LLVM-EXEGESIS(1)

💅 NAME

llvm-exegesis — the LLVM Machine Instruction Benchmark, bestie. That's the whole name, that's the whole vibe ✨

---

🫶 SYNOPSIS

llvm-exegesis [options]

---

🌸 DESCRIPTION

okay so llvm-exegesis is a benchmarking tool, and the way it works is genuinely iconic: it pulls on info that already lives inside LLVM to measure the characteristics of host machine instructions — things like latency, throughput, or port decomposition 👑

Here's the tea 🍓: you hand it an LLVM opcode name plus a benchmarking mode, and llvm-exegesis generates a lil code snippet that makes the execution as serial as possible (for latency) or as parallel as possible (for inverse throughput / uop decomposition). That's literally the whole trick.

Then that snippet gets jitted and executed on the host subtarget. The time taken — or, in the parallel case, the resource usage — is measured using hardware performance counters 💀 and the result gets printed out as YAML to standard output. No cap.

The main goal of this tool? To automatically (in)validate LLVM's TableDef scheduling models. That's the mission. To that end, it also gives you analysis of the results, which is lowkey the best part.

And plot twist 🎀: llvm-exegesis can also benchmark arbitrary user-provided code snippets. Not just LLVM instructions — your code. Obsessed.

---

🔥 EXAMPLE 1: BENCHMARKING INSTRUCTIONS

Assume you have an X86-64 machine (iykyk). To measure the latency of a single instruction, run:

$ llvm-exegesis -mode=latency -opcode-name=ADD64rr

Measuring the uop decomposition or inverse throughput of an instruction works basically the same way, periodt:

$ llvm-exegesis -mode=uops -opcode-name=ADD64rr
$ llvm-exegesis -mode=inverse_throughput -opcode-name=ADD64rr

The output is a YAML document — default is writing to stdout, but you can redirect that output to a file using -benchmarks-file 🦋:

---
key:
  opcode_name:     ADD64rr
  mode:            latency
  config:          ''
cpu_name:        haswell
llvm_triple:     x86_64-unknown-linux-gnu
num_repetitions: 10000
measurements:
  - { key: latency, value: 1.0058, debug_string: '' }
error:           ''
info:            'explicit self cycles, selecting one aliasing configuration.
Snippet:
ADD64rr R8, R8, R10
'
...

And if you want to measure the latency of all the instructions for the host architecture — every single one, main character energy — run:

$ llvm-exegesis -mode=latency -opcode-index=-1

---

🧚 EXAMPLE 2: BENCHMARKING A CUSTOM CODE SNIPPET

To measure the latency/uops of a custom piece of code, you can specify the snippets-file option. And bestie, - reads from standard input — cute:

$ echo "vzeroupper" | llvm-exegesis -mode=uops -snippets-file=-

Real-life code snippets typically depend on registers or memory, and llvm-exegesis does a vibe check on the liveliness of registers — meaning any register use has a corresponding def, or it's a "live in". If your code depends on the value of some registers, you have exactly two options 💖:

• Mark the register as requiring a definition. llvm-exegesis will automatically assign a value to the register. You do this with the directive LLVM-EXEGESIS-DEFREG <reg name> <hex_value>, where <hex_value> is a bit pattern used to fill <reg_name>. Heads up though: if <hex_value> is smaller than the register width, it will be sign-extended 🌈
• Mark the register as a "live in". llvm-exegesis will benchmark using whatever value was in these registers on entry. You do this with the directive LLVM-EXEGESIS-LIVEIN <reg name> 🌸

For example, the following code snippet depends on the values of XMM1 (which will be set by the tool) and the memory buffer passed in RDI (live in):

# LLVM-EXEGESIS-LIVEIN RDI
# LLVM-EXEGESIS-DEFREG XMM1 42
vmulps        (%rdi), %xmm1, %xmm2
vhaddps       %xmm2, %xmm2, %xmm3
addq $0x10, %rdi

---

⭐ EXAMPLE 3: ANALYSIS

Assuming you have a set of benchmarked instructions (either latency or uops) as YAML in the file /tmp/benchmarks.yaml, you can analyze the results using the following command:

      $ llvm-exegesis -mode=analysis \
-benchmarks-file=/tmp/benchmarks.yaml \
-analysis-clusters-output-file=/tmp/clusters.csv \
-analysis-inconsistencies-output-file=/tmp/inconsistencies.html

This will group the instructions into clusters with the same performance characteristics 🎀 The clusters get written out to /tmp/clusters.csv in the following format:

cluster_id,opcode_name,config,sched_class
...
2,ADD32ri8_DB,,WriteALU,1.00
2,ADD32ri_DB,,WriteALU,1.01
2,ADD32rr,,WriteALU,1.01
2,ADD32rr_DB,,WriteALU,1.00
2,ADD32rr_REV,,WriteALU,1.00
2,ADD64i32,,WriteALU,1.01
2,ADD64ri32,,WriteALU,1.01
2,MOVSX64rr32,,BSWAP32r_BSWAP64r_MOVSX64rr32,1.00
2,VPADDQYrr,,VPADDBYrr_VPADDDYrr_VPADDQYrr_VPADDWYrr_VPSUBBYrr_VPSUBDYrr_VPSUBQYrr_VPSUBWYrr,1.02
2,VPSUBQYrr,,VPADDBYrr_VPADDDYrr_VPADDQYrr_VPADDWYrr_VPSUBBYrr_VPSUBDYrr_VPSUBQYrr_VPSUBWYrr,1.01
2,ADD64ri8,,WriteALU,1.00
2,SETBr,,WriteSETCC,1.01
...

llvm-exegesis will also analyze the clusters to point out inconsistencies in the scheduling information 👀 The output is an html file. For example, /tmp/inconsistencies.html will contain messages like the following : [image]

Heads up bestie 🫶 — note that the scheduling class names will be resolved only when llvm-exegesis is compiled in debug mode. Otherwise only the class id will be shown. This does not invalidate any of the analysis results though, so don't panic 💜

💅 OPTIONS 💅

okay bestie, this is the part where we meet the whole cast of flags. every single one. no skipping, we're doing this right ✨

• -help — prints out a lil summary of all the command line options. your classic "wait what were the flags again" button 🫶
• -opcode-index=<LLVM opcode index> — picks the opcode to measure by index. set it to -1 and it'll measure literally every existing opcode (main character energy, honestly). see example 1 for the deets. and heads up: you must set either opcode-index, opcode-name, or snippets-file — pick one, bestie 💅
• -opcode-name=<opcode name 1>,<opcode name 2>,... — picks the opcode(s) to measure by name. you can stack several of them in a comma-separated list, go off. see example 1. and again: either opcode-index, opcode-name, or snippets-file must be set ✨
• -snippets-file=<filename> — points at the custom code snippet you wanna measure. see example 2. and yep, you guessed it: either opcode-index, opcode-name, or snippets-file must be set 🌸
• -mode=[latency|uops|inverse_throughput|analysis] — picks the run mode. ngl some modes come with extra requirements and options, so read the fine print:

latency mode can make use of either RDTSC or LBR. latency[LBR] is only available on X86 (at least Skylake, we don't do ancient). to run in latency mode, you gotta specify a positive value for x86-lbr-sample-period and -repetition-mode=loop.

in analysis mode, you also need to specify at least one of -analysis-clusters-output-file= and -analysis-inconsistencies-output-file= — it's giving "you can't sit with us unless you bring output files" 😭

• -x86-lbr-sample-period=<nBranches/sample> — sets the LBR sampling period, aka how many branches we let pass before we take a sample. when you give it a positive value and the mode is latency, we use LBRs for measuring 🔥

so like, on picking the "right" sampling period: small value is preferred, but if you sample too frequently you could get throttling. also use a prime number so you don't consistently skip certain blocks — lowkey genius, iykyk 👑

• -repetition-mode=[duplicate|loop|min] — sets the repetition mode, and each one has a whole personality:

duplicate creates a large, straight-line basic block stuffed with num-repetitions instructions (it repeats the snippet num-repetitions/snippet size times).

loop will optionally duplicate the snippet until the loop body contains at least loop-body-size instructions, then wraps the result in a loop that executes num-repetitions instructions — so, again, it's repeating the snippet num-repetitions/snippet size times.

loop mode (especially with loop unrolling) tends to hide the effects of the CPU frontend better on architectures that cache decoded instructions, but it costs you a register for counting iterations. if you're doing an analysis over many opcodes, it might be best to just use min mode instead — it runs each other mode and gives you the minimal measured result. slay 💅

• -num-repetitions=<Number of repetitions> — the target number of executed instructions. note that the actual repetition count of the snippet will be num-repetitions/snippet size. higher values = more accurate measurements but a longer benchmark. the eternal tradeoff 🙈
• -loop-body-size=<Preferred loop body size> — only effective for -repetition-mode=[loop|min]. instead of looping over the snippet directly, it first duplicates it so the loop body contains at least this many instructions. this can get the loop body cached in the CPU Op Cache / Loop Cache, which may have higher throughput than the CPU decoders ⭐
• -max-configs-per-opcode=<value> — the max number of configurations that can be generated for each opcode. by default this is 1, meaning we assume a single measurement is enough to characterize an opcode 👀

that assumption might not hold for all instructions: for example, the performance characteristics of the LEA instruction on X86 depend on the value of assigned registers and immediates. setting -max-configs-per-opcode larger than 1 lets llvm-exegesis explore more configurations to find out if some register or immediate assignments lead to different performance characteristics 🦋

• -benchmarks-file=</path/to/file> — the file to read (in analysis mode) or write (in latency/uops/inverse_throughput modes) benchmark results to. "-" uses stdin/stdout 🎀
• -analysis-clusters-output-file=</path/to/file> — if provided, writes the analysis clusters as CSV to this file. "-" prints to stdout. by default, this analysis is not run 💜
• -analysis-inconsistencies-output-file=</path/to/file> — if non-empty, writes inconsistencies found during analysis to this file. - prints to stdout. by default, this analysis is not run 🌈
• -analysis-clustering=[dbscan,naive] — picks the clustering algorithm. DBSCAN is the default 💅

naive clustering is better if you wanna do further work on the -analysis-inconsistencies-output-file= output — it creates one cluster per opcode and checks that the cluster is stable (all points are neighbours) ✨

• -analysis-numpoints=<dbscan numPoints parameter> — sets the numPoints parameter used for DBSCAN clustering (analysis mode, DBSCAN only) 🍓
• -analysis-clustering-epsilon=<dbscan epsilon parameter> — sets the epsilon parameter used for clustering of benchmark points (analysis mode) 🌸
• -analysis-inconsistency-epsilon=<epsilon> — sets the epsilon parameter used to detect when the cluster is different from the LLVM schedule profile values (analysis mode) 🔥
• -analysis-display-unstable-clusters — if there's more than one benchmark for an opcode, those benchmarks might not get clustered into the same cluster if the measured performance characteristics differ. by default all such opcodes are filtered out — this flag instead shows only the unstable opcodes 👑
• -ignore-invalid-sched-class=false — if set, ignore instructions that don't have a sched class (class idx = 0) 🙈
• -mcpu=<cpu name> — if set, measure the cpu characteristics using the counters for this CPU. super useful when creating new sched models (like, when the host CPU is unknown to LLVM) 🧚
• --dump-object-to-disk=true — by default, llvm-exegesis dumps the generated code to a temporary file so you can inspect it. you may disable it to speed up execution and save disk space 💀

---

🫶 EXIT STATUS 🫶

llvm-exegesis returns 0 on success 💖 otherwise, an error message gets printed to standard error, and the tool returns a non 0 value. not the vibe, but at least you know 😭

✨ AUTHOR ✨

Maintained by the LLVM Team (https://llvm.org/) — the group chat admins, fr fr 👑

🎀 COPYRIGHT 🎀

2003-2023, LLVM Project

---

and that's the whole page, bestie!! 🦋 this was page version 15, dated 2023-07-14, source LLVM-EXEGESIS(1) — go forth and benchmark, you absolute icon 💅✨