LLVM-MCA(1)

💅 llvm-mca — the LLVM Machine Code Analyzer, bestie edition

okay so real talk, this is llvm-mca(1) from the LLVM fam — the Machine Code Analyzer. she's a whole vibe, let's get into it 🫶

✨ NAME ✨

llvm-mca — LLVM Machine Code Analyzer. yes that's her full government name 💖

🌸 SYNOPSIS 🌸

llvm-mca [options] [input]

🦋 DESCRIPTION 🦋

okay so llvm-mca is a performance analysis tool and she is lowkey iconic. what she does is take the info that's already chilling inside LLVM (think scheduling models and stuff) and statically measure how fast machine code would go on a specific CPU. no cap, no running the code, she just knows. ⭐

performance here gets measured in terms of throughput AND processor resource consumption. currently she works for processors that have a backend where LLVM has a scheduling model available — so if your target's got the model, she's got you 👑

the main goal is not just predicting perf on the target (that would be too basic) — she also wants to help you diagnose potential performance issues. it's giving detective energy 🔍

so like, you hand her an assembly code sequence and she estimates the Instructions Per Cycle (IPC) plus hardware resource pressure. the analysis + reporting style was inspired by the IACA tool from Intel, iykyk 💅

for example, you can compile code with clang, spit out assembly, and pipe it straight into llvm-mca for analysis:

$ clang foo.c -O2 -target x86_64-unknown-unknown -S -o - | llvm-mca -mcpu=btver2

or if you're an Intel syntax girlie:

$ clang foo.c -O2 -target x86_64-unknown-unknown -mllvm -x86-asm-syntax=intel -S -o - | llvm-mca -mcpu=btver2

quick side note: llvm-mca detects Intel syntax by the presence of an .intel_syntax directive at the beginning of the input. and by default her output syntax matches whatever the input syntax was. matching energy, we love to see it 🎀

scheduling models aren't just for computing instruction latencies and throughput, bestie — they're also how she understands what processor resources are available and how to simulate them. main character energy fr fr 🌈

now, by design, the quality of the analysis llvm-mca conducts is inevitably affected by the quality of the scheduling models in LLVM. so like, garbage in → garbage out, that's just how it is 😭

if you see that the performance report is not accurate for a processor, please file a bug against the appropriate backend. do your part, queen 🫶

💜 OPTIONS 💜

so if input is - or just... omitted entirely (she's flexible like that), llvm-mca reads from standard input. otherwise she'll read from the filename you gave her.

and if the -o option is omitted, then llvm-mca sends output to standard output when the input is from standard input. and if -o specifies -, the output also goes to standard output. consistent queen 💅

• -help — prints a summary of the command line options. the classic "what even are my options" moment.
• -o <filename> — use <filename> as the output filename. peep the summary above for the deets 👀
• -mtriple=<target triple> — specify a target triple string.
• -march=<arch> — specify the architecture to analyze the code for. defaults to the host default target.
• -mcpu=<cpuname> — specify the processor to analyze the code for. by default the cpu name is autodetected from the host. she just knows 💫
• -output-asm-variant=<variant id> — specify the output assembly variant for the report the tool generates. on x86 the possible values are [0, 1]. a value of 0 (vic. 1) for this flag enables the AT&T (vic. Intel) assembly format for the code printed out by the tool in the analysis report. pick your fighter 🥊
• -print-imm-hex — prefers hex format for numeric literals in the output assembly that gets printed as part of the report.
• -dispatch=<width> — specify a different dispatch width for the processor. the dispatch width defaults to the field 'IssueWidth' in the processor scheduling model. if width is zero, then the default dispatch width is used.
• -register-file-size=<size> — specify the size of the register file. when specified, this flag limits how many physical registers are available for register renaming purposes. a value of zero here means "unlimited number of physical registers" — go off, infinite regs ✨
• -iterations=<number of iterations> — specify the number of iterations to run. if this flag is set to 0, then the tool sets the number of iterations to a default value (i.e. 100).
• -noalias=<bool> — if set, the tool assumes that loads and stores don't alias. this is the default behavior. no drama between your loads and stores, periodt.
• -lqueue=<load queue size> — specify the size of the load queue in the load/store unit that the tool emulates. by default the tool assumes an unbound number of entries in the load queue. a value of zero for this flag is ignored, and the default load queue size is used instead.
• -squeue=<store queue size> — specify the size of the store queue in the load/store unit that the tool emulates. by default the tool assumes an unbound number of entries in the store queue. a value of zero for this flag is ignored, and the default store queue size is used instead.
• -timeline — enable the timeline view. get the whole chronological glow-up 🕐
• -timeline-max-iterations=<iterations> — limit the number of iterations printed in the timeline view. by default the timeline view prints info for up to 10 iterations.
• -timeline-max-cycles=<cycles> — limit the number of cycles in the timeline view, or use 0 for no limit. by default the number of cycles is set to 80.
• -resource-pressure — enable the resource pressure view. this one's enabled by default 👑
• -register-file-stats — enable register file usage statistics.
• -dispatch-stats — enable extra dispatch statistics. this view collects and analyzes instruction dispatch events, plus static/dynamic dispatch stall events. this view is disabled by default.
• -scheduler-stats — enable extra scheduler statistics. this view collects and analyzes instruction issue events. this view is disabled by default.
• -retire-stats — enable extra retire control unit statistics. this view is disabled by default. she's retired, she's resting 💅
• -instruction-info — enable the instruction info view. this is enabled by default.
• -show-encoding — enable the printing of instruction encodings within the instruction info view.
• -show-barriers — enable the printing of LoadBarrier and StoreBarrier flags within the instruction info view.
• -all-stats — print all hardware statistics. this enables extra statistics related to the dispatch logic, the hardware schedulers, the register file(s), and the retire control unit. this option is disabled by default.
• -all-views — enable all the view. yes, all of them. every single one. go big or go home 🌈
• -instruction-tables — prints resource pressure information based on the static info available from the processor model. this differs from the resource pressure view because it doesn't require the code to be simulated. instead it prints the theoretical uniform distribution of resource pressure for every instruction in sequence. theory queen 🧚
• -bottleneck-analysis — print info about bottlenecks that affect the throughput. this analysis can be expensive, and it's disabled by default. bottlenecks get highlighted in the summary view. bottleneck analysis is currently not supported for processors with an in-order backend. noted 🫡
• -json — print the requested views in valid JSON format. the instructions and the processor resources get printed as members of special top level JSON objects. the individual views refer to them by index. HOWEVER — not all views are currently supported. for example, the report from the bottleneck analysis is not printed out in JSON. all the default views are currently supported though 💖
• -disable-cb — force usage of the generic CustomBehaviour and InstrPostProcess classes rather than using the target specific implementation. the generic classes never detect any custom hazards or make any post processing modifications to instructions. vanilla mode, basically 🍦

🔥 EXIT STATUS 🔥

llvm-mca returns 0 on success. otherwise, an error message gets printed to standard error, and the tool returns 1. she tells you when something's wrong, communication icon 💀

🎀 USING MARKERS TO ANALYZE SPECIFIC CODE BLOCKS 🎀

llvm-mca lets you optionally use special code comments to mark regions of assembly code you want analyzed. it's giving "highlight the good part" ✨

a comment starting with the substring LLVM-MCA-BEGIN marks the beginning of a code region. a comment starting with the substring LLVM-MCA-END marks the end of a code region. like so:

# LLVM-MCA-BEGIN
  ...
# LLVM-MCA-END

if no user-defined region is specified, then llvm-mca assumes a default region that contains every instruction in the input file. every region gets analyzed in isolation, and the final performance report is the union of all the reports generated for every code region. teamwork makes the dream work 🫶

code regions can have names. for example:

# LLVM-MCA-BEGIN A simple example
  add %eax, %eax
# LLVM-MCA-END

the code from that example defines a region named "A simple example" with a single instruction in it. and notice how the region name doesn't have to be repeated in the LLVM-MCA-END directive. in the absence of overlapping regions, an anonymous LLVM-MCA-END directive always ends the currently active user defined region. no need to introduce yourself twice, bestie 💅

example of nesting regions:

# LLVM-MCA-BEGIN foo
  add %eax, %edx
# LLVM-MCA-BEGIN bar
  sub %eax, %edx
# LLVM-MCA-END bar
# LLVM-MCA-END foo

example of overlapping regions:

# LLVM-MCA-BEGIN foo
  add %eax, %edx
# LLVM-MCA-BEGIN bar
  sub %eax, %edx
# LLVM-MCA-END foo
  add %eax, %edx
# LLVM-MCA-END bar

note that multiple anonymous regions cannot overlap. also, overlapping regions cannot have the same name. them's the rules 🙈

there is no support for marking regions from high-level source code, like C or C++. as a workaround, inline assembly directives may be used:

int foo(int a, int b) {
  __asm volatile("# LLVM-MCA-BEGIN foo":::"memory");
  a += 42;
  __asm volatile("# LLVM-MCA-END":::"memory");
  a *= b;
  return a;
}

however, this interferes with optimizations like loop vectorization and may have an impact on the code generated. this is because the __asm statements are seen as real code having important side effects, which limits how the code around them can be transformed. so if users want to make use of inline assembly to emit markers, then the recommendation is to always verify that the output assembly is equivalent to the assembly generated in the absence of markers. the Clang options to emit optimization reports can also help in detecting missed optimizations. don't let the markers sabotage your gains, fr fr 😭

🧚 HOW LLVM-MCA WORKS 🧚

llvm-mca takes assembly code as input. the assembly code gets parsed into a sequence of MCInst with the help of the existing LLVM target assembly parsers. the parsed sequence of MCInst is then analyzed by a Pipeline module to generate a performance report. that's the whole lore drop 🫶

The Pipeline module is literally the main character here — it simulates your machine code sequence running in a loop of iterations (default is 100), and while it's doing all that, it's quietly collecting a whole pile of execution-related statistics. Then at the end it prints a report from everything it gathered. Obsessed.

Here's an actual performance report the tool spat out for a dot-product of two packed float vectors of four elements, target x86, cpu btver2. You get this exact result by running the command below on the example at test/tools/llvm-mca/X86/BtVer2/dot-product.s:

$ llvm-mca -mtriple=x86_64-unknown-unknown -mcpu=btver2 -iterations=300 dot-product.s

Iterations:        300
Instructions:      900
Total Cycles:      610
Total uOps:        900

Dispatch Width:    2
uOps Per Cycle:    1.48
IPC:               1.48
Block RThroughput: 2.0

Instruction Info:
[1]: #uOps
[2]: Latency
[3]: RThroughput
[4]: MayLoad
[5]: MayStore
[6]: HasSideEffects (U)

[1]    [2]    [3]    [4]    [5]    [6]    Instructions:
 1      2     1.00                        vmulps      %xmm0, %xmm1, %xmm2
 1      3     1.00                        vhaddps     %xmm2, %xmm2, %xmm3
 1      3     1.00                        vhaddps     %xmm3, %xmm3, %xmm4

Resources:
[0]   - JALU0
[1]   - JALU1
[2]   - JDiv
[3]   - JFPA
[4]   - JFPM
[5]   - JFPU0
[6]   - JFPU1
[7]   - JLAGU
[8]   - JMul
[9]   - JSAGU
[10]  - JSTC
[11]  - JVALU0
[12]  - JVALU1
[13]  - JVIMUL

Resource pressure per iteration:
[0]    [1]    [2]    [3]    [4]    [5]    [6]    [7]    [8]    [9]    [10]   [11]   [12]   [13]
 -      -      -     2.00   1.00   2.00   1.00    -      -      -      -      -      -      -

Resource pressure by instruction:
[0]    [1]    [2]    [3]    [4]    [5]    [6]    [7]    [8]    [9]    [10]   [11]   [12]   [13]   Instructions:
 -      -      -      -     1.00    -     1.00    -      -      -      -      -      -      -     vmulps      %xmm0, %xmm1, %xmm2
 -      -      -     1.00    -     1.00    -      -      -      -      -      -      -      -     vhaddps     %xmm2, %xmm2, %xmm3
 -      -      -     1.00    -     1.00    -      -      -      -      -      -      -      -     vhaddps     %xmm3, %xmm3, %xmm4

So what's the tea ☕ — according to this report, that dot-product kernel ran 300 times, for a grand total of 900 simulated instructions. And the total number of simulated micro opcodes (uOps) is also 900. Cute.

The report is laid out in three main sections, okay? Section one collects a few performance numbers — its whole job is to give you a super quick overview of the performance throughput. The important performance indicators are IPC, uOps Per Cycle, and Block RThroughput (that's Block Reciprocal Throughput, iykyk).

Field DispatchWidth is the max number of micro opcodes dispatched to the out-of-order backend every simulated cycle. For processors with an in-order backend, DispatchWidth is the max number of micro opcodes issued to the backend every simulated cycle. Same energy, different backend.

IPC is computed by dividing the total number of simulated instructions by the total number of cycles. Math is mathing 💅

Field Block RThroughput is the reciprocal of the block throughput. Block throughput is a theoretical quantity, computed as the maximum number of blocks (a.k.a. iterations) that can be executed per simulated clock cycle in the absence of loop carried dependencies. And Block throughput is superiorly limited by the dispatch rate and the availability of hardware resources. So like, it can't just be anything.

In the absence of loop-carried data dependencies, the observed IPC tends toward a theoretical maximum which you can compute by dividing the number of instructions of a single iteration by the Block RThroughput.

Field ‘uOps Per Cycle’ is computed by dividing the total number of simulated micro opcodes by the total number of cycles. A delta between Dispatch Width and this field is a whole indicator of a performance issue 🚩. And in the absence of loop-carried data dependencies, the observed ‘uOps Per Cycle’ should tend toward a theoretical maximum throughput which can be computed by dividing the number of uOps of a single iteration by the Block RThroughput.

Field uOps Per Cycle is bounded from above by the dispatch width — because the dispatch width limits the maximum size of a dispatch group, periodt. Both IPC and ‘uOps Per Cycle’ are limited by the amount of hardware parallelism. The availability of hardware resources affects the resource pressure distribution, and it limits the number of instructions that can be executed in parallel every cycle. A delta between Dispatch Width and the theoretical maximum uOps per Cycle (computed by dividing the number of uOps of a single iteration by the Block RThroughput) is an indicator of a performance bottleneck caused by the lack of hardware resources. In general, the lower the Block RThroughput, the better. Lowkey the whole game.

In this example, uOps per iteration/Block RThroughput is 1.50. Since there are no loop-carried dependencies, the observed uOps Per Cycle is expected to approach 1.50 when the number of iterations tends to infinity ♾️. The delta between the Dispatch Width (2.00) and the theoretical maximum uOp throughput (1.50) is an indicator of a performance bottleneck caused by the lack of hardware resources — and the Resource pressure view can help you identify the problematic resource usage. She's pointing at the culprit 🫵

---

Section two of the report is the instruction info view ✨ It shows the latency and reciprocal throughput of every instruction in the sequence. It also reports extra info about the number of micro opcodes and opcode properties (a.k.a. ‘MayLoad’, ‘MayStore’, and ‘HasSideEffects’).

Field RThroughput is the reciprocal of the instruction throughput. Throughput is computed as the maximum number of instructions of a same type that can be executed per clock cycle in the absence of operand dependencies. In this example, the reciprocal throughput of a vector float multiply is 1 cycles/instruction. That's because the FP multiplier JFPM is only available from pipeline JFPU1. One lane, bestie.

Instruction encodings get displayed inside the instruction info view when the flag -show-encoding is specified 👀

Below is an example of -show-encoding output for the dot-product kernel:

Instruction Info:
[1]: #uOps
[2]: Latency
[3]: RThroughput
[4]: MayLoad
[5]: MayStore
[6]: HasSideEffects (U)
[7]: Encoding Size

[1]    [2]    [3]    [4]    [5]    [6]    [7]    Encodings:                    Instructions:
 1      2     1.00                         4     c5 f0 59 d0                   vmulps %xmm0, %xmm1, %xmm2
 1      4     1.00                         4     c5 eb 7c da                   vhaddps        %xmm2, %xmm2, %xmm3
 1      4     1.00                         4     c5 e3 7c e3                   vhaddps        %xmm3, %xmm3, %xmm4

The Encoding Size column shows the size in bytes of instructions. The Encodings column shows the actual instruction encodings (byte sequences in hex). Hex girlies rise 🧚

---

Section three is the Resource pressure view. This view reports the average number of resource cycles consumed every iteration by instructions, for every processor resource unit available on the target. The info is structured in two tables: table one reports the number of resource cycles spent on average every iteration; table two correlates the resource cycles to the machine instruction in the sequence. So, for example, every iteration of the instruction vmulps always executes on resource unit [6] (JFPU1 — floating point pipeline #1), consuming an average of 1 resource cycle per iteration. And note that on AMD Jaguar, vector floating-point multiply can only be issued to pipeline JFPU1, while horizontal floating-point additions can only be issued to pipeline JFPU0. No switching lanes, no negotiating.

The resource pressure view helps with identifying bottlenecks caused by high usage of specific hardware resources. Situations with resource pressure mainly concentrated on a few resources should, in general, be avoided 🙅‍♀️. Ideally, pressure should be uniformly distributed between multiple resources. Balance is the vibe. ✨

Timeline View 🕰️

The timeline view produces a detailed report of each instruction's state transitions through an instruction pipeline. It's enabled by the command line option -timeline. As instructions transition through the various stages of the pipeline, their states are depicted in the view report. These states are represented by the following characters:

• D : Instruction dispatched.
• e : Instruction executing.
• E : Instruction executed.
• R : Instruction retired.
• = : Instruction already dispatched, waiting to be executed.
• - : Instruction executed, waiting to be retired.

Below is the timeline view for a subset of the dot-product example located in test/tools/llvm-mca/X86/BtVer2/dot-product.s and processed by llvm-mca using the following command:

$ llvm-mca -mtriple=x86_64-unknown-unknown -mcpu=btver2 -iterations=3 -timeline dot-product.s

Timeline view:
                    012345
Index     0123456789

[0,0]     DeeER.    .    .   vmulps   %xmm0, %xmm1, %xmm2
[0,1]     D==eeeER  .    .   vhaddps  %xmm2, %xmm2, %xmm3
[0,2]     .D====eeeER    .   vhaddps  %xmm3, %xmm3, %xmm4
[1,0]     .DeeE-----R    .   vmulps   %xmm0, %xmm1, %xmm2
[1,1]     . D=eeeE---R   .   vhaddps  %xmm2, %xmm2, %xmm3
[1,2]     . D====eeeER   .   vhaddps  %xmm3, %xmm3, %xmm4
[2,0]     .  DeeE-----R  .   vmulps   %xmm0, %xmm1, %xmm2
[2,1]     .  D====eeeER  .   vhaddps  %xmm2, %xmm2, %xmm3
[2,2]     .   D======eeeER   vhaddps  %xmm3, %xmm3, %xmm4

Average Wait times (based on the timeline view):
[0]: Executions
[1]: Average time spent waiting in a scheduler's queue
[2]: Average time spent waiting in a scheduler's queue while ready
[3]: Average time elapsed from WB until retire stage

      [0]    [1]    [2]    [3]
0.     3     1.0    1.0    3.3       vmulps   %xmm0, %xmm1, %xmm2
1.     3     3.3    0.7    1.0       vhaddps  %xmm2, %xmm2, %xmm3
2.     3     5.7    0.0    0.0       vhaddps  %xmm3, %xmm3, %xmm4
       3     3.3    0.5    1.4       <total>

The timeline view is interesting because it shows instruction state changes during execution 🔥. It also gives you an idea of how the tool processes instructions executed on the target, and how their timing information might be calculated. Literally watching the instructions go through their little journey, it's giving main character energy for every single uOp 💅

The timeline view is split into two tables, bestie. Table number one is the glow-up timeline: instructions changing state over time, measured in cycles. Table two is called Average Wait times and it's the ✨stats✨ table — the one that helps you diagnose performance bottlenecks caused by long data dependencies and sub-optimal usage of hardware resources.

An instruction in the timeline view is ID'd by a pair of indices: the first one is the iteration, the second one is the instruction index (like, where it lives in the code sequence). This example was generated with 3 iterations — -iterations=3 — so the iteration indices go from 0-2 inclusively. Slay.

Skip the first and last column and everything else is in cycles, numbered sequentially starting from 0. 🎀

From the example output above, we know the following:

• Instruction [1,0] was dispatched at cycle 1.
• Instruction [1,0] started executing at cycle 2.
• Instruction [1,0] reached the write back stage at cycle 4.
• Instruction [1,0] was retired at cycle 10.

Instruction [1,0] (that's vmulps from iteration #1) does not have to wait in the scheduler's queue for the operands to become available. By the time vmulps is dispatched, the operands are already there, and pipeline JFPU1 is ready to serve another instruction. So it can be immediately issued on the JFPU1 pipeline — which is literally proven by the fact that it only spent 1cy in the scheduler's queue. 💅

There's a gap of 5 cycles between the write-back stage and the retire event. Why? Because instructions must retire in program order, so [1,0] has to wait for [0,2] to retire first (like, it has to wait until cycle 10). No cutting the line. 🙈

In the example, all instructions are in a RAW (Read After Write) dependency chain. Register %xmm2 written by vmulps is immediately used by the first vhaddps, and register %xmm3 written by the first vhaddps is used by the second vhaddps. Long data dependencies negatively impact the ILP (Instruction Level Parallelism). Not iconic of them.

In the dot-product example, there are anti-dependencies introduced by instructions from different iterations. But those dependencies can be removed at register renaming stage (at the cost of allocating register aliases, and therefore consuming physical registers). Trade-offs, babe.

The table Average Wait times helps diagnose performance issues caused by the presence of long latency instructions and potentially long data dependencies which may limit the ILP. The last row, <total>, shows a global average over all instructions measured. Note that llvm-mca, by default, assumes at least 1cy between the dispatch event and the issue event. 🫶

When the performance is limited by data dependencies and/or long latency instructions, the number of cycles spent while in the ready state is expected to be very small compared with the total number of cycles spent in the scheduler's queue. The difference between the two counters is a good indicator of how large of an impact data dependencies had on the execution of the instructions. When performance is mostly limited by the lack of hardware resources, the delta between the two counters is small. But the number of cycles spent in the queue tends to be larger (i.e., more than 1-3cy), especially when compared to other low latency instructions. 🦋

🔥 Bottleneck Analysis 🔥

The -bottleneck-analysis command line option enables the analysis of performance bottlenecks.

Heads up: this analysis is potentially expensive. It attempts to correlate increases in backend pressure (caused by pipeline resource pressure and data dependencies) to dynamic dispatch stalls. Expensive = think before you spam it, bestie. 💀

Below is an example of -bottleneck-analysis output generated by llvm-mca for 500 iterations of the dot-product example on btver2.

Cycles with backend pressure increase [ 48.07% ]
Throughput Bottlenecks:
  Resource Pressure       [ 47.77% ]
  - JFPA  [ 47.77% ]
  - JFPU0  [ 47.77% ]
  Data Dependencies:      [ 0.30% ]
  - Register Dependencies [ 0.30% ]
  - Memory Dependencies   [ 0.00% ]

Critical sequence based on the simulation:

              Instruction                         Dependency Information
 +----< 2.    vhaddps %xmm3, %xmm3, %xmm4
 |
 |    < loop carried >
 |
 |      0.    vmulps  %xmm0, %xmm1, %xmm2
 +----> 1.    vhaddps %xmm2, %xmm2, %xmm3         ## RESOURCE interference:  JFPA [ probability: 74% ]
 +----> 2.    vhaddps %xmm3, %xmm3, %xmm4         ## REGISTER dependency:  %xmm3
 |
 |    < loop carried >
 |
 +----> 1.    vhaddps %xmm2, %xmm2, %xmm3         ## RESOURCE interference:  JFPA [ probability: 74% ]

According to the analysis, throughput is limited by resource pressure and not by data dependencies. The analysis observed increases in backend pressure during 48.07% of the simulated run. Almost all those pressure increase events were caused by contention on processor resources JFPA/JFPU0. (Not the data's fault for once, king behavior from the scheduler.)

The critical sequence is the most expensive sequence of instructions according to the simulation. It's annotated to give you extra info about critical register dependencies and resource interferences between instructions.

Instructions from the critical sequence are expected to significantly impact performance. By construction, the accuracy of this analysis is strongly dependent on the simulation and (as always) by the quality of the processor model in llvm. So like… garbage in, garbage out, iykyk. 😭

Bottleneck analysis is currently not supported for processors with an in-order backend.

🌸 Extra Statistics to Further Diagnose Performance Issues 🌸

The -all-stats command line option enables extra statistics and performance counters for the dispatch logic, the reorder buffer, the retire control unit, and the register file. So much data. So much power. 👑

Below is an example of -all-stats output generated by llvm-mca for 300 iterations of the dot-product example discussed in the previous sections.

Dynamic Dispatch Stall Cycles:
RAT     - Register unavailable:                      0
RCU     - Retire tokens unavailable:                 0
SCHEDQ  - Scheduler full:                            272  (44.6%)
LQ      - Load queue full:                           0
SQ      - Store queue full:                          0
GROUP   - Static restrictions on the dispatch group: 0

Dispatch Logic - number of cycles where we saw N micro opcodes dispatched:
[# dispatched], [# cycles]
 0,              24  (3.9%)
 1,              272  (44.6%)
 2,              314  (51.5%)

Schedulers - number of cycles where we saw N micro opcodes issued:
[# issued], [# cycles]
 0,          7  (1.1%)
 1,          306  (50.2%)
 2,          297  (48.7%)

Scheduler's queue usage:
[1] Resource name.
[2] Average number of used buffer entries.
[3] Maximum number of used buffer entries.
[4] Total number of buffer entries.

 [1]            [2]        [3]        [4]
JALU01           0          0          20
JFPU01           17         18         18
JLSAGU           0          0          12

Retire Control Unit - number of cycles where we saw N instructions retired:
[# retired], [# cycles]
 0,           109  (17.9%)
 1,           102  (16.7%)
 2,           399  (65.4%)

Total ROB Entries:                64
Max Used ROB Entries:             35  ( 54.7% )
Average Used ROB Entries per cy:  32  ( 50.0% )

Register File statistics:
Total number of mappings created:    900
Max number of mappings used:         35

*  Register File #1 -- JFpuPRF:
   Number of physical registers:     72
   Total number of mappings created: 900
   Max number of mappings used:      35

*  Register File #2 -- JIntegerPRF:
   Number of physical registers:     64
   Total number of mappings created: 0
   Max number of mappings used:      0

If we look at the Dynamic Dispatch Stall Cycles table, we see the counter for SCHEDQ reports 272 cycles. This counter is incremented every time the dispatch logic is unable to dispatch a full group because the scheduler's queue is full. Full queue, no dispatch. Simple as that. 💅

Looking at the Dispatch Logic table, we see that the pipeline was only able to dispatch two micro opcodes 51.5% of the time. The dispatch group was limited to one micro opcode 44.6% of the cycles, which corresponds to 272 cycles. The dispatch statistics are displayed by either using the command option -all-stats or -dispatch-stats.

The next table, Schedulers, presents a histogram displaying a count, representing the number of micro opcodes issued on some number of cycles. In this case, of the 610 simulated cycles, single opcodes were issued 306 times (50.2%) and there were 7 cycles where no opcodes were issued. Zero. Nothing. Nada. 😭

The Scheduler's queue usage table shows the average and maximum number of buffer entries (i.e., scheduler queue entries) used at runtime. Resource JFPU01 reached its maximum (18 of 18 queue entries — maxed out, fully booked 🎀). Note that AMD Jaguar implements three schedulers:

• JALU01 — a scheduler for ALU instructions.
• JFPU01 — a scheduler floating point operations.
• JLSAGU — a scheduler for address generation.

The dot-product is a kernel of three floating point instructions (a vector multiply followed by two horizontal adds). That explains why only the floating point scheduler appears to be used. It's giving float-only energy. ✨

A full scheduler queue is either caused by data dependency chains or by a sub-optimal usage of hardware resources. Sometimes, resource pressure can be mitigated by rewriting the kernel using different instructions that consume different scheduler resources. Schedulers with a small queue are less resilient to bottlenecks caused by the presence of long data dependencies. The scheduler statistics are displayed by using the command option -all-stats or -scheduler-stats.

The next table, Retire Control Unit, presents a histogram displaying a count, representing the number of instructions retired on some number of cycles. In this case, of the 610 simulated cycles, two instructions were retired during the same cycle 399 times (65.4%) and there were 109 cycles where no instructions were retired. The retire statistics are displayed by using the command option -all-stats or -retire-stats. 🧚

Like, the very last table on the screen is Register File statistics ✨ — every physical register file (PRF) that the pipeline actually touches gets its own row in there. For AMD Jaguar specifically, there are two register files: one for floating-point registers (JFpuPRF) and one for integer registers (JIntegerPRF). The table is telling us that out of the 900 instructions processed, 900 mappings got created. And since this dot-product example only ever used floating point registers, it was JFPuPRF that did all the heavy lifting and made those 900 mappings 💅. But look — the pipeline only ever used a maximum of 35 out of 72 available register slots at any given moment. So like, we can conclude the floating point PRF was the only register file used for the example, and it was never once resource constrained. Bestie, she was chilling 🧚. You get this table with -all-stats or -register-file-stats, iykyk.

So for this example? The conclusion is that the IPC is mostly limited by data dependencies, not by resource pressure. Periodt.

🌈 Instruction Flow 🌈

This bit walks you through the instruction flow across llvm-mca's default pipeline, plus the functional units that get roped into the whole process 🎀.

The default pipeline runs this exact sequence of stages to process instructions:

• Dispatch — the instruction gets dispatched to the schedulers.
• Issue — the instruction gets issued to the processor pipelines.
• Write Back — the instruction is executed, and results are written back.
• Retire — the instruction is retired; writes are architecturally committed.

Meanwhile the in-order pipeline runs this sequence of stages instead:

• InOrderIssue — the instruction is issued to the processor pipelines.
• Retire — the instruction is retired; writes are architecturally committed.

Okay so here's the thing 👀 — llvm-mca assumes every instruction has already been decoded and plopped into a queue before the simulation even starts. So the instruction fetch and decode stages? Not modeled. At all. Frontend performance bottlenecks are simply not diagnosed. Also, llvm-mca does not model branch prediction. Nope, not a vibe it does 😭.

💫 Instruction Dispatch 💫

During the dispatch stage, instructions get picked in program order out of a queue of already-decoded instructions, and they're dispatched in groups to the simulated hardware schedulers 🫶.

The size of a dispatch group depends on how available the simulated hardware resources are. The processor dispatch width defaults to the value of IssueWidth in LLVM's scheduling model.

An instruction can be dispatched if:

• The size of the dispatch group is smaller than the processor's dispatch width.
• There are enough entries in the reorder buffer.
• There are enough physical registers to do register renaming.
• The schedulers are not full.

Scheduling models can optionally specify which register files are available on the processor, and llvm-mca uses that info to initialize register file descriptors ✨. You can cap how many physical registers are globally available for register renaming by using -register-file-size. A value of zero for this option means unbounded — like, no limits, main character energy 👑. And by knowing how many registers are available for renaming, the tool can predict dispatch stalls caused by the lack of physical registers. Iconic.

The number of reorder buffer entries an instruction consumes depends on how many micro-opcodes the target scheduling model specifies for that instruction. The reorder buffer is the one tracking the progress of instructions that are "in-flight", and retiring them in program order. The number of entries in the reorder buffer defaults to whatever value is specified by the field MicroOpBufferSize in the target scheduling model.

Instructions dispatched to the schedulers consume scheduler buffer entries. llvm-mca queries the scheduling model to figure out the set of buffered resources an instruction consumes. Buffered resources are treated like scheduler resources — same energy, fr fr 💜.

🔥 Instruction Issue 🔥

Each processor scheduler implements a buffer of instructions. An instruction has to wait in the scheduler's buffer until its input register operands become available 🥺. Only at that point does the instruction become eligible for execution and get to potentially be issued (potentially out-of-order) for execution. Instruction latencies are computed by llvm-mca with a little help from the scheduling model.

llvm-mca's scheduler is designed to simulate multiple processor schedulers. It's responsible for tracking data dependencies and dynamically selecting which processor resources get consumed by instructions. It delegates the management of processor resource units and resource groups to a resource manager 💅. The resource manager is the one selecting the resource units consumed by instructions. For example, if an instruction consumes 1cy of a resource group, the resource manager picks one of the available units from the group; by default, the resource manager uses a round-robin selector to guarantee that resource usage is uniformly distributed between all units of a group. Fair and balanced, we love to see it ⭐.

Internally, llvm-mca's scheduler groups instructions into three sets:

• WaitSet: a set of instructions whose operands are not ready.
• ReadySet: a set of instructions ready to execute.
• IssuedSet: a set of instructions executing.

Depending on operand availability, instructions dispatched to the scheduler land in either the WaitSet or the ReadySet 🦋.

Every single cycle, the scheduler checks if instructions can be moved from the WaitSet to the ReadySet, and whether instructions from the ReadySet can be issued to the underlying pipelines. And the algorithm prioritizes older instructions over younger instructions — respect your elders, bestie 🙈.

🎀 Write-Back and Retire Stage 🎀

Issued instructions get moved from the ReadySet to the IssuedSet. There, instructions wait until they reach the write-back stage. At that exact point, they get removed from the queue and the retire control unit is notified 💖.

When instructions are executed, the retire control unit flags the instruction as "ready to retire."

Instructions are retired in program order. The register file gets notified of the retirement so it can free the physical registers that were allocated for the instruction back during the register renaming stage. Giving back what was borrowed, we stan a sustainable queen 🌸.

🦋 Load/Store Unit and Memory Consistency Model 🦋

To simulate out-of-order execution of memory operations, llvm-mca uses a simulated load/store unit (LSUnit) to simulate the speculative execution of loads and stores ✨.

Each load (or store) consumes an entry in the load (or store) queue. You can use the flags -lqueue and -squeue to limit the number of entries in the load and store queues respectively. The queues are unbounded by default — infinite vibes 💫.

The LSUnit implements a relaxed consistency model for memory loads and stores. The rules are:

1. A younger load is allowed to pass an older load only if there are no intervening stores or barriers between the two loads.
2. A younger load is allowed to pass an older store provided that the load does not alias with the store.
3. A younger store is not allowed to pass an older store.
4. A younger store is not allowed to pass an older load.

By default, the LSUnit optimistically assumes that loads do not alias (-noalias=true) store operations 🫶. Under this assumption, younger loads are always allowed to pass older stores. Essentially, the LSUnit does not even attempt to run any alias analysis to predict when loads and stores do not alias with each other. It's giving "we'll cross that bridge when we get there."

Note that, in the case of write-combining memory, rule 3 could be relaxed to allow reordering of non-aliasing store operations 👀. That being said, at the moment there is no way to further relax the memory model (-noalias is the only option). Essentially, there's no option to specify a different memory type (e.g., write-back, write-combining, write-through; etc.) and consequently no way to weaken, or strengthen, the memory model.

Other limitations are:

• The LSUnit does not know when store-to-load forwarding may occur.
• The LSUnit does not know anything about cache hierarchy and memory types.
• The LSUnit does not know how to identify serializing operations and memory fences.

The LSUnit does not attempt to predict if a load or store hits or misses the L1 cache. It only knows if an instruction "MayLoad" and/or "MayStore." For loads, the scheduling model provides an "optimistic" load-to-use latency (which usually matches the load-to-use latency for when there is a hit in the L1D) 😭.

llvm-mca does not (on its own) know about serializing operations or memory-barrier like instructions. The LSUnit used to conservatively use an instruction's "MayLoad", "MayStore", and unmodeled side effects flags to determine whether an instruction should be treated as a memory-barrier. But that was inaccurate in general, so it got changed — now each instruction has an IsAStoreBarrier and IsALoadBarrier flag 💅. These flags are mca specific and default to false for every instruction. If any instruction should have either of these flags set, it should be done within the target's InstrPostProcess class. For an example, look at the X86InstrPostProcess::postProcessInstruction method within llvm/lib/Target/X86/MCA/X86CustomBehaviour.cpp. Screenshot it, memorize it, you're welcome 🎀.

A load/store barrier consumes one entry of the load/store queue. A load/store barrier enforces ordering of loads/stores 🚦. A younger load cannot pass a load barrier. Also, a younger store cannot pass a store barrier. A younger load has to wait for the memory/load barrier to execute. A load/store barrier is "executed" when it becomes the oldest entry in the load/store queue(s). That also means, by construction, all of the older loads/stores have been executed. It's like the line leader finally reaching the front — everything ahead of it already went through 👑.

In conclusion, the full set of load/store consistency rules are:

1. A store may not pass a previous store.
2. A store may not pass a previous load (regardless of -noalias).
3. A store has to wait until an older store barrier is fully executed.
4. A load may pass a previous load.
5. A load may not pass a previous store unless -noalias is set.
6. A load has to wait until an older load barrier is fully executed.

🌟 In-order Issue and Execute 🌟

In-order processors are modelled as a single InOrderIssueStage stage. It bypasses Dispatch, Scheduler and Load/Store unit entirely — skipping the whole queue, no cap 🧚. Instructions are issued as soon as their operand registers are available and their resource requirements are met. Multiple instructions can be issued in one cycle according to the value of the IssueWidth parameter in LLVM's scheduling model.

Once issued, an instruction gets moved to the IssuedInst set until it's ready to retire. llvm-mca ensures that writes are committed in-order 💖. However, an instruction is allowed to commit writes and retire out-of-order if the RetireOOO property is true for at least one of its writes. Plot twist, but a cute one 🌈.

🛠️ Custom Behaviour 🛠️

okay so here's the tea ☕ — some instructions just do not express themselves perfectly inside their scheduling model, and because of that llvm-mca isn't always able to simulate them perfectly. and like... modifying the scheduling model isn't always a viable option either (maybe because the instruction is modeled incorrectly on purpose, or because the instruction's behaviour is genuinely quite complex 👀). that's literally what the CustomBehaviour class is for — you use it in these cases to enforce proper instruction modeling, often by customizing data dependencies and detecting hazards that llvm-mca has no way of knowing about. no cap. 💅

llvm-mca comes with one generic and multiple target specific CustomBehaviour classes. the generic class gets used if the -disable-cb flag is used, or if a target specific CustomBehaviour class doesn't exist for that target. (the generic class does nothing. it's giving placeholder energy 💀) currently, the CustomBehaviour class is only a part of the in-order pipeline, but bestie there are plans to add it to the out-of-order pipeline in the future ✨

CustomBehaviour's main method is checkCustomHazard() — it uses the current instruction plus a list of all instructions still executing within the pipeline to figure out if the current instruction should be dispatched. as output, the method returns an integer representing the number of cycles that the current instruction must stall for (this can be an underestimate if you don't know the exact number, and a value of 0 represents no stall 🙌).

if you'd like to add a CustomBehaviour class for a target that doesn't already have one, go refer to an existing implementation to see how to set it up. the classes are implemented within the target specific backend (for example /llvm/lib/Target/AMDGPU/MCA/) so that they can access backend symbols. iykyk 🧚

---

👑 Custom Views 👑

llvm-mca comes with several Views such as the Timeline View and Summary View. these Views are generic and can work with most (if not all) targets — iconic behaviour honestly 🌈

if you wish to add a new View to llvm-mca and it does not require any backend functionality that isn't already exposed through MC layer classes (MCSubtargetInfo, MCInstrInfo, etc.), please add it to the /tools/llvm-mca/View/ directory. however, if your new View is target specific AND requires unexposed backend symbols or functionality, you can define it in the /lib/Target/<TargetName>/MCA/ directory. pick your lane bestie 🫶

to enable this target specific View, you will have to use this target's CustomBehaviour class to override the CustomBehaviour::getViews() methods. there are 3 variations of these methods based on where you want your View to appear in the output: getStartViews(), getPostInstrInfoViews(), and getEndViews(). these methods return a vector of Views, so you'll want to return a vector containing all of the target specific Views for the target in question. all of them. every single one. no skipping 🔥

because these target specific (and backend dependent) Views require the CustomBehaviour::getViews() variants, these Views will not be enabled if the -disable-cb flag is used. that's the trade-off, sorry bestie 😭

enabling these custom Views does not affect the non-custom (generic) Views. keep using the usual command line arguments to enable / disable those Views. they're unbothered, moisturized, happy, in their lane 💅✨

---

💖 AUTHOR 💖

maintained by the LLVM Team (https://llvm.org/). they're the main characters, fr fr 👑

---

🎀 COPYRIGHT 🎀

2003-2023, LLVM Project — yeah, that's the whole thing, slay 💜

---

and that's a wrap on the whole page bestie!! 🎉🦋 this gorgeous little manual page is from llvm-mca version 15, dated 2023-07-14 (the source/version + date footer, but make it cute 🎀). go forth, benchmark those instruction streams, and remember: the vibe check never fails ✨💖🌸🍓🧚💜