LLAMA-BENCH(1)

💅 LLAMA-BENCH(1) — User Commands, bestie edition

okay so like, welcome to llama-bench, the manual page, section 1 (User Commands). yes i'm the one explaining it to you. no i will not be normal about it. let's go 🦋

---

✨ NAME ✨

llama-bench — it's literally called llama-bench. that's it. that's the name. iconic minimalism, no notes 💅

---

🌸 DESCRIPTION 🌸

so the vibe is:

usage: llama-bench [options]

that's the whole thing. llama-bench takes options and then it does its little benchmarking thing. periodt.

---

🎀 options: 🎀

• -h, --help — help. the classic. when in doubt, ask for help, bestie 🫶
• --numa <distribute|isolate|numactl> — numa mode. default: disabled. so if you don't say anything, it stays off, okay? 😌
• -r, --repetitions <n> — how many times each test gets run. default: 5. yes, five times, we're thorough in this household 💖
• --prio <-1|0|1|2|3> — process/thread priority. default: 0.
• --delay <0...N> (seconds) — a lil pause between each test. default: 0. she needs a breather 🧚
• -o, --output <csv|json|jsonl|md|sql> — the output format that gets printed to stdout. default: md.
• -oe, --output-err <csv|json|jsonl|md|sql> — the output format that gets printed to stderr. default: none. so nothing by default, chill 😌
• --list-devices — lists the available devices and then peaces out. bye 👋
• -v, --verbose — verbose output. she talks a LOT. love that for her 🗣️
• --progress — prints test progress indicators. little dashboard moment ✨
• --no-warmup — skips the warmup runs before benchmarking. no stretching, straight into it 🏃♀️
• -fitt, --fit-target <MiB> — fits the model to device memory with this margin per device, in MiB. default: off.
• -fitc, --fit-ctx <n> — the minimum ctx size for --fit-target. default: 4096.

---

🧪 test parameters: 🧪

• -m, --model <filename> — default: models/7B/ggml-model-q4_0.gguf. that's the model file, bestie 🍓
• -hf, -hfr, --hf-repo <user>/<model>[:quant] — Hugging Face model repository. the quant is optional and it's case-insensitive; it defaults to Q4_K_M, OR it falls back to the first file in the repo if Q4_K_M doesn't exist. example: ggml-org/GLM-4.7-Flash-GGUF:Q4_K_M. default: unused.
• -hff, --hf-file <file> — the Hugging Face model file. if you specify this one, it overrides the quant in --hf-repo. default: unused.
• -hft, --hf-token <token> — Hugging Face access token. default: the value from the HF_TOKEN environment variable 🔑
• --offline — offline mode: forces use of cache and prevents network access. default: disabled. airplane mode but for benchmarking ✈️
• -p, --n-prompt <n> — default: 512.
• -n, --n-gen <n> — default: 128.
• -pg <pp,tg> — default: (empty, bestie. nothing. the void.) 🕳️
• -d, --n-depth <n> — default: 0.
• -b, --batch-size <n> — default: 2048.
• -ub, --ubatch-size <n> — default: 512.
• -ctk, --cache-type-k <t> — default: f16.
• -ctv, --cache-type-v <t> — default: f16.
• -t, --threads <n> — default: <num_cpus>. she'll figure out your cpu count herself, so smart 💅
• -C, --cpu-mask <hex,hex> — default: 0x0.
• --cpu-strict <0|1> — default: 0.
• --poll <0...100> — default: 50.
• -ngl, --n-gpu-layers <n> — default: -1.
• -ncmoe, --n-cpu-moe <n> — default: 0.
• -sm, --split-mode <none|layer|row|tensor> — default: layer.
• -mg, --main-gpu <i> — default: 0.
• -nkvo, --no-kv-offload <0|1> — default: 0.
• -fa, --flash-attn <on|off|auto> — default: auto.
• -dev, --device <dev0/dev1/...> — default: auto.
• -lm, --load-mode <auto|none|mmap|mlock|mmap+mlock|dio> — default: auto.
• -mmp, --mmap <0|1> — DEPRECATED IN FAVOUR OF --load-mode. like, she's retired, stop calling her 📵
• -dio, --direct-io <0|1> — also DEPRECATED IN FAVOUR OF --load-mode. same energy. rest in peace 🪦
• -embd, --embeddings <0|1> — default: 0.
• -ts, --tensor-split <ts0/ts1/..> — default: 0.
• -ot, --override-tensor <tensor name pattern>=<buffer type>;... — default: disabled.
• -nopo, --no-op-offload <0|1> — default: 0.
• --no-host <0|1> — default: 0.

---

⭐ real talk though, and this part is IMPORTANT, so don't scroll past it:

Multiple values can be given for each parameter by separating them with ',' or by specifying the parameter multiple times. Ranges can be given as 'first-last' or 'first-last+step' or 'first-last*mult'.

like. you can just hand her a whole list. a lil buffet of values. she'll run them all. obsessed honestly 😭💖

---

and that's the whole page, bestie!! this was the debian build, dated September 2026 — mwah, love u, go benchmark something 🔥👑💜