LLAMA-PERPLEXITY(1)

📦
version 10438 · 2 versions
, in ubuntu stonking

💖 LLAMA-PERPLEXITY(1) 💖 — User Commands

okay bestie, sit down, hydrate, and let me tell you about llama-perplexity — it's lowkey one of the most iconic little tools in the llama.cpp fam, and this is the part of its manual where we spill the common params, the shared settings basically every llama.cpp binary shares. it's giving "universal cheat codes" and honestly? obsessed. 🎀

✨ NAME ✨

llama-perplexity — literally just llama-perplexity. that's the name, that's the whole vibe, no notes. 💅

🦋 DESCRIPTION 🦋

okay so the manual doesn't do a big dramatic intro here — it just dives straight into the shared options block and goes off. main character energy fr fr. here's the whole thing, every single flag, no crumbs left behind:

💫 ----- common params ----- 💫

  • -h, --help, --usage — print usage and exit. the classic "i'm lost bestie" button. 🫶
  • --version — show version and build info. proof you're running the right build, no cap.
  • -cl, --cache-list — show list of models in cache. window-shopping your downloads. ✨
  • --completion-bash — print source-able bash completion script for llama.cpp. yes, it will make your shell finish your sentences. iconic. 💅
  • -t, --threads N — number of CPU threads to use during generation (default: <num_cpus>)
    • env: LLAMA_ARG_THREADS
  • -tb, --threads-batch N — number of threads to use during batch and prompt processing (default: same as --threads)
  • -C, --cpu-mask M — CPU affinity mask: arbitrarily long hex. Complements cpu-range (default: "")
  • -Cr, --cpu-range lo-hi — range of CPUs for affinity. Complements --cpu-mask
  • --cpu-strict <0|1> — use strict CPU placement (default: 0)
  • --prio N — set process/thread priority : low(-1), normal(0), medium(1), high(2), realtime(3) (default: 0)
  • --poll <0...100> — use polling level to wait for work (0 - no polling, default: 50)
  • -Cb, --cpu-mask-batch M — CPU affinity mask: arbitrarily long hex. Complements cpu-range-batch (default: same as --cpu-mask)
  • -Crb, --cpu-range-batch lo-hi — ranges of CPUs for affinity. Complements --cpu-mask-batch
  • --cpu-strict-batch <0|1> — use strict CPU placement (default: same as --cpu-strict)
  • --prio-batch N — set process/thread priority : 0-normal, 1-medium, 2-high, 3-realtime (default: 0)
  • --poll-batch <0|1> — use polling to wait for work (default: same as --poll)
  • -c, --ctx-size N — size of the prompt context (default: 512, 0 = loaded from model)
    • env: LLAMA_ARG_CTX_SIZE
  • -n, --predict, --n-predict N — number of tokens to predict (default: -1, -1 = infinity)
    • env: LLAMA_ARG_N_PREDICT
  • -b, --batch-size N — logical maximum batch size (default: 2048)
    • env: LLAMA_ARG_BATCH
  • -ub, --ubatch-size N — physical maximum batch size (default: 512)
    • env: LLAMA_ARG_UBATCH
  • --keep N — number of tokens to keep from the initial prompt (default: 0, -1 = all)
  • --swa-full — use full-size SWA cache (default: false) [more info: https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055]
    • env: LLAMA_ARG_SWA_FULL
  • -fa, --flash-attn [on|off|auto] — set Flash Attention use ('on', 'off', or 'auto', default: 'auto')
    • env: LLAMA_ARG_FLASH_ATTN
  • -p, --prompt PROMPT — prompt to start generation with; for system message, use -sys
  • --perf, --no-perf — whether to enable internal libllama performance timings (default: false)
    • env: LLAMA_ARG_PERF
  • -f, --file FNAME — a file containing the prompt (default: none)
  • -bf, --binary-file FNAME — binary file containing the prompt (default: none)
  • -e, --escape, --no-escape — whether to process escapes sequences (\n, \r, \t, \', \", \\) (default: false)
  • --rope-scaling {none,linear,yarn} — RoPE frequency scaling method, defaults to linear unless specified by the model
    • env: LLAMA_ARG_ROPE_SCALING_TYPE
  • --rope-scale N — RoPE context scaling factor, expands context by a factor of N
    • env: LLAMA_ARG_ROPE_SCALE
  • --rope-freq-base N — RoPE base frequency, used by NTK-aware scaling (default: loaded from model)
    • env: LLAMA_ARG_ROPE_FREQ_BASE
  • --rope-freq-scale N — RoPE frequency scaling factor, expands context by a factor of 1/N
    • env: LLAMA_ARG_ROPE_FREQ_SCALE
  • --yarn-orig-ctx N — YaRN: original context size of model (default: 0 = model training context size)
    • env: LLAMA_ARG_YARN_ORIG_CTX
  • --yarn-ext-factor N — YaRN: extrapolation mix factor (default: -1.00, 0.0 = full interpolation)
    • env: LLAMA_ARG_YARN_EXT_FACTOR
  • --yarn-attn-factor N — YaRN: scale sqrt(t) or attention magnitude (default: -1.00)
    • env: LLAMA_ARG_YARN_ATTN_FACTOR
  • --yarn-beta-slow N — YaRN: high correction dim or alpha (default: -1.00)
    • env: LLAMA_ARG_YARN_BETA_SLOW
  • --yarn-beta-fast N — YaRN: low correction dim or beta (default: -1.00)
    • env: LLAMA_ARG_YARN_BETA_FAST
  • -kvo, --kv-offload, -nkvo, --no-kv-offload — whether to enable KV cache offloading (default: enabled)
    • env: LLAMA_ARG_KV_OFFLOAD
  • --repack, -nr, --no-repack — whether to enable weight repacking (default: enabled)
    • env: LLAMA_ARG_REPACK
  • --no-host — bypass host buffer allowing extra buffers to be used
    • env: LLAMA_ARG_NO_HOST
  • -ctk, --cache-type-k TYPE — KV cache data type for K
    • allowed values: f32, f16, bf16, q8_0, q4_0, q4_1, iq4_nl, q5_0, q5_1 (default: f16)
    • env: LLAMA_ARG_CACHE_TYPE_K
  • -ctv, --cache-type-v TYPE — KV cache data type for V
    • allowed values: f32, f16, bf16, q8_0, q4_0, q4_1, iq4_nl, q5_0, q5_1 (default: f16)
    • env: LLAMA_ARG_CACHE_TYPE_V
  • -dt, --defrag-thold N — KV cache defragmentation threshold (DEPRECATED) 💀
    • env: LLAMA_ARG_DEFRAG_THOLD
  • -np, --parallel N — number of parallel sequences to decode (default: 1)
    • env: LLAMA_ARG_N_PARALLEL
  • --mlock — DEPRECATED in favor of --load-mode: force system to keep model in RAM rather than swapping or compressing
    • env: LLAMA_ARG_MLOCK
  • --mmap, --no-mmap — DEPRECATED in favor of --load-mode: whether to memory-map model. (if mmap disabled, slower load but may reduce pageouts if not using mlock)
    • env: LLAMA_ARG_MMAP
  • -dio, --direct-io, -ndio, --no-direct-io — DEPRECATED in favor of --load-mode: use DirectIO if available
    • env: LLAMA_ARG_DIO
  • -lm, --load-mode MODE — model loading mode (default: auto)
    • auto: mmap, unless a device does not support it
    • none: no special loading mode
    • mmap: memory-map model (if mmap disabled, slower load but may reduce pageouts if not using mlock)
    • mlock: force system to keep model in RAM rather than swapping or compressing
    • mmap+mlock: mmap + force system to keep model in RAM rather than swapping or compressing
    • dio: use DirectIO if available
    • env: LLAMA_ARG_LOAD_MODE
  • --numa TYPE — attempt optimizations that help on some NUMA systems
    • distribute: spread execution evenly over all nodes
    • isolate: only spawn threads on CPUs on the node that execution started on
    • numactl: use the CPU map provided by numactl
    • if run without this previously, it is recommended to drop the system page cache before using this see https://github.com/ggml-org/llama.cpp/issues/1437
    • env: LLAMA_ARG_NUMA
  • -dev, --device <dev1,dev2,..> — comma-separated list of devices to use for offloading (none = don't offload)
    • use --list-devices to see a list of available devices
    • env: LLAMA_ARG_DEVICE
  • --list-devices — print list of available devices and exit
  • -ot, --override-tensor <tensor name pattern>=<buffer type>,... — override tensor buffer type
    • env: LLAMA_ARG_OVERRIDE_TENSOR
  • -cmoe, --cpu-moe — keep all Mixture of Experts (MoE) weights in the CPU
    • env: LLAMA_ARG_CPU_MOE
  • -ncmoe, --n-cpu-moe N — keep the Mixture of Experts (MoE) weights of the first N layers in the CPU
    • env: LLAMA_ARG_N_CPU_MOE
  • -ngl, --gpu-layers, --n-gpu-layers N — max. number of layers to store in VRAM, either an exact number, 'auto', or 'all' (default: auto)
    • env: LLAMA_ARG_N_GPU_LAYERS
  • -sm, --split-mode {none,layer,row,tensor} — how to split the model across multiple GPUs, one of:
    • none: use one GPU only
    • layer (default): split layers and KV across GPUs (pipelined)
    • row: split weight across GPUs by rows (parallelized)
    • tensor: split weights and KV across GPUs (parallelized, EXPERIMENTAL)
    • env: LLAMA_ARG_SPLIT_MODE
  • -ts, --tensor-split N0,N1,N2,... — fraction of the model to offload to each GPU, comma-separated list of proportions, e.g. 3,1
    • env: LLAMA_ARG_TENSOR_SPLIT
  • -mg, --main-gpu INDEX — the GPU to use for the model (with split-mode = none), or for intermediate results and KV (with split-mode = row) (default: 0)
    • env: LLAMA_ARG_MAIN_GPU
  • -fit, --fit [on|off] — whether to adjust unset arguments to fit in device memory ('on' or 'off', default: 'on')
    • env: LLAMA_ARG_FIT
  • -fitt, --fit-target MiB0,MiB1,MiB2,... — target margin per device for --fit, comma-separated list of values, single value is broadcast across all devices, default: 1024
    • env: LLAMA_ARG_FIT_TARGET
  • -fitc, --fit-ctx N — minimum ctx size that can be set by --fit option, default: 4096
    • env: LLAMA_ARG_FIT_CTX
  • --check-tensors — check model tensor data for invalid values (default: false)
  • --override-kv KEY=TYPE:VALUE,... — advanced option to override model metadata by key. to specify multiple overrides, either use comma-separated values. types: int, float, bool, str. example:
--override-kv tokenizer.ggml.add_bos_token=bool:false,tokenizer.ggml.add_eos_token=bool:false
  • --op-offload, --no-op-offload — whether to offload host tensor operations to device (default: true)
  • --lora FNAME — path to LoRA adapter (use comma-separated values to load multiple adapters)
  • --lora-scaled FNAME:SCALE,... — path to LoRA adapter with user defined scaling (format: FNAME:SCALE,...) note: use comma-separated values
  • --control-vector FNAME — add a control vector note: use comma-separated values to add multiple control vectors
  • --control-vector-scaled FNAME:SCALE,... — add a control vector with user defined scaling SCALE note: use comma-separated values (format: FNAME:SCALE,...)
  • --control-vector-layer-range START END — layer range to apply the control vector(s) to, start and end inclusive
  • -m, --model FNAME — model path to load
    • env: LLAMA_ARG_MODEL
  • -mu, --model-url MODEL_URL — model download url (default: unused)
    • env: LLAMA_ARG_MODEL_URL
  • -dr, --docker-repo [<repo>/]<model>[:quant]

and that's where part one dips out bestie — the -dr, --docker-repo entry is literally just getting started and the rest is pulling up in the next chunk like a dramatic season finale. keep scrolling, we're not done yet 💜

  • (bestie we literally got cut off mid-bullet, so here's the rest of this one 💀) — Docker Hub model repository. repo is optional, and it defaults to ai/. quant is optional, and it defaults to :latest. example: gemma3 (default: unused) (env: LLAMA_ARG_DOCKER_REPO)
  • -hf, -hfr, --hf-repo <user>/<model>[:quant] — Hugging Face model repository, the one you actually want 90% of the time 🦋. quant is optional, case-insensitive, and defaults to Q4_K_M — or it falls back to the first file in the repo if Q4_K_M doesn't exist. mmproj also gets downloaded automatically if available. to disable that, add --no-mmproj. example: ggml-org/GLM-4.7-Flash-GGUF:Q4_K_M (default: unused) (env: LLAMA_ARG_HF_REPO)
  • -hff, --hf-file FILE — Hugging Face model file. If you specify it, it will override the quant in --hf-repo (default: unused) (env: LLAMA_ARG_HF_FILE)
  • -hft, --hf-token TOKEN — Hugging Face access token (default: value from HF_TOKEN environment variable) (env: HF_TOKEN)
  • --log-disable — the entire description is just "Log disable", and honestly? that's the whole vibe. it disables logging. periodt 💅
  • --log-file FNAME — Log to file (env: LLAMA_ARG_LOG_FILE)
  • --log-colors [on|off|auto] — Set colored logging ('on', 'off', or 'auto', default: 'auto'). 'auto' enables colors when output is to a terminal (env: LLAMA_ARG_LOG_COLORS)
  • -v, --verbose, --log-verbose — Set verbosity level to infinity (i.e. log all messages, useful for debugging). infinity, bestie. not a number. infinity 🔥
  • --offline — Offline mode: forces use of cache, prevents network access (env: LLAMA_ARG_OFFLINE)
  • -lv, --verbosity, --log-verbosity N — Set the verbosity threshold. Messages with a higher verbosity will be ignored. Values:
    • 0: generic output
    • 1: error
    • 2: warning
    • 3: info
    • 4: trace (more info)
    • 5: debug

(default: 3) (env: LLAMA_ARG_LOG_VERBOSITY)

  • --log-prefix, --no-log-prefix — Enable prefix in log messages (env: LLAMA_ARG_LOG_PREFIX)
  • --log-timestamps, --no-log-timestamps — Enable timestamps in log messages (env: LLAMA_ARG_LOG_TIMESTAMPS)
  • --spec-draft-type-k, -ctkd, --cache-type-k-draft TYPE — KV cache data type for K for the draft model. allowed values: f32, f16, bf16, q8_0, q4_0, q4_1, iq4_nl, q5_0, q5_1 (default: f16) (env: LLAMA_ARG_SPEC_DRAFT_CACHE_TYPE_K)
  • --spec-draft-type-v, -ctvd, --cache-type-v-draft TYPE — KV cache data type for V for the draft model. allowed values: f32, f16, bf16, q8_0, q4_0, q4_1, iq4_nl, q5_0, q5_1 (default: f16) (env: LLAMA_ARG_SPEC_DRAFT_CACHE_TYPE_V)

🎲 sampling params 🎲

okay this is the part where the model gets its little personality transplant. buckle up 🌸

  • --samplers SAMPLERS — samplers that will be used for generation in the order, separated by ; (default: penalties;dry;top_n_sigma;top_k;typ_p;top_p;min_p;xtc;temperature)
  • -s, --seed SEED — RNG seed (default: -1, use random seed for -1)
  • --sampler-seq, --sampling-seq SEQUENCE — simplified sequence for samplers that will be used (default: edskypmxt)
  • --ignore-eos — ignore end of stream token and continue generating (implies --logit-bias EOS-inf)
  • --temp, --temperature N — temperature (default: 0.80)
  • --top-k N — top-k sampling (default: 40, 0 = disabled) (env: LLAMA_ARG_TOP_K)
  • --top-p N — top-p sampling (default: 0.95, 1.0 = disabled)
  • --min-p N — min-p sampling (default: 0.05, 0.0 = disabled)
  • --top-nsigma, --top-n-sigma N — top-n-sigma sampling (default: -1.00, -1.0 = disabled)
  • --xtc-probability N — xtc probability (default: 0.00, 0.0 = disabled)
  • --xtc-threshold N — xtc threshold (default: 0.10, 1.0 = disabled)
  • --typical, --typical-p N — locally typical sampling, parameter p (default: 1.00, 1.0 = disabled)
  • --repeat-last-n N — last n tokens to consider for penalize (default: 64, 0 = disabled)
  • --repeat-penalty N — penalize repeat sequence of tokens (default: 1.00, 1.0 = disabled)
  • --presence-penalty N — repeat alpha presence penalty (default: 0.00, 0.0 = disabled)
  • --frequency-penalty N — repeat alpha frequency penalty (default: 0.00, 0.0 = disabled)
  • --dry-multiplier N — set DRY sampling multiplier (default: 0.00, 0.0 = disabled)
  • --dry-base N — set DRY sampling base value (default: 1.75)
  • --dry-allowed-length N — set allowed length for DRY sampling (default: 2)
  • --dry-penalty-last-n N — set DRY penalty for the last n tokens (default: 64, 0 = disable)
  • --dry-sequence-breaker STRING — add sequence breaker for DRY sampling, clearing out default breakers ('\n', ':', '"', '*') in the process; use "none" to not use any sequence breakers
  • --adaptive-target N — adaptive-p: select tokens near this probability (valid range 0.0 to 1.0; negative = disabled) (default: -1.00) (more info: https://github.com/ggml-org/llama.cpp/pull/17927)
  • --adaptive-decay N — adaptive-p: decay rate for target adaptation over time. lower values are more reactive, higher values are more stable. (valid range 0.0 to 0.99) (default: 0.90)
  • --dynatemp-range N — dynamic temperature range (default: 0.00, 0.0 = disabled)
  • --dynatemp-exp N — dynamic temperature exponent (default: 1.00)
  • --mirostat N — use Mirostat sampling. Top K, Nucleus and Locally Typical samplers are ignored if used. (default: 0, 0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0)
  • --mirostat-lr N — Mirostat learning rate, parameter eta (default: 0.10)
  • --mirostat-ent N — Mirostat target entropy, parameter tau (default: 5.00)
  • -l, --logit-bias TOKEN_ID(+/-)BIAS — modifies the likelihood of token appearing in the completion, i.e. --logit-bias 15043+1 to increase likelihood of token ' Hello', or --logit-bias 15043-1 to decrease likelihood of token ' Hello'
  • --grammar GRAMMAR — BNF-like grammar to constrain generations (see samples in grammars/ dir)
  • --grammar-file FNAME — file to read grammar from
  • -j, --json-schema SCHEMA — JSON schema to constrain generations (https://json-schema.org/), e.g. {} for any JSON object. For schemas w/ external $refs, use --grammar + example/json_schema_to_grammar.py instead
  • -jf, --json-schema-file FILE — File containing a JSON schema to constrain generations (https://json-schema.org/), e.g. {} for any JSON object. For schemas w/ external $refs, use --grammar + example/json_schema_to_grammar.py instead
  • -bs, --backend-sampling — enable backend sampling (experimental) (default: disabled) (env: LLAMA_ARG_BACKEND_SAMPLING)
it's giving "we're still cooking this one". experimental means experimental, bestie 🧚

🧪 example-specific params 🧪

  • -kvu, --kv-unified, -no-kvu, --no-kv-unified — use single unified KV buffer shared across all sequences (default: enabled if number of slots is auto) (env: LLAMA_ARG_KV_UNIFIED)
  • --context-shift, --no-context-shift — whether to use context shift on infinite text generation (default: disabled) (env: LLAMA_ARG_CONTEXT_SHIFT)
  • --chunks N — max number of chunks to process (default: -1, -1 = all)
  • --warmup, --no-warmup — whether to perform warmup with an empty run (default: enabled)
  • --hellaswag — compute HellaSwag score over random tasks from datafile supplied with -f
  • --hellaswag-tasks N — number of tasks to use when computing the HellaSwag score (default: 400)
  • --winogrande — compute Winogrande score over random tasks from datafile supplied with -f
  • --winogrande-tasks N — number of tasks to use when computing the Winogrande score (default: 0)
  • --multiple-choice — compute multiple choice score over random tasks from datafile supplied with -f
  • --multiple-choice-tasks N — number of tasks to use when computing the multiple choice score (default: 0)
  • --kl-divergence — computes KL-divergence to logits provided via --kl-divergence-base
  • --save-all-logits, --kl-divergence-base FNAME — set logits file
  • --ppl-stride N — stride for perplexity calculation (default: 0)
  • --ppl-output-type <0|1> — output type for perplexity calculation (default: 0)

and that's a wrap on the flags, bestie 🎀 you made it to the end of a very long options list and honestly? main character energy. go run something unhinged with --temp 2.0 and tell me how it went 🫶

this page ships from debian and its date stamp says September 2026, and yeah, it's the one and only LLAMA-PERPLEXITY(1) manual — love her, respect her, read her before you file a bug 💜

📦 where this page comes from