LLAMA-SPECULATIVE-SIMPLE(1)

📦
version 10438 · 2 versions
, in ubuntu stonking

💖 LLAMA-SPECULATIVE-SIMPLE(1) 💖

okay so bestie, you found the manual page for llama-speculative-simple — it lives in User Commands land, and yes the title is literally just the command name twice over because that's how man pages do. we're gonna read it together, no skimming, every single flag gets its moment 💅✨

🌸 NAME 🌸

llama-speculative-simple — bestie. that's it. that's the name. she's llama-speculative-simple. iconic, no notes 🎀

🦋 DESCRIPTION 🦋

okay so this is the whole rundown, and it kicks off with the common params section — the shared ~vibes~ that a whole bunch of llama.cpp tools all use. buckle up.

🍓 common params 🍓

  • -h, --help, --usage — prints the usage info and then dips out. your classic "what do i even do here" button 🌈
  • --version — shows you the version and the build info. receipts only 💅
  • -cl, --cache-list — shows you the list of models that are chillin' in your cache 👑
  • --completion-bash — prints a bash completion script for llama.cpp that you can source. tab-completion slay ✨
  • -t, --threads N — how many CPU threads to use while generating (default: <num_cpus>). required vibe: knowing how many cores you have 🧚
    • env: LLAMA_ARG_THREADS
  • -tb, --threads-batch N — number of threads to use during batch and prompt processing (default: same as --threads)
  • -C, --cpu-mask M — CPU affinity mask: arbitrarily long hex. Complements cpu-range (default: "")
  • -Cr, --cpu-range lo-hi — range of CPUs for affinity. Complements --cpu-mask
  • --cpu-strict <0|1> — use strict CPU placement (default: 0)
  • --prio N — set process/thread priority: low(-1), normal(0), medium(1), high(2), realtime(3) (default: 0)
  • --poll <0...100> — use polling level to wait for work (0 - no polling, default: 50)
  • -Cb, --cpu-mask-batch M — CPU affinity mask: arbitrarily long hex. Complements cpu-range-batch (default: same as --cpu-mask)
  • -Crb, --cpu-range-batch lo-hi — ranges of CPUs for affinity. Complements --cpu-mask-batch
  • --cpu-strict-batch <0|1> — use strict CPU placement (default: same as --cpu-strict)
  • --prio-batch N — set process/thread priority: 0-normal, 1-medium, 2-high, 3-realtime (default: 0)
  • --poll-batch <0|1> — use polling to wait for work (default: same as --poll)
  • -c, --ctx-size N — size of the prompt context (default: 0, 0 = loaded from model). your model's little attention span, but make it configurable 🫶
    • env: LLAMA_ARG_CTX_SIZE
  • -n, --predict, --n-predict N — number of tokens to predict (default: -1, -1 = infinity). yes bestie, -1 means go forever, she's in her main character era 🌟
    • env: LLAMA_ARG_N_PREDICT
  • -b, --batch-size N — logical maximum batch size (default: 2048)
    • env: LLAMA_ARG_BATCH
  • -ub, --ubatch-size N — physical maximum batch size (default: 512). the other batch size. yes there's two. don't ask 💀
    • env: LLAMA_ARG_UBATCH
  • --keep N — number of tokens to keep from the initial prompt (default: 0, -1 = all)
  • --swa-full — use full-size SWA cache (default: false). (more info at https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
    • env: LLAMA_ARG_SWA_FULL
  • -fa, --flash-attn [on|off|auto] — set Flash Attention use ('on', 'off', or 'auto', default: 'auto')
    • env: LLAMA_ARG_FLASH_ATTN
  • -p, --prompt PROMPT — prompt to start generation with; for the system message, use -sys. she's the opening line, the conversation starter 💬
  • --perf, --no-perf — whether to enable internal libllama performance timings (default: false)
    • env: LLAMA_ARG_PERF
  • -f, --file FNAME — a file containing the prompt (default: none)
  • -bf, --binary-file FNAME — binary file containing the prompt (default: none)
  • -e, --escape, --no-escape — whether to process escapes sequences (\n, \r, \t, \', \", \\) (default: true)
  • --rope-scaling {none,linear,yarn} — RoPE frequency scaling method, defaults to linear unless specified by the model
    • env: LLAMA_ARG_ROPE_SCALING_TYPE
  • --rope-scale N — RoPE context scaling factor, expands context by a factor of N
    • env: LLAMA_ARG_ROPE_SCALE
  • --rope-freq-base N — RoPE base frequency, used by NTK-aware scaling (default: loaded from model)
    • env: LLAMA_ARG_ROPE_FREQ_BASE
  • --rope-freq-scale N — RoPE frequency scaling factor, expands context by a factor of 1/N
    • env: LLAMA_ARG_ROPE_FREQ_SCALE
  • --yarn-orig-ctx N — YaRN: original context size of model (default: 0 = model training context size)
    • env: LLAMA_ARG_YARN_ORIG_CTX
  • --yarn-ext-factor N — YaRN: extrapolation mix factor (default: -1.00, 0.0 = full interpolation)
    • env: LLAMA_ARG_YARN_EXT_FACTOR
  • --yarn-attn-factor N — YaRN: scale sqrt(t) or attention magnitude (default: -1.00)
    • env: LLAMA_ARG_YARN_ATTN_FACTOR
  • --yarn-beta-slow N — YaRN: high correction dim or alpha (default: -1.00)
    • env: LLAMA_ARG_YARN_BETA_SLOW
  • --yarn-beta-fast N — YaRN: low correction dim or beta (default: -1.00)
    • env: LLAMA_ARG_YARN_BETA_FAST
  • -kvo, --kv-offload, -nkvo, --no-kv-offload — whether to enable KV cache offloading (default: enabled)
    • env: LLAMA_ARG_KV_OFFLOAD
  • --repack, -nr, --no-repack — whether to enable weight repacking (default: enabled)
    • env: LLAMA_ARG_REPACK
  • --no-host — bypass host buffer allowing extra buffers to be used
    • env: LLAMA_ARG_NO_HOST
  • -ctk, --cache-type-k TYPE — KV cache data type for K
    • allowed values: f32, f16, bf16, q8_0, q4_0, q4_1, iq4_nl, q5_0, q5_1 (default: f16)
    • env: LLAMA_ARG_CACHE_TYPE_K
  • -ctv, --cache-type-v TYPE — KV cache data type for V
    • allowed values: f32, f16, bf16, q8_0, q4_0, q4_1, iq4_nl, q5_0, q5_1 (default: f16)
    • env: LLAMA_ARG_CACHE_TYPE_V
  • -dt, --defrag-thold N — KV cache defragmentation threshold (DEPRECATED)
    • env: LLAMA_ARG_DEFRAG_THOLD
  • -np, --parallel N — number of parallel sequences to decode (default: 1)
    • env: LLAMA_ARG_N_PARALLEL
  • --mlock — DEPRECATED in favor of --load-mode: force system to keep model in RAM rather than swapping or compressing
    • env: LLAMA_ARG_MLOCK
  • --mmap, --no-mmap — DEPRECATED in favor of --load-mode: whether to memory-map model. (if mmap disabled, slower load but may reduce pageouts if not using mlock)
    • env: LLAMA_ARG_MMAP
  • -dio, --direct-io, -ndio, --no-direct-io — DEPRECATED in favor of --load-mode: use DirectIO if available
    • env: LLAMA_ARG_DIO
  • -lm, --load-mode MODE — model loading mode (default: auto)
    • auto: mmap, unless a device does not support it
    • none: no special loading mode
    • mmap: memory-map model (if mmap disabled, slower load but may reduce pageouts if not using mlock)
    • mlock: force system to keep model in RAM rather than swapping or compressing
    • mmap+mlock: mmap + force system to keep model in RAM rather than swapping or compressing
    • dio: use DirectIO if available
    • env: LLAMA_ARG_LOAD_MODE
  • --numa TYPE — attempt optimizations that help on some NUMA systems
    • distribute: spread execution evenly over all nodes
    • isolate: only spawn threads on CPUs on the node that execution started on
    • numactl: use the CPU map provided by numactl
    • if run without this previously, it is recommended to drop the system page cache before using this
    • see https://github.com/ggml-org/llama.cpp/issues/1437
    • env: LLAMA_ARG_NUMA
  • -dev, --device <dev1,dev2,..> — comma-separated list of devices to use for offloading (none = don't offload)
    • use --list-devices to see a list of available devices
    • env: LLAMA_ARG_DEVICE
  • --list-devices — print list of available devices and exit. window shopping but for GPUs 👀
  • -ot, --override-tensor <tensor name pattern>=<buffer type>,... — override tensor buffer type
    • env: LLAMA_ARG_OVERRIDE_TENSOR
  • -cmoe, --cpu-moe — keep all Mixture of Experts (MoE) weights in the CPU
    • env: LLAMA_ARG_CPU_MOE
  • -ncmoe, --n-cpu-moe N — keep the Mixture of Experts (MoE) weights of the first N layers in the CPU
    • env: LLAMA_ARG_N_CPU_MOE
  • -ngl, --gpu-layers, --n-gpu-layers N — max. number of layers to store in VRAM, either an exact number, 'auto', or 'all' (default: auto)
    • env: LLAMA_ARG_N_GPU_LAYERS
  • -sm, --split-mode {none,layer,row,tensor} — how to split the model across multiple GPUs, one of:
    • none: use one GPU only
    • layer (default): split layers and KV across GPUs (pipelined)
    • row: split weight across GPUs by rows (parallelized)
    • tensor: split weights and KV across GPUs (parallelized, EXPERIMENTAL)
    • env: LLAMA_ARG_SPLIT_MODE
  • -ts, --tensor-split N0,N1,N2,... — fraction of the model to offload to each GPU, comma-separated list of proportions, e.g. 3,1
    • env: LLAMA_ARG_TENSOR_SPLIT
  • -mg, --main-gpu INDEX — the GPU to use for the model (with split-mode = none), or for intermediate results and KV (with split-mode = row) (default: 0)
    • env: LLAMA_ARG_MAIN_GPU
  • -fit, --fit [on|off] — whether to adjust unset arguments to fit in device memory ('on' or 'off', default: 'on')
    • env: LLAMA_ARG_FIT
  • -fitt, --fit-target MiB0,MiB1,MiB2,... — target margin per device for --fit, comma-separated list of values, single value is broadcast across all devices, default: 1024
    • env: LLAMA_ARG_FIT_TARGET
  • -fitc, --fit-ctx N — minimum ctx size that can be set by --fit option, default: 4096
    • env: LLAMA_ARG_FIT_CTX
  • --check-tensors — check model tensor data for invalid values (default: false)
  • --override-kv KEY=TYPE:VALUE,... — advanced option to override model metadata by key. to specify multiple overrides, either use comma-separated values. types: int, float, bool, str. example: --override-kv tokenizer.ggml.add_bos_token=bool:false,tokenizer.ggml.add_eos_token=bool:false
  • --op-offload, --no-op-offload — whether to offload host tensor operations to device (default: true)
  • --lora FNAME — path to LoRA adapter (use comma-separated values to load multiple adapters)
  • --lora-scaled FNAME:SCALE,... — path to LoRA adapter with user defined scaling (format: FNAME:SCALE,...) note: use comma-separated values
  • --control-vector FNAME — add a control vector note: use comma-separated values to add multiple control vectors
  • --control-vector-scaled FNAME:SCALE,... — add a control vector with user defined scaling SCALE note: use comma-separated values (format: FNAME:SCALE,...)
  • --control-vector-layer-range START END — layer range to apply the control vector(s) to, start and end inclusive
  • -m, --model FNAME — model path to load 👑 the star of the show, fr fr
    • env: LLAMA_ARG_MODEL
  • -mu, --model-url MODEL_URL — model download url (default: unused)
    • env: LLAMA_ARG_MODEL_URL

okay bestie that was a lot and we're not even done yet — more parts are coming, so hold on to your ponytail and don't go anywhere 🫶✨

  • -dr, --docker-repo [<repo>/]<model>[:quant] — the Docker Hub model repo, bestie 🐳 repo is optional and defaults to ai/, quant is optional and defaults to :latest. example: gemma3 (default: unused) (env: LLAMA_ARG_DOCKER_REPO)
  • -hf, -hfr, --hf-repo <user>/<model>[:quant] — the Hugging Face model repo. quant is optional, case-insensitive, defaults to Q4_K_M, and if Q4_K_M doesn't exist it falls back to the first file in the repo. mmproj also gets downloaded automatically if it's available — to turn that off, add --no-mmproj. example: ggml-org/GLM-4.7-Flash-GGUF:Q4_K_M (default: unused) (env: LLAMA_ARG_HF_REPO)
  • -hff, --hf-file FILE — a specific Hugging Face model file 💅 if you specify it, it overrides the quant in --hf-repo (default: unused) (env: LLAMA_ARG_HF_FILE)
  • -hft, --hf-token TOKEN — your Hugging Face access token 🔑 (default: the value from the HF_TOKEN environment variable) (env: HF_TOKEN)
  • --log-disable — log disable 🙈
  • --log-file FNAME — log to a file (env: LLAMA_ARG_LOG_FILE)
  • --log-colors [on|off|auto] — set colored logging ('on', 'off', or 'auto', default: 'auto'). 'auto' enables colors when output is to a terminal (env: LLAMA_ARG_LOG_COLORS)
  • -v, --verbose, --log-verbose — crank the verbosity level to infinity, aka log all the messages, which is genuinely useful for debugging 🔍
  • --offline — offline mode: forces use of cache, prevents network access ✈️ (env: LLAMA_ARG_OFFLINE)
  • -lv, --verbosity, --log-verbosity N — set the verbosity threshold. messages with a higher verbosity than this get ignored. values:
    • 0: generic output
    • 1: error
    • 2: warning
    • 3: info
    • 4: trace (more info)
    • 5: debug

(default: 3)

(env: LLAMA_ARG_LOG_VERBOSITY)

  • --log-prefix, --no-log-prefix — enable prefix in log messages (env: LLAMA_ARG_LOG_PREFIX)
  • --log-timestamps, --no-log-timestamps — enable timestamps in log messages (env: LLAMA_ARG_LOG_TIMESTAMPS)
  • --spec-draft-type-k, -ctkd, --cache-type-k-draft TYPE — KV cache data type for K, for the draft model 🎀 allowed values: f32, f16, bf16, q8_0, q4_0, q4_1, iq4_nl, q5_0, q5_1 (default: f16) (env: LLAMA_ARG_SPEC_DRAFT_CACHE_TYPE_K)
  • --spec-draft-type-v, -ctvd, --cache-type-v-draft TYPE — KV cache data type for V, for the draft model 💖 allowed values: f32, f16, bf16, q8_0, q4_0, q4_1, iq4_nl, q5_0, q5_1 (default: f16) (env: LLAMA_ARG_SPEC_DRAFT_CACHE_TYPE_V)

✨ sampling params ✨

  • --samplers SAMPLERS — the samplers that will be used for generation, in order, separated by ';' (default: penalties;dry;top_n_sigma;top_k;typ_p;top_p;min_p;xtc;temperature)
  • -s, --seed SEED — RNG seed 🎲 (default: -1, use random seed for -1)
  • --sampler-seq, --sampling-seq SEQUENCE — the simplified sequence for samplers that will be used (default: edskypmxt)
  • --ignore-eos — ignore end of stream token and keep on generating, no stopping 🏃‍♀️ (implies --logit-bias EOS-inf)
  • --temp, --temperature N — temperature 🔥 (default: 0.80)
  • --top-k N — top-k sampling (default: 40, 0 = disabled) (env: LLAMA_ARG_TOP_K)
  • --top-p N — top-p sampling (default: 0.95, 1.0 = disabled)
  • --min-p N — min-p sampling (default: 0.05, 0.0 = disabled)
  • --top-nsigma, --top-n-sigma N — top-n-sigma sampling (default: -1.00, -1.0 = disabled)
  • --xtc-probability N — xtc probability (default: 0.00, 0.0 = disabled)
  • --xtc-threshold N — xtc threshold (default: 0.10, 1.0 = disabled)
  • --typical, --typical-p N — locally typical sampling, parameter p (default: 1.00, 1.0 = disabled)
  • --repeat-last-n N — last n tokens to consider for penalize (default: 64, 0 = disabled)
  • --repeat-penalty N — penalize repeat sequence of tokens (default: 1.00, 1.0 = disabled)
  • --presence-penalty N — repeat alpha presence penalty (default: 0.00, 0.0 = disabled)
  • --frequency-penalty N — repeat alpha frequency penalty (default: 0.00, 0.0 = disabled)
  • --dry-multiplier N — set the DRY sampling multiplier (default: 0.00, 0.0 = disabled)
  • --dry-base N — set the DRY sampling base value (default: 1.75)
  • --dry-allowed-length N — set the allowed length for DRY sampling (default: 2)
  • --dry-penalty-last-n N — set the DRY penalty for the last n tokens (default: 64, 0 = disable)
  • --dry-sequence-breaker STRING — add a sequence breaker for DRY sampling, which clears out the default breakers ('\n', ':', '"', '*') in the process; use "none" to not use any sequence breakers 🚫
  • --adaptive-target N — adaptive-p: select tokens near this probability (valid range 0.0 to 1.0; negative = disabled) (default: -1.00) [(more info)](https://github.com/ggml-org/llama.cpp/pull/17927) — the link is https://github.com/ggml-org/llama.cpp/pull/17927 if you wanna peek 👀
  • --adaptive-decay N — adaptive-p: decay rate for target adaptation over time. lower values are more reactive, higher values are more stable. (valid range 0.0 to 0.99) (default: 0.90)
  • --dynatemp-range N — dynamic temperature range (default: 0.00, 0.0 = disabled)
  • --dynatemp-exp N — dynamic temperature exponent (default: 1.00)
  • --mirostat N — use Mirostat sampling. Top K, Nucleus and Locally Typical samplers are ignored if used, jsyk 💀 (default: 0, 0 = disabled, 1 = Mirostat, 2 = Mirostat 2.0)
  • --mirostat-lr N — Mirostat learning rate, parameter eta (default: 0.10)
  • --mirostat-ent N — Mirostat target entropy, parameter tau (default: 5.00)
  • -l, --logit-bias TOKEN_ID(+/-)BIAS — modifies the likelihood of a token appearing in the completion, i.e. --logit-bias 15043+1 to increase likelihood of token ' Hello', or --logit-bias 15043-1 to decrease likelihood of token ' Hello' 🎚️
  • --grammar GRAMMAR — BNF-like grammar to constrain generations (see samples in the grammars/ dir)
  • --grammar-file FNAME — file to read grammar from
  • -j, --json-schema SCHEMA — JSON schema to constrain generations (https://json-schema.org/), e.g. {} for any JSON object. For schemas w/ external $refs, use --grammar + example/json_schema_to_grammar.py instead
  • -jf, --json-schema-file FILE — file containing a JSON schema to constrain generations (https://json-schema.org/), e.g. {} for any JSON object. For schemas w/ external $refs, use --grammar + example/json_schema_to_grammar.py instead
  • -bs, --backend-sampling — enable backend sampling (experimental) (default: disabled) (env: LLAMA_ARG_BACKEND_SAMPLING)
it says experimental bestie, so like… be aware 👀

🔮 speculative params 🔮

  • --spec-draft-hf, -hfd, -hfrd, --hf-repo-draft <user>/<model>[:quant] — same as --hf-repo, but for the draft model (default: unused) (env: LLAMA_ARG_SPEC_DRAFT_HF_REPO)
  • --spec-draft-threads, -td, --threads-draft N — number of threads to use during generation (default: same as --threads)
  • --spec-draft-threads-batch, -tbd, --threads-batch-draft N — number of threads to use during batch and prompt processing (default: same as --threads-draft)
  • --spec-draft-cpu-mask, -Cd, --cpu-mask-draft M — draft model CPU affinity mask. Complements cpu-range-draft (default: same as --cpu-mask)
  • --spec-draft-cpu-range, -Crd, --cpu-range-draft lo-hi — ranges of CPUs for affinity. Complements --cpu-mask-draft
  • --spec-draft-cpu-strict, --cpu-strict-draft <0|1> — use strict CPU placement for draft model (default: same as --cpu-strict)
  • --spec-draft-prio, --prio-draft N — set draft process/thread priority: 0-normal, 1-medium, 2-high, 3-realtime (default: 0)
  • --spec-draft-poll, --poll-draft <0|1> — use polling to wait for draft model work (default: same as --poll)
  • --spec-draft-cpu-mask-batch, -Cbd, --cpu-mask-batch-draft M — draft model CPU affinity mask. Complements cpu-range-draft (default: same as --cpu-mask)
  • --spec-draft-cpu-range-batch, -Crbd, --cpu-range-batch-draft lo-hi — ranges of CPUs for affinity. Complements --cpu-mask-draft-batch)
  • --spec-draft-cpu-strict-batch, --cpu-strict-batch-draft <0|1> — use strict CPU placement for draft model (default: --cpu-strict-draft)
  • --spec-draft-prio-batch, --prio-batch-draft N — set draft process/thread priority: 0-normal, 1-medium, 2-high, 3-realtime (default: 0)
  • --spec-draft-poll-batch, --poll-batch-draft <0|1> — use polling to wait for draft model work (default: --poll-draft)
  • --spec-draft-override-tensor, -otd, --override-tensor-draft <tensor name pattern>=<buffer type>,... — override tensor buffer type for draft model
  • --spec-draft-cpu-moe, -cmoed, --cpu-moe-draft — keep all Mixture of Experts (MoE) weights in the CPU for the draft model (env: LLAMA_ARG_SPEC_DRAFT_CPU_MOE)
  • --spec-draft-n-cpu-moe, --spec-draft-ncmoe, -ncmoed, --n-cpu-moe-draft N — keep the Mixture of Experts (MoE) weights of the first N layers in the CPU for the draft model (env: LLAMA_ARG_SPEC_DRAFT_N_CPU_MOE)
  • --spec-draft-n-max N — number of tokens to draft for speculative decoding (default: 3) (env: LLAMA_ARG_SPEC_DRAFT_N_MAX)
  • --spec-draft-n-min N — minimum number of draft tokens to use for speculative decoding (default: 0) (env: LLAMA_ARG_SPEC_DRAFT_N_MIN)
  • --spec-draft-p-split, --draft-p-split P — speculative decoding split probability (default: 0.10) (env: LLAMA_ARG_SPEC_DRAFT_P_SPLIT)
  • --spec-draft-p-min, --draft-p-min P — minimum speculative decoding probability (greedy) (default: 0.00) (env: LLAMA_ARG_SPEC_DRAFT_P_MIN)
  • --spec-draft-backend-sampling, --no-spec-draft-backend-sampling — offload draft sampling to the backend (default: enabled) (env: LLAMA_ARG_SPEC_DRAFT_BACKEND_SAMPLING)
  • --spec-draft-device, -devd, --device-draft <dev1,dev2,..> — comma-separated list of devices to use for offloading the draft model (none = don't offload). use --list-devices to see a list of available devices 🖥️
  • --spec-draft-ngl, -ngld, --gpu-layers-draft, --n-gpu-layers-draft N — max. number of draft model layers to store in VRAM, either an exact number, 'auto', or 'all' (default: auto) (env: LLAMA_ARG_N_GPU_LAYERS_DRAFT)
  • --spec-draft-model, -md, --model-draft FNAME — draft model for speculative decoding (default: unused) (env: LLAMA_ARG_SPEC_DRAFT_MODEL)
  • --spec-type — a comma-separated list of the types of speculative decoding you wanna use, bestie. The options are none, draft-simple, draft-eagle3, draft-mtp, draft-dflash, draft-dspark, ngram-simple, ngram-map-k, ngram-map-k4v, ngram-mod, ngram-cache (default: none)

(env: LLAMA_ARG_SPEC_TYPE)

  • --spec-ngram-mod-n-min N — the minimum number of ngram tokens to use for ngram-based speculative decoding (default: 48)
  • --spec-ngram-mod-n-max N — the maximum number of ngram tokens to use for ngram-based speculative decoding (default: 64)
  • --spec-ngram-mod-n-match N — the ngram-mod lookup length (default: 24)
  • --spec-ngram-simple-size-n N — the ngram size N for ngram-simple speculative decoding, aka the length of the lookup n-gram (default: 12)
  • --spec-ngram-simple-size-m N — the ngram size M for ngram-simple speculative decoding, aka the length of the draft m-gram (default: 48)
  • --spec-ngram-simple-min-hits N — the minimum hits for ngram-simple speculative decoding (default: 1)
  • --spec-ngram-map-k-size-n N — the ngram size N for ngram-map-k speculative decoding, aka the length of the lookup n-gram (default: 12)
  • --spec-ngram-map-k-size-m N — the ngram size M for ngram-map-k speculative decoding, aka the length of the draft m-gram (default: 48)
  • --spec-ngram-map-k-min-hits N — the minimum hits for ngram-map-k speculative decoding (default: 1)
  • --spec-ngram-map-k4v-size-n N — the ngram size N for ngram-map-k4v speculative decoding, aka the length of the lookup n-gram (default: 12)
  • --spec-ngram-map-k4v-size-m N — the ngram size M for ngram-map-k4v speculative decoding, aka the length of the draft m-gram (default: 48)
  • --spec-ngram-map-k4v-min-hits N — the minimum hits for ngram-map-k4v speculative decoding (default: 1)
  • --draft, --draft-n, --draft-max N — girl this argument has been REMOVED 😭 use --spec-draft-n-max or --spec-ngram-mod-n-max instead (env: LLAMA_ARG_DRAFT_MAX)
  • --draft-min, --draft-n-min N — also removed, we don't do that anymore 💀 use --spec-draft-n-min or --spec-ngram-mod-n-min instead (env: LLAMA_ARG_DRAFT_MIN)

----- example-specific params -----

  • -co, --color [on|off|auto] — colorize the output so you can tell the prompt and user input apart from the generations, like a little visual slay for your eyeballs 🎨

It takes 'on', 'off', or 'auto' (default: 'auto').

And 'auto' means colors turn on when the output is going to a terminal. iconic behaviour, honestly ✨

aaaaand that's the whole page, bestie!! 🫶 This masterpiece is from the Debian crew, September 2026, and it lives at LLAMA-SPECULATIVE-SIMPLE(1). Go forth and speculate responsibly 💅🌸

📦 where this page comes from