NIXTAMAL-MANIFEST(5)

📦
version 2.4.0 · 2 versions
, in nixpkgs unstable

🌸 NAME 🌸

Nixtamal Manifest(5) — setting up for pinning down inputs. No cap, that's the whole assignment. This page is lowkey the blueprint for how Nixtamal pins inputs, and we're gonna walk it together 💅

✨ SYNOPSIS ✨

Okay so Nixtamal uses KDL for its manifest describing inputs. At the highest level, this includes:

  • manifest version
  • default hashing algorithm
  • patches (optional)
  • list of inputs
    • the input kind & its specific attributes
    • a command to check if ‘fresh’
    • hashing information
    • patches to apply to the input
Note: A KDL Schema file is shipped alongside this documentation for use with schema-aware editors and validators. It can be found in the installation directory under share/nixtamal/manifest.kdl. Schema girlies stay winning.

🎀 Default manifest.kdl 🎀

Here's the default vibe, fr:

version "0.5.0"
inputs {
        nixpkgs {
                archive {
                        url "https://github.com/NixOS/nixpkgs/archive/{{fresh_value}}.tar.gz"
                }
                hash algorithm=SHA-256
                fetch-time eval
                fresh-cmd {
                        $ git ls-remote "https://github.com/NixOS/nixpkgs.git" --refs "refs/heads/nixos-unstable"
                | cut -f1
                }
        }
}

🌈 TOP-LEVEL NODES 🌈

  • version — version of the Nixtamal spec the manifest.kdl is using. Iconic little version stamp.
  • default-hash-algorithm — hash algorithm to use by default for inputs when the input does not note its hash algorithm. Defaults to SHA-256.
  • patches — map of patches to be applied to inputs where the patch name should be unique. Each patch has a URL (first argument). See Patches.
  • default-fetch-time — default fetch time for inputs when not specified at the input level. Can be eval (fetch during Nix evaluation using builtins.fetch*) or build (fetch during build using pkgs.fetch*). Defaults to eval.
  • default-gc — EXPERIMENTAL: default garbage collection (GC) tracking behavior for inputs when not specified at the input level. Can be track or skip (allow garbage collection). Defaults to skip. Experimental means bestie, read the label.
  • inputs — map of inputs to be pinned where the input+node name should be unique & will be used in the Nix output as well as logs & errors. See Input node.

🔥 FETCH TIME 🔥

Inputs can be fetched either during Nix evaluation or during build time:

  • eval — fetch the input during Nix evaluation using builtins.fetch* functions. This is the default behavior and is suitable for inputs that need to be available during evaluation, such as the bootstrap Nixpkgs.
  • build — fetch the input during build time using pkgs.fetch* functions. This is suitable for most inputs and allows for better caching and parallel fetching.

The fetch time can be set globally using default-fetch-time or per-input using the fetch-time property on file, archive, and git input types.

Caution! Inputs with patches cannot use eval fetch time, as patches are applied during build time. The system will automatically enforce this constraint.

🗑️ GARBAGE COLLECTION 🗑️

The gc per-input property & document-level default-gc controls whether the input's Nix store path is protected from Nixtamal-controlled garbage collection by creating a silo GC root. The feature requires the NIXTAMAL_EXPERIMENTAL_GC environment variable. Silo GC root. Very mysterious. Very protected.

  • track — protect from garbage collection (when NIXTAMAL_EXPERIMENTAL_GC is set).
  • skip — allow garbage collection (default).

🩹 PATCHES 🩹

Patches are defined at the top-level and can be applied to any input. This allows defining a patch once and applying it to multiple inputs. Reuse queen behavior.

  • prop: name — a string that specifies the path name in the Nix store. The Nix store has a very conservative set of values, narrower than ASCII, that are allowed as path which will definitely & unfortunately affect Unicode usage. Defaults to $NAME-patch if possible, otherwise warns for default behavior.
  • prop: frozen — a boolean (frozen=#true) that prevents trying refresh or otherwise get a new fresh value for this patch.
  • prop: gc — override the GC tracking behavior for this patch. Can be track or skip. When absent, the value from default-gc is used. See Garbage collection.
  • url — templated node URL or file reference for the patch. Supports https://, http://, and file:// URLs.
  • fresh-cmd — command (with or without pipes using $ & | nodes) that can be shelled out to return a string that will be locked as the fresh command value which can be used both to prevent unnecessary prefetching, but also for use in a Templated node.
  • hash — optional node for hash algorithm information. The algorithm property will be used when prefetching, locking, & for integrity verification. The optional expected property may be used to assert a known hash. If not specified, no hash verification is performed. Defaults to the top-level default-hash-algorithm or SHA-256.

👑 INPUT NODE 👑

At a high level these should be seen as:

  • prop: frozen — a boolean (frozen=#true) that prevents trying refresh or otherwise get a new fresh value for this input.
  • prop: gc — override the GC tracking behavior for this input. Can be track or skip. When absent, the value from default-gc is used. See Garbage collection.
  • prop: name — a string that specifies the path name in the Nix store. The Nix store has a very conservative set of values, narrower than ASCII, that are allowed as path which will definitely & unfortunately affect Unicode usage. Defaults to $NAME-src if possible, otherwise warns for default behavior.
  • "kind" — there are specific nodes for each different type of supported fetchers/prefetchers: file, archive, git, darcs, pijul, fossil (with more to come in the future). Fetcher expansion pack loading.
  • hash — an optional node for hash algorithm information for an input. The algorithm property will be used when prefetching, locking, & for importing (which falls back to top-level default-hash-algorithm or defined default SHA-256). The optional expected property may be used to assert a known hash.
Caution! The bootstrapping Nixpkgs pin (either manually set or using nixpkgs-nixtamal or nixpkgs as defaults) must be SHA-256 to be compatible with builtins.fetchTarball.
  • fresh-cmd — command (with or without pipes using $ & | nodes) that can be shelled out to return a string that will be locked as the fresh command value which can be used both to prevent unnecessary prefetching, but also for use in a Templated node.
  • patches — list of patch names (as arguments) to apply to this input. Patches are defined at the top-level in the Patches section.

📁 File 📁

  • prop: fetch-time — property to set when to fetch this input: eval or build. Defaults to the top-level default-fetch-time or build.
  • url — templated node URL reference for the input.
  • mirrors — templated node URL mirror references for the input.

📦 Archive 📦

  • prop: fetch-time — property to set when to fetch this input: eval or build. Defaults to the top-level default-fetch-time or build.
  • url — templated node URL reference for the input.
  • mirrors — templated node URL mirror references for the input.
  • extension — annotation to help pkgs.fetchzip with the correct extension type for the download (such as a /download URL that is a .tar.gz).

See: https://nixos.org/manual/nixpkgs/unstable/#sec-pkgs-fetchers-fetchzip Link drop, but make it documentation.

🌿 Git 🌿

  • prop: fetch-time — property to set when to fetch this input: eval or build. Defaults to the top-level default-fetch-time or build.
  • repository — templated node repository reference for the input.
  • mirrors — templated node repository mirror references for the input.
Warning: Probably not yet supported upstream.
  • "reference" — branch or tag or ref node as the reference point for getting stable reference. tag provides convenience over using ref with full tag paths.
  • submodules — leaf node for enabling submodules on a repository.
  • lfs — leaf node for enabling Git LFS on a repository.

🧶 Darcs 🧶

  • repository — templated node repository reference for the input.
  • mirrors — templated node repository mirror references for the input.
Note: Recently upstreamed. See: https://github.com/NixOS/nixpkgs/pull/467172
  • "reference" — context or tag node as the reference point for getting stable reference; in the case of Darcs, if neither is supplied a context will be assumed & copied from nix-prefetch-darcs.

🪶 Pijul 🪶

  • remote — templated node remote reference for the input.
  • mirrors — templated node remote mirror references for the input.
Note: Recently upstreamed. See: https://github.com/NixOS/nixpkgs/pull/467890
  • "reference" — channel or state or change (not recommended) node as the reference point for getting stable reference; if unsure, try channel main.

🦴 Fossil 🦴

  • repository — templated node repository reference for the input.
  • "reference" — branch or tag or check-in node as the reference point for getting stable reference.

🧚 TEMPLATED NODE 🧚

Some nodes have values with string substitution via Jingoo https://tategakibunko.github.io/jingoo/templates/templates.en.html, which is probably overkill, but could give you flexibility with if statements. The templated nodes include:

  • inputs >> file > url
  • inputs >> file > mirrors
  • inputs >> archive > url
  • inputs >> archive > mirrors
  • inputs >> git > repository
  • inputs >> git > mirrors
  • inputs >> darcs > repository
  • inputs >> darcs > mirrors
  • inputs >> pijul > remote
  • inputs >> pijul > mirrors
  • inputs >> fossil > repository
  • inputs >> fresh-cmd > $
  • inputs >> fresh-cmd > |

The input kind affects the values for substitution:

okay so each of these input flavors has its own lil field list — and the types are doing so much work here, bestie 👀💅

💾 file

  • name — string — the input name, periodt.
  • fresh_value — string nullable — the fresh command return value.

📦 archive

  • name — string — the input name.
  • fresh_value — string nullable — the fresh command return value.

🌿 git

  • name — string — the input name.
  • fresh_value — string nullable — the fresh command return value.
  • branch — string nullable — branch name.
  • ref — string nullable — reference name.
  • datetime — string nullable — datetime of latest revision.
  • lfs — bool — repository uses LFS.
  • submodules — bool — repository uses submodules.
  • rev / revision — string nullable — latest revision.

🌀 darcs

  • name — string — the input name.
  • fresh_value — string nullable — the fresh command return value.
  • context — string nullable — path to context file.
  • tag — string nullable — tag.
  • datetime — string nullable — datetime of latest patch.
  • weak_hash — string nullable — latest weak hash of the repository.

🧚 pijul

  • name — string — the input name.
  • fresh_value — string nullable — the fresh command return value.
  • channel — string nullable — remote channel.
  • change — string nullable — change.
  • datetime — string nullable — datetime of latest patch.
  • state — string nullable — latest state of the remote or supplied state.

🦴 fossil

  • name — string — the input name.
  • fresh_value — string nullable — the fresh command return value.
  • branch — string nullable — branch name.
  • tag — string nullable — tag.
  • checkin — string nullable — check-in.
  • date — string nullable — date of latest patch.

🌈 Input showcase 🌈

Okay bestie, gather round — these are the little show-and-tell configs, and they are lowkey iconic. No cap, every snippet here is doing its own main-character thing. 🫶

🌀 Darcs using exposed WeakHash to avoid needless refresh

Here’s the Darcs setup using an exposed WeakHash so it doesn’t do a needless refresh. The config is literally:

          nixtamal {
                  darcs {
                          repository "https://darcs.toastal.in.th/nixtamal/stable/"
                          mirrors "https://smeder.ee/~toastal/nixtamal.darcs"
                  }
                  fresh-cmd {
                          $ curl -sL "https://darcs.toastal.in.th/nixtamal/stable/_darcs/weak_hash"
                  }
          }

🍓 Local directory checking for latest modification

Checking a local directory for the latest modification? This soupault-plugins moment is giving exactly that:

          soupault-plugins {
                  file {
                          url "file:///home/toastal/my-project"
                  }
                  fresh-cmd {
                          $ find "/home/toastal/my-project" "-print0"
                          | xargs "-0" stat -c %Y
                          | sort -n
                          | tail -n1
                  }
          }

🦋 Eval time file with mirror + templated nodes

This one is an eval-time file with a mirror plus templated nodes, and the {{fresh_value}} placeholder is serving in both URLs — obsessed:

          mozilla-tls-guidelines {
                  file fetch-time=eval {
                          url "https://ssl-config.mozilla.org/guidelines/{{fresh_value}}.json"
                          mirrors "https://raw.githubusercontent.com/mozilla/ssl-config-generator/refs/tags/v{{fresh_value}}/src/static/guidelines/{{fresh_value}}.json"
                  }
                  fresh-cmd {
                          $ curl -sL "https://wiki.mozilla.org/Security/Server_Side_TLS"
                          | htmlq -w -t "table.wikitable:last-of-type > tbody > tr:nth-child(2) > td:first-child"
                          | head -n1
                  }
          }

💜 Basic Pijul with BLAKE3 hash

Basic Pijul, but make it a BLAKE3 hash. The vibes are immaculate:

          pijul {
                  pijul {
                          remote "https://nest.pijul.com/pijul/pijul"
                          channel main
                  }
                  hash algorithm=BLAKE3
          }

🎀 Inputs with patches

Patches are the main character here, fr fr. The patches block and the inputs block are literally this:

          patches {
                  nixpkgs-pr123 "https://github.com/NixOS/nixpkgs/pull/123.diff"
                  my-fix "./patches/my-fix.patch"
          }

          inputs {
                  nixpkgs {
                          git {
                                  repository "https://github.com/NixOS/nixpkgs.git"
                                  ref "refs/heads/nixos-unstable"
                          }
                          patches "nixpkgs-pr123" "my-fix"
                  }
                  nixpkgs-stable {
                          git {
                                  repository "https://github.com/NixOS/nixpkgs.git"
                                  ref "refs/heads/nixos-24.05"
                          }
                          patches "my-fix"
                  }
          }

Local patches (starting with ./ or ../) are applied directly from the repository and & be tracked by your VCS. “Remote” patches (meaning not local to the repository such as HTTPS, absolute paths with file:, & so forth) are fetched & hashed during nixtamal lock.

👑 Author 👑

toastal

That’s the whole page, bestie 💖 The footer is serving version 1.4.0, source Nixtamal Manifest(5), and the date slot is empty (), so that’s the tea. Byeee ✨🎀

📦 where this page comes from