bup-midx(1)

📦
version 0.34 · 10 versions
, in fedora 44, nixpkgs unstable

💖 bup-midx(1) 💖

okay bestie, so this is the manual for bup-midx, from the General Commands Manual section of the bup suite. grab a snack, we're learning about multi-index files ✨

✨ NAME ✨

bup-midx — make a multi-index (.midx) file out of a whole bunch of .idx files. yes, that's literally the whole job. she's focused. 💅

🦋 SYNOPSIS 🦋

bup midx [-o outfile] <-a|-f|idxnames...>

🌸 DESCRIPTION 🌸

so bup midx takes one or more git pack index (.idx) files and creates a multi-index (.midx) file out of them. that's it, that's the gig, and she eats every time. 👑

important lil note: you should no longer need to run this command by hand, bestie. it gets run automatically by bup-save(1) and similar commands. she's on autopilot now. it's giving self-sufficient. 😌

🎀 OPTIONS 🎀

  • -o, --output=filename.midx — use the filename you pick for the .midx file. if you don't say anything, the default is auto-generated and honestly she'll name herself. 💖
  • -a, --auto — automatically generate new .midx files for any .idx files where it would be appropriate. she decides. she's iconic like that. ✨
  • -f, --force — force generation of a single new .midx file containing all your .idx files, even if other .midx files already exist. heads up: this will result in the fastest backup performance, but it may take a long time to run. so like, slay, but also be patient. ⏳
  • --dir=packdir — specify the directory containing the .idx/.midx files to work with. the default is $BUP_DIR/objects/pack. yes that's a variable, yes it matters, write it down. 📝
  • --max-files — maximum number of .idx files to open at a time. use this if you have an especially small number of file descriptors available, so that midx can actually complete (though possibly non-optimally) even if it can't open all your .idx files at once. the default value of this option should be fine for most people, so like, don't stress unless you're in the file-descriptor trenches. 🙈
  • --check — validate a .midx file by making sure all objects in its contained .idx files exist inside the .midx. may be useful for debugging. detective mode activated. 🔍💀

🌈 EXAMPLES 🌈

$ bup midx -a
Merging 21 indexes (2278559 objects).
Table size: 524288 (17 bits)
Reading indexes: 100.00% (2278559/2278559), done.
midx-b66d7c9afc4396187218f2936a87b865cf342672.midx

look at her go. 21 indexes, 2278559 objects, zero complaints, output file named. ate and left no crumbs. 🔥

💜 DISCUSSION 💜

okay this is the lore part, lock in. 🧚

by default, bup uses git-formatted pack files, which consist of a pack file (containing objects) and an idx file (containing a sorted list of object names and their offsets in the .pack file). two files, one destiny, you know how it is. 🫶

normal idx files are convenient because it means you can use git(1) to access your backup datasets. however — and this is the plot twist — idx files can get slow when you have a lot of very large packs (which git typically doesn't have, but bup often does). 😭

bup .midx files consist of a single sorted list of all the objects contained in all the .pack files it references. this list can be binary searched in about log2(m) steps, where m is the total number of objects. math but make it fashionable. 💅

to further speed up the search, midx files also have a variable-sized fanout table that reduces the first n steps of the binary search. with the help of this fanout table, bup can narrow down which page of the midx file a given object id would be in (if it exists) with a single lookup. thus, typical searches will only need to swap in two pages: one for the fanout table, and one for the object id. efficiency? she invented it. ⭐

midx files are most useful when creating new backups, since searching for a nonexistent object in the repository necessarily requires searching through all the index files to ensure that it does not exist. (searching for objects that do exist can be optimized; for example, consecutive objects are often stored in the same pack, so we can search that one first using an MRU algorithm.) so yeah, main character energy for the midx whenever new backups are happening. 👑

🌟 SEE ALSO 🌟

💖 BUP 💖

part of the bup(1) suite. the whole squad, honestly. 🌸

👑 AUTHORS 👑

Avery Pennarun apenwarr@gmail.com — ty for your service, legend. 🫡

and that's the whole page, bestie!! this was Bup 0.34, dated 2026-09-01, and it's the bup-midx(1) manual — go forth, let her auto-generate your midx files, and don't run her by hand unless you truly must. periodt. 💅✨

📦 where this page comes from