bup-memtest(1)

📦
version 0.34 · 10 versions
, in fedora 44, nixpkgs unstable

✨ NAME ✨

okay so this is bup-memtest — it tests bup's memory usage statistics, bestie 💖 like literally it's the vibe-check tool for how much RAM your bup repo is eating.

🌸 SYNOPSIS 🌸

bup memtest [options...]

🦋 DESCRIPTION 🦋

So here's the tea, bestie: bup memtest opens up the list of pack indexes in your bup repository, and then it goes searching through that list for a series of nonexistent objects — and after every single cycle it prints out memory usage statistics. Every cycle. No skips. Periodt. 💅

Now, because of the way Unix systems work, the output will usually show a big ol' (and, like, unchanging) value in the VmSize column. Why? Because just mapping the index files in the first place takes a certain amount of virtual address space. It's giving... dramatic numbers for no reason. 😭

But here's the thing — that virtual memory usage is entirely virtual. It does not take any of your actual RAM. Zero. None. It's fake. It's a vibe. Over time, bup uses parts of the indexes, and those need to be loaded from disk, and that is what causes an increase in the VmRSS column. So VmRSS is the one you actually care about. 👑

💖 OPTIONS 💖

  • -n, --number=number — sets the number of objects to search for during each cycle (aka before it prints a line of output). 🍓
  • -c, --cycles=cycles — sets the number of cycles, which is to say, the number of lines of output after the first one. The first line of output is always 0 — that's the baseline, the before-picture, before it searches for any objects at all. 🌸
  • --ignore-midx — ignore any .midx files created by bup midx. This lets you compare memory performance with and without using midx, which is honestly so iconic of it. 🌈
  • --existing — search for existing objects instead of searching for random nonexistent ones. This can greatly affect memory usage and performance, no cap. Note that most of the time, bup save spends most of its time searching for nonexistent objects, since existing ones are probably in unmodified files that we won't be trying to back up anyway. So the default behaviour reflects real bup performance more accurately. But you might want this option anyway, just to make sure you haven't made searching for existing objects much worse than before. 🧚

⭐ EXAMPLES ⭐

First one, the default vibe:

$ bup memtest -n300 -c5
PackIdxList: using 1 index.
               VmSize      VmRSS     VmData      VmStk
        0    20824 kB    4528 kB    1980 kB      84 kB
      300    20828 kB    5828 kB    1984 kB      84 kB
      600    20828 kB    6844 kB    1984 kB      84 kB
      900    20828 kB    7836 kB    1984 kB      84 kB
     1200    20828 kB    8736 kB    1984 kB      84 kB
     1500    20828 kB    9452 kB    1984 kB      84 kB

And now with --ignore-midx, look at all those indexes:

$ bup memtest -n300 -c5 --ignore-midx
PackIdxList: using 361 indexes.
               VmSize      VmRSS     VmData      VmStk
        0    27444 kB    6552 kB    2516 kB      84 kB
      300    27448 kB   15832 kB    2520 kB      84 kB
      600    27448 kB   17220 kB    2520 kB      84 kB
      900    27448 kB   18012 kB    2520 kB      84 kB
     1200    27448 kB   18388 kB    2520 kB      84 kB
     1500    27448 kB   18556 kB    2520 kB      84 kB

👑 DISCUSSION 👑

When you're optimizing bup indexing, your first goal is to keep the VmRSS reasonably low. That's the assignment. However — and this is the important however — it might eventually be necessary to swap in all the indexes, simply because you're searching for a lot of objects, and this will cause your RSS to grow as large as VmSize eventually. Just how it goes. 🙈

And the key word here is eventually. ✨ As long as VmRSS grows reasonably slowly, the amount of disk activity caused by accessing pack indexes is reasonably small — we love a slow burn. If it grows quickly though? bup will probably spend most of its time swapping index data from disk instead of actually running your backup, so backups will run very slowly. Not slay. 💀

The whole purpose of bup memtest is to give you an idea of how fast your memory usage is growing, and to help in optimizing bup for better memory use. If you have memory problems, you might be asked to send the output of bup memtest to help diagnose the problems. So like, keep that receipt. 🎀

Tip: try using bup midx -a or bup midx -f to see if it helps reduce your memory usage. 🫶
Trivia: index memory usage in bup (or git) is only really a problem when adding a large number of previously unseen objects. This is because for each object, we need to absolutely confirm that it isn't already in the database, which requires us to search through all the existing pack indexes to ensure that none of them contain the object in question. In the more obvious case of searching for objects that do exist, the objects being searched for are typically related in some way, which means they probably all exist in a small number of packfiles, so memory usage will be constrained to just those packfile indexes. Constrained. Focused. Main character energy. 💅
Since git users typically don't add a lot of files in a single run, git doesn't really need a program like bup midx. bup, on the other hand, spends most of its time backing up files it hasn't seen before, so its memory usage patterns are different.

🔥 SEE ALSO 🔥

🌈 BUP 🌈

Part of the bup(1) suite. Obviously. 💖

🫶 AUTHORS 🫶

Avery Pennarun apenwarr@gmail.com

okay bestie, that's the whole page, and this beauty is from Bup 0.34, dated 2026-09-01 — go forth and monitor your VmRSS like the icon you are, mwah 💋💜

📦 where this page comes from