borg-compression(1)

πŸ’… NAME πŸ’…

borg-compression β€” the whole entire tea on compression β˜•οΈβœ¨


🫢 DESCRIPTION 🫢

okay so first things first, bestie: mixing different compression methods inside one repo? literally not a problem, not even a little bit. deduplication happens on the source data chunks, not on the compressed or encrypted data. so the compression method is just vibes on top, it doesn't mess with dedup. periodt πŸ’…

and here's the part that lowkey matters: if some specific chunk was once compressed and stored into the repo, then making another backup that also uses this chunk will not change the stored chunk. so if you roll with different compression specs across your backups, whichever one stores a chunk first decides its compression. first come, first compressed. see also borg recreate for that whole situation.

Compression is lz4 by default. if you want something else, you have to say something else β€” it will not read your mind. valid compression specifiers are:

  • none β€” do not compress. we're going in raw, no notes πŸ™ˆ
  • lz4 β€” use lz4 compression. very high speed, very low compression. (the default, the moment, the main character πŸ‘‘)
  • zstd[,L] β€” use zstd ("zstandard") compression, a modern wide-range algorithm. if you don't explicitly give the compression level L (ranging from 1 to 22), it will use level 3. ⚠️ archives compressed with zstd are not compatible with borg < 1.1.4, no cap.
  • zlib[,L] β€” use zlib ("gz") compression. medium speed, medium compression. if you don't explicitly give the compression level L (ranging from 0 to 9), it will use level 6. and listen β€” giving level 0 (which means "no compression", but it still has zlib protocol overhead) is usually pointless, you better use none compression instead. like why 😭
  • lzma[,L] β€” use lzma ("xz") compression. low speed, high compression. if you don't explicitly give the compression level L (ranging from 0 to 9), it will use level 6. giving levels above 6 is pointless and counterproductive, because it does not compress better due to the buffer size used by borg β€” but it wastes lots of CPU cycles and RAM. so it's giving unsustainable, bestie πŸ’€
  • auto,C[,L] β€” use a built-in heuristic to decide per chunk whether to compress or not. the heuristic tries with lz4 whether the data is compressible. for incompressible data, it will not use compression (uses none). for compressible data, it uses the given C[,L] compression β€” with C[,L] being any valid compression specifier. this can be helpful for media files which often cannot be compressed much more. πŸŽ€
  • obfuscate,SPEC,C[,L] β€” use compressed-size obfuscation to make fingerprinting attacks based on the observable stored chunk size more difficult. (iconic behaviour honestly ✨)

about that obfuscation one, note:

  • you must combine this with encryption, or it won't make any sense. fr fr, do not skip this.
  • your repo size will be bigger, of course. bigger repo, bigger feelings 🫢
  • a chunk is limited by the constant MAX_DATA_SIZE (cur. ~20MiB).

The SPEC value determines how the size obfuscation works:

🌸 Relative random reciprocal size variation (multiplicative) 🌸

Size will increase by a factor, relative to the compressed data size. smaller factors are used often, larger factors rarely.

available factors:

 1:     0.01 ..        100
 2:     0.1  ..      1,000
 3:     1    ..     10,000
 4:    10    ..    100,000
 5:   100    ..  1,000,000
 6: 1,000    .. 10,000,000

example probabilities for SPEC 1:

 90   %  0.01 ..   0.1
  9   %  0.1  ..   1
  0.9 %  1    ..  10
  0.09% 10    .. 100

πŸ¦‹ Randomly sized padding up to the given size (additive) πŸ¦‹

110: 1kiB (2 ^ (SPEC - 100))
...
120: 1MiB
...
123: 8MiB (max.)

🧚 Padmé padding (deterministic) 🧚

250: pads to sums of powers of 2, max 12% overhead

Uses the PadmΓ© algorithm to deterministically pad the compressed size to a sum of powers of 2, limiting overhead to 12%. see https://lbarman.ch/blog/padme/ for details if you're curious like that πŸ’œ

πŸŽ€ Examples πŸŽ€

okay and here's the part where you just copy-paste, no thoughts, head empty, slay:

borg create --compression lz4 REPO::ARCHIVE data
borg create --compression zstd REPO::ARCHIVE data
borg create --compression zstd,10 REPO::ARCHIVE data
borg create --compression zlib REPO::ARCHIVE data
borg create --compression zlib,1 REPO::ARCHIVE data
borg create --compression auto,lzma,6 REPO::ARCHIVE data
borg create --compression auto,lzma ...
borg create --compression obfuscate,110,none ...
borg create --compression obfuscate,3,auto,zstd,10 ...
borg create --compression obfuscate,2,zstd,6 ...
borg create --compression obfuscate,250,zstd,3 ...

πŸ‘‘ AUTHOR πŸ‘‘

The Borg Collective πŸ’–


and that's the whole page, bestie!! this was borg-compression(1), from the borg backup tool, dated 2026-07-18. go forth and compress responsibly, ily 🫢✨

πŸ“¦ where this page comes from