lrzip(1)

🌸 NAME 🌸

okay bestie, meet lrzip — she's a large-file compression program and she's that girl 💅 (this is the manual for lrzip(1), yes, we're doing the whole thing, sit down, grab a snack 🍓)

---

✨ SYNOPSIS ✨

lrzip [OPTIONS] <file>
lrzip -d [OPTIONS] <file>
lrunzip [OPTIONS] <file>
lrzcat [OPTIONS] <file>
lrztar [lrzip options] <directory>
lrztar -d [lrzip options] <directory>
lrzuntar [lrzip options] <directory>
lrz [lrz options] <directory>
LRZIP=NOCONFIG [lrzip|lrunzip] [OPTIONS] <file>

---

💜 DESCRIPTION 💜

LRZIP is a file compression program built to really pop off on absolutely massive files that are full of long distance redundancy. no cap, that's her whole thing 🔥

lrztar is a wrapper for LRZIP that makes compressing and decompressing whole directories way less of a whole ordeal ✨

---

🎀 OPTIONS SUMMARY 🎀

here's the vibe check on all the options for lrzip, quick and cute:

💖 General options

• -c, --check — check the integrity of the file written on decompression
• -d, --decompress — decompress
• -e, --encrypt[=password] — password protected sha512/aes128 encryption on compression
• -h, -?, --help — show help
• -H, --hash — display md5 hash integrity information
• -i, --info — show compressed file information
• -q, --quiet — don't show compression progress
• -Q, --very-quiet — don't show any output
• -r, --recursive — operate recursively on directories
• -t, --test — test compressed file integrity
• -v[v], --verbose — Increase verbosity
• -V, --version — show version

🌈 Options affecting output

• -D, --delete — delete existing files
• -f, --force — force overwrite of any existing files
• -k, --keep-broken — keep broken or damaged output files
• -o, --outfile filename — specify the output file name and/or path
• -O, --outdir directory — specify the output directory when -o is not used
• -S, --suffix suffix — specify compressed suffix (default '.lrz')

🦋 Options affecting compression

• -b, --bzip2 — bzip2 compression
• -g, --gzip — gzip compression using zlib
• -l, --lzo — lzo compression (ultra fast)
• -n, --no-compress — no backend compression - prepare for other compressor
• -z, --zpaq — zpaq compression (best, extreme compression, extremely slow)

👑 Low level options

• -L, --level level — set lzma/bzip2/gzip compression level (1-9, default 7)
• -N, --nice-level value — Set nice value to value (default 19)
• -p, --threads value — Set processor count to override number of threads
• -m, --maxram size — Set maximum available ram in hundreds of MB, overrides detected amount of available ram
• -T, --threshold — Disable LZ4 compressibility testing
• -U, --unlimited — Use unlimited window size beyond ramsize (potentially much slower)
• -w, --window size — maximum compression window in hundreds of MB, default chosen by heuristic dependent on ram and chosen compression

---

The LRZIP=NOCONFIG environment variable setting can be used to bypass lrzip.conf.

The TMP environment variable will be used for storage of temporary files when needed. TMPDIR may also be stored in the lrzip.conf file.

If no filenames or "-" is specified, stdin/out will be used. 📥📤

---

🔥 OPTIONS 🔥

💫 General options

• -c — this one turns on integrity checking of the file that gets written to disk on decompression. so like, all decompression is already tested internally inside lrzip with either crc32 or md5 hash checking, depending on which version of the archive you've got. but the file written to disk can still get corrupted for other reasons — other userspace problems like faulty library versions, drivers, hardware failure, that whole cursed category 😭. enabling this option makes lrzip do an md5 hash check on the file that's written to disk. when the archive has the md5 value stored inside it, that's what it gets compared against. otherwise it's compared to the value calculated during decompression. it's the extra guarantee that the file written is literally the same as the original archived one 🫶
• -d — Decompress. if you don't use this option then lrzip looks at the name that was used to launch the program instead 👀. if that name contains the string "lrunzip" then -d is automatically set. if it contains the string "lrzcat" then the -d -o - options are automatically set. sneaky. iconic.
• -e
• --encrypt[=password] — Encrypt. this option enables high grade password encryption using a combo of multiply sha512 hashed password, random salt and aes128 CBC encryption 🔐✨. passwords up to 500 characters long are supported, and the encryption mechanism used virtually guarantees that the same file created with the same password will never be the same. like, ever. furthermore, the password hashing is increased according to the date the file is encrypted, increasing the number of CPU cycles required for each password attempt in accordance with Moore's law, so the difficulty of attempting brute force attacks scales proportionally to the power of modern computers. she said "good luck with that" 💅
• -h|-? — Print an options summary page
• -H — this shows the md5 hash value calculated on compressing or decompressing an lrzip archive. by default all compression has the md5 value calculated and stored in all archives since version 0.560. on decompression, when an md5 value has been found, it will be calculated and used for integrity checking. if the md5 value is not stored in the archive, it will not be calculated unless you explicitly specify it with this option, or check integrity (see below) has been requested 👑
• -i — this shows information about a compressed file. it shows the compressed size, the decompressed size, the compression ratio, what compression was used, and what hash checking will be used for internal integrity checking 📊. note: the compression mode is detected from the first block only, and it will show no compression used if the first block was incompressible — even if later blocks were compressible 🙈. if verbose options -v or -vv are added, a breakdown of all the internal blocks and progressively more information pertaining to them will also be shown.
• -q — if this option is specified then lrzip will not show the percentage progress while compressing. heads up: compression happens in bursts with lzma compression, which is the default compression. that means it will progress very rapidly for short periods and then stop for long periods 💀 (not broken, just dramatic.)
• -Q — if this option is specified then lrzip will not show any output to the console except for error messages 🤫
• -r — if this option is specified, lrzip will recursively enter the directories specified, compressing or decompressing every file individually in the same directory. note: for better compression it is recommended to instead combine files in a tar file rather than compress them separately, either manually or with the lrztar helper 🎀
• -t — this tests the compressed file integrity. it does this by decompressing it to a temporary file and then deleting it. in and out, no trace, main character energy ✨
• -v[v] — Increases verbosity. -vv will print more messages than -v 📢
• -V — Print the lrzip version number

🌈 Options affecting output

• -D — if this option is specified then lrzip will delete the source file after successful compression or decompression. when this option is not specified then the source files are not deleted.
• -f — if this option is not specified (Default) then lrzip will not overwrite any existing files. if you set this option then rzip will silently overwrite any files as needed. (yes, silent. no asking. 😳)
• -k — this option will keep broken or damaged files instead of deleting them. when compression or decompression is interrupted either by user or error, or a file decompressed fails an integrity check, it is normally deleted by LRZIP. with -k, she keeps it 🫶
• -o — Set the output file name. if this option is not set then the output file name is chosen based on the input name and the suffix. the -o option cannot be used if more than one file name is specified on the command line ⚠️
• -O — Set the output directory for the default filename. this option cannot be combined with -o ⚠️
• -S — Set the compression suffix. the default is '.lrz'.

🦋 Options affecting compression

• -b — Bzip2 compression. uses bzip2 compression for the 2nd stage, much like the original rzip does.
• -g — Gzip compression. uses gzip compression for the 2nd stage. uses libz compress and uncompress functions.
• -l — LZO Compression. if this option is set then lrzip will use the ultra fast lzo compression algorithm for the 2nd stage. this mode of compression gives bzip2 like compression at the speed it would normally take to simply copy the file, giving excellent compression/time value ⚡🔥
• -n — No 2nd stage compression. if this option is set then lrzip will only perform the long distance redundancy 1st stage compression. while this does not compress any faster than LZO compression, it produces a smaller file that then responds better to further compression (by eg another application), also reducing the compression time substantially ✨
• -z — ZPAQ compression. uses ZPAQ compression which is from the PAQ family of compressors known for having some of the highest compression ratios possible, but at the cost of being extremely slow on both compress and decompress (4x slower than lzma, which is the default) 🐌💀

fr fr, -z is giving "best compression, worst patience." know what you're signing up for.

Low level options 🧠✨

• -L 1..9 — sets the compression level, 1 through 9. Default is level 7 which is the good-all-rounder sweet spot. Lowkey important: the compression level is strongly tied to how much memory lrzip eats, so go peep the -w option if you wanna know more. 👀
• -N value — the default nice value is 19. This one sets the priority scheduling for the lrzip backup or decompression. Valid nice values run from -20 to 19. > Heads up bestie: this does NOT speed up or slow down compression. It's just vibes scheduling, not a performance knob. 💅
• -p value — sets the processor count, which decides how many threads get run. Normally lrzip just scales itself according to the number of CPUs it detects, but this overrides that, in case you wanna use fewer CPUs to either ease the load on your machine or squeeze out better compression. Setting it to 1 will max out compression but won't even try to use more than one CPU. 🧚
• -T — disables the LZ4 compressibility threshold testing that happens when a slower compression back-end is in play. That LZ4 testing is normally done for the slower back-ends, LZMA and ZPAQ, and the reasoning is chef's kiss: if something is completely incompressible by LZ4 then it's gonna be incompressible by them too. So if a block fails to compress with the very fast LZ4, lrzip won't bother trying that block with the slower compressor — instant time savings. Turn this option on and it bypasses the LZ4 testing entirely and attempts to compress every block regardless. 🔥
• -U — unlimited window size, the main character of big-file compression. If this is set and the file being compressed doesn't fit in the available ram, lrzip will use a moving second buffer as a "sliding mmap" which emulates having infinite ram. That gives the most possible compression in the first rzip stage, which can improve compression of ultra large files when they're bigger than the available ram. But — and this is the catch — it runs progressively slower the larger the difference between ram and the file size. So it's best reserved for when you want the smallest possible size on a very large file and the time taken is just not important. 🐌💖
• -w n — sets the maximum allowable compression window size to n, measured in hundreds of megabytes. This is the amount of memory lrzip searches during its first stage of pre-compression, and it's the main thing determining how much benefit lrzip gives you over ordinary compression with the 2nd stage algorithm. If you don't set it (which is recommended, fr fr), the value chosen comes from an internal heuristic in lrzip that uses the most memory that's reasonable, without any hard upper limit. It's limited to 2GB on 32bit machines. And lrzip will always shrink the window size down to the biggest it can be without running out of memory. 🫶

---

🎀 INSTALLATION 🎀

make install, or honestly just install lrzip somewhere in your search path. Iconic either way. ✨

---

🌈 COMPRESSION ALGORITHM 🌈

LRZIP operates in two stages. The first stage finds and encodes large chunks of duplicated data over potentially very long distances in the input file. The second stage is to use a compression algorithm to compress the output of that first stage. The compression algorithm can be chosen to be optimised for extreme size (zpaq), size (lzma — the default), speed (lzo), legacy (bzip2 or gzip), or it can be omitted entirely, doing only the first stage. A one-stage-only compressed file can almost always improve both the compression size and the speed done by a subsequent compression program. 😌👑

The key difference between lrzip and other well known compression algorithms is its ability to take advantage of very long distance redundancy. The well known deflate algorithm used in gzip uses a maximum history buffer of 32k. The block sorting algorithm used in bzip2 is limited to 900k of history. The history buffer in lrzip can be any size long, not even limited by available ram. Not me flexing but okay. 💅

It is quite common these days to need to compress files that contain long distance redundancies. For example, when compressing a set of home directories, several users might have copies of the same file, or of quite similar files. It's also common to have a single file containing large duplicated chunks over long distances — like pdf files containing repeated copies of the same image. Most compression programs won't be able to take advantage of this redundancy, and thus might achieve a much lower compression ratio than lrzip can achieve. Their loss, honestly. 💀

---

🍓 FILES 🍓

LRZIP recognises a configuration file that contains default settings. This configuration is searched for in the current directory, /etc/lrzip, and $HOME/.lrzip. The configuration filename must be lrzip.conf. Don't try to name it anything else, she won't answer. 🙈

---

🦋 ENVIRONMENT 🦋

By default, lrzip will search for and use a configuration file, lrzip.conf. If the user wishes to bypass the file, a startup ENV variable may be set.

LRZIP = NOCONFIG [lrzip|lrunzip] [OPTIONS] <file>

which will force lrzip to ignore the configuration file. Periodt. 🎀

---

💜 HISTORY — Notes on rzip by Andrew Tridgell 💜

The ideas behind rzip were first implemented in 1998 while I was working on rsync. That version was too slow to be practical, and was replaced by this version in 2003.

LRZIP was created by the desire to have better compression and/or speed by Con Kolivas, on blending the lzma and lzo compression algorithms with the rzip first stage, and extending the compression windows to scale with increasing ram sizes. History class but make it slay. ✨

---

😭 BUGS 😭

Nil known.

---

⭐ SEE ALSO ⭐

lrzip.conf(5), lrunzip(1), lrzcat(1), lrztar(1), lrzuntar(1), lrz(1), bzip2(1), gzip(1), lzop(1), rzip(1), zip(1)

---

🔥 DIAGNOSTICS 🔥

Exit status is normally 0; if an error occurs, exit status is 1, and usage errors are 2. Know your numbers, bestie. 💅

---

👑 AUTHOR and CREDITS 👑

lrzip is being extensively bastardised from rzip by Con Kolivas.

rzip was written by Andrew Tridgell.

lzma was written by Igor Pavlov.

lzo was written by Markus Oberhumer.

zpaq was written by Matt Mahoney.

Peter Hyman added informational output, updated LZMA SDK, and added lzma multi-threading capabilities.

If you wish to report a problem, or make a suggestion, then please consult the git repository at: https://github.com/ckolivas/lrzip

lrzip is released under the GNU General Public License version 2. Please see the file COPYING for license details.

---

okay that's the whole page bestie, no cap 🫶 this one's from February 2022, and it's lrzip(1) — go forth and compress, you absolute icon 💖🌸