DUFF(1)

💅 duff(1) — the duplicate file finder, bestie edition 🎀

okay so. you know how you have like 47 copies of the same screenshot in your Downloads folder and it's giving chaos?? duff is the local legend that finds all those clusters of duplicates for you. it lives in the LOCAL manual, section 1, and it's about to be your new obsession. ✨

---

🌸 NAME 🌸

duff — duplicate file finder 🦋

---

🫶 SYNOPSIS 🫶

duff [-0HLPaeqprtz] [-d function] [-f format] [-l limit] [file ...]
duff [-h]
duff [-v]

---

💖 DESCRIPTION 💖

so basically, duff reports clusters of duplicates in whatever files and/or directories you hand it. in the default mode it prints a customizable header, and then the names of all the files in the cluster. in excess mode it skips the header and instead, for each cluster, prints the names of all but the first of the files it includes.

if you don't give it any files as arguments, duff just reads file names from stdin. lowkey convenient.

🚨 important lore: as of version 0.4, duff ignores symbolic links to files, because that behavior was conceptually broken. so the -H, -L and -P options now apply only to directories. periodt.

the following options are available:

---

✨ OPTIONS ✨

• -0 — if it's reading file names from stdin, assume they're null-terminated instead of separated by newlines. also, when printing file names and cluster headers, terminate them with null characters instead of newlines. this is the moment for file names with whitespace or other non-standard characters, bestie — it's giving bulletproof. 💅
• -H — follow symbolic links listed on the command line. this overrides any previous -L or -P option. note that this only applies to directories, because symbolic links to files are never followed.
• -L — follow all symbolic links. this overrides any previous -H or -P option. note that this only applies to directories, because symbolic links to files are never followed.
• -P — don't follow any symbolic links. this overrides any previous -H or -L option. this is the default. note that this only applies to directories, because symbolic links to files are never followed.
• -a — include hidden files and directories when searching recursively. yes bestie, the dotfiles too. 👑
• -d function — the message digest function to use. the supported functions are sha1, sha256, sha384 and sha512. the default is sha1. 🌈
• -e — excess mode. list all but one file from each cluster of duplicates. also suppresses output of the cluster header. this is the one when you want to automate removal of duplicate files and you genuinely don't care which duplicates get removed. 🙈
• -f format — set the format of the cluster header. if the header is set to the empty string, no header line is printed. 🎀

the following escape sequences are available:

• %n — the number of files in the cluster.
• %c — a legacy synonym for %d, for compatibility reasons.
• %d — the message digest of files in the cluster. this may not be combined with -t, as no digest is calculated.
• %i — the one-based index of the file cluster.
• %s — the size, in bytes, of a file in the cluster.
• %% — a % character.

the default format string when using -t is:

%n files in cluster %i (%s bytes)

the default format string for other modes is:

%n files in cluster %i (%s bytes, digest %d)

• -h — display help information and exit. 🫶
• -l limit — the minimum size of files to be sampled. if the size of files in a cluster is equal or greater than the specified limit, duff will sample and compare a few bytes from the start of each file before calculating a full digest. this is strictly an optimization and does not affect which files are considered by duff. the default limit is zero bytes, i.e. to use sampling on all files. ✨
• -q — quiet mode. suppress warnings and error messages. shhh. 🤫
• -p — physical mode. make duff consider physical files instead of hard links. if specified, multiple hard links to the same physical file will not be reported as duplicates. 💅
• -r — recursively search into all specified directories. go deep bestie. 🦋
• -t — thorough mode. distrust digests as a guarantee for equality. in thorough mode, duff compares files byte by byte when their sizes match. no cap, it checks. 🔥
• -v — display version information and exit. ⭐
• -z — do not consider empty files to be equal. this option prevents empty files from being reported as duplicates. 🍓

---

🌈 EXAMPLES 🌈

the command:

duff -r foo/

lists all duplicate files in the directory foo and its subdirectories. iconic.

the command:

duff -e0 * | xargs -0 rm

removes all duplicate files in the current directory.

😭 use with care!! note that you have no control over which files in each cluster are selected by -e (excess mode). this is main character energy with consequences.

the command:

find . -name '*.h' -type f | duff

lists all duplicate header files in the current directory and its subdirectories. 💜

the command:

find . -name '*.h' -type f -print0 | duff -0 | xargs -0 -n1 echo

lists all duplicate header files in the current directory and its subdirectories, correctly handling file names containing whitespace. note the use of xargs and echo to remove the null separators again before listing. the way that pipeline just works. 🧚

---

💀 DIAGNOSTICS 💀

duff exits 0 on success, and >0 if an error occurs.

---

🎀 SEE ALSO 🎀

find(1), xargs(1)

---

👑 AUTHORS 👑

Camilla Berglund <elmindreda@elmindreda.org>

---

🙈 BUGS 🙈

• duff doesn't check whether the same file has been specified twice on the command line. this will lead it to report files listed multiple times as duplicates when not using -p (physical mode). note that this problem only affects files, not directories. 🫠
• duff no longer (as of version 0.4) reports symbolic links to files as duplicates, as they're by definition always duplicates. this may break scripts relying on the previous behavior. 💅
• if the underlying files are modified while duff is running, all bets are off. this is not really a bug, but it can still bite you. 🦋

---

and that's the whole page bestie, slay. this local manual page came to you from Debian, dated January 18, 2012 — yes it's an elder but it's still eating, and duff(1) remains that girl. 💖✨🎀