rdfind(1)

📦
version 1.8.0 · 4 versions
, in alpine 3.24, arch rolling, fedora 43, fedora 44, nixpkgs unstable

🌸 rdfind — the manual page, but make it make sense 🌸

okay bestie, welcome to rdfind(1) 💖 this is the page for rdfind, a lil program that goes and finds duplicate files for you. she's simple, she's powerful, she will absolutely eat your disk space back. let's get into it 🫶

✨ NAME ✨

rdfind — finds duplicate files 💅

🦋 SYNOPSIS 🦋

rdfind [ options ] directory1 | file1 [ directory2 | file2 ] ...

💖 DESCRIPTION 💖

rdfind finds duplicate files across and/or within several directories. she's smart about it too — she only calculates checksums when it's actually necessary (no wasted work, we love an efficient queen 👑). and directories get searched recursively, so she goes all the way down, no cap.

if two (or more) equal files show up, the program decides which one is the original and the rest get called duplicates. how? by ranking the files against each other and picking the one with the highest rank. full deets are in the RANKING section below, iykyk 🌈

by default, she takes no action other than creating a file with the detected files and showing you the possible amount of saved space. so like, she tells you the tea but doesn't spill it ☕

if you need better control over the ranking than what's given, you can use some preprocessor which sorts the file names in the order you want, and then run the program using xargs. there are examples below for how to use find and xargs together with rdfind — iconic combo fr fr.

to include files or directories that have names starting with -, use rdfind ./- so you don't confuse them with options 💀 (yes this trips everyone up, you're not alone)

👑 RANKING 👑

so given two or more equal files, the one with the highest rank gets selected as the original and the rest are duplicates. the rules run from start until an original has been found — like a little tournament 🏆

given two files A and B which have equal size and content, the ranking goes like this:

  • if A was found while scanning an input argument earlier than B, A is higher ranked.
  • if A was found at a directory depth lower than B, A is higher ranked (A is closer to the root).
  • if A and B are found during scanning of the same input argument and share the same directory depth, then who ranks highest depends on whether deterministic operation is enabled. this is on by default, see the -deterministic option.
    • if enabled, which one ranks highest is unspecified but deterministic.
    • if disabled, the one that was reported first from the file system is highest ranked.

🎀 OPTIONS 🎀

searching options etc: 🔍

  • -ignoreempty true|false — ignore empty files. setting this to true (the default) is equivalent to -minsize 1, and false is equivalent to -minsize 0.
  • -minsize N — ignores files with less than N bytes. default is 1, meaning empty files are ignored.
  • -maxsize N — ignores files with N bytes or more. default is 0, which means this check is disabled.
  • -followsymlinks true|false — follow symlinks. default is false.
  • -removeidentinode true|false — removes items found which have identical inode and device ID. default is true.
  • -checksum none|md5|sha1|sha256|sha512|xxh128 — what type of checksum to be used: md5, sha1, sha256, sha512 or xxh128. the default is sha1 since version 1.4.0. xxh128 is a very fast checksum, but not of cryptographic quality. xxh support is optional and requires that rdfind was configured with --with-xxhash. in case xxh is used but there is no support, an error is returned. checksum none can be used to skip checksumming altogether.
not recommended!! 🚨 in case files of the same size have contents that differ, it is likely they are falsely considered duplicates, leading to file removal (depending on other options). like, actually read that twice bestie 😭
  • -buffersize N — chunksize in bytes when calculating the checksum for files. smaller or bigger can improve performance dependent on filesystem and checksum algorithm. the default is 1 MiB, the maximum allowed is 128MiB (inclusive).
  • -firstbytessize N — size in bytes when scanning the first bytes of each file, prior to full checksumming. setting this to 0 means skipping the step entirely.
  • -lastbytessize N — size in bytes when scanning the last bytes of each file, prior to full checksumming. setting this to 0 means skipping the step entirely.
  • -deterministic true|false — if set (the default), sort files of equal rank in an unspecified but deterministic order. this makes the behaviour independent of in which order files are listed when querying the file system. so stable, but mysterious. main character energy honestly 💅

action options: 💥

  • -makesymlinks true|false — replace duplicate files with symbolic links. default is false.
  • -makehardlinks true|false — replace duplicate files with hard links. default is false.
  • -makeresultsfile true|false — make a results file in the current directory. default is true. if the file exists, it is overwritten 🫣 this does not affect whether items are deleted. see -dryrun for how to disable deletions.
  • -outputname name — make the results file name to be name instead of the default results.txt.
  • -deleteduplicates true|false — delete (unlink) files. default is false.

general options: 🌈

  • -progress true|false — show progress during elimination. defaults to false.
  • -sleep Xms — sleeps X milliseconds between reading each file, to reduce load. default is 0 (no sleep). note that only a few values are supported at present: 0,1-5,10,25,50,100 milliseconds.
  • -n, -dryrun true|(false) — by default, rdfind does nothing except creating a results file. in case one of the action flags like -deleteduplicates is set, dryrun can be set to let rdfind print what it would do, instead of actually doing it. the default is false.
  • -h, -help, --help — displays a brief help message.
  • -v, -version, --version — displays the version number.

🌟 EXAMPLES 🌟

search for duplicate files in the home directory and a backup directory:

rdfind ~ /mnt/backup

delete duplicates in a backup directory:

rdfind -deleteduplicates true /mnt/backup

search for duplicate files in directories called foo:

find . -type d -name foo -print0 |xargs -0 rdfind

🍓 FILES 🍓

results.txt (the default name is results.txt and can be changed with option outputname, see above).

the results file results.txt will contain one row per duplicate file found, along with a header row explaining the columns. a text describes why the file is considered a duplicate:

  • DUPTYPE_UNKNOWN — some internal error 💀
  • DUPTYPE_FIRST_OCCURRENCE — the file that is considered to be the original. 👑
  • DUPTYPE_WITHIN_SAME_TREE — files in the same tree (found when processing the directory in the same input argument as the original)
  • DUPTYPE_OUTSIDE_TREE — the file is found during processing another input argument than the original.

🌸 ENVIRONMENT 🌸

(this section is empty in the original — nothing to report, bestie 🤷♀️)

💜 DIAGNOSTICS 💜

(also empty in the original. silence is golden ✨)

⭐ EXIT VALUES ⭐

  • 0 on success, nonzero otherwise.

🙈 BUGS/FEATURES 🙈

  • when specifying the same directory twice, it keeps the first encountered as the most important (original), and the rest as duplicates. this might not be what you want 👀
  • the symlink creates absolute links. this might not be what you want. to create relative links instead, you may use the symlinks(8) command, which is able to convert absolute links to relative links. (noted, bookmarked 📌)
  • older versions unfortunately contained a misspelling on the word occurrence. this is now corrected (since 1.3), which might affect user scripts parsing the output file written by rdfind. so if your script broke, it's giving "it's not a bug it's a spelling fix" 😭

🚨 SECURITY CONSIDERATIONS 🚨

avoid manipulating the directories while rdfind is reading. rdfind is quite brittle in that case. especially, when deleting or making links, rdfind can be subject to a symlink attack. use with care!! 🫶

rfind was written with the intention of deduplicating directories owned by the same user. it does not handle file ownership and permissions. it is not recommended to run it across directories with several owners.

💖 AUTHOR 💖

Paul Dreik 2006-2026, reachable at rdfind@pauldreik.se

rdfind can be found at https://rdfind.pauldreik.se/

do you find rdfind useful? drop me a line! it is always fun to hear from people who actually use it and what data collections they run it on 🥹 (this is so sweet actually)

🌈 THANKS 🌈

several persons have helped with suggestions and improvements: Niels Möller, Carl Payne and Salvatore Ansani. thanks also to you who tested the program and sent me feedback 🫶

🎀 VERSION 🎀

1.8.0 (release date 2026-02-22)

this program is distributed under GPLv2 or later, at your option.

🔥 SEE ALSO 🔥

md5sum, sha1sum, find, symlinks

and that's the whole page, bestie — this was rdfind(1) from the rdfind manual, version 1.8.0, dated Feb 2026. go forth and deduplicate responsibly 💅✨💖

📦 where this page comes from