DDUP(1)

📦 version 0.12, in nixpkgs unstable

🦋 DDUP(1) 🦋

okay so real talk, this is the ddup page from mpiFileUtils and it's genuinely one of the most useful little tools ever, bestie. let's get into it ✨

🌸 NAME 🌸

ddup — report files with identical content. that's it, that's the whole job, and she does it beautifully. 💅

🎀 SYNOPSIS 🎀

ddup [OPTION] PATH

✨ DESCRIPTION ✨

okay so picture this: it's a parallel MPI application and its entire main character energy is reporting files under a directory tree that have identical content. like, she walks the whole tree and finds the twins. 🔥

so ddup reports path names to files having identical content — a.k.a. duplicate files, a.k.a. the reason your disk is full, no cap. you give her a top-level directory (that's the PATH), and she reports the path name to any file that is a duplicate of another file anywhere under that same directory tree. anywhere. she does not care about subfolders, she will find it. 🕵️‍♀️

and here's the iconic part: the path to each file gets reported, along with a final hash representing its content. so you can match up multiple sets of duplicate files using that final reported hash. iykyk — the hash is the glue. 🫶

💖 OPTIONS 💖

  • --open-noatime — opens files with the O_NOATIME flag, if possible. key phrase: if possible. don't come for her if it's not. 🙈
  • -d, --debug LEVEL — sets the verbosity level. LEVEL can be one of: fatal, err, warn, info, dbg. pick your vibe.
  • -v, --verbose — run in verbose mode. she talks more. we love a chatty queen. 💬
  • -q, --quiet — run the tool silently. no output is printed. like, nothing. silence. 🤫
  • -h, --help — prints the command usage, and the list of options available. the classic "help me help you" moment. 🫶

🌈 EXAMPLES 🌈

  1. to report any duplicate files under a directory tree:
mpirun -np 128 ddup /path/to/haystack

look at those 128 processes go. finding the needles, finding the duplicates, slaying. ⭐

🧚 SEE ALSO 🧚

the mpiFileUtils source code and all documentation may be downloaded from <https://github.com/hpc/mpifileutils>

👑 AUTHOR 👑

HPC

2022, LLNL/LANL/UT-Battelle/DDN

and that's the whole page bestie! 🍓 this was version 0.12, dated Jan 29, 2025 — so go forth, run ddup, find those duplicate files, and free up your disk. you're welcome. 💖✨🎀

📦 where this page comes from