CPDUP(1)

📦 version 1.18, in fedora 43, fedora 44

💖 CPDUP(1) — the manual, but make it make sense 💖

okay bestie, gather round, because cpdup from the General Commands Manual is genuinely one of the most useful little gremlins in your toolbox and i need you to know her 💅

✨ NAME ✨

cpdup — mirror filesystems 🪞

🌸 SYNOPSIS 🌸

cpdup [-C] [-v[v[v]]] [-d] [-n] [-u] [-I] [-f] [-F ssh-arg] [-s0] [-i0]
      [-j0] [-l] [-q] [-o] [-m] [-H path] [-M file] [-V] [-VV] [-S] [-R]
      [-k] [-K file] [-X file] [-x] [[user@]host:]source_dir
      [[user@]host:]target_dir

🦋 DESCRIPTION 🦋

so basically? cpdup makes an exact mirror copy of the source into the destination, and it will create and delete files and directories as necessary to get there. she is not playing around 🫶

UTimes, hardlinks, softlinks, devices, permissions, and flags are all mirrored. iconic behavior honestly.

By default, cpdup asks for confirmation if any file or directory needs to be removed from the destination, and it does not copy files which it believes to have already been synchronized (by observing that the source and destination files' sizes and mtimes match). efficient queen ✨

cpdup does not cross mount points in either the source or the destination. and as a safety measure — very much the "bestie checking your texts before you send them" energy — cpdup refuses to replace a destination directory with a file.

The following options are available:

  • -C — if the source or target is a remote host, request that the ssh(1) session be compressed. this is literally the same as -F -C.
  • -v[v[v]] — set verboseness. by default cpdup does not report its progress except when asking for confirmation. a single -v will only report modifications made to the destination. -vv will report directories as they are being traversed as well as modifications made to the destination. and -vvv will cause all files and directories to be reported whether or not modifications are made. yes she will yap. that's the point 💀
  • -d — print directories as they are being traversed. useful to watch the progress; this typically produces much less output than -vv.
  • -n — go through the motions but don't actually make any changes to the target. a dress rehearsal, no cap.
  • -u — causes the output generated by -v and -d to be unbuffered. this can be useful for obtaining prompt progress updates through a pipe. no more waiting around for the vibes to flush 🫶
  • -I — will cause cpdup to print a summary at the end with performance counters. numbers!! we love numbers.
  • -f — forces file updates to occur even if the files appear to be the same. if the -H option is used, this option will force a byte for byte comparison between the original file and the file in the hardlink path, even if all the stat info matches, but will still use a hardlink if they match.
  • -F ssh-arg — pass ssh-arg to ssh. for example “-F -p222”. note the lack of a space 👀
  • -s0 — disable the disallow-file-replaces-directory safety feature. this safety feature is enabled by default to prevent user mistakes from blowing away everything accidentally. and bestie, people do blow away everything accidentally, so maybe don't 🥲
  • -i0 — do not request confirmation when removing something. no take-backs, no "wait—", just gone.
  • -j0 — do not try to recreate CHR or BLK devices.
  • -l — line buffer verbose output.
  • -q — quiet operation. shhh 🤫
  • -o — do not remove any files, just overwrite/add. the gentle option 🎀
  • -m — generate and maintain a MD5 checkfile called .MD5.CHECKSUMS in each directory on the source and do an MD5 check on each file of the destination when the destination appears to be the same as the source. if the check fails, the source is recopied to the destination. when you specify a destination directory, the MD5 checkfile is only updated as needed and may not be updated even if modifications are made to a source file. if you do not specify a destination directory the cpdup command forcefully regenerates the MD5 checkfile for every file in the source.
  • -M file — works the same as -m but allows you to specify the name of the MD5 checkfile.
  • -H path — cpdup will create a hardlink from a file found under path to the target instead of copying the source to the target if the file found via path is identical to the source. note that a remote host specification should not be used for this option's path, but the path will be relative to the target machine.

this allows one to use cpdup to create incremental backups of a filesystem. create a direct ‘level 0’ backup, and then specify the level 0 backup path with this option when creating an incremental backup to a different target directory. this method works so long as the filesystem does not hit a hardlink limit. if the system does hit a hardlink limit, cpdup will generate a warning and copy the file instead. note that cpdup must record file paths for any hardlinked file while operating and therefore uses a great deal more memory when dealing with hardlinks or hardlink-based backups. example use:

cpdup -i0 -s0 -I -H /backup/home.l0 /home /backup/home.l1
WARNING: if this option is used cpdup must record the paths for all files it encounters while it operates and it is possible that you may run the process out of memory. 😭

the file found via the hardlink path will be byte-by-byte compared with the source if the -V or -f option is also used, otherwise only the stat info is checked to determine whether it matches the source.

  • -V — this forces the contents of regular files to be verified, even if the files appear to the be the same. whereas the -f (force) option forces a copy regardless, this option will avoid rewriting the target if everything matches and the contents are verified to be the same. checking before acting? that's growth 💅
  • -VV — this works the same as -V but ignores mtime entirely, making it suitable for comparing HAMMER master and slave filesystems or copies made without mtime retention.
  • -S — this places cpdup into slave mode and is used to initiate the slave protocol on a remote machine. this option is not intended to be used by humans. hands off, bestie, that's not for us 🙈
  • -R — place the slave into read-only mode. can only be used when the source is remote. useful for unattended backups via SSH keys.
  • -k — generate and maintain a FSMID checkfile called .FSMID.CHECK in each directory on the target. cpdup will check the FSMID for each source file or directory against the checkfile on the target and will not copy the file or recurse through the directory when a match occurs. any source file or directory with the same name as the checkfile will be ignored. the FSMID will be re-checked after the copy has been completed and cpdup will loop on that directory or file until it is sure it has an exact copy. she is persistent ✨
Warning: FSMID is not always supported by a filesystem and may not be synchronized if a crash occurs. DragonFly will simulate an FSMID when it is otherwise not supported by the filesystem, and users should be aware that simulated FSMIDs may change state in such cases even if the underlying hierarchy does not due to cache flushes. additionally, the FSMID may not reflect changes made to remote filesystems by other hosts. for example, using these options with NFS mounted sources will not work well. so like… know that going in 🫠
  • -K file — works the same as -k but allows you to specify the name of the FSMID checkfile.
  • -x — causes cpdup to use the exclusion file .cpignore in each directory on the source to determine which files to ignore. when this option is used, the exclusion filename itself is automatically excluded from the copy. if this option is not used then the filename .cpignore is not considered special and will be copied along with everything else. iykyk 📝
  • -X file — works the same as -x but allows you to specify the name of the exclusion file. this file is automatically excluded from the copy. only one exclusion file may be specified.

🌈 REMOTE COPYING 🌈

cpdup can mirror directory structures across machines and can also do third-party copies. this also works between machines that use different byte order. ssh(1) sessions are used and cpdup is run on the remote machine(s) in slave mode. you can use the -F option to pass additional flags to the ssh command if necessary.

The syntax of remote path specifications is similar to scp(1). in particular, that means that a local path containing a colon must be preceded by a slash to prevent it being considered a remote host: ‘foo:bar’ causes cpdup to look for a directory called ‘bar’ on host ‘foo’, while ‘./foo:bar’ denotes the directory ‘foo:bar’ on the local machine. the colon is lowkey a trap if you're not paying attention 🎀

⭐ DIAGNOSTICS ⭐

The cpdup utility exits 0 if no error occurred and >0 if an error occurred.

💜 SEE ALSO 💜

cp(1), cpio(1), scp(1), ssh(1), tar(1)

👑 HISTORY 👑

The cpdup command was originally created to update servers at BEST Internet circa 1997 and was placed under the FreeBSD copyright for inclusion in the ports area in 1999. The program was written by Matthew Dillon, Dima Ruban, and later significantly improved by Oliver Fromme. legends only, honestly 🫶

💀 BUGS 💀

UFS(5) has a hardlink limit of 32767. many programs, in particular CVS with regards to its CVS/Root file, will generate a lot of hard links. when using the -H option it may not be possible for cpdup to maintain these hard links. if this occurs, cpdup will be forced to copy the file instead of link it, and thus not be able to make a perfect copy of the filesystem. not her fault, the filesystem said no 🥲

When so-called sparse files (i.e. files with "holes") are copied, the holes will be filled in the target files, so they occupy more physical disk space than the source files. your disk said "i'm full" and the holes said "bet" 🙃

For compatibility reasons, the slave protocol is not as efficient for writing remote files as it is for reading them. Therefore it is recommended to run cpdup on the target machine when making remote copies, so the source machine is remote. If you do it the other way, cpdup will run somewhat slower. do it the right way bestie, your future self will thank you 💅

and that's the whole page, bestie — this one ships with the Debian edition of the General Commands Manual, dated November 24, 2009 🎀 go forth and mirror responsibly, mwah 💖✨

📦 where this page comes from