pdsh(1)

💖 NAME 💖

okay bestie, welcome to the pdsh(1) manual page — yes it's filed under the General Commands Manual but it's really all about these two icons:

• pdcp — copy files to groups of hosts in parallel 🫶
• rpdcp — (reverse pdcp) copy files from a group of hosts in parallel

---

📜 SYNOPSIS 📜

pdcp [options]... src [src2...] dest
rpdcp [options]... src [src2...] dir

---

✨ DESCRIPTION ✨

So pdcp is literally a variant of the rcp(1) command. But here's the glow-up: rcp(1) copies files to one single remote host, and pdcp copies files to multiple remote hosts in parallel. main character energy fr fr.

⚠️ but no cap: pdcp does not recognize files in the format ``rname@rhost:path,'' so all source files must be on the local host machine. no remote sources, periodt.

Destination nodes must be listed on the pdcp command line using a suitable target nodelist option (peep the OPTIONS section below 👇). And every single destination node you list must have pdcp installed, otherwise the copy just won't succeed. 💀

okay now the drama part. When pdcp receives SIGINT (that's ctrl-C bestie), it lists the status of current threads. A second SIGINT within one second terminates the program. Pending threads may be canceled by issuing ctrl-Z within one second of ctrl-C. And "pending threads" means the ones that haven't been initiated yet, or that are still in the process of connecting to the remote host. iykyk. 😭

Just like pdsh(1), the functionality of pdcp may be supplemented by dynamically loadable modules. In pdcp, those modules can provide:

• a new connect protocol (replacing the standard rsh(1) protocol)
• filtering options (e.g. excluding hosts that are down)
• and/or host selection options (e.g. -a selects all nodes from a local config file)

By default, pdcp requires at least one "rcmd" module to be loaded (to provide the channel for remote copy). no module, no copy, simple as that 💅

---

🔄 REVERSE PDCP 🔄

rpdcp performs a reverse parallel copy. Instead of copying files to remote hosts, files are retrieved from remote hosts and stored locally. obsessed.

Every directory or file that gets retrieved will be stored with its remote hostname appended to the filename. And the destination file must be a directory when this option is used.

In all other respects, rpdcp is exactly like pdcp, so literally everything else in this manual about pdcp also applies to rpdcp. 🫶

---

🧚 RCMD MODULES 🧚

The method by which pdcp connects to remote hosts may be selected at runtime using the -R option (see OPTIONS below). This is ultimately implemented via dynamically loadable modules, so the list of available options may be different from installation to installation. 💀

A list of currently available rcmd modules is printed when using any of the -h, -V, or -L options. The default rcmd module will also be displayed with the -h and -V options.

Here's the list of rcmd modules currently distributed with pdcp:

• rsh — uses an internal, thread-safe implementation of BSD rcmd(3) to run commands using the standard rsh(1) protocol.
• ssh — uses a variant of popen(3) to run multiple copies of the ssh(1) command. ✨
• mrsh — this module uses the mrsh(1) protocol to execute jobs on remote hosts. The mrsh protocol uses a credential based authentication, forgoing the need to allocate reserved ports. In other aspects, it acts just like rsh.
• krb4 — the krb4 module allows users to execute remote commands after authenticating with kerberos. Of course, the remote rshd daemons must be kerberized. 👑
• xcpu — the xcpu module uses the xcpu service to execute remote commands.

---

🌸 OPTIONS 🌸

The list of available pdcp options is determined at runtime by supplementing the list of standard pdcp options with any options provided by loaded rcmd and misc modules. In some cases, options provided by modules may conflict with each other. In those cases, the modules are incompatible and the first module loaded wins.

🎀 Standard target nodelist options 🎀

• -w TARGETS,... — target and or filter the specified list of hosts. Do not use with any other node selection options (e.g. -a, -g, if they are available). No spaces are allowed in the comma-separated list. Arguments in the TARGETS list may include normal host names, a range of hosts in hostlist format (see HOSTLIST EXPRESSIONS), or a single - character to read the list of hosts on stdin.

If a host or hostlist is preceded by a - character, this causes those hosts to be explicitly excluded. If the argument is preceded by a single ^ character, it is taken to be the path to a file containing a list of hosts, one per line. If the item begins with a / character, it is taken as a regular expression on which to filter the list of hosts (a regex argument may also be optionally trailed by another /, e.g. /node.*/). A regex or file name argument may also be preceeded by a minus - to exclude instead of include thoses hosts.

A list of hosts may also be preceded by "user@" to specify a remote username other than the default, or "rcmd_type:" to specify an alternate rcmd connection type for these hosts. When used together, the rcmd type must be specified first, e.g. "ssh:user1@host0" would use ssh to connect to host0 as user "user1." 💅

• -x host,host,... — exclude the specified hosts. May be specified in conjunction with other target node list options such as -a and -g (when available). Hostlists may also be specified to the -x option (see the HOSTLIST EXPRESSIONS section below). Arguments to -x may also be preceeded by the filename (^) and regex (/) characters as described above, in which case the resulting hosts are excluded as if they had been given to -w and preceeded with the minus - character.

🦋 Standard pdcp options 🦋

• -h — output usage menu and quit. A list of available rcmd modules will be printed at the end of the usage message.
• -q — list option values and the target nodelist and exit without action. (no notes, just vibes 🫶)
• -b — disable the ctrl-C status feature so that a single ctrl-C kills the parallel copy. (Batch Mode)
• -r — copy directories recursively.
• -p — preserve modification time and modes.
• -e PATH — explicitly specify path to the remote pdcp binary instead of using the locally executed path. Can also be set via the environment variable PDSH_REMOTE_PDCP_PATH.
• -l user — this option may be used to copy files as another user, subject to authorization. For BSD rcmd, this means the invoking user and system must be listed in the user´s .rhosts file (even for root). 👀
• -t seconds — set the connect timeout. Default is 10 seconds.
• -f number — set the maximum number of simultaneous remote copies to number. The default is 32.
• -R name — set rcmd module to name. This option may also be set via the PDSH_RCMD_TYPE environment variable. A list of available rcmd modules may be obtained via either the -h or -L options.
• -M name,... — when multiple misc modules provide the same options to pdsh, the first module initialized "wins" and subsequent modules are not loaded. The -M option allows a list of modules to be specified that will be force-initialized before all others, in-effect ensuring that they load without conflict (unless they conflict with each other). This option may also be set via the PDSH_MISC_MODULES environment variable.
• -L — list info on all loaded pdcp modules and quit.
• -d — include more complete thread status when SIGINT is received, and display connect and command time statistics on stderr when done. Also, display any module loading errors on stderr, including option conflicts.
• -V — output pdcp version information, along with a list of currently loaded modules, and exit.

---

🌈 HOSTLIST EXPRESSIONS 🌈

As noted in the sections above, pdcp accepts ranges of hostnames in the general form:

prefix[n-m,l-k,...]

where n < m and l < k, etc., as an alternative to explicit lists of hosts. This form should not be confused with regular expression character classes (also denoted by ``[]''). For example, foo[19] does not represent foo1 or foo9, but rather represents a degenerate range: foo19. 😭

This range syntax is meant only as a convenience on clusters with a prefixNN naming convention, and specification of ranges should not be considered necessary — the list foo1,foo9 could be specified as such, or by the range foo[1,9]. either way bestie, we slay.

Some examples of range usage follow:

Copy /etc/hosts to foo01,foo02,...,foo05

pdcp -w foo[01-05] /etc/hosts /etc

Copy /etc/hosts to foo7,foo9,foo10

pdcp -w foo[7,9-10] /etc/hosts /etc

Copy /etc/hosts to foo0,foo4,foo5

pdcp -w foo[0-5] -x foo[1-3] /etc/hosts /etc

As a reminder to the reader (that's you 🫵), some shells will interpret brackets ([ and ]) for pattern matching. Depending on your shell, it may be necessary to enclose ranged lists within quotes. For example, in tcsh, the first example above should be executed as:

pdcp -w "foo[01-05]" /etc/hosts /etc

---

👑 ORIGIN 👑

Pdsh/pdcp was originally a rewrite of IBM dsh(1) by Jim Garlick <garlick@llnl.gov> on LLNL's ASCI Blue-Pacific IBM SP system. It is now also used on Linux clusters at LLNL. iconic origin story honestly 💅

---

💀 LIMITATIONS 💀

When using ssh for remote execution, stderr of ssh to be folded in with that of the remote command. When invoked by pdcp, it is not possible for ssh to prompt for confirmation if a host key changes, prompt for passwords if RSA keys are not configured properly, etc. 🙈 Finally, the connect timeout is only adjustable with ssh when the underlying ssh implementation supports it, and pdsh has been built to use the correct option.

---

🔥 SEE ALSO 🔥

• pdsh(1)

---

okay bestie that's the whole page 💖 this one came out of the linux-gnu build, and it lives in the pdsh(1) manual — so like, go forth and copy files to a whole group of hosts at once. slay, ate, no crumbs. 🎀✨🦋