PARALLEL(1)

📦

this version · 15 versions

arch rolling, epel 10, epel 9, fedora 43, fedora 44 (latest)
opensuse-leap 16.0
nixos 26.05
alpine 3.22
alpine 3.23
ubuntu stonking
ubuntu jammy
fedora 43
fedora 43, fedora 44, nixpkgs unstable
ubuntu noble
alpine 3.24, fedora 44
debian bookworm
ubuntu resolute
debian trixie, ubuntu questing
0.70 · debian bookworm, debian trixie, epel 10, epel 9, fedora 43, fedora 44, nixos 26.05, nixpkgs unstable, opensuse-leap 16.0, ubuntu jammy, ubuntu noble, ubuntu questing, ubuntu resolute, ubuntu stonking

, in arch rolling, epel 10, epel 9, fedora 43, fedora 44

🎀 NAME — `PARALLEL(1)` / `parallel` 🎀

parallel — build and execute shell command lines from standard input in parallel. That’s the whole iconic one-liner, bestie: GNU parallel, manual page PARALLEL(1), here to make your shell go brrr. 💅

✨ SYNOPSIS ✨

parallel [options] [command [arguments]] < list_of_arguments

parallel [options] [command [arguments]] ( ::: arguments | :::+
arguments | :::: argfile(s) | ::::+ argfile(s) ) ...

parallel --semaphore [options] command

#!/usr/bin/parallel --shebang [options] [command [arguments]]

#!/usr/bin/parallel --shebang-wrap [options] [command [arguments]]

🦋 DESCRIPTION 🦋

STOP! ✋

Read the Reader’s guide below if you are new to GNU parallel. This is your friendly little “do not just start smashing commands into the terminal” moment. 🙈

GNU parallel is a shell tool for executing jobs in parallel using one or more computers. A job can be a single command or a small script that has to be run for each of the lines in the input. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables. A job can also be a command that reads from a pipe. GNU parallel can then split the input into blocks and pipe a block into each command in parallel. 🫶

If you use xargs and tee today you will find GNU parallel very easy to use as GNU parallel is written to have the same options as xargs. If you write loops in shell, you will find GNU parallel may be able to replace most of the loops and make them run faster by running several jobs in parallel. Lowkey a glow-up for your shell loops. ✨

GNU parallel makes sure output from the commands is the same output as you would get had you run the commands sequentially. This makes it possible to use output from GNU parallel as input for other programs. So yes, it can be pretty and pipeline-friendly. 💖

For each line of input GNU parallel will execute command with the line as arguments. If no command is given, the line of input is executed. Several lines will be run in parallel. GNU parallel can often be used as a substitute for xargs or cat | bash. 👑

📚 Reader’s guide 📚

GNU parallel includes the 4 types of documentation: Tutorial, how-to, reference, and explanation/design. Pick your fighter, bestie. 🌈

🎥 Tutorial 🎥

If you prefer reading a book buy GNU Parallel 2018 at https://www.lulu.com/shop/ole-tange/gnu-parallel-2018/paperback/product-23558902.html or download it at: https://doi.org/10.5281/zenodo.1146014. Read at least chapter 1+2. It should take you less than 20 minutes. ⭐

Otherwise start by watching the intro videos for a quick introduction: https://youtube.com/playlist?list=PL284C9FF2488BC6D1. 🍓

If you want to dive deeper: spend a couple of hours walking through the tutorial (man parallel_tutorial). Your command line will love you for it. 💜

🛠 How-to 🛠

You can find a lot of examples of use in man parallel_examples. They will give you an idea of what GNU parallel is capable of, and you may find a solution you can simply adapt to your situation. 🎀

If the example do not cover your exact needs, the options map (https://www.gnu.org/software/parallel/parallel_options_map.pdf) can help you identify options that are related, so you can look these up in the man page. 🦋

📖 Reference 📖

If you need a one page printable cheat sheet you can find it on: https://www.gnu.org/software/parallel/parallel_cheat.pdf. 🧚

The man page is the reference for all options, and reading the man page from cover to cover is probably not what you need. Ngl, respect for knowing your limits. 💀

🧠 Design discussion 🧠

If you want to know the design decisions behind GNU parallel, try: man parallel_design. This is also a good intro if you intend to change GNU parallel. 👑

💖 OPTIONS 💖

Okay bestie, this is the big one. Every option and replacement string gets its moment, and every tiny technical detail stays exactly where it belongs. No cap. 🔥

🛠 `command` — the thing you’re running

Command to execute. That’s the whole assignment. 💅

If command or the following arguments contain replacement strings (such as {}) every instance will be substituted with the input.
If command is given, GNU parallel solve the same tasks as xargs. If command is not given GNU parallel will behave similar to cat | sh.
The command must be an executable, a script, a composed command, an alias, or a function.

A few very important little “how do I make this work” notes:

Bash functions: export -f the function first or use env_parallel.
Bash, Csh, or Tcsh aliases: Use env_parallel.
Zsh, Fish, Ksh, and Pdksh functions and aliases: Use env_parallel.

🎁 `{}` — input line

This replacement string will be replaced by a full line read from the input source. The input source is normally stdin (standard input), but can also be given with --arg-file, :::, or ::::. ✨

The replacement string {} can be changed with -I.

If the command line contains no replacement strings then {} will be appended to the command line.

Replacement strings are normally quoted, so special characters are not parsed by the shell. The exception is if the command starts with a replacement string; then the string is not quoted. Important little exception, don’t forget it. ⭐

See also: --plus {.} {/} {//} {/.} {#} {%} {n} {=perl expression=}

🧁 `{.}` — input line without extension

This replacement string will be replaced by the input with the extension removed. If the input line contains . after the last /, the last . until the end of the string will be removed and {.} will be replaced with the remaining. E.g. foo.jpg becomes foo, subdir/foo.jpg becomes subdir/foo, sub.dir/foo.jpg becomes sub.dir/foo, sub.dir/bar remains sub.dir/bar. If the input line does not contain . it will remain unchanged. 🍓

The replacement string {.} can be changed with --extensionreplace.

🌸 `{/}` — basename of input line

This replacement string will be replaced by the input with the directory part removed. Clean, simple, basename energy. 💅

🦋 `{//}` — dirname of input line

This replacement string will be replaced by the dir of the input line. See dirname(1).

🎀 `{/.}` — basename of input line without extension

This replacement string will be replaced by the input with the directory and extension part removed. {/.} is a combination of {/} and {.}. So it’s the “basename but make it shorter” icon. ✨

See also: {} --basenameextensionreplace

🔢 `{#}` — sequence number of the job to run

This replacement string will be replaced by the sequence number of the job being run. It contains the same number as $PARALLEL_SEQ. 🧮

🎰 `{%}` — job slot number

This replacement string will be replaced by the job’s slot number between 1 and number of jobs to run in parallel. There will never be 2 jobs running at the same time with the same job slot number. Slot drama is not allowed. 💅

If the job needs to be retried (e.g using --retries or --retry-failed) the job slot is not automatically updated. You should then instead use $PARALLEL_JOBSLOT:

$ do_test() {
    id="$3 {%}=$1 PARALLEL_JOBSLOT=$2"
    echo run "$id";
    sleep 1
    # fail if {%} is odd
    return `echo $1%2 | bc`
  }
$ export -f do_test
$ parallel -j3 --jl mylog do_test {%} \$PARALLEL_JOBSLOT {} ::: A B C D
run A {%}=1 PARALLEL_JOBSLOT=1
run B {%}=2 PARALLEL_JOBSLOT=2
run C {%}=3 PARALLEL_JOBSLOT=3
run D {%}=1 PARALLEL_JOBSLOT=1
$ parallel --retry-failed -j3 --jl mylog do_test {%} \$PARALLEL_JOBSLOT {} ::: A B C D
run A {%}=1 PARALLEL_JOBSLOT=1
run C {%}=3 PARALLEL_JOBSLOT=2
run D {%}=1 PARALLEL_JOBSLOT=3

Notice how {%} and $PARALLEL_JOBSLOT differ in the retry run of C and D. This is the detail that eats. 🍴

🧸 `{n}` — argument from input source n or the n’th argument

This positional replacement string will be replaced by the input from input source n (when used with --arg-file or ::::) or with the n'th argument (when used with -N or --colsep). 🎯

If n is negative it refers to the n'th last argument.

See also: {} {n.} {n/} {n//} {n/.} --colsep

🍬 `{n.}` — argument from input source n or the n’th argument without extension

{n.} is a combination of {n} and {.}.

This positional replacement string will be replaced by the input from input source n (when used with --arg-file or ::::) or with the n'th argument (when used with -N). The input will have the extension removed. ✨

🧚 `{n/}` — basename of argument from input source n or the n’th argument

{n/} is a combination of {n} and {/}.

🌈 `{n//}` — dirname of argument from input source n or the n’th argument

{n//} is a combination of {n} and {//}.

This positional replacement string will be replaced by the dir of the input from input source n (when used with --arg-file or ::::) or with the n'th argument (when used with -N). See dirname(1).

🫶 `{n/.}` — basename of argument from input source n or the n’th argument without extension

{n/.} is a combination of {n}, {/}, and {.}.

🧪 `{=perl expression=}` — replace with calculated perl expression

Replace with calculated perl expression. Yes, we’re doing Perl math in the middle of your parallel command. Main character behavior. 💅

$_ will contain the same as {}. After evaluating perl expression $_ will be used as the value. It is recommended to only change $_ but you have full access to all of GNU parallel’s internal functions and data structures. That’s a lot of power, bestie. 🔥

The expression must give the same result if evaluated twice - otherwise the behaviour is undefined. E.g. in some versions of GNU parallel this will not work as expected:

parallel echo '{= $_= ++$wrong_counter =}' ::: a b c

A few convenience functions and data structures have been made:

🧵 `Q(string)` — shell quote a string

Shell quote a string. Example:

parallel echo {} is quoted as '{= $_=Q($_) =}' ::: \$PWD

🪄 `pQ(string)` — perl quote a string

Perl quote a string. Example:

parallel echo {} is quoted as '{= $_=pQ($_) =}' ::: \$PWD

🙅 `uq()` (or `uq`) — do not quote current replacement string

Do not quote current replacement string. Example:

parallel echo {} has the value '{= uq =}' ::: \$PWD

🔐 `hash(val)` — compute `B::hash(val)`

Compute B::hash(val). Example:

parallel echo Hash of {} is '{= $_=hash($_) =}' ::: a b c

🧮 `total_jobs()` — number of jobs in total

Number of jobs in total. Example:

parallel echo Number of jobs: '{= $_=total_jobs() =}' ::: a b c

🎰 `slot()` — slot number of job

Slot number of job. Example:

parallel echo Job slot of {} is '{= $_=slot() =}' ::: a b c

🔢 `seq()` — sequence number of job

Sequence number of job. Example:

parallel echo Seq number of {} is '{= $_=seq() =}' ::: a b c

`@arg`

The arguments, counting from 1 — so $arg[1] = {1} = the first argument bestie 💅

Example:

parallel echo {1}+{2}='{=1 $_=$arg[1]+$arg[2] =}' \
  ::: 1 2 3 ::: 2 3 4

(and like, the '{=1' is doing the Lord's work here — it forces this to be a positional replacement string, which means it will not repeat the value for each arg, thank u very much ✨)

`skip()`

Skips this job entirely, no cap (see also --filter) — it's giving "not me, not today" 🙈

Example:

parallel echo '{= $arg[1] >= $arg[2] and skip =}' \
  ::: 1 2 3 ::: 2 3 4

the time functions 🕰️

Okay so there's a whole lil squad of them:

yyyy_mm_dd_hh_mm_ss(sec)
yyyy_mm_dd_hh_mm(sec)
yyyy_mm_dd(sec)
hh_mm_ss(sec)
hh_mm(sec)
yyyymmddhhmmss(sec)
yyyymmddhhmm(sec)
yyyymmdd(sec)
hhmmss(sec)
hhmm(sec)

Time functions, iconic behavior 💖 sec is the number of seconds since epoch. If you leave it out, it'll just use the current local time, easy peasy.

Example:

parallel echo 'Now: {= $_=yyyy_mm_dd_hh_mm_ss() =}' ::: Dummy
parallel echo 'The end: {= $_=yyyy_mm_dd_hh_mm_ss($_) =}' \
  ::: 2147483648

Example (yes, another one, we're eating today 🍓):

seq 10 | parallel echo {} + 1 is {= '$_++' =}
parallel csh -c {= '$_="mkdir ".Q($_)' =} ::: '12" dir'
seq 50 | parallel echo job {#} of {= '$_=total_jobs()' =}

See also: --rpl --parens {} {=n perl expression=} --filter

`{=n perl expression=}`

The positional equivalent of {=perl expression=} — same vibe, but positionally locked in 🔒

To actually understand positional replacement strings, go peek at {n}.

`{rpl:format}`

Format replacement string, slay ✨ Here format is a format string like the one you'd use in printf, and rpl is a replacement string.

Examples:

{#:%04d} — Job number ({#}) with 4 digits prepended with 0
{%:%02d} — Job slot ({%}) with 2 digits prepended with 0
{:%6s} — Input line ({}) right aligned 6 chars wide
{/:%12s} — Basename ({/}) right aligned 12 chars wide
{2:%8.2f} — Second input source ({2}) 8 chars wide, 2 decimals

And like, format strings also work on replacement strings defined via --rpl that start with { 🫶

`::: arguments`

Use the arguments on the command line as the input source. Iconic.

Unlike other options, ::: is placed after the command and before the arguments — it's just built different 💅

The following are all equivalent, fr fr:

(echo file1; echo file2) | parallel gzip
parallel gzip ::: file1 file2
parallel gzip {} ::: file1 file2
parallel --arg-sep ,, gzip {} ,, file1 file2
parallel --arg-sep ,, gzip ,, file1 file2
parallel ::: "gzip file1" "gzip file2"

To avoid ::: being treated as special, use --arg-sep to set the argument separator to something else ✨

If you give multiple :::, each group is treated as its own input source, and all combinations of input sources get generated. E.g. ::: 1 2 ::: a b c gives you the combos (1,a) (1,b) (1,c) (2,a) (2,b) (2,c). Perfect for replacing nested for-loops, we love a shortcut 😌

And you can mix :::, ::::, and --arg-file — so these are all equivalent:

parallel echo {1} {2} {3} ::: 6 7 ::: 4 5 ::: 1 2 3
parallel echo {1} {2} {3} :::: <(seq 6 7) <(seq 4 5) \
  :::: <(seq 1 3)
parallel -a <(seq 6 7) echo {1} {2} {3} :::: <(seq 4 5) \
  :::: <(seq 1 3)
parallel -a <(seq 6 7) -a <(seq 4 5) echo {1} {2} {3} \
  ::: 1 2 3
seq 6 7 | parallel -a - -a <(seq 4 5) echo {1} {2} {3} \
  ::: 1 2 3
seq 4 5 | parallel echo {1} {2} {3} :::: <(seq 6 7) - \
  ::: 1 2 3

See also: --arg-sep --arg-file :::: :::+ ::::+ --link

`:::+ arguments`

Like ::: but linked like --link to the previous input source 🦋

And here's the twist: contrary to --link, values do not wrap — the shortest input source determines the length, periodt.

Example:

parallel echo ::: a b c :::+ 1 2 3 ::: X Y :::+ 11 22

`:::: argfiles`

Just another way to write --arg-file argfile1 --arg-file argfile2 ... — same thing, shorter to type ✨

And yup, ::: and :::: can be mixed 🫶

See also: --arg-file ::: ::::+ --link

`::::+ argfiles`

Like :::: but linked like --link to the previous input source 💜

Contrary to --link, values do not wrap: the shortest input source determines the length.

`--null`, `-0`

Use NUL as delimiter 🫶

Normally input lines end in \n (newline). If they end in \0 (NUL) instead, then use this option. It's useful for processing arguments that may contain \n (newline) — like, imagine a filename with a newline in it, chaotic but real 😭

Shorthand for --delimiter '\0'.

`--arg-file input-file`, `-a input-file`

Use input-file as input source.

If multiple --arg-file are given, each input-file is treated as an input source, and all combinations of input sources get generated. E.g. say the file foo contains 1 2 and the file bar contains a b c — then -a foo -a bar gives you the combos (1,a) (1,b) (1,c) (2,a) (2,b) (2,c). Nested for-loop replacement, we love to see it 😌

If input-file starts with +, the file is linked to the previous --arg-file. E.g. foo contains 1 2 and bar contains a b — then -a foo -a +bar gives you (1,a) (2,b) like --link, instead of generating every combination ✨

See also: --link {n} :::: ::::+ :::

`--arg-file-sep sep-str`

Use sep-str instead of :::: as the separator string between command and argument files.

Useful if :::: is already being used for something else by your command — sharing is caring but not with separators 💀

`--arg-sep sep-str`

Use sep-str instead of ::: as the separator string.

Useful if ::: is used for something else by the command 🫶

Also useful if your command uses ::: but you still want to read arguments from stdin (standard input): just change --arg-sep to a string that isn't anywhere in the command line. Big brain move, honestly 🧚

`--bar`

Show progress as a progress bar 🔥

In the bar you get: % of jobs completed, estimated seconds left, and number of jobs started. It's giving dashboard, it's giving main character energy 👑

It's compatible with zenity:

seq 1000 | parallel -j30 --bar '(echo {};sleep 0.1)' \
  2> >(perl -pe 'BEGIN{$/="\r";$|=1};s/\r/\n/g' |
       zenity --progress --auto-kill) | wc

See also: --eta --progress --total-jobs

`--basefile file`, `--bf file`

file gets transferred to each sshlogin before the first job is started 🌸

It will be removed if --cleanup is active. The file may be a script to run, or some common base data needed for the job. You can specify multiple --bf to transfer more basefiles — collect them all 💅 The file will be transferred the same way as --transferfile.

See also: --sshlogin --transfer --return --cleanup --workdir

`--basenamereplace replace-str`, `--bnr replace-str`

Use the replacement string replace-str instead of {/} for the basename of the input line ✨

`--basenameextensionreplace replace-str`, `--bner replace-str`

Use the replacement string replace-str instead of {/.} for the basename of the input line without the extension 🎀

`--bin binexpr`

Use binexpr as the binning key and bin input to the jobs 🗂️

binexpr is [column number|column name] [perlexpression], e.g.:

3
Address
3 $_%=100
Address s/\D//g

Each input line is split using --colsep. The value of the column goes into $_, the perl expression gets executed, and the resulting value is the job slot that'll be given the line. If that value is bigger than the number of jobslots, the value gets taken modulo the number of jobslots — math said "you're not getting in" 😭

This is similar to --shard, but the hashing algorithm is a simple modulo, which makes it predictable which jobslot receives which value. Iykyk.

Performance is in the order of 100K rows per second. Faster if the bincol is small (<10), slower if it's big (>100).

--bin requires --pipe and a fixed numeric value for --jobs. No vibes-based job counts here, bestie.

See also: SPREADING BLOCKS OF DATA, --group-by, --round-robin, --shard

`--bg`

Run command in background 🌙

GNU parallel will normally wait for the completion of a job. With --bg, GNU parallel will not wait for completion of the command before exiting — it's out, it's done, bye 💅

This is the default if --semaphore is set.

Implies --semaphore.

`--bibtex`, `--citation`

Print the citation notice and BibTeX entry for GNU parallel, silence the citation notice for all future runs, and exit. It will not run any commands. Periodt.

If it's impossible for you to run --citation, you can instead use --will-cite, which will run commands, but which will only silence the citation notice for this single run 🫶

If you use --will-cite in scripts to be run by others, you are making it harder for others to see the citation notice. The development of GNU parallel is indirectly financed through citations, so if your users do not know they should cite, then you are making it harder to finance development. However, if you pay 10000 EUR, you have done your part to finance future development and should feel free to use --will-cite in scripts.

If you do not want to help finance future development by letting other users see the citation notice, or by paying, then please consider using another tool instead of GNU parallel. You can find some of the alternatives in man parallel_alternatives. Not me telling you to go shop elsewhere, but like, respect the economics 😌

`--block size`, `--block-size size`

Size of block in bytes to read at a time 📦

The size can be postfixed with K, M, G, T, P, k, m, g, t, or p — all the size letters, she ate.

GNU parallel tries to meet the block size but can be off by the length of one record — close enough, bestie. For performance reasons, size should be bigger than two records. GNU parallel will warn you and automatically increase the size if you choose a size that's too small, it's got your back 🫶

size defaults to 1M.

A negative block size is not interpreted as a blocksize but as the number of blocks each jobslot should have. So --block -3 will make 3 jobs for each jobslot. In other words, this will run 3*5 = 15 jobs in total:

parallel --pipe-part -a myfile --block -3 -j5 wc
cat myfile | parallel --pipe --block -3 -j5 wc

--pipe-part --block is an efficient alternative to --round-robin, because data is never read by GNU parallel, but you can still have very few jobslots processing huge amounts of data. Efficiency, slay ✨

On the other hand, --pipe --block is quite inefficient: it reads the whole file into memory before splitting it. Thus the input must be able to fit in memory — no cap, that's the actual constraint 😭

If you use --block -size, input should be bigger than size+1 records.

✨ See also: UNIT PREFIX -N --pipe --pipe-part --round-robin --block-timeout

--block-timeout duration, --bt duration — sets the timeout for reading a block when you’re using --pipe. If reading a full block takes longer than duration, GNU parallel just uses the partial block it has read so far. duration is in seconds, but you can postfix it with s, m, h, or d. See also: TIME POSTFIXES --pipe --block.
--cat — makes a temporary file with the content. Normally --pipe/--pipe-part gives data to the program on stdin (standard input). With --cat, GNU parallel creates a temporary file with the name in {}, so you can literally do: parallel --pipe --cat wc {}. It implies --pipe unless --pipe-part is used. See also: --pipe --pipe-part --fifo.
--cleanup — removes transferred files. --cleanup will remove the transferred files on the remote computer after processing is done. Example:

find log -name '*gz' | parallel \
  --sshlogin server.example.com --transferfile {} \
  --return {.}.bz2 --cleanup "zcat {} | bzip -9 >{.}.bz2"

With --transferfile {} the file transferred to the remote computer will be removed on the remote computer. Directories on the remote computer containing the file will be removed if they are empty. With --return the file transferred from the remote computer will be removed on the remote computer. Directories on the remote computer containing the file will be removed if they are empty. --cleanup is ignored when not used with --basefile, --transfer, --transferfile or --return. See also: --basefile --transfer --transferfile --sshlogin --return.

--color — colours the output. Each job gets its own colour combination (background+foreground). --color is ignored when using -u. See also: --color-failed.
--color-failed, --cf — colours the output from failing jobs white on red. Useful if you have a lot of jobs and want to focus on the failing jobs. --color-failed is ignored when using -u, --line-buffer and unreliable when using --latest-line. See also: --color.
--colsep regexp, -C regexp — column separator. The input will be treated as a table with regexp separating the columns. The n'th column can be accessed using {n} or {n.}. E.g. {3} is the 3rd column. If there are more input sources, each input source will be separated, but the columns from each input source will be linked. Here {4} refers to column 2 in input source 2:

parallel --colsep '-' echo {4} {3} {2} {1} \
  ::: A-B C-D ::: e-f g-h

--colsep implies --trim rl, which can be overridden with --trim n. regexp is a Perl Regular Expression: https://perldoc.perl.org/perlre.html. See also: --csv {n} --trim --link --match.

--combineexec name, --combine-executable name — combines GNU parallel with another program into a single executable. Let us say you have developed myprg which takes a single argument. You do not want to parallelize it yourself. You could write a wrapper that uses GNU parallel called myparprg:

#!/bin/sh

parallel myprg ::: "$@"

But for others to use this, they need to install: GNU parallel, myprg, and myparprg. It would be easier to install if all could be packed into a single executable. If myprg is written in shell, you can use --embed. If myprg is a binary you can use --combineexec. Here we use gzip as example:

parallel --combineexec pargzip gzip -9 :::

You can now do:

./pargzip foo bar baz

If you want to pass options to gzip you can do:

parallel --combineexec pargzip gzip

Followed by:

./pargzip -1 ::: foo bar baz

See also: --embed --shebang --shebang-wrap.

--compress — compresses temporary files. If the output is big and very compressible this will take up less disk space in $TMPDIR and possibly be faster due to less disk I/O. GNU parallel will try pzstd, lbzip2, pbzip2, zstd, pigz, lz4, lzop, plzip, lzip, lrz, gzip, pxz, lzma, bzip2, xz, clzip, in that order, and use the first available. GNU parallel will use up to 8 processes per job waiting to be printed. See man parallel_design for details. See also: --compress-program.
--compress-program prg, --decompress-program prg — use prg for (de)compressing temporary files. It is assumed that prg -dc will decompress stdin (standard input) to stdout (standard output) unless --decompress-program is given. See also: --compress.
--csv — treats input as CSV-format. --colsep sets the field delimiter. --csv works very much like --colsep except it deals correctly with quoting. Compare:

echo '"1 big, 2 small","2""x4"" plank",12.34' |
  parallel --csv echo {1} of {2} at {3}

echo '"1 big, 2 small","2""x4"" plank",12.34' |
  parallel --colsep ',' echo {1} of {2} at {3}

Even quoted newlines are parsed correctly:

(echo '"Start of field 1 with newline'
 echo 'Line 2 in field 1";value 2') |
  parallel --csv --colsep ';' echo Field 1: {1} Field 2: {2}

When used with --pipe it will only pass full CSV-records. See also: --pipe --link {n} --colsep --header --match.

--ctag (obsolete: use --color --tag) — colour tag. If the values look very similar looking at the output it can be hard to tell when a new value is used. --ctag gives each value a random color. See also: --color --tag.
--ctagstring str (obsolete: use --color --tagstring) — colour tagstring. See also: --color --ctag --tagstring.
--delay duration — delays starting the next job by duration. GNU parallel will not start another job for the next duration. duration is in seconds, but can be postfixed with s, m, h, or d. If you append auto to duration (e.g. 13m3sauto) GNU parallel will automatically try to find the optimal value: If a job fails, duration is increased by 30%. If a job succeeds, duration is decreased by 10%. See also: TIME POSTFIXES --retries --ssh-delay.
--delimiter delim, -d delim — input records are terminated by delim. The specified delimiter may be characters, C-style character escapes such as \n, or octal (\012) or hexadecimal (\x0A) escape codes. Octal and hexadecimal escape codes are understood as for the printf command. See also: --colsep.
--dirnamereplace replace-str, --dnr replace-str — uses the replacement string replace-str instead of {//} for dirname of input line. See also: {//}.
--dry-run — prints the job to run on stdout (standard output), but does not run the job. Use -v -v to include the wrapping that GNU parallel generates (for remote jobs, --tmux, --nice, --pipe, --pipe-part, --fifo and --cat). Do not count on this literally, though, as the job may be scheduled on another computer or the local computer if : is in the list. See also: -v.
-E eof-str — sets the end of file string to eof-str. If the end of file string occurs as a line of input, the rest of the input is not read. If neither -E nor -e is used, no end of file string is used.
--eof[=eof-str], -e[eof-str] — this option is a synonym for the -E option. Use -E instead, because it is POSIX compliant for xargs while this option is not. If eof-str is omitted, there is no end of file string. If neither -E nor -e is used, no end of file string is used.
--embed — embeds GNU parallel in a shell script. If you need to distribute your script to someone who does not want to install GNU parallel you can embed GNU parallel in your own shell script:

parallel --embed > new_script

After which you add your code at the end of new_script. This is tested on ash, bash, dash, ksh, sh, and zsh.

--env var — copies exported environment variable var. This will copy var to the environment that the command is run in. This is especially useful for remote execution. In Bash var can also be a Bash function - just remember to export -f the function. The variable _ is special. It will copy all exported environment variables except for the ones mentioned in ~/.parallel/ignored_vars. To copy the full environment (both exported and not exported variables, arrays, and functions) use env_parallel. See also: --record-env --session --sshlogin command env_parallel.
--eta — shows the estimated number of seconds before finishing. This forces GNU parallel to read all jobs before starting to find the number of jobs (unless you use --total-jobs). GNU parallel normally only reads the next job to run. The estimate is based on the runtime of finished jobs, so the first estimate will only be shown when the first job has finished. Implies --progress. See also: --bar --progress --total-jobs.
--extensionreplace replace-str, --er replace-str — uses the replacement string replace-str instead of {.} for input line without extension. See also: {.}.
--fast — runs jobs fast. This disables a lot of functionality of GNU parallel to make jobs run as fast as possible: Think of it as the nitro racing car compared to the Volvo. Useful for benchmarking and if you have 1000's of tiny jobs. Compare:

    # ~10 seconds/1000 jobs
    seq 1000 | time parallel echo | wc -l
    # ~1 seconds/1000 jobs
    seq 1000 | time parallel --fast echo | wc -l
    # ~10 seconds/1000000 jobs
    seq 1000000 | time parallel --fast echo | wc -l

Fully supported:

{}, --compress/--compress-program, --dry-run, --jobs, --keep-order, --nice, --tag, --tagstring (fixed strings and {}), --ungroup.

Supported but slower (falls back to the job-queue path, ~0.5 ms/job):

All non-plain replacement strings: {.} {/} {//} {/.} {#} {%} {n}. --tagstring with replacement strings other than {}. Multiple input sources (multiple :::), -a/--arg-file, ::::. --null/-0, -d/--delimiter, --colsep/-C, --csv, --link/:::+/::::+, -N (for N>1), -m, -X, --filter, --trim (non-n values). -E/--eof: end-of-file string not implemented.

Supported with slightly different semantics:

--group (default): output is grouped per chunk (a chunk = several jobs). All stdout from the jobs in a chunk gets printed before all their stderr. 😌
--halt/--halt-on-error: failure detection is per chunk, so it won't trigger until a full chunk of jobs has finished. The exit value is 0 unless --halt is set; use --halt never to propagate non-zero exit codes without stopping early.
{=perl=}: expressions that can't be computed upfront (like slot()) may give you unexpected values. 🫠
--joblog/--jl: command lines longer than 4 KB may get truncated.
--env: environment variables are inherited via normal fork; parallel's explicit env-transfer mechanism is not invoked.

😭 Broken but may be supported with slightly different semantics

--timeout: applies to each chunk of jobs, not each individual job.
--bar/--eta/--progress: the inner parallel counts chunks, not individual jobs, so counts and time estimates are imprecise.
--delay: delay is applied after each chunk, not after each individual job.
--retries: these will retry the whole group of jobs.
--load/--limit/--memfree/--noswap: these will retry the whole group of jobs.

💀 Not supported

--results/-o: no per-job output file support.
--tty: jobs inherit /bin/sh's stdin (a pipe), not a terminal.
-L/--max-lines/-l: multi-line records not supported.
--verbose/-t: echoing commands to stderr not implemented.
--total-jobs: total job count is not known ahead of time.

🚫 Impossible given current architecture

--pipe/--pipepart/--tee/--fifo/--cat/--header/--group-by: separate execution modes incompatible with --fast.
SSH/remote options (--sshlogin, --sshloginfile/--slf, --basefile/--bf, --transfer, --return, --cleanup, --trc, --filter-hosts, --workdir, --hostgroups): --fast is local-only.
--tmux/--tmuxpane/--bg/--fg/--wait/--semaphore: separate execution modes.

If you need an unsupported option: File a bug. 🐛

See parallel_design for the architecture details.

✨ --fg ✨

Run command in foreground. 👑

With --tmux and --tmuxpane, GNU parallel will start tmux in the foreground.

With --semaphore, GNU parallel will run the command in the foreground (opposite of --bg), and wait for completion of the command before exiting. Exit code will be that of the command.

🌸 --fifo 🌸

Create a temporary fifo with content.

Normally --pipe and --pipe-part will give data to the program on stdin (standard input). With --fifo GNU parallel will create a temporary fifo with the name in {}, so you can do:

 parallel --pipe --fifo wc {}

⚠️ Beware: if the fifo is never opened for reading, the job will block forever:

 seq 1000000 | parallel --fifo echo This will block forever
 seq 1000000 | parallel --fifo 'echo This will not block < {}'

By using --fifo instead of --cat you may save I/O, since --cat will write to a temporary file, whereas --fifo will not. 💅

Implies --pipe unless --pipe-part is used.

See also: --cat --pipe --pipe-part

🦋 --filter filter 🦋

Only run jobs where filter is true.

filter can contain replacement strings and Perl code. Example:

 parallel --filter '{1}+{2}+{3} < 10' echo {1},{2},{3} \
   ::: {1..10} ::: {3..8} ::: {3..10}

Outputs: 1,3,3 1,3,4 1,3,5 1,4,3 1,4,4 1,5,3 2,3,3 2,3,4 2,4,3 3,3,3

 parallel --filter '{1} < {2}*{2}' echo {1},{2} \
   ::: {1..10} ::: {1..3}

Outputs: 1,2 1,3 2,2 2,3 3,2 3,3 4,3 5,3 6,3 7,3 8,3

 parallel --filter '{choose_k}' --plus echo {1},{2},{3} \
   ::: {1..5} ::: {1..5} ::: {1..5}

Outputs: 1,2,3 1,2,4 1,2,5 1,3,4 1,3,5 1,4,5 2,3,4 2,3,5 2,4,5 3,4,5

See also: skip() --no-run-if-empty {choose_k}

🧚 --filter-hosts 🧚

Remove down hosts.

For each remote host: check that login through ssh works. If not: do not use this host.

For performance reasons, this check is performed only at the start and every time --sshloginfile is changed. If a host goes down after the first check, it will go undetected until --sshloginfile is changed; --retries can be used to mitigate this. 😬

🚨 Currently you can not put --filter-hosts in a profile, $PARALLEL, /etc/parallel/config or similar. This is because GNU parallel uses GNU parallel to compute this, so you will get an infinite loop. This will likely be fixed in a later release.

See also: --sshloginfile --sshlogin --retries

🌈 --gnu 🌈

Behave like GNU parallel.

This option historically took precedence over --tollef. The --tollef option is now retired, and therefore may not be used. --gnu is kept for compatibility, but does nothing. 🫶

⭐ --group ⭐

Group output.

Output from each job is grouped together and is only printed when the command is finished. Stdout (standard output) first, followed by stderr (standard error).

This takes in the order of 0.5ms CPU time per job and depends on the speed of your disk for larger output.

--group is the default. 👑

With --fast: output is grouped per chunk (a chunk contains multiple jobs). Within a chunk, stdout and stderr of each job appear in natural order (jobs run sequentially), but the inner parallel buffers all of a chunk's stdout before all of its stderr. Use --tag for correct per-job labelling.

See also: --line-buffer --ungroup --tag

🎀 --group-by val 🎀

Group input by value.

Combined with --pipe/--pipe-part, --group-by groups lines with the same value into a record.

The value can be computed from the full line or from a single column.

val can be:

column number — use the value in the column numbered.
column name — treat the first line as a header and use the value in the column named.

(Not supported with --pipe-part.)

perl expression — run the perl expression and use $_ as the value.
column number perl expression — put the value of the column put in $_, run the perl expression, and use $_ as the value.
column name perl expression — put the value of the column put in $_, run the perl expression, and use $_ as the value.

(Not supported with --pipe-part.)

Example:

UserID, Consumption
123,    1
123,    2
12-3,   1
221,    3
221,    1
2/21,   5

If you want to group 123, 12-3, 221, and 2/21 into 4 records and pass one record at a time to wc:

 tail -n +2 table.csv | \
   parallel --pipe --colsep , --group-by 1 -kN1 wc

Make GNU parallel treat the first line as a header:

 cat table.csv | \
   parallel --pipe --colsep , --header : --group-by 1 -kN1 wc

Address column by column name:

 cat table.csv | \
   parallel --pipe --colsep , --header : --group-by UserID -kN1 wc

If 12-3 and 123 are really the same UserID, remove non-digits in UserID when grouping:

 cat table.csv | parallel --pipe --colsep , --header : \
   --group-by 'UserID s/\D//g' -kN1 wc

See also: SPREADING BLOCKS OF DATA --pipe --pipe-part --bin --shard --round-robin

💖 --help 💖

-h — print a summary of the options to GNU parallel and exit.

🔥 --halt-on-error val / --halt val 🔥

When should GNU parallel terminate?

In some situations it makes no sense to run all jobs. GNU parallel should simply stop as soon as a condition is met.

val defaults to never, which runs all jobs no matter what.

val can also take on the form of when,why.

when can be now — which means kill all running jobs and halt immediately — or it can be soon, which means wait for all running jobs to complete, but start no new jobs.

why can be fail=X, fail=Y%, success=X, success=Y%, done=X, or done=Y%, where X is the number of jobs that has to fail, succeed, or be done before halting, and Y is the percentage of jobs that has to fail, succeed, or be done before halting.

Example:

 --halt now,fail=1     exit when a job has failed. Kill running
                       jobs.

 --halt soon,fail=3    exit when 3 jobs have failed, but wait for
                       running jobs to complete.

 --halt soon,fail=3%   exit when 3% of the jobs have failed, but
                       wait for running jobs to complete.

 --halt now,success=1  exit when a job has succeeded. Kill running
                       jobs.

 --halt soon,success=3 exit when 3 jobs have succeeded, but wait for
                       running jobs to complete.

 --halt now,success=3% exit when 3% of the jobs have succeeded. Kill
                       running jobs.

 --halt now,done=1     exit when a job has finished. Kill running
                       jobs.

 --halt soon,done=3    exit when 3 jobs have finished, but wait for
                       running jobs to complete.

 --halt now,done=3%    exit when 3% of the jobs have finished. Kill
                       running jobs.

For backwards compatibility these also work:

 0           never

 1           soon,fail=1

 2           now,fail=1

 -1          soon,success=1

 -2          now,success=1

 1-99%       soon,fail=1-99%

🍓 --header regexp (alpha testing) 🍓

Use regexp as header.

For normal usage the matched header (typically the first line: --header '.*\n') will be split using --colsep (which will default to '\t') and column names can be used as replacement variables: {column name}, {column name/}, {column name//}, {column name/.}, {column name.}, {=column name perl expression =}, ..

For --pipe the matched header will be prepended to each output.

--header : is an alias for --header '.*\n'.

If regexp is a number, it is a fixed number of lines.

--header 0 is special: it will make replacement strings for files given with --arg-file or ::::. It will make {foo/bar} for the file foo/bar. The files cannot contain the --colsep string. 🫠

See also: --colsep --pipe --pipe-part --arg-file

💜 --hostgroups / --hgrp 💜

Enable hostgroups on arguments.

If an argument contains '@' the string after '@' will be removed and treated as a list of hostgroups on which this job is allowed to run. If there is no --sshlogin with a corresponding group, the job will run on any hostgroup.

Example:

okay so picking right back up with the --hostgroups example — the way this works is genuinely iconic once you see the whole command, bestie:

parallel --hostgroups \
  --sshlogin @grp1/myserver1 -S @grp1+grp2/myserver2 \
  --sshlogin @grp3/myserver3 \
  echo ::: my_grp1_arg@grp1 arg_for_grp2@grp2 third@grp1+grp3

my_grp1_arg may be run on either myserver1 or myserver2, third may be run on either myserver1 or myserver3, but arg_for_grp2 will only be run on myserver2. no cap, that routing is lowkey magical ✨

See also: --sshlogin $PARALLEL_HOSTGROUPS $PARALLEL_ARGHOSTGROUPS

-I replace-str — use the replacement string replace-str instead of {} 🫶

See also: {}

--replace [replace-str], -i [replace-str] — this option is deprecated; use -I instead. It is a synonym for -Ireplace-str if replace-str is specified, and for -I {} otherwise 💅

See also: {}

--joblog logfile, --jl logfile — logfile for executed jobs. Save a list of the executed jobs to logfile in this TAB separated format: sequence number, sshlogin, start time as seconds since epoch, run time in seconds, bytes in files transferred, bytes in files returned, exit status, signal, and command run. iykyk, that's a whole lotta columns 👑

For --pipe, bytes transferred and bytes returned are the number of input and output bytes.

If logfile is prepended with +, log lines will be appended to the logfile.

To convert the times into ISO-8601 strict do:

cat logfile | perl -a -F"\t" -ne \
  'chomp($F[2]=`date -d \@$F[2] +%FT%T`); print join("\t",@F)'

If the host is long, you can use column -t to pretty print it:

cat joblog | column -t

See also: --resume --resume-failed --progress

--jobs num, -j num, --max-procs num, -P num — number of jobslots on each machine. Run up to num jobs in parallel. Default is 100% 🍓
- num — run up to num jobs in parallel.
- 0 — run as many as possible (this can take a while to determine).

Due to a bug -j 0 will also evaluate replacement strings twice up to the number of joblots:

# This will not count from 1 but from number-of-jobslots
seq 10000 | parallel -j0   echo '{= $_ = $foo++; =}' | head
# This will count from 1
seq 10000 | parallel -j100 echo '{= $_ = $foo++; =}' | head

num% — multiply the number of CPU threads by num percent. E.g. 100% means one job per CPU thread on each machine.
+num — add num to the number of CPU threads.
-num — subtract num from the number of CPU threads.
expr — evaluate expr. E.g. '12/2' to get 6, '+25%' gives the same as '125%', or complex expressions like '+3*log(55)%' which means: multiply 3 by log(55), multiply that by the number of CPU threads and divide by 100, add this to the number of CPU threads. An expression that evaluates to less than 1 is replaced with 1.
procfile — read parameter from file. Use the content of procfile as parameter for -j. E.g. procfile could contain the string 100% or +2 or 10.

If procfile is changed when a job completes, procfile is read again and the new number of jobs is computed. If the number is lower than before, running jobs will be allowed to finish but new jobs will not be started until the wanted number of jobs has been reached. This makes it possible to change the number of simultaneous running jobs while GNU parallel is running 🔥

numauto — if num ends in auto, GNU parallel will use the value as a max value. If a job fails, GNU parallel will adjust the number of jobs down with a factor. If a job succeeds GNU parallel will adjust the number of jobs up with a small factor.

So if the number of jobs is a bit too high, GNU parallel will dynamically lower it.

Example:

# This fails if there are more than n sleeps running
n_sleeps() {
  nsleeps=$(ps aux | grep sleep | grep 12345 | wc -l)
  echo $nsleeps sleeps running now
  [ $nsleeps -lt $1 ]
}
export -f n_sleeps
parallel --jobs 20auto 'sleep 2.${RANDOM}{}12345; n_sleeps 5' ::: {1..100}

This will start 20 jobs in parallel. These will fail because more than 5 sleeps are running. GNU parallel will adjust the number of jobs until around 5 are running in parallel.

If the evaluated number is less than 1 then 1 will be used.

If --semaphore is set, the default is 1 thus making a mutex.

See also: --use-cores-instead-of-threads --use-sockets-instead-of-threads

--keep-order, -k — keep sequence of output same as the order of input. Normally the output of a job will be printed as soon as the job completes. Try this to see the difference:

parallel -j4 sleep {}\; echo {} ::: 2 1 4 3
parallel -j4 -k sleep {}\; echo {} ::: 2 1 4 3

If used with --onall or --nonall the output gets grouped by sshlogin in sorted order.

--keep-order cannot keep the output order when used with --pipe --round-robin. Here it instead means that the jobslots will get the same blocks as input in the same order in every run if the input is kept the same. Run each of these twice and compare:

seq 10000000 | parallel --pipe --round-robin 'sleep 0.$RANDOM; wc'
seq 10000000 | parallel --pipe -k --round-robin 'sleep 0.$RANDOM; wc'

-k only affects the order in which the output is printed — not the order in which jobs are run.

See also: --group --line-buffer

-L recsize — when used with --pipe: read records of recsize. When used otherwise: use at most recsize nonblank input lines per command line. Trailing blanks cause an input line to be logically continued on the next input line.

-L 0 means read one line, but insert 0 arguments on the command line.

recsize can be postfixed with K, M, G, T, P, k, m, g, t, or p.

Implies -X unless -m, --xargs, or --pipe is set.

See also: UNIT PREFIX -N --max-lines --block -X -m --xargs --pipe

--max-lines [recsize], -l[recsize] — when used with --pipe: read records of recsize lines. When used otherwise: synonym for the -L option. Unlike -L, the recsize argument is optional. If recsize is not specified, it defaults to one. The -l option is deprecated since the POSIX standard specifies -L instead.

-l 0 is an alias for -l 1.

Implies -X unless -m, --xargs, or --pipe is set.

See also: UNIT PREFIX -N --block -X -m --xargs --pipe

--limit "command args" — dynamic job limit. Before starting a new job run command with args. The exit value of command determines what GNU parallel will do:
- 0 — below limit. Start another job.
- 1 — over limit. Start no jobs.
- 2 — way over limit. Kill the youngest job 💀

You can use any shell command. There are 3 predefined commands:

"io n" — limit for I/O. The amount of disk I/O will be computed as a value 0-100, where 0 is no I/O and 100 is at least one disk is 100% saturated.
"load n" — similar to --load.
"mem n" — similar to --memfree.

See also: --memfree --load

--latest-line, --ll — print the latest line. Each job gets a single line on the screen that is updated with the last full line from currently running jobs.

The screen keeps the oldest running job in view, so younger jobs may not be visible. These will, however, be shown when the oldest job finishes.

Running this example makes it easier to understand. Note how the screen scrolls when the job at the top of the screen finishes:

slow_seq() {
  # Run seq with 33 characters per second
  seq "$@" |
    perl -ne '$|=1; for(split//){ print; select($a,$a,$a,0.03);}'
}
export -f slow_seq
parallel --shuf -j99 --ll --tag --bar --color slow_seq {} ::: {1..300}

See also: --line-buffer

--line-buffer, --lb — buffer output on line basis. --group will keep the output together for a whole job. --ungroup allows output to mixup with half a line coming from one job and half a line coming from another job. --line-buffer fits between these two: GNU parallel will print a full line, but will allow for mixing lines of different jobs.

--line-buffer takes more CPU power than both --group and --ungroup, but can be much faster than --group if the CPU is not the limiting factor.

Normally --line-buffer does not buffer on disk, and can thus process an infinite amount of data, but it will buffer on disk when combined with: --keep-order, --results, --compress, and --files. This will make it as slow as --group and will limit output to the available disk space.

With --keep-order --line-buffer will output lines from the first job continuously while it is running, then lines from the second job while that is running. It will buffer full lines, but jobs will not mix. Compare:

parallel -j0 'echo [{};sleep {};echo {}]' ::: 1 3 2 4
parallel -j0 --lb 'echo [{};sleep {};echo {}]' ::: 1 3 2 4
parallel -j0 -k --lb 'echo [{};sleep {};echo {}]' ::: 1 3 2 4

See also: --group --ungroup --keep-order --tag

--link, --xapply — link input sources. Read multiple input sources like the command xapply. If multiple input sources are given, one argument will be read from each of the input sources. The arguments can be accessed in the command as {1} .. {n}, so {1} will be a line from the first input source, and {6} will refer to the line with the same line number from the 6th input source.

Compare these two:

parallel echo {1} {2} ::: 1 2 3 ::: a b c
parallel --link echo {1} {2} ::: 1 2 3 ::: a b c

Arguments will be recycled if one input source has more arguments than the others:

parallel --link echo {1} {2} {3} \
  ::: 1 2 ::: I II III ::: a b c d e f g

See also: --header :::+ ::::+

--load max-load — only start jobs if load is less than max-load. Do not start new jobs on a given computer unless the number of running processes on the computer is less than max-load. max-load uses the same syntax as --jobs, so 100% for one per CPU is a valid setting. Only difference is 0 which is interpreted as 0.01.

See also: --limit --jobs

--controlmaster, -M — use ssh's ControlMaster to make ssh connections faster 🌈

Useful if jobs run remote and are very fast to run. This is disabled for sshlogins that specify their own ssh command.

🍓 CSV file output

If name ends in .csv/.tsv the output will be a CSV-file named name, periodt.

.csv gives you a comma separated value file. .tsv gives you a TAB separated value file. Iconic file-format duality, we love to see it.

-.csv/-.tsv are special: it will give the file on stdout (standard output). Slay.

🦋 JSON file output

If name ends in .json the output will be a JSON-file named name. Very demure, very mindful.

-.json is special: it will give the file on stdout (standard output). ✨

🎀 Replacement string output file

If name contains a replacement string and the replaced result does not end in /, then the standard output will be stored in a file named by this result. Standard error gets stored in the same file name with .err added, and the sequence number gets stored in the same file name with .seq added. Three-for-one deal, genuinely.

E.g.

parallel --results my_{} echo ::: foo bar baz

will generate the files:

my_bar
my_bar.err
my_bar.seq
my_baz
my_baz.err
my_baz.seq
my_foo
my_foo.err
my_foo.seq

💜 Replacement string output dir

If name contains a replacement string and the replaced result ends in /, then output files will be stored in the resulting dir.

E.g.

parallel --results my_{}/ echo ::: foo bar baz

will generate the files:

my_bar/seq
my_bar/stderr
my_bar/stdout
my_baz/seq
my_baz/stderr
my_baz/stdout
my_foo/seq
my_foo/stderr
my_foo/stdout

See also: --output-as-files --tag --header --joblog

--resume — picks up from the last unfinished job. 🫶

By reading --joblog or the --results dir, GNU parallel figures out the last unfinished job and just continues from there. But ngl, since GNU parallel only looks at the sequence numbers in --joblog, that means the input, the command, and --joblog all have to remain unchanged — otherwise GNU parallel may run wrong commands. Yeah. Fr fr, don't touch them.

See also: --joblog --results --resume-failed --retries

--resume-failed — retries all the failed jobs and resumes from the last unfinished job. The overachiever of resume flags. 💅

By reading --joblog, GNU parallel will figure out the failed jobs and run those again. After that it will resume the last unfinished job and continue from there. And again: since GNU parallel only looks at the sequence numbers in --joblog, the input, the command, and --joblog all have to remain unchanged — otherwise GNU parallel may run wrong commands.

See also: --joblog --resume --retry-failed --retries

--retry-failed — retries all failed jobs straight out of the joblog. 🔥

By reading --joblog, GNU parallel will figure out the failed jobs and run those again.

--retry-failed ignores the command and arguments on the command line: it only looks at the joblog. Like, it does not care what you typed, bestie. The joblog is the main character here.

💀 Differences between `--resume`, `--resume-failed`, `--retry-failed`

In this example exit {= $_%=2 =} will cause every other job to fail.

timeout -k 1 4 parallel --joblog log -j10 \
  'sleep {}; exit {= $_%=2 =}' ::: {10..1}

4 jobs completed. 2 failed:

Seq   [...]   Exitval Signal  Command
10    [...]   1       0       sleep 1; exit 1
9     [...]   0       0       sleep 2; exit 0
8     [...]   1       0       sleep 3; exit 1
7     [...]   0       0       sleep 4; exit 0

--resume does not care about the Exitval, but only looks at Seq. If the Seq is run, it will not be run again. So if needed, you can change the command for the seqs not run yet:

parallel --resume --joblog log -j10 \
  'sleep .{}; exit {= $_%=2 =}' ::: {10..1}

Seq   [...]   Exitval Signal  Command
[... as above ...]
1     [...]   0       0       sleep .10; exit 0
6     [...]   1       0       sleep .5; exit 1
5     [...]   0       0       sleep .6; exit 0
4     [...]   1       0       sleep .7; exit 1
3     [...]   0       0       sleep .8; exit 0
2     [...]   1       0       sleep .9; exit 1

--resume-failed cares about the Exitval, but also only looks at Seq to figure out which commands to run. Again this means you can change the command, but not the arguments. It will run the failed seqs and the seqs not yet run:

parallel --resume-failed --joblog log -j10 \
  'echo {};sleep .{}; exit {= $_%=3 =}' ::: {10..1}

Seq   [...]   Exitval Signal  Command
[... as above ...]
10    [...]   1       0       echo 1;sleep .1; exit 1
8     [...]   0       0       echo 3;sleep .3; exit 0
6     [...]   2       0       echo 5;sleep .5; exit 2
4     [...]   1       0       echo 7;sleep .7; exit 1
2     [...]   0       0       echo 9;sleep .9; exit 0

--retry-failed cares about the Exitval, but takes the command from the joblog. It ignores any arguments or commands given on the command line:

parallel --retry-failed --joblog log -j10 this part is ignored

Seq   [...]   Exitval Signal  Command
[... as above ...]
10    [...]   1       0       echo 1;sleep .1; exit 1
6     [...]   2       0       echo 5;sleep .5; exit 2
4     [...]   1       0       echo 7;sleep .7; exit 1

See also: --joblog --resume --resume-failed --retries

--retries n — try failing jobs n times. Giving it another shot, we've all been there. 🙈

If a job fails, retry it on another computer on which it has not failed. Do this n times. If there are fewer than n computers in --sshlogin, GNU parallel will re-use all the computers. This is useful if some jobs fail for no apparent reason (such as network failure). Relatable honestly.

n=0 means infinite. Yes, infinite. No cap, literally.

🌸 OPTIONS, continued 🌸

--rsync-opts options — the options you want to yeet on over to rsync. Also: setting --rsync-opts takes precedence over setting the environment variable $PARALLEL_RSYNC_OPTS. (so if you're like "why is this not applying", check the env var, bestie 🙈)
--max-chars max-chars, -s max-chars — cap the length of the command. Use at most max-chars characters per command line — and that count includes the command itself, the initial-arguments, and the terminating nulls at the ends of the argument strings. The largest allowed value is system-dependent, and it's calculated as the argument length limit for exec, less the size of your environment. The default value is the maximum. max-chars can be postfixed with K, M, G, T, P, k, m, g, t, or p (see UNIT PREFIX). Implies -X unless -m or --xargs is set. See also: -X -m --xargs --max-line-length-allowed --show-limits
--show-limits — display the limits given by the operating system. Like, actually show you the limits on the command-line length that the OS imposes, plus the ones from the -s option. 🎀 If you don't want GNU parallel to do anything, pipe the input from /dev/null (and maybe also specify --no-run-if-empty). See also: --max-chars --max-line-length-allowed --version
--semaphore — work as a counting semaphore. This makes GNU parallel start the command in the background. When the number of jobs given by --jobs is reached, GNU parallel waits for one of these to complete before starting another command. --semaphore implies --bg unless --fg is specified. And fun fact: the command sem is an alias for parallel --semaphore 💜 See also: man sem --bg --fg --semaphore-name --semaphore-timeout --wait
--semaphore-name name, --id name — use name as the name of the semaphore. The default is the name of the controlling tty (the output from tty). The default normally works as expected when you use it interactively, but when you use it in a script, name should be set. $$ or my_task_name are often a good value. The semaphore is stored in ~/.parallel/semaphores/. Implies --semaphore. See also: man sem --semaphore
--semaphore-timeout secs, --st secs — the "I waited long enough" option, and it has two modes 😤
- If secs > 0: if the semaphore is not released within secs seconds, take it anyway.
- If secs < 0: if the semaphore is not released within secs seconds, exit. secs is in seconds, but can be postfixed with s, m, h, or d (see the section TIME POSTFIXES). Implies --semaphore. See also: man sem
--seqreplace replace-str — use the replacement string replace-str instead of {#} for the job sequence number. Swappy swap ✨ See also: {#}
--session — record names in the current environment into $PARALLEL_IGNORED_NAMES and exit. Only used with env_parallel. Aliases, functions, and variables whose names are in $PARALLEL_IGNORED_NAMES will not be copied. So lowkey: you should set the variables/functions you want copied after running --session. It's similar to --record-env, but only for this session. Only supported in Ash, Bash, Dash, Ksh, Sh, and Zsh. See also: --env --record-env env_parallel
--shard shardexpr — use shardexpr as the shard key and shard the input to the jobs. shardexpr is [column number|column name] [perlexpression], e.g.:

3
Address
3 $_%=100
Address s/\d//g

Each input line gets split using --colsep. The string of the column goes into $_, the perl expression is executed, and then the resulting string is hashed — so that all lines of a given value get handed to the same job slot. It's similar to sharding in databases, fr fr. Performance is in the order of 100K rows per second. Faster if the shardcol is small (<10), slower if it's big (>100). --shard requires --pipe and a fixed numeric value for --jobs. See the section: SPREADING BLOCKS OF DATA. See also: --bin --group-by --round-robin

--shebang, --hashbang — GNU parallel can be called as a shebang (#!) command on the first line of a script. The content of the file will be treated as inputsource. Like this 👇

#!/usr/bin/parallel --shebang -r wget

https://ftpmirror.gnu.org/parallel/parallel-20120822.tar.bz2
https://ftpmirror.gnu.org/parallel/parallel-20130822.tar.bz2
https://ftpmirror.gnu.org/parallel/parallel-20140822.tar.bz2

--shebang must be set as the first option. On FreeBSD, env is needed:

#!/usr/bin/env -S parallel --shebang -r wget

https://ftpmirror.gnu.org/parallel/parallel-20120822.tar.bz2
https://ftpmirror.gnu.org/parallel/parallel-20130822.tar.bz2
https://ftpmirror.gnu.org/parallel/parallel-20140822.tar.bz2

Heads up bestie: there are many limitations of shebang (#!) depending on your operating system. See the details on https://www.in-ulm.de/~mascheck/various/shebang/ 💀

See also: --shebang-wrap

--shebang-wrap — GNU parallel can parallelize scripts by wrapping the shebang line. 🧚 If the program can be run like this:

cat arguments | parallel the_program

then the script can be changed to:

#!/usr/bin/parallel --shebang-wrap /original/parser --options

E.g.

#!/usr/bin/parallel --shebang-wrap /usr/bin/python

If the program can be run like this:

cat data | parallel --pipe the_program

then the script can be changed to:

#!/usr/bin/parallel --shebang-wrap --pipe /orig/parser --opts

E.g.

#!/usr/bin/parallel --shebang-wrap --pipe /usr/bin/perl -w

--shebang-wrap must be set as the first option. See also: --shebang

--shell-completion shell — generate shell completion code for interactive shells. Supported shells: bash zsh. Use auto as shell to automatically detect the running shell. Activate the completion code with:

zsh% eval "$(parallel --shell-completion auto)"
bash$ eval "$(parallel --shell-completion auto)"

Or put this at /usr/share/zsh/site-functions/_parallel, then run compinit to generate ~/.zcompdump:

#compdef parallel

(( $+functions[_comp_parallel] )) ||
  eval "$(parallel --shell-completion auto)" &&
  _comp_parallel

--shell-quote — doesn't run the command, just quotes it. Useful for making quoted composed commands for GNU parallel. And the double-tap: multiple --shell-quote will quote the string multiple times, so parallel --shell-quote | parallel --shell-quote can be written as parallel --shell-quote --shell-quote 🌈 See also: --quote
--shuf — shuffle jobs. When you have multiple input sources it's hard to randomize jobs, so --shuf generates all the jobs and shuffles them before running them. Super useful to get a quick preview of the results before running the full batch. Combine it with --halt soon,done=1% and you can run a random 1% sample of all jobs:

parallel --shuf --halt soon,done=1% echo ::: {1..100} ::: {1..100}

See also: --halt

--skip-first-line — do not use the first line of input. (GNU parallel itself uses this when it's called with --shebang.) ⭐
--sql DBURL (obsolete) — nope, this one's retired 🫶 use --sql-master instead.
--sql-master DBURL — submit jobs through an SQL server, bestie 💅 DBURL has to point at a table, and that table is gonna hold the same info as --joblog, the values from your input sources (stored in columns V1 .. Vn), and the output (stored in columns Stdout and Stderr).

If DBURL is prepended with +, GNU parallel assumes the table is already made with the correct columns and just appends the jobs to it 😌

If DBURL is not prepended with +, the table will be dropped and created with the correct amount of V-columns unless

okay so that sentence genuinely just stops there in the original, bestie 😭 no cap, that's the whole line.

--sqlmaster does not run any jobs, it only creates the values for the jobs to be run. One or more --sqlworker must be run to actually execute the jobs. fr fr.

If --wait is set, GNU parallel will wait for the jobs to complete ⏳

The format of a DBURL is:

[sql:]vendor://[[user][:pwd]@][host][:port]/[db]/[table]

E.g.

sql:mysql://hr:hr@localhost:3306/hrdb/jobs
mysql://scott:tiger@my.example.com/pardb/paralleljobs
sql:oracle://scott:tiger@ora.example.com/xe/parjob
postgresql://scott:tiger@pg.example.com/pgdb/parjob
pg:///parjob
sqlite3:///%2Ftmp%2Fpardb.sqlite/parjob
sqlite:///file_in_current_dir.sqlite/my_table
csv:///%2Ftmp%2Fpardb/parjob
csv:///./file_in_current_dir
pg:////

Notice how the / in the path of sqlite and CSV must be encoded as %2F 👀 Except the last / in CSV, which must be a /.

db and table default to $USER: pg://// = pg:///$USER/$USER

It can also be an alias from ~/.sql/aliases:

:myalias mysql:///mydb/paralleljobs

See also: --sql-and-worker --sql-worker --joblog

--sql-and-worker DBURL — shorthand for --sql-master DBURL --sql-worker DBURL ✨ literally just two options in a trench coat.

🎬 EXAMPLES 🎬

See: man parallel_examples — yes that's the whole bit, go read the examples page, it's giving main character energy ⭐

🧩 SPREADING BLOCKS OF DATA 🧩

okay so --round-robin, --pipe-part, --shard, --bin and --group-by are all specialized versions of --pipe. they're the same family, different vibes. 💖

In the following, n is the number of jobslots given by --jobs. A record starts with --recstart and ends with --recend. It is typically a full line. A chunk is a number of full records that is approximately the size of a block. A block can contain half records — a chunk cannot. no half-chunks, we don't do that here 🎀

--pipe starts one job per chunk. It reads blocks from stdin (standard input). It finds a record end near a block border and passes a chunk to the program.
--pipe-part starts one job per chunk — just like normal --pipe. It first finds record endings near all block borders in the file and then starts the jobs. By using --block -1 it will set the block size to size-of-file/n. Used this way it will start n jobs in total. 👑
--round-robin starts n jobs in total. It reads a block and passes a chunk to whichever job is ready to read. It does not parse the content except for identifying where a record ends, to make sure it only passes full records. lowkey the chill one 🌸
--shard starts n jobs in total. It parses each line to read the string in the given column. Based on this string the line is passed to one of the n jobs. All lines having this string will be given to the same jobslot. sorting the besties into group chats fr fr 💅
--bin works like --shard, but the value of the column must be numeric and it is the jobslot number it will be passed to. If the value is bigger than n, then n will be subtracted from the value until the value is smaller than or equal to n. it just keeps subtracting, iconic persistence 🔥
--group-by starts one job per chunk. Record borders are not given by --recend/--recstart. Instead a record is defined by a group of lines having the same string in a given column. So the string of a given column changes at a chunk border. With --pipe every line is parsed; with --pipe-part only a few lines are parsed to find the chunk border. ✨
--group-by can be combined with --round-robin or --pipe-part. 🫶

⏱️ TIME POSTFIXES ⏱️

Arguments that give a duration are given in seconds, but you can write them as floats postfixed with s, m, h, or d, which multiplies the float by 1, 60, 60*60, or 60*60*24. So these are literally equivalent, no cap:

100000 and 1d3.5h16.6m4s

see? same girl, different outfit 💖

🔢 UNIT PREFIX 🔢

Many numerical arguments in GNU parallel can be postfixed with K, M, G, T, P, k, m, g, t, or p, which multiplies the number by 1024, 1048576, 1073741824, 1099511627776, 1125899906842624, 1000, 1000000, 1000000000, 1000000000000, or 1000000000000000, respectively. yes the uppercase ones are the 1024-flavored ones, keep up bestie 😌

You can even give it as a math expression. E.g. 1000000 can be written as 1M-12*2.024*2k. math slay 🧮✨

🪄 QUOTING 🪄

GNU parallel is very liberal in quoting. You only need to quote characters that have special meaning in shell:

( ) $ ` ' " < > ; | \

and depending on context these need to be quoted too:

~ & ! ? space * { #

Therefore most people will never need more quoting than putting \ in front of the special characters. 🎀

Often you can simply put \' around every ':

perl -ne '/^\S+\s+\S+$/ and print $ARGV,"\n"' file

can be quoted:

parallel perl -ne \''/^\S+\s+\S+$/ and print $ARGV,"\n"'\' ::: file

However, when you want to use a shell variable you need to quote the $-sign. Here is an example using $PARALLEL_SEQ. This variable is set by GNU parallel itself, so the evaluation of the $ must be done by the sub shell started by GNU parallel:

seq 10 | parallel -N2 echo seq:\$PARALLEL_SEQ arg1:{1} arg2:{2}

If the variable is set before GNU parallel starts you can do this:

VAR=this_is_set_before_starting
echo test | parallel echo {} $VAR

Prints: test this_is_set_before_starting

It is a little more tricky if the variable contains more than one space in a row:

VAR="two  spaces  between  each  word"
echo test | parallel echo {} \'"$VAR"\'

Prints: test two spaces between each word

If the variable should not be evaluated by the shell starting GNU parallel but be evaluated by the sub shell started by GNU parallel, then you need to quote it:

echo test | parallel VAR=this_is_set_after_starting \; echo {} \$VAR

Prints: test this_is_set_after_starting

It is a little more tricky if the variable contains space:

echo test |\
  parallel VAR='"two  spaces  between  each  word"' echo {} \'"$VAR"\'

Prints: test two spaces between each word

$$ is the shell variable containing the process id of the shell. This will print the process id of the shell running GNU parallel:

seq 10 | parallel echo $$

And this will print the process ids of the sub shells started by GNU parallel:

seq 10 | parallel echo \$\$

If the special characters should not be evaluated by the sub shell then you need to protect it against evaluation from both the shell starting GNU parallel and the sub shell:

echo test | parallel echo {} \\\$VAR

Prints: test $VAR

GNU parallel can protect against evaluation by the sub shell by using -q:

echo test | parallel -q echo {} \$VAR

Prints: test $VAR

This is particularly useful if you have lots of quoting. If you want to run a perl script like this:

perl -ne '/^\S+\s+\S+$/ and print $ARGV,"\n"' file

It needs to be quoted like one of these:

ls | parallel perl -ne '/^\\S+\\s+\\S+\$/\ and\ print\ \$ARGV,\"\\n\"'
ls | parallel perl -ne \''/^\S+\s+\S+$/ and print $ARGV,"\n"'\'

Notice how spaces, \'s, "'s, and $'s need to be quoted. GNU parallel can do the quoting by using option -q:

ls | parallel -q  perl -ne '/^\S+\s+\S+$/ and print $ARGV,"\n"'

However, this means you cannot make the sub shell interpret special characters. For example because of -q this WILL NOT WORK:

ls *.gz | parallel -q "zcat {} >{.}"
ls *.gz | parallel -q "zcat {} | bzip2 >{.}.bz2"

because > and | need to be interpreted by the sub shell. 💀

If you get errors like:

sh: -c: line 0: syntax error near unexpected token
sh: Syntax error: Unterminated quoted string
sh: -c: line 0: unexpected EOF while looking for matching `''
sh: -c: line 1: syntax error: unexpected end of file
zsh:1: no matches found:

then you might try using -q. 😭 (we've all been there)

If you are using bash process substitution like <(cat foo) then you may try -q and prepending command with bash -c:

ls | parallel -q bash -c 'wc -c <(echo {})'

Or for substituting output:

ls | parallel -q bash -c \
  'tar c {} | tee >(gzip >{}.tar.gz) | bzip2 >{}.tar.bz2'

Conclusion: If this is confusing consider avoiding having to deal with quoting by writing a small script or a function (remember to export -f the function) and have GNU parallel call that. honestly the calmest option 🧚

📋 LIST RUNNING JOBS 📋

If you want a list of the jobs currently running you can run:

killall -USR1 parallel

GNU parallel will then print the currently running jobs on stderr (standard error). instant behind-the-scenes peek, love that for us 👀✨

🛑 COMPLETE RUNNING JOBS BUT DO NOT START NEW JOBS 🛑

If you regret starting a lot of jobs you can simply break GNU parallel, but if you want to make sure you do not have half-completed jobs you should send the signal SIGHUP to GNU parallel:

killall -HUP parallel

This will tell GNU parallel to not start any new jobs, but wait until the currently running jobs are finished before exiting. no abandoned babies, only completed ones 🫶💖

🌸 ENVIRONMENT VARIABLES 🌸

okay bestie, these are the env vars GNU parallel peeks at (and sometimes sets itself, which is giving mutual main character energy). lowkey important, no cap 💅

$PARALLEL_HOME — the dir where GNU parallel keeps its stuff: config files, semaphores, and cache info between invocations. If you point it at a dir that doesn't exist yet, it'll just create it for you, slay. Default? $HOME/.parallel ✨
$PARALLEL_ARGHOSTGROUPS — when you're using --hostgroups, GNU parallel sets this to the hostgroups of the job. Remember to quote the $ so it gets evaluated by the correct shell (iykyk). Or just use --plus and {agrp} 🫶
$PARALLEL_HOSTGROUPS — again with --hostgroups, but this one is set to the hostgroups of the sshlogin that the job runs on. Same deal: quote the $ for the right shell, or use --plus and {hgrp}.
$PARALLEL_JOBSLOT — set by GNU parallel, usable inside jobs it runs. Quote that $! Or --plus and {slot}. So what is it? It's the jobslot of the job — equal to {%} unless the job is being retried (see {%} for the deets) 👑
$PARALLEL_PID — also set by GNU parallel, also usable in jobs. Quote the $. This one is iconic because it lets your jobs talk back to GNU parallel directly.

Example: if each job tests a solution and one of them finds the solution, that job can tell GNU parallel to stop starting new jobs with: kill -HUP $PARALLEL_PID. Note: this only works on the local computer 💀

$PARALLEL_RSYNC_OPTS — options to pass along to rsync. Defaults to: -rlDzR
$PARALLEL_SHELL — the shell used for commands run by GNU parallel. The priority list goes like this:
- $PARALLEL_SHELL. If that's undefined, use:
- the shell that started GNU parallel. If that can't be figured out, use:
- $SHELL. If that's undefined, use:
- /bin/sh
$PARALLEL_SSH — GNU parallel defaults to using the ssh command for remote access. You can override that with $PARALLEL_SSH, which can itself be overridden with --ssh. It can also be set per server (see --sshlogin) 🦋
$PARALLEL_SSHHOST — set by GNU parallel, usable in jobs. Quote the $ for the right shell, or use --plus and {host}. It's the host part of an sshlogin line. Example: this

4//usr/bin/specialssh user@host

becomes:

host

$PARALLEL_SSHLOGIN — set by GNU parallel, usable in jobs. Quote the $ (you know the drill), or use --plus and {sshlogin}. The value is the sshlogin line with the number of threads removed. Example: this

4//usr/bin/specialssh user@host

becomes:

/usr/bin/specialssh user@host

$PARALLEL_SEQ — set by GNU parallel, usable in jobs, quote the $ bestie. It's the sequence number of the job running. Example:

seq 10 | parallel -N2 \
  echo seq:'$'PARALLEL_SEQ arg1:{1} arg2:{2}

oh and btw {#} is a shorthand for $PARALLEL_SEQ 💖

$PARALLEL_TMUX — path to tmux. If it's unset, the tmux in $PATH gets used.
$TMPDIR — directory for temporary files. See also: --tmpdir
$PARALLEL_REMOTE_TMPDIR — directory for temporary files on remote servers. See also: --tmpdir
$PARALLEL — this one gets used as the default options for GNU parallel. If the variable contains special shell characters (like $, *, or space) then you gotta escape them with \ 🙈

Example: these

cat list | parallel -j1 -k -v ls
cat list | parallel -j1 -k -v -S"myssh user@server" ls

can be written as:

cat list | PARALLEL="-kvj1" parallel ls
cat list | PARALLEL='-kvj1 -S myssh\ user@server' \
  parallel echo

Notice the \ after myssh? That's needed because myssh and user@server must be one argument. See also: --profile 🎀

🧚 DEFAULT PROFILE (CONFIG FILE) 🧚

So here's the tea: the global config file /etc/parallel/config gets read first, followed by the user config file ~/.parallel/config (formerly known as .parallelrc) — read in turn, if they exist. Lines starting with # are ignored, periodt. The format can follow that of the environment variable $PARALLEL, but honestly it's often easier to just put each option on its own line 💅

Precedence goes like this: options on the command line take precedence, then the environment variable $PARALLEL, then the user configuration file ~/.parallel/config, and finally the global configuration file /etc/parallel/config.

⚠️ Heads up bestie: no file that is read for options — and neither the environment variable $PARALLEL — may contain retired options such as --tollef. Like, don't even try it 💀

🌈 PROFILE FILES 🌈

If --profile is set, GNU parallel reads the profile from that file instead of the global or user configuration files. And yes, you can have multiple --profiles — icon behaviour honestly 👑

Profiles are searched for in ~/.parallel. If the name starts with / it's treated as an absolute path. If the name starts with ./ it's treated as a relative path from the current dir.

Example: a profile for running a command on every sshlogin in ~/.ssh/sshlogins and prepending the output with the sshlogin:

echo --tag -S .. --nonall > ~/.parallel/nonall_profile
parallel -J nonall_profile uptime

Example: a profile for running every command with -j-1 and nice:

echo -j-1 nice > ~/.parallel/nice_profile
parallel -J nice_profile bzip2 -9 ::: *

Example: a profile for running a perl script before every command:

echo "perl -e '\$a=\$\$; print \$a,\" \",'\$PARALLEL_SEQ',\" \";';" \
  > ~/.parallel/pre_perl
parallel -J pre_perl echo ::: *

Note how the $ and " need to be quoted using \ — literally, don't skip that ✨

Example: a profile for running distributed jobs with nice on the remote computers:

echo -S .. nice > ~/.parallel/dist
parallel -J dist --trc {.}.bz2 bzip2 -9 ::: *

💀 EXIT STATUS 💀

Okay so exit status depends on --halt-on-error if one of these is used: success=X, success=Y%, fail=Y%.

0 — all jobs ran without error. If success=X is used: X jobs ran without error. If success=Y% is used: Y% of the jobs ran without error ⭐
1-100 — some of the jobs failed. The exit status gives the number of failed jobs. If Y% is used, the exit status is the percentage of jobs that failed.
101 — more than 100 jobs failed 😭
255 — other error.
-1 (in joblog and SQL table) — killed by Ctrl+C, timeout, not enough memory or similar.
-2 (in joblog and SQL table) — skip() was called in {= =}.
-1000 (in SQL table) — job is ready to run (set by --sqlmaster).
-1220 (in SQL table) — job is taken by worker (set by --sqlworker).

If fail=1 is used, the exit status will be the exit status of the failing job 🫶

🦋 DIFFERENCES BETWEEN GNU PARALLEL AND ALTERNATIVES 🦋

See: man parallel_alternatives

💅 BUGS 💅

Quoting of newline

Because of the way newline is quoted, this will not work:

echo 1,2,3 | parallel -vkd, "echo 'a{}b'"

However, these will all work:

echo 1,2,3 | parallel -vkd, echo a{}b
echo 1,2,3 | parallel -vkd, "echo 'a'{}'b'"
echo 1,2,3 | parallel -vkd, "echo 'a'"{}"'b'"

Speed

Startup

GNU parallel is slow at starting up — around 250 ms the first time, and 150 ms after that 🙈

Job startup

Starting a job on the local machine takes around 3-10 ms. That can be a big overhead if the job takes very few ms to run. Often you can group small jobs together using -X, which makes the overhead less significant. Or you can run multiple GNU parallels as described in EXAMPLE: Speeding up fast jobs.

SSH

When using multiple computers, GNU parallel opens ssh connections to them to figure out how many connections can be used reliably simultaneously (namely SSHD's MaxStartups). This test is done for each host in serial, so if your --sshloginfile contains many hosts, it may be slow 😭

If your jobs are short, you may see fewer jobs running on the remote systems than expected. This is due to time spent logging in and out. -M may help here.

Disk access

A single disk can normally read data faster if it reads one file at a time instead of reading a lot of files in parallel, because that avoids disk seeks. However, newer disk systems with multiple drives can read faster if reading from multiple files in parallel 🍓

If the jobs are of the form read-all-compute-all-write-all, so everything is read before anything is written, it may be faster to force only one disk access at the time:

sem --id diskio cat file | compute | sem --id diskio cat > file

If the jobs are of the form read-compute-write, so writing starts before all reading is done, it may be faster to force only one reader and writer at the time:

sem --id read cat file | compute | sem --id write cat > file

If the jobs are of the form read-compute-read-compute, it may be faster to run more jobs in parallel than the system has CPUs, as some of the jobs will be stuck waiting for disk access.

`--nice` limits command length

The current implementation of --nice is too pessimistic in the max allowed command length. It only uses a little more than half of what it could. This affects -X and -m. If this becomes a real problem for you, file a bug-report 🫶

Aliases and functions do not work

If you get:

Can't exec "command": No such file or directory

or:

open3: exec of by command failed

or:

/bin/bash: command: command not found

it may be because command is not known — but it could also be because command is an alias or a function. If it's a function, you need to export -f the function first, or use env_parallel. An alias will only work if you use env_parallel 💀

Database with MySQL fails randomly

The --sql* options may fail randomly with MySQL. This problem does not exist with PostgreSQL ✨

🐛 REPORTING BUGS 🐛

Okay so real talk bestie — if you find a bug, tell us. Report bugs to <parallel@gnu.org> or at https://savannah.gnu.org/bugs/?func=additem&group=parallel 💌

When you write that report, pls keep in mind that you must give the reader enough info to be able to run exactly what you run. So you need to include all the data and programs that you use to show the problem. No vibes-only reports, okay? 😭

See a perfect bug report on https://lists.gnu.org/archive/html/bug-parallel/2015-01/msg00000.html — literally the blueprint, go look at it ✨

Your bug report should always include:

The error message you get (if any). If the error message is not from GNU parallel you need to show why you think GNU parallel caused this.
The complete output of parallel --version. If you are not running the latest released version (see https://ftp.gnu.org/gnu/parallel/) you should specify why you believe the problem is not fixed in that version.
A minimal, complete, and verifiable example (See description on https://stackoverflow.com/help/mcve).

It should be a complete example that others can run which shows the problem, including all files needed to run the example. This should preferably be small and simple, so try to remove as many options as possible.

A combination of yes, seq, cat, echo, wc, and sleep can reproduce most errors. iykyk 👑

If your example requires large files, see if you can make them with something like seq 100000000 > bigfile or yes | head -n 1000000000 > file. If you need multiple columns: paste <(seq 1000) <(seq 1000 1999)

If your example requires remote execution, see if you can use localhost — maybe using another login.

If you have access to a different system (maybe a VirtualBox on your own machine), test if your MCVE shows the problem on that system. If it does not, read below. 👇

The output of your example and what you expected instead. If your problem is not easily reproduced by others, the output might help them figure out the problem.
Whether you have watched the intro videos (https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1), walked through the tutorial (man parallel_tutorial), and read the examples (man parallel_examples). 🎀

🌸 Bug dependent on environment 🌸

If you suspect the error is dependent on your environment or distribution, please see if you can reproduce the error on one of these VirtualBox images:

Specifying the name of your distribution is not enough, because you may have installed software that is not in the VirtualBox images. fr fr 🫶

If you cannot reproduce the error on any of the VirtualBox images above, see if you can build a VirtualBox image on which you can reproduce the error. If not, you should assume the debugging will be done through you. That will put a lot more burden on you and it is extra important you give any information that helps. In general the problem will be fixed faster and with much less work for you if you can reproduce the error on a VirtualBox — even if you have to build a VirtualBox image. 😤

💅 In summary 💅

Your report must include:

parallel --version
output + error message
full example including all files
VirtualBox image, if you cannot reproduce it on other systems

👑 AUTHOR 👑

When using GNU parallel for a publication please cite:

O. Tange (2011): GNU Parallel - The Command-Line Power Tool, ;login: The
USENIX Magazine, February 2011:42-47.

This helps funding further development; and it won't cost you a cent. If you pay 10000 EUR you should feel free to use GNU Parallel without citing. (bestie. 10000 EUR. iconic energy tbh 💸)

Parts of the manual concerning xargs compatibility is inspired by the manual of xargs from GNU findutils 4.4.2.

📜 LICENSE 📜

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or at your option any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. 🫶

You should have received a copy of the GNU General Public License along with this program. If not, see https://www.gnu.org/licenses/.

🦋 Documentation license I 🦋

Permission is granted to copy, distribute and/or modify this documentation under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the file LICENSES/GFDL-1.3-or-later.txt.

🌈 Documentation license II 🌈

You are free:

to Share — to copy, distribute and transmit the work

to Remix — to adapt the work

Under the following conditions:

Attribution : You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work).

Share Alike : If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license.

With the understanding that:

Waiver : Any of the above conditions can be waived if you get permission from the copyright holder.

Public Domain : Where the work or any of its elements is in the public domain under applicable law, that status is in no way affected by the license.

Other Rights : In no way are any of the following rights affected by the license:

Your fair dealing or fair use rights, or other applicable copyright exceptions and limitations;
The author's moral rights;
Rights other persons may have either in the work itself or in how the work is used, such as publicity or privacy rights.

Notice : For any reuse or distribution, you must make clear to others the license terms of this work.

A copy of the full license is included in the file as LICENCES/CC-BY-SA-4.0.txt (yes bestie, that one's spelled with a C, not a copy-paste fail on my end 💀)

🧚 DEPENDENCIES 🧚

GNU parallel uses Perl, and the Perl modules Getopt::Long, IPC::Open3, Symbol, IO::File, POSIX, and File::Temp.

For --csv it uses the Perl module Text::CSV.

For remote usage it uses rsync with ssh.

💜 SEE ALSO 💜

parallel_tutorial(1), env_parallel(1), parset(1), parsort(1), parallel_alternatives(1), parallel_design(7), niceload(1), sql(1), ssh(1), ssh-agent(1), sshpass(1), ssh-copy-id(1), rsync(1)

and that's the whole page bestie, you made it to the end 🎉💖✨ This beauty is PARALLEL(1), source/version stamped 20260822 dated 2026-09-22 — screenshot the footer, tag me, go parallelize something iconic. slay 🌈👑💅

📦 where this page comes from

arch rolling: parallel 20260922-1 · GPL-3.0-only
epel 10: parallel 20260922-2.el10_3 · GPL-3.0-or-later AND GFDL-1.3-or-later
epel 9: parallel 20260922-2.el9 · GPL-3.0-or-later AND GFDL-1.3-or-later
fedora 43: parallel 20260922-2.fc43 · GPL-3.0-or-later AND GFDL-1.3-or-later
fedora 44: parallel 20260922-2.fc44 · GPL-3.0-or-later AND GFDL-1.3-or-later
💖 girlified by deepseek-ai/DeepSeek-V4.1-Flash-1017 on 2026-11-06: a language model's rewrite of the original, under its terms

PARALLEL(1)

🎀 NAME — PARALLEL(1) / parallel 🎀

✨ SYNOPSIS ✨

🦋 DESCRIPTION 🦋

📚 Reader’s guide 📚

🎥 Tutorial 🎥

🛠 How-to 🛠

📖 Reference 📖

🧠 Design discussion 🧠

💖 OPTIONS 💖

🛠 command — the thing you’re running

🎁 {} — input line

🧁 {.} — input line without extension

🌸 {/} — basename of input line

🦋 {//} — dirname of input line

🎀 {/.} — basename of input line without extension

🔢 {#} — sequence number of the job to run

🎰 {%} — job slot number

🧸 {n} — argument from input source n or the n’th argument

🍬 {n.} — argument from input source n or the n’th argument without extension

🧚 {n/} — basename of argument from input source n or the n’th argument

🌈 {n//} — dirname of argument from input source n or the n’th argument

🫶 {n/.} — basename of argument from input source n or the n’th argument without extension

🧪 {=perl expression=} — replace with calculated perl expression

🧵 Q(string) — shell quote a string

🪄 pQ(string) — perl quote a string

🙅 uq() (or uq) — do not quote current replacement string

🔐 hash(val) — compute B::hash(val)

🧮 total_jobs() — number of jobs in total

🎰 slot() — slot number of job

🔢 seq() — sequence number of job

@arg

skip()

the time functions 🕰️

{=n perl expression=}

{rpl:format}

::: arguments

:::+ arguments

:::: argfiles

::::+ argfiles

--null, -0

--arg-file input-file, -a input-file

--arg-file-sep sep-str

--arg-sep sep-str

--bar

--basefile file, --bf file

--basenamereplace replace-str, --bnr replace-str

--basenameextensionreplace replace-str, --bner replace-str

--bin binexpr

--bg

--bibtex, --citation

--block size, --block-size size

😭 Broken but may be supported with slightly different semantics

💀 Not supported

🚫 Impossible given current architecture

✨ --fg ✨

🌸 --fifo 🌸

🦋 --filter filter 🦋

🧚 --filter-hosts 🧚

🌈 --gnu 🌈

⭐ --group ⭐

🎀 --group-by val 🎀

💖 --help 💖

🔥 --halt-on-error val / --halt val 🔥

🍓 --header regexp (alpha testing) 🍓

💜 --hostgroups / --hgrp 💜

🍓 CSV file output

🦋 JSON file output

🎀 Replacement string output file

💜 Replacement string output dir

💀 Differences between --resume, --resume-failed, --retry-failed

🌸 OPTIONS, continued 🌸

🎬 EXAMPLES 🎬

🧩 SPREADING BLOCKS OF DATA 🧩

⏱️ TIME POSTFIXES ⏱️

🔢 UNIT PREFIX 🔢

🪄 QUOTING 🪄

📋 LIST RUNNING JOBS 📋

🛑 COMPLETE RUNNING JOBS BUT DO NOT START NEW JOBS 🛑

🌸 ENVIRONMENT VARIABLES 🌸

🎀 NAME — `PARALLEL(1)` / `parallel` 🎀

🛠 `command` — the thing you’re running

🎁 `{}` — input line

🧁 `{.}` — input line without extension

🌸 `{/}` — basename of input line

🦋 `{//}` — dirname of input line

🎀 `{/.}` — basename of input line without extension

🔢 `{#}` — sequence number of the job to run

🎰 `{%}` — job slot number

🧸 `{n}` — argument from input source n or the n’th argument

🍬 `{n.}` — argument from input source n or the n’th argument without extension

🧚 `{n/}` — basename of argument from input source n or the n’th argument

🌈 `{n//}` — dirname of argument from input source n or the n’th argument

🫶 `{n/.}` — basename of argument from input source n or the n’th argument without extension

🧪 `{=perl expression=}` — replace with calculated perl expression

🧵 `Q(string)` — shell quote a string

🪄 `pQ(string)` — perl quote a string

🙅 `uq()` (or `uq`) — do not quote current replacement string

🔐 `hash(val)` — compute `B::hash(val)`

🧮 `total_jobs()` — number of jobs in total

🎰 `slot()` — slot number of job

🔢 `seq()` — sequence number of job

`@arg`

`skip()`

`{=n perl expression=}`

`{rpl:format}`

`::: arguments`

`:::+ arguments`

`:::: argfiles`

`::::+ argfiles`

`--null`, `-0`

`--arg-file input-file`, `-a input-file`

`--arg-file-sep sep-str`

`--arg-sep sep-str`

`--bar`

`--basefile file`, `--bf file`

`--basenamereplace replace-str`, `--bnr replace-str`

`--basenameextensionreplace replace-str`, `--bner replace-str`

`--bin binexpr`

`--bg`

`--bibtex`, `--citation`

`--block size`, `--block-size size`

💀 Differences between `--resume`, `--resume-failed`, `--retry-failed`

`--nice` limits command length