pipexec(1)

💖 pipexec — the manual page, bestie edition 💖

okay so this is pipexec(1), a User Commands page, and honestly? it's giving legend behaviour. sit down, grab a strawberry 🍓, and let me tell you about the girl who can build a whole directed graph of processes and pipes. yes. cycles. yes. we're doing this.

---

✨ NAME ✨

pipexec — create a directed graph of processes and pipes 🦋

---

✨ SYNOPSIS ✨

pipexec [OPTION]... [PROCESS DESCRIPTION]... [PIPE DESCRIPTION]...

---

✨ DESCRIPTION ✨

so like. pipexec builds an arbitrary network (a directed graph, no cap) of processes with pipes in between — and yes bestie, even cycles are possible. she literally overcomes the shortcomings of shells, which are typically only able to create non-cyclic trees. shells could NEVER 💅

pipexec also monitors all its child processes and is able to restart the whole network of processes and pipes if one crashes. so if you want to run a network of processes inside a SYSV-init or systemd configuration — she's your girl. main character energy fr fr 👑

---

✨ OPTIONS ✨

• -h — print help and version information. the "hi bestie what is this" button 🫶
• -l logfd — use the given file descriptor for text logging. if a 's' is specified, syslog is used. example: specifying '2' means log to stderr.
• -j logfd — use the given file descriptor for json logging. if a 's' is specified, syslog is used. example: specifying '2' means log to stderr. and because this is meant to be parsed by other programs, this is an official and supported interface, described in the JSON LOGGING chapter. it's giving contract 📜
• -p pidfile — with pipexec it is possible to handle pipes within SYSV-init scripts. in some environments (e.g. RHEL6, Debian7) the start and stop routines need a pid file. if this option is given, pipexec writes its own pid into the file shortly after start of pipexec.
• -k — if one sub-process (child) gets killed and this option is given, all other sub-processes are also killed. afterwards all processes are restarted. dramatic? yes. necessary? also yes 😭
• -s sleep_time — the time interval in seconds before a restart. this option makes only sense when also the '-k' option is specified. (note the typo in the original, she's iconic, we don't fix her 💀)

---

✨ BACKGROUND ✨

inside a shell it's possible to start processes and redirect the output to other processes.

example:

cat Chap1.txt Chap2.txt | grep bird | wc -l

three processes are created: the standard output (file descriptor, aka fd, 1) of the cat process is connected to the standard input (fd 0) of the grep command, and the standard output of the grep command is connected to the standard input (fd 0) of the wc process.

please note that the assignment between names and file descriptor numbers is pure historical and has no technical background. it's just vibes from 1970s, iykyk 🎀

example:

find / 1> >(grep .txt) 2> >(wc >/tmp/w.log)

in this more complex example, fd 1 of the find process is connected to fd 0 of grep and fd 2 is connected to fd 0 of wc.

the limitation of using this way of specifying processes and pipes is that it is not possible to have any cycles. it is impossible to e.g. pass a fd of wc either to grep or to find. tragic. heartbreaking. 💀

pipexec overcomes these limitations. she makes it possible to link any two arbitrary file descriptors in a set of processes. slay. ✨

---

✨ USAGE ✨

when building up a network of processes and pipes, there is the need to specify each element separately.

the processes will be the nodes in the network (directed graph), the connections of the file descriptors between two processes are the edges. each node (process) has a unique name assigned to it. this makes it possible to differentiate between using the same command more than once. no name collisions, we don't do that here 💅

the format of specifying a process is

[ NAME /path/to/command arg1 arg2 ... argN ]

the first parameter NAME must be a unique name. the second parameter must be the full path of the command to execute. please note that always the full path must be specified — there is no PATH environment variable handling (execv(2) is used internally to span new processes). the following parameters are the parameters passed to the command.

the whole definition must be enclosed in square brackets. the square brackets must be given separately — before and after them must be a space. yes, a space. respect the space. 🫶

the format of specifying a pipe between processes is

{NAME_1:FD1>NAME_2:FD2}

example:

{LS:1>GREP:0}

the names are the names of the processes, the numbers are the number of the file descriptor that should be used to build the pipe in between. when using pipexec from a shell (like bash) there is the need to escape the brackets or use quotation marks. (or, you know, suffer. your call 😌)

---

✨ JSON LOGGING ✨

pipexec can log in JSON format. this is an official supported interface which is defined in this chapter. this can be seen as stable as no changes will be made during small version upgrades. of course additions will be made also within minor upgrades, but this should be fine because of the underlying JSON format. she's flexible like that 🌸

each JSON log object (e.g. line) will be in a separate line.

the following keys are always present:

• timestamp — the timestamp is the time(0) (seconds since epoch) when the log in generated.
• pipexec_pid — the pid of the pipexec process.
• id — an id that specifies defined log objects.
• type — an indicator in which internal module this log was generated.
• serverity — one of debug, info, warning, error. (yes it's spelled serverity in the original, we keep it exactly, no cap ✋)
• message — a short message describing the event.

depending on the id there might be additional fields which are described in the next section.

🦋 the ids 🦋

• id = 0 — id of 0 is a special case: this is used for internal logs only. the logs are not documented and may change at any time. the information can be used to get an idea what currently happens in pipexec. it's giving backstage pass 🎀
• id = 1 — this gives information about the process ids (pids) of the commands. the field command contains the command (i.e. the part in the command before the colon). the field command_pid contains the pid of the command.
• id = 2 — this log message is emitted when a child (command) exits. it contains some information about the exit status and termination reason of the command. the field command_pid contains the pid of the command which just terminated. the field status is the value which is set by waitpid(2). normal_exit, child_status and child_signaled are WIFEXITED, WEXITSTATUS and WIFSIGNALED of the status respectively.

---

✨ RETURN ✨

pipexec returns 1 if any of the child processes fails, else 0 is returned. simple. honest. iconic. 👑

---

✨ EXAMPLES ✨

the shell command

cat Chap1.txt Chap2.txt | grep bird | wc -l

is equivalent to

pipexec [ CAT /bin/cat Chap1.txt Chap2.txt ] \
  [ GREP /usr/bin/grep bird ] [ WC /usr/bin/wc -l ] \
  "{CAT:1>GREP:0}" "{GREP:1>WC:0}"

the pipexec equivalent is longer and more complex in this example. but pipexec can build cycles that are impossible within a shell:

pipexec [ A /bin/cmd1 ] [ B /bin/cmd2 ] "{A:1>B:0}" "{B:1>A:0}"

when using json log, you get output like:

{"timestamp":1655706460,"pipexec_pid":42850,"id":1,"type":"exec","serverity":"info","message":"New child forked","command":"A","command_pid":"42851"}
{"timestamp":1655706886,"pipexec_pid":42869,"id":2,"type":"tracing","serverity":"info","message":"child exit","command_pid":"42870","status":"1","normal_exit":"1","child_status":"1","child_signaled":"0"}

for more examples see the ptee(1) and peet(1) man pages. 🫶

---

✨ SEE ALSO ✨

bash(1), ptee(1), peet(1), execv(2)

---

✨ AUTHOR ✨

written by Andreas Florath (andreas@florath.net) 🎀

---

✨ COPYRIGHT ✨

copyright © 2015,2022 by Andreas Florath (andreas@florath.net). license GPLv2+: GNU GPL version 2 or later <http://gnu.org/licenses/gpl.html>.

---

and that's the whole page bestie!! 🌈 this was User Commands, dated 2022-07-18, and it lives in section pipexec(1) of the manual. go build your little process graph, you absolute legend. mwah 💜💅✨