Reliable(3pm)

📦
version 1.16 · 3 versions
, in ubuntu noble, ubuntu resolute, ubuntu stonking

💖 Reliable(3pm) — the manual, but make it make sense 💖

From the User Contributed Perl Documentation collection, bestie. This is Proc::Reliable — yes that's the whole name, it's giving confident. Version 3pm, the perl module page. Sit down, get a snack, we're doing this. 🍓

✨ NAME ✨

Proc::Reliable -- run external processes reliably, with so many options. Like, an unreasonable amount. We love her for it. 👑

✨ SYNOPSIS ✨

Okay so first things first, you gotta bring her into the group chat:

use Proc::Reliable;

Create a new process object

$myproc = Proc::Reliable->new();

Run a subprocess and collect its output

$output = $myproc->run("/bin/ls -l");

Check for problems (the vibe check, essentially)

if($myproc->status()) {
  print("problem!\n");
}

Run another subprocess, keeping stdout and stderr separated. Also, send the subprocess some data on stdin:

$msg = "Hello World\n");
$p->want_single_list(0);
$stdout = $p->run("/usr/bin/fastmail - foo@bar.com", $msg);
if($p->status()) {
  print("problem: ", $p->stderr(), "\n");
}

Another way to get output

($stdout, $stderr, $status, $msg) = $p->run("/bin/ls -l");
Not me pointing out that stray ) in the $msg line up there — it's in the original, bestie, so it stays. Copy it exactly and don't come for me. 💀

✨ OPTIONS ✨

🌈 Run Modes

Five whole ways to launch a thing. Pick your fighter:

$p->run("shell-command-line");  # Launch a shell process
$p->run("cmdline", "data");     # Launch a shell process with stdin data
$p->run(["cmd", "arg1", ...]);  # Bypass shell processing of arguments
$p->run(sub { ... });           # Launch a perl subroutine
$p->run(\&subroutine);          # Launch a perl subroutine

🎀 Option settings below represent defaults

Everything in this block is the default, okay? So if you don't touch it, this is what you get. No cap.

$p->num_tries(1);           # execute the program only once
$p->time_per_try(60);       # time per try 60 sec
$p->maxtime(60);            # set overall timeout
$p->time_btw_tries(5);      # time between tries 5 sec
$p->want_single_list();     # return STDOUT and STDERR together
$p->accept_no_error();      # Re-try if any STDERR output
$p->pattern_stdout($pat);   # require STDOUT to match regex $pat
$p->pattern_stderr($pat);   # require STDERR to match regex $pat
$p->allow_shell(1);         # allowed to use shell for operation
$p->child_exit_time(1.0);   # timeout for child to exit after it closes stdout
$p->sigterm_exit_time(0.5); # timeout for child to exit after sigterm
$p->sigkill_exit_time(0.5); # timeout for child to exit after sigkill
$p->input_chunking(0);      # feed stdin data line-by-line to subprocess
$p->stdin_error_ok(0);      # ok if child exits without reading all stdin
$p->stdout_cb(undef);       # callback function for line-by-line stdout
$p->stderr_cb(undef);       # callback function for line-by-line stderr

📬 Getting output

This is the part where she tells you what happened. Main character recap energy:

$out = $p->stdout();        # stdout produced by last run()
$err = $p->stderr();        # stderr produced by last run()
$stat = $p->status();       # exit code produced by last run()
$msg = $p->msg();           # module messages produced by last run()

🐛 Debug

Proc::Reliable::debug($level);         # Turn debug on

✨ OVERVIEW ✨

Okay so real talk. Proc::Reliable is a class for simple, reliable and configurable subprocess execution in perl. In particular — and this is the moment, this is her whole thing — it is especially useful for managing the execution of 'problem' programs, the ones likely to fail, hang, or otherwise behave in an unruly manner. We've all got one. 🙈

She includes all the functionality of the backticks operator and the system() functions, plus many common uses of fork() and exec(), open2() and open3(). And she incorporates a number of options, including:

  • sending data to the subprocess on STDIN
  • collecting STDOUT and STDERR separately or together
  • killing hung processes
  • timouts
  • automatic retries
Yes, "timouts". That's what it says. We keep it. It's canon. 🫶

✨ DESCRIPTION ✨

A new process object is created by:

$myproc = Proc::Reliable->new();

The default will run a subprocess only once with a 60-second timeout. Either shell-like command lines or references to perl subroutines can be specified for launching a process in background. A simple list process, for example, can be started via the shell as:

$out = $myproc->run("ls");

To separate stdout, stderr, and exit status:

($out, $err, $status, $msg) = $myproc->run("ls");

The output data is also stored within the $myproc object for later retrieval. You can also run a perl subroutine in a subprocess, with:

$myproc->run(sub { return <*>; });

The run Method will try to run the named process. If the process times out (after time_per_try seconds) or has an error defined as unacceptable and you would like to re-run it, you can use the num_tries option. Use the time_btw_tries option to set the number of seconds between runs. This can repeat until maxtime seconds have elapsed. 🔁

When using num_tries, the user can specify what constitutes an unacceptable error of STDOUT or STDERR output -- i.e. demanding a retry. One common shorthand is to have the run method retry if there is any return from STDERR:

$myproc->accept_no_error();    # Re-try if any STDERR
$myproc->pattern_stdout($pat); # require STDOUT to match regex $pat
$myproc->pattern_stderr($pat); # require STDERR to match regex $pat

Subprocess completion is detected when the process closes all filehandles. The process must then exit before child_exit_time expires, or it will be killed. If the subprocess does not exit, it is sent a TERM signal unless sigterm_exit_time is 0. Then if it does not exit before sigterm_exit_time expires, it is sent a KILL signal unless sigkill_exit_time is 0. Then if it does not exit before sigkill_exit_time expires an error is generated. Waiting is done in 0.01 second increments. ⏱️

Big important one: Proc::Reliable is not MT-Safe due to signals usage. That's the actual technical caveat, do not ignore her, she's serious. 💅

✨ to be continued, bestie — there's a part 2 ✨

✨ METHODS ✨

okay bestie, here's the whole menu of methods you get to play with 💅

  • new (Constructor) — spin up a brand new instance of this class by writing either
$proc = new Proc::Reliable;   or   $proc = Proc::Reliable->new();

and yes, new will happily take any valid configuration options you throw at it:

$proc = Proc::Reliable->new('maxtime' => 200, 'num_tries' => 3);
  • run — kick off a brand new process and scoop up the standard output and standard error, each through its own separate pipe. 🫶
 $out = $proc->run("program-name");
($out, $err, $status, $msg) = $proc->run("program-name");

by default, if you only take a single return value, stdout and stderr get combined into one single stream and handed back. but if you grab all 4 return values, stdout and stderr come separated, and the program exit status comes along too. $msg holds messages from Proc::Reliable whenever errors pop off. call want_single_list(1) to force stdout and stderr to merge, and want_single_list(0) to force them apart. the results from run() get stashed as member data too, like this:

 $proc->want_single_list(0);
 $proc->run("program");
 if($proc->status) {
   print($proc->stderr);
   exit;
 }
 else {
   print($proc->stdout);
 }

program exit status comes back in the same format as exec(): bits 0-7 get set if the program exited from a signal, bits 8-15 are the exit status on a normal program exit. periodt.

you can also set up callbacks to run a function of your choice as each line of stdout and stderr is produced by the child process, using the stdout_cb and stderr_cb options. 🎀

there are a bunch of other options too. you can even feed the forked program data on stdin via a second argument to run():

 $myinput = "hello\ntest\n";
 $output = $proc->run("program-name", $myinput);

the first option to run() supports three forms: 1) a string containing the command string to execute — this incurs shell parsing. 2) an arrayref containing a split command string to execute — this bypasses shell parsing. 3) a coderef to a perl function. the first two options get executed via exec(), so the specifics of incurring shell parsing are exactly the same.

the second option to run() supports two forms: 1) a string containing data to feed on stdin 2) a stringref pointing to data to feed on stdin.

you can start execution of an independent Perl function (like "eval", except with timeout, retries, etc.). simply hand over the function reference like

 $output = $proc->run(\&perl_function);

or supply an unnamed subroutine:

 $output = $proc->run( sub { sleep(1) } );

the run Method returns after the function finishes, one way or another. 💀

  • debug — switches debug messages on and off — Proc::Reliable::debug(1) flips them on, Proc::Reliable::debug(0) keeps Proc::Reliable quiet. 🤫
  • maxtime — return or set the maximum time in seconds per run method call. Default is 300 seconds (i.e. 5 minutes).
  • num_tries — return or set the maximum number of tries the run method will attempt an operation if there are unallowed errors. Default is 5.
  • time_per_try — return or set the maximum time in seconds for each attempt which run makes of an operation. multiple tries in case of error can go longer than this. Default is 30 seconds.
  • time_btw_tries — return or set the time in seconds between attempted operations in case of unacceptable error. Default is 5 seconds.
  • child_exit_time — when the subprocess closes stdout, it is assumed to have completed normal operation. it's expected to exit within the amount of time specified. if it does not exit, it will be killed (with SIGTERM). this option can be disabled by setting to '0'. values are in seconds, with a resolution of 0.01.
  • sigterm_exit_time — if the time_per_try or max_time has been exceeded, or if child_exit_time action has not succeeded, the subprocess will be killed with SIGTERM. this option specifies the amount of time to allow the process to exit after closing stdout. this option can be disabled by setting to '0'. values are in seconds, with a resolution of 0.01.
  • sigkill_exit_time — similar to sigterm_exit_time, but a SIGKILL is sent instead of a SIGTERM. when both options are enabled, the SIGTERM is sent first and SIGKILL is then sent after the specified time only if the subprocess is still alive. this option can be disabled by setting to '0'. values are in seconds, with a resolution of 0.01.
  • input_chunking — if data is being written to the subprocess on stdin, this option will cause the module to split() the input data at linefeeds, and only feed the subprocess a line at a time. this option typically would be used when the subprocess is an application with a command prompt and does not work properly when all the data is fed on stdin at once. the module will feed the subprocess one line of data on stdin, and will then wait until some data is produced by the subprocess on stdout or stderr. it will then feed the next line of data on stdin. 🧚
  • stdout_cb — set up a callback function to get stdout data from the child line-by-line. the function you supply will be called whenever the child prints a line onto stdout. this is the only way to get output from the child while it is still running — the normal method will give you all the output at once after the child exits. 👑
  • stderr_cb — similar to stdout_cb for stderr data.

🌸 REQUIREMENTS 🌸

I recommend using at least perl 5.003. (yes, we're keeping it vintage, it's giving retro chic 💜)

👑 AUTHORS 👑

Proc::Reliable by Dan Goldwater <dgold at zblob dot com>

Based on Proc::Short, written by John Hanju Kim <jhkim@fnal.gov>.

Contributions by Stephen Cope and Jason Robertson.

Copyright 2001 by Dan Goldwater, all rights reserved. Copyright 1999 by John Hanju Kim, all rights reserved.

This program is free software, you can redistribute it and/or modify it under the same terms as Perl itself.

okay that's the whole page bestie!! 🫶 this cutie was served up as perl v5.36.0, dated 2022-10-22, and it's Reliable(3pm) — go forth and run some processes, you absolute icon 💅✨🌈

📦 where this page comes from