TAKTUK(1)

📦
version 3.7.8 · 6 versions
, in ubuntu resolute, ubuntu stonking

💖 TAKTUK(1) — TakTuk Deployment Engine 💖

okay bestie, sit down, hydrate, and let me tell you about TakTuk — it's a tool for large scale remote execution deployment, and honestly? it's giving main character energy in the cluster-administration world. 🌈

✨ NAME ✨

TakTuk — a tool for large scale remote execution deployment 🫶

🌸 SYNOPSIS 🌸

taktuk [-behinsvzMP] [-c connector] [-d limit] [-f filename] [-g
duration] [-l login] [-m hostname [-[ args ... -]]] [-o stream=template]
[-t timeout] [-u limit] [-w size] [-B parameter=expression] [-C
separators] [-E character] [-F filename] [-G hostname [-[ args ... -]]]
[-L hostname] [-I interpreter] [-O separators] [-R stream=filename] [-S
files] [-T command] [-V path ] [-W scheme] [commands ... ]

like, look at all those flags. she's busy. she's booked. 💅

🦋 DESCRIPTION 🦋

so, TakTuk is a tool for broadcasting the remote execution of one or more commands to a set of one or more distant machines. that's the whole vibe. 🎀

she combines local parallelization (using concurrent deployment processes) and work distribution (using an adaptive work-stealing algorithm) to achieve both scalability and efficiency. yes, both. she doesn't choose. she ate and left no crumbs. ✨

TakTuk is especially suited to interactive tasks involving several distant machines and parallel remote executions. think: clusters administration and parallel program debugging. that's her moment, that's her stage. 👑

TakTuk also provides a basic communication layer to programs it executes. this communication layer uses the communication infrastructure set up by TakTuk during its deployment. it's available both for the Perl and the C langages, and it's described in TakTuk(3) and taktukcomm(3) respectively. iykyk 📚

💅 OPTIONS 💅

⚠️ Caution bestie, this is the important part: in TakTuk, options are parsed in the order given on the command line. this means TakTuk is not POSIX compliant regarding options order. this matters a lot, because some of the options change the behavior of following ones (and only these ones — e.g. -l applies to following -m options). so order is everything, no cap. 🔥

the defaults settings of TakTuk can be obtained by using the --print-defaults option. 🧚

the following options are given by category in alphabetical order. periodt.

🚀 DEPLOYMENT OPTIONS 🚀

  • -b, --begin-group — begins a new deployment group. a deployment group acts both as a context for most options (options set within a group have no effect outside the group) and as an isolation for deployment tasks scheduling (deployment tasks within a group cannot be stolen by nodes deployed in other groups). she's contained, she's protected, she's her own little world. 🫧
  • -c connector, --connector connector — defines the connector command used to contact the following remote machines. 🔌
  • -d limit, --dynamic limit — turns dynamic mode (work stealing) on or off for the deployment of all the following remote machines specifications. if limit is positive or null, TakTuk uses the dynamic mode with its value as a maximal arity (0 = no maximal arity). a negative value for limit turns dynamic mode off.
⚠️ Warning: currently it is a bad idea to use several -d options with positive values on the same command line. don't do it bestie, i'm begging 😭
  • -e, --end-group — ends a deployment group. bye girl, it was fun 💜
  • -f filename, --machines-file filename — filename is the name of a file that contains remote machines names (equivalent to several -m options). in each line of this file, the first word is considered as a machine name and the remaining as comments. 📄
  • -l login, --login login — sets the login name for the following hosts. this assumes that the connector command accepts -l option (this is the case for ssh, the default). 🪪
  • -m hostname, --machine hostname — hostname is the name of a remote machine on which TakTuk has to be deployed. hostname might contain list of values specified inside brackets and exclusion lists (see "HOSTNAMES SPECIFICATION"). in such case, the depolyment will be performed on all the matching hosts. this option can be followed by any number of arguments enclosed between -[ and -] which will be transmitted as arguments of the remote TakTuk that will execute on all the target hosts. 🖥️✨
  • -s, --self-propagate — propagates the TakTuk executable through the following connectors and thus eliminates the need for a TakTuk installation on remote machines. she travels. she's everywhere. 🦋
  • -z, --dont-self-propagate — cancels a preceding -s option. never mind, she stays home 🙈
  • -F filename, --args-file filename — filename is the name of a file that contains additional options for TakTuk. the parsing of the file is done when encountering this option. 📁
  • -G hostname, --gateway hostname — same as -m hostname, except that the deployed node(s) are considered as forward-only nodes: they will not be numbered and will not be targeted by subsequent diffusion commands. as an exception, execution of TakTuk commands on those nodes is still possible by giving them as deployment arguments of the -G option. they're the middle men, the pass-through girls, the gateways 💫
  • -I command, --perl-interpreter command — command is the name of the command to use as a Perl interpreter when auto propagating or when using the taktuk_perl command. 🐪
  • -L hostname, --localhost hostname — changes the name of localhost as viewed by TakTuk. it's still her, just rebranded 🏠
  • -S files, --send-files files — gives to TakTuk a list of files to be sent upon successful remote connection.
⚠️ use with caution as it will slow down deployment (the files are transferred before the end of the deployment).

the intended use is NOT to transfer application data files (rather use the put command to transfer such files). this option should be used to transfer files that are necessary to perform further connections from the remote node. files is a comma separated list of pairs local_file:remote_file. regarding the transfer itself, it behaves as the put command. 📦

  • -T command, --taktuk-command command — command is the name of the TakTuk command (default is $0, the name used to launch TakTuk). this is the command used once connected to initiate the remote TakTuk engine. note that the effect of this switch is void when using the -s option. 🔧
  • -V list, --path-value list — set the PATH environment variable to the given list (as usual, a colon separated directories list) on remote hosts to which TakTuk connects. this is performed very early and is taken into account even for taktuk or perl invocations at connection time. 🌱

🎀 COMMAND LINE PARSING OPTIONS 🎀

  • -C separators set, --command-separator separators set — replaces the default set of characters that separates successive TakTuk commands. ✂️
  • -E character, --escape-character character — defines an escape character that can be used to protect any following character from TakTuk interpretation. this protection applies nearly to the whole TakTuk command line. 🛡️
  • -O separators set, --option-separator separators set — replaces the default set of characters that separates successive words when reading options and commands from a file (using the -F switch). 📖

more parts coming bestie, don't go anywhere 💖

✨ I/O OPTIONS ✨

  • -o stream=template
  • --output-template stream=template — okay so this sets an output template spec for one of the output streams, picked by the name you give it. And bestie, listen: if you only give a stream name and no template, it disables that stream entirely. Iconic little on/off switch tbh. The available streams are:
    • "connector" (errors from the connector command) — this stream holds, in $line, whatever a connector command printed to stderr. The fields that matter here include $host, $peer and the others listed below. 💅
    • "error", "output" (error and output of remotely executed commands) — these streams hold, in $line, what an executed command printed to stdout or stderr (respectively, obviously). Relevant fields include $host, $command, $pid and the others listed below.
    • "info" (general information such as help summary or version) — this stream holds, in $line, what taktuk itself printed in response to commands like "help" or "version".
    • "message" (messages from running processes) — every output of this stream has, in $line, a message sent with target "output" (through TakTuk out-of-band communications facilities) by some processus started using TakTuk commands. Relevant fields for this stream include $from and $to and the others listed below.
    • "state" (internal state of TakTuk instance) — this stream holds, in $line, the state change code matching some TakTuk events. The codes go like this:
  0 - TakTuk is ready
  1 - TakTuk is numbered
  2 - TakTuk terminated
  3 - connection failed
  4 - connection initialized
  5 - connection lost
  6 - command started
  7 - command failed
  8 - command terminated
  9 - numbering update failed
 10 - pipe input started
 11 - pipe input failed
 12 - pipe input terminated
 13 - file reception started
 14 - file reception failed
 15 - file reception terminated
 16 - file send failed
 17 - Invalid target
 18 - No target
 19 - Message delivered
 20 - Invalid destination
 21 - Destination not available anymore
 22 - Wait complete
 23 - Wait reduce complete

The function event_msg($) can be used in the template to translate this code into a string that describes the event. Relevant fields include $host, $position, $rank, $count and the others listed below. 🦋

  • "status" (commands exit status) — this stream holds, in $line, the return code of executed commands. Relevant fields include $host, $pid and the others listed below.
  • "taktuk" (internal messages, warnings and errors) — this stream holds, in $line, error, warning and debug messages outputed by TakTuk itself. Relevant fields include $line_number, $package and the others listed below.

So a template is a Perl expression that should evaluate to the string you eventually see on screen. Within a specification, some variables might be used depending on the concerned stream:

  • $command (not for "taktuk" nor "info") — the command line execution that generated output.
  • $count — the total number of TakTuk instances.
  • $eol — an empty string or a newline depending on what character ended command output.
  • $filename ("taktuk" only) — the file in which a TakTuk diagnostic function has been called.
  • $from ("message" only) — the logical number of the TakTuk instance from which the message has been sent.
  • $host — the name of the host executing the current TakTuk instance
  • $init_date ("connector" and "state" only) — the time in seconds since Jan. 1, 1970 at which the remote TakTuk instance completed its initialization (including ssh connection time and self propagation if any).
  • $level ("taktuk" only) — the level of the diagnostic function called.
  • $level_name ("taktuk" only) — the name associated with the level of the diagnostic function called.
  • $line — the raw data outputed by a command or passed to a diagnostic function, any trailing newline removed (see $eol).
  • $line_number ("taktuk" only) — the line number at which a TakTuk diagnostic function has been called.
  • $package ("taktuk" only) — the package in which a TakTuk diagnostic function has been called.
  • $peer ("connector" and "state" only) — the remote machine for which a connector command outputed information.
  • $peer_position ("connector" and "state" only) — position on the command line of the remote host. This position is expressed as a path in the hierarchy of hosts expressed on the command line (dot separated position number at each level of imbrication).
  • $peers_given ("state" only) — a space separated list of remote nodes that has been given to the remote peer for deployment (whatever their deployment state). Each remote node is specified as a colon separated couple made of its hostname and its position on the command line. This variable has only a meaningful value when the event outputed to the stream "state" is a connection loss (event number 5).
  • $pid (not for "taktuk" nor "info") — the pid of the command line executed (that produced output).
  • $position — position on the command line of the current host. This position is expressed as a path in the hierarchy of hosts expressed on the command line (dot separated position number at each level of imbrication).
  • $rank — the logical number of TakTuk instance in which output is collected.
  • $reply_date ("connector" and "state" only) — the time in seconds since Jan. 1, 1970 at which the connector command replied to the local TakTuk instance (note that this does not include the initialization exange occurring before the remote TakTuk is considered as functional).
  • $start_date ("output", "error", "connector" and "status") — the time in seconds since Jan. 1, 1970 at which the command started (taken just after the call to fork()).
  • $init_date ("connector" and "state" only) — the time in seconds since Jan. 1, 1970 at which the remote TakTuk completed its initialization.
  • $stop_date ("connector" and "status" only) — the time in seconds since Jan. 1, 1970 at which the command terminated (after the closing of its stdout).
  • $to ("message" only) — the logical number (or set of such numbers) of the TakTuk instance to which the message has been sent.
  • $type — the name of the output stream (connector, error, output, ...).
  • $user_scalar — a scalar, global to the package, initially initialized to undef, that the user is free to use for any purpose.

At the end of the day, the specification is evaluated for each line of the concerned stream and the result printed on the root node. Note that the newline has to be added explicitly as "\n" in the template if needed. Lowkey the most important sentence here ngl — no newline, no line breaks, your whole output becomes one long cursed string. 😭

WARNING: take care of your specification, if the Perl syntax is not correct lots of awfull compilation error messages will be displayed and TakTuk execution will fail. Fr fr, it's giving wall-of-red-text energy. Protect your peace and double check those quotes. 💀
  • -R stream=filename
  • --output-redirect stream=filename — redirects a given stream output to a given file (located at the root node). Notice that the given filename comply with Perl open syntax: for instance, one might redirect to a file descriptor X using the name ">&=X". See -o option for more details about streams output. 🎀

🌸 PERFORMANCE TUNING OPTIONS 🌸

okay bestie, this is the section where you make TakTuk go zoom 💨 these are the knobs and dials that change how fast stuff happens, and some of them are lowkey delicate so read carefully, mwah 💖

  • -B parameter=expression, --worksteal-behavior parameter=expression — ⚠️ caution: this feature should be considered as experimental and could be removed from future releases. fr fr, don't build your whole life around it. What it does is finely tune the behavior of the work-stealing algorithm by setting some of its parameters to Perl expressions. Those Perl expressions can use the variables $last_given and $available, which are respectively equal to the number of tasks given to a stealer last time it stole, and the total number of remaining deployment tasks. The parameters you can set are:
    • initial — the number of tasks given to a remote TakTuk instance the first time it sends a steal request to the local host. first impression energy ✨
    • growth — the number of tasks given to a remote TakTuk instance each time it performs a steal to the local host after the first time.
    • limit — an upper bound to the value resulting from the evaluation of "growth". no infinite glow-up allowed 🚫
  • -g duration, --time-granularity duration — sets to duration (in seconds) the maximal interval between timeouts checks. usually checks are made more often, bestie: at each message received, a timeouts check is made. so this is the worst case gap 🕐
  • -n, --no-numbering — disables TakTuk logical numbering, and the variables TAKTUK_COUNT and TAKTUK_RANK are not defined in executed commands. The upside? it removes the global synchronization happening at the end of the deployment, which makes the deployment more efficient. speed with a side of danger, we love a chaotic queen 💅
⚠️ WARNING: use this option only before any remote node specification (-m or -f) otherwise you might get serious synchronization issues in TakTuk. And using TakTuk point-to-point communication along with this option will fail and produce TakTuk warnings. like, it's giving "I ignored the warning and now everything is on fire" 😭
  • -t timeout, --timeout timeout — sets the timeout (in seconds) for following connectors. 0 means no timeout. heads up: this option overrides the timeout of the connector command (ssh for instance). When the delay expires, the connector command is simply killed. no gentle goodbyes, just gone 💀
  • -u limit, --cache-limit limit — limits the number of memory bytes TakTuk might use as an internal write buffer. For most users, setting such a limit is a bad idea: in some rare situations it might result in a deadlock in TakTuk, caused by parallel blocking writes between an application and TakTuk itself. Setting this option to a negative value disables the limit, which is the default. 🙈

That said though — if you use TakTuk to transfer large files, because of I/O bandwidth disparities in various parts of a system, TakTuk memory use might grow too large and performance can be severely degraded when the system starts swapping. In those situations, limiting the size of the internal cache will keep TakTuk in main memory and preserve the performance. so it's a "situational slay", not an everyday slay 🌈

  • -w size, --window size — sets initial window to the given size (which equals pipeline width).
  • -W scheme, --window-adaptation scheme — sets the windows adaptation scheme to number (default is 0, where 0: no adaptation, 1: implementation in progress). yeah 1 is literally unfinished, we're all works in progress bestie 🎀

🦋 MISCELLANEOUS OPTIONS 🦋

the little grab-bag of extras, iykyk ✨

  • -M, --my — makes the next option local (not inherited). This means the next option is set only for the local host and is not propagated to children TakTuk instances. Has no effect if the next option is already a local one.
  • -h, --help — prints a short description of TakTuk on the screen. the "help me" button, we've all been there 🫶
  • -i, --interactive — forces TakTuk interactive mode even after some batch commands given on the command line.
  • -v, --version — prints the TakTuk version. flex the number 📌
  • -P, --print-defaults — prints the defaults settings used by TakTuk, including environment variables taken into account.

💜 INTERNAL OPTIONS 💜

ngl, these options are not useful for most users. They are used either internally by TakTuk itself or for development purposes. you can peek but it's giving backstage pass 🎭

  • -p packagename, --print-package packagename — prints the content of package packagename extracted from the taktuk code in execution.
  • -r, --not-root — current TakTuk instance is not the root node. This is an internal switch used by spawned TakTuk instances. This is usually not useful for most users.
  • -D scope=level, --debug scope=level — sets debugging level (1..4) for scope. The scope might be a TakTuk package name or "default", and the lowest the level, the more verbose the output. 🔍

👑 COMMANDS 👑

okay this is the main character section, periodt. After the options parsing, TakTuk expects some commands either on the remaining of the command line (that's batch mode) or on the standard input (that's interactive mode). These commands are actions to be performed by TakTuk using the logical network infrastructure set up during the deployment. ✨

By default, commands might be separated by ; or newlines. And for all the commands, any non ambiguous prefix can be used instead of their full name. In interactive mode, TakTuk has support for "readline" (history, command line editing) if installed on your system. ⭐

When TakTuk commands accept arguments, they should be enclosed into matching delimiters — those are indicated by * below. In other words, * might be replaced either by any non alphanumeric character or by a pair of matching braces, brackets or parenthesis. These delimiters must be separated from their content (using the options separator). If the argument contains a closing delimiter preceded by a separator, then it is probably a good idea to escape it (see the -E option) or to protect the whole arguments string if given on the command line. 💅

Taktuk understands the following commands:

  • [ set specification ] command — Sends the execution of command to all the peers belonging to the given set. For more details about the set specification, see section "SET AND TARGETS SPECIFICATION".
  • broadcast command — Broadcasts the execution of command on all the remote peers, not including the node initiating the broadcast. it's giving "everyone but me" 📣
  • downcast command — Spreads the execution of command on all the children of the node initiating the downcast, not including itself.
  • exec parameters * command line * — Executes the "command line" on the local machine. Inputs/outputs of the execution are multiplexed and forwarded to the TakTuk root. This command accepts optional parameters that enable the attachment of callbacks triggered by timeouts to the execution of the command. See section "EXEC PARAMETERS" for more details.
  • get * src * * dest * — Copies (a) remote source(s) (present on the remote node(s)) to a destination on the node executing this command. The type of source and destination is quite similar to those accepted by the "cp -r" command (that is file or directories). This command also performs some kind of magic with its parameters: shell environment variables present in the source or destination names are replaced by their respective value on the remote and local nodes. In addition, the variables $host, $rank and $position of the remote node (which are the same as in the -o option) can also be used in both names. ✨
  • help — Prints a very concise TakTuk help.
  • input [ target target ] * data * and input [ target target ] data * data * — Sends the given data as input to commands in execution on the local machine as specified by the given target. If no target is given, the data is sent to all commands. For more details about targets, see "SET AND TARGETS SPECIFICATION".
  • input [ target target ] close — Closes the inputs descriptor of executing commands on the local machine as specified by the given target. If no target is given, the inputs of all commands are closed.
  • input [ target target ] file * filename * — Sends the content of a file (which must be local to the node executing the input file) as input to commands in execution on the local machine as specified by the given target. If no target is given, the data is sent to all commands.
⚠️ WARNING: this command is not atomic. If you manage somehow to initiate a input file command from two different TakTuk instances, data will probably be interleaved. In this case you should synchronize the two instances. This is not required when spreading files only from the root node. 😭
  • input [ target target ] line * data * — Sends the given data with an additional newline as input to commands in execution on the local machine as specified by the given target. If no target is given, the data is sent to all commands.
  • input [ target target ] pipe * filename * — Treat the parameter as a file (which must be local to the node executing the input pipe) which behavior is the same as a pipe: data can arrive continuously. TakTuk adds it as one of its input channels and sends data as input to commands when available. If given, the target specifies to which command the data are to be sent, otherwise it is sent to all commands. 🍓
  • kill [ target target ] <signal> — Sends the given signal to local commands processes groups. If no signal is given, sends a TERM signal (signal 15). If given, the target specifies to which command the signal is to be sent, otherwise it is sent to all commands. For more details about targets, see "SET AND TARGETS SPECIFICATION". 💀
  • message [ target target ] * data * and message [ target target ] data * data * — Sends the given data as a message to commands in execution on the local machine as specified by the given target (see TakTuk(3) and taktukcomm(3) for more details about messages). If no target is given, the data is sent to the first command performing a receive and not already targeted by another message. For more details about targets, see "SET AND TARGETS SPECIFICATION".
  • message [ target target ] file * filename * — Sends the content of a file (which must be local to the node executing the message file) as a message to commands in execution on the local machine as specified by the given target (see TakTuk(3) and taktukcomm(3) for more details about messages). If no target is given, the data is sent to the first command performing a receive and not already targeted by another message.
⚠️ WARNING: this command is not atomic either. If you manage somehow to initiate a message file command from two different TakTuk instances, data will probably be interleaved. In this case you should synchronize the two instances. This is not required when spreading files only from the root node. 🧚
  • message [ target target ] line * data * — drops your data into a message with an extra newline glued on the end, and ships it off to the commands running on the local machine, exactly as the target says. (Wanna know the ✨lore✨ of messages? TakTuk(3) and taktukcomm(3) got you.) No target? Then it just yeets the data at the first command that's performing a receive and that hasn't already been claimed by some other message. first come first served bestie 💅
  • message [ target target ] pipe * data * — here data gets treated as a filename, and that file has to live on the node that's running the input pipe. Its whole vibe is being a pipe: data can show up continuously, nonstop, forever. TakTuk just adds it to its roster of input channels and fires data off as a message to commands the second it's available. (Again, TakTuk(3) and taktukcomm(3) for the message lore.) Give it a target and that's who receives the data; skip it and it goes to the first command performing a receive that isn't already targeted by another message. 🫶
  • network
  • network state — prints out the current TakTuk deployment tree, like a little family tree but make it infrastructure 🌳. The numbers in parenthesis match up with the peer rank in the logical TakTuk numbering, plus the peer's ready state. And if the deployment isn't complete yet? The tree will be showing you "connecting ..." leaves. patience, queen 👑
  • network cancel — cancels all the ongoing connections. Which means: numbering of TakTuk instances and execution of TakTuk commands may immediately kick off. no takebacks.
  • network renumber — completely recomputes the logical numbering of TakTuk instances. This is especially useful when you're adding new nodes to the deployed network after the initial numbering already happened. glow up, get renumbered ✨
  • network update — updates the logical numbering of TakTuk instances without touching numbers that already exist. Heads up though: this does not always succeed, because TakTuk numbers things with a depth first scheme. For every node of the tree it can't update while keeping the TakTuk numbering scheme, it sends an event 9 in the stream 'state'. Same use case as above: super handy when adding new nodes to the deployed network after initial numbering. 🦋
  • option name * value *
  • option * line * — two flavors, one command. First form: change a single option, given by its name. Second form: parse an additional options line. Either way it happens on the node(s) executing this command. (More deets over in TakTuk options.) Oh and in the first form, name can be either the short or the long version, your choice bestie.
WARNING: new nodes added to the TakTuk network with this command are not numbered. You're gonna need to use network renumbering or update afterwards to get the TakTuk logical numbering. don't skip this step 😭
  • put * src * * dest * — copies a local source (as in, sitting on the node executing this command) over to a destination on all the remote nodes given as a prefix of the command. The kinds of source and destination it accepts are basically the same as what cp -r takes — so, files or directories. And it does a little bit of ✨magic✨ with its parameters: shell environment variables hanging out in the source or destination names get swapped for their respective values on the local and remote nodes. On top of that, the variables $host, $rank and $position of the local node — same ones as in the -o option — can be used in both names too. iconic behavior, honestly 👑
  • synchronize command — forces the given command to sit there and wait for the completion of deployment, nodes numbering, and any previous commands, before it executes. This is the move for global commands like "broadcast", which does not wait for nodes numbering. Meanwhile multicast commands (the ones that use a set specification as a prefix) already wait for the nodes numbering and the completion of previous commands. In doubt? Just always synchronize before a "broadcast", "downcast" or "wait reduce". trust 💜
  • taktuk_perl * arguments * — forks a perl interpreter on the local node, exactly as if you'd run the command exec perl arguments. The difference? This interpreter already got fetched with the "taktuk" package, which comes with point-to-point communication routines — "TakTuk::send" and "TakTuk::recv", see taktukcomm(3). 🧚
WARNING: the parser that analyses this command's arguments has limitations, so you have to give arguments (even if they're empty) and you have to use '--' if you give any option to the perl interpreter — even if options are the only thing you're giving it. fr fr.
  • version — prints the TakTuk version. that's it, that's the command 🌸
  • quit — quits the TakTuk engine and shuts down the logical communication network that got established during deployment. bye bestie 💀
  • wait target target — waits — as in, suspends the execution of the commands that come after it — in the local TakTuk instance, for the completion of some local process that depends on the given target. That target can be any target specification except "output". Wanna know more about targets? Peep "SET AND TARGETS SPECIFICATION". 🎀
  • wait reduce target target — can only be used in the root node, no exceptions. It waits (again: suspends execution of the following commands) for the completion of some processes across all the deployed TakTuk instances, depending on the given target. And yes, this target can be any target specification except "output". Here's the tea though: unlike the plain wait commands, this one gets synchronized the same way as "broadcast". So if you're mixing it with non-broadcasted commands, you should explicitly synchronize it — otherwise it will run before them. 🙈

✨ EXEC PARAMETERS ✨

The TakTuk command exec is flexible like that — it accepts optional parameters. You use them to specify a target id for the command, or to attach actions that get triggered by timeouts onto commands execution. An exec command accepts any number of parameters, and they're interpreted from left to right with this syntax:

  • target value — attaches the (numerical) value as the target id of the executed command. This target id is what any subsequent command that makes use of targets will use — stuff like "input", "message", "kill" or "wait".
Caution: this value overwrites any target id that could have been automatically assigned by TakTuk. So yeah — it is not recommended to mix explicitly assigned target ids with automatically assigned ones. pick a lane 💅
  • timeout value — creates a brand new timeout specification. When the duration expressed by value runs out, it triggers its attached callbacks. If it has no attached callback, it sends a TERM signal to the command. And if the command execution finishes before the timeout duration ends, the timeout gets canceled. 🔥
  • kill value — attaches a callback to the last defined timeout. That callback sends a signal — the number of which is the given value — to the timeouted command.
  • action command — attaches a callback to the last defined timeout. That callback executes the given TakTuk command. If it executes some other command, it can use the environment variable "TAKTUK_PID", which holds the pid of the timeouted command. The given TakTuk command can be any valid TakTuk command — just no command separator allowed. 🌈

Quick note, bestie: each timeout can have any number of attached callbacks. They get processed in the exact order they're given as parameters. no queue-jumping 💖

🌈 ENVIRONMENT 🌈

Variables that change TakTuk default behavior

Some of TakTuk's default settings can be changed on some host using environment variables. These settings get propagated just like other options, and they're overridden by propagated settings and command line options.

To change some default setting, use the variable TAKTUK_NAME — where NAME is the name of the according long option in upper case, with dashes replaced by underscores. For an option that takes a complex value (like "--debug"), just add an underscore and the field you want to change, in upper case, at the end of the name. Running taktuk --print-defaults will hand you examples of the names used to change default settings. And note: defining a default setting in the environment that TakTuk doesn't use has no effect. zero. nada. 🫠

You can also change some TakTuk default settings locally, without propagating the change through the deployment tree. For that, use the variable TAKTUK_MY_NAME, where name is defined the same way as above. And as before, these local settings get overridden by propagated settings and command line options.

Variables set by TakTuk in remotely executed commands

TakTuk sets the following environment variables for all the commands it executes:

  • TAKTUK_CONTROL_READ, TAKTUK_CONTROL_WRITE — file descriptors, used internally by TakTuk.
  • TAKTUK_COUNT — the total number of successfully deployed TakTuk instances.
  • TAKTUK_HOSTNAME — local node hostname as given to TakTuk (on the command line).
  • TAKTUK_PIDS — list of pids (separated by spaces) of commands executed by the local TakTuk instance.
  • TAKTUK_TARGET — target number of the currently executing process (see "SET AND TARGETS SPECIFICATION" for more details).
  • TAKTUK_POSITION — host position on the command line.
  • TAKTUK_RANK — the logical rank of the local instance.
  • TAKTUK_FATHER — the logical rank of the taktuk instance that spawned this one. 👑
  • TAKTUK_CHILD_MIN — the logical rank of the lowest numbered descendent from the current taktuk instance (-1 if the local TakTuk instance has no children).
  • TAKTUK_CHILD_MAX — the logical rank of the highest numbered descendent from the current taktuk instance (-1 if the local TakTuk instance has no children).

HOSTNAMES SPECIFICATION 🌈

okay bestie so when you hand TakTuk a hostname it's not always just a boring lil machine name — it can be a whole complex hosts-lists specification 💖

In its general form, an hostname is made of an host set plus an optional exclusion set, and those two are separated by a slash. Each of those sets is a comma separated list of host templates. Each one of those templates is made of a constant part (the characters that live outside brackets) and optional range parts (the characters that live inside brackets). Each range part is a comma separated list of intervals or single values. Each interval is made of two single values separated by a dash. And yeah — this is true for all hostnames given to TakTuk, both with -m and -f. 💅

In other words, these expressions are all valid host specifications:

node1
node[19]
node[1-3]
node[1-3],otherhost/node2
node[1-3,5]part[a-b]/node[3-5]parta,node1partb

and they respectively expand to:

node1
node19
node1 node2 node3
node1 node3 otherhost
node1parta node2parta node2partb node3partb node5partb
⚠️ not me having to say this but these lists are NOT regular expressions, fr fr. node[19] is node19, it is not node1, node2, ...., node9. Periodt. 🫶

Intervals are implemented using the perl magical auto increment feature, so you can totally use alphanumeric values as interval bounds (go peek at the perl documentation, operator ++ for the limitations of this auto increment). 🦋

SET AND TARGETS SPECIFICATION 🎀

The TakTuk command line and the TakTuk::send routine both accept a set specification as destination host(s). A set specification is made of interval specifications separated by slashes. An interval specification is either a single number, two numbers separated by a dash, or a single number followed by a plus symbol (that last case matches the interval that goes from that number all the way to the highest numbered TakTuk destination). And obvi the two numbers specifying an interval must be given in increasing order. 👑

The remote peers included in a set specification are all the peers whose logical number belongs to at least one interval of the set. Here's some examples bestie:

1

the peer numbered 1

2-7

the peers numbered 2,3,4,5,6 and 7

2-4/1/10

the peers numbered 1,2,3,4 an 10

3+

the peers from 3 to the highest numbered

5+/1

the peers from 5 to the highest numbered and the peer 1

The target number is a number assigned by TakTuk to all processes it executes (whether they were successfully started or not, using exec or taktuk_perl commands). By default this number starts from 0 and goes to the total number of processes that have been executed since TakTuk launch minus one. Target processes of a TakTuk::send or a TakTuk command can be expressed with the same syntax as in the case of sets. 🧚

Plus TakTuk understands several special targets:

  • all — targets all processes: meaning the command gets applied to all executing local processes (message or input data are duplicated and sent to all of them). This is the default for the input and kill commands. 👑
  • any — targets the first eligible process. In the case of a message, this is the first process that issues a TakTuk::recv and that is not already the target of another message. This is the default for the message command. ✨
  • output — targets the output stream message rather than a process. 💫

EXAMPLES 🌸

The following examples illustrate the basic use of TakTuk on a few machines and the use of developer options. And just so you know — TakTuk is designed to scale to much more peers than the number involved in these examples. it's giving main character energy, no cap. 💅

Basic usage 💖

simple deployment

the simplest way to use TakTuk is to make it selfpropagate with option -s. In this case, the basic remote execution of hostname on the host toto.nowhere.com can be written:

taktuk -s -m toto.nowhere.com broadcast exec [ hostname ]

In this example, -s asks TakTuk to propagate its own code on remote hosts. It can be removed by installing the taktuk executable on toto.nowhere.com. By the following we will assume that TakTuk is installed on all the remote hosts. 🫶

The -m toto.nowhere.com describes the set of remote hosts to be contacted by TakTuk and broadcast exec [ hostname ] is a command that will be executed by the TakTuk interpreter.

This example can be written in many other ways. In interactive mode, the same execution might become:

taktuk -m toto.nowhere.com

here TakTuk is blocked waiting for commands from stdin. Thus, we just have to type:

broadcast exec { hostname }
Ctrl-D

here you can notice that parameters to the exec TakTuk command (as all commands parameters) can be enclosed in any reasonable pair of delimiters. We might also write the list of hosts involved in the command in a file machine that contains:

toto.nowhere.com

and the TakTuk command becomes:

taktuk -f machine broadcast exec - hostname -

We could also use another file options that contains:

-f machine

and use it as the options line given to TakTuk:

taktuk -F options broadcast exec \( hostname \)

Finally, everything could be stored in a last file command_line that contains:

-f machine broadcast exec = hostname =

and the following command achieve the same result:

taktuk -F command_line

All of these variants have the same effect: they execute hostname on toto.nowhere.com and the output of the program is forwarded to the localhost. In this case:

toto.nowhere.com: hostname: somepid: output > toto.nowhere.com

parameters braces

notice that braces for command parameters must be separated from their content. Thus, they can contain other braces as long as no single closing brace is part of the content:

taktuk -m localhost broadcast exec [ 'if [ $RANDOM -gt 10000 ];then echo greater;else echo lower;fi' ]

In this example, quotes are necessary to prevent the shell from interpreting the $ and ; characters and to prevent the closing brace for if to be considered as closing the exec command. In this case the variable will be interpolated only on remote hosts. This same example can also be expressed using shortcuts and intercative mode:

taktuk -m localhost -E%

then type:

b e [ if [ $RANDOM -gt 10000 %];then echo greater;else echo lower;fi ]
Ctrl-D

Notice the closing bracket used in the test that should not be interpreted as the closing bracket for exec arguments. In such case, a simpler solution is probably to use another kind of braces

taktuk -m localhost

and then:

b e { if [ $RANDOM -gt 10000 ];then echo greater;else echo lower;fi }
Ctrl-D

Usually, if you want to be safe, you can quote all commands parameters. Nevertheless, notice that parameters should not be quoted in interactive mode as input lines are not interpreted by the shell. 🙈

exec parameters

commands executed by TakTuk can be timeouted using exec parameters. For instance if you want to execute some command and send it a TERM signal after two seconds, just type:

taktuk -m localhost broadcast exec timeout 2 [ sleep 10 ]

the callback executed when a timeout occurs can also be something else than a TERM signal. This can be another signal (KILL for instance):

taktuk -m localhost broadcast exec timeout 2 kill 9 [ sleep 10 ]

or any valid TakTuk command:

taktuk -m localhost broadcast exec timeout 2 action broadcast exec [ echo hello ] [ sleep 10 ]

or even several timeouts and several callbacks:

taktuk -m localhost b e t 2 a e [ echo hello ] k 30 t 10 k 9 [ sleep 5 ]

in this last example, the command sleep 5 is executed by TakTuk. After 2 seconds, the first timeout will be triggered, it will execute the command echo hello and send a USR1 signal to the first command (sleep 5). The second timeout is set to 10 seconds. Thus, it will never occur as the sleep 5 command will be terminated before its expiration. 🔥

topology

Usually, TakTuk deploys itself using a relatively flat tree because its default window size is quite large (10 simultaneous ongoing connections). Using a smaller window size will result in a deeper tree although it also depends on the local load of the deployment nodes. You can use the TakTuk network state command to print the tree constructed by TakTuk. 🌳

Notice that it is usually a bad idea to use a too large window as it results in too much local load and bad distribution of work (something like 10 is often sufficient).

You can also force TakTuk to use more specific topologies. For instance, to execute echo $$ using a flat-tree as deployment topology, just disable work-stealing in TakTuk:

taktuk -d -1 -m host1 -m host2 -m host3 broadcast exec [ 'echo $$' ]

and to use a chain-like topology, either encode the topology in arguments structure:

taktuk -m host1 -[ -m host2 -[ -m host3 -] -] broadcast exec [ 'echo $$' ]

or limits the arity of the dynamic tree to 1:

taktuk -d 1 -m host1 -m host2 -m host3 broadcast exec [ 'echo $$' ]

Finally, the default will use a dynamically constructed topology:

taktuk -d 0 -m host1 -m host2 -m host3 broadcast exec [ 'echo $$' ]

lightweight grid deployment

On a lightweight grid, because of locality issues (ldap cache, network topology, ...) it seems interesting to separate the deployment of each subcluster. To do this, it is possible to enclose the nodes of each cluster into a separate deployment group:

taktuk -b -m node1.cluster1 -m node2.cluster1 -m node3.cluster1 -m node4.cluster1 -e -b -m node1.cluster2 -m node2.cluster2 -m node3.cluster2 -m node4.cluster2 -e broadcast exec [ hostname ]

This command has the effect of deploying TakTuk on two clusters (cluster 1 and 2) made of four nodes (node 1 to 4), preventing deployed nodes from one cluster to be used to deploy nodes from the other cluster. Finally, once the deployment is complete, it executes the command hostname on all these nodes. ✨

executing a distinct command on each host

each distinct host can be given its own command using arguments:

./taktuk -m host1 -[ exec [ hostname ] -] -m host2 -[ exec [ id ] -] -m host3 -[ exec [ 'echo $TAKTUK_RANK; ls' ] -] quit

but this could also be given using set specification (in this case logical number are used for hosts):

or in interactive mode:

./taktuk -m host1 -m host3 -m host8
1 exec [ hostname ]
2 exec [ id ]
3 exec [ echo $TAKTUK_RANK; ls ]
Ctrl-D
Nevertheless keep in mind that in general these logical numbers do not match the position of hosts on the command line. 💀 like, do not assume, bestie.

script diffusion 🌸

spreading and executing a perl script named "essai.pl" on three hosts, knowing that "taktuk" and "essai.pl" are only present on the root node is straightforward with TakTuk:

taktuk -s -m host1 -m host2 -m host3
broadcast exec [ perl -- - ]
broadcast input file [ essai.pl ]
broadcast input close
Ctrl-D

files diffusion and collection 🎀

since version 3.4, files transfer is supported directly by TakTuk. Notice that this transfer uses the TakTuk network (usually a tree composed of ssh connections by default). This network is not efficient for the transfer of large files (both topologically and from a protocol point of view), so keep this in mind when using this feature.

copying a file named "message.txt" to the "/tmp" directory of each remote host is thus as easy as:

taktuk -s -m host1 -m host2 -m host3
broadcast put [ message.txt ] [ /tmp ]
Ctrl-D

but the older method still works (and does almost the same as the previous command):

taktuk -s -m host1 -m host2 -m host3
broadcast exec [ cat - >/tmp/message.txt ]
broadcast input file [ message.txt ]
broadcast input close
Ctrl-D

although it requires to be more careful about shell interpretation when typing everything directly on the command line:

taktuk -s -m host1 -m host2 -m host3  broadcast exec [ 'cat - >/tmp/message.txt' ]\;broadcast input file [ message.txt ]

notice in this latter command that the "input close" is not necessary as TakTuk closes inputs of all spawned commands when quitting. ✨

the "get" command also makes possible things that were previously very difficult in TakTuk, files collecting. The following command gets the file "/tmp/message.txt" from each remote host and copies it locally to "message-number.txt" where "number" is the logical rank of the source node:

taktuk -s -m host1 -m host2 -m host3
broadcast get [ /tmp/message.txt ] [ message-$rank.txt ]
Ctrl-D

finally, it seems important to mention that "put/get" commands can copy directories and keep files permissions unchanged. 👑

communication 💜

TakTuk can also be used to establish a point-to-point communication. Assume the file "communication.pl" contains the following Perl script:

my $rank = TakTuk::get('rank');
my $count = TakTuk::get('count');
if ($rank == 1)
  {
    print "I'm process 1\n";
    if ($count > 1)
      {
        TakTuk::send(to=>2, body=>"Hello world");
      }
  }
elsif ($rank == 2)
  {
    print "I'm process 2\n";
    my ($from, $message) = TakTuk::recv();
    print "Process $to received $message from $from\n";
  }

then the execution of the following command:

taktuk -m localhost -m localhost broadcast taktuk_perl [ - ]\;broadcast input file [ communication.pl ]

would produce an output similar to:

Astaroth.local: taktuk_perl: 3523: output > I'm process 2
Astaroth.local: taktuk_perl: 3523: output > Process 2 received Hello world from 1
Astaroth.local: taktuk_perl: 3523: status > 0
Astaroth.local: taktuk_perl: 3524: output > I'm process 1
Astaroth.local: taktuk_perl: 3524: status > 0

if the file "communication.pl" was placed in the login directory of the user, this could have also been executed by the more simple:

taktuk -m localhost -m localhost broadcast taktuk_perl [ communication.pl ]

output templates and redirections 🦋

TakTuk is also capable of making each site echo its rank without command status information:

taktuk -o status -m host1 -m host2 broadcast exec [ 'echo $TAKTUK_RANK' ]

or removing the prompt before each line of output from commands:

taktuk -o output='"$line\n"' -m host1 -m host2 broadcast exec [ 'echo $TAKTUK_RANK' ]

or even changing the prompt to make it display only the stream type:

taktuk -o default='"$type > $line\n"' -m host1 -m host2 broadcast exec [ 'echo $TAKTUK_RANK' ]

and it also possible to redirect the status to file descriptor 2 only for the second host:

taktuk -m host1 -R status=2 -m host2 broadcast exec [ 'echo $TAKTUK_RANK' ]

and so on... 🌈

Developer usage 🧚

debugging TakTuk 🐛

debugging in TakTuk is made using "debug", "warning" and "error" functions of the package "diagnostic". These routines, depending on the debugging level of the package in which they are called, produce an output that is propagated up to the root node and printed on screen.

By default the debugging level of packages is set to 2 (everything is printed out except "debug" messages). It might be changed for each package using the -D option. For instance the following code executes "true" on "toto.nowhere.com" and prints out every bit of internal messaging:

taktuk -D default=1 -m toto.nowhere.com broadcast exec [ true ]

but one could have executed the same command keeping only messages from the "scheduler" package:

taktuk -D scheduler=1 -m toto.nowhere.com broadcast exec [ true ]

or ensuring an execution exempted of any warning or error messages:

taktuk -D default=4 -m toto.nowhere.com broadcast exec [ true ]

internal messages server 👀

the internal message server used in TakTuk for the management of logical network construction, commands execution and I/O forwarding can be exposed using the -r option:

taktuk -r
Notice that in this mode the behavior of TakTuk can seem very cryptic. This is not intended for ordinary users. 🙈

💀 BUGS 💀

The development of TakTuk is still in progress, so there are propably numbers of bugs. For now, the following characteristics (some of them are not really bugs) have been identified :

hanged commands 😭

currently TakTuk do not quit (even if asked to do so) if some local commands do not terminate upon the closing of their standard input. Interrupting TakTuk actually terminate the engine but do not kill these commands. We still have to decide if this is the appropriate behavior.

command order 🔥

broadcasts and multicast (set specifications) are not synchronized the same way in TakTuk. Thus, broadcast commands given after multicast commands might be executed before these last ones. Use the "synchronize" command to avoid this.

fork failures 💅

this is not really a bug : when a machine is highly loaded, fork may fail. If this happen to a local command, a distributed application may deadlock, because the taktuk instances numbering is correct but some instance didn't fork the local command. The simplest fix is to use timeouts on TakTuk::recv().

You might also want to have a look at:

http://taktuk.gforge.inria.fr/Bugs.txt

where all the temporary bugs are listed version by version. 👑

✨ SEE ALSO ✨

taktukcomm(3), TakTuk(3), TakTuk::Pilot(3)

💖 AUTHOR 💖

The original concept of TakTuk has been proposed by Cyrille Martin in his PhD thesis. People involved in this work include Jacques Briat, Olivier Richard, Thierry Gautier and Guillaume Huard.

The author of the version 3 (perl version) and current maintainer of the package is Guillaume Huard.

TakTuk is provided under the terms of the GNU General Public License version 2 or later.

okay bestie that's the whole page, you made it, and honestly? iconic. 🫶 This whole thing comes to you from perl v5.40.1, dated 2025-09-30, and it's page TAKTUK(1) — so go forth and broadcast responsibly, mwah 💅💖

📦 where this page comes from