nettee(1)

📦 version 0.3.5, in nixos 26.05, nixpkgs unstable

🌸 nettee(1) — the ✨nettee✨ manual, bestie edition 🎀

okay so grab a snack because we're about to learn nettee, from the nettee Manual. it's giving network chaos in the best way possible 💅

✨ NAME ✨

nettee — literally a network "tee" program, no cap 🫖

✨ SYNOPSIS ✨

nettee [options]

✨ DESCRIPTION ✨

okay so here's the vibe: nettee passes a data stream to one or more child nodes using a "daisy chain" method. on each node, nettee may also direct the stream to a file or pipe. it lets absolutely huge amounts of data get distributed quickly to multiple nodes on a switched network, at a rate limited only by the network bandwidth 🔥. the distribution tree is typically linear for each network switch, but it may branch when nodes contain multiple interfaces. for maximum throughput, only one instance of nettee should utilize each network interface. also — important — data streams are not encrypted, so confidential or sensitive information should not be transmitted using this tool on open networks 🙈. like, fr fr, don't do it.

when nettee starts, it waits for a connection from the single parent node before attempting to connect to its child node(s). consequently, nettee may be started on the nodes in any order (by a script, rsh, ssh, PXE boot, and so forth — iconic flexibility honestly ✨). typically only the root node will be set to log messages, so that the progress of the transfer may be monitored. transmission errors are detected by comparing the total number of bytes read by each child node with the number of bytes transmitted to that child 👑

nettee may also be used to perform processing of the data streams on each node. when used in this manner, the data may flow either up or down the distribution tree 🦋. additionally, on each node data may be consumed, created, or redirected. these actions are carried out by a child process on each node, said child process being a program which understands a simple command language used to communicate back and forth with the nettee parent process. this method may be used, for instance, to merge N presorted data files into one large sorted data stream. obsessed with the versatility tbh 💖

🌈 Distribution tree

the distribution tree is formed by a group of nodes running nettee. the topology of this tree is constrained in that it must possess one root node, may branch at each node to no more than 8 child nodes, and may not contain any loops. the root node has no parent node and at least one child node. internal nodes have both a parent and at least one child node. tip nodes connect to a parent but no child nodes. the topology is specified through the use of the command line options -next and -root. the direction of data flow within the tree is defined by the value of the -flow option 🌊.

for maximum throughput on a switched network, generally each node should connect to as many child nodes as it possesses network interfaces. if a group of systems on a switched network contain only one network interface each, the most efficient distribution tree is a simple linear chain ⛓️. specifying N child nodes on a single network interface will reduce the throughput to each of those nodes to 1/N of the interface's maximum throughput. this will also occur if nettee is used on a shared (as opposed to switched) network. math is mathing, bestie 😭

💀 Error Handling

by default, severe errors cause the entire chain to abort. by utilizing the -colrf, -colwf, -conrf, or -conwf, options, nettee may be instructed to do its best to continue processing in the event of certain read or write failures of the data stream. note that failures which occur while the distribution chain is forming are still fatal events 🚨. the -connf option allows the program to use a truncated distribution tree if errors are encountered during tree formation. in conjunction with the -next option supplied with alternate targets in each hostlist, -connf allows the program to build alternative distribution trees in order to connect around failed nodes. if the node above the failed node is allowed to emit messages and errors (for instance: -v 5) messages similar to these will be sent to the log destination (-log):

Failures detected in child 0 [node34]: NWF
Failures detected in child 1 [node35]: NONE
Failures detected in chain: NWF

the first type of message describes the failures that were detected in the named child node, that is, those named in the -next option. the second message describes failures that were detected anywhere further on in the chain. the error codes currently defined are:

  • NONE — no errors 🌸
  • NWF — network write failure
  • LWF — local write failure
  • NRF — network read failure
  • LRF — local read failure
  • BBC — child returned incorrect byte count
  • BSTAT — child returned unknown or bad status
  • NNF — could not connect to (one or more) child nodes 💔

🎀 Exit Status

nettee will normally emit an EXIT_SUCCESS status (0 on Unix). this is true even if the errors were detected and handled in the node itself or in a child node. nettee will emit an EXIT_FAILURE status if it was forced to close by an unhandled event such as a timeout, write failure, or unexpected socket closure 😤.

🦋 Version compatibility

  • Version 0.3.1 added the command line options -burst, and -sync ✨
  • Version 0.3.1 with the command line options -flow push may be employed in a distribution tree with older 0.1.x versions of nettee.
  • Version 0.2 added the command line options -root, -buf_*, -conrf, and -conlrf, and dropped the option -in in nettee.

✨ OPTIONS ✨

  • -branch — the distribution tree is branched 🌳. if the first host in a hostlist cannot be reached and -branch is present, all of the remaining hosts in that list will become child nodes. this provides one level of failover for branching distribution tree topologies. should a child node not be able to connect, the parent node can attempt instead to connect to all of the missing child node's children, so long as these are also listed in the hostlist. if -branch is not present, nettee assumes that the distribution tree is a linear chain. for a linear topology the hostlist will be traversed only until a connection is made to one host.
  • -buf_size N — set the size in bytes of the transfer buffers 🧺. increasing the size of the buffer may be useful in acheiving maximum throughput when the nodes in the distribution tree are not always working at 100% capacity, for instance, when some other task is running. the default is 1048576.
  • -buf_minfree N — set the minimum free storage in each buffer before more data will be read in. the default is 128.
  • -buf_net_minwrite N
  • -buf_lcl_minwrite N — set the minimum stored data in a buffer before data will be written out. if N is larger than 1, data may remain in the output buffer until more data is added to the buffer or it is flushed at program exit. for this reason such a value may not be combined with the -stm option ⚠️. additionally, if an output buffer fills to the point where less than N bytes remain free, the input(s) will not be read until such time as this buffer has emptied sufficiently. otherwise, if an output buffer is very nearly full and is only emptying slowly, a very high rate of cycling may result, where only small numbers of bytes at a time are read and written. the values for buffers associated with network and local IO may be specified separately using the net and lcl forms of these commands. the default is 1.
  • -buf_net_maxread N
  • -buf_net_maxwrite N
  • -buf_lcl_maxread N
  • -buf_lcl_maxwrite N — set the maximum number of bytes to read/write from a buffer at once, before other IO operations are allowed to proceed 🚦. the maxread and maxwrite options may be used to force better interleaving of IO operations. if the buffers are large and these values are set to a very high value, a fast data source/sink may completely fill/empty an entire buffer at once, never allowing time for other IO operations to run. for instance, if input is from a one gigabyte file, and both the buffer size and the maxread value are also one gigabyte, nettee will move data continuously from the input file to the input buffer until it fills — without pausing to do anything else 😳. the values for buffers associated with network and local IO may be specified separately using the net and lcl forms of these commands. the default is 131072.
  • -burst — in this mode network and local transfers are completely separated 💥. first the network moves data down the chain until either the end of input data or the buffer fills. at this point all pending network writes are completed, then the buffer of data is written to the local output. then the cycle repeats. this mode overrides the values of buf_minfree, buf_lcl_minwrite, and buf_lcl_maxwritee and sets them to 1, 1, and 2147483647, respectively. may not be combined with -stm or -process 🚫.
  • -cmd COMMAND — mandatory option when either -in socket or -out socket are used. COMMAND is the program or script that writes into, or reads from, the socket 📡. since only a single COMMAND may be specified, socket may not be applied to both -in and -out at the same time. additionally, the -cmd and -process options may not be used together. when -cmd is used with -in socket, a child process running COMMAND reads data from a disk or other device and writes the resulting data stream to stdout. when -cmd is used with -out socket, a child process running COMMAND reads the datastream from stdin and writes the processed data to a disk or other device. typically the COMMAND string invokes tar or some other archiving program. in some instances using sockets and -cmd will be faster than using the same command in a pipe due to the larger buffer size used for the socket 🏃‍♀️💨. run nettee -hexamples to see a usage example.
  • -colrf — Continue on Local Read Failure 💜. normally the failure of a read of the data stream from a local input will be fatal and the entire distribution tree will collapse immediately. when -colrf is set and a local read failure occurs on a node, that node will continue to relay data along the chain. the top node will emit an error message when this occurs so that a subsequent analysis with other tools may locate the node(s) which failed. there may be instances where this is useful, but in general continuing on this type of failure will result in expected information not being present in the data stream 🫠.
  • -colwf — Continue on Local Write Failure. Okay so normally, if writing the data stream to your local output fails, it's game over — fatal error, the entire distribution tree collapses immediately. 😭 (This is usually what happens when data gets written to disk and a partition fills up, or there's an ownership problem. A full disk failure might start out looking exactly like this, but it often goes on to crash the node, which also gives you a network write failure. Combo deal, unfortunately.) But when -colwf is set and a local write failure happens on a node, that node keeps relaying data along the tree 🌈. The node that failed didn't process the data stream correctly locally, but every other node is completely unaffected. The top node emits an error message when this happens, so you can do a follow-up analysis with other tools and track down which node(s) fumbled. 🕵️
  • -connf WAIT — Continue on Next Node Failure. Give each node in a -next hostlist WAIT seconds to connect ⏰. After that, each successive host in the hostlist gets WAIT seconds to join, and if none of them make it, no data gets sent to any of those hosts, periodt. If -connf isn't specified (or the wait time is set to zero seconds), the program just waits forever for a connection to the first node in each hostlist 💀. When -connf is present, these three hostlists are all equivalent: node1, [node1], [node1,_EOC_]. In every case it either connects to node1, or that hostlist results in no connection to another machine at all. And if -root is also specified and no child nodes connect, the program exits with an error — because at that point there's no distribution tree left to distribute anything. 🫠
  • -conrf — Continue on Network Read Failure. Normally, if reading the data stream from the previous node fails, that's fatal and the whole distribution tree collapses instantly 😭 (usually because a node crashed while nettee was running). When -conrf is set and a network read failure hits a node — which means the previous node is gone — that node keeps processing the data stream locally but makes no further attempts to transfer data from the previous node in the tree. So the transfer can still finish on the section of the tree below the failed node ✨. Ngl though, it's unlikely you'll find many situations where this switch is actually useful, since the data from the other side of the failed node is just… gone. 🙈
  • -conwf — Continue on Network Write Failure. Same deal, other direction: normally a failed write of the data stream to the next node is fatal and the whole tree collapses immediately 😭 (again, usually a node crashing mid-run). When -conwf is set and a network write failure happens — meaning the next node has failed — the node keeps processing the data stream locally but stops trying to transfer data to the next node in the tree. This lets the transfer complete normally on the tree down to the node just above the failed one 🫶. The top node emits an error message when this happens so a later analysis with other tools can locate the node(s) that failed.
  • -flow DIR — Sets the direction data flows through the distribution tree 🦋. DIR is either push (data flows away from the root node) or pull (data flows toward the root node). Default is push. The pull option may only be used together with the -process option. On top of that, -flow defines the default settings for -in and -out:

For -flow push those defaults are:

  • root node: -in - -out none
  • internal node(s): -in none -out -
  • tip node(s): -in none -out -

For -flow pull those defaults are:

  • root node: -in none -out -
  • internal node(s): -in none -out -
  • tip node(s): -in - -out none
  • -h — Print nettee help information. 🎀
  • -hexamples — Print nettee examples. Obsessed, honestly.
  • -herrors — Print nettee error status codes.
  • -hprocess — Print verbose info on the -process option 💖. Describes the interface between nettee and the CMD started by the -process option. See also the nettee_cmd(3) man page.
  • -i — Print version, license, and copyright information. 📜
  • -in SRC1(,SRC2,SRC3...) — Reads local data from SRC, which can be one of four values:
    • none — no local input
    • - — reads from stdin
    • socket — read the output of a command from a socket
    • filename — reads from a file

You can only use more than one SRC if -process is also specified ✨. When you do use more than one SRC, only one of them may be from stdin, and none may be from a socket or none. If there's no -in option present, the default value comes from whatever -flow says. 💅

  • -log LDST — Errors and messages get written to LDST, which can be one of two values:
    • - — writes to stderr
    • filename — writes to a file

If no -log option is present, the program writes messages to stderr. 🫶

  • -name STRING — Specify the node name used in messages (<=127 characters) 🏷️. If you don't supply it, the values of the environmental variables MYHOSTNAME and HOSTNAME get checked first, and if those aren't defined, the result of a gethostname() call is used.
  • -next HOSTLISTS — Writes data to child node destination[s] hostlist1(,hostlist2(,hostlist3(...))), where the hostlist entries are separated by commas or spaces 🌸. A hostlist is either a single hostname, or a comma separated list of hostnames enclosed in square brackets. Example: node1,[node2,node3],[node4,node5,node6],node7. The bracketed form lets you do automatic failover when unreachable nodes show up, and when -connf is specified. How the alternatives in the hostlist get handled is set by the -branch parameter.

For a linear topology, the first hostname in the list is tried, then the next, and so on until one host connects or the list runs out. For a branching topology, if the first host doesn't connect, connections get attempted to all of the remaining entries in the list 🔥. There may be 1-8 hostnames in each hostlist. The number of hostlists determines the topology of the distribution chain. Use a linear distribution chain (a single hostlist) when all nodes share a single network switch. Use a forked distribution chain (multiple hostlists) when nodes are connected to two or more network switches, or for large numbers of nodes, to reduce the number of nodes from the root to the tips of the distribution tree. 🌈

The End of Chain condition (no child write) is indicated by a hostname value of ., "", or _EOC_. An End of Chain condition is also indicated by the absence of a -next option. 💅

  • -out DST — Writes data locally to DST, which can be one of four values:
    • none — write nothing locally
    • - — write to stdout
    • socket — write to a command through a socket
    • filename — write to a file

If no -out option is present, the default is set by the -flow option. 🦋

  • -p, -port PORT — First of two consecutive ports used for communication 📡. If no -port option is present, the program uses the default value of 9997.
  • -process CMD — Specifies the command to be executed instead of the normal copy operation 👑. The CMD binary calls API subroutines to communicate with nettee, and it passes messages back and forth along with the data. While this allows for complex processing in a nettee context, a better choice in most cases would be to implement this complex multinode process using OpenMPI. Input comes from the source(s) specified by -in and from the data distribution tree. Output goes to the local destination specified by -out and to the data distribution tree.

Whenever two or more data sources exist on a node, some form of merge is required, and -process must be specified to supply that function ✨. For -flow push, the distribution tree may branch whether or not -process is set, but -process must be used if local input is to be accepted on nonroot nodes. For -flow pull, only if -process is set may the distribution tree branch or local input be accepted on internal nodes. May not be combined with the -cmd option. See also -hcommand, which provides more details.

  • -progress COUNT — If -v 8 has been set, nettee emits a status message every COUNT bytes 📈. The default of one message every 10000000 bytes is appropriate for a 100baseT network. Adjust COUNT up or down as needed for faster or slower networks.
  • -q — Suppresss "ignored signal" messages. 🤫 (yes, three s's, she's just like that, don't @ me)
  • -root — Defines the current node as the root node of the distribution tree 🌳. If this isn't specified, a node will sit there waiting for an incoming connection request from the node that's about to become its parent. -root tells the root node that no such request is coming, and that it should start connecting to the child node(s) specified by the -next option. The distribution tree always assembles down from the root node, even if -flow pull is specified. 💜
  • -stm EOS — Stream text through a nettee chain until the string EOS is encountered, then exit ✨. This lets short text messages traverse the chain without waiting for a buffer to fill. And since the text message can traverse the nettee chain very rapidly, it can be piped into execinput (or any other program that will execute its stdin as commands) to produce essentially simultaneous execution on all target nodes 🔥. The EOS string is not passed through the data chain and its length is ignored. When used to start further nettee processes on the target nodes, PORT values must be chosen to avoid interference. While this mode may be convenient for setting up Beowulf nodes, it is exceedingly dangerous for general use, since any command introduced into the command stream will execute on all chain nodes as if submitted by the owner of the nettee process on that node 😭. Run nettee -hexamples to see a usage example.
⚠️ bestie, that last one is not a drill. -stm + execinput = every node runs whatever you pipe in, as the nettee process owner. Great for setting up Beowulf nodes. Terrifying everywhere else. 💀
  • -sync — force local output onto the disk before closing the data stream. Default vibe? leave it in disk cache — literally [DEFAULT - leave it in disk cache].
  • -t WAIT — wait up to WAIT seconds for a connection from the parent node in the distribution tree or for data to be received. If neither of those events happens, nettee exits with an error 😭. A value of 0 waits forever and will only exit on an end of data condition. If no -t is present, the program uses a default WAIT value of 0. The -connf WAIT and -w options control timeouts for connections to child nodes 🫶.
  • -v VERBOSE — VERBOSE is a bit mask which controls the types of warning and error messages which are sent to the -log destination. Bit values indicate:
    • 1 show error messages
    • 2 show command line settings
    • 4 show messages
    • 8 show periodic status messages during transfer
    • 16 prepend nodename to all messages

Use a VERBOSE value of 0 to eliminate all messages. If no -v is present, the program uses a default VERBOSE value of 1.

  • -w — wait for the next node to boot or attach to the network. If not specified and the next node is not reachable, nettee will exit with an error no matter what the -t WAIT and -connf WAIT timeout values are. No cap, that's the warning bestie ⚠️

🎀 COPYRIGHTS 🎀

  • Copyright: 2009 David Mathog and Caltech.
  • Copyright: Felix Rauch and ETH Zurich

📜 LICENSE 📜

Freely distributed under the second GNU General Public License (GPL 2). periodt ✨

👑 AUTHOR 👑

  • David Mathog
  • Biology Division, Caltech

That's the whole page, bestie! 💖 This was nettee 0.3.5, dated NOV 2019, page nettee(1) — slay and stay safe out there 🌸🫶

📦 where this page comes from