cstream(1)

📦
version 4.0.0 · 3 versions
, in debian bookworm, debian trixie, ubuntu noble, ubuntu resolute, ubuntu stonking

✨ cstream(1) ✨

the manual page, but it's the group chat version 💖 (the manual name is literally cstream, and it lives in section 1, General Commands Manual, just so you know what you're reading 🫶)

💅 NAME

cstream — direct data streams, with bandwidth limiting, FIFO, audio, duplication and extended reporting support. she does it all, no cap. 🎀

🦋 SYNOPSIS

cstream [-b num] [-B num] [-i filename] [-I string] [-l] [-n num]
        [-o filename] [-O string] [-p filename] [-t num] [-T num]
        [-v num] [-V] [filename]

🌸 DESCRIPTION

Okay so cstream filters data streams, and honestly? It's giving dd(1) energy — like the UNIX tool dd(1), but with a glow-up ✨. It's got a more traditional commandline syntax, support for precise bandwidth limiting and reporting, and support for FIFOs. And bestie, the data limits and throughput rate calculation will work for files > 4 GB. Not a vibe issue. Ever. 💪

Cstream reads from the standard input and writes to the standard output, if no filenames are given. It will also 'generate' or 'sink' data if desired. main character energy tbh — it can just make data appear out of nowhere or make it disappear. 🪄

🌈 Options:

  • -b num — Set the block size used for read/write to num. The default is 8192 bytes. 🧱
  • -B num — Buffer input up to num bytes before writing. The default is the blocksize. It is an error to set this to anything below the blocksize. Super useful when writing tapes and similar that prefer few large writes of many small. 📼 (big chunky writes only, ty)
  • -c num — Concurrent operation. Use a separate process for output. This is especially useful in combination with the -B option.
    • 0 = use one process only (default)
    • 1 = read process will buffer
    • 2 = write process will buffer
    • 3 = both processes will buffer.
In combination with a large buffer size this will often load your memory heavily, every time the reader transfers the buffer it collected to the writer. If you use -c 3 and have a buffer size of 128 Megabytes 256 MB of memory will be touched at once. 😭 that's a lot of RAM babe, be careful.
  • -i num
  • -o num — Set the file names to use for input or output, respectively. If the output file name is -, data will just be discarded. If the input file name is -, data will be generated 'out of the void' 🕳️✨. If these options aren't given, stdin/stout will be used. If you need to give -o or -i options and want stdin/stdout, specify the empty string, like this:
cstream -i''

If TCP support has been compiled in (default), hostname:portnumber will try to connect to the specified host at the specified port and :portnumber will open a TCP socket on the local machine and wait for a connection to arrive.

SECURITY NOTE: cstream includes no mechanism to restrict the hosts that may connect to this port. Unless your machine has other network filters, anyone will be able to connect. bestie. please. 💀
  • -I string
  • -O string — Specify the type of input and output file, respectively.
    • If string includes 'f', a fifo will be created. 🚰
    • If string includes 'F', a rsync(2) will be issued before closing the output file.
    • If string includes 'a', the file will be assumed to be a opensound-compatible audio device and will be switched to CD-like settings. 🎧
    • If string includes 't', a copy of the stream will be sent to file descriptor 3.
    • If string includes 'N', TCP will not be used for that file even if the name has a ":".
  • -l — Include line count in statistics. 🔢
  • -n num — Limit the total amount of data to num. If there is more input available, it will be discarded, cstream will exit after the limit has been reached. If there is less input, the limit will not be reached and no error will be signaled. (so chill about it fr fr)

num may have a trailing 'k', 'm' or 'g' which means Kilobytes, Megabytes or Gigabytes (where Kilo = 1024). This applies to all numeric options. 🔥

  • -p filename — Write the process id of cstream to filename. If cstream uses a separate writer process (option -c), this is the pid of the parent (reader) process. 🪪
  • -t num — Limit the throughput of the data stream to num bytes/second. Limiting is done at the input side, you can rely on cstream not accepting more than this rate. If the number you give is positive, cstream accumulates errors and tries to keep the overall rate at the specified value, for the whole session. If you give a negative number, it is an upper limit for each read/write system call pair. In other words: the negative number will never exceed that limit, the positive number will exceed it to make good for previous underutilization. 🏃♀️💨
  • -T num — Report throughput every num seconds. ⏱️
  • -v num — Set verbose level to num. By default, it is set to 0, which means no messages are displayed as long as no errors occur. A value of 1 means that total amount of data and throughput will be displayed at the end of program run. A value of 2 means the transfer rate since the end of the first read/write pair will also be reported (useful when there is an initial delay). A value of 3 means there will also be separate measurements for read and write. This option is resource-consuming and currently isn't implemented. A value of 4 means that notices about each single read/write will be displayed. High values include all message types of lower values. 🗣️
  • -V — Print version number to stdout and exit with 0. 👑
  • filename — A single filename as the last argument without an option switch will be used as input file if -i has not been used. 📁

🧚 Signals (yes bestie, she listens)

  • SIGUSR1
  • SIGINFO — Sending SIGUSR1 (or SIGINFO, which is usually mapped to Control-T on you keyboard) to cstream causes it to display throughput rates to stderr. The stream will continue as if nothing happened. like, zero drama. iconic. 🌈
  • SIGUSR2 — Exit and report throughput rates, if requested. ✌️
  • SIGHUP — I found myself sending SIGHUP accidentally too often. But ignoring or misusing SIGHUP is not an option for me. Thus, when cstream received SIGHUP, it will wait 5 seconds for another SIGHUP, to give users a chance to correct a possible mistake. If no additional SIGHUP is received, cstream kills itself with SIGHUP. 😭 a five-second grace period, she's so forgiving 💖

✨ EXAMPLES ✨

Okay bestie, this is the part where it all clicks. Real commands, real vibes. 💖

Benchmark speed test, but make it a file

cstream -o tmpfile -v 1 -n 384m -i -

Writes 384 Megabytes of unspecified data to file tmpfile and display verbose throughput rate. Honestly? This makes a good benchmark, because the speed of /dev/null varies too much from system to system. 💅

Read it back and yeet it

cstream -i tmpfile -v 1 -n 384m -o -

Read the same file back in and discard data. Round trip, no notes. 🦋

The chill watchable log reader

cstream -b 2000  -t 10000 /var/log/messages

Will display the file in a more or less watchable speed. Not too fast, not too slow, just right. 🌸

dump(1) to tape, with a pidfile so you can poke it later

dump 0sf 400000 - / | cstream -v 1 -b 32768 -o /dev/rst0 -p pidfile
kill -USR1 `cat pidfile`

Write the output from dump(1) to tape. Each time the signal is sent, the throughput and data rate so far will be displayed. Live stats, iconic. 👑

Fake soundcard era

cstream -t 176400 -i /dev/dsp0 -I f -o -

Makes kind of a soundcard emulator which may be used to test audio applications that need something to write to that limits the data rate as a real soundcard does. This obviously doesn't work when the application tries to write data using mmap(2) and the application has to ignore errors when it tries to set soundcard parameters using ioctl(2). 😭

Same energy, but now you can spy on the data

cstream -t 176400 -i /dev/dsp0 -I f -o /dev/dsp1 -O f

Similar soundcard emulator, except that it allows you to grab the data your applications sends to it from the other fifo, while still having precise timing. 🎀

Network audio, CD quality, straight to the soundcard

cstream -Oa -o /dev/dsp0 myhost.mydomain.com:17324

Connects port 3333 on host myhost.mydomain.com and whatever data it finds there will be sent to the soundcard, with appropriate settings for CD quality stereo play. 🔥

Now be the server, slay

cstream -i myaudiofile.raw -o :17324

This will open a TCP server on port 17324 and waits until someone connects (for example, the commandline from the previous example). Then it will send the contents of myaudiofile.raw down the TCP stream (for the previous audio example, typically a CD audio track like you get from the tosha or cdparanoia utilities). ✨

O_DIRECT file writing, the whole saga

cstream -OD -o myfile

Write to file myfile with O_DIRECT. That usually means that the filesystem buffer cache will not try to cache this file. You can use that to prevent copying operations from eating up physical memory. Note that when cstream encounters a write error it will switch the output file from O_DIRECT to a normal file and write all further blocks without O_DIRECT if writes without O_DIRECT succeed. In practice that usually means that your last block, if not a multiple of the filesystem block size, will still be written into the file (the maximum amount of data written without O_DIRECT is your blocksize minus one). That way cstream ensures that the output file has the length of the input, however odd the length was and no matter what restrictions your OS places on O_DIRECT output. Again, cstream will not pad the output to the block size, you get the same file and file size as if not using O_DIRECT, at the cost of switching to non-O_DIRECT whenever a block is not the right size. 💀

Buffering TCP so O_DIRECT doesn't get dramatic

cstream -i :3333 | dd obs=8192 | ./cstream -omyfile -v7 -OD

This is what you need to do to buffer TCP input, so that the last cstream will not switch away from O_DIRECT prematurely because of short reads. If your input can do short reads (e.g. from TCP), and you want to ensure that O_DIRECT stays in effect, you need a buffer between the TCP stream and the O_DIRECT stream. Since cstream does not yet support different input and output block sizes, dd is suitable here. Note that this is only necessary if the OS requires multiples of the filesystem block size for O_DIRECT. At the time of this writing this construct is needed on Linux for using TCP streams with O_DIRECT, but it is not needed on FreeBSD. 🐧

O_SYNC, the "it's on disk, I promise" mode

cstream -OS -o myfile

Writes to file myfile with O_SYNC. This means by the time the system call returns the data is known to be on disk. This is not the same thing as O_DIRECT. O_DIRECT can do its own buffering, with O_SYNC there is no buffering at all. At the time of this writing, O_SYNC on both Linux and FreeBSD is very slow (1/5th to 1/10th of normal write) and O_DIRECT is reasonably fast (1/4th to 1/2 of normal write). You can combined O_SYNC and O_DIRECT. 🌈

🌸 ERRORS 🌸

Here's the tea on exit codes, bestie. 🫶

  • Exit code 0 means success. We love a win. 💖
  • Exit code 1 means a commandline syntax usage error. Oops, typo era. 😭
  • Exit code 2 means other errors, especially system errors. Not our fault, fr fr. 💀

👑 Bugs 👑

There should be an option to begin writing directly after the first read ended and then fill the buffer with reads in the background. Right now writing will not begin before the reader has filled the buffer completely for the first time. Patience era, unfortunately. 🙈

Not a bug: the code to do O_DIRECT is reasonably sophisticated. It will fall back to normal I/O on errors. But before doing that it knows about both filesystem blocksize requirements (will default I/O blocksize to whatever the filesystem of the output file is in) and page alignment requirements (I/O will happen from a page-aligned buffer). However, the combination of concurrent read/writes (-c options) and O_DIRECT has not been tested beyond basic verification that it gets some tests right. ⭐

💜 SEE ALSO 💜

🦋 HISTORY 🦋

cstream was initially written by Martin Cracauer in 1998. For updates and more information see http://www.cons.org/cracauer/cstream.html. A legend, honestly. Periodt. ✨

And that's a wrap on this page, bestie! This one's the Debian build, from March, 30, 1999, filed under cstream(1). Go forth and stream responsibly. 💅💖

📦 where this page comes from