BUFFER(1)

✨ BUFFER(1) ✨

okay bestie, welcome to the General Commands Manual 💅 this is buffer, and she is a very fast reblocking program 🏃♀️💨 no cap.

---

🌸 NAME 🌸

buffer — very fast reblocking program 🦋

---

💖 SYNTAX 💖

buffer [-S size] [-b blocks] [-s size] [-z size] [-m size] [-p
percentage] [-u microseconds] [-B] [-t] [-Z] [-i filename] [-o filename]
[-d]

---

✨ OPTIONS ✨

• -i filename — use filename as the input file instead of whatever's coming in. default is stdin 🫶
• -o filename — use filename as the output file. default is stdout 💖
• -S size — every time a chunk of this size has been written, she prints how much has been written so far. also prints the total throughput. by default this is not set.
• -s size — size in bytes of each block. the default blocksize is 10k, which matches the normal output of the tar(1) program 👑
• -z size — literally just combines the -S and -s flags. two-for-one, we love a bundle 🎀
• -b blocks — number of blocks to allocate to the shared memory circular buffer. defaults to the number required to fill up the shared memory requested.
• -m size — maximum size of the shared memory chunk to allocate for the circular queue. defaults to one megabyte.
• -p percentage — only start a write when the given percentage of the internal queue is full. a percentage around 75 often proves best. defaults to zero.
• -u microseconds — after every write, pause for this many microseconds. defaults to zero. (and like, surprisingly? a small sleep, 100 usecs, after each write can greatly enhance throughput on some drives. the way that works is wild fr fr 😭)
• -B — force each block written to be padded out to the blocksize. some tape and cartridge drives need this. defaults to unpadded. this only affects the last block written.
• -t — on exiting, print a brief message to stderr showing the total number of bytes written.
• -Z — if reading/writing directly to a character device (like a tape drive), then after each gigabyte perform an lseek to the start of the file.

⚠️ use this flag with extreme care. it can only be used on devices where an lseek does not rewind the tape but does reset the kernel's position flags. it's used to allow more than 2 gigabytes to be written. like… girl. be careful. 💀

• -d — print debug information to stderr during operation.

Sizes are a number with an optional trailing character. a 'b' multiplies the size by 512, a 'k' by 1024 and an 'm' by a meg. iykyk 💅

---

🦋 DESCRIPTION 🦋

buffer reads from standard input, reblocking to the given blocksize, and writes each block to standard output. that's the whole vibe ✨

internally buffer is a pair of processes communicating via a large circular queue held in shared memory. the reader process only has to block when the queue is full, and the writer process only when the queue is empty. buffer is designed to try and keep the writer side continuously busy so that it can stream when writing to tape drives. when used to write tapes with an intervening network buffer, it can result in a considerable increase in throughput 📈

the default settings for buffer are normally good enough, ngl. but if you're a heavy tape user, then it is worth your while trying out various different combinations of options. in particular, running a buffer at both ends of the pipe can provide a substantial increase — see the last example below 🫶

---

🌈 EXAMPLES 🌈

$ buffer < /etc/termcap > /dev/rst8

$ tar cf - . | rsh somehost 'buffer > /dev/rst8'

$ dump fu - | rsh somehost 'buffer -s 16k > /dev/nrst8'
$ tar cf - . | buffer |
   rsh somehost 'buffer -S 500K -p 75 > /dev/rst0'

---

👑 BUFFER LIMITS 👑

there are 2 limits in buffer affecting the maximum total buffer size: the maximum number of blocks (2048) and the maximum size of a block (512kB). this results in a maximum total buffer size of 1GB.

note that there's also a kernel limit for the maximum size of a shared memory segment (used by buffer internally), which is typically much lower than the limits in buffer. so if buffer complains about not being able to create a shared memory segment, this limit might have to be raised using the command sysctl kernel.shmmax=XXX (see also sysctl(1) and proc(5)). 🙈

---

💀 BUGS 💀

internally, for printing purposes, buffer counts in terms of the number of kilobytes output. so if the blocksize you use is not a whole number of kilobytes, then the numbers printed will be inaccurate. not me getting caught out by rounding 😭

---

🎀 THANKS 🎀

thanks to Kevin Twidle <kpt@doc.ic.ac.uk> for a lot of early suggestions and patches to make it work with non-tar/dump tapes to exabyte drives. 🫶

thanks to Andi Karrer <karrer@bernina.ethz.ch>, Rumi Zahir <rumi@iis.ethz.ch> and Christoph Wicki <wicki@iis.ethz.ch> for patches to make buffer work when trying to write single tape files of greater than 2 gigabytes. absolute icons, all of them 👑

---

💜 COPYRIGHT 💜

copyright (C) 1990, 1991 by Lee McLoughlin. periodt.

---

🌟 SEE ALSO 🌟

dd(1), tar(1), rsh(1)

---

okay that's the whole page bestie, mwah 💖 this one's from 14 May 1990, and yeah it's still filed under BUFFER(1) — the General Commands Manual could never be more iconic. go stream some tapes 🎀🦋✨