VENTI-FMT(8)

💖 NAME 💖

okay so this is VENTI-FMT(8), the manual for the whole little crew that preps and maintains a venti server: buildindex, checkarenas, checkindex, conf, fmtarenas, fmtbloom, fmtindex, fmtisect, syncindex — prepare and maintain a venti server. periodt 💅

✨ SYNOPSIS ✨

venti/fmtarenas [ -4Z ] [ -a arenasize ] [ -b blocksize ] name file

venti/fmtisect [ -1Z ] [ -b blocksize ] name file

venti/fmtbloom [ -n nblocks | -N nhash ] [ -s size ] file

venti/fmtindex [ -a ] venti.conf

venti/conf [ -w ] partition [ configfile ]

venti/buildindex [ -bd ] [ -i isect ] ... [ -M imemsize ] venti.conf

venti/checkindex [ -f ] [ -B blockcachesize ] venti.conf tmp

venti/checkarenas [ -afv ] file

🌸 DESCRIPTION 🌸

These commands are the setup, maintenance, and debugging besties for venti servers, no cap. For the big-picture overview of the venti system, see venti(7), and for the overview of the data structures the venti server uses, see venti(8). 🦋

Note that the units for the various sizes in the following commands can be specified by appending or to indicate kilobytes, megabytes, or gigabytes respectively. (That is literally what the page says, bestie 😭)

🎀 Formatting 🎀

To prepare a server for its first-ever use, the arena partitions and the index sections have to be formatted individually, with fmtarenas and fmtisect. Then the whole collection of index sections gets combined into one venti index with fmtindex. It's giving assembly line, but make it storage. ✨

Fmtarenas formats the given file, typically a disk partition, into an arena partition. The arenas in the partition are given names of the form name%d, where %d is replaced with a sequential number starting at 0.

Options to fmtarenas are:

• -a arenasize — The arenas are of arenasize bytes. The default is 512M, which was picked to balance the number of arenas against the ability to copy an arena to external media such as recordable CDs and tapes. old-school media representation, we love to see it 💿
• -b blocksize — The size, in bytes, for read and write operations to the file. The size is recorded in the file, and is used by applications that access the arenas. The default is 8k.
• -4 — Create a version 4 arena partition for backwards compatibility with old servers. The default is version 5, used by the current venti server.
• -Z — Do not zero the data sections of the arenas. Using this option reduces the formatting time but should only be used when it is known that the file was already zeroed. (Version 4 only; version 5 sections are not and do not need to be zeroed.) do not skip the zero check unless you know, bestie 🫶

Fmtisect formats the given file, typically a disk partition, as a venti index section with the specified name. Each of the index sections in a venti configuration must have a unique name. unique names only, it's the law 💅

Options to fmtisect are:

• -b bucketsize — The size of an index bucket, in bytes. All the index sections within a index must have the same bucket size. The default is 8k. (the synopsis writes this one as -b blocksize, bestie — same flag, different label energy)
• -1 — Create a version 1 index section for backwards compatibility with old servers. The default is version 2, used by the current venti server.
• -Z — Do not zero the index. Using this option reduces the formatting time but should only be used when it is known that the file was already zeroed. (Version 1 only; version 2 sections are not and do not need to be zeroed.)

Fmtbloom formats the given file as a Bloom filter (see venti(7)). The options are:

• -n nblock | -N nhash — The number of blocks expected to be indexed by the filter or the number of hash functions to use. If the -n option is given, it is used, along with the total size of the filter, to compute an appropriate nhash. (the synopsis says -n nblocks, plural — keep that detail in your heart) ✨
• -s size — The size of the Bloom filter. The default is the total size of the file. In either case, size is rounded down to a power of two.

The file argument in the commands above can be of the form file:lo-hi to specify a range of the file. lo and hi are specified in bytes but can have the usual k, m, or g suffixes. Either lo or hi may be omitted. This notation eliminates the need to partition raw disks on non-Plan 9 systems. raw disk girlies, this one's for you 🦋

Fmtindex reads the configuration file venti.conf and initializes the index sections to form a usable index structure. The arena files and index sections must have previously been formatted using fmtarenas and fmtisect respectively. do not skip steps, bestie, it will not slay 😭

The function of a venti index is to map a SHA1 fingerprint to a location in the data section of one of the arenas. The index is composed of blocks, each of which contains the mapping for a fixed range of possible fingerprint values. Fmtindex determines the mapping between SHA1 values and the blocks of the collection of index sections. Once this mapping has been determined, it cannot be changed without rebuilding the index. The basic assumption in the current implementation is that the index structure is sufficiently empty that individual blocks of the index will rarely overflow. The total size of the index should be about 2% to 10% of the total size of the arenas, but the exact percentage depends both on the index block size and the compressed size of blocks stored. See the discussion in venti(8) for more. math girlie moment, but make it storage 🔥

Fmtindex also computes a mapping between a linear address space and the data section of the collection of arenas. The -a option can be used to add additional arenas to an index. To use this feature, add the new arenas to venti.conf after the existing arenas and then run fmtindex -a.

• -a — Add additional arenas to an index, using the whole "add new arenas to venti.conf after the existing arenas, then run fmtindex -a" situation described above. append-only era 💅

A copy of the above mappings is stored in the header for each of the index sections. These copies enable buildindex to restore a single index section without rebuilding the entire index. backup header realness 🫶

To make it easier to bootstrap servers, the configuration file can be stored in otherwise empty space at the beginning of any venti partitions using conf. A partition so branded with a configuration file can be used in place of a configuration file when invoking any of the venti commands. By default, conf prints the configuration stored in partition. When invoked with the -w flag, conf reads a configuration file from configfile (or else standard input) and stores it in partition.

• -w — Read a configuration file from configfile (or else standard input) and store it in partition. Without -w, conf just prints the configuration stored in partition. read and write energy, iconic ✨

🦋 Checking and Rebuilding 🦋

Buildindex populates the index for the Venti system described in venti.conf. The index must have previously been formatted using fmtindex. This command is typically used to build a new index for a Venti system when the old index becomes too small, or to rebuild an index after media failure. Small errors in an index can usually be fixed with checkindex, but checkindex requires a large temporary workspace and buildindex does not. trade-offs, bestie, they're everywhere 😭

Options to buildindex are:

• -b — Reinitialise the Bloom filter, if any. bloom filter reset slay 🌸
• -d — Dumb mode; run all three passes.
• -i isect — Only rebuild index section isect; may be repeated to rebuild multiple sections. The name is special and just reads the arenas.
• -M imemsize — The amount of memory, in bytes, to use for caching raw disk accesses while running buildindex. (This is not a property of the created index.) The usual suffices apply. The default is 256M.

Checkindex examines the Venti index described in venti.conf. The program detects various error conditions including: blocks that are not indexed, index entries for blocks that do not exist, and duplicate index entries. If requested, an attempt can be made to fix errors that are found. error hunting era 🔍

The tmp file, usually a disk partition, must be large enough to store a copy of the index. This temporary space is used to perform a merge sort of index entries generated by reading the arenas. give tmp room to breathe, no cap 🫶

Options to checkindex are:

• -B blockcachesize — The amount of memory, in bytes, to use for caching raw disk accesses while running checkindex. The default is 8k.
• -f — Attempt to fix any errors that are found. fix-it era, but only if you mean it 💅

Checkarenas examines the Venti arenas contained in the given file. The program detects various error conditions, and optionally attempts to fix any errors that are found. arena checkup, very self-care 🧚

Options to checkarenas are:

• -a — For each arena, scan the entire data section. If this option is omitted, only the end section of the arena is examined. full scan vs little peek, choose wisely 👑
• -f — Attempt to fix any errors that are found.
• -v — Increase the verbosity of output. more words, more lore ✨

🌈 SOURCE 🌈

/src/cmd/venti/srv

⭐ SEE ALSO ⭐

venti(7), venti(8)

💀 BUGS 💀

Buildindex should allow an individual index section to be rebuilt. not the vibe, but it is what it is 😭

---

okay that's the whole VENTI-FMT(8) page, bestie: source is /src/cmd/venti/srv, version/manual is VENTI-FMT(8), and no date is printed in this copy — mwah, go format something iconic 💖🫶💅