MTBL_WRITER(3)

💖 MTBL_WRITER(3) 💖

okay bestie, welcome to the mtbl_writer(3) page — this is the one where we make MTBL files from scratch. no cap, it's lowkey the most important page in the whole friend group. 🦋

---

✨ NAME ✨

mtbl_writer — create an MTBL file. that's it. that's the job. 💅

---

🌸 SYNOPSIS 🌸

first things first, you gotta pull in the header:

#include <mtbl.h>

🎀 Writer objects 🎀

these are the lil guys that actually do the writing:

struct mtbl_writer *
mtbl_writer_init(const char *fname, const struct mtbl_writer_options *wopt);

struct mtbl_writer *
mtbl_writer_init_fd(int fd, const struct mtbl_writer_options *wopt);

void
mtbl_writer_destroy(struct mtbl_writer **w);

mtbl_res
mtbl_writer_add(struct mtbl_writer *w,
        const uint8_t *key, size_t len_key,
        const uint8_t *val, size_t len_val);

🧚 Writer options 🧚

and these are the settings you can hand it before it starts:

struct mtbl_writer_options *
mtbl_writer_options_init(void);

void
mtbl_writer_options_destroy(struct mtbl_writer_options **wopt);

void
mtbl_writer_options_set_compression(
        struct mtbl_writer_options *wopt,
        mtbl_compression_type compression_type);

void
mtbl_writer_options_set_compression_level(
        struct mtbl_writer_options *wopt,
        int compression_level);

void
mtbl_writer_options_set_block_size(
        struct mtbl_writer_options *wopt,
        size_t block_size);

void
mtbl_writer_options_set_block_restart_interval(
        struct mtbl_writer_options *wopt,
        size_t block_restart_interval);

---

💫 DESCRIPTION 💫

so here's the whole vibe. MTBL files get written to disk by: making an mtbl_writer object, calling mtbl_writer_add() once for every single key-value entry, and then calling mtbl_writer_destroy(). that's the ritual. 🕯️

mtbl_writer_add() copies the key-value pairs from you into the mtbl_writer object. keys come in as a pointer to a buffer, key, plus the length of that buffer, len_key. values come in as a pointer to a buffer, val, plus the length of that buffer, len_val. easy.

⚠️ okay this part is serious, bestie. keys must be in sorted, lexicographical byte order. and the same key may not be added to an mtbl_writer more than once. periodt. if your input entries aren't sorted, or might contain duplicate keys, then you should be using the mtbl_sorter(3) interface instead. don't fight it, just use the sorter.

you can make an mtbl_writer object by calling mtbl_writer_init() with an fname argument naming a file to be created. the filename must not already exist on the filesystem. or, you can call mtbl_writer_init_fd() with an fd argument that's an open, writable file descriptor.

here's the tea on that fd: before you call it, if you move the fd's cursor via write(2) or lseek(2), that has the effect of reserving the file's initial bytes for non-MTBL use. MTBL will only write at, or above, the current offset. after that, only mtbl_writer may touch this fd (including closing it), and no other writes may be made to the underlying file above the first offset of the MTBL data. the MTBL data has to be last in the file. no exceptions. 🫶

if the wopt parameter you pass to mtbl_writer_init() or mtbl_writer_init_fd() is non-NULL, then all the parameters you set up in the mtbl_writer_options object get configured into the mtbl_writer object. slay.

🌈 Writer options 🌈

💜 compression

tells it which compression algorithm to use on the data blocks. the possible values are MTBL_COMPRESSION_NONE, MTBL_COMPRESSION_SNAPPY, MTBL_COMPRESSION_LZ4, MTBL_COMPRESSION_LZ4HC, MTBL_COMPRESSION_ZSTD, or MTBL_COMPRESSION_ZLIB (which is the default, iykyk).

💜 compression_level

the compression level to use on the data blocks — but only if the algorithm you picked actually supports different compression levels. the ones that do are MTBL_COMPRESSION_LZ4HC (0-16), MTBL_COMPRESSION_ZSTD (1-22), and MTBL_COMPRESSION_ZLIB (0-9). the other compression algorithms just ignore this option entirely. and if you give a level outside the range the algorithm supports, it gets silently clamped to the minimum or maximum supported level. no error, no warning, just vibes. if you don't specify it, a reasonable default will be used.

💜 block_size

the maximum size of the uncompressed data blocks, in bytes. the default is 8 kilobytes.

💜 block_restart_interval

how frequently to restart the intra-block key prefix compression. the default is every 16 keys.

---

⭐ RETURN VALUE ⭐

mtbl_writer_init() and mtbl_writer_init_fd() return NULL on failure, and non-NULL on success. 🌟

mtbl_writer_add() returns mtbl_res_success if the key-value entry was successfully copied into the mtbl_writer object, and mtbl_res_failure if it wasn't — for instance, if there's been a key-ordering violation. 😭 yes, that's the classic one.

---

🔥 SEE ALSO — nothing else listed here, bestie, this page is a solo act. 👑

---

and that's the whole page!! this was MTBL_WRITER(3) from the MTBL manual, dated 12/11/2016. mwah, go write some files, you absolute icon. 💖✨🎀🦋🌸💅🫶👑