libinn_dbz(3)

💖 libinn_dbz(3) — the dbz page, bestie edition 💖

InterNetNews Documentation ✨ Manual page: libinn_dbz(3). Buckle up, we're doing database routines today and honestly? They slay.

---

🌸 NAME 🌸

dbz — Database routines for InterNetNews. Yeah that's the whole name section, it's giving minimalism 💅

---

✨ SYNOPSIS ✨

Okay so here's the whole vibe upfront, copy-paste ready, no notes:

#include <inn/dbz.h>

#define DBZMAXKEY              ...
#define DBZ_INTERNAL_HASH_SIZE ...

typedef enum
{
    DBZSTORE_OK,
    DBZSTORE_EXISTS,
    DBZSTORE_ERROR
} DBZSTORE_RESULT;

typedef enum
{
    INCORE_NO,
    INCORE_MEM,
    INCORE_MMAP
} dbz_incore_val;

typedef struct {
    bool writethrough;
    dbz_incore_val pag_incore;
    dbz_incore_val exists_incore;
    bool nonblock;
} dbzoptions;

typedef struct {
    char hash[DBZ_INTERNAL_HASH_SIZE];
} __attribute__((__packed__)) erec;

extern bool dbzinit(const char *name);
extern bool dbzclose(void);

extern bool dbzfresh(const char *name, off_t size);
extern bool dbzagain(const char *name, const char *oldname);
extern bool dbzexists(const HASH key);
extern bool dbzfetch(const HASH key, off_t *value);
extern DBZSTORE_RESULT dbzstore(const HASH key, off_t data);
extern bool dbzsync(void);
extern long dbzsize(off_t contents);
extern void dbzsetoptions(const dbzoptions options);
extern void dbzgetoptions(dbzoptions *options);

---

🦋 DESCRIPTION 🦋

These functions are an indexing system for rapid random access to a text file — and from here on out we're calling that text file the base file, because that's just the name, iykyk.

dbz stores offsets into the base file so it can snatch them back up fast later. All retrievals are keyed on a hash value that comes from the HashMessageID function over in libinn(3). Cute, right? Everything is vibing off that one hash.

dbzinit opens a database — an index into the base file called name — and that database consists of the files name.dir, name.index, and name.hash, which must already exist. (If the database is brand new, they should be zero-length files. Yes, empty files. It's fine. It's a vibe.) After that, all subsequent accesses go to that database until you call dbzclose to close it. 💫 One important twist: when the tagged hash format is in use — that's when --enable-tagged-hash was given at configure time — then a name.pag file is used instead of .index and .hash.

dbzfetch searches the database for the specified key, and if there's anything for that key it assigns the offset of the base file for the corresponding key to value.

dbzstore stores the key-data pair in the database. It returns "DBZSTORE_EXISTS" for duplicates (meaning entries that already exist), and "DBZSTORE_OK" when it worked. It will fail with "DBZSTORE_ERROR" if the database files are not writable or not opened, or if any other error happens. 😭

dbzexists will verify whether or not the given hash exists — that's it, that's the job. And dbz is optimized for this operation specifically, so it may be significantly faster than dbzfetch. Iconic.

dbzfresh is a variant of dbzinit for creating a new database, except you get more control over the details. The size parameter specifies the size of the first hash table within the database, in number of key-value pairs. Performance will be best if the number of key-value pairs stored in the database does not exceed about 2/3 of size, or 1/2 of size when the tagged hash format is used. (The dbzsize function, given the expected number of key-value pairs, will suggest a database size that meets these criteria — so let the function cook.) Assuming that an fseek offset is 4 bytes, the .index file will be 4 * size bytes. The .hash file will be "DBZ_INTERNAL_HASH_SIZE" * size bytes (the .dir file is tiny and roughly constant in size) until the number of key-value pairs exceeds about 80% of size. And ngl, nothing awful will happen if the database grows beyond 100% of size, but accesses will slow down quite a bit and the .index and .hash files will grow somewhat. So, like, it's allowed, it's just not the moment. 💀

dbz stores up to "DBZ_INTERNAL_HASH_SIZE" bytes — by default, 4 bytes if the tagged hash format is used, 6 otherwise — of the Message-ID's hash in the .hash file to confirm a hit. This eliminates the need to read the base file just to handle collisions. Efficiency queen. 👑

A size of "0" given to dbzfresh is synonymous with the local default. That normal default is suitable for tables of about 6,000,000 key-value pairs (or 500,000 key-value pairs when the tagged hash format is used). And that same default value is the one used by dbzinit. 💅

When databases are regenerated periodically — as it is the case for the history file — it's simplest to pick the parameters for a new database based on the old one. This also permits some memory of past sizes of the old database, so that a new database size can be chosen to cover expected fluctuations. dbzagain is a variant of dbzinit for creating a new database as a new generation of an old database. The database files for oldname must exist. dbzagain is equivalent to calling dbzfresh with a size equal to the result of applying dbzsize to the largest number of entries in the oldname database and its previous 10 generations. Generational, fr fr. 🧬

When many accesses are being done by the same program, dbz is massively faster if its first hash table is in memory. If the pag_incore flag is set to "INCORE_MEM", an attempt is made to read the table in when the database is opened, and dbzclose writes it out to disk again (if it was read successfully and has been modified). dbzsetoptions can be used to set the pag_incore and exists_incore flags to different values, which should be:

• "INCORE_NO" — read from disk
• "INCORE_MEM" — read from memory
• "INCORE_MMAP" — read from a mmap'ed file

...and you do that for the .hash and .index files separately. Note that this does not affect the status of a database that has already been opened. The default is "INCORE_NO" for the .index file and "INCORE_MMAP" for the .hash file. The attempt to read the table in may fail due to memory shortage; in this case dbz fails with an error. 😭

Real talk though: stores to an in-memory database are not (in general) written out to the file until dbzclose or dbzsync. So if robustness in the presence of crashes or concurrent accesses is crucial, in-memory databases should probably be avoided — or the writethrough option should be set to true, which tells it to systematically write to the filesystem in addition to updating the in-memory database. Protect yourself, bestie. 🫶

If the nonblock option is true, then writes to the .hash and .index files will be done using non-blocking I/O. This can be significantly faster if your platform supports non-blocking I/O with files. It is only applicable if you're not mmap'ing the database.

dbzsync causes all buffers etc. to be flushed out to the files. It's typically used as a precaution against crashes or concurrent accesses when a dbz-using process will be running for a long time. Heads up: it's a somewhat expensive operation, especially for an in-memory database. 💸

Concurrent reading of databases is fairly safe, but there is no (inter)locking, so concurrent updating is not. Don't do it. That's not the vibe. 🚫

An open database occupies three stdio streams and two file descriptors. Memory consumption is negligible except for in-memory databases (and stdio buffers).

---

🎀 DIAGNOSTICS 🎀

Functions returning bool values return true for success and false for failure. Simple as that. ✨

dbzinit attempts to have errno set plausibly on return, but otherwise this is not guaranteed. An errno of "EDOM" from dbzinit indicates that the database did not appear to be in dbz format. That's your signal, girlie. 🚨

If "DBZTEST" is defined at compile-time, then a main() function will be included. This will do performance tests and integrity test.

---

💀 BUGS 💀

• Unlike dbm, dbz will refuse to dbzstore with a key already in the database. The user is responsible for avoiding this. Not the program's problem, respectfully. 🙈
• The RFC5322 case mapper implements only a first approximation to the hideously-complex RFC5322 case rules. Hideously. Complex. We said what we said. 😭
• dbz no longer tries to be call-compatible with dbm in any way. It's a whole new era. Periodt.

---

👑 HISTORY 👑

The original dbz was written by Jon Zeeff <zeeff@b-tech.ann-arbor.mi.us>. Later contributions by David Butler and Mark Moraes. Extensive reworking, including this documentation, by Henry Spencer <henry@zoo.toronto.edu> as part of the C News project. MD5 code borrowed from RSA. Extensive reworking to remove backward compatibility and to add hashes into dbz files by Clayton O'Neill <coneill@oneill.net>. Rewritten into POD by Julien Elie. A lineage, honestly. 🌈

---

🌟 SEE ALSO 🌟

• dbm(3)
• history(5)
• libinn(3)

---

And that's the page, bestie!! You just read libinn_dbz(3) from INN 2.7.4, dated 2026-08-20. Go forth and hash your Message-IDs responsibly. Mwah. 💖✨🦋🌸💅