readers(3)

✨ readers — the Reader Lock Table ✨

hiii bestie 🫶 welcome to readers(3), straight out of the Library Functions Manual. this page is lowkey the most elegant part of LMDB, fr fr: no locks for reading, just a table and good vibes. let's get into it 💅

---

🌈 SYNOPSIS 🌈

🦋 Data Structures

the whole cast of characters, in order:

• struct MDB_rxbody
• struct MDB_reader
• struct MDB_txbody
• struct MDB_txninfo

🎀 Macros

• #define DEFAULT_READERS 126
• #define CACHELINE 64
• #define MDB_LOCK_FORMAT

---

💖 Detailed Description 💖

okay so here's the iconic plot twist: readers don't acquire any locks for their data access. not one. instead they just... record their transaction ID in the reader table. that's it, that's the whole move ✨

the reader mutex is needed just to find an empty slot in the reader table. the slot's address is saved in thread-specific data, so that subsequent read transactions started by the same thread need no further locking to proceed. obsessed. 👑

if MDB_NOTLS is set, the slot address is not saved in thread-specific data.

and no reader table is used at all if the database is on a read-only filesystem, or if MDB_NOLOCK is set. so don't go looking for it. it's not there 🙈

since the database uses multi-version concurrency control, readers don't actually need any locking. this table is used to keep track of which readers are using data from which old transactions, so that we'll know when a particular old transaction is no longer in use. old transactions that have discarded any data pages can then have those pages reclaimed for use by a later write transaction. recycling era 💚

the lock table is constructed such that reader slots are aligned with the processor's cache line size, and any slot is only ever used by one thread. this alignment guarantees that there will be no contention or cache thrashing as threads update their own slot info, and it also eliminates any need for locking when accessing a slot. it's giving "designed by someone who actually knows what a cache line is" 💅

a writer thread will scan every slot in the table to determine the oldest outstanding reader transaction. any freed pages older than this will be reclaimed by the writer. and the writer doesn't use any locks when scanning this table. which means there's no guarantee that the writer will see the most up-to-date reader info — but that's not required for correct operation! all we need is to know the upper bound on the oldest reader, we don't care at all about the newest reader. so the only consequence of reading stale information here is that old pages might hang around a while longer before being reclaimed. and honestly? that's actually good anyway 🌸, because the longer we delay reclaiming old pages, the more likely it is that a string of contiguous pages can be found after coalescing old pages from many old transactions together. slay.

---

📚 Data Structure Documentation 📚

🍓 struct MDB_rxbody

the information we store in a single slot of the reader table. in addition to a transaction ID, we also record the process and thread ID that owns a slot, so that we can detect stale information — e.g. threads or processes that went away without cleaning up. (the ghosting is REAL 😭)

Note: we currently don't check for stale records. we simply re-init the table when we know that we're the only process opening the lock file.

Data Fields

• volatile txnid_t mrb_txnid
• volatile MDB_PID_T mrb_pid
• volatile MDB_THR_T mrb_tid

Field Documentation

• volatile txnid_t MDB_rxbody::mrb_txnid — Current Transaction ID when this transaction began, or (txnid_t)-1. multiple readers that start at the same time will probably have the same ID here. again, it's not important to exclude them from anything; all we need to know is which version of the DB they started from, so we can avoid overwriting any data used in that particular version.
• volatile MDB_PID_T MDB_rxbody::mrb_pid — the process ID of the process owning this reader txn.
• volatile MDB_THR_T MDB_rxbody::mrb_tid — the thread ID of the thread owning this txn.

🧚 struct MDB_reader

the actual reader record, with cacheline padding. padding is self-care, bestie 🫶

Data Fields

union {
   MDB_rxbody mrx
   char pad [(sizeof(MDB_rxbody)+CACHELINE-1)
          &~(CACHELINE-1)]"
} mru

Field Documentation

• char MDB_reader::pad[(sizeof(MDB_rxbody)+CACHELINE-1) &~(CACHELINE-1)] — cache line alignment.

👑 struct MDB_txbody

the header for the reader table. the table resides in a memory-mapped file — and ngl, this is a different file than the one used for the main database. don't mix them up 💀

for POSIX, the actual mutexes reside in the shared memory of this mapped file. on Windows, mutexes are named objects allocated by the kernel; we store the mutex names in this mapped file so that other processes can grab them. this same approach is also used on MacOSX/Darwin (using named semaphores) since MacOSX doesn't support process-shared POSIX mutexes. for these cases where a named object is used, the object name is derived from a 64 bit FNV hash of the environment pathname. as such, naming collisions are extremely unlikely. if a collision occurs, the results are unpredictable. 😬 extremely unlikely ≠ impossible, so respect the risk.

Data Fields

• uint32_t mtb_magic
• uint32_t mtb_format
• mdb_mutex_t mtb_rmutex
• volatile txnid_t mtb_txnid
• volatile unsigned mtb_numreaders

Field Documentation

• uint32_t MDB_txbody::mtb_magic — Stamp identifying this as an LMDB file. it must be set to MDB_MAGIC.
• uint32_t MDB_txbody::mtb_format — Format of this lock file. must be set to MDB_LOCK_FORMAT.
• mdb_mutex_t MDB_txbody::mtb_rmutex — Mutex protecting access to this table. this is the reader table lock used with LOCK_MUTEX().
• volatile txnid_t MDB_txbody::mtb_txnid — the ID of the last transaction committed to the database. this is recorded here only for convenience; the value can always be determined by reading the main database meta pages.
• volatile unsigned MDB_txbody::mtb_numreaders — the number of slots that have been used in the reader table. this always records the maximum count — it is not decremented when readers release their slots.

🌟 struct MDB_txninfo

the actual reader table definition. the blueprint. the main character ✨

Data Fields

union {
   MDB_txbody mtb
   char pad [(sizeof(MDB_txbody)+CACHELINE-1)
          &~(CACHELINE-1)]"
} mt1
union {
   mdb_mutex_t mt2_wmutex
   char pad [(MNAME_LEN+CACHELINE-1)
          &~(CACHELINE-1)]"
} mt2
MDB_reader mti_readers [1]

---

🎀 Macro Definition Documentation 🎀

• #define DEFAULT_READERS 126 — number of slots in the reader table. this value was chosen somewhat arbitrarily (iconic behavior tbh). 126 readers plus a couple mutexes fit exactly into 8KB on the author's development machine. applications should set the table size using mdb_env_set_maxreaders().
• #define CACHELINE 64 — the size of a CPU cache line in bytes. we want our lock structures aligned to this size to avoid false cache line sharing in the lock table. this value works for most CPUs. for Itanium this should be 128.
• #define MDB_LOCK_FORMAT — value:

((uint32_t) \
 ((MDB_LOCK_VERSION) \
  /* Flags which describe functionality */ \
  + (((MDB_PIDLOCK) != 0) << 16)))

lockfile format signature: version, features and field layout 💅

---

✍️ Author ✍️

generated automatically by Doxygen for LMDB from the source code. not a human — just Doxygen doing its little thing 🤖✨

---

aaand that's the whole page, bestie! this one came from LMDB, readers(3) — go forth and read locklessly. mwah 💖💅🦋