libtalloc_pools(3)

💖 NAME 💖

libtalloc_pools — Chapter 5: Memory pools 🫧✨

---

🧚 Memory pools 🧚

okay bestie so picture this: allocating brand-new memory is expensive. like, a whole operation. and big programs can have thousands of malloc() calls for a single computation, where every single one of those calls only grabs a teeny tiny bit of memory. that adds up to an undesirable slowdown of the application 😭 but!! we can dodge that slowdown by lowering the number of malloc() calls — and we do it with a memory pool. 💅

so what even is a memory pool? it's a preallocated memory space with a fixed size. when we need to allocate new data, we just take the amount we want out of the pool instead of asking the system for fresh memory. the way this works is by creating a pointer that points inside the preallocated memory. ✨

⚠️ a pool must not be reallocated — that would change its location, and every pointer that was pointing inside the pool would become invalid. so a memory pool needs a very good estimate of the memory space it'll require. no cap.

the talloc library has its own implementation of a memory pool, and it's highly transparent for the programmer. the only thing you gotta do is initialize a new pool context using talloc_pool() — and then you can use it exactly like any other context. iconic tbh 👑

refactoring existing code (the talloc kind) to take advantage of a memory pool is genuinely so simple, because the pool context has these properties:

• if we're allocating data on a pool context, it takes the desired amount of memory from the pool,
• if the context is a descendant of the pool context, it takes the space from the pool as well,
• if the pool doesn't have a sufficient portion of memory left, it will create a new non-pool context, leaving the pool intact 🫶

/* allocate 1KiB in a pool */
TALLOC_CTX *pool_ctx = talloc_pool(NULL, 1024);

/* Take 512B from the pool, 512B is left there */
void *ptr = talloc_size(pool_ctx, 512);

/* 1024B > 512B, this will create new talloc chunk outside
   the pool */
void *ptr2 = talloc_size(ptr, 1024);

/* The pool still contains 512 free bytes
 * this will take 200B from them. */
void *ptr3 = talloc_size(ptr, 200);

/* This will destroy context 'ptr3' but the memory
 * is not freed, the available space in the pool
 * will increase to 512B. */
talloc_free(ptr3);

/* This will free memory taken by 'pool_ctx'
 * and 'ptr2' as well. */
talloc_free(pool_ctx);

---

okay so all of that is super convenient, but there's one big issue you have to keep in mind, bestie. if the parent of a talloc pool child gets changed to a parent that's outside of this pool, the whole pool memory will not be freed until the child is freed. so we have to be very careful when stealing a descendant of a pool context. 😭

TALLOC_CTX *mem_ctx = talloc_new(NULL);
TALLOC_CTX *pool_ctx = talloc_pool(NULL, 1024);
struct foo *foo = talloc(pool_ctx, struct foo);

/* mem_ctx is not in the pool */
talloc_steal(mem_ctx, foo);

/* pool_ctx is marked as freed but the memory is not
   deallocated, accessing the pool_ctx again will cause
   an error */
talloc_free(pool_ctx);

/* This deallocates the pool_ctx. */
talloc_free(mem_ctx);

ngl it's often better to copy the memory we want instead of stealing it, just to dodge this whole problem. if we don't need to retain the context name (i.e. to keep the type information), we can use talloc_memdup() to do it. 🎀

but!! copying the memory out of the pool may discard all the performance boost the pool gave us — depends on the size of the copied memory. so the code should be well profiled before taking this path. in general, the golden rule is: if we need to steal from the pool context, we should not use a pool context. periodt. 💀

---

💖 that's the whole chapter, bestie — go forth and pool responsibly 💖

source: talloc, Version 2.0 — and yes, this was the libtalloc_pools(3) page, mwah 😘💅🌸