POOL_CACHE_INIT(9)

🎀 pool_cache_init(9) 🎀

okay so this is the Kernel Developer's Manual page for pool_cache_init(9), and it also covers pool_cache_destroy because they're a package deal 💖. Manual page name: POOL_CACHE_INIT(9). Buckle up, bestie 🫶

---

🌸 NAME 🌸

pool_cache_init, pool_cache_destroy — per CPU caching of pool items, literally.

✨ SYNOPSIS ✨

#include <sys/pool.h>

void
pool_cache_init(struct pool *pp);

void
pool_cache_destroy(struct pool *pp);

🦋 DESCRIPTION 🦋

So by default, pools protect their internal state using a single lock, which means concurrent access to a pool may suffer contention on that lock 😭. The pool API provides support for caching free pool items on each CPU, and you can enable that to mitigate against this contention. It's giving "stop fighting over one lock" energy.

When per CPU caches are enabled on a pool, each CPU maintains an active and inactive list of free pool items. A global depot of free lists is initialised in the pool structure to store excess lists of free items that may accumulate on CPUs.

pool_cache_init() allocates the free lists on each CPU, initialises the global depot of free lists, and enables their use to handle pool_get(9) and pool_put(9) operations. Slay.

pool_cache_destroy() disables the use of the free lists on each CPU, returns items cached on all the free lists in the subsystem back to the normal pool allocator, and finally frees the per CPU data structures. Cleanup era 💅.

Once per CPU caches are enabled, items returned to a pool with pool_put(9) are placed on the current CPU's active free list. If the active list becomes full, it becomes the inactive list and a new active list is initialised for the free item to go on. If an inactive list already exists when the active list becomes full, the inactive list is moved to the global depot of free lists before the active list is moved into its place.

Attempts to allocate items with pool_get(9) first try to get an item from the active free list on the CPU it is called on. If the active free list is empty but an inactive list of items is available, the inactive list is moved back into place as the active list so it can satisfy the request. If no lists are available on the current CPU, an attempt to allocate a free list from the global depot is made. Finally, if no free list is available, pool_get(9) falls through to allocating a pool item normally.

The maximum number of items cached on a free list is dynamically scaled for each pool based on the contention on the lock around the global depot of free lists. A garbage collector runs periodically to recover idle free lists and make the memory they consume available to the system for use elsewhere. Very mindful, very demure 🧚.

Information about the current state of the per CPU caches and counters of operations they handle are available via sysctl(2), or displayed in the pcache view in systat(1).

🧺 The kinfo_pool_cache struct 🧺

The kinfo_pool_cache struct provides information about the global state of a pool's caches via a node for each pool under the CTL_KERN, KERN_POOL, KERN_POOL_CACHE sysctl(2) MIB hierarchy.

struct kinfo_pool_cache {
        uint64_t        pr_ngc;
        unsigned int    pr_len;
        unsigned int    pr_nitems;
        unsigned int    pr_contention;
};

• pr_ngc — indicates the number of times the garbage collector has recovered an idle item free list.
• pr_len — shows the maximum number of items that can be cached on a CPU's active free list.
• pr_nitems — shows the number of free items that are currently stored in the global depot.
• pr_contention — indicates the number of times that there was contention on the lock protecting the global depot.

🧸 The kinfo_pool_cache_cpus struct 🧸

The kinfo_pool_cache_cpus struct provides information about the number of times the cache on a CPU handled certain operations. These counters may be accessed via a node for each pool under the CTL_KERN, KERN_POOL, KERN_POOL_CACHE_CPUS sysctl(2) MIB hierarchy. This sysctl returns an array of kinfo_pool_cache_cpus structures sized by the number of CPUs found in the system. The number of CPUs in the system can be read from the CTL_HW, HW_NCPUFOUND sysctl MIB.

And yes, the prose says kinfo_pool_cache_cpus while the struct tag in the code block is kinfo_pool_cache_cpu — I'm keeping both exactly as printed 👀💀

struct kinfo_pool_cache_cpu {
        unsigned int    pr_cpu;
        uint64_t        pr_nget;
        uint64_t        pr_nfail;
        uint64_t        pr_nput;
        uint64_t        pr_nlget;
        uint64_t        pr_nlfail;
        uint64_t        pr_nlput;
};

• pr_cpu — indicates which CPU performed the relevant operations.
• pr_nget and pr_nfail — show the number of times the CPU successfully or unsuccessfully handled a pool_get(9) operation respectively.
• pr_nput — shows the number of times the CPU handled a pool_put(9) operation.
• pr_nlget and pr_nlfail — show the number of times the CPU successfully or unsuccessfully requested a list of free items from the global depot.
• pr_nlput — shows the number of times the CPU pushed a list of free items to the global depot.

🌈 CONTEXT 🌈

pool_cache_init() and pool_cache_destroy() can be called from process context. No cap.

🔥 CODE REFERENCES 🔥

The pool implementation is in the file sys/kern/subr_pool.c.

👑 SEE ALSO 👑

• systat(1)
• sysctl(2)
• pool_get(9)

💀 CAVEATS 💀

Because the intention of per CPU pool caches is to avoid having all CPUs coordinate via shared data structures for handling pool_get(9) and pool_put(9) operations, any limits set on the pool with pool_sethardlimit(9) are ignored. If limits on the memory used by a pool with per CPU caches enabled are needed, they must be enforced by a page allocator specified when a pool is set up with pool_init(9). That's the tea, bestie 🍓.

---

okay that's the whole page! Source/version/date sign-off: this came from Debian, dated January 12, 2018, and the footer page name was POOL_CACHE_INIT(9). slay 💅✨