DPCPU(9)

💖 dpcpu(9) — Kernel Developer's Manual 💖

okay bestie, gather round, because today we're learning about dpcpu, aka the Kernel Dynamic Per-CPU Memory Allocator, and honestly? it's giving main character energy for SMP kernels. no cap. this is the dpcpu(9) page and it's about to be your new obsession. 🦋

---

✨ NAME ✨

dpcpu — Kernel Dynamic Per-CPU Memory Allocator

like, it literally does what it says on the tin, and the tin is iconic. 💅

---

🌸 SYNOPSIS 🌸

#include <sys/pcpu.h>

🎀 Per-CPU Variable Definition and Declaration 🎀

DPCPU_DEFINE(type, name);

DPCPU_DEFINE_STATIC(type, name);

DPCPU_DECLARE(type, name);

⭐ Current CPU Accessor Functions ⭐

DPCPU_PTR(name);

DPCPU_GET(name);

DPCPU_SET(name, value);

🦋 Named CPU Accessor Functions 🦋

DPCPU_ID_PTR(cpu, name);

DPCPU_ID_GET(cpu, name);

DPCPU_ID_SET(cpu, name, value);

---

💜 DESCRIPTION 💜

okay so here's the tea: dpcpu instantiates one whole instance of a global variable for every single CPU in the system. not one shared one, not a fake one, an actual per-CPU one. queen behavior fr fr. 👑

dynamically allocated per-CPU variables get defined with DPCPU_DEFINE(), which defines a variable called name with type type. and when we say arbitrary C types, we mean it — structures and arrays are totally welcome, no gatekeeping here. if you don't provide any initialization, then every per-CPU instance of that variable is zero-filled (i.e., as though it were allocated in BSS):

DPCPU_DEFINE(int, foo_int);

you can also statically initialize values right there in the definition, which makes every per-CPU instance start out with that value:

DPCPU_DEFINE(int, foo_int) = 1;

and for the values that can be defined as static, you gotta use DPCPU_DEFINE_STATIC() — it's the law bestie:

DPCPU_DEFINE_STATIC(int, foo_int);

DPCPU_DECLARE() produces a declaration of the per-CPU variable, which is exactly what you want for sticking in header files. 🎀

the current CPU's variable instance is reachable through three little icons:

• DPCPU_PTR — returns a pointer to the per-CPU instance
• DPCPU_GET — retrieves the value of the per-CPU instance
• DPCPU_SET — sets the value of the per-CPU instance

and if you wanna talk to instances of variables belonging to specific CPUs? that's what DPCPU_ID_PTR, DPCPU_ID_GET, and DPGPU_ID_SET are for — they take an extra CPU ID argument, cpu. 🌈

---

🔥 Synchronization 🔥

🚨 bestie alert 🚨 this part is the one you cannot skip. like, this is the whole vibe check of the page.

so on top of the ordinary synchronization concerns you already have with global variables — which may imply the use of atomic(9), mutex(9), or other kernel synchronization primitives — there's an extra little plot twist: thread migration could dynamically change the instance of a variable being accessed by a thread between operations. yeah. the variable you're poking at can change underneath you. this requires additional care when reasoning about and protecting per-CPU variables. 💀

for example, it may be desirable to protect access using critical_section(9) to prevent both preemption and migration during use. alternatively, it may be desirable to cache the CPU ID at the start of a sequence of accesses, using suitable synchronization to make non-atomic sequences safe in the presence of migration.

and here's the whole thing in code, look at her go:

DPCPU_DEFINE_STATIC(int, foo_int);
DPCPU_DEFINE_STATIC(struct mutex, foo_lock);

void
foo_int_increment(void)
{
    int cpu, value;

    /* Safe as atomic access. */
    atomic_add_int(DPCPU_PTR(foo_int), 1);

    /*
     * Protect with a critical section, which prevents preemption
     * and migration.  However, access to instances from remote CPUs
     * is not safe, as critical sections prevent concurrent access
     * only from the current CPU.
     */
    critical_enter();
    value = DPCPU_GET(foo_int);
    value++;
    DPCPU_SET(foo_int, value);
    critical_exit();

    /*
     * Protect with a per-CPU mutex, tolerating migration, but
     * potentially accessing the variable from multiple CPUs if
     * migration occurs after reading curcpu.  Remote access to a
     * per-CPU variable is safe as long as the correct mutex is
     * acquired.
     */
    cpu = curcpu;
    mtx_lock(DPCPU_ID_PTR(cpu, foo_lock));
    value = DPCPU_ID_GET(cpu, foo_int);
    value++;
    DPCPU_ID_SET(cpu, foo_int);
    mtx_unlock(DPCPU_ID_PTR(cpu, foo_lock));
}

like, read the comments bestie, they're literally telling you the whole story. 🙈

---

👀 SEE ALSO 👀

• atomic(9)
• critical_enter(9)
• mutex(9)

go read these next, they're all in the friend group. 🫶

---

📜 HISTORY 📜

dpcpu was first introduced by Jeff Roberson in FreeBSD 8.0. and this manual page was written by Robert N. M. Watson. legends only. 👑

---

and that's the whole page bestie!! ✨ this was the Debian build of DPCPU(9) from July 5, 2018 — yeah she's an elder but she still ate. 💅🌸💖 periodt. 🦋