UVM_HOTPLUG(9)

📦 version 11.0, in nixpkgs unstable

💖 UVM_HOTPLUG(9) — memory hotplug manager 💖

okay bestie, welcome to UVM_HOTPLUG(9) from the Kernel Developer's Manual. It's literally the page for the kernel's memory hotplug manager, and it is giving experimental main character energy. 🫶

✨ NAME ✨

gather round, because the whole hotplug squad is here: uvm_physseg_init, uvm_physseg_valid_p, uvm_physseg_get_start, uvm_physseg_get_end, uvm_physseg_get_avail_start, uvm_physseg_get_avail_end, uvm_physseg_get_pg, uvm_physseg_get_pmseg, uvm_physseg_get_free_list, uvm_physseg_get_start_hint, uvm_physseg_set_start_hint, uvm_physseg_get_next, uvm_physseg_get_prev, uvm_physseg_get_first, uvm_physseg_get_last, uvm_physseg_get_highest_frame, uvm_physseg_find, uvm_page_physload, uvm_page_physunload, uvm_page_physunload_force, uvm_physseg_plug, uvm_physseg_unplug, uvm_physseg_set_avail_start, and uvm_physseg_set_avail_end — together they're the memory hotplug manager. 💅

🌸 SYNOPSIS 🌸

#include <uvm/uvm_physseg.h>

void
uvm_physseg_init(void);

uvm_physseg_t
uvm_page_physload(paddr_t start, paddr_t end, paddr_t avail_start,
    paddr_t avail_end, int free_list);

bool
uvm_page_physunload(uvm_physseg_t upm, int free_list, paddr_t *paddrp);

bool
uvm_page_physunload_force(uvm_physseg_t upm, int free_list,
    paddr_t *paddrp);

bool
uvm_physseg_plug(paddr_t pfn, size_t npages, uvm_physseg_t *upmp);

bool
uvm_physseg_unplug(paddr_t pfn, size_t npages);

💅 DESCRIPTION 💅

So these utility routines are the ones that let you tell uvm(9) about system memory segments. Obsessed. When the kernel is compiled with 'options UVM_HOTPLUG', memory segments are handled in a dynamic data structure (rbtree(3)) compared to a static array when not. That means kernel code can add or remove information about memory segments at any point after boot — hence, “hotplug”. ✨

uvm_page_physload(), uvm_page_physunload(), and uvm_page_physunload_force() are legacy interfaces which may be removed in the future. They must never be used after uvm_init(9). no cap, that part is serious.

WARNING: This is an experimental feature and should not be used in production environments. Furthermore, attempting to "hotplug" without 'options UVM_HOTPLUG' after boot will almost certainly end in a panic(9). 💀

🎀 USAGE 🎀

INITIALIZING HOTPLUG

The function uvm_physseg_init() initializes the hotplug subsystem. This is expected to happen exactly once, at boot time, and from MD code. 🧚

PLUGGING IN MEMORY

uvm_page_physload() registers uvm(9) with a memory segment span, and on a specified free_list. It must be called at system boot time as part of setting up memory management. The arguments describe the start and end of the physical addresses of the segment, and the available start and end addresses of pages not already in use. If a system has memory banks of different speeds the slower memory should be given a higher free_list value.

The arguments are:

  • start — Starting page frame number of the physical memory segments.
  • end — Ending page frame number of the physical memory segments.
  • avail_start — Available starting page frame number of the physical memory segments.
  • avail_end — Available ending page frame number of the physical memory segments.
  • free_list — The free list types are defined in the Machine Dependent code.

This function returns a valid uvm_physseg_t handle when a successful plug occurs, else it will return UVM_PHYSSEG_TYPE_INVALID when the plug fails. 🫶

uvm_physseg_plug() registers uvm(9) with a memory segment span. It can also be called to initiate a hotplug and register a newly "hotplugged" physical memory range into the VM. Unlike uvm_page_physload() this function can, if 'options UVM_HOTPLUG' is enabled at compile time, be used after uvm_init(9). The arguments describe the start page frame, the number of pages to plug starting from the start page frame and an optional return variable, which points to a valid uvm_physseg_t handle when a successful plug occurs.

The arguments are:

  • pfn — Starting page frame number of the physical memory segment.
  • npages — Total number of pages from the starting page frame number to plug in.
  • upmp — If upmp is not NULL, then on a successful plug, a valid pointer to the uvm_physseg_t handle for the segment which was plugged is returned.

This function returns true when a successful plug occurs, false otherwise. ⭐

UNPLUGGING MEMORY

The functions uvm_page_physunload(), uvm_page_physunload_force(), and uvm_physseg_unplug() make uvm(9) forget about previously registered memory segments or portions of such. 🦋

uvm_page_physunload() unloads pages from a segment (from the front or from the back) depending on its availability. When the last page is removed, the segment handle is invalidated and supporting metadata is freed.

Note: This function can only be used during boot time. Pages, once unloaded, are unregistered from uvm and are therefore assumed to be managed by the code which called uvm_page_physunload(9) (usually boot time MD code, for boottime memory "allocation"). 🙈

The arguments are:

  • upm — The handle identifying segment from which we are trying to unload memory.
  • free_list — The free list types are defined in the Machine Dependent code.
  • paddrp — The pointer to the physical address that was unloaded.

If the unload was successful, true is returned, false otherwise. 💜

uvm_page_physunload_force() unconditionally unloads pages from a segment. When the last page is removed, the segment handle is invalidated and supporting metadata is freed.

Note: This function can only be used during boot time. Pages, once unloaded, are unregistered from uvm and are therefore assumed to be managed by the code which called uvm_page_physunload_force(9) (usually boot time MD code, for boottime memory "allocation"). 🙈

The arguments are:

  • upm — The handle identifying segment from which we are trying to unload memory.
  • free_list — The free list types are defined in the Machine Dependent code.
  • paddrp — The pointer to the physical address that was unloaded.

If the unload was successful true is returned, false otherwise. 🍓

uvm_physseg_unplug() can be called to unplug an existing physical memory segment. Unlike uvm_page_physunload() and uvm_page_physunload_force(), it can be called after uvm_init(9), if 'options UVM_HOTPLUG' is enabled at compile time. uvm_hotplug(9) makes no effort to manage the state of the underlying physical memory. It is up to the caller to ensure that it is not in use, either by uvm(9), or by any other sub-system. Further, any hardware quiescing that may be required is the responsibility of MD code. The arguments describe the start page frame and the number of pages to unplug.

The arguments are:

  • pfn — Starting page frame number of the physical memory segment.
  • npages — Total number of pages from the starting page frame number to unplug.

Returns true or false depending on success or failure respectively. 🌈

✨ UTILITY FUNCTIONS ✨

bool
uvm_physseg_valid_p(uvm_physseg_t upm)

paddr_t
uvm_physseg_get_start(uvm_physseg_t upm)

paddr_t
uvm_physseg_get_end(uvm_physseg_t upm)

paddr_t
uvm_physseg_get_avail_start(uvm_physseg_t upm)

paddr_t
uvm_physseg_get_avail_end(uvm_physseg_t upm)

struct vm_page *
uvm_physseg_get_pg(uvm_physseg_t upm, paddr_t index)

struct pmap_physseg *
uvm_physseg_get_pmesg(uvm_physseg_t upm)

int
uvm_physseg_get_free_list(uvm_physseg_t upm)

u_int
uvm_physseg_get_start_hint(uvm_physseg_t upm)

bool
uvm_physseg_set_start_hint(uvm_physseg_t upm, u_int start_hint)

uvm_physseg_t
uvm_physseg_get_next(uvm_physseg_t upm)

uvm_physseg_t
uvm_physseg_get_prev(uvm_physseg_t upm)

uvm_physseg_t
uvm_physseg_get_first(void)

uvm_physseg_t
uvm_physseg_get_last(void)

paddr_t
uvm_physseg_get_highest_frame(void)

paddr_t
uvm_physseg_find(paddr pframe, psize_t *offsetp)

void
uvm_physseg_set_avail_start(uvm_physseg_t upm, paddr_t avail_start)

void
uvm_physseg_set_avail_end(uvm_physseg_t upm, paddr_t avail_end)
  • uvm_physseg_valid_p() — vibe check for a handle: returns true if the uvm_physseg_t you passed in is valid, and false if it's not. 💅
  • uvm_physseg_get_start() — if a valid uvm_physseg_t handle is passed in, it returns the starting physical address of the segment as a paddr_t. If the handle is invalid, you get (paddr_t) -1. No cap. ✨
  • uvm_physseg_get_end() — same deal but for the ending physical address. Valid handle → paddr_t end address; invalid handle → (paddr_t) -1. 🎀
  • uvm_physseg_get_avail_start() — returns the available starting physical address of the segment as paddr_t when the handle is valid. Invalid handle? (paddr_t) -1. 🦋
  • uvm_physseg_get_avail_end() — returns the available ending physical address of the segment as paddr_t for a valid handle. Invalid handle gives (paddr_t) -1. 🌸
  • uvm_physseg_get_pg() — give it a valid uvm_physseg_t handle plus an index, and it returns the struct vm_page * object chillin' at that location. 🫶
  • uvm_physseg_get_pmseg() — okay so the signature above says uvm_physseg_get_pmesg but the description calls it uvm_physseg_get_pmseg() — iconic typo energy, we keep both. With a valid handle it returns the struct pmap_physseg * object contained in the handle. 😭
  • uvm_physseg_get_free_list() — valid handle in, and it returns the free_list type that the current segment is associated with. Return type is int. 🔥
  • uvm_physseg_get_start_hint() — valid handle in, and it returns the start_hint type for the current segment. Return type is u_int. ⭐
  • uvm_physseg_set_start_hint() — pass a valid handle plus a start_hint, and the value gets set in the segment. Returns true to say “slay, it worked”. Invalid handle → false. 💀
  • uvm_physseg_get_next() — valid handle → next valid uvm_physseg_t handle in the sequence. But if the handle you passed is the last segment in the sequence, it returns UVM_PHYSSEG_TYPE_INVALID_OVERFLOW. Passing an invalid handle is not fatal and returns UVM_PHYSSEG_TYPE_INVALID. 🙈
  • uvm_physseg_get_prev() — valid handle → previous valid uvm_physseg_t handle in the sequence. If you're at the first segment, it returns UVM_PHYSSEG_TYPE_INVALID_EMPTY. Passing an invalid handle is not fatal and returns UVM_PHYSSEG_TYPE_INVALID. 🍓
  • uvm_physseg_get_first() — returns the first valid uvm_physseg_t handle in the sequence. If there are no valid handles in the sequence yet, you get UVM_PHYSSEG_TYPE_INVALID_EMPTY. 🧚
  • uvm_physseg_get_last() — returns the last valid uvm_physseg_t handle in the sequence. If there are no valid handles in the sequence yet, it's UVM_PHYSSEG_TYPE_INVALID_EMPTY. 💜
  • uvm_physseg_get_highest_frame() — returns the frame number of the highest registered physical page frame, type paddr_t. There's a lil XXX: note: “Searching on empty sequences are not yet processed in the function.” 👑
  • uvm_physseg_find() — searches for a given segment containing the page frame (paddr_t) you pass in. If a segment that falls between the starting and ending addresses is found, the corresponding uvm_physseg_t handle is returned; otherwise UVM_PHYSSEG_TYPE_INVALID is returned. The second parameter, if not set to NULL, gets the offset value of the page frame passed in with respect to the starting address set to the appropriate psize_t value if the search was successful in finding the segment. 🔍
  • uvm_physseg_set_avail_start() — if you pass a valid uvm_physseg_t handle along with the available starting physical address of the segment (type paddr_t), the value is set in the segment. 🌈
  • uvm_physseg_set_avail_end() — same but for the available ending physical address of the segment (type paddr_t). Valid handle → value set in the segment. 💖

🌸 NOTES 🌸

okay so the big never energy here: uvm_physseg_plug() and uvm_physseg_unplug() must never be used after uvm_init(9) in a kernel build where 'options UVM_HOTPLUG' is not enabled. Periodt. 🚨

🩺 DIAGNOSTICS 🩺

  • Tests for uvm_physseg_init are in tests/sys/uvm. 🩺
  • Unit / functional tests live in tests/sys/uvm/t_uvm_physseg.c. These tests focus on the expected working of the uvm_physseg_init API and its utility functions. 💅
  • Load tests can be found in tests/sys/uvm/t_uvm_physseg_load.c. These tests focus on stressing the uvm_physseg_init implementation in order to make performance comparisons between kernel builds with and without 'options UVM_HOTPLUG'. 🔥

🔍 CODE REFERENCES 🔍

The uvm hotplug feature is implemented in sys/uvm/uvm_physseg.c. The uvm hotplug API is exported via sys/uvm/uvm_physseg.h. iykyk. 🫶

👀 SEE ALSO 👀

📜 HISTORY 📜

This API emerged out of the need to insert new pages at runtime in the Xen x86/balloon(4) driver. iconic origin story tbh. ✨

👑 AUTHORS 👑

  • Cherry G. Mathew ⟨cherry@NetBSD.org⟩ — designed and integrated the API. 👑
  • Santhosh N. Raju ⟨santhosh.raju@gmail.com⟩ — implemented the dynamic segment handling code and all tests for this API. 🧚
  • Nick Hudson ⟨skrll@NetBSD.org⟩ — contributed bugfixes and testing on a wide range of hardware ports. 🫶

That's the whole page, bestie! This edition is from Debian, dated January 17, 2020, and the manpage is UVM_HOTPLUG(9). Stay iconic! 💖✨

📦 where this page comes from