MEMKINDALLOCATOR(3)

💅 MEMKINDALLOCATOR — libmemkind::static_kind::allocator<T> 💅

okay bestie, welcome to MEMKINDALLOCATOR(3), the man page you didn't know you needed but absolutely do 🫶 this is the whole vibe on the C++ allocator that plugs straight into your STL containers so they can pull memory from static kinds. buckle up, it's giving ✨systems programming✨ but make it cute.

---

✨ NAME ✨

libmemkind::static_kind::allocator<T> — the C++ allocator that plays nice with the C++ standard library allocator concepts 💖

Note: memkind_allocator.h functionality is considered as stable API (STANDARD API). no cap, it's official.

---

🌸 SYNOPSIS 🌸

#include <memkind_allocator.h>

Link with -lmemkind

libmemkind::static_kind::allocator(libmemkind::kinds kind);
template <typename U> libmemkind::static_kind::allocator<T>::allocator(const libmemkind::static_kind::allocator<U>&) noexcept;
template <typename U> libmemkind::static_kind::allocator(const allocator<U>&& other) noexcept;
libmemkind::static_kind::allocator<T>::~allocator();
T *libmemkind::static_kind::allocator<T>::allocate(std::size_t n) const;
void libmemkind::static_kind::allocator<T>::deallocate(T *p, std::size_t n) const;
template <class U, class... Args> void libmemkind::static_kind::allocator<T>::construct(U *p, Args... args) const;
void libmemkind::static_kind::allocator<T>::destroy(T *p) const;

---

🦋 DESCRIPTION 🦋

So libmemkind::static_kind::allocator<T> is the allocator you throw at your STL containers when you want them allocating from static kinds memory. the memory management underneath is all the memkind library doing its thing — go read memkind(3) if you want the full lore, fr fr.

libmemkind::kinds is what tells the allocator which flavour of memory you want. every kind comes with its own lil personality (different characteristics, you know how it is). here's the whole lineup, no skips:

• libmemkind::kinds::DEFAULT — basic allocation using standard memory and the default page size. the vanilla of the group, and we respect that 💅
• libmemkind::kinds::HIGHEST_CAPACITY — allocates from the NUMA node(s) that has the highest capacity out of every node in the system. biggest fish energy.
• libmemkind::kinds::HIGHEST_CAPACITY_PREFERRED — same as libmemkind::kinds::HIGHEST_CAPACITY, EXCEPT if the highest-capacity NUMA node in the local domain doesn't have enough memory to satisfy your request, the allocation falls back on other memory NUMA nodes. 🫶 Note: for this kind, allocation will not succeed if there are two or more NUMA nodes tied for the highest capacity. no tiebreakers here bestie, it just gives up.
• libmemkind::kinds::HIGHEST_CAPACITY_LOCAL — allocates from a NUMA node with the highest capacity among all NUMA nodes in the local domain. NUMA nodes share a local domain when they're for the same set of CPUs (think socket or sub-NUMA cluster). Note: if multiple NUMA nodes in the same local domain tie for highest capacity, allocation goes to the NUMA node with the worse latency attribute. this kind needs the locality info described in SYSTEM CONFIGURATION, so don't skip that section.
• libmemkind::kinds::HIGHEST_CAPACITY_LOCAL_PREFERRED — same as libmemkind::kinds::HIGHEST_CAPACITY_LOCAL, except if the highest-capacity NUMA node can't satisfy the request, it falls back on other memory NUMA nodes. the "and if not, whatever" version ✨
• libmemkind::kinds::LOWEST_LATENCY_LOCAL — allocates from a NUMA node with the lowest latency among all NUMA nodes in the local domain. NUMA nodes have the same local domain for a set of CPUs associated with them, e.g. socket or sub-NUMA cluster. Note: if multiple NUMA nodes in that local domain tie for lowest latency, allocation goes to the NUMA node with smaller memory capacity. this kind requires the locality <em>and</em> memory performance characteristics info from SYSTEM CONFIGURATION.
• libmemkind::kinds::LOWEST_LATENCY_LOCAL_PREFERRED — same as libmemkind::kinds::LOWEST_LATENCY_LOCAL, except if the lowest-latency NUMA node doesn't have enough memory, it falls back on other memory NUMA nodes.
• libmemkind::kinds::HIGHEST_BANDWIDTH_LOCAL — allocates from a NUMA node with the highest bandwidth among all NUMA nodes in the local domain. same local-domain deal: NUMA nodes have the same local domain for a set of CPUs associated with them, e.g. socket or sub-NUMA cluster. Note: if multiple NUMA nodes in the same local domain tie for highest bandwidth, allocation goes to the NUMA node with smaller memory capacity. this kind needs the locality and memory performance characteristics info from SYSTEM CONFIGURATION.
• libmemkind::kinds::HIGHEST_BANDWIDTH_LOCAL_PREFERRED — same as libmemkind::kinds::HIGHEST_BANDWIDTH_LOCAL, except if the highest-bandwidth NUMA node can't satisfy the request, it falls back on other memory NUMA nodes.
• libmemkind::kinds::HUGETLB — allocates from standard memory using huge pages. Note: this kind needs the huge pages configuration from SYSTEM CONFIGURATION, bestie, go set it up.
• libmemkind::kinds::INTERLEAVE — allocates pages interleaved across all NUMA nodes, with transparent huge pages disabled.
• libmemkind::kinds::HBW — allocates from the closest high bandwidth memory NUMA node at the time of allocation. if there's not enough high bandwidth memory to satisfy the request, errno is set to ENOMEM and the allocated pointer is set to NULL. 😭 Note: this kind requires the memory performance characteristics info described in SYSTEM CONFIGURATION.
• libmemkind::kinds::HBW_ALL — same as libmemkind::kinds::HBW, except the decision about which NUMA node is "closest" gets postponed until the first write. main character energy, arrives fashionably late.
• libmemkind::kinds::HBW_HUGETLB — same as libmemkind::kinds::HBW, except the allocation is backed by huge pages. Note: needs the huge pages configuration from SYSTEM CONFIGURATION.
• libmemkind::kinds::HBW_ALL_HUGETLB — the combo meal: libmemkind::kinds::HBW_ALL properties + libmemkind::kinds::HBW_HUGETLB properties together. Note: yep, needs the huge pages configuration from SYSTEM CONFIGURATION.
• libmemkind::kinds::HBW_PREFERRED — same as libmemkind::kinds::HBW, except if there's not enough high bandwidth memory, the allocation falls back on standard memory. the chill one 🌈
• libmemkind::kinds::HBW_PREFERRED_HUGETLB — same as libmemkind::kinds::HBW_PREFERRED, except the allocation is backed by huge pages. Note: huge pages configuration from SYSTEM CONFIGURATION required, you know the drill.
• libmemkind::kinds::HBW_INTERLEAVE — same as libmemkind::kinds::HBW, except the pages supporting the allocation are interleaved across all high bandwidth nodes, and transparent huge pages are disabled.
• libmemkind::kinds::REGULAR — allocates from regular memory using the default page size. "regular" here means general purpose memory from the NUMA nodes that contain CPUs. just a normal girl, living in a normal NUMA node 💅
• libmemkind::kinds::DAX_KMEM — allocates from the closest persistent memory NUMA node at the time of allocation. if there's not enough memory in that closest persistent memory NUMA node, errno is set to ENOMEM and the allocated pointer is set to NULL. 😭
• libmemkind::kinds::DAX_KMEM_ALL — allocates from the closest persistent memory NUMA node available at the time of allocation. if there's not enough memory on <em>any</em> of the persistent memory NUMA nodes, errno is set to ENOMEM and the allocated pointer is set to NULL.
• libmemkind::kinds::DAX_KMEM_PREFERRED — same as libmemkind::kinds::DAX_KMEM, except if the closest persistent memory NUMA node can't satisfy the request, the allocation falls back on other memory NUMA nodes. Note: for this kind, allocation will not succeed if two or more persistent memory NUMA nodes are at the same shortest distance from the same CPU that the process is eligible to run on. that eligibility check happens when the application starts, so like… plan accordingly 🫶
• libmemkind::kinds::DAX_KMEM_INTERLEAVE — same as libmemkind::kinds::DAX_KMEM, except the pages supporting the allocation are interleaved across all persistent memory NUMA nodes.

---

every public member type and function matches up with the standard library allocator concepts and definitions, iconic behaviour honestly. the current implementation supports the C++11 standard. periodt.

Template arguments:

• T is an object type aliased by value_type.
• U is an object type.

Note:

T *libmemkind::static_kind::allocator<T>::allocate(std::size_t n) allocates memory using memkind_malloc(). it throws std::bad_alloc when:

• n = 0
• or there isn't enough memory to satisfy the request.

libmemkind::static_kind::allocator<T>::deallocate(T *p, std::size_t n) deallocates memory associated with the pointer returned by allocate(), using memkind_free().

---

🧚 SYSTEM CONFIGURATION 🧚

the interfaces for getting 2MB (HUGETLB) memory need huge pages actually allocated in the kernel's huge page pool. so like, do that first bestie.

HUGETLB (huge pages)

you can read the current number of "persistent" huge pages from the /proc/sys/vm/nr_hugepages file. the proposed way of setting hugepages is:

sudo sysctl vm.nr_hugepages=<number_of_hugepages>

more info can be found here: https://www.kernel.org/doc/Documentation/vm/hugetlbpage.txt

the interfaces for obtaining locality information are provided by the libhwloc dependency. functionality based on locality requires that the memkind library is configured and built with support for libhwloc:

./configure --enable-hwloc

the interfaces for obtaining memory performance characteristics information are based on HMAT (Heterogeneous Memory Attribute Table) — https://uefi.org/sites/default/files/resources/ACPI_6_3_final_Jan30.pdf. functionality based on memory performance characteristics requires that the platform configuration <em>fully</em> supports HMAT, AND that the memkind library is configured and built with support for libhwloc:

./configure --enable-hwloc

Note: for a given target NUMA Node, the OS exposes <em>only</em> the performance characteristics of the best performing NUMA node. that's it, that's the tea ☕

libhwloc can be reached on: https://www.open-mpi.org/projects/hwloc

---

👑 COPYRIGHT 👑

Copyright (C) 2019 - 2021 Intel Corporation. All rights reserved.

---

🌈 SEE ALSO 🌈

memkind(3)

---

and that's the whole page, bestie!! 🎀 this was brought to you by Intel Corporation, dated 2019-09-24, and the manual is MEMKINDALLOCATOR(3). go forth and allocate responsibly, ily 💖✨🦋