LIBPMEM2(7)

📦
version 2.1.0 · 7 versions
, in fedora 43

💖 libpmem2 — persistent memory support library 💖

okay bestie, welcome to libpmem2, straight outta the PMDK Programmer's Manual 📖✨. This is the persistent memory support library, and yeah it's a lot, but I got you — grab a snack 🍓, we're doing this together.

✨ SYNOPSIS ✨

#include <libpmem2.h>
cc ... -lpmem2

that's it, that's the whole vibe check. include the header, link the lib, slay 💅

🌸 DESCRIPTION 🌸

So libpmem2 gives you low-level persistent memory (pmem) support for apps that use direct access storage (DAX) — and DAX is storage that supports load/store access without paging blocks from a block storage device. Some types of non-volatile memory DIMMs (NVDIMMs) give you this kind of byte addressable access to storage. Usually a persistent-memory-aware file system is what exposes that direct access to your app. And when you memory map a file from that kind of file system, boom 💥 — you get load/store, non-paged access to pmem.

This library is for apps that use persistent memory directly — no library-supplied transactions, no library-supplied memory allocation, none of that. If you want a higher-level library, there's one that currently builds on libpmem (the previous variation of libpmem2), and it's available and recommended for most applications, see:

  • libpmemobj(7) — a general use persistent memory API, providing memory allocation and transactional operations on variable-sized objects. 👑

The libpmem2 library gives you a comprehensive set of functions for robust use of Persistent Memory. It relies on three core concepts: struct pmem2_src source, struct pmem2_config config and struct pmem2_map map 🫶

  • source — an object describing the data source for mapping. The data source can be a file descriptor or an anonymous mapping. The APIs dedicated for creating source are: pmem2_source_from_fd(3), pmem2_source_from_anon(3).
  • config — an object containing parameters that are used to create a mapping from a source. The configuration structure must always be provided to create a mapping, but the only required parameter to set in the config is granularity. The granularity should by set using the dedicated libpmem2 function pmem2_config_set_required_store_granularity(3), which defines a maximum permitted granularity requested by the user. For more info about the granularity concept, read the GRANULARITY section below 👇

In addition to the granularity setting, libpmem2 provides multiple optional functions to configure the target mapping, e.g. pmem2_config_set_length(3) to set length which will be used for mapping, or pmem2_config_set_offset(3) which will be used to map the contents from the specified location of the source, or pmem2_config_set_sharing(3) which defines the behavior and visibility of writes to the mapping's pages.

On top of the basic "managing the virtual address mapping" stuff, libpmem2 also gives you optimized functions for modifying the mapped data. That includes data flushing and memory copying, bestie.

To get the proper function for data flushing, use: pmem2_get_flush_fn(3), pmem2_get_persist_fn(3) or pmem2_get_drain_fn(3). To get the proper function for copying to persistent memory, use the map getters: pmem2_get_memcpy_fn(3), pmem2_get_memset_fn(3), pmem2_get_memmove_fn(3).

The libpmem2 API also provides support for the badblock and unsafe shutdown state handling. 😭 (we love a library that plans for the worst)

To read or clear badblocks, the following functions are provided: pmem2_badblock_context_new(3), pmem2_badblock_context_delete(3), pmem2_badblock_next(3) and pmem2_badblock_clear(3).

To handle unsafe shutdown in the application, the following functions are provided: pmem2_source_device_id(3), pmem2_source_device_usc(3). More detailed information about unsafe shutdown detection and unsafe shutdown count can be found in the libpmem2_unsafe_shutdown(7) man page. 📚

🌈 GRANULARITY 🌈

The libpmem2 library introduces the concept of granularity, and through it you may easily distinguish between different levels of storage performance capabilities available to the application, as related to the power-fail protected domain. The way data reaches this protected domain differs based on the platform and storage device capabilities. iykyk 💅

Traditional block storage devices (SSD, HDD) must use system API calls such as msync(), fsync() to write data reliably. Invoking these functions flushes the data to the medium with page granularity. In the libpmem2 library, this type of flushing behavior is called PMEM2_GRANULARITY_PAGE.

In systems with persistent memory support, a power-fail protected domain may cover different sets of resources: either the memory controller, or the memory controller AND CPU caches. For this reason, libpmem2 distinguishes two types of granularity for persistent memory: PMEM2_GRANULARITY_CACHE_LINE and PMEM2_GRANULARITY_BYTE.

If the power-fail protected domain covers only the memory controller, then the CPU appropriate cache lines must be flushed for the data to be considered persistent. This granularity type is called PMEM2_GRANULARITY_CACHE_LINE. Depending on the architecture, there are different types of machine instructions for flushing cache lines (e.g., CLWB, CLFLUSHOPT, CLFLUSH for Intel x86_64 architecture). Usually, to ensure the ordering of stores, such instructions must be followed by a barrier (e.g., SFENCE). ⚡

The third type of granularity, PMEM2_GRANULARITY_BYTE, applies to platforms where the power-fail protected domain covers both the memory controller and the CPU caches. In such cases, cache flush instructions are no longer needed, and the platform itself guarantees the persistence of data. But barriers might still be required for ordering. ✨

The library declares these granularity levels in the pmem2_granularity enum, which the application must set in pmem2_config to the appropriate level for a mapping to succeed. Your software should set this config parameter to a value that most accurately represents the target hardware characteristics and the storage patterns of the application. For example, a database storage engine that operates on large logical pages that reside either on SSDs or PMEM should set this value to PMEM2_GRANULARITY_PAGE. The library will create mappings where the new map granularity is lower or equal to the requested one. For example, a mapping with PMEM2_GRANULARITY_CACHE_LINE can be created for the required granularity PMEM2_GRANULARITY_PAGE, but not vice versa. 🚫

💀 CAVEATS 💀

libpmem2 relies on the library destructor being called from the main thread. For this reason, all functions that might trigger destruction (e.g. dlclose(3)) should be called in the main thread. Otherwise some of the resources associated with that thread might not be cleaned up properly. 😭 don't do it to yourself.

🧚 ENVIRONMENT 🧚

libpmem2 can change its default behavior based on the following environment variables. These are primarily intended for testing and are generally not required.

  • PMEM2_FORCE_GRANULARITY=val

Setting this environment variable to val forces libpmem2 to use the persist method specific for the forced granularity and skip the granularity autodetecting mechanism. The concept of granularity is described in the GRANULARITY section above. This variable is intended for use during library testing.

The val argument accepts the following text values:

  • BYTE — force byte granularity.
  • CACHE_LINE — force cache line granularity.
  • PAGE — force page granularity.

Granularity values listed above are case-insensitive.

NOTE: The value of PMEM2_FORCE_GRANULARITY is not queried (and cached) at library initialization time, but read during each pmem2_map_new(3) call.

This means that PMEM2_FORCE_GRANULARITY may still be set or modified by the program until the first attempt to map a file.

  • PMEM_NO_CLWB=1

Setting this environment variable to 1 forces libpmem2 to never issue the CLWB instruction on Intel hardware, falling back to other cache flush instructions on that hardware instead (CLFLUSHOPT or CLFLUSH). Without this setting, libpmem2 will always use the CLWB instruction for flushing processor caches on platforms that support this instruction. This variable is intended for use during library testing, but may be required for some rare cases when using CLWB has a negative impact on performance. 🙈

  • PMEM_NO_CLFLUSHOPT=1

Setting this environment variable to 1 forces libpmem2 to never issue the CLFLUSHOPT instruction on Intel hardware, falling back to the CLFLUSH instructions instead. Without this environment variable, libpmem2 will always use the CLFLUSHOPT instruction for flushing processor caches on platforms that support the instruction, but where CLWB is not available. This variable is intended for use during library testing.

  • PMEM_NO_MOVNT=1

Setting this environment variable to 1 forces libpmem2 to never use the non-temporal move instructions on Intel hardware. Without this environment variable, libpmem2 will use the non-temporal instructions for copying larger ranges to persistent memory on platforms that support these instructions. This variable is intended for use during library testing.

  • PMEM_MOVNT_THRESHOLD=val

This environment variable allows overriding the minimum length of the pmem2_memmove_fn operations for which libpmem2 uses non-temporal move instructions. Setting this environment variable to 0 forces libpmem2 to always use the non-temporal move instructions if available. It has no effect if PMEM_NO_MOVNT is set to 1. This variable is intended for use during library testing.

🐛 DEBUGGING 🐛

Okay bestie, listen up, because this part is the glow-up. There are two whole versions of libpmem2 chillin' on a dev system and they are not the same girl.

The normal version — the one you get when your program is linked with the -lpmem2 option — is optimized for performance. She's the popular one. She skips checks that would slow her down, and she never logs any trace info, and she does not do run-time assertions. She's fast and she's unbothered. Periodt.

Then there's the second version, and you get to her when your program uses the libraries under /usr/lib/pmdk_debug. She's the one with run-time assertions and trace points. The classic way to slide into her DMs is to set the environment variable LD_LIBRARY_PATH to /usr/lib/pmdk_debug or /usr/lib64/pmdk_debug, whichever one is appropriate for you. ✨

Debugging output is controlled with the following environment variables. And bestie, heads up: these variables have no effect on the non-debug version of the library. She doesn't even see them. 💅

  • PMEM2_LOG_LEVEL — the value of this one enables trace points in the debug version of the library, like this:
    • 0 — this is the default level when PMEM2_LOG_LEVEL isn't set. At this level, no log messages are emitted. Silent. Mysterious. Iconic.
    • 1 — additional details on any errors detected get logged, on top of returning the errno-based errors as usual. The same info may be retrieved using pmem2_errormsg().
    • 2 — a trace of basic operations is logged.
    • 3 — enables a very verbose amount of function call tracing in the library.
    • 4 — enables voluminous and fairly obscure tracing information that is likely only useful to the libpmem2 developers. (Devs-only content, iykyk 🫶)

Unless PMEM2_LOG_FILE is set, debugging output is written to stderr.

  • PMEM2_LOG_FILE — specifies the name of a file where all logging information should be written. Cute detail: if the last character in the name is -, the PID of the current process will be appended to the file name when the log file is created. If PMEM2_LOG_FILE is not set, output is written to stderr.

🌸 EXAMPLE 🌸

For examples using the libpmem2 API, go look at https://pmem.io/pmdk/libpmem2. 🦋

The basic example uses libpmem2 to flush changes made to raw, memory-mapped persistent memory.

⚠️ WARNING: There is nothing transactional about the persist from pmem2_get_persist_fn(3) call in this example. Interrupting the program may result in a partial write to pmem. Use a transactional library such as libpmemobj(7) to avoid torn updates.

Like, genuinely bestie, do not play with that one. 💀

👑 ACKNOWLEDGEMENTS 👑

libpmem2 builds on the persistent memory programming model recommended by the SNIA NVM Programming Technical Work Group: https://snia.org/nvmp 🌈

⭐ SEE ALSO ⭐

and that's the whole page, bestie!! 🎀 this one came from the PMDK girlies and it's dated 2025-07-24 — go forth and map your memory safely, mwah 💖✨💅

📦 where this page comes from