RPMEM_CREATE(3)

💖 RPMEM_CREATE(3) 💖 — PMDK Programmer's Manual, but make it slay

okay so bestie, you're in the librpmem corner of the PMDK Programmer's Manual and honestly? this page is the whole friend group. it's giving rpmem_create(), rpmem_open(), rpmem_set_attr(), rpmem_close(), and rpmem_remove() — the most commonly used functions for remote access to persistent memory 🌈 remote pools from literally across the network, no cap.

---

✨ NAME ✨

rpmem_create(), rpmem_open(), rpmem_set_attr(), rpmem_close(), rpmem_remove() — the most commonly used functions for remote access to persistent memory 🫶

---

🌸 SYNOPSIS 🌸

#include <librpmem.h>

RPMEMpool *rpmem_create(const char *target, const char *pool_set_name,
    void *pool_addr, size_t pool_size, unsigned *nlanes,
    const struct rpmem_pool_attr *create_attr);
RPMEMpool *rpmem_open(const char *target, const char *pool_set_name,
    void *pool_addr, size_t pool_size, unsigned *nlanes,
    struct rpmem_pool_attr *open_attr);
int rpmem_set_attr(RPMEMpool *rpp, const struct rpmem_pool_attr *attr);
int rpmem_close(RPMEMpool *rpp);
int rpmem_remove(const char *target, const char *pool_set_name, int flags);

---

👑 DESCRIPTION 👑

🦋 rpmem_create()

so rpmem_create() makes a remote pool on a given target node, and it uses the pool set file pool_set_name to map that remote pool. and pool_set_name? it's a relative path in the root config directory on the target node, iykyk. for the pool set file format and its options, go peek at poolset(5).

pool_addr is a pointer to the associated local memory pool, and that pool has size pool_size. both pool_addr and pool_size must be aligned to the system's page size (see sysconf(3)) — like, non-negotiable 💅. the size of the remote pool must be at least pool_size. see REMOTE POOL SIZE below for the details, bestie.

nlanes points to the maximum number of lanes the caller is requesting. once the remote pool is successfully created, *nlanes gets set to the maximum number of lanes supported by both the local and remote nodes. see LANES below 👀

the create_attr structure holds the attributes used for creating the remote pool. if create_attr is not NULL, you get a pool with internal metadata 🎀 that metadata lives in the first 4096 bytes of the pool and can be read when you open the remote pool with rpmem_open(). and to stop the user from overwriting the pool metadata, that region is not accessible to the user via rpmem_persist() — hands off, it's protected 🙈

if create_attr is NULL or zeroed, the remote pool set file must contain the NOHDRS option. in that case the remote pool is created with no internal metadata in it, and the entire pool space is available to the user. see rpmem_persist(3) for details.

🦋 rpmem_open()

rpmem_open() opens the existing remote pool with set file pool_set_name on remote node target. pool_set_name is a relative path in the root config directory on the target node. pool_addr is a pointer to the associated local memory pool of size pool_size. both pool_addr and pool_size must be aligned to the system's page size (see sysconf(3)). the size of the remote pool must be at least pool_size. see REMOTE POOL SIZE below for details.

nlanes points to the maximum number of lanes the caller is requesting. upon successful opening of the remote pool, *nlanes is set to the maximum number of lanes supported by both the local and remote nodes. see LANES below, bestie 💜

🦋 rpmem_set_attr()

rpmem_set_attr() overwrites the pool's attributes. the attr structure contains the attributes used for overwriting the remote pool attributes that were passed to rpmem_create() back at pool creation time. if attr is NULL, a zeroed structure with attributes will be used. the new attributes get stored in the pool's metadata ✨

🦋 rpmem_close()

rpmem_close() closes the remote pool rpp. all resources are released on both the local and remote nodes. the remote pool itself persists on the remote node though — it does not disappear, and you can re-open it later using rpmem_open() 🌈

🦋 rpmem_remove()

rpmem_remove() removes the remote pool with set file name pool_set_name from node target. pool_set_name is a relative path in the root config directory on the target node. by default, only the pool part files are removed — the pool set file is left untouched, periodt. if the pool is not consistent, rpmem_remove() fails 😭

the flags argument decides the behavior of rpmem_remove(). flags may be either 0 or the bitwise OR of one or more of these flags:

• RPMEM_REMOVE_FORCE — ignore errors when opening an inconsistent pool. the pool set file must still be in an appropriate format for the pool to be removed, no skipping that part 💅
• RPMEM_REMOVE_POOL_SET — remove the pool set file after removing the pool described by this pool set 🎀

---

🔥 RETURN VALUE 🔥

on success, rpmem_create() returns an opaque handle to the remote pool, for use in subsequent librpmem calls. if any error prevents the remote pool from being created, rpmem_create() returns NULL and sets errno appropriately 💀

on success, rpmem_open() returns an opaque handle to the remote pool for subsequent librpmem calls. if the open_attr argument is not NULL, the remote pool attributes are returned in the provided structure. if the remote pool was created without internal metadata, zeroes are returned in the open_attr structure on a successful call to rpmem_open(). if any error prevents the remote pool from being opened, rpmem_open() returns NULL and sets errno appropriately.

on success, rpmem_set_attr() returns 0. on error, it returns -1 and sets errno appropriately.

on success, rpmem_close() returns 0. on error, it returns a non-zero value and sets errno appropriately.

on success, rpmem_remove() returns 0. on error, it returns a non-zero value and sets errno appropriately.

---

🎀 NOTES 🎀

🌸 REMOTE POOL SIZE 🌸

the size of a remote pool depends on the configuration in the pool set file on the remote node (see poolset(5)). here's the tea:

• if no pool set options are used in the remote pool set file, the remote pool size is the sum of the sizes of all part files, decreased by 4096 bytes per part file. that's because 4096 bytes of each part file are utilized for storing internal metadata.
• if the SINGLEHDR option is used in the remote pool set file, the remote pool size is the sum of sizes of all part files, decreased once by 4096 bytes. in this case only the first part contains internal metadata.
• if a remote pool set file contains the NOHDRS option, the remote pool size is the sum of sizes of all its part files. in this case none of the parts contains internal metadata.

for other consequences of using the SINGLEHDR and NOHDRS options, see rpmem_persist(3). also: RPMEM_MIN_PART and RPMEM_MIN_POOL in <librpmem.h> define the minimum size allowed by librpmem for a part file and a remote pool, respectively — the floor, the bare minimum, the itty bitty limit 💅

🌸 LANES 🌸

the term lane means an isolated path of execution. the underlying hardware utilized by both local and remote nodes may have limited resources that restrict the maximum number of parallel rpmem_persist(3) operations. the maximum number of supported lanes is returned by the rpmem_open() and rpmem_create() function calls.

the caller passes the maximum number of lanes requested in *nlanes. if the pool is successfully created or opened, *nlanes is updated to reflect the minimum of the number of lanes requested by the caller and the maximum number of lanes supported by the underlying hardware. and the application is obligated to use at most the returned number of lanes in parallel — stay in your lane, literally 🫶

rpmem_persist(3) does not provide any locking mechanism; so any serialization of calls must be performed by the application if required. no one else is gonna do it for you, bestie.

each lane requires a separate connection, represented by a file descriptor. if the system runs out of free file descriptors during rpmem_create() or rpmem_open(), these functions will fail 😭 see nofile in limits.conf(5) for more details.

⚠️ heads up: lanes are not free, and file descriptors are finite. plan accordingly or cry.

---

🌈 SEE ALSO 🌈

rpmem_persist(3), sysconf(3), limits.conf(5), libpmemobj(7), librpmem(7) and <https://pmem.io> ✨

---

and that's the whole page, bestie 🎀 this was PMDK - rpmem API version 1.3, dated 2021-09-24, living its best life as RPMEM_CREATE(3). go forth and remote-pool responsibly 💖✨🦋