IBV_REG_MR(3)

💖 IBV_REG_MR(3) 💖 — libibverbs Programmer's Manual

okay so bestie, welcome to the page where we register memory regions 💅 this is ibv_reg_mr and friends from the libibverbs Programmer's Manual, and no cap, if you're doing RDMA you're gonna need this one. sit down, grab a boba, let's go 🧋✨

---

🌸 NAME 🌸

ibv_reg_mr, ibv_reg_mr_iova, ibv_reg_dmabuf_mr, ibv_reg_mr_ex, ibv_dereg_mr — register or deregister a memory region (MR) 🫶

like, you're telling the NIC "hey, this chunk of memory? it's mine, it's registered, please respect it" 👑

---

✨ SYNOPSIS ✨

#include <infiniband/verbs.h>

struct ibv_mr *ibv_reg_mr(struct ibv_pd *pd, void *addr,
                          size_t length, int access);

struct ibv_mr *ibv_reg_mr_iova(struct ibv_pd *pd, void *addr,
                               size_t length, uint64_t hca_va,
                               int access);

struct ibv_mr *ibv_reg_dmabuf_mr(struct ibv_pd *pd, uint64_t offset,
                                 size_t length, uint64_t iova,
                                 int fd, int access);

struct ibv_mr *ibv_reg_mr_ex(struct ibv_pd *pd, struct ibv_mr_init_attr *mr_init_attr");

int ibv_dereg_mr(struct ibv_mr *mr);

(yes that last mr_init_attr"); is exactly how it shows up, we do not edit the sacred text 🙈 copy-paste it as-is and the compiler will decide your fate)

---

🦋 DESCRIPTION 🦋

registering, the classic way 💫

ibv_reg_mr() registers a memory region (MR) that belongs to the protection domain pd. The MR's starting address is addr and its size is length — so like, from here to here, that's the vibe. The argument access describes the desired memory protection attributes; it's either 0 or the bitwise OR of one or more of these flags ⬇️

• IBV_ACCESS_LOCAL_WRITE — Enable Local Write Access ✍️
• IBV_ACCESS_REMOTE_WRITE — Enable Remote Write Access 📡
• IBV_ACCESS_REMOTE_READ — Enable Remote Read Access 👀
• IBV_ACCESS_REMOTE_ATOMIC — Enable Remote Atomic Operation Access (if supported) ⚛️
• IBV_ACCESS_FLUSH_GLOBAL — Enable Remote Flush Operation with global visibility placement type (if supported) 🌍
• IBV_ACCESS_FLUSH_PERSISTENT — Enable Remote Flush Operation with persistence placement type (if supported) 💾
• IBV_ACCESS_MW_BIND — Enable Memory Window Binding 🪟
• IBV_ACCESS_ZERO_BASED — Use byte offset from beginning of MR to access this MR, instead of a pointer address 🅾️
• IBV_ACCESS_ON_DEMAND — Create an on-demand paging MR 🍽️
• IBV_ACCESS_HUGETLB — Huge pages are guaranteed to be used for this MR, applicable with IBV_ACCESS_ON_DEMAND in explicit mode only 🐘
• IBV_ACCESS_RELAXED_ORDERING — This setting allows the NIC to relax the order that data is transferred between the network and the target memory region. Relaxed ordering allows network initiated writes (such as incoming message send or RDMA write operations) to reach memory in an arbitrary order. This can improve the performance of some applications. However, relaxed ordering has the following impact: RDMA write-after-write message order is no longer guaranteed. (Send messages will still match posted receive buffers in order.) Back-to-back network writes that target the same memory region leave the region in an unknown state. Relaxed ordering does not change completion semantics, such as data visibility. That is, a completion still ensures that all data is visible, including data from prior transfers. Relaxed ordered operations will also not bypass atomic operations. 🌀

bestie listen — if IBV_ACCESS_REMOTE_WRITE or IBV_ACCESS_REMOTE_ATOMIC is set, then IBV_ACCESS_LOCAL_WRITE must be set too. no exceptions, the kernel is not playing 😭

local read access is always enabled for the MR, always, no flag needed, it comes free with the fit 💁‍♀️

to create an implicit ODP MR, IBV_ACCESS_ON_DEMAND should be set, addr should be 0 and length should be SIZE_MAX 🫧

if IBV_ACCESS_HUGETLB is set, then the application is aware that for this MR all pages are huge and must promise it will never do anything to break huge pages 🤞 (this is a pinky promise with the kernel, fr fr)

---

ibv_reg_mr_iova() — the "i'll pick my own address" edition 🎀

ibv_reg_mr_iova is the same as the normal reg_mr, except that the user is allowed to specify the virtual base address of the MR when accessed through a lkey or rkey. The offset in the memory region is computed as 'addr + (iova - hca_va)'. Specifying 0 for hca_va has the same effect as IBV_ACCESS_ZERO_BASED ⭐

---

ibv_reg_dmabuf_mr() — dma-buf girlies rise 🌈

ibv_reg_dmabuf_mr() registers a dma-buf based memory region (MR) associated with the protection domain pd. The MR starts at offset of the dma-buf and its size is length. The dma-buf is identified by the file descriptor fd. The argument iova specifies the virtual base address of the MR when accessed through a lkey or rkey. It must have the same page offset as offset. 👑

The argument access describes the desired memory protection attributes; it's similar to the ibv_reg_mr case except that only the following flags are supported: IBV_ACCESS_LOCAL_WRITE, IBV_ACCESS_REMOTE_WRITE, IBV_ACCESS_REMOTE_READ, IBV_ACCESS_REMOTE_ATOMIC, IBV_ACCESS_RELAXED_ORDERING.

not me listing the allowed flags twice, it's giving "read the menu before you order" 💀

---

ibv_reg_mr_ex() — the everything bagel 🥯

ibv_reg_mr_ex is an API that enables all the variants of the other ibv_reg_mr_xxx() that are described in this man page. It has the mr_init_attr->comp_mask field to let the application mark which fields are applicable. In addition, it includes the mr_init_attr->dmah which can be used to include an ibv_dmah object that will be used for that MR. The other fields on the input pointer have the same meaning as the fields described in this man page for the other verbs. main character energy honestly 👑

---

ibv_dereg_mr() — the breakup 💔

ibv_dereg_mr() deregisters the MR mr. it's over, you're freeing it, moving on, slay ✨

---

🔥 RETURN VALUE 🔥

ibv_reg_mr() / ibv_reg_mr_iova() / ibv_reg_dmabuf_mr() / ibv_reg_mr_ex() returns a pointer to the registered MR, or NULL if the request fails 😭

the local key (L_Key) field lkey is used as the lkey field of struct ibv_sge when posting buffers with ibv_post_* verbs, and the the remote key (R_Key) field rkey is used by remote processes to perform Atomic and RDMA operations. The remote process places this rkey as the rkey field of struct ibv_send_wr passed to the ibv_post_send function. 🗝️✨ (that double "the the" is in the source, we keep the lore intact, iykyk)

ibv_dereg_mr() returns 0 on success, or the value of errno on failure (which indicates the failure reason) 💅

---

💀 NOTES 💀

• ibv_dereg_mr() fails if any memory window is still bound to this MR. so unbind first bestie, don't just yank it 🚨
• ibv_dereg_mr_ex() — one among mr_init_attr->fd and mr_init_attr->addr is required, both can't come together. it's one or the other, we don't do situationships here 🙅‍♀️

---

👑 SEE ALSO 👑

ibv_alloc_pd(3), ibv_post_send(3), ibv_post_recv(3), ibv_post_srq_recv(3) 🫶 go read those next, they're the supporting cast

---

💜 AUTHORS 💜

Dotan Barak <dotanba@gmail.com> — thank you for your service legend 🫡🌸

---

that's the whole page bestie, you ate it 💅 this was libibverbs, dated 2006-10-31, and the page is IBV_REG_MR(3) — now go register something responsibly, ily, bye 🎀✨🦋