io_uring_...ed_buffers(7)

💖 io_uring_provided_buffers — the provided buffer rings overview 💖

okay bestie, so this is a page from the Linux Programmer's Manual about io_uring's provided buffer rings, and honestly? it's giving genius. let's get into it 🫶

---

✨ NAME ✨

io_uring_provided_buffers — the whole entire overview of io_uring provided buffer rings 💅

---

🌸 DESCRIPTION 🌸

So, provided buffer rings let applications hand the kernel a whole pool of buffers, and then the kernel gets to dynamically pick one at operation completion time. No cap. This is lowkey perfect for operations where you literally cannot know upfront how much buffer you're gonna need — like receiving data from network sockets, or reading from pipes. 🌈

🦋 Why use provided buffers?

Traditional I/O is so rigid bestie — the app has to specify a buffer at the moment it submits the request. But for receive operations on sockets or reads from pipes, that's a whole problem: you don't know how much data is gonna show up, so you're stuck doing one of these:

• Allocate a big huge buffer for every single pending operation, which is just wasting memory 😭
• Use tiny little buffers and then potentially need multiple operations
• Limit how many pending operations you have, just to control memory usage

Provided buffer rings fix all of that by letting the kernel select an appropriately-sized buffer from a shared pool at completion time. Multiple operations can share the same buffer pool, and buffers are only consumed when data actually arrives. iconic behaviour honestly ✨

Provided buffers are most beneficial for:

• Network servers with so many concurrent connections 👑
• Applications receiving variable-length messages
• Scenarios where memory efficiency is important

🧚 Buffer ring concepts

A provided buffer ring is a circular buffer, shared between the application and the kernel:

• The application adds buffers to the ring by writing entries and advancing the tail
• The kernel consumes buffers from the ring by reading entries and advancing the head
• Each buffer has a unique buffer ID (bid) within its buffer group
• Buffer groups are identified by a buffer group ID (bgid)

Multiple buffer rings can exist simultaneously, each one with a different buffer group ID. Operations then specify which buffer group to use. 💜

🎀 Setting up a buffer ring

Buffer rings get set up with io_uring_setup_buf_ring(3), which handles the allocation, the registration, and the initialization for you:

struct io_uring_buf_ring *br;
int bgid = 1;  /* buffer group ID */
int err;

br = io_uring_setup_buf_ring(ring, 128, bgid, 0, &err);
if (!br) {
    fprintf(stderr, "buffer ring setup failed: %d\n", err);
    return err;
}

⭐ The ring must have a power-of-two number of entries, up to a maximum of 32768 (2^15). that's the vibe check, don't fight it.

Alternatively, applications can use io_uring_register_buf_ring(3) for more control over the setup process, including kernel-allocated rings using the IOU_PBUF_RING_MMAP flag. fancy 💅

🍓 Adding buffers to the ring

Buffers get added with io_uring_buf_ring_add(3), and then made visible to the kernel with io_uring_buf_ring_advance(3):

int mask = io_uring_buf_ring_mask(128);

for (int i = 0; i < 128; i++) {
    void *buf = malloc(4096);
    io_uring_buf_ring_add(br, buf, 4096, i, mask, i);
}
io_uring_buf_ring_advance(br, 128);

Each buffer gets assigned a buffer ID — that's the third parameter. Buffer IDs should be unique within the buffer group, but they can totally be reused once a buffer gets returned. ♻️

🔥 Using provided buffers in operations

To use provided buffers, set the IOSQE_BUFFER_SELECT flag on the SQE and specify the buffer group ID:

struct io_uring_sqe *sqe = io_uring_get_sqe(ring);
io_uring_prep_recv(sqe, sockfd, NULL, 4096, 0);
io_uring_sqe_set_flags(sqe, IOSQE_BUFFER_SELECT);
io_uring_sqe_set_buf_group(sqe, bgid);

Note that addr is set to NULL (or just ignored) since the kernel will select the buffer itself. The len field specifies the maximum amount of data to receive. 📏

Operations that support provided buffers include:

• IORING_OP_READ / IORING_OP_RECV
• IORING_OP_READV (single vector only!)
• IORING_OP_RECVMSG

💀 Handling completions

When an operation using provided buffers completes, the CQE tells you which buffer was used:

• IORING_CQE_F_BUFFER is set in cqe->flags
• The buffer ID is in the upper 16 bits of cqe->flags, extractable via cqe->flags >> IORING_CQE_BUFFER_SHIFT
• cqe->res contains the number of bytes transferred

struct io_uring_cqe *cqe;
io_uring_wait_cqe(ring, &cqe);

if (cqe->flags & IORING_CQE_F_BUFFER) {
    int bid = cqe->flags >> IORING_CQE_BUFFER_SHIFT;
    void *buf = buffers[bid];  /* application's buffer tracking */
    int len = cqe->res;

    /* process data in buf */
    process_data(buf, len);

    /* return buffer to ring for reuse */
    io_uring_buf_ring_add(br, buf, 4096, bid, mask, 0);
    io_uring_buf_ring_advance(br, 1);
}
io_uring_cqe_seen(ring, cqe);

😭 If no buffer was available when the operation completed, the operation fails with -ENOBUFS.

👑 Multishot operations

Provided buffers are particularly powerful with multishot operations like io_uring_prep_recv_multishot(3). A single SQE can generate multiple completions, and each one consumes a buffer from the ring:

struct io_uring_sqe *sqe = io_uring_get_sqe(ring);
io_uring_prep_recv_multishot(sqe, sockfd, NULL, 0, 0);
io_uring_sqe_set_flags(sqe, IOSQE_BUFFER_SELECT);
io_uring_sqe_set_buf_group(sqe, bgid);

Completions with IORING_CQE_F_MORE set mean more completions are coming, bestie. The multishot operation keeps going until an error occurs, the buffer ring gets exhausted, or the operation is canceled. 🔥

🌈 Incremental buffer consumption

Buffer rings can be set up with the IOU_PBUF_RING_INC flag to enable incremental consumption. In this mode, big large buffers can be partially consumed across multiple operations:

• Completions with IORING_CQE_F_BUF_MORE indicate the buffer will be used for more completions
• Each completion picks up exactly where the previous one left off
• The buffer is only returned when it's consumed completely, or on error

This is useful for registering large buffer regions that get consumed in smaller chunks. 🧩

🎀 Returning buffers

When you're done with a buffer, return it to the ring using io_uring_buf_ring_add(3) followed by io_uring_buf_ring_advance(3). And for efficiency when you're processing multiple CQEs at once, use io_uring_buf_ring_cq_advance(3) to advance both the CQ and the buffer ring in a single operation. efficiency slay 💅

⭐ Buffer ring status

Applications can query how many buffers are available using io_uring_buf_ring_available(3), which returns the number of buffers the kernel has not yet consumed. And the current kernel head position can be retrieved with io_uring_buf_ring_head(3). 🔍

🫶 Cleaning up

Buffer rings are freed using io_uring_free_buf_ring(3), which unregisters the ring and frees the ring memory (if it was allocated by io_uring_setup_buf_ring(3)). And listen — applications must free the individual buffers themselves. that part's on you bestie 🙈

---

💜 NOTES 💜

• Buffer ring entries must be a power of two, maximum 32768.
• Buffer IDs are 16-bit values (0-65535).
• If no buffer is available when an operation needs one, the operation fails with -ENOBUFS. Applications should ensure the ring is adequately stocked. (like, please keep it fed 🍓)
• Provided buffers cannot be used with registered (fixed) buffers. These are separate mechanisms, don't mix them up.
• For multishot receives, ensure buffers are returned to the ring promptly to avoid running out. 🏃♀️

🕰️ Legacy provided buffers

Earlier kernels supported provided buffers via IORING_OP_PROVIDE_BUFFERS and IORING_OP_REMOVE_BUFFERS. That mechanism required submitting SQEs just to add or remove buffers, which added latency and overhead. The ring-based mechanism described above supersedes this approach and should be used for all new applications. The legacy interface remains for backwards compatibility only. 🫡

---

🌟 SEE ALSO 🌟

• io_uring(7)
• io_uring_setup_buf_ring(3)
• io_uring_free_buf_ring(3)
• io_uring_register_buf_ring(3)
• io_uring_unregister_buf_ring(3)
• io_uring_buf_ring_add(3)
• io_uring_buf_ring_advance(3)
• io_uring_buf_ring_cq_advance(3)
• io_uring_buf_ring_available(3)
• io_uring_prep_recv_multishot(3)

---

and that's the whole page, bestie!! 💖✨🎀🦋

this lil manual page came from the Linux gang, dated January 18, 2025 — go forth and stock your buffer rings, mwah 💅🫶