io_uring_...ed_buffers(7)

📦 version 2.15, in alpine 3.24, arch rolling, nixpkgs unstable, ubuntu resolute, ubuntu stonking

🎀 io_uring_...ed_buffers(7) — bestie, let's talk registered buffers 🎀

okay so hi!! 👋 you're reading io_uring_...ed_buffers(7), which is the io_uring registered buffers overview page from the Linux Programmer's Manual. the title is a lil abbreviated in the header (iykyk) but the real name is io_uring_registered_buffers and it is literally about making your I/O not slow. let's get into it 💅

✨ NAME ✨

io_uring_registered_buffers — the whole overview of io_uring registered buffers, no cap 💖

🦋 DESCRIPTION 🦋

Registered buffers are a performance optimization feature of io_uring. they let applications pre-register a set of buffers with the kernel, and when buffers are registered, the kernel pins the memory and creates long-term mappings. that means the overhead of mapping and unmapping buffer memory for every single I/O operation just… goes away. iconic. ✨

💭 Why use registered buffers?

so like, for every I/O operation that transfers data between user space and the kernel, the kernel has to do a whole lil to-do list on the buffer memory:

  • verify the memory is accessible to the process
  • pin the pages in memory so they don't get swapped out
  • set up kernel mappings to access the memory

each one is fast on its own, sure, but they add up when you're doing a bazillion tiny I/O operations. by registering buffers once upfront, you pay those costs only once, and all the I/O operations after that can use the pre-mapped buffers directly. periodt. 🫶

Registered buffers are most beneficial for applications that:

  • perform many small I/O operations
  • reuse the same buffers repeatedly
  • need the lowest possible per-I/O overhead

🌸 Registering buffers

you register buffers with io_uring_register_buffers(3) or io_uring_register_buffers_tags(3). the buffers get described using an array of struct iovec structures, like this:

struct iovec iovecs[2];
iovecs[0].iov_base = buf1;
iovecs[0].iov_len = 4096;
iovecs[1].iov_base = buf2;
iovecs[1].iov_len = 8192;

ret = io_uring_register_buffers(ring, iovecs, 2);

the buffers must be anonymous memory — allocated via malloc(3), mmap(2) with MAP_ANONYMOUS, or similar. file-backed memory is not supported. not the vibe, don't do it 🙈

there's a limit of 1 GiB per individual buffer. huge pages are supported, and heads up: the entire huge page will be pinned even if only part of it is used.

the buffers get charged against the user's RLIMIT_MEMLOCK resource limit on kernels before 5.12. on kernel 5.12 and later with IORING_FEAT_NATIVE_WORKERS support, cgroup memory accounting is used instead and no memlock limit applies. slay.

unless you're running as root, if buffer registration fails with ENOMEM, the memlock limit may need to be increased. check the current limit with:

ulimit -l

and bump it for the current shell session with:

ulimit -l unlimited

for a permanent change, edit /etc/security/limits.conf or use setrlimit(2) programmatically with RLIMIT_MEMLOCK.

💅 Using registered buffers

to use a registered buffer in an I/O operation, use the fixed buffer variants of the prep functions:

zero-copy send operations can also use registered buffers:

these functions take a buf_index parameter that says which registered buffer to use — it's 0-indexed into the array you passed to io_uring_register_buffers(3).

the memory range used for the I/O operation must fall within the bounds of the registered buffer. but it is valid to use only a portion of a registered buffer for an operation. 🫶

/* Use first 1024 bytes of registered buffer 0 */
io_uring_prep_read_fixed(sqe, fd, buf1, 1024, offset, 0);

/* Use registered buffer 1 */
io_uring_prep_write_fixed(sqe, fd, buf2, 2048, offset, 1);

🧚 Sparse buffer registration

applications can register a sparse buffer table using io_uring_register_buffers_sparse(3). this creates a table with empty slots that can be filled in later using io_uring_register_buffers_update_tag(3). super useful when you don't know the full set of buffers at registration time. no cap.

/* Create sparse table with 10 slots */
ret = io_uring_register_buffers_sparse(ring, 10);

/* Later, fill in slot 3 */
struct iovec iov = { .iov_base = buf, .iov_len = 4096 };
ret = io_uring_register_buffers_update_tag(ring, 3, &iov, NULL, 1);

🏷️ Buffer tagging

when you use io_uring_register_buffers_tags(3) or io_uring_register_buffers_update_tag(3), each buffer can be associated with a tag value. when a buffer is unregistered — either explicitly or by replacing it — and there are no more in-flight operations using that buffer, a completion queue entry is posted with user_data set to the tag value and all other fields zeroed.

this is the moment the application finds out it's safe to free or reuse the buffer memory. chef's kiss. 👑

🔄 Updating registered buffers

registered buffers can be updated in place using io_uring_register_buffers_update_tag(3). this can:

  • replace an existing buffer with a new one
  • fill in a sparse slot
  • remove a buffer by setting the iovec to zero length

updating buffers does not immediately free resources. the old buffer stays valid until all in-flight operations complete. be patient bestie. 🫶

🗑️ Unregistering buffers

buffers get unregistered with io_uring_unregister_buffers(3). this releases all registered buffers. buffers are also automatically unregistered when the io_uring instance is destroyed.

applications don't need to explicitly unregister buffers before shutting down the ring. however, page unpinning may happen asynchronously, so pages may not be immediately available after ring destruction. 👀

🌈 Cloning buffers

registered buffers can be cloned from one ring to another using io_uring_clone_buffers(3) or io_uring_clone_buffers_offset(3). this lets multiple rings share the same set of registered buffers without re-registering them. sharing is caring fr fr 💜

📝 NOTES 📝

  • registered buffers provide the most benefit for small, frequent I/O operations where the per-operation overhead is significant.
  • for large I/O operations, the buffer mapping overhead is small relative to the actual I/O time, so registered buffers may not provide much benefit.
  • the maximum number of registered buffers is limited by available kernel memory and the RLIMIT_MEMLOCK limit (on older kernels).
  • registered buffers cannot be used with provided buffer rings (IOSQE_BUFFER_SELECT). these are separate mechanisms for different use cases. don't mix them up 🙈

👀 SEE ALSO 👀

io_uring(7), io_uring_registered_files(7), setrlimit(2), io_uring_register_buffers(3), io_uring_register_buffers_tags(3), io_uring_register_buffers_sparse(3), io_uring_register_buffers_update_tag(3), io_uring_unregister_buffers(3), io_uring_prep_read_fixed(3), io_uring_prep_write_fixed(3), io_uring_prep_send_zc(3), io_uring_prep_sendmsg_zc(3), io_uring_clone_buffers(3)

okay that's the whole page bestie, you ate that 🍓 this one came from Linux (the Linux Programmer's Manual, page io_uring_...ed_buffers(7)) and it's dated January 18, 2025. go register some buffers, you've got this 💖✨🎀

📦 where this page comes from