userfaultfd(2)

💖 NAME 💖

so like, this is the manual page for userfaultfd(2) — a Linux system call that makes you a brand-new file descriptor so you can handle page faults in user space. yes. YOU. handling your own page faults. the kernel is literally handing you the aux. iconic behavior honestly 🫶

---

📚 LIBRARY 📚

Standard C library (libc, -lc) — the usual bestie.

---

✨ SYNOPSIS ✨

#include <fcntl.h>             /* Definition of O_* constants */
#include <sys/syscall.h>       /* Definition of SYS_* constants */
#include <linux/userfaultfd.h> /* Definition of UFFD_* constants */
#include <unistd.h>

int syscall(SYS_userfaultfd, int flags);

⚠️ bestie listen — glibc gives you no wrapper for userfaultfd(), so you have to call it through syscall(2). there is no cute little shortcut. periodt.

---

🦋 DESCRIPTION 🦋

userfaultfd() creates a new userfaultfd object that lets you delegate page-fault handling to a user-space application, and it returns a file descriptor pointing at that new object. You configure this new object using ioctl(2). lowkey the whole point of the thing 💅

once the userfaultfd object is set up, your app can use read(2) to receive userfaultfd notifications. those reads can be blocking or non-blocking, depending on the value of flags you used when creating the userfaultfd, or on later fcntl(2) calls. you're in control, queen.

the following values can be bitwise ORed into flags to change how userfaultfd() behaves:

• O_CLOEXEC — turns on the close-on-exec flag for the new userfaultfd file descriptor. go read the O_CLOEXEC description in open(2) for the full tea.
• O_NONBLOCK — enables non-blocking operation for the userfaultfd object. same deal, O_NONBLOCK in open(2) has the details.
• UFFD_USER_MODE_ONLY — a userfaultfd-specific flag that showed up in Linux 5.11. when you set it, the userfaultfd object can only handle page faults that came from user space on the registered regions. if a kernel-originated fault gets triggered on the registered range with this userfaultfd, a SIGBUS signal gets delivered. no cap.

when the last file descriptor referring to a userfaultfd object is closed, every memory range that was registered with that object gets unregistered, and any unread events get flushed. bye bye 🧹

userfaultfd supports a few modes of registration:

• UFFDIO_REGISTER_MODE_MISSING (since Linux 4.10) — when registered in this mode, user space gets a page-fault notification whenever a missing page is accessed. the faulted thread gets stopped from executing until the fault is resolved from user space by either an UFFDIO_COPY or an UFFDIO_ZEROPAGE ioctl.
• UFFDIO_REGISTER_MODE_MINOR (since Linux 5.13) — when registered in this mode, user space gets a page-fault notification when a minor page fault happens. that means the backing page is chillin' in the page cache, but the page table entries don't exist yet. the faulted thread stops until user space resolves it with an UFFDIO_CONTINUE ioctl.
• UFFDIO_REGISTER_MODE_WP (since Linux 5.7) — when registered in this mode, user space gets a page-fault notification when a write-protected page gets written to. the faulted thread stops until user space write-unprotects the page using an UFFDIO_WRITEPROTECT ioctl.
• UFFDIO_REGISTER_MODE_RWP (since Linux 7.2) — when registered in this mode, user space gets a page-fault notification on any access — read or write — to a page that's present within the range. by default, the faulted thread stops until user space removes the protection using a UFFDIO_RWPROTECT ioctl. BUT if UFFD_FEATURE_RWP_ASYNC was negotiated, the kernel restores access in place and the faulted thread just keeps going, no blocking, no drama ✨ additionally — UFFDIO_REGISTER_MODE_RWP and UFFDIO_REGISTER_MODE_WP cannot be combined on the same range. if you try to register with both bits set it fails with EINVAL. go read the "Userfaultfd read-write-protect mode" section below for the plot.

multiple modes can be enabled at the same time for the same memory range. multitasking queen 👑

since Linux 4.14, a userfaultfd page-fault notification can selectively embed faulting thread ID info into the notification. you have to enable it explicitly with the UFFD_FEATURE_THREAD_ID feature bit when you initialize the userfaultfd context. by default, thread ID reporting is disabled.

---

🎀 Usage 🎀

the userfaultfd mechanism is designed so that a thread in a multithreaded program can do user-space paging for the other threads in the process. when a page fault happens for one of the regions registered to the userfaultfd object, the faulting thread gets put to sleep 😴 and an event is generated that can be read via the userfaultfd file descriptor. the fault-handling thread reads events off that file descriptor and services them using the operations described in ioctl_userfaultfd(2). when it's servicing those page fault events, the fault-handling thread can trigger a wake-up for the sleeping thread. bestie behavior fr fr.

it's totally possible for the faulting threads and the fault-handling threads to run in the context of different processes. in that case they might belong to different programs, and the program running the faulting threads won't necessarily cooperate with the program handling the page faults. in this non-cooperative mode, the process that monitors userfaultfd and handles page faults needs to be aware of changes to the faulting process's virtual memory layout, otherwise: memory corruption 💀. be careful.

since Linux 4.11, userfaultfd can also notify the fault-handling threads about changes in the virtual memory layout of the faulting process. and on top of that, if the faulting process invokes fork(2), the userfaultfd objects associated with the parent may be duplicated into the child process, and the userfaultfd monitor gets notified (via the UFFD_EVENT_FORK described below) about the file descriptor associated with the userfault objects created for the child process. this means the userfaultfd monitor can perform user-space paging for the child process too 🍼

unlike page faults, which have to be synchronous and require an explicit or implicit wakeup, all other events are delivered asynchronously, and the non-cooperative process resumes execution as soon as the userfaultfd manager executes read(2). so the userfaultfd manager should carefully synchronize its calls to UFFDIO_COPY with the processing of events. no sloppy timing, okay?

the current asynchronous model of event delivery is optimal for single threaded non-cooperative userfaultfd manager implementations. it's giving custom-fit 👑

since Linux 5.7, userfaultfd can do synchronous page dirty tracking using the brand new write-protect register mode. you should check against the feature bit UFFD_FEATURE_PAGEFAULT_FLAG_WP before using this feature. similar to the original userfaultfd missing mode, write-protect mode generates a userfaultfd notification when the protected page gets written to. the user needs to resolve the page fault by unprotecting the faulted page and kicking the faulted thread to continue. for more info, go look at the "Userfaultfd write-protect mode" section.

since Linux 7.2, userfaultfd can do read-write-protection tracking, which traps every access (read or write) to a page present within a registered range. you should check against the feature bit UFFD_FEATURE_RWP before using this feature, and optionally negotiate UFFD_FEATURE_RWP_ASYNC so the kernel auto-restores page permissions on fault without delivering a notification. this mode is intended for working-set tracking by VM memory managers and similar callers; cold pages can then be evicted using independent kernel interfaces. more info's in the "Userfaultfd read-write-protect mode" section 🌈

---

🧚 Userfaultfd operation 🧚

after the userfaultfd object is created with userfaultfd(), the application must enable it using the UFFDIO_API ioctl(2) operation. this operation allows a two-step handshake between the kernel and user space — first to figure out what API version and features the kernel supports, then to enable the features user space actually wants. you must do this before any of the other ioctl(2) operations described below, otherwise those operations fail with the EINVAL error. no skipping steps bestie.

after a successful UFFDIO_API operation, the application registers memory address ranges using the UFFDIO_REGISTER ioctl(2) operation. once a UFFDIO_REGISTER operation completes successfully, any page fault that occurs in the requested memory range — and that satisfies the mode defined at registration time — gets forwarded by the kernel to the user-space application. the application can then use various (like, UFFDIO_COPY, UFFDIO_ZEROPAGE, or UFFDIO_CONTINUE) ioctl(2) operations to resolve the page fault. slay.

since Linux 4.14, if the application sets the UFFD_FEATURE_SIGBUS feature bit using the UFFDIO_API ioctl(2), no page-fault notification gets forwarded to user space. instead a SIGBUS signal is delivered to the faulting process. with this feature, userfaultfd can be used for robustness purposes — to just catch any access to areas within the registered address range that don't have pages allocated, without having to listen to userfaultfd events at all. no userfaultfd monitor will be required for handling those memory accesses. for example, this feature is useful for applications that want to stop the kernel from automatically allocating pages and filling holes in sparse files when the hole is accessed through a memory mapping.

the UFFD_FEATURE_SIGBUS feature is implicitly inherited through fork(2) if it's used in combination with UFFD_FEATURE_FORK. inheriting the vibe, exactly.

details of the various ioctl(2) operations live in ioctl_userfaultfd(2).

since Linux 4.11, events other than page-fault may be enabled during the UFFDIO_API operation.

up to Linux 4.11, userfaultfd could only be used with anonymous private memory mappings. since Linux 4.11, userfaultfd can also be used with hugetlbfs and shared memory mappings. glow up 💖

---

🔥 Userfaultfd write-protect mode (since Linux 5.7) 🔥

since Linux 5.7, userfaultfd supports write-protect mode for anonymous memory. you need to first check that this feature is available using the UFFDIO_API ioctl against the feature bit UFFD_FEATURE_PAGEFAULT_FLAG_WP before using it.

since Linux 5.19, write-protection mode is also supported on shmem and hugetlbfs memory types. you can detect it with the feature bit UFFD_FEATURE_WP_HUGETLBFS_SHMEM. expanding the empire 🎀

To register with userfaultfd write-protect mode, you gotta kick off the UFFDIO_REGISTER ioctl with the mode UFFDIO_REGISTER_MODE_WP set. And bestie, it is totally legal to monitor the same memory range with multiple modes at once. Like, you can absolutely do UFFDIO_REGISTER with the mode set to UFFDIO_REGISTER_MODE_MISSING | UFFDIO_REGISTER_MODE_WP. 💅

When the only thing you've registered is UFFDIO_REGISTER_MODE_WP, user space will not get any notification when a missing page gets written. Nope. Instead, user space gets a write-protect page-fault notification only when an existing-but-write-protected page got written. Periodt.

Once the UFFDIO_REGISTER ioctl is done and dusted with UFFDIO_REGISTER_MODE_WP mode set, you can write-protect any existing memory inside the range using the ioctl UFFDIO_WRITEPROTECT, where uffdio_writeprotect.mode should be set to UFFDIO_WRITEPROTECT_MODE_WP. easy.

When a write-protect event goes down, user space receives a page-fault notification whose uffd_msg.pagefault.flags has the UFFD_PAGEFAULT_FLAG_WP flag set. 💖 Note this though: since only writes can trigger this kind of fault, write-protect notifications will always have the UFFD_PAGEFAULT_FLAG_WRITE bit set right alongside the UFFD_PAGEFAULT_FLAG_WP bit. they come as a duo, no cap.

To resolve a write-protection page fault, the user should initiate another UFFDIO_WRITEPROTECT ioctl, whose uffd_msg.pagefault.flags should have the flag UFFDIO_WRITEPROTECT_MODE_WP cleared upon the faulted page or range. that's the whole trick. ✨

---

🌸 Userfaultfd minor fault mode (since Linux 5.13) 🌸

Since Linux 5.13, userfaultfd supports minor fault mode. In this mode, fault messages are produced not for major faults (the ones where the page was missing), but rather for minor faults — where a page exists in the page cache, but the page table entries are not yet present. subtle but important, iykyk.

The user needs to first check availability of this feature using the UFFDIO_API ioctl with the appropriate feature bits set before using this feature: UFFD_FEATURE_MINOR_HUGETLBFS since Linux 5.13, or UFFD_FEATURE_MINOR_SHMEM since Linux 5.14. 👑

To register with userfaultfd minor fault mode, the user needs to initiate the UFFDIO_REGISTER ioctl with mode UFFD_REGISTER_MODE_MINOR set.

When a minor fault occurs, user space will receive a page-fault notification whose uffd_msg.pagefault.flags will have the UFFD_PAGEFAULT_FLAG_MINOR flag set.

To resolve a minor page fault, the handler should decide whether or not the existing page contents need to be modified first. If so, this should be done in-place via a second, non-userfaultfd-registered mapping to the same backing page (e.g., by mapping the shmem or hugetlbfs file twice — yes, twice, iconic behavior). Once the page is considered "up to date", the fault can be resolved by initiating an UFFDIO_CONTINUE ioctl, which installs the page table entries and (by default) wakes up the faulting thread(s). 🫶

Minor fault mode supports only hugetlbfs-backed (since Linux 5.13) and shmem-backed (since Linux 5.14) memory.

---

🦋 Userfaultfd read-write-protect mode (since Linux 7.2) 🦋

Since Linux 7.2, userfaultfd supports read-write-protect mode. Unlike write-protect mode, every single access — read or write — to a protected page generates a userfaultfd notification. She's thorough like that. 💅 It works on anonymous, shmem, and hugetlbfs mappings.

The user needs to first check availability of this feature using the UFFDIO_API ioctl against the feature bit UFFD_FEATURE_RWP before using this mode. See UFFDIO_API(2const) for the recommended discovery sequence. 📖

To register with userfaultfd read-write-protect mode, the user needs to initiate the UFFDIO_REGISTER ioctl with mode UFFDIO_REGISTER_MODE_RWP set. Heads up though: UFFDIO_REGISTER_MODE_RWP cannot be combined with UFFDIO_REGISTER_MODE_WP. However, it can be combined with UFFDIO_REGISTER_MODE_MISSING when the caller also wants notifications for fresh page populations.

After registration, the user can read-write-protect any existing memory within the range using the UFFDIO_RWPROTECT ioctl where uffdio_rwprotect.mode is set to UFFDIO_RWPROTECT_MODE_RWP. Quick thing: read-write protection only affects pages that are currently populated in the range. Unpopulated addresses stay unpopulated and fall through to the normal missing-page path on first access. no drama, they just get the standard treatment.

For anonymous mappings, protection is preserved across page reclaim (the marker rides on the swap entry, cute) and migration. For shmem and file-backed mappings, protection is dropped when the backing page is reclaimed and must be re-armed by the caller. Protection is also not preserved across operations that explicitly drop the underlying page:

• MADV_DONTNEED on anonymous memory
• hole-punch on shmem
• truncation of a file mapping

Callers must re-arm the range with UFFDIO_RWPROTECT after any such operation. don't forget, bestie. 🙈

When an access fault happens against a protected page, user space will receive a page-fault notification whose uffd_msg.pagefault.flags field has the UFFD_PAGEFAULT_FLAG_RWP bit set.

To resolve a read-write-protect page fault, the user initiates another UFFDIO_RWPROTECT ioctl whose uffdio_rwprotect.mode has the UFFDIO_RWPROTECT_MODE_RWP flag cleared. This restores the original VMA permissions on the affected pages and wakes any blocked threads (unless UFFDIO_RWPROTECT_MODE_DONTWAKE is also set). 🎀

If UFFD_FEATURE_RWP_ASYNC was negotiated alongside UFFD_FEATURE_RWP, the kernel resolves access faults in place without delivering a notification: page permissions are restored automatically and the faulting thread just... continues. main character energy, honestly. ✨ Callers can later reconstruct which pages were touched by inspecting the PAGE_IS_ACCESSED bit returned by the PAGEMAP_SCAN ioctl described in ioctl_userfaultfd(2) and Documentation/admin-guide/mm/pagemap.rst in the Linux kernel source.

The async mode can be toggled at runtime using the UFFDIO_SET_MODE ioctl, which lets a single userfaultfd switch between async detection and synchronous eviction without re-registering the range. glow up without the redo. 💜

---

🧚 Reading from the userfaultfd structure 🧚

Each read(2) from the userfaultfd file descriptor returns one or more uffd_msg structures, each of which describes a page-fault event or an event required for the non-cooperative userfaultfd usage:

struct uffd_msg {
    __u8  event;            /* Type of event */
    ...
    union {
        struct {
            __u64 flags;    /* Flags describing fault */
            __u64 address;  /* Faulting address */
            union {
                __u32 ptid; /* Thread ID of the fault */
            } feat;
        } pagefault;

        struct {            /* Since Linux 4.11 */
            __u32 ufd;      /* Userfault file descriptor
                               of the child process */
        } fork;

        struct {            /* Since Linux 4.11 */
            __u64 from;     /* Old address of remapped area */
            __u64 to;       /* New address of remapped area */
            __u64 len;      /* Original mapping size */
        } remap;

        struct {            /* Since Linux 4.11 */
            __u64 start;    /* Start address of removed area */
            __u64 end;      /* End address of removed area */
        } remove;
        ...
    } arg;

    /* Padding fields omitted */
} __packed;

If multiple events are available and the buffer you supplied is big enough, read(2) returns as many events as will fit in that buffer. But if the buffer you handed to read(2) is smaller than the size of the uffd_msg structure, the read(2) fails with the error EINVAL. size matters, unfortunately. 💀

The fields set in the uffd_msg structure are as follows:

• event — the type of event. Depending on the event type, different fields of the arg union represent the details you need for processing that event. The non-page-fault events are generated only when the appropriate feature was enabled during the API handshake with the UFFDIO_API ioctl(2).

The following values can appear in the event field:

• UFFD_EVENT_PAGEFAULT (since Linux 4.3) — a page-fault event. The page-fault details live in the pagefault field.
• UFFD_EVENT_FORK (since Linux 4.11) — generated when the faulting process invokes fork(2) (or clone(2) without the CLONE_VM flag). The event details live in the fork field.
• UFFD_EVENT_REMAP (since Linux 4.11) — generated when the faulting process invokes mremap(2). The event details live in the remap field.
• UFFD_EVENT_REMOVE (since Linux 4.11) — generated when the faulting process invokes madvise(2) with MADV_DONTNEED or MADV_REMOVE advice. The event details live in the remove field.
• UFFD_EVENT_UNMAP (since Linux 4.11) — generated when the faulting process unmaps a memory range, either explicitly using munmap(2) or implicitly during mmap(2) or mremap(2). The event details live in the remove field.

• pagefault.address — the address that triggered the page fault. like, the exact spot. 📍
• pagefault.flags — a bit mask of flags that describe the event. For UFFD_EVENT_PAGEFAULT, the following flags may appear:
  • UFFD_PAGEFAULT_FLAG_WP — if this flag is set, then the fault was a write-protect fault.
  • UFFD_PAGEFAULT_FLAG_MINOR — if this flag is set, then the fault was a minor fault.
  • UFFD_PAGEFAULT_FLAG_RWP (since Linux 7.2) — if this flag is set, then the fault was a read-write-protect fault.
  • UFFD_PAGEFAULT_FLAG_WRITE — if this flag is set, then the fault was a write fault.

And if none of UFFD_PAGEFAULT_FLAG_WP, UFFD_PAGEFAULT_FLAG_MINOR, nor UFFD_PAGEFAULT_FLAG_RWP are set, then the fault was a missing fault. the classic. ✨

• pagefault.feat.pid — the thread ID that triggered the page fault.
• fork.ufd — the file descriptor associated with the userfault object created for the child created by fork(2).
• remap.from — the original address of the memory range that was remapped using mremap(2).
• remap.to — the new address of the memory range that was remapped using mremap(2).
• remap.len — the original size of the memory range that was remapped using mremap(2).
• remove.start — the start address of the memory range that was freed using madvise(2) or unmapped
• remove.end — the end address of the memory range that was freed using madvise(2) or unmapped

A read(2) on a userfaultfd file descriptor can fail with the following errors:

• EINVAL — bestie, the userfaultfd object hasn't been enabled yet, like it's literally still waking up 💅 it needs the UFFDIO_API ioctl(2) operation first, no cap.

If the O_NONBLOCK flag is enabled in the associated open file description, the userfaultfd file descriptor can be monitored with poll(2), select(2), and epoll(7) 🌈 when events are available, the file descriptor indicates as readable. But if the O_NONBLOCK flag is not enabled, then poll(2) (always) indicates the file as having a POLLERR condition, and select(2) indicates the file descriptor as both readable and writable 😭 like it's giving mixed signals fr fr.

---

💖 RETURN VALUE 💖

On success, userfaultfd() returns a new file descriptor that refers to the userfaultfd object — slay ✨ on error, -1 is returned, and errno is set to indicate the error. Periodt.

---

🚨 ERRORS 🚨

• EINVAL — an unsupported value was specified in flags 🙈 bestie that flag is not it.
• EMFILE — the per-process limit on the number of open file descriptors has been reached 💀 too many fds open, no more room.
• ENFILE — the system-wide limit on the total number of open files has been reached 😭 the whole system is full.
• ENOMEM — insufficient kernel memory was available 🫠 the kernel is lowkey struggling.
• EPERM (since Linux 5.2) — the caller is not privileged (does not have the CAP_SYS_PTRACE capability in the initial user namespace), and /proc/sys/vm/unprivileged_userfaultfd has the value 0 🔒 no permission, bestie.

---

✨ STANDARDS ✨

Linux 👑 that's the standard.

---

🌸 HISTORY 🌸

Linux 4.3 🎀 she's been around.

Support for hugetlbfs and shared memory areas and non-page-fault events was added in Linux 4.11 🦋 glow up.

Read-write-protect mode (UFFDIO_REGISTER_MODE_RWP, UFFD_FEATURE_RWP, UFFDIO_RWPROTECT) was added in Linux 7.2, together with UFFD_FEATURE_RWP_ASYNC and the UFFDIO_SET_MODE runtime mode toggle 💅 iconic behavior.

---

📝 NOTES 📝

The userfaultfd mechanism can be used as an alternative to traditional user-space paging techniques based on the use of the SIGSEGV signal and mmap(2) ✨ it can also be used to implement lazy restore for checkpoint/restore mechanisms, as well as post-copy migration to allow (nearly) uninterrupted execution when transferring virtual machines and Linux containers from one host to another 🌈 main character energy for real.

---

💀 BUGS 💀

If the UFFD_FEATURE_EVENT_FORK is enabled and a system call from the fork(2) family is interrupted by a signal or failed, a stale userfaultfd descriptor might be created 🫠 in this case, a spurious UFFD_EVENT_FORK will be delivered to the userfaultfd monitor 😭 not the vibe we wanted, bestie.

✨ EXAMPLES ✨

okay bestie, here's a whole entire working program that shows off the userfaultfd mechanism, no cap. It spins up two threads: one of them is the page-fault handler for the process, and it handles the pages in a demand-page zero region that got made with mmap(2). 🦋

The program takes exactly one command-line argument: the number of pages that will be created in a mapping whose page faults get handled via userfaultfd. The flow is: create a userfaultfd object → create an anonymous private mapping of the size you asked for → register that mapping's address range with the UFFDIO_REGISTER ioctl(2) operation → then spawn a second thread whose whole job is handling page faults. 💅

The main thread then walks through the pages of the mapping, fetching bytes from successive pages. Since those pages have not been accessed yet, the first access of a byte in each page triggers a page-fault event on the userfaultfd file descriptor. ✨

Each page-fault event gets handled by that second thread, which just sits in a loop processing input from the userfaultfd file descriptor. Every loop iteration, the second thread calls poll(2) first to check the state of the file descriptor, then reads an event from it. All these events should be UFFD_EVENT_PAGEFAULT events, and the thread handles them by copying a page of data into the faulting region using the UFFDIO_COPY ioctl(2) operation. 👑

Here's what we see when running the program, fr fr:

$ ./userfaultfd_demo 3;
Address returned by mmap() = 0x7fd30106c000

fault_handler_thread():
    poll() returns: nready = 1; POLLIN = 1; POLLERR = 0
    UFFD_EVENT_PAGEFAULT event: flags = 0; address = 7fd30106c00f
        (uffdio_copy.copy returned 4096)
Read address 0x7fd30106c00f in main(): A
Read address 0x7fd30106c40f in main(): A
Read address 0x7fd30106c80f in main(): A
Read address 0x7fd30106cc0f in main(): A

fault_handler_thread():
    poll() returns: nready = 1; POLLIN = 1; POLLERR = 0
    UFFD_EVENT_PAGEFAULT event: flags = 0; address = 7fd30106d00f
        (uffdio_copy.copy returned 4096)
Read address 0x7fd30106d00f in main(): B
Read address 0x7fd30106d40f in main(): B
Read address 0x7fd30106d80f in main(): B
Read address 0x7fd30106dc0f in main(): B

fault_handler_thread():
    poll() returns: nready = 1; POLLIN = 1; POLLERR = 0
    UFFD_EVENT_PAGEFAULT event: flags = 0; address = 7fd30106e00f
        (uffdio_copy.copy returned 4096)
Read address 0x7fd30106e00f in main(): C
Read address 0x7fd30106e40f in main(): C
Read address 0x7fd30106e80f in main(): C
Read address 0x7fd30106ec0f in main(): C

---

🎀 Program source 🎀

here's the whole thing, literally, go off:

/* userfaultfd_demo.c

   Licensed under the GNU General Public License version 2 or later.
*/
#define _GNU_SOURCE
#include <err.h>
#include <errno.h>
#include <fcntl.h>
#include <inttypes.h>
#include <linux/userfaultfd.h>
#include <poll.h>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <sys/syscall.h>
#include <unistd.h>

static int page_size;

static void *
fault_handler_thread(void *arg)
{
    long  uffd;   /* userfaultfd file descriptor */

    static int      fault_cnt = 0; /* Number of faults so far handled */
    static char     *page = NULL;
    static struct uffd_msg  msg;  /* Data read from userfaultfd */

    uffd = (long) arg;

    /* Create a page that will be copied into the faulting region.  */

    if (page == NULL) {
        page = mmap(NULL, page_size, PROT_READ | PROT_WRITE,
                    MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
        if (page == MAP_FAILED)
            err(EXIT_FAILURE, "mmap");
    }

    /* Loop, handling incoming events on the userfaultfd
       file descriptor.  */

    for (;;) {
        int                 nready;
        ssize_t             nread;
        struct pollfd       pollfd;
        struct uffdio_copy  uffdio_copy;

        /* See what poll() tells us about the userfaultfd.  */

        pollfd.fd = uffd;
        pollfd.events = POLLIN;
        nready = poll(&pollfd, 1, -1);
        if (nready == -1)
            err(EXIT_FAILURE, "poll");

        printf("\nfault_handler_thread():\n");
        printf("    poll() returns: nready = %d; "
               "POLLIN = %d; POLLERR = %d\n", nready,
               (pollfd.revents & POLLIN) != 0,
               (pollfd.revents & POLLERR) != 0);

        /* Read an event from the userfaultfd.  */

        nread = read(uffd, &msg, sizeof(msg));
        if (nread == 0) {
            printf("EOF on userfaultfd!\n");
            exit(EXIT_FAILURE);
        }

        if (nread == -1)
            err(EXIT_FAILURE, "read");

        /* We expect only one kind of event; verify that assumption.  */

        if (msg.event != UFFD_EVENT_PAGEFAULT) {
            fprintf(stderr, "Unexpected event on userfaultfd\n");
            exit(EXIT_FAILURE);
        }

        /* Display info about the page-fault event.  */

        printf("    UFFD_EVENT_PAGEFAULT event: ");
        printf("flags = %w64x; ", msg.arg.pagefault.flags);
        printf("address = %w64x\n", msg.arg.pagefault.address);

        /* Copy the page pointed to by 'page' into the faulting
           region.  Vary the contents that are copied in, so that it
           is more obvious that each fault is handled separately.  */

        memset(page, 'A' + fault_cnt % 20, page_size);
        fault_cnt++;

        uffdio_copy.src = (unsigned long) page;

        /* We need to handle page faults in units of pages(!).
           So, round faulting address down to page boundary.  */

        uffdio_copy.dst = (unsigned long) msg.arg.pagefault.address &
                                             ~(page_size - 1);
        uffdio_copy.len = page_size;
        uffdio_copy.mode = 0;
        uffdio_copy.copy = 0;
        if (ioctl(uffd, UFFDIO_COPY, &uffdio_copy) == -1)
            err(EXIT_FAILURE, "ioctl-UFFDIO_COPY");

        printf("        (uffdio_copy.copy returned %w64d)\n",
               uffdio_copy.copy);
    }
}

int
main(int argc, char *argv[])
{
    int        s;
    char       *addr;   /* Start of region handled by userfaultfd */
    long       uffd;    /* userfaultfd file descriptor */
    size_t     size, i;  /* Size of region handled by userfaultfd */
    pthread_t  thr;     /* ID of thread that handles page faults */
    struct uffdio_api       uffdio_api;
    struct uffdio_register  uffdio_register;

    if (argc != 2) {
        fprintf(stderr, "Usage: %s num-pages\n", argv[0]);
        exit(EXIT_FAILURE);
    }

    page_size = sysconf(_SC_PAGE_SIZE);
    size = strtoull(argv[1], NULL, 0) * page_size;

    /* Create and enable userfaultfd object.  */

    uffd = syscall(SYS_userfaultfd, O_CLOEXEC | O_NONBLOCK);
    if (uffd == -1)
        err(EXIT_FAILURE, "userfaultfd");

    /* NOTE: Two-step feature handshake is not needed here, since this
       example doesn't require any specific features.

       Programs that *do* should call UFFDIO_API twice: once with
       `features = 0` to detect features supported by this kernel, and
       again with the subset of features the program actually wants to
       enable.  */
    uffdio_api.api = UFFD_API;
    uffdio_api.features = 0;
    if (ioctl(uffd, UFFDIO_API, &uffdio_api) == -1)
        err(EXIT_FAILURE, "ioctl-UFFDIO_API");

    /* Create a private anonymous mapping.  The memory will be
       demand-zero paged--that is, not yet allocated.  When we
       actually touch the memory, it will be allocated via
       the userfaultfd.  */

    addr = mmap(NULL, size, PROT_READ | PROT_WRITE,
                MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
    if (addr == MAP_FAILED)
        err(EXIT_FAILURE, "mmap");

    printf("Address returned by mmap() = %p\n", addr);

    /* Register the memory range of the mapping we just created for
       handling by the userfaultfd object.  In mode, we request to track
       missing pages (i.e., pages that have not yet been faulted in).  */

    uffdio_register.range.start = (unsigned long) addr;
    uffdio_register.range.len = size;
    uffdio_register.mode = UFFDIO_REGISTER_MODE_MISSING;
    if (ioctl(uffd, UFFDIO_REGISTER, &uffdio_register) == -1)
        err(EXIT_FAILURE, "ioctl-UFFDIO_REGISTER");

    /* Create a thread that will process the userfaultfd events.  */

    s = pthread_create(&thr, NULL, fault_handler_thread, (void *) uffd);
    if (s != 0) {
        errc(EXIT_FAILURE, s, "pthread_create");
    }

    /* Main thread now touches memory in the mapping, touching
       locations 1024 bytes apart.  This will trigger userfaultfd
       events for all pages in the region.  */

    i = 0xf;    /* Ensure that faulting address is not on a page
                   boundary, in order to test that we correctly
                   handle that case in fault_handling_thread().  */
    while (i < size) {
        char  c;

        c = addr[i];
        printf("Read address %p in %s(): ", addr + i, __func__);
        printf("%c\n", c);
        i += 1024;
        usleep(100000);         /* Slow things down a little */
    }

    exit(EXIT_SUCCESS);
}

---

🌈 SEE ALSO 🌈

go read these next, bestie, they're the whole lore:

• fcntl(2), ioctl(2), ioctl_userfaultfd(2), madvise(2), mmap(2)
• Documentation/admin-guide/mm/userfaultfd.rst in the Linux kernel source tree

---

and that's a wrap on the whole page 🫶 this was Linux man-pages 6.20, dated 2026-10-01 — iconic, ate, no crumbs left. go handle some page faults, you got this 💖✨🎀