rseq(2)

💅 rseq(2) 💅

okay so welcome to the System Calls Manual page for rseq, bestie — yes, the restartable sequences syscall, and yes it's giving kernel-level main character energy 💖. This is part 1, so grab a snack 🍓.

✨ NAME ✨

rseq — restartable sequences system call

🎀 LIBRARY 🎀

Standard C library (libc, -lc)

🌸 SYNOPSIS 🌸

#include <linux/rseq.h>       /* Definition of RSEQ_* constants */
#include <sys/syscall.h>      /* Definition of SYS_* constants */
#include <unistd.h>

int syscall(SYS_rseq, struct rseq *rseq, uint32_t rseq_len,
            int flags, uint32_t sig);

note bestie: glibc provides no wrapper for rseq(), necessitating the use of syscall(2). Like, you literally have to call it through syscall(2) — no cap.

💖 DESCRIPTION 💖

okay so the rseq() ABI accelerates specific user-space operations by registering a per-thread data structure shared between kernel and user-space 🦋. This data structure can be read from or written to by user-space to skip otherwise expensive system calls — slay, honestly.

A restartable sequence is a sequence of instructions guaranteed to be executed atomically with respect to other threads and signal handlers on the current CPU ✨. If its execution does not complete atomically, the kernel changes the execution flow by jumping to an abort handler defined by user-space for that restartable sequence. literally the kernel yanking you onto the abort path, periodt.

Using restartable sequences requires you to register a rseq() ABI per-thread data structure (struct rseq) through the rseq() system call. Only one rseq() ABI can be registered per thread, so user-space libraries and applications must follow a user-space ABI defining how to share this resource. The ABI defining how to share this resource between applications and libraries is defined by the C library. Allocation of the per-thread rseq() ABI and its registration to the kernel is handled by glibc since version 2.35.

The rseq() ABI per-thread data structure contains a rseq_cs field which points to the currently executing critical section. For each thread, a single rseq critical section can run at any given point. Each critical section needs to be implemented in assembly.

The rseq() ABI accelerates user-space operations on per-cpu data by defining a shared data structure ABI between each user-space thread and the kernel.

It allows user-space to perform update operations on per-cpu data without requiring heavy-weight atomic operations. highkey a glow-up for perf.

The term CPU used in this documentation refers to a hardware execution context. For instance, each CPU number returned by sched_getcpu() is a CPU. The current CPU means the CPU on which the registered thread is running.

Restartable sequences are atomic with respect to preemption (making it atomic with respect to other threads running on the same CPU), as well as signal delivery (user-space execution contexts nested over the same thread). They either complete atomically with respect to preemption on the current CPU and signal delivery, or they are aborted.

Restartable sequences are suited for update operations on per-cpu data.

Restartable sequences can be used on data structures shared between threads within a process, and on data structures shared between threads across different processes.

Some examples of operations that can be accelerated or improved by this ABI:

• Memory allocator per-cpu free-lists,
• Querying the current CPU number,
• Incrementing per-CPU counters,
• Modifying data protected by per-CPU spinlocks,
• Inserting/removing elements in per-CPU linked-lists,
• Writing/reading per-CPU ring buffers content.
• Accurately reading performance monitoring unit counters with respect to thread migration.

🚨 bestie, huge warning: Restartable sequences must not perform system calls. Doing so may result in termination of the process by a segmentation fault.

The rseq argument is a pointer to the thread-local struct rseq to be shared between kernel and user-space.

The structure struct rseq is an extensible structure. Additional feature fields can be added in future kernel versions. Its layout is as follows:

🦋 Structure alignment 🦋

This structure is aligned on either 32-byte boundary, or on the alignment value returned by getauxval() invoked with AT_RSEQ_ALIGN if the structure size differs from 32 bytes.

🍓 Structure size 🍓

This structure size needs to be at least 32 bytes. It can be either 32 bytes, or it needs to be large enough to hold the result of getauxval() invoked with AT_RSEQ_FEATURE_SIZE. Its size is passed as parameter to the rseq() system call.

#include <linux/rseq.h>

struct rseq {
    __u32 cpu_id_start;
    __u32 cpu_id;
    union {
        /* ... */
    } rseq_cs;
    __u32 flags;
    __u32 node_id;
    __u32 mm_cid;
} __attribute__((aligned(32)));

💅 Fields 💅

🌟 cpu_id_start 🌟

Always-updated value of the CPU number on which the registered thread is running. Its value is guaranteed to always be a possible CPU number, even when rseq() is not registered. Its value should always be confirmed by reading the cpu_id field before user-space performs any side-effect (e.g. storing to memory).

This field is always guaranteed to hold a valid CPU number in the range [ 0 .. nr_possible_cpus - 1 ]. It can therefore be loaded by user-space and used as an offset in per-cpu data structures without having to check whether its value is within the valid bounds compared to the number of possible CPUs in the system.

Initialized by user-space to a possible CPU number (e.g., 0), updated by the kernel for threads registered with rseq().

For user-space applications executed on a kernel without rseq() support, the cpu_id_start field stays initialized at 0, which is indeed a valid CPU number. It is therefore valid to use it as an offset in per-cpu data structures, and only validate whether it's actually the current CPU number by comparing it with the cpu_id field within the rseq critical section. If the kernel does not provide rseq() support, that cpu_id field stays initialized at -1, so the comparison always fails, as intended. 💀

This field should only be read by the thread which registered this data structure. Aligned on 32-bit.

It is up to user space to implement a fall-back mechanism for scenarios where rseq() is not available.

🌟 cpu_id 🌟

Always-updated value of the CPU number on which the registered thread is running. Initialized by user-space to -1, updated by the kernel for threads registered with rseq().

This field should only be read by the thread which registered this data structure. Aligned on 32-bit.

🌟 rseq_cs 🌟

The rseq_cs field is a pointer to a struct rseq_cs. It is NULL when no rseq assembly block critical section is active for the registered thread. Setting it to point to a critical section descriptor (struct rseq_cs) marks the beginning of the critical section.

Initialized by user-space to NULL.

Updated by user-space, which sets the address of the currently active rseq_cs at the beginning of assembly instruction sequence block, and set to NULL by the kernel when it restarts an assembly instruction sequence block, as well as when the kernel detects that it is preempting or delivering a signal outside of the range targeted by the rseq_cs. Also needs to be set to NULL by user-space before reclaiming memory that contains the targeted struct rseq_cs.

Read and set by the kernel.

This field should only be updated by the thread which registered this data structure. Aligned on 64-bit.

🌟 flags 🌟

Flags indicating the restart behavior for the registered thread. This is mainly used for debugging purposes. Can be a combination of:

• RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT — Inhibit instruction sequence block restart on preemption for this thread. This flag is deprecated since Linux 6.1.
• RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL — Inhibit instruction sequence block restart on signal delivery for this thread. This flag is deprecated since Linux 6.1.
• RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE — Inhibit instruction sequence block restart on migration for this thread. This flag is deprecated since Linux 6.1.

Initialized by user-space, used by the kernel.

🌟 node_id 🌟

Always-updated value of the current NUMA node ID.

Initialized by user-space to 0.

Updated by the kernel. Read by user-space with single-copy atomicity semantics. This field should only be read by the thread which registered this data structure. Aligned on 32-bit.

🌟 mm_cid 🌟

Contains the current thread's concurrency ID (allocated uniquely within a memory map).

Updated by the kernel. Read by user-space with single-copy atomicity semantics. This field should only be read by the thread which registered this data structure. Aligned on 32-bit.

This concurrency ID is within the possible cpus range, and is temporarily (and uniquely) assigned while threads are actively running within a memory map. If a memory map has fewer threads than cores, or is limited to run on few cores concurrently through sched affinity or cgroup cpusets, the concurrency IDs will be values close to 0, thus allowing efficient use of user-space memory for per-cpu data structures.

The layout of struct rseq_cs version 0 is as follows:

💜 struct rseq_cs version 0 — Structure alignment 💜

This structure is aligned on 32-byte boundary.

💜 struct rseq_cs version 0 — Structure size 💜

This structure has a fixed size of 32 bytes.

#include <linux/rseq.h>

struct rseq_cs {
    __u32   version;
    __u32   flags;
    __u64   start_ip;
    __u64   post_commit_offset;
    __u64   abort_ip;
} __attribute__((aligned(32)));

✨ Fields ✨

okay the actual struct rseq_cs field breakdown is literally next — part 2, bestie, don't leave me hanging 🫶💖

• version — the version of this struct, babes 🎀 it should be initialized to 0, no questions asked.
• flags — the flags that say how this struct should behave when it restarts. Can be a combo of these bad boys:
  • RSEQ_CS_FLAG_NO_RESTART_ON_PREEMPT — stops the instruction-sequence block from restarting on preemption for this critical section. heads up though: this flag is deprecated since Linux 6.1 💀
  • RSEQ_CS_FLAG_NO_RESTART_ON_SIGNAL — stops the restart on signal delivery for this critical section. also deprecated since Linux 6.1, rip 🪦
  • RSEQ_CS_FLAG_NO_RESTART_ON_MIGRATE — stops the restart on migration for this critical section. and yeah, deprecated since Linux 6.1 too bestie 😭
• start_ip — the instruction pointer address of the first instruction in the sequence of consecutive assembly instructions. like, where the whole thing kicks off ✨
• post_commit_offset — the offset (measured from the start_ip address) of the address right after the last instruction in that sequence of consecutive assembly instructions 🫶
• abort_ip — the instruction pointer address where execution flow gets yeeted to in case the sequence of consecutive assembly instructions aborts 🙈

---

Ok so the rseq_len argument? It's literally the size of the struct rseq you're registering. that's it, that's the whole vibe 📏

The flags argument is 0 for registration, and RSEQ_FLAG_UNREGISTER for unregistration. two moods 💅

The sig argument is the 32-bit signature that's expected before the abort handler code even runs. think of it as the secret handshake 🤝

A single library per process should keep the struct rseq inside a per-thread data structure — one per thread, no sharing chaos. The cpu_id field should be initialized to -1, and the cpu_id_start field should be initialized to a possible CPU value (typically 0). lowkey important 🌸

Each thread is responsible for registering and unregistering its own struct rseq — nobody else is doing it for you, bestie, this is main character energy 👑 And no more than one struct rseq address can be registered per thread at a given time. one at a time, periodt 💯

Reclaiming the memory of a struct rseq object must only happen after either an explicit rseq unregistration is performed, or after the thread exits. do not skip this step, I'm begging 😭

In a typical usage scenario, the thread that registers the struct rseq will be the one doing loads and stores from/to that structure. That said, it is however also allowed to read that structure from other threads — slay, sharing is caring 🦋 The struct rseq field updates performed by the kernel provide relaxed atomicity semantics (atomic store, without memory ordering), which guarantee that other threads performing relaxed atomic reads (atomic load, without memory ordering) of the cpu number fields will always observe a consistent value. no cap, it's genuinely consistent 🔥

---

✨ RETURN VALUE ✨

A return value of 0 means success, we ate 💖 On error, -1 is returned, and errno is set appropriately. classic.

---

🎀 ERRORS 🎀

• EINVAL — either flags contains an invalid value, or rseq contains an address which is not appropriately aligned, or rseq_len contains an incorrect size. three ways to fumble it 🙈
• ENOSYS — the rseq() system call is not implemented by this kernel. your kernel said "not me" 💀
• EFAULT — rseq is an invalid address.
• EBUSY — restartable sequence is already registered for this thread. somebody got here first 😤
• EPERM — the sig argument on unregistration does not match the signature received on registration. the secret handshake did not handshake 🫠

---

🌈 VERSIONS 🌈

The rseq() system call was added in Linux 4.18. throwback era ⭐

---

🦋 STANDARDS 🦋

rseq() is Linux-specific. sorry other OSes, this one's ours 💅

---

🌸 SEE ALSO 🌸

• sched_getcpu(3)
• membarrier(2)
• getauxval(3)

---

and that's the whole page, bestie, you're officially rseq-pilled 🧚💜 this one came out of Linux man-pages (unreleased) and the date stamp on it is just literally (date) — vague, iconic, we love a timestamp that refuses to commit 💀🎀