SYSTEMD-SYSEXT(8)

📦
version 262.1 · 3 versions
, in arch rolling

🎀 NAME 🎀

  • systemd-sysext
  • systemd-sysext.service
  • systemd-sysext-initrd.service
  • systemd-sysext-sysroot.service
  • systemd-confext
  • systemd-confext.service
  • systemd-confext-initrd.service
  • systemd-confext-sysroot.service

— Activates System Extension Images ✨ (yes bestie, that's the manual systemd-sysext(8), we're doing this 💖)

💫 SYNOPSIS 💫

systemd-sysext [OPTIONS...] COMMAND

systemd-sysext.service

systemd-confext [OPTIONS...] COMMAND

systemd-confext.service

🌸 DESCRIPTION 🌸

okay so systemd-sysext is literally the girl who activates and deactivates system extension images. and what are those? they're images that can — dynamically, at runtime — extend your /usr/ and /opt/ directory hierarchies with extra files. 🦋 this is highkey perfect for immutable system images, where a /usr/ and/or /opt/ hierarchy lives on a read-only file system and you wanna extend it temporarily at runtime without making any persistent modifications. no cap. 💅

System extension images should carry files and directories that look basically like a regular operating system tree. when one or more of these images get activated, their /usr/ and /opt/ hierarchies get combined with the same hierarchies of the host OS via overlayfs, and the host /usr/ and /opt/ get overmounted with that ("merging" 💖). when they get deactivated, the mount point is disassembled — which reveals the unmodified original host version of the hierarchy again ("unmerging" ✨). so merging = the extension's resources suddenly appear below /usr/ and /opt/ as if they were shipped inside the base OS image itself. unmerging = they vanish again, leaving behind only the files that came with the base OS image. it's giving Cinderella at midnight, honestly. 🕛

now, the important bit: files and directories inside the extension images that live outside of /usr/ and /opt/ do not get merged, which means they have zero effect when you put them in a system extension image. in particular, files in /etc/ and /var/ that are included in a system extension image will not show up in those hierarchies after activation. 🙈 don't do it, bestie.

System extension images are strictly read-only by default. 🔒 on mutable host file systems, /usr/ and /opt/ become read-only while extensions are merged, unless mutability is enabled. mutability can be turned on via the --mutable= option and the Mutable= option in the configuration file — go read "Mutability" below for the full tea. 🍓

Various command options can be configured globally via configuration files. see sysext.conf(5) for the details. ✨

System extensions are supposed to be purely additive, meaning they're supposed to include only files that don't already exist in the underlying basic OS image. however, the underlying mechanism (overlayfs) does also allow overlaying or removing files — but it's recommended not to make use of that. just because you can doesn't mean you should, periodt. 💀

System extension images may be provided in the following formats:

  1. Plain directories or btrfs subvolumes containing the OS tree
  2. Disk images with a GPT disk label, following the UAPI.2 Discoverable Partitions Specification[1]
  3. Disk images lacking a partition table, with a naked Linux file system (e.g. erofs, squashfs or ext4)

These image formats are the exact same ones that systemd-nspawn(1) supports via its --directory=/--image= switches, and the ones the service manager supports via RootDirectory=/RootImage=. just like those, they may optionally carry Verity authentication information. 👑

System extensions get searched for in the directories /etc/extensions/, /run/extensions/ and /var/lib/extensions/. the first two listed directories are not suitable for carrying large binary images, but they're still useful for carrying symlinks pointing at them. 🪄 the primary place for installing system extensions is /var/lib/extensions/. any directories found in these search directories are considered directory based extension images; any files with the .raw suffix are considered disk image based extension images. when invoked in the initrd, the additional directory /.extra/sysext/ is included in the directories that are searched for extension images. note however that by default a tighter image policy applies to images found there though, see below. 👀 this directory is populated by systemd-stub(7) with extension images found in the system's EFI System Partition.

During boot, system and configuration extension images are activated automatically if the systemd-sysext.service and systemd-confext.service services are enabled. 🎀 note that these services run only after the underlying file systems where system and configuration extensions may be located have been mounted. to make it possible to ship resources that are processed by subsystems running in the earliest boot stages (for example, system services or systemd-sysusers(8) definitions), the systemd-sysext-sysroot.service and systemd-confext-sysroot.service initrd services are provided. currently, these services cannot be used to merge system extensions from /sysroot/var/lib/extensions/ and configuration extensions from /sysroot/var/lib/confexts/ when the /var/ partition is split off. those extensions get merged later, by the systemd-sysext.service and systemd-confext.service services during the main OS boot process. 🦋

Also, go look at the Portable Services[2] page for a simple mechanism for shipping system services in disk images, in a similar fashion to OS extensions. note the differences in isolation between these two mechanisms: while system extensions directly extend the underlying OS image with additional files that appear as if they were shipped in the OS image itself — and thus imply no security isolation — portable services imply service-level sandboxing in one way or another. binaries shipped in system extensions can only link against libraries from the host if the extension is bound to the version of the host OS (see further below) or otherwise they have to use static linking. portable services, however, are fully decoupled from the host since they ship their own library dependencies. 💅

The systemd-sysext.service and systemd-confext.service services are guaranteed to finish start-up before basic.target is reached; i.e., by the time regular services initialize (those which do not use DefaultDependencies=no), the files and directories provided by system and configuration extensions are available in /usr/, /opt/, and /etc/ and may be accessed. ✨ she's punctual, she's reliable, she's that girl. ⭐

System and configuration extensions can also be used to extend the initrd, and the systemd-sysext-initrd.service and systemd-confext-initrd.service initrd services are provided. note that some limitations apply: resources that are used in the earliest boot stages of the initrd (e.g. system services) cannot be updated. 😭

Note that there is no concept of enabling/disabling installed system extension images: all installed extension images are automatically activated at boot. however, you can place an empty directory named like the extension (no .raw) in /etc/extensions/ to "mask" an extension with the same name in a system folder with lower precedence. it's also possible to disable automatic merging altogether using the rd.systemd.sysext=, rd.systemd.confext=, systemd.sysext=, and systemd.confext= kernel command line options. 🔥 note that systemd-sysext-sysroot.service and systemd-confext-sysroot.service are controlled by the systemd.sysext= and systemd.confext= options, since these services merge system and configuration extensions for the main system, not for the initrd.

A simple mechanism for version compatibility is enforced: a system extension image must carry a /usr/lib/extension-release.d/extension-release.NAME file, which must match its image name, and that gets compared with the host os-release file: the contained ID= fields have to match unless "_any" is set for the extension. if the extension ID= is not "_any", the SYSEXT_LEVEL= field (if defined) has to match. if the latter isn't defined, the VERSION_ID= field has to match instead. if the extension defines the ARCHITECTURE= field and the value isn't "_any" it has to match the kernel's architecture reported by uname(2) — but the architecture identifiers used are the same as for ConditionArchitecture= described in systemd.unit(5). 👑

EXTENSION_RELOAD_MANAGER= can be set to 1 if the extension requires a service manager reload after application of the extension. EXTENSION_RESTART_UNITS= and EXTENSION_RELOAD_OR_RESTART_UNITS= can be set to a whitespace-separated list of unit names (e.g. EXTENSION_RESTART_UNITS="my.service other.service") that systemd-sysext/systemd-confext should restart resp. reload (or restart, if the unit does not support reloading) after merging or refreshing the extension, and after unmerging it. use EXTENSION_RESTART_UNITS= for units whose binary the extension replaces (a reload would not pick up the new binary), and EXTENSION_RELOAD_OR_RESTART_UNITS= for units the extension extends with configuration drop-ins only (so reload-capable services are not needlessly interrupted 💅). if a unit has disappeared, e.g., on unmerge because it was part of the extension, it is stopped instead. if the same unit is listed in both fields, EXTENSION_RESTART_UNITS= takes precedence. listing any unit in either field implies EXTENSION_RELOAD_MANAGER=1, since the service manager would otherwise not yet (or no longer) see the unit files shipped by the extension. 👀

note: these settings do not take effect when the extension is merged for the system already from the initrd, and extensions should still ship a ".wants/" or ".upholds/" symlink from the unit's install target to ensure the unit starts at boot. 🫶

System extensions should not ship a /usr/lib/os-release file (because that would get merged into the host /usr/ tree, overriding the host OS version data, which is not desirable 🙅♀️). the extension-release file follows the same format and semantics, and carries the same content, as the os-release file of the OS — but it describes the resources carried in the extension image. ✨

The systemd-confext concept follows the same principle as the systemd-sysext(8) functionality, but instead of working on /usr and /opt, confext will extend only /etc. files and directories contained in the confext images outside of the /etc/ hierarchy are not merged, and hence have no effect when included in the image. the formats for these images are the same as sysext images. the merged hierarchy will be mounted with "nosuid" and (if not disabled via --noexec=false) "noexec". 🔒

Just like sysexts, confexts are strictly read-only by default. merging confexts on mutable host file systems will result in /etc/ becoming read-only. as with sysexts, mutability can be enabled via the --mutable= option. refer to "Mutability" below for more information. 💖

Okay so for confexts, we go looking in /run/confexts/, /var/lib/confexts/, /usr/lib/confexts/ and /usr/local/lib/confexts/ — but bestie, that first directory listed? Not the vibe for carrying big binary images. It's still useful for holding symlinks pointing at them tho. The primary spot for installing configuration extensions is /var/lib/confexts/. Anything that's a directory in those search paths counts as a directory based confext image, and anything with the .raw suffix counts as a disk image based confext image. Simple as.

And again, literally just like sysext images, confext images carry a /etc/extension-release.d/extension-release.NAME file that has to match the image name (with the classic escape hatch of the user.extension-release.strict xattr(7)), and the content is one or more of ID=, VERSION_ID=, and CONFEXT_LEVEL. Then confext images get checked and matched against the base OS layer. 💅

🌈 USES 🌈

The main character use case for system images is immutable environments where debugging and development tools should optionally be made available, but not baked into the immutable base OS image itself — like, strace(1) and gdb(1) should be an optionally installable addition so debugging/development is easier, iykyk. But do not misunderstand system extension images as some generic software packaging framework, because no dependency scheme exists: system extensions should carry all the files they need themselves, except for the ones already shipped in the underlying host system image. Typically, system extension images get built at the same time as the base OS image — inside the same build system, periodt.

Another use case for the whole system extension concept is temporarily overriding OS supplied resources with newer ones. Like, say you wanna install a locally compiled development version of some low-level component over the immutable OS image without doing a full OS rebuild or touching the nominally immutable image. (e.g. "install" a locally built package with

DESTDIR=/var/lib/extensions/mytest make install && systemd-sysext refresh

making it available in /usr/ as if it was installed in the OS image itself.) And this case works no matter whether the underlying host /usr/ is managed as an immutable disk image or is a traditional package manager controlled (i.e. writable) tree. Slay. ✨

With systemd-confext you can do runtime reconfiguration of OS services. Sometimes you need to swap certain configuration parameter values or restart only one specific service without deploying new code or a complete OS deployment. Basically we wanna tie the most frequently configured options to runtime updateable flags that can be changed without a system reboot. This helps reduce servicing times whenever the OS configuration needs changing. It's also a reliable tool for managing configuration, because all old configuration files disappear when the systemd-confext image is removed. Chef's kiss. 🎀

💅 MUTABILITY 💅

By default, merging system extensions on mutable host file systems will render the /usr/ and /opt/ hierarchies read-only. Merging configuration extensions has the same effect on /etc/. Mutable mode is what lets writes to these locations happen while extensions are merged.

The following modes are supported:

  1. disabled: Force immutable mode even if write routing directories exist below /var/lib/extensions.mutable/. This is the default.
  2. auto: Automatic mode. Mutability is disabled by default and only enabled if a corresponding write routing directory exists below /var/lib/extensions.mutable/.
  3. enabled: Force mutable mode and automatically create write routing directories below /var/lib/extensions.mutable/ when required.
  4. import: Force immutable mode like disabled above, but merge the contents of directories below /var/lib/extensions.mutable/ into the host file system.
  5. ephemeral: Force mutable mode like enabled above, but instead of using write routing directory below /var/lib/extensions.mutable/, systemd-sysext will use empty ephemeral directories. This means that the modifications made in the merged hierarchies will be gone when the hierarchies are unmerged.
  6. ephemeral-import: Force mutable mode like ephemeral above, but instead of ignoring the contents of write routing directories under /var/lib/extensions.mutable/, merge them into the host file system, like import does.

See "Options" below on specifying modes using the --mutable= command line option. 👑

With exception of the ephemeral mode, the mutable mode routes writes to subdirectories in /var/lib/extensions.mutable/. So:

  • Writes to /usr/ are directed to /var/lib/extensions.mutable/usr/
  • writes to /opt/ are directed to /var/lib/extensions.mutable/opt/, and
  • writes to /etc/ land in /var/lib/extensions.mutable/etc/.

If usr/, opt/, or etc/ in /var/lib/extensions.mutable/ are symlinks, then writes are directed to the symlinks' targets. Consequently, to retain mutability of a host file system, create symlinks

/var/lib/extensions.mutable/etc/ → /etc/
/var/lib/extensions.mutable/usr/ → /usr/
/var/lib/extensions.mutable/opt/ → /opt/

to route writes back to the original base directory hierarchy. 🦋

Alternatively, a temporary file system may be mounted to /var/lib/extensions.mutable/, or symlinks in /var/lib/extensions.mutable/ may point to sub-directories on a temporary file system (e.g. below /tmp/) to only allow ephemeral changes. Note that this is not the same as ephemeral mode, because the temporary file system will still exist after unmerging.

Added in version 256.

✨ COMMANDS ✨

The following commands are understood by both the sysext and confext concepts:

  • status — When invoked without any command verb, or when status is specified, the current merge status is shown, separately (for both /usr/ and /opt/ of sysext and for /etc/ of confext).

Added in version 248.

  • merge — Merges all currently installed system extension images into /usr/ and /opt/, by overmounting these hierarchies with an "overlayfs" file system combining the underlying hierarchies with those included in the extension images. This command will fail if the hierarchies are already merged. For confext, the merge happens into the /etc/ directory instead.

Added in version 248.

  • unmerge — Unmerges all currently installed system extension images from /usr/ and /opt/ for sysext and /etc/, for confext, by unmounting the "overlayfs" file systems created by merge prior.

Added in version 248.

  • refresh — A combination of unmerge and merge: if already mounted the existing "overlayfs" instance is unmounted temporarily, and then replaced by a new version. This command is useful after installing/removing system extension images, in order to update the "overlayfs" file system accordingly. If no system extensions are installed when this command is executed, the equivalent of unmerge is executed, without establishing any new "overlayfs" instance. Note that currently there's a brief moment where neither the old nor the new "overlayfs" file system is mounted. This implies that all resources supplied by a system extension will briefly disappear — even if it exists continuously during the refresh operation. 😭

Added in version 248.

  • list — A brief list of installed extension images is shown.

Added in version 248.

  • -h, --help — Print a short help text and exit.
  • --version — Print a short version string and exit.

✨ OPTIONS ✨

okay bestie, here's the whole flag buffet 🍓 every single one, no skips, we eat the entire plate.

  • --root= — do all the work relative to the root directory you name here. like, instead of dropping the "overlayfs" mount at the top-level host /usr/ and /opt/ hierarchies (that's sysext) or /etc/ (that's confext), it goes underneath whatever root dir you specify instead. ✨

Added in version 248.

  • --force — when merging system extensions into /usr/ and /opt/ for sysext, or /etc/ for confext, just ignore version incompatibilities. like, force the merge regardless of whether the version info baked into the images actually matches the host or not. yolo but make it official 💅

Added in version 248.

  • --always-refresh=yes|no — when refreshing system extensions on /usr/ and /opt/ for sysext, or /etc/ for confext, this makes it ignore the fact that the already-merged extensions match what would be merged. by default, if nothing changed, the refresh gets skipped entirely. and heads up: changes you make to an extension directory while it's merged get ignored without this flag (unless some other extension got changed). also, changing contents while merged is undefined behavior in overlayfs, so... be normal about it 😭

Added in version 260.

  • --image-policy=policy — takes an image policy string as its argument, exactly like systemd.image-policy(7) says. this policy gets enforced when operating on system extension disk images. if you don't specify one, the default is "root=verity+signed+encrypted+unprotected+absent:usr=verity+signed+encrypted+unprotected+absent" for system extensions — meaning only the root and /usr/ file systems inside the image get used. for configuration extensions the default is "root=verity+signed+encrypted+unprotected+absent". and if you're running in the initrd and operating on a system extension image stored in the /.extra/sysext/ directory, a slightly stricter policy is used by default: "root=signed+absent:usr=signed+absent". see above for the deets 🫶

Added in version 254.

  • --mutable=BOOL|auto|import|ephemeral|ephemeral-import|help — sets the mutable mode. the special value "help" will list the known values for you. 🌈
    • no — force immutable mode even if write routing directories are present. this is the default.

Added in version 256.

  • auto — turn on mutable mode individually for /usr/, /opt/, and /etc/ if write routing sub-directories or symlinks are present in /var/lib/extensions.mutable/; disable it otherwise. peep the "Mutability" section above for more on write routing. 💜

Added in version 256.

  • yes — force mutable mode. write routing directories get created in /var/lib/extensions.mutable/ if they're not there already.

Added in version 256.

  • import — immutable mode, but the contents of the write routing directories in /var/lib/extensions.mutable/ also get merged into the host file system.

Added in version 256.

  • ephemeral — force mutable mode, but the contents of the write routing directories in /var/lib/extensions.mutable/ get ignored, and any modifications you make to the host file system get discarded after unmerge. poof 💀

Added in version 256.

  • ephemeral-import — force mutable mode, with the contents of the write routing directories in /var/lib/extensions.mutable/ being merged into the host file system, but the modifications made to the host file system get discarded after unmerge. best of both worlds, kinda 🦋

Added in version 256.

  • help — list the known values and exit immediately.

Added in version 259.

Added in version 256.

  • --noexec=BOOL — when merging configuration extensions into /etc/, the "MS_NOEXEC" mount flag is used by default. this option lets you disable that.

Added in version 254.

  • --no-reload — when used with merge, unmerge or refresh, don't reload the daemon after making the changes, even if an extension that got applied requires a reload via EXTENSION_RELOAD_MANAGER= set to 1. this also skips processing any units listed via EXTENSION_RESTART_UNITS= or EXTENSION_RELOAD_OR_RESTART_UNITS=. no reload, no problem ✨

Added in version 255.

  • --no-pager — don't pipe the output into a pager.
  • --no-legend — don't print the legend, i.e. the column headers and that little footer with hints.
  • --json=MODE — shows the output formatted as JSON. expects one of "short" (the shortest possible output, no redundant whitespace or line breaks), "pretty" (a pretty version of the same, with indentation and line breaks) or "off" (to turn JSON output off — this one's the default).

🎀 EXIT STATUS 🎀

if it works, you get 0 back. that's the whole vibe. slay. 💖

👑 SEE ALSO 👑

🌸 NOTES 🌸

  1. UAPI.2 Discoverable Partitions Specification https://uapi-group.org/specifications/specs/discoverable_partitions_specification
  2. Portable Services https://systemd.io/PORTABLE_SERVICES

and that's the whole page bestie!! this was systemd 262.1, the SYSTEMD-SYSEXT(8) girlie herself, and she ate every single time 💅✨ no cap, fr fr. go merge something iconic. 🫶🌈💖

📦 where this page comes from
  • arch rolling: systemd 262.1-1 · LGPL-2.1-or-later AND CC0-1.0 AND GPL-2.0-or-later AND MIT-0
  • 💖 girlified by deepseek-ai/DeepSeek-V4.1-Flash on : a language model's rewrite of the original, under its terms