SYSTEMD-NSPAWN(1)

๐Ÿ’… SYSTEMD-NSPAWN(1) ๐Ÿ’…

the systemd-nspawn manual page โ€” spawn a command or OS in a lightweight container, bestie ๐Ÿซถ

๐ŸŒธ NAME ๐ŸŒธ

systemd-nspawn โ€” Spawn a command or OS in a lightweight container

โœจ SYNOPSIS โœจ

systemd-nspawn [OPTIONS...] [COMMAND [ARGS...]]

systemd-nspawn --boot [OPTIONS...] [ARGS...]

๐Ÿ’– DESCRIPTION ๐Ÿ’–

okay so systemd-nspawn may be used to run a command or OS in a lightweight namespace container. In many ways it is similar to chroot(1), but more powerful since it virtualizes the file system hierarchy, as well as the process tree, the various IPC subsystems, and the host and domain names. it's giving โ€œchroot but make it iconicโ€ โœจ

systemd-nspawn may be invoked on any directory tree containing an operating system tree, using the --directory= command line option. By using the --machine= option an OS tree is automatically searched for in a couple of locations, most importantly in /var/lib/machines/, the suggested directory to place OS container images installed on the system. ๐Ÿ”Ž

In contrast to chroot(1), systemd-nspawn may be used to boot full Linux-based operating systems in a container. periodt. ๐Ÿ’…

systemd-nspawn limits access to various kernel interfaces in the container to read-only, such as /sys/, /proc/sys/, or /sys/fs/selinux/. The host's network interfaces and the system clock may not be changed from within the container. Device nodes may not be created. The host system cannot be rebooted and kernel modules may not be loaded from within the container. โš ๏ธ This sandbox can easily be circumvented from within the container if user namespaces are not used. This means that untrusted code must always be run in a user namespace, see the discussion of the --private-users= option below. no cap, that part matters.

Use a tool like dnf(8), debootstrap(8), or pacman(8) to set up an OS directory tree suitable as file system hierarchy for systemd-nspawn containers. See the Examples section below for details on suitable invocation of these commands. ๐Ÿ› ๏ธ

As a safety check systemd-nspawn will verify the existence of /usr/lib/os-release or /etc/os-release in the container tree before booting a container (see os-release(5)). It might be necessary to add this file to the container tree manually if the OS of the container is too old to contain this file out-of-the-box. โœ…

systemd-nspawn may be invoked directly from the interactive command line or run as system service in the background. In this mode each container instance runs as its own service instance; a default template unit file systemd-nspawn@.service is provided to make this easy, taking the container name as instance identifier. Note that different default options apply when systemd-nspawn is invoked by the template unit file than interactively on the command line. Most importantly the template unit file makes use of the --boot option which is not the default in case systemd-nspawn is invoked from the interactive command line. Further differences with the defaults are documented along with the various supported options below. ๐Ÿค 

The machinectl(1) tool may be used to execute a number of operations on containers. In particular it provides easy-to-use commands to run containers as system services using the systemd-nspawn@.service template unit file. ๐ŸŽ›๏ธ

Along with each container a settings file with the .nspawn suffix may exist, containing additional settings to apply when running the container. See systemd.nspawn(5) for details. Settings files override the default options used by the systemd-nspawn@.service template unit file, making it usually unnecessary to alter this template file directly. ๐ŸŽ€

Note that systemd-nspawn will mount file systems private to the container to /dev/, /run/, and similar. These will not be visible outside of the container, and their contents will be lost when the container exits. ๐Ÿ’€

Note that running two systemd-nspawn containers from the same directory tree will not make processes in them see each other. The PID namespace separation of the two containers is complete and the containers will share very few runtime objects except for the underlying file system. Rather use machinectl(1)'s login or shell commands to request an additional login session in a running container.

systemd-nspawn implements the Container Interface[1] specification. ๐Ÿ“œ

While running, containers invoked with systemd-nspawn are registered with the systemd-machined(8) service that keeps track of running containers, and provides programming interfaces to interact with them. ๐Ÿฆ‹

๐Ÿ”“ UNPRIVILEGED OPERATION ๐Ÿ”“

systemd-nspawn may be invoked with or without privileges. The full functionality is currently only available when invoked with privileges. When invoked without privileges, various limitations apply, including, but not limited to:

  • Only disk image based containers are supported (i.e. --image=). Directory based ones (i.e. --directory=) are only supported if owned by the "foreign" UID range.
  • Only --private-network and --network-veth networking modes are supported.

When running in unprivileged mode, some needed functionality is provided via systemd-mountfsd.service(8) and systemd-nsresourced.service(8). ๐Ÿซถ

โš™๏ธ OPTIONS โš™๏ธ

If option --boot is specified, the arguments are used as arguments for the init program. Otherwise, COMMAND specifies the program to launch in the container, and the remaining arguments are used as arguments for this program. If --boot is not used and no arguments are specified, a shell is launched in the container. ๐Ÿš

The following options are understood:

  • -q, --quiet โ€” turns off any status output by the tool itself. When this switch is used, the only output from nspawn will be the console output of the container OS itself.

โœจ Added in version 209.

  • --settings=MODE โ€” controls whether systemd-nspawn shall search for and use additional per-container settings from .nspawn files. Takes a boolean or the special values override or trusted.

If enabled (the default), a settings file named after the machine (as specified with the --machine= setting, or derived from the directory or image file name) with the suffix .nspawn is searched in /etc/systemd/nspawn/ and /run/systemd/nspawn/. If it is found there, its settings are read and used. If it is not found there, it is subsequently searched in the same directory as the image file or in the immediate parent of the root directory of the container. In this case, if the file is found, its settings will be also read and used, but potentially unsafe settings are ignored. Note that in both these cases, settings on the command line take precedence over the corresponding settings from loaded .nspawn files, if both are specified. Unsafe settings are considered all settings that elevate the container's privileges or grant access to additional resources such as files or directories of the host. For details about the format and contents of .nspawn files, consult systemd.nspawn(5).

If this option is set to override, the file is searched, read and used the same way, however, the order of precedence is reversed: settings read from the .nspawn file will take precedence over the corresponding command line options, if both are specified.

If this option is set to trusted, the file is searched, read and used the same way, but regardless of being found in /etc/systemd/nspawn/, /run/systemd/nspawn/ or next to the image file or container root directory, all settings will take effect, however, command line arguments still take precedence over corresponding settings.

If disabled, no .nspawn file is read and no settings except the ones on the command line are in effect.

โœจ Added in version 226.

  • --cleanup โ€” clean up left-over mounts and underlying mount points used by the container, and exit without invoking any containers. This may be useful when the previous invocation of systemd-nspawn was unexpectedly terminated. This requires at least one of -M/--machine=, -D/--directory=, or -i/--image= to determine the mounts to be cleaned up.

โœจ Added in version 257.

๐Ÿ–ผ๏ธ Image Options ๐Ÿ–ผ๏ธ

  • -D, --directory= โ€” directory to use as file system root for the container.

If neither --directory=, nor --image= is specified the directory is determined by searching for a directory named the same as the machine name specified with --machine=. See machinectl(1) section "Files and Directories" for the precise search path.

In place of the directory path a .v/ versioned directory may be specified, see systemd.v(7) for details.

If neither --directory=, --image=, nor --machine= are specified, the current directory will be used. May not be specified together with --image=.

  • --template= โ€” directory or btrfs subvolume to use as template for the container's root directory. If this is specified and the container's root directory (as configured by --directory=) does not yet exist it is created as btrfs snapshot (if supported) or plain directory (otherwise) and populated from this template tree. Ideally, the specified template path refers to the root of a btrfs subvolume, in which case a simple copy-on-write snapshot is taken, and populating the root directory is instant. If the specified template path does not refer to the root of a btrfs subvolume (or not even to a btrfs file system at all), the tree is copied (though possibly in a 'reflink' copy-on-write scheme โ€” if the file system supports that), which can be substantially more time-consuming. Note that the snapshot taken is of the specified directory or subvolume, including all subdirectories and subvolumes below it, but excluding any sub-mounts. May not be specified together with --image= or --ephemeral.

Note that this switch leaves hostname, machine ID and all other settings that could identify the instance unmodified.

โœจ Added in version 219.

  • -x, --ephemeral โ€” if specified, the container is run with a temporary snapshot of its file system that is removed immediately when the container terminates. May not be specified together with --template=.

Note that this switch leaves hostname, machine ID and all other settings that could identify the instance unmodified. Please note that โ€” as with --template= โ€” taking the temporary snapshot is more efficient on file systems that support subvolume snapshots or 'reflinks' natively (btrfs or new xfs) than on more traditional file systems that do not (ext4). Note that the snapshot taken is of the specified directory or subvolume, including all subdirectories and subvolumes below it, but excluding any sub-mounts. ๐ŸŒˆ

  • --ephemeral (continued from above) โ€” with this one, literally zero modifications to the container image stick around. Nothing persists. Nada. If you want other ways to lock down how much of the image survives runtime, go look at --volatile= (it's right below, bestie ๐Ÿ’œ).

Added in version 219.

  • -i, --image= โ€” the disk image you wanna mount as the container's root directory. You give it a path to a regular file or to a block device node. And that file/block device has to contain one of these, no cap:
    • An MBR partition table with a single partition of type 0x83 that's marked bootable.
    • A GUID partition table (GPT) with a single partition of type 0fc63daf-8483-4772-8e79-3d69d8477de4.
    • A GUID partition table (GPT) with a marked root partition, which gets mounted as the root directory of the container. GPT images can optionally also carry a home and/or a server data partition, and those get mounted to the appropriate places in the container. All of these partitions have to be identified by the partition types defined by the UAPI.2 Discoverable Partitions Specification[2].
    • No partition table at all โ€” just a single file system spanning the whole image.

On GPT images, if an EFI System Partition (ESP) is discovered, it's automatically mounted to /efi (or /boot as a fallback) โ€” but only in case a directory by that name exists and is empty. ๐ŸŽ€

Partitions encrypted with LUKS are automatically decrypted. And on GPT images, dm-verity data integrity hash partitions get set up if the root hash for them is specified using the --root-hash= option.

Single file system images (i.e. file systems with no partition table wrapped around them) can be opened using dm-verity if the integrity data is passed via the --root-hash= and --verity-data= options (and optionally --root-hash-sig=).

Any other partitions โ€” foreign partitions, swap partitions, whatever โ€” are not mounted. And this option may not be specified together with --directory= or --template=. Pick a lane, babe. ๐Ÿ’…

In place of the image path you can also specify a .v/ versioned directory โ€” see systemd.v(7) for the deets.

Added in version 211.

  • --image-policy= policy โ€” takes an image policy string as its argument, exactly as described in systemd.image-policy(7). The policy gets enforced when operating on the disk image you specified via --image=, see above. If you don't specify it, it defaults to:
root=verity+signed+encrypted+unprotected+absent:usr=verity+signed+encrypted+unprotected+absent:home=encrypted+unprotected+absent:srv=encrypted+unprotected+absent:esp=unprotected+absent:xbootldr=unprotected+absent:tmp=encrypted+unprotected+absent:var=encrypted+unprotected+absent

which basically means: all recognized file systems in the image get used, but the swap partition does not. iykyk ๐Ÿฆ‹

Added in version 254.

  • --mstack= โ€” mount stack to mount the root directory for the container from. Takes a path to a directory implementing systemd.mstack(7). This is the one for running a container of an "overlayfs" assembled from a bunch of layers โ€” possibly writable, and beefed up with extra bind mounts. Main character layering energy โœจ

Added in version 260.

  • --oci-bundle= โ€” takes the path to an OCI runtime bundle to invoke, as specified in the OCI Runtime Specification[3]. In this case, no .nspawn file gets loaded, and the root directory plus various settings are read from the OCI runtime JSON data โ€” but data passed on the command line takes precedence. (Command line stays the queen ๐Ÿ‘‘.)

Added in version 242.

  • --read-only โ€” mounts the container's root file system (and any other file systems contained in the container image) read-only. Heads up: this does not affect additional mounts you make with --bind=, --tmpfs= and similar options. Also, this mode is implied if the container image file or directory is marked read-only itself, and it's also implied if --volatile= is used. In that case, the container image on disk is strictly read-only, while changes are permitted but kept non-persistently in memory only. More deets below. ๐ŸงŠ
  • --volatile, --volatile= MODE โ€” boots the container in volatile mode. Let me break this down because there are modes, bestie:

When no mode parameter is passed โ€” or when mode is specified as yes โ€” full volatile mode is enabled. That means the root directory gets mounted as a mostly unpopulated "tmpfs" instance, and /usr/ from the OS tree gets mounted into it read-only. So the system starts up with a read-only OS image, but pristine state and configuration, and any changes are lost on shutdown. ๐Ÿซง

When the mode parameter is specified as state, the OS tree is mounted read-only, but /var/ is mounted as a writable "tmpfs" instance into it. So the system starts up with read-only OS resources and configuration, but pristine state โ€” and any changes to that state are lost on shutdown.

When the mode parameter is specified as overlay, the read-only root file system gets combined with a writable tmpfs instance through "overlayfs", so it looks like it normally would, but any changes are applied to the temporary file system only and are lost when the container is terminated. ๐Ÿ˜ญ

When the mode parameter is specified as no โ€” that's the default โ€” the whole OS tree is made available writable (unless --read-only is specified, see above).

Ok so important caveat time, don't skip this: if you choose one of the volatile modes, its effect is limited to the root file system (or /var/ in the case of state), and any other mounts placed in the hierarchy are unaffected โ€” regardless of whether they're established automatically (e.g. the EFI system partition that might be mounted to /efi/ or /boot/) or explicitly (e.g. through an additional command line option such as --bind=, see below). This means: even if --volatile=overlay is used, changes to /efi/ or /boot/ are prohibited in case such a partition exists in the container image you're operating on. And even if --volatile=state is used, the hypothetical file /etc/foobar is potentially writable if --bind=/etc/foobar is used to mount it from outside the read-only container /etc/ directory. Sneaky, right? ๐Ÿ™ˆ

The --ephemeral option is closely related to this setting, and provides similar behaviour by making a temporary, ephemeral copy of the whole OS image and executing that. For further details, see above.

The --tmpfs= and --overlay= options provide similar functionality, but for specific sub-directories of the OS image only. Details below.

This option gives containers roughly what the "systemd.volatile=" kernel command line switch gives host systems. See kernel-command-line(7) for details.

Also, setting this option to yes or state will only work correctly with operating systems in the container that can boot up with only /usr/ mounted, and that are able to automatically populate /var/ (and /etc/ in the case of "--volatile=yes"). Specifically, that means operating systems following the historic split of /bin/ and /lib/ (and related directories) from /usr/ โ€” i.e. where the former are not symlinks into the latter โ€” are not supported by "--volatile=yes" as container payload. The overlay option doesn't require any particular preparations in the OS, but do note that "overlayfs" behaviour differs from regular file systems in a number of ways, and hence compatibility is limited. ๐Ÿ’€

Added in version 216.

  • --root-hash= โ€” takes a data integrity (dm-verity) root hash, specified in hexadecimal. This option enables data integrity checks using dm-verity, if the image you're using contains the appropriate integrity data (see above). The hash you specify must match the root hash of the integrity data, and it's usually at least 256 bits long (and hence 64 formatted hexadecimal characters) โ€” in the case of SHA256, for example. ๐Ÿ”

If this option isn't specified, but the image file carries the "user.verity.roothash" extended file attribute (see xattr(7)), then the root hash is read from there, also as formatted hexadecimal characters. If the extended file attribute isn't found (or isn't supported by the underlying file system), but a file with the .roothash suffix is found next to the image file, bearing otherwise the same name โ€” except if the image has the .raw suffix, in which case the root hash file must not have it in its name โ€” then the root hash is read from that and automatically used, also as formatted hexadecimal characters. It just finds it. Iconic. ๐ŸŽ€

Note that this configures the root hash for the root file system. Disk images may also contain separate file systems for the /usr/ hierarchy, which may be Verity protected as well. The root hash for that protection may be configured via the "user.verity.usrhash" extended file attribute, or via a .usrhash file adjacent to the disk image โ€” following the same format and logic as for the root hash for the root file system described here. And just so you know: there's currently no switch to configure the root hash for /usr/ from the command line. Tragic, I know. ๐Ÿ˜ญ

Also see the RootHash= option in systemd.exec(5).

Added in version 233.

  • --root-hash-sig= โ€” takes a PKCS7 signature of the --root-hash= option. The semantics are the same as for the RootHashSignature= option, see systemd.exec(5). โœ๏ธ

Added in version 246.

  • --verity-data= โ€” takes the path to a data integrity (dm-verity) file. This option enables data integrity checks using dm-verity, if a root-hash is passed and if the image you're using doesn't itself contain the integrity data. The integrity data must be matched by the root hash. If this option isn't specified, but a file with the .verity suffix is found next to the image file, bearing otherwise the same name โ€” except if the image has the .raw suffix, in which case the verity data file must not have it in its name โ€” then the verity data is read from it and automatically used. ๐Ÿฆ‹

Added in version 246.

  • --pivot-root= โ€” pivots the specified directory to / inside the container, and then either unmounts the container's old root, or pivots it to another specified directory. Takes one of two things:
    • A path argument โ€” in which case the specified path will be pivoted to / and the old root will be unmounted;
    • or a colon-separated pair of new root path and pivot destination for the old root. The new root path gets pivoted to /, and the old / gets pivoted to the other directory.

Both paths must be absolute, and are resolved in the container's file system namespace. No relative nonsense here, bestie. ๐Ÿ’…

Like, this one's for containers that have, like, several bootable directories sitting in them โ€” think multiple ostree(1) deployments, bestie. It literally emulates what the boot loader and the initrd do normally: pick which directory gets mounted as the root, then start the container's PID 1 in it. Iconic behavior, honestly. ๐ŸŽ€

Added in version 233.


โœจ Execution Options โœจ

  • -a, --as-pid2 โ€” runs your shell (or whatever program you picked) as process ID (PID) 2 instead of PID 1 (init). So here's the tea: by default, if you don't use this option and you don't use --boot, the program you picked runs as PID 1 โ€” and that mode is only suitable for programs that actually know about the special semantics PID 1 has on UNIX. Like, PID 1 has to reap every process that gets reparented to it, and it should implement sysvinit-compatible signal handling (specifically: it needs to reboot on SIGINT, reexecute on SIGTERM, reload configuration on SIGHUP, and so on). Messy. ๐Ÿ™ˆ With --as-pid2, a minimal stub init process runs as PID 1 and your selected program is executed as PID 2 โ€” so it does not need to implement any of those special semantics. The stub init will reap processes as necessary and react to signals appropriately. Bestie, use this mode for basically everything unless the command has been modified to run correctly as PID 1. Or, in other words: this switch should be used for pretty much all commands, except when the command refers to an init or shell implementation โ€” those are generally capable of running correctly as PID 1. This option may not be combined with --boot. ๐Ÿ’…

Added in version 229.

  • -b, --boot โ€” automatically searches for an init program and invokes it as PID 1, instead of a shell or a user-supplied program. If you use this option, the arguments you specified on the command line are used as arguments for the init program. This option may not be combined with --as-pid2.

Okay so here's the relationship between the invocation modes and --as-pid2 (see above), laid out all cute:

  • Neither --as-pid2 nor --boot specified โ€” the passed parameters are interpreted as the command line, which is executed as PID 1 in the container.
  • --as-pid2 specified โ€” the passed parameters are interpreted as the command line, which is executed as PID 2 in the container. A stub init process is run as PID 1.
  • --boot specified โ€” an init program is automatically searched for and run as PID 1 in the container. The passed parameters are used as invocation parameters for this process.
Heads up: --boot is the default mode of operation if the systemd-nspawn@.service template unit file is used. โญ

When --boot is used, the passed parameters are processed the same way the kernel processes its command line before handing it to PID 1: any "KEY=VALUE" assignment whose "KEY" does not contain a "." is exported as an environment variable for PID 1 (with "-" in the key replaced by "_", as if specified via --setenv=), while all other parameters (including "systemd.*=" assignments) are passed as arguments to the init program. fr fr, that's exactly how it works. ๐Ÿงš

  • --chdir= โ€” changes to the specified working directory before invoking the process in the container. Expects an absolute path in the container's file system namespace. ๐Ÿฆ‹

Added in version 229.

  • -E NAME[=VALUE], --setenv= NAME[=VALUE] โ€” specifies an environment variable to pass to the init process in the container. You can use this to override the default variables or to set additional ones. It may be used more than once to set multiple variables. When "=" and VALUE are omitted, the value of the variable with the same name in the program environment will be used. ๐Ÿ’–

Added in version 209.

  • -u, --uid= โ€” after transitioning into the container, changes to the specified user defined in the container's user database. And like every other systemd-nspawn feature: this is not a security feature and provides protection against accidental destructive operations only. No cap. This option was previously named --user=; the old name is still accepted but deprecated. The short option -u is compatible with both old and new versions of systemd-nspawn. ๐Ÿ‘‘
โš ๏ธ Real talk: if credentials are used in combination with a non-root --uid= (e.g.: --set-credential= or --load-credential=), then --no-new-privileges=yes must be used, and --boot or --as-pid2 must not be used โ€” because the credentials would otherwise be unreadable by the container due to missing privileges after switching to the specified user. ๐Ÿ˜ญ
  • --kill-signal= โ€” specifies the process signal to send to the container's PID 1 when nspawn itself receives SIGTERM, in order to trigger an orderly shutdown of the container. Defaults to SIGRTMIN+3 if --boot is used (on systemd-compatible init systems SIGRTMIN+3 triggers an orderly shutdown). If --boot is not used and this option is not specified, the container's processes are terminated abruptly via SIGKILL. ๐Ÿ’€ For a list of valid signals, see signal(7).

Added in version 220.

  • --notify-ready= โ€” configures support for notifications from the container's init process. --notify-ready= takes a boolean. If false, systemd-nspawn notifies the calling service manager with a "READY=1" message when the init process is created. If true, it waits for a "READY=1" message from the init process in the VM before sending its own to the service manager. For more details about notifications see sd_notify(3). โœจ

Defaults to false. (Note that this is unlike the option of the same name to systemd-vmspawn(1) that defaults to true.) ๐Ÿ™ˆ

And okay, bonus lore: if systemd-nspawn itself is invoked with a $NOTIFY_SOCKET set in its environment (i.e. it is itself supervised by a service manager that uses the sd_notify(3) protocol), "FDSTORE=1" and "FDSTOREREMOVE=1" messages received from the container payload (along with any accompanying file descriptors and "FDNAME=" tag) are forwarded one level up to the enveloping service manager. This allows the file descriptor store of services running inside the container to be preserved across container restarts (and, transitively, across restarts, re-execs, soft-reboots and LUO-based kexecs of any outer service manager), provided FileDescriptorStoreMax=/FileDescriptorStorePreserve=yes are configured on the unit running systemd-nspawn. See the File Descriptor Store[4] documentation for details. ๐Ÿซถ

Added in version 231.

  • --suppress-sync= โ€” expects a boolean argument. If true, turns off any form of on-disk file system synchronization for the container payload. This means all system calls such as sync(2), fsync(), syncfs(), โ€ฆ will execute no operation, and the O_SYNC/O_DSYNC flags to open(2) and related calls will be made unavailable.
โš ๏ธ Potentially dangerous, bestie: assumed data integrity guarantees to the container payload are not actually enforced (i.e. data assumed to have been written to disk might be lost if the system is shut down abnormally). ๐Ÿ˜ญ However, this can dramatically improve container runtime performance โ€” as long as those guarantees are not required or desirable, for example because any data written by the container is of temporary, redundant nature, or just an intermediary artifact that will be further processed and finalized by a later step in a pipeline. Defaults to false.

Added in version 250.


๐ŸŒธ System Identity Options ๐ŸŒธ

  • -M, --machine= โ€” sets the machine name for this container. This name may be used to identify this container during its runtime (for example in tools like machinectl(1) and similar), and it's used to initialize the container's hostname (which the container can choose to override, however). If not specified, the last component of the root directory path of the container is used, possibly suffixed with a random identifier in case --ephemeral mode is selected. And if the root directory selected is the host's root directory, the host's hostname is used as default instead. ๐ŸŽ€

Added in version 202.

  • --hostname= โ€” controls the hostname to set within the container, if it's different from the machine name. Expects a valid hostname as argument. If you use this option, the kernel hostname of the container will be set to this value; otherwise it will be initialized to the machine name as controlled by the --machine= option described above. The machine name is used for various aspects of identification of the container from the outside, while the kernel hostname configurable with this option is useful for the container to identify itself from the inside. It's usually a good idea to keep both forms of identification synchronized, to avoid confusion. So it's hence recommended to avoid usage of this option, and use --machine= exclusively. ๐Ÿ’… Note that regardless of whether the container's hostname is initialized from the name set with --hostname= or the one set with --machine=, the container can later override its kernel hostname freely on its own as well. Slay.

Added in version 239.

  • --uuid= โ€” sets the UUID for the container to whatever you pass in. The init system will seed /etc/machine-id from it, but only if that file isn't populated yet โ€” so heads up, this literally does nothing if /etc/machine-id in the container already has content. ๐Ÿ’…

๐ŸŽ€ Property Options ๐ŸŽ€

  • -S, --slice= โ€” drops the container into the slice you name instead of the default machine.slice. Key caveat bestie: this only applies if the machine runs in its own scope unit, meaning not with --keep-unit. โœจ

Added in version 206.

  • --property= โ€” slaps a unit property onto the scope unit registered for the machine. Again, only if the machine gets its own scope unit (so no --keep-unit). Takes unit property assignments in the exact same format as systemctl set-property โ€” super handy for setting memory limits and that kind of thing on the container. ๐Ÿซถ

Added in version 220.

  • --register= โ€” controls whether the container gets registered with systemd-machined(8). Takes a boolean argument or "auto", and defaults to "auto". Turn it on when the container runs a full Operating System (more specifically: a system and service manager as PID 1) โ€” it's what makes the container reachable via machinectl(1) and visible in tools like ps(1). If the container does not run a service manager, set this to "no". And when it's "auto", registration is attempted but any failures are just ignored. ๐Ÿ˜Œ

Added in version 209.

  • --keep-unit โ€” instead of spinning up a fresh transient scope unit for the container, just reuse the service or scope unit that systemd-nspawn was already invoked in. If --register=yes is set, that unit is what gets registered with systemd-machined(8). Reach for this when systemd-nspawn is invoked from inside a service unit whose whole entire purpose is running one single systemd-nspawn container. Not available if you run it from a user session, sorry. ๐Ÿ™ˆ

Also important: passing --keep-unit kills the effect of --slice= and --property=. Want to disable any kind of unit allocation or registration with systemd-machined? Use --keep-unit together with --register=no. periodt.

Added in version 209.


๐Ÿฆ‹ User Namespacing Options ๐Ÿฆ‹

  • --private-users= โ€” controls user namespacing. When enabled, the container runs with its own private set of UNIX user and group ids (UIDs and GIDs). This means mapping the private UIDs/GIDs used inside the container (starting at the container's root user 0 and going up) onto a range of UIDs/GIDs on the host that aren't used for anything else (usually the range beyond the host's UID/GID 65536). The parameter works like this: โญ
    1. If one or two colon-separated numbers are given, user namespacing is turned on. The first number is the first host UID/GID to assign to the container, the second is how many host UIDs/GIDs to assign. Omit the second and it assigns 65536 UIDs/GIDs.
    2. If the parameter is "yes", user namespacing is turned on. The UID/GID range gets determined automatically from the file ownership of the root directory of the container's directory tree. For this to work, prep the directory tree in advance and make sure every file and directory in it is owned by UIDs/GIDs in the range you want to use. Also make sure any file ACLs in play exclusively reference UIDs/GIDs in the appropriate range. In this mode the number of UIDs/GIDs assigned to the container is 65536, and the owner UID/GID of the root directory must be a multiple of 65536.
    3. The special value "pick" turns on user namespacing with the UID/GID range chosen automatically. Step one: it reads the file owner UID/GID of the root directory of the container's directory tree, then checks that no other container is currently using it. If that check passes, that determined UID/GID range is used, just like with "yes". If the check fails (so the range from the root directory's file owner is already in use elsewhere), a new โ€” currently unused โ€” UID/GID range of 65536 UIDs/GIDs is randomly chosen between the host UID/GIDs of 524288 and 1878982656, always starting at a multiple of 65536, and consistently hashed from the machine name where possible. This setting implies --private-users-ownership=auto (see below), which can mean the files and directories in the container's directory tree end up owned by the appropriate users of whichever range got picked. Using this makes user namespace behavior fully automatic. ๐Ÿ˜ญ Fair warning though: the first invocation of a previously unused container image might pick a brand-new UID/GID range for it, which means the (possibly expensive) file ownership adjustment has to happen. Subsequent invocations are cheap โ€” unless the picked UID/GID range got reassigned to a different use by then, obviously.
    4. If the parameter is "no", user namespacing is turned off. This is the default when systemd-nspawn is invoked directly. (Note that the systemd-nspawn@.service unit enables private users.) ๐Ÿ’€ This option is not secure and must not be used to run untrusted code.
    5. If the parameter is "identity", user namespacing is employed with an identity mapping for the first 65536 UIDs/GIDs. This is mostly equivalent to --private-users=0:65536. It does not provide UID/GID isolation โ€” since all host and container UIDs/GIDs are chosen identically โ€” but it does provide process capability isolation, and it may be useful if proper user namespacing with distinct UID maps isn't possible. ๐Ÿ’€ This option is not secure and must not be used to run untrusted code.
    6. If the parameter is "managed", user namespacing is employed in managed mode, i.e. allocation of a UID range is delegated to systemd-nsresourced.service(8). This mode is selected by default if invoked unprivileged, but you can also request it explicitly when privileged. In this mode a 64K UID range is automatically picked.

Bestie, the rec is to assign at least 65536 UIDs/GIDs to each container so the usable UID/GID range inside covers 16 bits. For best security, don't hand out overlapping UID/GID ranges to multiple containers. So yeah โ€” good idea to use the upper 16 bits of the host 32-bit UIDs/GIDs as the container identifier, while the lower 16 bits encode the container UID/GID actually used. This is in fact exactly the behavior enforced by --private-users=pick. ๐Ÿ‘‘

When user namespaces are used, the GID range assigned to each container is always chosen identical to the UID range.

In most cases, --private-users=managed (or, when privileged, --private-users=pick too) is the recommended option โ€” user namespacing is advised for security, and this option massively enhances container security while operating fully automatically in most cases. slay. โœจ

Note that the picked UID/GID range is not written to /etc/passwd or /etc/group. In fact, the allocation of the range isn't stored persistently at all, except possibly in the file ownership of the files and directories of the container โ€” see --private-users-ownership=.

Also note: when user namespacing is used without UID mapping (see below), file ownership on disk reflects that, and all of the container's files and directories are owned by the container's effective user and group IDs. Which means copying files from and to the container image requires correcting the numeric UID/GID values according to the UID/GID shift applied. ๐ŸŽ€

And for fully unprivileged operation in "managed" mode, any directory image should be owned by the foreign UID range.

Added in version 220.

  • --private-users-ownership= โ€” controls how the container image's UIDs and GIDs get adjusted to match the UID/GID range chosen with --private-users= (see above). Takes one of "off" (leave the image exactly as is), "chown" (recursively chown() the container's directory tree as needed), "map" (to use transparent ID mapping mounts from UID 0 to the target UID range), "foreign" (the same, but from the foreign UID range base), or "auto" to automatically use "map" or "foreign" where available and applicable, and "chown" where not. ๐ŸŒˆ

If "chown" is selected, every file and directory in the container's directory tree gets adjusted so it's owned by the appropriate UIDs/GIDs selected for the container (see above). This operation is potentially expensive, since it involves iterating through the full directory tree of the container. And besides actual file ownership, file ACLs get adjusted as well. ๐Ÿ˜ญ

Typically "foreign" or "map" is the best choice โ€” it transparently maps UIDs/GIDs in memory as needed without modifying the image, and without needing an expensive recursive adjustment operation. However, it isn't available on all file systems currently.

The --private-users-ownership=auto option is implied if --private-users=pick is used. This option has no effect if user namespacing is not used.

Oh also โ€” systemd-dissect(1)'s --shift switch can be used to shift UID/GID ownership from or to the 0, foreign, or a specific container UID/GID base outside of any systemd-nspawn invocation. ๐Ÿซถ

Added in version 230.

  • --private-users-delegate= โ€” takes a non-negative integer. Requests that the specified number of additional 64K UID/GID ranges get delegated into the container's user namespace. These delegated ranges are mapped 1:1 (i.e. the same UID/GID values are used inside and outside the user namespace) and can be used by nested containers to allocate their own transient UID/GID ranges via systemd-nsresourced.service(8). ๐Ÿงš

This option requires --private-users=managed, since the delegation is performed by systemd-nsresourced.service(8) as part of the user namespace allocation. The maximum number of delegated ranges is 16. Defaults to 0, i.e. no delegation.

When you hand this option a non-zero value, the systemd-nsresourced.service(8) Varlink socket (/run/systemd/io.systemd.NamespaceResource) gets automatically bind-mounted into the container, along with the discovery symlinks it needs in /run/systemd/userdb/ and /run/varlink/registry/. Which means processes inside the container can hit up systemd-nsresourced on the host and allocate nested user namespaces out of the delegated ranges. iykyk ๐Ÿ’ซ

Added in version 260.

  • -U โ€” if your kernel has the user namespaces feature, this is literally just shorthand for --private-users=pick --private-users-ownership=auto. No user namespaces on your kernel? Then it means --private-users=no. Either way, it does the work for you ๐Ÿ’…

Fun fact: -U is the default if you're using the systemd-nspawn@.service template unit file.

Bestie, careful โ€” you can undo the effect of --private-users-ownership=chown (or plain -U) on the filesystem, by redoing the whole operation with the first UID set to 0: systemd-nspawn ... --private-users=0 --private-users-ownership=chown

Added in version 230.

๐ŸŒ Networking Options ๐ŸŒ

  • --private-network โ€” cuts the container's networking off from the host, completely. Every network interface goes away inside the container except the loopback device, plus whatever you named with --network-interface= and whatever got set up with --network-veth. Heads up: using this option adds the CAP_NET_ADMIN capability to the set the container keeps โ€” you can turn that off again with --drop-capability=. And if you don't pass this option (or one of the ones below that implies it), the container gets full access to the host network. ๐Ÿ™ˆ
  • --network-interface= โ€” assigns the network interface you name to the container. You can pass just a single interface name, which refers to the name on the host, or a colon-separated pair of interfaces: first one is the host name, second one is the name inside the container. When the container exits, the interface gets moved back to the calling namespace and renamed to its original name. And yes, --network-interface= implies --private-network. You can use this option more than once to shove multiple network interfaces into the container ๐Ÿซถ

Ngl, any interface you specify this way has to already exist the moment the container starts. So if the container is meant to boot automatically via a systemd-nspawn@.service unit file instance, it's probably smart to add a unit file drop-in to the service instance (like /etc/systemd/system/systemd-nspawn@foobar.service.d/50-network.conf) containing something like:

[Unit]
Wants=sys-subsystem-net-devices-ens1.device
After=sys-subsystem-net-devices-ens1.device

That little guy makes sure activating the container service waits until the "ens1" network interface has actually shown up. Which you need, because hardware probing is fully asynchronous and network interfaces might only get discovered later in the boot process โ€” after the container would have normally started without these explicit dependencies. โœจ

Added in version 209.

  • --network-macvlan= โ€” creates a "macvlan" interface off the Ethernet network interface you specify and adds it to the container. Takes either a single interface name (the host's name) or a colon-separated pair, where the first references the host name and the second the name in the container. A "macvlan" interface is a virtual interface that adds a second MAC address to an already-existing physical Ethernet link. If you don't define a container interface name, the interface inside the container gets named after the host interface with "mv-" stuck on the front. Note that --network-macvlan= implies --private-network. Use it more than once and you get multiple network interfaces in the container ๐Ÿ’–

Same deal as --network-interface=: the underlying Ethernet network interface must already exist when the container starts, so those unit file drop-ins described above are probably gonna be useful.

Added in version 211.

  • --network-ipvlan= โ€” creates an "ipvlan" interface off the Ethernet network interface you specify and adds it to the container. Takes a single interface name (host side) or a colon-separated pair โ€” first is the host name, second is the container name. An "ipvlan" interface is a virtual interface, kinda like a "macvlan" interface, except it uses the same MAC address as the underlying interface. If no container interface name is defined, the interface in the container is named after the host interface prefixed with "iv-". And yep, --network-ipvlan= implies --private-network. Multiple uses = multiple interfaces ๐ŸŒธ

Again, exactly like --network-interface=: the underlying Ethernet network interface has to exist already at container start time, and similar unit file drop-ins might come in clutch.

Added in version 219.

  • -n, --network-veth โ€” creates a virtual Ethernet link ("veth") between host and container. The host side of that link shows up as a network interface named after the container's name (whatever you gave --machine=), prefixed with "ve-". The container side gets the name "host0". And --network-veth implies --private-network, obviously ๐Ÿ’…

So here's the tea: systemd-networkd.service(8) ships a network file by default โ€” /usr/lib/systemd/network/80-container-ve.network โ€” matching the host-side interfaces created this way. It's got the settings to enable automatic address provisioning on the new virtual link via DHCP, plus automatic IP routing onto the host's external network interfaces. It also ships /usr/lib/systemd/network/80-container-host0.network, matching the container-side interface created this way, with settings for client side address assignment via DHCP. So if systemd-networkd is running on both the host and inside the container, automatic IP communication from the container to the host just works, with further connectivity out to the external network. Slay ๐Ÿ”ฅ

Also: --network-veth is the default if the systemd-nspawn@.service template unit file is used.

Ok important caveat ๐Ÿ‘‘ โ€” on Linux, network interface names max out at 15 characters, but container names can be up to 64 characters long. Since this option derives the host-side interface name from the container name, the name may get truncated. So you gotta make sure interface names stay unique in that case โ€” or, even better, just don't pick container names longer than 12 characters and dodge the truncation entirely. If the name does get truncated, systemd-nspawn automatically appends a 4-digit hash value to reduce the chance of collisions. But bestie, that hash algorithm is not collision-free. (Peep systemd.net-naming-scheme(7) for details on older naming algorithms for this interface.) Alternatively, you can use the --network-veth-extra= option, which lets you configure the host-side interface name freely, independent of the container name โ€” though that might need a bit more extra configuration if you want bridging similar to --network-bridge= ๐Ÿ™‚

Added in version 209.

  • --network-veth-extra= โ€” adds an extra virtual Ethernet link between host and container. Takes a colon-separated pair: host interface name and container interface name. You can leave the second one out, in which case the container and host sides get the same name. This switch is independent of --network-veth and โ€” plot twist โ€” can be used multiple times, and lets you configure the network interface names. Note that --network-bridge= has no effect on interfaces created with --network-veth-extra= ๐Ÿฆ‹

Added in version 228.

  • --network-bridge= โ€” adds the host side of the Ethernet link created with --network-veth to the Ethernet bridge interface you name. Expects a valid network interface name of a bridge device as the argument. Note that --network-bridge= implies --network-veth. If you use this option, the host side of the Ethernet link uses the "vb-" prefix instead of "ve-". And regardless of which naming prefix you end up with, the same Linux network interface name length limits apply, along with all the complications that creates (details above, bestie, scroll up ๐Ÿ˜ญ).

Same as --network-interface=: the underlying bridge network interface must already exist when the container starts, so those unit file drop-ins described earlier might help.

Added in version 209.

  • --network-zone= โ€” creates a virtual Ethernet link ("veth") to the container and adds it to an automatically managed Ethernet bridge interface. That bridge interface is named after whatever argument you passed, prefixed with "vz-". It gets created automatically when the first container configured for that name starts, and removed automatically when the last container configured for that name exits. So every bridge interface configured this way only exists as long as at least one container referencing it is running. It's very similar to --network-bridge=, except for this automatic creation/removal of the bridge device โœจ

This setting makes it easy to drop multiple related containers onto a common, virtual Ethernet-based broadcast domain โ€” which we're calling a "zone" ๐ŸŽ€. Each container can only be part of one zone, but each zone can contain any number of containers. Each zone is referenced by its name. Names can be whatever you want (as long as they form valid network interface names once prefixed with "vz-"), and it's enough to pass the same name to the --network-zone= switch of the various concurrently running containers to join them all in one zone. Periodt.


Okay so, heads up bestie: systemd-networkd.service(8) includes by default a network file /usr/lib/systemd/network/80-container-vz.network matching the bridge interfaces created this way, and that file has settings to enable automatic address provisioning on the created virtual network via DHCP, plus automatic IP routing onto the host's external network interfaces. Using --network-zone= is hence in most cases fully automatic and sufficient to connect multiple local containers in a joined broadcast domain to the host, with further connectivity to the external network. Added in version 230.

  • --network-namespace-path= โ€” takes the path to a file representing a kernel network namespace that the container shall run in. The specified path should refer to a (possibly bind-mounted) network namespace file, as exposed by the kernel below /proc/$PID/ns/net. This makes the container enter the given network namespace. One of the typical use cases is to give a network namespace under /run/netns created by ip-netns(8), for example, --network-namespace-path=/run/netns/foo. Note that this option cannot be used together with other network-related options, such as --private-network or --network-interface=. Added in version 236.
  • -p, --port= โ€” if private networking is enabled, maps an IP port on the host onto an IP port on the container. Takes a protocol specifier (either "tcp" or "udp"), separated by a colon from a host port number in the range 1 to 65535, separated by a colon from a container port number in the range from 1 to 65535. The protocol specifier and its separating colon may be omitted, in which case "tcp" is assumed. The container port number and its colon may be omitted, in which case the same port as the host port is implied. This option is only supported if private networking is used, such as with --network-veth, --network-zone=, --network-bridge=. Added in version 219.

๐Ÿ›ก๏ธ Security Options ๐Ÿ›ก๏ธ

  • --capability= โ€” list one or more additional capabilities to grant the container. Takes a comma-separated list of capability names, see capabilities(7) for more information. Note that the following capabilities will be granted in any way: CAP_AUDIT_CONTROL, CAP_AUDIT_WRITE, CAP_CHOWN, CAP_DAC_OVERRIDE, CAP_DAC_READ_SEARCH, CAP_FOWNER, CAP_FSETID, CAP_IPC_OWNER, CAP_KILL, CAP_LEASE, CAP_LINUX_IMMUTABLE, CAP_MKNOD, CAP_NET_BIND_SERVICE, CAP_NET_BROADCAST, CAP_NET_RAW, CAP_SETFCAP, CAP_SETGID, CAP_SETPCAP, CAP_SETUID, CAP_SYS_ADMIN, CAP_SYS_BOOT, CAP_SYS_CHROOT, CAP_SYS_NICE, CAP_SYS_PTRACE, CAP_SYS_RESOURCE, CAP_SYS_TTY_CONFIG. Also CAP_NET_ADMIN is retained if --private-network is specified. If the special value "all" is passed, all capabilities are retained. If the special value of "help" is passed, the program will print known capability names and exit. This option sets the bounding set of capabilities which also limits the ambient capabilities as given with the --ambient-capability=. Added in version 186.
  • --drop-capability= โ€” specify one or more additional capabilities to drop for the container. This allows running the container with fewer capabilities than the default (see above). If the special value of "help" is passed, the program will print known capability names and exit. This option sets the bounding set of capabilities which also limits the ambient capabilities as given with the --ambient-capability=. Added in version 209.
  • --ambient-capability= โ€” specify one or more additional capabilities to pass in the inheritable and ambient set to the program started within the container. The value "all" is not supported for this setting. All capabilities specified here must be in the set allowed with the --capability= and --drop-capability= options. Otherwise, an error message will be shown. This option cannot be combined with the boot mode of the container (as requested via --boot). If the special value of "help" is passed, the program will print known capability names and exit. Added in version 248.
  • --no-new-privileges= โ€” takes a boolean argument. Specifies the value of the PR_SET_NO_NEW_PRIVS flag for the container payload. Defaults to off. When turned on the payload code of the container cannot acquire new privileges, i.e. the setuid file bit as well as file system capabilities will not have an effect anymore. See prctl(2) for details about this flag. Added in version 239.
  • --system-call-filter= โ€” alter the system call filter applied to containers. Takes a space-separated list of system call names or group names (the latter prefixed with "@", as listed by the syscall-filter command of systemd-analyze(1)). Passed system calls will be permitted. The list may optionally be prefixed by "~", in which case all listed system calls are prohibited. If this command line option is used multiple times the configured lists are combined. If both a positive and a negative list (that is one system call list without and one with the "~" prefix) are configured, the negative list takes precedence over the positive list. Note that systemd-nspawn always implements a system call allow list (as opposed to a deny list!), and this command line option hence adds or removes entries from the default allow list, depending on the "~" prefix. Note that the applied system call filter is also altered implicitly if additional capabilities are passed using the --capabilities=. Added in version 235.
  • --restrict-address-families= โ€” restrict the socket address families accessible to the container. Takes a space-separated list of address family names, such as AF_INET, AF_INET6 or AF_UNIX. When prefixed with "~" the listed address families will be prohibited, otherwise they will be permitted (allowlisted). Use the special value "none" to prohibit all address families. This option may be specified more than once, in which case the configured lists are combined. If both a positive and a negative list are configured, the negative list takes precedence over the positive list. Note that currently this option defaults to no restrictions, i.e. all address families are accessible. In a future version of systemd, the default will be changed to restrict address families to AF_INET, AF_INET6 and AF_UNIX. Use --restrict-address-families= (with an empty argument) or set RestrictAddressFamilies= in a .nspawn file to opt out of filtering explicitly. Added in version 261.
  • -Z, --selinux-context= โ€” sets the SELinux security context to be used to label processes in the container. Added in version 209.
  • -L, --selinux-apifs-context= โ€” sets the SELinux security context to be used to label files in the virtual API file systems in the container. Added in version 209.

๐Ÿงธ Resource Options ๐Ÿงธ

  • --rlimit= โ€” sets the specified POSIX resource limit for the container payload. Expects an assignment of the form "LIMIT=SOFT:HARD" or "LIMIT=VALUE", where LIMIT should refer to a resource limit type, such as RLIMIT_NOFILE or RLIMIT_NICE. The SOFT and HARD fields should refer to the numeric soft and hard resource limit values. If the second form is used, VALUE may specify a value that is used both as soft and hard limit. In place of a numeric value the special string "infinity" may be used to turn off resource limiting for the specific type of resource. This command line option may be used multiple times to control limits on multiple limit types. If used multiple times for the same limit type, the last use wins. For details about resource limits see setrlimit(2). By default resource limits for the container's init process (PID 1) are set to the same values the Linux kernel originally passed to the host init system. Note that some resource limits are enforced on resources counted per user, in particular RLIMIT_NPROC. This means that unless user namespacing is deployed (i.e. --private-users= is used, see above), any limits set will be applied to the resource usage of the same user on all local containers as well as the host. This means particular care needs to be taken with these limits as they might be triggered by possibly less trusted code. Example: "--rlimit=RLIMIT_NOFILE=8192:16384". Added in version 239.
  • --oom-score-adjust= โ€” changes the OOM ("Out Of Memory") score adjustment value for the container payload. This controls /proc/self/oom_score_adj which influences the preference with which this container is terminated when memory becomes scarce. For details see proc(5). Takes an integer in the range -1000...1000. Added in version 239.
  • --cpu-affinity= โ€” controls the CPU affinity of the container payload. Takes a comma separated list of CPU numbers or number ranges (the latter's start and end value separated by dashes). See sched_setaffinity(2) for details. Added in version 239.
  • --personality= โ€” control the architecture ("personality") reported by uname(2) in the container. Currently, only "x86" and "x86-64" are supported. This is useful when running a 32-bit container on a 64-bit host. If this setting is not used, the personality reported in the container is the same as the one reported on the host. Added in version 209.

๐Ÿ”Œ Integration Options ๐Ÿ”Œ

  • --resolv-conf= โ€” configures how /etc/resolv.conf inside of the container shall be handled (i.e. DNS configuration synchronization from host to container). Takes one of "off", "copy-host", "copy-static", "copy-uplink", "copy-stub", "replace-host", "replace-static", "replace-uplink", "replace-stub", "bind-host", "bind-static", "bind-uplink", "bind-stub", "delete" or "auto". If set to "off" the /etc/resolv.conf file in the container is left as it is included in the image, and neither modified nor bind mounted over. If set to "copy-host", the /etc/resolv.conf file from the host is copied into the container, unless the file exists already and is not a regular file (e.g. a symlink). Similarly, if "replace-host" is used the file is copied, replacing any existing inode, including symlinks. Similarly, if "bind-host" is used, the file is bind mounted from the host into the container.
  • If it's "copy-static", "replace-static" or "bind-static", the static resolv.conf that ships with systemd-resolved.service(8) โ€” bestie, that's literally /usr/lib/systemd/resolv.conf โ€” gets copied or bind mounted into the container. ๐Ÿ’ซ
  • If it's "copy-uplink", "replace-uplink" or "bind-uplink", it's the uplink resolv.conf managed by systemd-resolved.service (specifically: /run/systemd/resolve/resolv.conf) that gets copied or bind mounted in. ๐ŸŒˆ
  • If it's "copy-stub", "replace-stub" or "bind-stub", then it's the stub resolv.conf from systemd-resolved.service (specifically: /run/systemd/resolve/stub-resolv.conf) getting the copy-or-bind treatment. โœจ
  • If it's "delete", the /etc/resolv.conf file inside the container just gets yeeted โ€” deleted, if it exists. ๐Ÿ’€
  • And finally, if it's "auto", the file is left exactly as it is when private networking is turned on (see --private-network). Otherwise: if systemd-resolved.service is running, its stub resolv.conf file is used, and if it's not running, the host's /etc/resolv.conf file is used. In those latter cases the file gets copied if the image is writable, and bind mounted if it isn't. ๐Ÿซถ

Real talk, here's the rec: use "copy-..." or "replace-..." if you want the container to be able to make changes to the DNS configuration all on its own, deviating from the host's settings. Otherwise, "bind" is preferable, because it means direct changes to /etc/resolv.conf in the container are not allowed, since it's a read-only bind mount. (But lowkey note: if the container has enough privileges, it might simply go ahead and unmount the bind mount anyway ๐Ÿ™ˆ.) Also note that both when the file is bind mounted and when it's copied, no further propagation of configuration is generally done after the one-time early initialization โ€” and that's because the file is usually updated through copying and renaming. Defaults to "auto".

Added in version 239. ๐ŸŽ€


--timezone=

Configures how /etc/localtime inside the container (i.e. local timezone synchronization from host to container ๐Ÿ•) shall be handled. Takes one of "off", "copy", "bind", "symlink", "delete" or "auto".

  • "off" โ€” the /etc/localtime file in the container is left as it is included in the image, and neither modified nor bind mounted over. Leave it alone, it's fine. ๐Ÿ’…
  • "copy" โ€” the /etc/localtime file of the host is copied into the container. ๐Ÿ“‹
  • "bind" โ€” similarly, the file is bind mounted from the host into the container. ๐Ÿงฒ
  • "symlink" โ€” a symlink is created pointing from /etc/localtime in the container to the timezone file in the container that matches the timezone setting on the host. ๐Ÿ”—
  • "delete" โ€” the file in the container is deleted, should it exist. ๐Ÿ—‘๏ธ
  • "auto" โ€” if the /etc/localtime file of the host is a symlink, then "symlink" mode is used, and "copy" otherwise โ€” except if the image is read-only, in which case "bind" is used instead. Iconic adaptive behaviour, honestly. โœจ

Defaults to "auto".

Added in version 239. ๐Ÿฆ‹


Control whether the container's journal shall be made visible to the host system. If enabled, it allows viewing the container's journal files from the host (but not vice versa ๐Ÿ‘€). Takes one of "no", "host", "try-host", "guest", "try-guest", "auto".

  • "no" โ€” the journal is not linked. Nope, nothing to see here. ๐Ÿ™…
  • "host" โ€” the journal files are stored on the host file system (beneath /var/log/journal/machine-id) and the subdirectory is bind-mounted into the container at the same location. ๐Ÿ 
  • "guest" โ€” the journal files are stored on the guest file system (beneath /var/log/journal/machine-id) and the subdirectory is symlinked into the host at the same location. ๐Ÿ‘ป
  • "try-host" and "try-guest" โ€” same energy, but they don't fail if the host doesn't have persistent journaling enabled, or if the container is in the --ephemeral mode. Very forgiving, we stan. ๐Ÿซถ
  • "auto" (the default) โ€” if the right subdirectory of /var/log/journal exists, it will be bind mounted into the container. If the subdirectory does not exist, no linking is performed at all. So effectively: boot a container once with "guest" or "host" and the journal gets linked persistently, and if you use the default of "auto" further on, it's already linked. Big brain. ๐Ÿง โœจ
Note that --link-journal=try-guest is the default if the systemd-nspawn@.service template unit file is used. ๐ŸŽ€

Added in version 187. ๐Ÿ’œ


-j

Equivalent to --link-journal=try-guest. Short and sweet, we love a shortcut. ๐Ÿ’…

Added in version 187. โญ


--forward-journal=

Forward the container's journal to the host by starting systemd-journal-remote(8) listening on a Unix socket that is bind-mounted into the container. The container's systemd-journald(8) connects to the socket via the journal.forward_to_socket credential and streams journal entries to the host in real-time. ๐Ÿ”ฅ Takes a path to a journal file or directory where the received entries will be stored. If the path ends in .journal, entries are written to a single file; otherwise, entries are split per host into the specified directory. ๐Ÿ“‚

Added in version 261. ๐ŸŒธ


--forward-journal-max-use=BYTES, --forward-journal-keep-free=BYTES, --forward-journal-max-file-size=BYTES, --forward-journal-max-files=N

These options configure the corresponding settings of systemd-journal-remote(8) when forwarding journal entries from the container. See journal-remote.conf(5) for the descriptions of these settings. ๐Ÿ“–

Added in version 261. ๐Ÿฆ‹


๐Ÿงฒ Mount Options ๐Ÿงฒ

--bind=, --bind-ro=

Bind mount a file or directory from the host into the container. Takes one of:

  • a path argument โ€” in which case the specified path will be mounted from the host to the same path in the container, or
  • a colon-separated pair of paths โ€” in which case the first specified path is the source in the host, and the second path is the destination in the container, or
  • a colon-separated triple of source path, destination path and mount options.

The source path may optionally be prefixed with a + character. If so, the source path is taken relative to the image's root directory. This permits setting up bind mounts within the container image, which is lowkey so useful. ๐ŸŽ€ The source path may be specified as empty string, in which case a temporary directory below the host's /var/tmp/ directory is used. It is automatically removed when the container is shut down. ๐Ÿงน If the source path is not absolute, it is resolved relative to the current working directory. The --bind-ro= option creates read-only bind mounts. Backslash escapes are interpreted, so \: may be used to embed colons in either path. This option may be specified multiple times for creating multiple independent bind mount points. โœจ

Mount options are comma-separated. rbind and norbind control whether to create a recursive or a regular bind mount. Defaults to rbind. noidmap, idmap, rootidmap and owneridmap control ID mapping. ๐Ÿ†”

Using idmap, rootidmap or owneridmap requires support by the source filesystem for user/group ID mapped mounts. Defaults to noidmap. ๐Ÿซถ

With x being the container's UID range offset, y being the length of the container's UID range, and p being the owner UID of the bind mount source inode on the host:

  • If noidmap is used, any user z in the range 0 ... y seen from inside of the container is mapped to x + z in the x ... x + y range on the host. Other host users are mapped to nobody inside the container. ๐Ÿ‘ป
  • If idmap is used, any user z in the UID range 0 ... y as seen from inside the container is mapped to the same z in the same 0 ... y range on the host. Other host users are mapped to nobody inside the container. โœจ
  • If rootidmap is used, the user 0 seen from inside of the container is mapped to p on the host. Other host users are mapped to nobody inside the container. ๐Ÿ‘‘
  • If owneridmap is used, the owner of the target directory inside of the container is mapped to p on the host. Other host users are mapped to nobody inside the container. ๐ŸŽ€

Whichever ID mapping option is used, the same mapping will be used for users and groups IDs. If rootidmap or owneridmap are used, the group owning the bind mounted directory will have no effect. ๐Ÿ’…

Heads up bestie: when this option is used in combination with --private-users, the resulting mount points will be owned by the nobody user. That's because the mount and its files and directories continue to be owned by the relevant host users and groups, which do not exist in the container, and thus show up under the wildcard UID 65534 (nobody). ๐Ÿ˜ญ If such bind mounts are created, it is recommended to make them read-only, using --bind-ro=. Alternatively you can use the "idmap" mount option to map the filesystem IDs. ๐Ÿ› ๏ธ

Added in version 198. ๐Ÿ’œ


--bind-user=

Binds the home directory of the specified user on the host into the container. Takes the name of an existing user on the host as argument. May be used multiple times to bind multiple users into the container. This does two things, and honestly? Both are kind of iconic:

  1. The user's home directory is bind mounted from the host into /run/host/home/, using an idmapped mount to map the host user's UID/GID to its assigned UID/GID in the container. ๐Ÿ โœจ
  2. A JSON user and group record is generated in /run/userdb/ that describes the mapped user. It contains a minimized representation of the host's user record, adjusted to the UID/GID and home directory path assigned to the user in the container. The nss-systemd(8) glibc NSS module will pick up these records from there and make them available in the container's user/group databases. ๐Ÿ“‡

The combination of the two operations above ensures that it is possible to log into the container using the same account information as on the host. ๐Ÿ”‘ The user is only mapped transiently, while the container is running, and the mapping itself does not result in persistent changes to the container (except maybe for log messages generated at login time, and similar). Note that in particular the UID/GID assignment in the container is not made persistently. โš ๏ธ

If the user is mapped transiently, it is best to not allow the user to make persistent changes to the container. If the user leaves files or directories owned by the user, and those UIDs/GIDs are reused during later container invocations (possibly with a different --bind-user= mapping), those files and directories will be accessible to the "new" user. ๐Ÿ˜ฌ So like, be careful with that one, fr fr.

The user/group record mapping only works if the container is running systemd 249 or newer, with nss-systemd set up properly in nsswitch.conf. Peep nss-systemd(8) for the deets ๐Ÿ’–

Okay so heads up, the user record that gets carried from the host into the container contains the UNIX password hash of the user โ€” that's what makes seamless logins inside the container possible. Which is slay, but also: if the container is less trusted than the host, it is hence important to use a strong UNIX password hash function (like yescrypt or similar, with the "$y$" hash prefix). Protect urself bestie ๐Ÿ”

When you bind a user from the host into the container, checks run to make sure the username isn't already known inside the container. On top of that, it checks that the UID/GID handed to it isn't currently defined in the container's user/group databases. Both of these checks go straight at the container's /etc/passwd and /etc/group, so fair warning โ€” they might not catch existing accounts living in other databases ๐Ÿ™ˆ

Added in version 249.

--bind-user-shell=

When you use it with --bind-user=, it includes the specified shell in the user records of users bound into the container. Takes either a boolean or an absolute path.

  • If false (which is the default), no shell is passed in the user records for users bound into the container. Which means bound users use the container's default shell.
  • If true, the shells specified by the host user records are included in the user records of all users bound into the container.
  • If you pass an absolute path, that path is set as the shell for user records of all users bound into the container.
Not me โš ๏ธ โ€” this will not check whether the specified shells actually exist in the container.

This operation is only supported in combination with --bind-user=.

Added in version 258.

*--bind-user-group=NAME*

When you use it with --bind-user=, it includes the specified group as an auxiliary group in the user records of users bound into the container. Takes a group name.

Again โš ๏ธ โ€” this will not check whether the specified groups exist in the container.

This operation is only supported in combination with --bind-user=.

Added in version 259.

--inaccessible=

Makes the specified path inaccessible in the container. What it does is over-mount the specified path (which, ps, must exist in the container) with a file node of the same type that is empty and has the most restrictive access mode supported. It's basically the icon move for masking files, directories and other file system objects from the container payload. You can use this option more than once in case all the specified paths are masked ๐Ÿ’…

Added in version 242.

--tmpfs=

Mounts a tmpfs file system into the container. Takes a single absolute path argument that says where to mount the tmpfs instance to โ€” and in that case the directory access mode gets chosen as 0755, owned by root/root โ€” or, optionally, a colon-separated pair of path and mount option string that's used for mounting, in which case the kernel default for access mode and owner gets chosen, unless you specify otherwise. Backslash escapes are interpreted in the path, so "\:" can be used to embed colons in the path ๐Ÿซถ

Heads up ๐Ÿ’€ โ€” this option cannot be used to replace the root file system of the container with a temporary file system. The --volatile= option described below provides similar functionality though, with a focus on implementing stateless operating system images. So like, not the same thing, bestie.

Added in version 214.

--overlay=, --overlay-ro=

Combines multiple directory trees into one overlay file system and mounts it into the container. Takes a list of colon-separated paths to the directory trees to combine and the destination mount point.

Backslash escapes are interpreted in the paths, so "\:" can be used to embed colons in the paths.

If you specify three or more paths, then the last specified path is the destination mount point in the container, and all the paths specified before it refer to directory trees on the host and get combined in the specified order into one overlay file system. So the left-most path is the lowest directory tree, and the second-to-last path is the highest directory tree in the stacking order. If you use --overlay-ro= instead of --overlay=, you get a read-only overlay file system. And if you create a writable overlay file system, every change made to it is written to the highest directory tree in the stacking order โ€” i.e. the second-to-last specified ๐Ÿ”ฅ

If you specify only two paths, then the second specified path is used both as the top-level directory tree in the stacking order as seen from the host, and as the mount point for the overlay file system in the container. At least two paths have to be specified โ€” no cap, that's the floor.

The source paths may optionally be prefixed with a "+" character. If so, they're taken relative to the image's root directory. The uppermost source path may also be specified as an empty string, in which case a temporary directory below the host's /var/tmp/ is used. That directory gets removed automatically when the container shuts down. This behaviour is useful in order to make read-only container directories writable while the container is running. For example, use --overlay=+/var::/var in order to automatically overlay a writable temporary directory on a read-only /var/ directory. If a source path isn't absolute, it's resolved relative to the current working directory โœจ

For details about overlay file systems, see Overlay Filesystem[5]. Also please note: the semantics of overlay file systems are substantially different from normal file systems, in particular regarding reported device and inode information. Device and inode information may change for a file while it's being written to, and processes might see out-of-date versions of files at times ๐Ÿ˜ญ Note that this switch automatically derives the "workdir=" mount option for the overlay file system from the top-level directory tree, making it a sibling of it. It is hence essential that the top-level directory tree is not a mount point itself, since the working directory must be on the same file system as the top-most directory tree. Also note that the "lowerdir=" mount option receives the paths to stack in the opposite order of this switch.

And the same warning as above ๐Ÿ’€ โ€” this option cannot be used to replace the root file system of the container with an overlay file system. The --volatile= option described above provides similar functionality, with a focus on implementing stateless operating system images.

Added in version 220.


โœจ Input/Output Options โœจ

*--console=MODE*

Configures how to set up standard input, output and error output for the container payload, plus the /dev/console device for the container. Takes one of interactive, read-only, passive, pipe or autopipe.

  • interactive โ€” a pseudo-TTY is allocated and made available as /dev/console in the container. It's then bi-directionally connected to the standard input and output passed to systemd-nspawn.
  • read-only โ€” similar, but only the output of the container is propagated and no input from the caller is read.
  • passive โ€” a pseudo TTY is allocated, but it's not connected anywhere.
  • pipe โ€” no pseudo TTY is allocated, but the standard input, output and error output file descriptors passed to systemd-nspawn are passed on โ€” as they are โ€” to the container payload, see the following paragraph.
  • autopipe โ€” operates like interactive when systemd-nspawn is invoked on a terminal, and like pipe otherwise.

Defaults to interactive if systemd-nspawn is invoked from a terminal, and read-only otherwise.

In pipe mode, /dev/console will not exist in the container. Which means the container payload generally cannot be a full init system, since init systems tend to require /dev/console to be available. But on the flip side, in this mode container invocations can be used within shell pipelines ๐ŸŽ€ Why? Because intermediary pseudo TTYs do not permit independent bidirectional propagation of the end-of-file (EOF) condition, which is necessary for shell pipelines to work correctly.

Okay genuine warning time ๐Ÿšจ โ€” pipe mode should be used carefully, since passing arbitrary file descriptors to less trusted container payloads might open up unwanted interfaces for access by the container payload. For example, if a passed file descriptor refers to a TTY of some form, APIs such as TIOCSTI may be used to synthesize input that might be used for escaping the container ๐Ÿ’€ Hence pipe mode should only be used if the payload is sufficiently trusted, or when the standard input/output/error output file descriptors are known safe โ€” for example pipes.

Added in version 242.

--pipe, -P

Equivalent to --console=pipe. That's it, that's the whole option ๐Ÿ’…

Added in version 242.

*--background=COLOR*

Changes the terminal background color to the specified ANSI color for as long as the container runs. The color you specify should be an ANSI X3.64 SGR background color, i.e. strings such as "40", "41", โ€ฆ, "47", "48;2;...", "48;5;...". See ANSI Escape Code (Wikipedia)[6] for details. Assign an empty string to disable any coloring ๐ŸŽจ

Added in version 256.


๐ŸŒธ Credentials ๐ŸŒธ

*--load-credential=ID*:PATH, *--set-credential=ID*:VALUE

Passes a credential to the container. These two options correspond to the LoadCredential= and SetCredential= settings in unit files. See systemd.exec(5) for details about these concepts, along with the syntax of the option's arguments.

Note: when systemd-nspawn runs as a systemd system service, it can propagate the credentials it received via LoadCredential=/SetCredential= to the container payload. A systemd service manager running as PID 1 in the container can further propagate them to the services it itself starts. It's thus possible to easily propagate credentials from a parent service manager to a container manager service, and from there into its payload. This can even be done recursively ๐Ÿ‘‘ iykyk

In order to embed binary data into the credential data for --set-credential=, use C-style escaping (i.e. "\n" to embed a newline, or "\x00" to embed a NUL byte). And note that the invoking shell might already apply unescaping once, hence this might require double escaping! ๐Ÿ˜ญ

so like, systemd-sysusers.service(8) and systemd-firstboot(1) are the ones that actually read the credentials you set up this way ๐Ÿ’– โ€” they use them to set the container's root password and root shell, plus the system locale, keymap and timezone, all during that container's first boot. This is lowkey iconic when you pair it with --volatile=yes, because there literally every single boot looks like a first boot (since everything you write to /etc/ gets yeeted on each container reboot cycle ๐Ÿ˜ญ). Peep the man pages for the deets. Example time:

# systemd-nspawn -i image.raw \
          --volatile=yes \
          --set-credential=firstboot.locale:de_DE.UTF-8 \
          --set-credential=passwd.hashed-password.root:'$y$j9T$yAuRJu1o5HioZAGDYPU5d.$F64ni6J2y2nNQve90M/p0ZP0ECP/qqzipNyaY9fjGpC' \
          -b

So that command right there boots the image file image.raw in volatile mode โ€” aka with an empty /etc/ and /var/ โœจ. The container payload goes "ohhh this is a first boot" and fires up systemd-firstboot.service, which then reads those two credentials you passed and uses them to configure the system's initial locale and the root password. Chef's kiss. ๐Ÿ‘‘

Added in version 247.


๐ŸŒˆ Other ๐ŸŒˆ

  • --system, --user โ€” picks whether you're running in system scope or user scope ๐Ÿซถ. This bad boy controls which service manager and which machined instance you talk to, plus the operational defaults like user namespace mode and private networking. If you don't pick either one, --system is implied when you're running as root, and --user otherwise. No cap.
โš ๏ธ Real talk, bestie: for backwards compatibility, --user followed by a positional argument gets interpreted as the deprecated --user=NAME form (which is now --uid=). So if you actually wanna use --user for scope selection and you've got positional arguments coming after it, separate them with --. Don't get caught out ๐Ÿ’€

Added in version 261.

  • --no-pager โ€” please do NOT pipe the output into a pager, thank you very much ๐Ÿ™ˆ.
  • -h, --help โ€” prints a short help text and dips โœŒ๏ธ.
  • --version โ€” prints a short version string and dips โœŒ๏ธ.
  • --no-ask-password โ€” don't bug the user for authentication for privileged operations ๐ŸŽ€.

โŒจ๏ธ HOTKEYS โŒจ๏ธ

okay so when you're in interactive mode (i.e. the default --console=interactive), there are a few special keyboard shortcuts that control the container runtime ๐Ÿ‘‘. The catch: you gotta type these shortcuts within 1s or they won't do anything โ€” instead they just get forwarded to the container as regular ol' keypresses ๐Ÿ˜ญ. Speedrun vibes fr fr.

  • Ctrl-] Ctrl-] Ctrl-] โ€” immediately terminates the container, killing all processes ๐Ÿ’€. Three taps, no survivors.
  • Ctrl-] Ctrl-] r โ€” issues a reboot request to the container ๐Ÿ”„.

Added in version 258.

  • Ctrl-] Ctrl-] p โ€” issues a shutdown request to the container ๐ŸŒ™.

Added in version 258.

๐ŸŒธ ENVIRONMENT ๐ŸŒธ

okay bestie, this is the part where we talk about the environment variables that systemd-nspawn and friends actually pay attention to ๐Ÿ‘€ these are all inherited from your shell, so if something's acting weird, check here first. no cap.


๐Ÿ’œ $SYSTEMD_LOG_LEVEL

this one sets the maximum log level of emitted messages โ€” anything with a higher (i.e. less important) level gets suppressed, like, bye ๐Ÿ’… it takes a comma-separated list of values, and each value can be either one of these (in order of decreasing importance): emerg, alert, crit, err, warning, notice, info, debug, or an integer in the range 0...7. see syslog(3) if you want the whole lore.

each value may optionally be prefixed with one of console, syslog, kmsg or journal followed by a colon, to set the max log level for that specific log target only. so like SYSTEMD_LOG_LEVEL=debug,console:info means "log at debug level everywhere except the console, which should be info level" โ€” iconic, honestly.

note that the global maximum log level takes priority over any per-target maximum log levels. so don't get cute and think your per-target setting wins. it doesn't. ๐Ÿ™ˆ

๐Ÿ’œ $SYSTEMD_LOG_COLOR

a boolean, bestie. if true, messages written to the tty get colored according to priority ๐ŸŒˆ

this setting is only useful when messages are written directly to the terminal, because journalctl(1) and other tools that display logs will color messages based on the log level all on their own. they don't need your help. โœจ

๐Ÿ’œ $SYSTEMD_LOG_TIME

a boolean. if true, console log messages get prefixed with a timestamp โฐ

again, only useful when messages are written directly to the terminal or a file, because journalctl(1) and friends attach timestamps based on the entry metadata on their own. they got this. ๐Ÿซถ

๐Ÿ’œ $SYSTEMD_LOG_LOCATION

a boolean. if true, messages get prefixed with a filename and line number in the source code where the message originates ๐Ÿ”

note that the log location is often attached as metadata to journal entries anyway, so likeโ€ฆ it's a bit redundant? but including it directly in the message text can nevertheless be convenient when debugging programs. key word: debugging. yw ๐Ÿ’…

๐Ÿ’œ $SYSTEMD_LOG_TID

a boolean. if true, messages get prefixed with the current numerical thread ID (TID) ๐Ÿงต

note that this information is attached as metadata to journal entries anyway. including it directly in the message text can nevertheless be convenient when debugging programs. same energy as the last one, fr fr.

๐Ÿ’œ $SYSTEMD_LOG_TARGET

the destination for log messages. pick one:

  • console โ€” log to the attached tty
  • console-prefixed โ€” log to the attached tty but with prefixes encoding the log level and "facility", see syslog(3)
  • kmsg โ€” log to the kernel circular log buffer
  • journal โ€” log to the journal
  • journal-or-kmsg โ€” log to the journal if available, and to kmsg otherwise
  • auto โ€” determine the appropriate log target automatically, the default
  • null โ€” disable log output

๐Ÿ’œ $SYSTEMD_LOG_RATELIMIT_KMSG

whether to ratelimit kmsg or not. takes a boolean. defaults to "true". if disabled, systemd will not ratelimit messages written to kmsg. so like, you can turn the firehose on if you really want ๐Ÿ”ฅ

๐Ÿ’œ $SYSTEMD_PAGER, $PAGER

the pager to use when --no-pager is not given. $SYSTEMD_PAGER is used if set; otherwise $PAGER is used. if neither $SYSTEMD_PAGER nor $PAGER are set, a set of well-known pager implementations is tried in turn, including less(1) and more(1), until one is found. if no pager implementation is discovered, no pager is invoked. periodt.

setting those environment variables to an empty string or the value "cat" is equivalent to passing --no-pager. cute, right? ๐ŸŽ€

note: if $SYSTEMD_PAGERSECURE is not set, $SYSTEMD_PAGER and $PAGER can only be used to disable the pager (with "cat" or ""), and are otherwise ignored. so don't think you can sneak a custom pager in there without setting the secure one too. iykyk.

๐Ÿ’œ $SYSTEMD_LESS

override the options passed to less (by default "FRSXMK").

users might want to change two options in particular, and honestly? valid:

๐Ÿ”ฅ K

this option instructs the pager to exit immediately when Ctrl+C is pressed. to allow less to handle Ctrl+C itself to switch back to the pager command prompt, unset this option.

if the value of $SYSTEMD_LESS does not include "K", and the pager that is invoked is less, Ctrl+C will be ignored by the executable, and needs to be handled by the pager. so likeโ€ฆ know what you're signing up for ๐Ÿ’€

๐Ÿ”ฅ X

this option instructs the pager to not send termcap initialization and deinitialization strings to the terminal. it is set by default to allow command output to remain visible in the terminal even after the pager exits. slay, output stays visible.

nevertheless, this prevents some pager functionality from working โ€” in particular, paged output cannot be scrolled with the mouse. ๐Ÿ˜ญ tradeoffs, bestie.

note that setting the regular $LESS environment variable has no effect for less invocations by systemd tools. they do their own thing.

see less(1) for more discussion ๐Ÿซถ

๐Ÿ’œ $SYSTEMD_LESSCHARSET

override the charset passed to less (by default "utf-8", if the invoking terminal is determined to be UTF-8 compatible) ๐ŸŒ

note that setting the regular $LESSCHARSET environment variable has no effect for less invocations by systemd tools. same deal as above.

๐Ÿ’œ $SYSTEMD_PAGERSECURE

okay this one is the important one, so buckle up ๐Ÿšจ common pager commands like less(1), in addition to "paging" (i.e. scrolling through the output), support opening of or writing to other files and running arbitrary shell commands. when commands are invoked with elevated privileges โ€” for example under sudo(8) or pkexec(1) โ€” the pager becomes a security boundary. care must be taken that only programs with strictly limited functionality are used as pagers, and unintended interactive features like opening or creation of new files or starting of subprocesses are not allowed. "Secure mode" for the pager may be enabled as described below, if the pager supports that (most pagers are not written in a way that takes this into consideration). it is recommended to either explicitly enable "secure mode" or to completely disable the pager using --no-pager or PAGER=cat when allowing untrusted users to execute commands with elevated privileges. like, genuinely. don't skip this. ๐Ÿ™

this option takes a boolean argument. when set to true, the "secure mode" of the pager is enabled. in "secure mode", LESSSECURE=1 will be set when invoking the pager, which instructs the pager to disable commands that open or create new files or start new subprocesses. currently only less(1) is known to understand this variable and implement "secure mode". ๐Ÿ‘‘

when set to false, no limitation is placed on the pager. setting SYSTEMD_PAGERSECURE=0 or not removing it from the inherited environment may allow the user to invoke arbitrary commands. that's the scary path, bestie. ๐Ÿ’€

when $SYSTEMD_PAGERSECURE is not set, systemd tools attempt to automatically figure out if "secure mode" should be enabled and whether the pager supports it. "Secure mode" is enabled if the effective UID is not the same as the owner of the login session, see geteuid(2) and sd_pid_get_owner_uid(3), or when running under sudo(8) or similar tools ($SUDO_UID is set [7]). in those cases, SYSTEMD_PAGERSECURE=1 will be set and pagers which are not known to implement "secure mode" will not be used at all. no exceptions ๐Ÿ’…

note that this autodetection only covers the most common mechanisms to elevate privileges and is intended as convenience. it is recommended to explicitly set $SYSTEMD_PAGERSECURE or disable the pager. fr fr, don't rely on the vibes.
note that if the $SYSTEMD_PAGER or $PAGER variables are to be honoured โ€” other than to disable the pager โ€” $SYSTEMD_PAGERSECURE must be set too. they're a package deal ๐ŸŽ€

๐Ÿ’œ $SYSTEMD_COLORS

takes a boolean argument, or a special value. by default (unset), systemd and related utilities will use colors in their output if possible ๐ŸŒˆ if $COLORTERM is set to "truecolor" or "24bit", 24-bit colors will be enabled, 256 colors otherwise, unless $NO_COLOR or $TERM indicates colors are disabled. got it? good.

๐ŸŒธ true

same as unset, except that $NO_COLOR is ignored. so you're overriding the override ๐Ÿ‘€

๐ŸŒธ false

the output will be monochrome. boring but valid ๐Ÿ–ค

๐ŸŒธ "16", "256", "24bit"

always use the base 16 ANSI colors, 256 colors, or 24 bit color, respectively. no debate. ๐Ÿ’…

๐ŸŒธ "auto-16", "auto-256", "auto-24bit"

use the given quantity of colours, subject to $TERM, and what the console is connected to. a little more chill about it โœจ

๐Ÿ’œ $SYSTEMD_URLIFY

the value must be a boolean. controls whether clickable links should be generated in the output for terminal emulators supporting this ๐Ÿ”— this can be specified to override the decision that systemd makes based on $TERM and other conditions. so if you want the links (or don't), here's your moment ๐Ÿซถ

โœจ EXAMPLES โœจ

Okay bestie, this is the fun part โ€” real little recipes you can steal. ๐Ÿ’…

๐ŸŽ€ Example 1. Download an Ubuntu TAR image and open a shell in it

# importctl pull-tar -mN https://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64-root.tar.xz
# systemd-nspawn -M jammy-server-cloudimg-amd64-root

This downloads and verifies the specified .tar image, then uses systemd-nspawn(1) to open a shell in it. Literally grab-and-go. ๐Ÿซถ

๐ŸŒธ Example 2. Build and boot a minimal Fedora distribution in a container

# dnf -y --releasever=44 --installroot=/var/lib/machines/f44 \
      --use-host-config --setopt=install_weak_deps=0 \
      --repo=fedora --repo=updates install \
      passwd dnf fedora-release nano util-linux systemd systemd-networkd
# systemd-nspawn -bD /var/lib/machines/f44

(Omit --use-host-config when using dnf <= 4.) This installs a minimal Fedora distribution into the directory /var/lib/machines/f44 and then boots that OS in a namespace container.

And because the installation is located underneath the standard /var/lib/machines/ directory, you can also start the machine with systemd-nspawn -M f44. Convenient? Obsessed. โœจ

๐Ÿฆ‹ Example 3. Spawn a shell in a container of a minimal Debian unstable distribution

# debootstrap unstable ~/debian-tree/
# systemd-nspawn -D ~/debian-tree/

This installs a minimal Debian unstable distribution into the directory ~/debian-tree/ and then spawns a shell from this image in a namespace container.

debootstrap supports Debian[8], and Ubuntu[9] out of the box, so the same command can be used to install any of those. For other distributions from the Debian family, a mirror has to be specified, see debootstrap(8).

๐ŸŒˆ Example 4. Boot a minimal Arch Linux distribution in a container

# pacstrap -c ~/arch-tree/ base
# systemd-nspawn -bD ~/arch-tree/

This installs a minimal Arch Linux distribution into the directory ~/arch-tree/ and then boots an OS in a namespace container in it. Main character distro behavior, no cap. ๐Ÿ’€

๐Ÿ“ Example 5. Install the OpenSUSE Tumbleweed rolling distribution

# zypper --root=/var/lib/machines/tumbleweed ar -c \
      https://download.opensuse.org/tumbleweed/repo/oss tumbleweed
# zypper --root=/var/lib/machines/tumbleweed refresh
# zypper --root=/var/lib/machines/tumbleweed install --no-recommends \
      systemd shadow zypper openSUSE-release vim
# systemd-nspawn -M tumbleweed passwd root
# systemd-nspawn -M tumbleweed -b

๐Ÿงš Example 6. Boot into an ephemeral snapshot of the host system

# systemd-nspawn -D / -xb

This runs a copy of the host system in a snapshot which is removed immediately when the container exits. All file system changes made during runtime will be lost on shutdown, hence. So yes, it's giving temporary, gorgeous, absolutely disposable. ๐Ÿ™ˆ

๐Ÿ‘‘ Example 7. Run a container with SELinux sandbox security contexts

# chcon system_u:object_r:svirt_sandbox_file_t:s0:c0,c1 -R /srv/container
# systemd-nspawn -L system_u:object_r:svirt_sandbox_file_t:s0:c0,c1 \
      -Z system_u:system_r:svirt_lxc_net_t:s0:c0,c1 -D /srv/container /bin/sh

๐Ÿ’œ Example 8. Run a container with an OSTree deployment

# systemd-nspawn -b -i ~/image.raw \
      --pivot-root=/ostree/deploy/$OS/deploy/$CHECKSUM:/sysroot \
      --bind=+/sysroot/ostree/deploy/$OS/var:/var

๐Ÿ’– EXIT STATUS ๐Ÿ’–

The exit code of the program executed in the container is returned. That's it โ€” whatever the program inside says, that's what you get back. Simple, iconic, periodt.

๐ŸŒŸ SEE ALSO ๐ŸŒŸ

๐ŸŽ€ NOTES ๐ŸŽ€

  1. Container Interface โ€” https://systemd.io/CONTAINER_INTERFACE
  2. UAPI.2 Discoverable Partitions Specification โ€” https://uapi-group.org/specifications/specs/discoverable_partitions_specification
  3. OCI Runtime Specification โ€” https://github.com/opencontainers/runtime-spec/blob/master/spec.md
  4. File Descriptor Store โ€” https://systemd.io/FILE_DESCRIPTOR_STORE
  5. Overlay Filesystem โ€” https://docs.kernel.org/filesystems/overlayfs.html
  6. ANSI Escape Code (Wikipedia) โ€” https://en.wikipedia.org/wiki/ANSI_escape_code#SGR_(Select_Graphic_Rendition)_parameters
  7. It is recommended for other tools to set and check $SUDO_UID as appropriate, treating it is a common interface.
  8. Debian โ€” https://www.debian.org
  9. Ubuntu โ€” https://www.ubuntu.com
  10. Arch Linux โ€” https://www.archlinux.org
  11. OpenSUSE Tumbleweed โ€” https://software.opensuse.org/distributions/tumbleweed

Footer vibe check: this is systemd 261.3, page SYSTEMD-NSPAWN(1) โ€” no date on the footer, bestie โ€” and that's a wrap! ๐Ÿ’–๐ŸŽ€โœจ

๐Ÿ“ฆ where this page comes from