BOOTC-FILESYSTEM(7)

🎀 NAME 🎀

bootc-filesystem — the Filesystem chapter, and yes it's pronounced "the filesystem chapter" 💾✨

So you opened the bootc-filesystem(7) page. Welcome bestie, sit down, we're talking about where your bytes live.

---

🌈 DESCRIPTION 🌈

Okay so, like the other chapters already gossiped about: right now the bootc project depends on the ostree project (https://github.com/ostreedev/ostree/) to store the base container image. And ALSO there's a containers/storage instance (https://github.com/containers/storage) hanging out to handle logically bound images (go read bootc-logically-bound-images(7) for that tea 🫖).

BUT. The whole point is that bootc is trying to be a "fresh, new container-native interface", and ostree is just an implementation detail. Like, backstage. Don't @ it.

First thing, and this one is a strong rec, not a cute little suggestion: if you're a bootc consumer, use the ostree composefs backend (https://ostreedev.github.io/ostree/composefs/) 💅 To do that, make sure you've got a /usr/lib/ostree/prepare-root.conf that contains at least:

[composefs]
enabled = true

This is the move that makes your entire / a read-only filesystem, which is so important for achieving correct semantics. No cap. 🔥

---

🧠 Understanding container build/runtime vs deployment 🧠

When you're running as a container (like, say, as part of a container build), the filesystem is fully mutable — deliberately! — so that derivation works. Wanna know more about container builds? Peep the build guidance in bootc-building-images(7).

Everything else in this document is describing the state of the system when it's "deployed" onto a physical or virtual machine and being managed by bootc. Different mode, different rules, iykyk.

---

⏰ Timestamps ⏰

bootc uses ostree, and ostree currently squashes all timestamps to zero 😭 (https://ostreedev.github.io/ostree/repo/#content-objects). This is now viewed as an implementation bug, and it's gonna get changed in the future, fr fr. Want the receipts? There's a tracker issue: https://github.com/bootc-dev/bootc/issues/20.

---

🗺️ Understanding physical vs logical root with /sysroot 🗺️

When the system is fully booted, you're booted into the equivalent of a chroot. The "physical" host root filesystem gets mounted at /sysroot. For more, go look at filesystem: sysroot in bootc-sysroot(7).

This chroot filesystem is called a "deployment root" 💖 And every remaining filesystem path below in this doc is part of a deployment root, which is the final target for the system boot. Which deployment is the target? That's decided by the ostree= kernel commandline argument. Iconic behaviour honestly.

---

🌸 /usr 🌸

The overall recommendation: keep all operating system content in /usr, with directories like /bin being symbolic links to /usr/bin, etc. See UsrMove (https://fedoraproject.org/wiki/Features/UsrMove) for an example.

HOWEVER — and this is the plot twist — with composefs enabled, /usr is not different from /. They're literally part of the same immutable image 🫶 So there's no fundamental need to do a full "UsrMove" on a bootc system. Chill.

✨ /usr/local ✨

The OSTree upstream recommendation says: make /usr/local a symbolic link to /var/usrlocal. But here's the thing — the whole emphasis of a bootc-oriented system is on users deriving custom container images as the default entrypoint. So the recommendation here is that base images configure /usr/local to be a regular directory (i.e. the default). Normal directory. Nothing fancy. 💅

Projects that want to produce "final" images — ones that aren't themselves intended to be derived from, generally — can enable that symbolic link in derived builds. Their call, their vibe.

---

🎀 /etc 🎀

The /etc directory contains mutable persistent state by default. That said, it's supported (and encouraged) to enable the ostree-prepare-root (https://ostreedev.github.io/ostree/man/ostree-prepare-root.html), see below as well.

When you're in persistent mode, /etc inherits the OSTree semantics of performing a 3-way merge (https://ostreedev.github.io/ostree/atomic-upgrades/#assembling-a-new-deployment-directory) across upgrades. In a nutshell:

• The new default /etc is used as a base
• The diff between current and previous /etc is applied to the new /etc
• Locally modified files in /etc different from the default /usr/etc (of the same deployment) will be retained

You can view the state via ostree admin config-diff 👀 Heads up though: the "diff" here includes metadata (uid, gid, extended attributes), so changing any of those will also mean that updated files from the image are not applied. Careful.

The implementation of this defaults to being executed by ostree-finalize-staged.service at shutdown time, before the new bootloader entry is created.

Why design it this way? Because in practice today, so many components of a Linux system end up shipping default configuration files in /etc. And even if the default package doesn't, the software often only looks for config files there by default anyway. So here we are. 🙈

Some other image-based update systems don't have distinct "versions" of /etc — it may get populated only at install time and left untouched forever after. But that creates "hysteresis" where the state of the system's /etc is strongly influenced by the initial image version. Which leads to problems like: a change to /etc/sudoers (just to give one simple example) would require external intervention to apply. Not slay. 😭

For more on configuration file best practices, see Building in bootc-building-images(7).

To emphasize again (because it matters): it's recommended to enable etc.transient if possible 💜 Though when you use that, you may need to store some machine-specific state in e.g. the kernel commandline, if applicable.

🧚 /usr/etc 🧚

The /usr/etc tree is generated client side, and it contains the default container image's view of /etc. This should generally be considered an internal implementation detail of bootc/ostree.

Do not explicitly put files into this location. It can create undefined behavior. 🚨 There's a check for this in bootc container lint.

🦋 systemd-confext and systemd-sysext 🦋

systemd-confext (https://www.freedesktop.org/software/systemd/man/systemd-confext.html) and systemd-sysext (https://www.freedesktop.org/software/systemd/man/systemd-sysext.html) are not currently supported on bootc-managed systems.

These tools may be present in the operating system image (for example, because they ship in the systemd package), but bootc has not validated them on deployed systems. Using them on a bootc host is not recommended. There are two categories of concern to be aware of 👇

Conflicts with transient overlays

confext and sysext apply their content by stacking overlayfs mounts. This can conflict with the non-default use of transient overlays for / and /etc. bootc systems commonly use composefs for the root filesystem, and may also enable transient root and/or transient /etc, each of which also uses overlayfs stacking. The Linux kernel limits filesystem stacking depth (FILESYSTEM_MAX_STACK_DEPTH = 2), so combining these features can exhaust the available depth and cause confext or sysext overlays to fail. 💀

The /etc case in particular is expected to become tractable: configuring the composefs backend with etc = none (there is not yet a corresponding ostree knob) leaves /etc free for confext to manage. Progress! 🌱

OS-specific integration concerns

Beyond the bootc-specific overlay concerns above, there may be additional integration issues (for example, SELinux labeling) that are specific to a particular OS or distribution. Consult your OS/distribution documentation for guidance on confext and sysext 📚

For per-host configuration, use the patterns described in Building images: Configuration (bootc-building-images(7)) instead:

• image-embedded configuration (prefer /usr where possible)
• persistent /etc with day-2 configuration management tools
• or machine-local kernel arguments via rpm-ostree kargs or /usr/lib/bootc/kargs.d

For more on the design rationale, see Relationship with systemd "particles" (bootc-systemd-particles(7)).

---

🍓 /var 🍓

Content in /var persists by default 💾 It is however supported to make it or its subdirectories mount points (whether network or tmpfs). There is exactly one /var. If it's not a distinct partition, then it's automatically made a bind from /ostree/deploy/$stateroot/var and shared across "deployments" (bootloader entries). One /var to rule them all, bestie 👑

You may include content in /var in your image — and reference base images may have a few basic directories such as /var/tmp (in order to ease use in container builds).

BUT. It is very important to understand that content included in /var in the container image acts like a Docker VOLUME /var. This means its contents are unpacked only from the initial image — subsequent changes to /var in a container image are not automatically applied. 😱

A common case is applications wanting some directory structure (e.g. /var/lib/postgresql) to be pre-created. The recommendation? Use systemd tmpfiles.d (https://www.freedesktop.org/software/systemd/man/latest/tmpfiles.d.html) for this. An even better approach where applicable is StateDirectory= (https://www.freedesktop.org/software/systemd/man/latest/systemd.exec.html#RuntimeDirectory=) in units. Chef's kiss 👨‍🍳💋

As of bootc 1.1.6, the bootc container lint command will check for missing tmpfiles.d entries and warn. Free vibe check, take it.

Note this is very different from the handling of /etc. The rationale: /etc is relatively small configuration files, and the expected configuration files are often bound to the operating system binaries in /usr.

But /var has arbitrarily large data (system logs, databases, etc.). It would also not be expected to be rolled back if the operating system state is rolled back. A simple example: an apt|dnf downgrade of postgresql should not affect the physical database in general in /var/lib/postgres. Similarly, a bootc update or rollback should not affect this application data. Keep the code and the data in separate lanes, always 🛣️

Having /var separate also makes it work cleanly to "stage" new operating system updates before applying them (they're downloaded and ready, but only take effect on reboot). So satisfying.

In general, this is the same rationale for Docker VOLUME: decouple the application code from its data.

💖 Other directories 💖

Okay so real talk — you cannot ship content in /run or /proc or any other API Filesystems (that's a whole systemd concept, link right here: https://www.freedesktop.org/wiki/Software/systemd/APIFileSystems/) inside your container images. Just don't. It's not supported, bestie. 🚫

Besides those, for other toplevel directories like /usr and /opt, they get lifecycled along with the container image. That's just how it is. ✨

🍓 /opt

In the default suggested model of using composefs (as described up above 💅), /opt is read-only, sitting right alongside the other toplevels like /usr.

Problem tho: some software — especially those "3rd party" deb/rpm packages 👀 — expects to be able to write to a subdirectory of /opt, like /opt/examplepkg.

So go check out building images (see bootc-building-images(7)) for the recommendations on how to build container images and adjust the filesystem for situations like this. 🛠️

But for some use cases, it may lowkey just be easier to allow some level of mutability. There are two options for that, and each one comes with its own separate trade-offs: transient roots and state overlays. Pick your fighter. 💪

🌈 Other toplevel directories

Creating other toplevel directories and content (like /afs, /arbitrarymountpoint) — or in general, further nested data — is supported! Just create the directory as part of your container image build process, e.g. RUN mkdir /arbitrarymountpoint. These directories will be lifecycled with the container image state, and they appear immutable by default, exactly the same as all the other directories like /usr and /opt. 🎀

Wanna mount separate filesystems in there? Totally doable with the usual mechanisms: /etc/fstab, systemd .mount units, etc.

🧚 SELinux for arbitrary toplevels

Heads up: operating systems using SELinux may use a label such as default_t for unknown toplevel directories, and some processes might not be able to access that. 😭 In this situation you currently may need to also ensure a label is defined for them in the file contexts.

---

⭐ Enabling transient root ⭐

This feature enables a fully transient writable rootfs by default. Iconic. To do this, set the

[root]
transient = true

option in /usr/lib/ostree/prepare-root.conf.

In particular, this will let software write (transiently, i.e. until the next reboot 🔁) to all top-level directories, including /usr and /opt, with symlinks to /var for content that should persist.

This can be combined with etc.transient as well (below 👇).

More on prepare-root: https://ostreedev.github.io/ostree/man/ostree-prepare-root.html

Note that regenerating the initramfs is required when changing this file. Don't skip that step, bestie, or you'll be confused and sad. 🙈

🦋 Dynamic mountpoints with transient-ro 🦋

The transient-ro option allows privileged users to create dynamic toplevel mountpoints at runtime while keeping the filesystem read-only by default. This is particularly useful for applications that need to bind mount host paths that may be platform-specific or dynamic. Very main character energy. 👑

👑 Use cases

This feature addresses scenarios where:

• Applications need to bind mount host directories that match the host's absolute paths
• Platform-specific mountpoints are required (e.g., /Users on macOS)
• Dynamic mountpoints need to be created after deployment but before application startup
• The filesystem should remain read-only for regular processes

🔧 Configuration

To enable this feature, add the following to /usr/lib/ostree/prepare-root.conf:

[root]
transient-ro = true

✨ How it works

When transient-ro=true is set:

1. The overlayfs upper directory is mounted read-only by default
2. Privileged processes can remount it as writable only in a new mount namespace, and perform arbitrary changes there, such as creating new toplevel mountpoints
3. These mountpoints persist for the current boot but do not survive reboots or upgrades
4. Regular processes continue to see a read-only filesystem

A privileged process can achieve this using standard Linux commands. For example:

# unshare -m -- /bin/sh -c 'mount -o remount,rw / && mkdir /new-mountpoint'

💜 Example: Podman machine integration

A common use case is with podman machine on macOS, where the VM needs to bind mount host paths like /Users/username into the VM. With transient-ro, the system can:

1. Create the /Users directory dynamically at runtime
2. Bind mount the host's /Users directory to the VM's /Users
3. Keep the rest of the filesystem read-only for security

---

🌸 Enabling transient etc 🌸

The default (per above) is to have /etc persist. If however you don't need to use it for any per-machine state, then enabling a transient /etc is a great way to reduce the amount of possible state drift. No cap. Set the

[etc]
transient = true

option in /usr/lib/ostree/prepare-root.conf.

This can be combined with root.transient as well (above ☝️).

More on prepare-root: https://ostreedev.github.io/ostree/man/ostree-prepare-root.html

Note that regenerating the initramfs is required when changing this file. Again. Yes, again. 💀

🎀 Enabling state overlays 🎀

This feature enables a writable overlay on top of /opt (or really, any toplevel or subdirectory baked into the image that is normally read-only).

The semantics here are, ngl, somewhat nuanced:

• Changes persist across reboots by default
• During updates, new files from the container image override any locally modified version

The advantages are:

• It makes it very easy to make compatible applications that install into /opt.
• In contrast to transient root (above), a smaller surface of the filesystem is mutable.

The disadvantages are:

• There is no equivalent to this feature in the Docker/Podman ecosystem.
• It allows for some temporary state drift until the next update.

To enable this feature, instantiate the ostree-state-overlay@.service unit template on the target path. For example, for /opt:

RUN systemctl enable ostree-state-overlay@opt.service

💅 More generally dealing with /opt

Both transient root and state overlays above provide ways for packages that install in /opt to operate. However, for maximum immutability the best approach is simply to symlink just the parts of the /opt needed into /var. See the section on /opt in Image building and configuration guidance (see bootc-building-images(7)) for a more concrete example. You got this. 🫶

🔥 Increased filesystem integrity with fsverity 🔥

The bootc project uses composefs (https://github.com/composefs/composefs) by default for the root filesystem (using ostree's support for composefs). However, the default configuration as recommended for base images uses composefs in a mode that does not require signatures or fsverity.

bootc supports with ostree's model of hard requiring fsverity for underlying objects. Enabling this also causes bootc to error out at install time if the target filesystem does not enable fsverity. ⚠️

To enable this, inside your container build update /usr/lib/ostree/prepare-root.conf with:

[composefs]
enabled = verity

At the current time, there is no default recommended mechanism to check the integrity of the upper composefs. For more information about this, see this tracking issue: https://github.com/bootc-dev/bootc/issues/1190

Note that the default /etc and /var mounts are unaffected by this configuration. Because /etc in particular can easily contain arbitrary executable code (/etc/systemd/system unit files), many deployment scenarios that want to hard require fsverity will also want a "transient etc" model. 👀

🫶 Caveats 🫶

💀 Does not apply to logically bound images

The logically bound images (see bootc-logically-bound-images(7)) store is currently implemented using a separate mechanism, and configuring fsverity for the bootc storage has no effect on it.

😭 Enabling fsverity across upgrades

At the current time the integration is only for installation; there is not yet support for automatically ensuring that fsverity is enabled when upgrading from a state with composefs.enabled = yes to composefs.enabled = verity. Because older objects may not have fsverity enabled, the new system will likely fail at runtime to access these older files across the upgrade. So yeah, plan accordingly. 🙈

---

🌟 SEE ALSO 🌟

bootc(8), bootc-building-images(7), bootc-docs(7), bootc-logically-bound-images(7), bootc-sysroot(7), bootc-systemd-particles(7)

⭐ VERSION ⭐

v1.17.1

---

And that's the whole page, bestie!! 🎉💖✨ We made it to the end — this was BOOTC-FILESYSTEM(7), version v1.17.1, and yes that's the same BOOTC-FILESYSTEM(7) you see chilling in the footer. Go forth and filesystem responsibly. periodt. 👑🦋🌸