BTRFS(5)

💖 BTRFS(5) — the btrfs filesystem, but make it a group chat 💖

okay bestie, gather round 🫶 this is the manual page for btrfs — like, topics about the BTRFS filesystem (mount options, supported file attributes and other). the tools have their own pages, this one is the big-picture lore drop.

✨ NAME ✨

btrfs — topics about the BTRFS filesystem (mount options, supported file attributes and other). yes that's the whole name section, she's brief, she's iconic 💅

🌸 DESCRIPTION 🌸

So this doc is all about the BTRFS stuff that isn't tied to any specific tool. Currently it covers:

mount options
filesystem features
swapfile support
checksum algorithms
compression
sysfs interface
filesystem exclusive operations
filesystem limits
bootloader support
file attributes
zoned mode
control device
filesystems with multiple block group profiles
seeding device
RAID56 status and recommended practices
glossary
storage model, hardware considerations

like, a whole curriculum fr fr 📚✨

🎀 MOUNT OPTIONS 🎀

💅 BTRFS SPECIFIC MOUNT OPTIONS 💅

This bit is about the mount options that are BTRFS-only, bestie. For the generic ones, go read the mount(8) manual page — and also peek the section with BTRFS specifics down at btrfs-man5//#mount-options-generic. The options are sorted alphabetically (with the no prefix casually thrown away, she's not about that prefix life).

Note: Most mount options apply to the whole filesystem, and only the options in the first mounted subvolume actually take effect. This is because the implementation just… isn't there yet, and it may change in the future. Which means (for example) you literally cannot set per-subvolume nodatacow, nodatasum, or compress using mount options. It's supposed to get fixed eventually, but ngl it has proved to be difficult to implement correctly inside the Linux VFS framework. 💀

Mount options are processed in order, and only the last occurrence of an option takes effect — and it may disable other options due to constraints (see like nodatacow and compress for the drama). The output of the mount command shows which options have been applied. periodt.

acl, noacl — (default: on)

Turn support for POSIX Access Control Lists (ACLs) on or off. The acl(5) manual page has the whole ACL story if you want it. Also important: ACL support is build-time configurable (BTRFS_FS_POSIX_ACL), so if you ask for acl and the feature wasn't compiled in, the mount straight up fails. no vibes, just error.

autodefrag, noautodefrag — (since: 3.0, default: off)

Turns on automatic file defragmentation. When it's enabled, small random writes into files (in a range of tens of kilobytes — currently it's 64KiB) get detected and queued up for the defragmentation process. Fair warning, it may not be well suited for large database workloads.

Read latency may go up because it's reading the adjacent blocks that make up the range for defragmentation, and successive writes will merge the blocks in the new location.

Warning: Defragmenting with Linux kernel versions < 3.9 or ≥ 3.14-rc2, and also with Linux stable kernel versions ≥ 3.10.31, ≥ 3.12.12 or ≥ 3.13.4, will break up the reflinks of COW data (think files copied with cp --reflink, snapshots, or de-duplicated data). This may cause a considerable increase of space usage depending on how broken up the reflinks are. 😭

barrier, nobarrier — (default: on)

Makes sure all IO write operations actually make it through the device cache and are stored permanently when the filesystem is at its consistency checkpoint. Usually that means a flush command goes to the device that synchronizes all pending data and ordinary metadata blocks, then it writes the superblock and issues another flush.

The write flushes cost you a little and they also stop the IO block scheduler from reordering requests in a more effective way. Disabling barriers gets rid of that penalty but will most certainly lead to a corrupted filesystem in case of a crash or power loss. The ordinary metadata blocks could still be unwritten at the time the new superblock is stored permanently, while everyone was expecting that the block pointers to metadata were stored permanently before. 🙈

On a device with a volatile battery-backed write-back cache, the nobarrier option won't lead to filesystem corruption, because the pending blocks are supposed to make it to the permanent storage anyway.

clear_cache — Force clearing and rebuilding of the free space cache if something has gone wrong. This option only takes effect on the first read-write mount.

For free space cache v1, this only clears (and, unless nospace_cache is used, rebuilds) the free space cache for block groups that are modified while the filesystem is mounted with that option. To actually clear an entire free space cache v1, see btrfs check --clear-space-cache v1.

For free space cache v2, this clears the entire free space cache. To do that without needing to mount the filesystem, see btrfs check --clear-space-cache v2.

💀 DEPRECATED MOUNT OPTIONS 💀

List of mount options that have been removed, kept for backward compatibility. They're the ghosts of btrfs past 👻

recovery — (since: 3.2, default: off, deprecated since: 4.5)

Note: this option has been replaced by usebackuproot and should not be used, but it will work on 4.5+ kernels. Don't be that girl 🙈

inode_cache, noinode_cache — (removed in: 5.11, since: 3.0, default: off)

Note: the functionality has been removed in 5.11. Any stale data created by previous use of the inode_cache option can be removed by btrfs rescue clear-ino-cache. Cleanup era 🧹

check_int, check_int_data, check_int_print_mask=value — (removed in: 6.7, since: 3.0, default: off)

These debugging options control the behavior of the integrity checking module (the BTRFS_FS_CHECK_INTEGRITY config option is required). The main goal is to verify that all blocks from a given transaction period are properly linked. 🔎

check_int enables the integrity checker module, which examines all block write requests to ensure on-disk consistency — at a large memory and CPU cost. 💸

check_int_data includes extent data in the integrity checks, and implies the check_int option.

check_int_print_mask takes a bit mask of BTRFSIC_PRINT_MASK_* values as defined in fs/btrfs/check-integrity.c, to control the integrity checker module behavior.

See the comments at the top of fs/btrfs/check-integrity.c for more information. iykyk 📖

🌈 NOTES ON GENERIC MOUNT OPTIONS 🌈

Some of the general mount options from mount(8) that affect BTRFS and are worth mentioning. Group-chat PSA 💖

context — the context refers to the SELinux contexts and policy definitions passed as mount options. This works properly since version v6.8 (because the mount option parser of BTRFS was ported to a new API that also understood the options). Glow up complete ✨
noatime — under read intensive work-loads, specifying noatime significantly improves performance because no new access time information needs to be written. Without this option, the default is relatime, which only reduces the number of inode atime updates in comparison to the traditional strictatime. The worst case for atime updates under relatime occurs when many files are read whose atime is older than 24 h and which are freshly snapshotted. In that case the atime is updated and COW happens — for each file — in bulk. See also <https://lwn.net/Articles/499293/> — Atime and btrfs: a bad combination? (LWN, 2012-05-31). 😭

Note: noatime may break applications that rely on atime uptimes, like the venerable Mutt (unless you use maildir mailboxes). Choose your fighter 🥊

👑 MULTI-DEVICE SUPPORT 👑

Btrfs has built-in multi-device support, and supports several profiles that require multiple devices, including RAID0, RAID1, RAID1C3, RAID1C4, RAID10, RAID5 and RAID6. She contains multitudes, literally 🌈

However the multi-device support has some extra requirements/quirks:

Requires proper device scanning and registration 🧐 Like all multi-device storage solutions, btrfs has to scan and register all involved devices to mount the filesystem.

Normally it's done by udev, but for systems without udev, the end users are responsible for proper block file creation and scanning. Bestie you're on your own there fr fr.

Initramfs required if the rootfs has multiple devices 🌱 Without an initramfs, the kernel boot sequence doesn't create /dev/ with every block file, thus it's impossible to register all devices to fulfill the mount — even with the device= mount option.

It's strongly recommended to use an initramfs if btrfs is the rootfs. It's very easy and fast to add a new device to an existing btrfs, but without an initramfs the next mount will fail to mount the rootfs. Not the vibe 💀

Device path shown in the mount output 📍 Btrfs maintains an internal device path for each device. The device path is recorded during the initial device scan, and normally doesn't change during the lifespan of that device.

This can lead to inconvenience if the user space tool doesn't do proper parsing. 😤

For example, create a block file at /tmp/block_file, forget all devices, scan and mount that newly created block file, and then delete /tmp/block_file.

This will make btrfs register the device path using /tmp/block_file, even if that file is later deleted.

Tools like lsblk can still handle such cases by using device numbers, but a lot of other tools won't parse the mount point properly, as the device path no longer exists.

Normally this should not be a big deal, as udev is the first program to create those block files and scan them. Phew 🫶

✨ FILESYSTEM FEATURES ✨

okay so bestie, the base feature set of btrfs keeps growing over time 🌱. The backward compatibility is maintained, and every feature is optional — you literally have to explicitly ask for it, so nobody accidentally creates incompatibilities. periodt 💅

There are a few classes here, and each one has its own tool situation:

at mkfs time only — this is for the core structures, like the b-tree nodesize or the checksum algorithm. Peep mkfs.btrfs(8) for the deets 📖.
after mkfs, on an unmounted filesystem — these are the features that optimize internal structures or add new ones to support new functionality. See btrfstune(8). The command btrfs inspect-internal dump-super /dev/sdx dumps a superblock for you, and you can map the value of incompat_flags to the features listed below 🕵️‍♀️.
after mkfs, on a mounted filesystem — the features of a filesystem (with a given UUID) live in /sys/fs/btrfs/UUID/features/, one file per feature. The status is stored inside the file. A value of 1 means enabled and active, while 0 means the feature was enabled at mount time but got turned off afterwards 😴.

Wanna know if some particular feature can even be turned on for a mounted filesystem? Check the directory /sys/fs/btrfs/features/, one file per feature. A value of 1 means the feature can be enabled ✅.

Here's the list of features (see also mkfs.btrfs(8) section FILESYSTEM FEATURES, iykyk):

big_metadata — (since: 3.4, not shown since: 6.0) the filesystem uses nodesize for metadata blocks, and this can be bigger than the page size 🌸
block_group_tree — (since: 6.1) block group item representation using a dedicated b-tree. This can greatly reduce mount time for large filesystems — slay for big volumes ⚡
compress_lzo — (since: 2.6.38) the lzo compression has been used on the filesystem, either as a mount option or via btrfs filesystem defrag 🗜️
compress_zstd — (since: 4.14) the zstd compression has been used on the filesystem, either as a mount option or via btrfs filesystem defrag 🗜️
default_subvol — (since: 2.6.34) the default subvolume has been set on the filesystem 🏠
extended_iref — (since: 3.7) increased hardlink limit per file in a directory to 65536. Older kernels supported a varying number of hardlinks depending on the sum of all file name sizes that can be stored into one metadata block 🔗
free_space_tree — (since: 4.5) free space representation using a dedicated b-tree, the successor of v1 space cache ✨
metadata_uuid — (since: 5.0) the main filesystem UUID is the metadata_uuid, which stores the new UUID only in the superblock while all metadata blocks still have the UUID set at mkfs time. See btrfstune(8) for more 🆔
mixed_backref — (since: 2.6.31) the last major disk format change, improved backreferences, now default 👑
mixed_groups — (since: 2.6.37) mixed data and metadata block groups, i.e. the data and metadata are not separated and occupy the same block groups. This mode is suitable for small volumes, because there are no constraints on how the remaining space should be used (compared to the split mode, where empty metadata space cannot be used for data and vice versa) 👯
- on the other hand, the final layout is quite unpredictable and possibly highly fragmented, which means worse performance 😭
no_holes — (since: 3.14) improved representation of file extents where holes are not explicitly stored as an extent. Saves a few percent of metadata if sparse files are used 🕳️
raid1c34 — (since: 5.5) extended RAID1 mode with copies on 3 or 4 devices respectively 🛡️
raid_stripe_tree — (since: 6.7, CONFIG_BTRFS_EXPERIMENTAL) a separate tree for tracking file extents on RAID profiles 🌲
RAID56 — (since: 3.9) the filesystem contains or contained a RAID56 profile of block groups 💾
remap_tree — (since: 7.0, experimental) tree with virtual-to-virtual block address mapping 🗺️
rmdir_subvol — (since: 4.18) indicate that the rmdir(2) syscall can delete an empty subvolume just like an ordinary directory. Note that this feature only depends on the kernel version 👀
skinny_metadata — (since: 3.10) reduced-size metadata for extent references, saves a few percent of metadata 🪶
send_stream_version — (since: 5.10) number of the highest supported send stream version. Experimental features depend on CONFIG_BTRFS_EXPERIMENTAL 📤
simple_quota — (since: 6.7) simplified quota accounting 🧮
supported_checksums — (since: 5.5) list of checksum algorithms supported by the kernel module. The respective modules — or built-in implementations of the algorithms — need to be present to mount the filesystem. See section CHECKSUM ALGORITHMS 🔍
supported_sectorsizes — (since: 5.13) list of values that are accepted as sector sizes (mkfs.btrfs --sectorsize) by the running kernel 📏
supported_rescue_options — (since: 5.11) list of values for the mount option rescue that are supported by the running kernel. See btrfs(5) 🚑
vefity — (since: 5.15, CONFIG_FS_VERITY) the fs-verity read-only file-based authenticity protection can be enabled on filesystems 🔐
zoned — (since: 5.12, CONFIG_BLK_DEV_ZONED) zoned mode is allocation/write friendly to host-managed zoned devices. Allocation space is partitioned into fixed-size zones that must be updated sequentially. See section ZONED MODE 🌀

💾 SWAPFILE SUPPORT 💾

ok so a swapfile, when it's actually active, is literally a file-backed swap area. btrfs has had this since kernel 5.0 bestie 💖. Use swapon(8) to turn it on — and until you do that (or again, once you've turned it back off with swapoff(8)) it's just a regular file (with NODATACOW set), so all the special active-swapfile restrictions just don't apply to it. it's giving regular file energy 💅

but ngl there are some limitations, both in the btrfs implementation and in the Linux swap subsystem:

filesystem — must be only single device
filesystem — must have only single data profile
subvolume — cannot be snapshotted if it contains any active swapfiles
swapfile — must be preallocated (i.e. no holes) 🙈
swapfile — must be NODATACOW (i.e. also NODATASUM, no compression)

where do these come from? lowkey it's the COW-based design + the block mapping layer that makes all the fancy features (relocation, multi-device filesystems) possible. but the swap subsystem is over here expecting something simpler: no background changes to where the file's blocks live once they've been handed to swap. so the single-device + single-profile constraints? those are about the swapfile itself — the extents and where they sit. technically you can create a swapfile on a multi-device filesystem as long as the extents are all on one device, but you can't control that fr fr, it depends on free space fragmentation and how much unused space is available for new chunks.

and with active swapfiles, these whole-filesystem operations will skip some block groups or just straight up fail:

balance — block groups that have extents of any active swapfiles get skipped and reported through dmesg, the rest get processed normally
scrub — ditto ✨
resize grow — unaffected, she's fine 👑
resize shrink — works as long as the extents of any active swapfiles are outside of the shrunk range
device add — works if the new devices don't interfere with any already active swapfiles, though no new swapfile can be activated afterwards
device delete — if the device was added as above, it can also be deleted
device replace — ditto

so all this means your btrfs maintenance routines get hit HARD, specifically balance and scrub. they'll skip those block groups that have active-swapfile extents on them — and those block groups might also contain extents belonging to other files too 💀. so the swapfile's impact is amplified, way bigger than the swapfile itself. not the vibe.

for balance, this means data block groups may not be properly balanced, which causes unbalanced data/metadata usage, which can lead to early ENOSPC errors. 😭

for scrub, this means data block groups may not be verified, which degrades the integrity of data.

so like, be aware of the impact when you're running swapfiles alongside other data. and it's especially discouraged for your root filesystem. don't do that.

🚨 also: when there are no active swapfiles and a whole-filesystem exclusive operation is running (e.g. balance, device delete, shrink), you cannot temporarily activate swapfiles. the operation has to finish first, periodt.

to create + activate a swapfile, run these:

# truncate -s 0 swapfile
# chattr +C swapfile
# fallocate -l 2G swapfile
# chmod 0600 swapfile
# mkswap swapfile
# swapon swapfile

and since version 6.1 you can do the whole swapfile creation in a single command (except the activation, that's still separate):

# btrfs filesystem mkswapfile --size 2G swapfile
# swapon swapfile

ok important note: the UUID that the mkswap utility returns identifies the swap "filesystem", and because it's stored in a file, it's not generally visible or usable as an identifier — unlike if it were on a block device. so don't come for her about that 💅

once activated, the file shows up in /proc/swaps:

# cat /proc/swaps
Filename          Type          Size           Used      Priority
/path/swapfile    file          2097152        0         -2

you can make the swapfile as a one-time thing, or — once properly created — activate it on every boot with the swapon -a command (usually started by the service manager 🧚). just drop this entry into /etc/fstab, assuming the filesystem that provides the /path is already mounted at that point. you can set additional mount options relevant for the swapfile too (like priority — not the BTRFS mount options, bestie, different thing).

/path/swapfile        none        swap        defaults      0 0

so from that point on, the subvolume with the active swapfile cannot be snapshotted until the swapfile is deactivated again with swapoff. after that the swapfile is just a regular file and you can snapshot the subvolume again 🌸 — though heads up, doing that would prevent activating any swapfile that has been snapshotted. new swapfiles (not snapshotted) can still be created and activated tho, slay.

otherwise, an inactive swapfile does not affect the containing subvolume at all. activation just creates a temporary in-memory status and prevents some file operations, but it's not stored permanently. she's chill when she's off ✨

😴 HIBERNATION 😴

a swapfile can be used for hibernation but it's... not straightforward. before hibernation you have to write a resume offset to the file /sys/power/resume_offset, or set the kernel command line parameter resume_offset.

the value is the physical offset on the device. and note — this is not the same value that filefrag prints as physical offset! 🙈

btrfs uses mapping between logical and physical addresses, but here the physical can still map to one or more device-specific physical block addresses. it's the device-specific physical offset that's suitable as the resume offset. iykyk.

since version 6.1 there's a command btrfs inspect-internal map-swapfile that prints the device physical offset AND the adjusted value for /sys/power/resume_offset. note that the value is divided by page size, i.e. it's not the offset itself. 👑

# btrfs filesystem mkswapfile swapfile
# btrfs inspect-internal map-swapfile swapfile
Physical start: 811511726080
Resume offset:     198122980

for scripting and convenience, the option -r will print just the offset:

# btrfs inspect-internal map-swapfile -r swapfile
198122980

the map-swapfile command also verifies all the requirements — no holes, single device, etc. she checks everything herself 💅

🩺 TROUBLESHOOTING 🩺

if your swapfile activation fails, please verify you followed all the steps above, or check the system log (e.g. dmesg or journalctl) for more info. 🔍

notably, the swapon utility exits with a message that does NOT say what actually failed:

# swapon /path/swapfile
swapon: /path/swapfile: swapon failed: Invalid argument

the specific reason is likely printed to the system log by the btrfs module though:

# journalctl -t kernel | grep swapfile
kernel: BTRFS warning (device sda): swapfile must have single data profile

so yeah, always check the log bestie, the answer is in there 💖

✨ CHECKSUM ALGORITHMS ✨

okay so bestie, btrfs is out here checksumming your data AND your metadata by default 💖 no cap. The checksum gets calculated before the write even happens, and then verified again after the blocks come back off the devices. Here's the lil structural tea:

the whole metadata block keeps an inline checksum, stashed right there in the b-tree node header
each data block gets a detached checksum, living in the checksum tree

🫶 Note: because a data checksum is calculated literally just before submitting to the block device, btrfs has a strong requirement that the corresponding data block must not be modified until the writeback is finished. That's all fine and good for a buffered write, since btrfs has full control over its page cache. But a direct write (O_DIRECT) bypasses the page cache, and btrfs cannot control the direct IO buffer (since it can live in user space memory 😭). So it's literally possible for a user space program to modify its direct write buffer before the buffer is fully written back, and that can lead to a data checksum mismatch 💀. To avoid this, kernels starting with version 6.14 will force a direct write to fall back to buffered, if the inode requires a data checksum. Heads up: this brings an observable performance penalty 🙈. If you require true zero-copy direct writes, then set the NODATASUM flag for the inode and make sure the direct IO buffer is fully aligned to block size ✨. The kernel starting with version 7.3 will address the unstable buffer by bouncing the buffer if the inode requires data checksums. Even though this is not true zero-copy, it's still better than falling back to buffered — it achieves 95% of true zero-copy performance 👑

There are several checksum algorithms supported, bestie. The default and backward compatible one is crc32c. Since kernel 5.5 there are three more, each with different characteristics and trade-offs around speed and strength. This list is here to help you pick your fighter 💅:

CRC32C (32 bits digest) — the default, best backward compatibility. Very fast, modern CPUs have instruction-level support. Not collision-resistant, but still has good error detection capabilities 🥇
XXHASH (64 bits digest) — can be used as a CRC32C successor. Very fast, optimized for modern CPUs utilizing instruction pipelining, with good collision resistance and error detection 🔥
SHA256 (256 bits digest) — cryptographic-strength hash. Relatively slow, but with possible CPU instruction acceleration or specialized hardware cards. FIPS certified and in wide use 👑
BLAKE2b (256 bits digest) — cryptographic-strength hash. Relatively fast, with possible CPU acceleration using SIMD extensions. Not standardized, but based on BLAKE which was a SHA3 finalist, and in wide use. The algorithm used is BLAKE2b-256 that's optimized for 64-bit platforms 🦋

Now, the digest size affects the overall size of data block checksums stored in the filesystem. The metadata blocks have a fixed area up to 256 bits (32 bytes), so there's no increase there. Each data block has a separate checksum stored, though, with additional overhead of the b-tree leaves.

📊 the performance tea 📊

Approximate relative performance of the algorithms, measured against CRC32C using implementations on a 11th gen 3.6GHz intel CPU:

CRC32C — 470 Cycles/4KiB, Ratio 1.00, implementation: CPU instruction, PCL combination
XXHASH — 870 Cycles/4KiB, Ratio 1.9, implementation: reference impl.
SHA256 — 7600 Cycles/4KiB, Ratio 16, implementation: libgcrypt
SHA256 — 8500 Cycles/4KiB, Ratio 18, implementation: openssl
SHA256 — 8700 Cycles/4KiB, Ratio 18, implementation: botan
SHA256 — 32000 Cycles/4KiB, Ratio 68, implementation: builtin, CPU instruction
SHA256 — 37000 Cycles/4KiB, Ratio 78, implementation: libsodium
SHA256 — 78000 Cycles/4KiB, Ratio 166, implementation: builtin, reference impl.
BLAKE2b — 10000 Cycles/4KiB, Ratio 21, implementation: builtin/AVX2
BLAKE2b — 10900 Cycles/4KiB, Ratio 23, implementation: libgcrypt
BLAKE2b — 13500 Cycles/4KiB, Ratio 29, implementation: builtin/SSE41
BLAKE2b — 13700 Cycles/4KiB, Ratio 29, implementation: libsodium
BLAKE2b — 14100 Cycles/4KiB, Ratio 30, implementation: openssl
BLAKE2b — 14500 Cycles/4KiB, Ratio 31, implementation: kcapi
BLAKE2b — 14500 Cycles/4KiB, Ratio 34, implementation: builtin, reference impl.

🌈 Note: since kernel v6.16 the accelerated implementation is always used if available. slay.

Many kernels are configured with SHA256 as built-in and not as a module. Up to kernel v6.15 the accelerated versions are however provided by the modules and must be loaded explicitly (modprobe sha256) before mounting the filesystem to make use of them ⚠️. You can check in /sys/fs/btrfs/FSID/checksum which one is used. If you see sha256-generic, then you may want to unmount and mount the filesystem again. Changing that on a mounted filesystem is not possible, periodt 🙅‍♀️.

Check the file /proc/crypto — when the implementation is built-in, you'd find:

name         : sha256
driver       : sha256-generic
module       : kernel
priority     : 100
...

While accelerated implementation is e.g.:

name         : sha256
driver       : sha256-avx2
module       : sha256_ssse3
priority     : 170
...

🌸 COMPRESSION 🌸

Btrfs supports transparent file compression, bestie 🎀. There are three algorithms available: ZLIB, LZO and ZSTD (since v4.14), with various levels. The compression happens on the level of file extents, and the algorithm is selected by file property, mount option or by a defrag command 💫. You can have a single btrfs mount point that has some files that are uncompressed, some that are compressed with LZO, some with ZLIB, for instance (though you may not want it that way, it is supported — no cap 🫶).

Once the compression is set, all newly written data will be compressed, i.e. existing data are untouched ✨. Data are split into smaller chunks (128KiB) before compression to make random rewrites possible without a high performance hit. Due to the increased number of extents the metadata consumption is higher. The chunks are compressed in parallel 🔥.

The algorithms can be characterized as follows regarding the speed/ratio trade-offs:

ZLIB

slower, higher compression ratio 🐌
levels: 1 to 9, mapped directly, default level is 3
good backward compatibility

LZO

faster compression and decompression than ZLIB, worse compression ratio, designed to be fast ⚡
no levels
good backward compatibility

ZSTD

compression comparable to ZLIB with higher compression/decompression speeds and different ratio 💅
levels: -15..15, mapped directly, default is 3
support since 4.14
levels 1..15 supported since 5.1
levels -15..-1 supported since 6.15

The differences depend on the actual data set and cannot be expressed by a single number or recommendation, fr fr 😭. Higher levels consume more CPU time and may not bring a significant improvement; lower levels are close to real time.

💖 HOW TO ENABLE COMPRESSION 💖

okay so bestie, first thing you need to know: compression is usually turned on for the whole filesystem, set at the mount point. and here's the tea — those compression mount options are shared among all mounts of the same filesystem, whether they're bind mounts or subvolume mounts. so like, no sneaking different settings per mount, the vibe is communal. go peek at the MOUNT OPTIONS section in btrfs(5) for the full deets. 🫶

$ mount -o compress=zstd /dev/sdx /mnt

that right there turns on the zstd algorithm at its default level — which is 3, in case you were wondering. but you can set the level manually too, like zstd:3. ✨

the tradeoff, because there's always a tradeoff: higher levels compress better but cost you time. that extra time can bump up your write latency, so low levels are the move for real-time compression — on a reasonably fast CPU they won't cause any noticeable performance drops. slay. 💅

$ btrfs filesystem defrag -czstd file

this one starts defragmentation of the whole file and applies the compression, regardless of the mount option. as of version 6.14 you can also pass the compression level with the --level or -L argument. important though: the compression algorithm here is not persistent — it only applies to that one defragmentation command. any other writes get whatever other compression settings apply. periodt.

persistent settings on a per-file basis can be set two ways:

$ chattr +c file
$ btrfs property set file compression zstd

the first command uses the legacy file-attributes interface inherited from the ext2 filesystem, and it's not flexible — by default it sets zlib compression. the other command sets a property on the file with the algorithm you give it. (note: setting the level that way is not yet implemented. someday bestie, someday. 😭)

🌸 COMPRESSION LEVELS 🌸

level support timeline, because version history is important iykyk:

ZLIB level support was added in v4.14
LZO does not support levels at all (the kernel implementation provides only one)
ZSTD level support was added in v5.1
negative levels arrived in v6.15

ZLIB has 9 levels supported (1 to 9), mapping 1:1 from the mount option to the algorithm-defined level. the default is level 3, which gives a reasonably good compression ratio and is still reasonably fast. the difference in compression gain between levels 7, 8 and 9 is comparable, but the higher levels take longer. 💀

ZSTD support includes levels -15..15 — a subset of the full range of what ZSTD provides:

-15..-1 are real-time, with worse compression ratio
1..3 are near real-time, with good compression
4..8 are slower with improved compression
9..15 try even harder, though the resulting size may not be significantly improved

higher levels also require more memory, and since they need more CPU, your system performance gets affected. not the system performance bestie!! ⭐

Level 0 always maps to the default. and good news: the compression level does not affect compatibility. 🎀

🔥 EXCEPTIONS 🔥

any file that has been touched by the fallocate system call will always be excepted from compression — even if the force-compress mount option is used. yes, even then. 😤

the reason: a successful fallocate call must guarantee that future writes to the allocated range will not fail due to lack of space. that is genuinely difficult to guarantee in a COW filesystem. to reduce the chances of it happening, btrfs preallocates space and disables compression for the file.

as a workaround, you can trigger a compressed rewrite for such a file using the btrfs defrag command. but be aware — if the file gets touched again by the fallocate system call, it will be excepted again from compression for all the new data written to it. it's giving cursed, honestly. 🙈

💜 INCOMPRESSIBLE DATA 💜

files with already-compressed data, or data that just won't compress well within the CPU and memory constraints of the kernel implementations, go through a simple decision logic. if the first portion of data being compressed is not smaller than the original, compression of the whole file is disabled. 🚫

unless the filesystem is mounted with compress-force, in which case btrfs will try compressing every block, falling back to storing the uncompressed version for each block that ends up larger after compression. this is not optimal and is subject to optimizations and further development. (we love an honest king.)

if a file gets identified as incompressible, a flag is set — NOCOMPRESS — and it's sticky. on that file, compression won't be performed unless forced. the flag can also be set by chattr +m (since e2fsprogs 1.46.2), or by properties with value no or none. an empty value will reset it to the default that's currently applicable on the mounted filesystem. ✨

there are two ways to detect incompressible data:

actual compression attempt — data are compressed, and if the result is not smaller, it's discarded. so this depends on the algorithm and level
pre-compression heuristics — a quick statistical evaluation on the data is performed, and based on the result either compression is performed or skipped. the NOCOMPRESS bit is not set just by the heuristic, only if the compression algorithm does not make an improvement

$ lsattr file
---------------------m file

using force compression is not recommended. the heuristics are supposed to decide that, and compression algorithms internally detect incompressible data too. let the algorithm cook. 🍓

🦋 PRE-COMPRESSION HEURISTICS 🦋

the heuristics aim to do a few quick statistical tests on the compressed data, in order to avoid probably-costly compression that would turn out to be inefficient. compression algorithms could have internal detection of incompressible data too, but that leads to more overhead — the compression is done in another thread and it has to write the data anyway. the heuristic is read-only and can utilize cached memory. 💅

the tests performed are based on the following: data sampling, long repeated pattern detection, byte frequency, and Shannon entropy.

👑 COMPATIBILITY 👑

compression requires both data checksums and COW, so either nodatasum — whether mount option or inode flag — will result in no compression. that's just how it is.

direct IO reads of compressed data will always fall back to buffered reads.

direct IO write behavior depends on the inode flag and kernel version. ok so let's break it down:

before v7.3, for inodes with data checksums, direct IO writes always fall back to buffered writes, thus can generate compressed data if the mount option / inode flags allow that
for inodes without data checksums, direct IO writes will not populate page cache, and since the inode has no data checksums, no compressed data will be generated anyway
starting with kernel v7.3, no compression is performed, no matter if the inode has data checksums or not

the compression algorithms have been added over time, so version compatibility should also be considered — together with other tools that may access the compressed data, like bootloaders. 👑

🌈 SYSFS INTERFACE 🌈

btrfs has a sysfs interface to provide extra knobs. yes, extra knobs. we love a knob. ✨

the top level path is /sys/fs/btrfs/, and the main directory layout is the following:

features/ — All supported features — version 3.14
UUID/ — Mounted fs UUID — version 3.14
UUID/allocation/ — Space allocation info — version 3.14
UUID/bdi/ — Backing device info (writeback) — version 5.9
UUID/debug/ — Debugging information (optional) — version 5.6
UUID/devices/NODE/ — Symlink to each block device sysfs — version 5.6
UUID/devinfo/DEVID/ — Btrfs specific info for each device — version 5.6
UUID/discard/ — Discard stats and tunables — version 6.1
UUID/features/ — Features of the filesystem — version 3.14
UUID/qgroups/ — Global qgroup info — version 5.9
UUID/qgroups/LEVEL_ID/ — Info for each qgroup — version 5.9

for the /sys/fs/btrfs/features/ directory, each file means a supported feature of the current kernel. most files have value 0. otherwise it depends on the file — value 1 typically means the feature can be turned on a mounted filesystem. some directories may depend on a feature (qgroups) or a build config option (debug and CONFIG_BTRFS_DEBUG). 💖

For /sys/fs/btrfs/UUID/features/ directory, each file means an enabled feature on the mounted filesystem. 💅

The features share the same name in section FILESYSTEM FEATURES. iykyk bestie, the names match up so you can cross-reference and slay.

🦋 UUID

Files in /sys/fs/btrfs/UUID/ directory are:

bg_reclaim_threshold — (RW, since: 5.19) Used space percentage of total device space to start auto block group claim. Mostly for zoned devices. Lowkey a zoned-device thing 🌀
checksum — (RO, since: 5.5) The checksum used for the mounted filesystem. This includes both the checksum type (peep section CHECKSUM ALGORITHMS) and the selected implementation (depends on kernel, it used to be type of hardware acceleration):

crc32c (crc32c-lib)

clone_alignment — (RO, since: 3.16) The bytes alignment for clone and dedupe ioctls. No cap. 🧬
commit_stats — (RW, since: 6.0) The performance statistics for btrfs transaction commit since the first mount. For debugging or monitoring purposes. ✨

Writing into this file will reset the maximum commit duration (max_commit_ms) to 0. The file looks like:

commits 70649
last_commit_ms 2
max_commit_ms 131
total_commit_ms 170840

commits — number of transaction commits since the first mount
last_commit_ms — duration in milliseconds of the last commit
max_commit_ms — maximum time a transaction commit took since first mount or last reset
total_commit_ms — sum of all transaction commit times

exclusive_operation — (RO, since: 5.10) Show the running exclusive operation. Check section FILESYSTEM EXCLUSIVE OPERATIONS for details. 👀
generation — (RO, since: 5.11) Show the generation of the mounted filesystem.
label — (RW, since: 3.14) Show the current label of the mounted filesystem, followed by a newline (which is otherwise trimmed from the label). 🏷️
metadata_uuid — (RO, since: 5.0) Show the metadata UUID of the mounted filesystem. Check the metadata_uuid feature for more details.
nodesize — (RO, since: 3.14) Show the nodesize of the mounted filesystem.
quota_override — (RW, since: 4.13) Show the current quota override status:
- 0 — no quota override.
- 1 — quota override, writes will ignore the existing limit settings
read_policy — (RW, since: 5.11) Show the current policy for balancing reads from devices. Currently only pid (balance using the process id (pid) value) is supported. More balancing policies are available in experimental build, namely round-robin.
sectorsize — (RO, since: 3.14) Show the sectorsize of the mounted filesystem. 📏
temp_fsid — (RO, since 6.7) Indicate that this filesystem got assigned a temporary FSID at mount time, making possible to mount devices with the same FSID. 🫶

🎀 UUID/allocations

Files and directories in /sys/fs/btrfs/UUID/allocations directory are:

global_rsv_reserved — (RO, since: 3.14) The used bytes of the global reservation.
global_rsv_size — (RO, since: 3.14) The total size of the global reservation.
data/, metadata/ and system/ directories — (RO, since: 5.14) Space info accounting for the 3 block group types.

🌸 UUID/allocations/{data,metadata,system}

Files in /sys/fs/btrfs/UUID/allocations/data,metadata,system directory are:

bg_reclaim_threshold — (RW, since: 5.19) Reclaimable space percentage of block group's size (excluding permanently unusable space) to reclaim the block group. Can be used on regular or zoned devices.
bytes_* — (RO) Values of the corresponding data structures for the given block group type and profile that are used internally and may change rapidly depending on the load.

Complete list: bytes_may_use, bytes_pinned, bytes_readonly, bytes_reserved, bytes_used, bytes_zone_unusable 💀

chunk_size — (RW, since: 6.0) Show the chunk size. Can be changed for data and metadata (independently) and cannot be set for system block group type. Cannot be set for zoned devices as it depends on the fixed device zone size. Upper bound is 10% of the filesystem size, the value must be multiple of 256MiB and greater than 0. ⚠️
disk_total — (RO, since: 3.14) Total number of bytes on devices occupied by this block group type.
disk_used — (RO, since: 3.14) Portion of disk_total in bytes which is used by this block group type.
dynamic_reclaim — (RW, since: 6.11) Enable or disable dynamic block group reclaim. Use a calculated threshold instead of fixed value in bg_reclaim_threshold.
flags — (RO, since: 6.11) Raw decimal value of block group flags.
force_chunk_alloc — (RW, since: 6.0, CONFIG_BTRFS_DEBUG) (Privileged write.) Force allocation of a new chunk of this block group type. Note his can cause ENOSPC, users should know what they're doing. 😭

fr fr do not just poke at force_chunk_alloc for the plot. You can genuinely run yourself out of space and the kernel will be like "bestie I warned you."

periodic_reclaim — (RW, since: 6.11) Enable or disable periodic block group reclaim based on bg_reclaim_threshold.
reclaim_bytes — (RO, since: 6.11) Monotonically increasing counter of reclaimed bytes.
reclaim_count — (RO, since: 6.11) Monotonically increasing counter of block group reclaim attempts.
reclaim_errors — (RO, since: 6.11) Monotonically increasing counter of reclaim errors.
size_classes — (RO, since: 6.3) Numbers of block groups of a given class based on heuristics that measure extent length, age and fragmentation. 🧚

none 136
small 374
medium 282
large 93

total_bytes — (RO, since: 3.14) Total bytes.

💜 UUID/bdi

Symlink to the sysfs directory of the backing device info (BDI), which is related to writeback process and infrastructure. It's giving plumbing, but iconic plumbing. 🔧

⭐ UUID/devices

Files in /sys/fs/btrfs/UUID/devices directory are symlinks named after device nodes (e.g. sda, dm-0) and pointing to their sysfs directory.

🌈 UUID/devinfo

The directory contains subdirectories named after device ids (numeric values). Each subdirectory has information about the device of the given devid.

🍓 UUID/devinfo/DEVID

Files in /sys/fs/btrfs/UUID/devinfo/DEVID directory are:

error_stats — (RO, since: 5.14) Show device stats of this device, same as btrfs device stats (btrfs-device(8) <>). 👑

write_errs 0
read_errs 0
flush_errs 0
corruption_errs 0
generation_errs 0

fsid — (RO, since: 5.17) Show the fsid which the device belongs to. It can be different than the UUID if it's a seed device.
in_fs_metadata — (RO, since: 5.6) Show whether we have found the device. Should always be 1, as if this turns to 0, the DEVID directory would get removed automatically.
missing — (RO, since: 5.6) Show whether the device is considered missing by the kernel module. 🙈
replace_target — (RO, since: 5.6) Show whether the device is the replace target. If no device replace is running, this value is 0.
scrub_speed_max — (RW, since: 5.14) Show the scrub speed limit for this device. The unit is Bytes/s. 0 means no limit. The value can be set but is not persistent and not preserved beyond mount. ⏳
writeable — (RO, since: 5.6) Show if the device is writeable.

🦋 UUID/qgroups

Files in /sys/fs/btrfs/UUID/qgroups/ directory are:

drop_subtree_threshold — (RW, since: 6.1) Show the subtree drop threshold to automatically mark qgroup inconsistent.

When dropping large subvolumes with qgroup enabled, there would be a huge load for qgroup accounting. If we have a subtree whose level is larger than or equal to this value, we will not trigger qgroup account at all, but mark qgroup inconsistent to avoid the huge workload.

Default value is 3, which means that trees of low height will be accounted properly as this is sufficiently fast. The value was 8 until 6.13 where no subtree drop can trigger qgroup rescan making it less useful.

Lower value can reduce qgroup workload, at the cost of extra qgroup rescan to re-calculate the numbers. 💅

enabled — (RO, since: 6.1) Show if qgroup is enabled. Also, if qgroup is disabled, the qgroups directory will be removed automatically.
inconsistent — (RO, since: 6.1) Show if the qgroup numbers are inconsistent. If 1, it's recommended to do a qgroup rescan.
mode — (RO, since: 6.7) Show which quota mode is enabled, can be qgroup for full accounting or squota for simplified accounting.

🌸 UUID/qgroups/LEVEL_ID

Files in each /sys/fs/btrfs/UUID/qgroups/LEVEL_ID/ directory are:

exclusive — (RO, since: 5.9) Show the exclusively owned bytes of the qgroup.
limit_flags — (RO, since: 5.9) Show the numeric value of the limit flags. If 0, means no limit implied.
max_exclusive — (RO, since: 5.9) Show the limits on exclusively owned bytes.
max_referenced — (RO, since: 5.9) Show the limits on referenced bytes.
referenced — (RO, since: 5.9) Show the referenced bytes of the qgroup.
rsv_data — (RO, since: 5.9) Show the reserved bytes for data.
rsv_meta_pertrans — (RO, since: 5.9) Show the reserved bytes for per transaction metadata.
rsv_meta_prealloc — (RO, since: 5.9) Show the reserved bytes for preallocated metadata.

🎀 UUID/discard

Files in /sys/fs/btrfs/UUID/discard/ directory are:

discardable_bytes — (RO, since: 6.1) Show amount of bytes that can be discarded in the async discard and nodiscard mode. 🗑️
discardable_extents — (RO, since: 6.1) Show number of extents to be discarded in the async discard and nodiscard mode.
discard_bitmap_bytes — (RO, since: 6.1) Show amount of discarded bytes from data tracked as bitmaps.

`discard_extent_bytes`

(RO, since: 6.1) — shows you the amount of discarded extents from data tracked as bitmaps. 💅

`discard_bytes_saved`

(RO, since: 6.1) — shows the amount of bytes that were reallocated without being discarded.

`kbps_limit`

(RW, since: 6.1) — a tunable limit of kilobytes per second issued as discard IO in the async discard mode.

`iops_limit`

(RW, since: 6.1) — a tunable limit of the number of discard IO operations to be issued in the async discard mode.

`max_discard_size`

(RW, since: 6.1) — a tunable limit for the size of one IO discard request.

🚧 FILESYSTEM EXCLUSIVE OPERATIONS 🚧

okay so bestie, there are some operations that touch the whole filesystem, and they literally cannot run at the same time. Like, one at a time, in the group chat, no talking over each other. If you try to start one while another is already running, it will fail (see the exceptions below, because of course there are exceptions 🫶).

Since kernel 5.10 you can peek at whichever operation is currently running from /sys/fs/UUID/exclusive_operation, and it'll show you one of these values:

balance
balance paused (since 5.17)
device add
device delete
device replace
resize
swapfile activate
none

Enqueuing is supported for several btrfs subcommands, so you can kick a bunch of them off at once and they'll get serialized. queue girlies rise 🙈

And there's an exception: when a paused balance is hanging around, it allows a device add operation to start, because those two don't really collide — and honestly this is iconic because it means you can add more space so the balance can actually finish. 💖

📏 FILESYSTEM LIMITS 📏

maximum file name length

255

This limit is imposed by Linux VFS, bestie — the structures of BTRFS could totally store larger file names. it's not BTRFS's fault, it's the VFS's. 💀

maximum symlink target length

This one depends on the nodesize value. For 4KiB it's 3949 bytes, and for larger nodesize it's 4095 due to the system limit PATH_MAX.

Worth knowing: the symlink target may not be a valid path, i.e. the path name components can exceed the limits (NAME_MAX). There's no content validation at symlink(3) creation. no vibe check, no nothing. 🌸

maximum number of inodes

264 but this depends on the available metadata space, because the inodes are created dynamically. main character energy: they come into existence whenever they want.

Each subvolume is an independent namespace of inodes and thus their numbers, so the limit is per subvolume, not for the whole filesystem. ✨

inode numbers

Minimum number: 256 (for subvolumes), regular files and directories: 257, maximum number: (264 - 256).

The inode numbers that can be assigned to user created files are from the whole 64bit space — except the first 256 and the last 256 in that range, which are reserved for internal b-tree identifiers. 🔒

maximum file length

The inherent limit of BTRFS is 264 (16 EiB) but the practical limit of Linux VFS is 263 (8 EiB).

maximum number of subvolumes

The subvolume ids can go up to 248, but the number of actual subvolumes depends on the available metadata space. fr fr it's a "how much room do you have" situation, not a "here's your exact cap" situation.

The space consumed by all subvolume metadata — including bookkeeping of shared extents — can be large (MiB, GiB). And the range is not the full 64bit range, because of qgroups that use the upper 16 bits for another purposes. yes, that's the actual reason, not me being vague. 💅

maximum number of hardlinks of a file in a directory

65536 when the extref feature is turned on during mkfs (which is the default), and roughly 100 otherwise — and even that depends on the file name length that fits into one metadata node.

minimum filesystem size

The minimal size of each device depends on the mixed-bg feature. Without that (the default) it's about 109MiB, and with mixed-bg it's 16MiB. tiny king 🧚

👑 BOOTLOADER SUPPORT 👑

GRUB2 (https://www.gnu.org/software/grub) has the most advanced support of booting from BTRFS with respect to features. she ate, honestly.

U-Boot (https://www.denx.de/wiki/U-Boot/) has decent support for booting, but not all BTRFS features are implemented — so check the documentation, bestie, don't just assume. 🦋

In general, the first 1MiB on each device is unused, with the exception of the primary superblock that is on the offset 64KiB and spans 4KiB. The rest can be freely used by bootloaders or for other system information.

⚠️ Note that booting from a filesystem on zoned device is not supported. not even a little. 💀

🎀 FILE ATTRIBUTES 🎀

The btrfs filesystem supports setting file attributes or flags. And listen — there are old and new interfaces with confusing names, so let's just clarify that right now before anyone spirals:

attributes: chattr(1) or lsattr(1) utilities (the ioctls are FS_IOC_GETFLAGS and FS_IOC_SETFLAGS). due to the ioctl names, the attributes are also called flags. iykyk.
xflags: to distinguish from the previous, it's extended flags, with tunable bits similar to the attributes but extensible — and new bits will be added in the future (the ioctls are FS_IOC_FSGETXATTR and FS_IOC_FSSETXATTR, but they are not related to extended attributes that are also called xattrs). There's no standard tool to change the bits, but there's support in xfs_io(8) as command xfs_io -c chattr.

Attributes have constraints associated and not all combinations can be set — the order of setting them also matters. Most attributes apply to files and directories but the semantics may differ. For directories the attribute may only mean to set this attribute to all new files (inheritable in the list below). Some attributes need root privileges to be set. yes queen, sometimes you gotta sudo. 👑

Attributes

a (file, dir, root) — append only, new writes are always written at the end of the file. no cutting the line, babe. 🍓
A (file, dir) — no atime updates.
c (file, dir, inherited) — compress data, all data written after this attribute is set will be compressed. Please note that compression is also affected by the mount options or the parent directory attributes.

When set on a directory, all newly created files will inherit this attribute. This attribute cannot be set with m at the same time. pick a lane. 💅

C (file, dir, inherited) — no copy-on-write, file data modifications are done in-place.

When set on a directory, all newly created files will inherit this attribute.

⚠️ Note: Due to implementation limitations, this flag can be set/unset only on empty files. empty files only, she's picky. 😭

d (file) — no dump, makes sense with 3rd party tools like dump(8). On BTRFS the attribute can be set/unset but no other special handling is done. so like... she's there, she's just not doing much.
D (dir) — synchronous directory updates, for more details search open(2) for O_SYNC and O_DSYNC.
i (file, dir, root) — immutable, no file data and metadata changes allowed even to the root user as long as this attribute is set (obviously the exception is unsetting the attribute). ROOT PROOF. periodt. 🔥
m (file, dir) — no compression, permanently turn off compression on the given file. Any compression mount options will not affect this file. (chattr(1) support added in 1.46.2.)

When set on a directory, all newly created files will inherit this attribute. This attribute cannot be set with c at the same time. again: pick a lane. 🌈

S (file) — synchronous updates, for more details search open(2) for O_SYNC and O_DSYNC.
V (file, read-only) — fs-verity enabled on the file.

No other attributes are supported. For the complete list please refer to the chattr(1) manual page. 💜

XFLAGS

There's an overlap of letters assigned to the bits with the attributes, so heads up: this list refers to what xfs_io(8) provides:

i — immutable, same as the attribute.
a — append only, same as the attribute.
s — synchronous updates, same as the attribute S.
A — no atime updates, same as the attribute.
d — no dump, same as the attribute.

🌈 ZONED MODE 🌈

okay bestie, since version 5.12 btrfs supports something called zoned mode. It's a special on-disk format and allocation/write strategy that's friendly to zoned devices. Short version: a device gets partitioned into fixed-size zones and each zone can be updated append-only style, or reset. Since btrfs has no fixed data structures (except, like, the super blocks obviously), zoned mode only needs block placement that follows the device constraints. You can learn about the whole architecture at <https://zonedstorage.io> . 🦋

The devices are also called SMR/ZBC/ZNS, in host-managed mode. Not me warning you: there are devices that appear non-zoned but actually are, that's drive-managed and using zoned mode won't help. 😭

The zone size depends on the device, typical sizes are 256MiB or 1GiB. In general it must be a power of two. Emulated zoned devices like null_blk let you set various zone sizes. ✨

Requirements, limitations 🎀

all devices must have the same zone size
maximum zone size is 8GiB
minimum zone size is 4MiB
mixing zoned and non-zoned devices is possible, the zone writes are emulated, but this is namely for testing
the super block is handled in a special way and is at different locations than on a non-zoned filesystem:
- primary: 0B (and the next two zones)
- secondary: 512GiB (and the next two zones)
- tertiary: 4TiB (4096GiB, and the next two zones)

Incompatible features 💀

The main constraint of the zoned devices is lack of in-place update of the data. This is inherently incompatible with some features — and babe, I mean inherently, no workaround vibes here:

NODATACOW — overwrite in-place, cannot create such files
fallocate — preallocating space for in-place first write
mixed-bg — unordered writes to data and metadata, fixing that means using separate data and metadata block groups
booting — the zone at offset 0 contains superblock, resetting the zone would destroy the bootloader data

Initial support lacks some features but they're planned:

only single (data, metadata) and DUP (metadata) profile is supported
fstrim — due to dependency on free space cache v1

Super block 👑

As said above, super block is handled in a special way. In order to be crash safe, at least one zone in a known location must contain a valid superblock. This is implemented as a ring buffer in two consecutive zones, starting from known offsets 0B, 512GiB and 4TiB.

The values are different than on non-zoned devices. Each new super block is appended to the end of the zone, once it's filled, the zone is reset and writes continue to the next one. Looking up the latest super block needs to read offsets of both zones and determine the last written version.

The amount of space reserved for super block depends on the zone size. The secondary and tertiary copies are at distant offsets as the capacity of the devices is expected to be large, tens of terabytes. Maximum zone size supported is 8GiB, which would mean that e.g. offset 0-16GiB would be reserved just for the super block on a hypothetical device of that zone size. This is wasteful but required to guarantee crash safety. 🙈

Zone reclaim, garbage collection 🗑️

As the zones are append-only, overwriting data or COW changes in metadata make parts of the zones used but not connected to the filesystem structures. This makes the space unusable and grows over time. Once the ratio hits a (configurable) threshold a background reclaim process is started and relocates the remaining blocks in use to a new zone. The old one is reset and can be used again. slay.

This process may take some time depending on other background work or amount of new data written. It is possible to hit an intermittent ENOSPC. Some devices also limit number of active zones. 😭

Devices 🔥

Real hardware 💾

The WD Ultrastar series 600 advertises HM-SMR, i.e. the host-managed zoned mode. There are two more: DA (device managed, no zoned information exported to the system), HA (host aware, can be used as regular disk but zoned writes improve performance). There are not many devices available at the moment, the information about exact zoned mode is hard to find, check data sheets or community sources gathering information from real devices.

Note: zoned mode won't work with DM-SMR disks. periodt.

Ultrastar® DC ZN540 NVMe ZNS SSD (product brief <https://documents.westerndigital.com/content/dam/doc-library/en_us/assets/public/western-digital/collateral/product-brief/product-brief-ultrastar-dc-zn540.pdf>)

Emulated: `null_blk` 🧚

The driver null_blk provides memory backed device and is suitable for testing. There are some quirks setting up the devices. The module must be loaded with nr_devices=0 or the numbering of device nodes will be offset. The configfs must be mounted at /sys/kernel/config and the administration of the null_blk devices is done in /sys/kernel/config/nullb. The device nodes are named like /dev/nullb0 and are numbered sequentially. NOTE: the device name may be different than the named directory in sysfs!

Setup:

modprobe configfs
modprobe null_blk nr_devices=0

Create a device mydev, assuming no other previously created devices, size is 2048MiB, zone size 256MiB. There are more tunable parameters, this is a minimal example taking defaults:

cd /sys/kernel/config/nullb/
mkdir mydev
cd mydev
echo 2048 > size
echo 1 > zoned
echo 1 > memory_backed
echo 256 > zone_size
echo 1 > power

This will create a device /dev/nullb0 and the value of file index will match the ending number of the device node.

Remove the device:

rmdir /sys/kernel/config/nullb/mydev

Then continue with mkfs.btrfs /dev/nullb0, the zoned mode is auto-detected. ✨

For convenience, there's a script wrapping the basic null_blk management operations <https://github.com/kdave/nullb.git>, the above commands become:

nullb setup
nullb create -s 2g -z 256
mkfs.btrfs /dev/nullb0
...
nullb rm nullb0

Emulated: TCMU runner 🌸

TCMU is a framework to emulate SCSI devices in userspace, providing various backends for the storage, with zoned support as well. A file-backed zoned device can provide more options for larger storage and zone size. Please follow the instructions at <https://zonedstorage.io/projects/tcmu-runner/> .

Compatibility, incompatibility 🫶

the feature sets an incompat bit and requires new kernel to access the filesystem (for both read and write)
superblock needs to be handled in a special way, there are still 3 copies but at different offsets (0, 512GiB, 4TiB) and the 2 consecutive zones are a ring buffer of the superblocks, finding the latest one needs reading it from the write pointer or do a full scan of the zones
mixing zoned and non zoned devices is possible (zones are emulated) but is recommended only for testing
mixing zoned devices with different zone sizes is not possible
zone sizes must be power of two, zone sizes of real devices are e.g. 256MiB or 1GiB, larger size is expected, maximum zone size supported by btrfs is 8GiB

Status, stability, reporting bugs 🙋

The zoned mode has been released in 5.12 and there are still some rough edges and corner cases one can hit during testing. Please report bugs to <https://github.com/naota/linux/issues/> .

References 📚

<https://zonedstorage.io>
- <https://zonedstorage.io/projects/libzbc/> — libzbc is library and set of tools to directly manipulate devices with ZBC/ZAC support
- <https://zonedstorage.io/projects/libzbd/> — libzbd uses the kernel provided zoned block device interface based on the ioctl() system calls
<https://hddscan.com/blog/2020/hdd-wd-smr.html> — some details about exact device types
<https://lwn.net/Articles/853308/> — Btrfs on zoned block devices
<https://www.usenix.org/conference/vault20/presentation/bjorling> — Zone Append: A New Way of Writing to Zoned Storage

💅 CONTROL DEVICE 💅

okay so there's this character special device /dev/btrfs-control with major and minor numbers 10 and 234 (the device can be found under the misc category). iconic.

$ ls -l /dev/btrfs-control
crw------- 1 root root 10, 234 Jan  1 12:00 /dev/btrfs-control

The device accepts some ioctl calls that can perform following actions on the filesystem module:

scan devices for btrfs filesystem (i.e. to let multi-device filesystems mount automatically) and register them with the kernel module
similar to scan, but also wait until the device scanning process is finished for a given filesystem
get the supported features (can be also found under /sys/fs/btrfs/features)

The device is created when btrfs is initialized, either as a module or a built-in functionality and makes sense only in connection with that. Running e.g. mkfs without the module loaded will not register the device and will probably warn about that.

In rare cases when the module is loaded but the device is not present (most likely accidentally deleted), it's possible to recreate it by

# mknod --mode=600 /dev/btrfs-control c 10 234

or (since 5.11) by a convenience command

# btrfs rescue create-control-device

The control device is not strictly required but the device scanning will not work and a workaround would need to be used to mount a multi-device filesystem. The mount option device can trigger the device scanning during mount, see also btrfs device scan. ✨

👯‍♀️ FILESYSTEM WITH MULTIPLE PROFILES 👯‍♀️

ok bestie, so here's a thing that can happen and it lowkey stresses people out but it's fine: a btrfs filesystem can end up holding multiple block group profiles of the same type. 😳 How? Usually when a profile conversion using balance filters got interrupted (peep btrfs-balance(8) for that lore). So some btrfs commands will literally vibe-check the filesystem for this exact situation and hit you with a warning like this:

WARNING: Multiple block group profiles detected, see 'man btrfs(5)'.
WARNING:   Data: single, raid1
WARNING:   Metadata: single, raid1

and the matching output of btrfs filesystem df might be giving something like:

WARNING: Multiple block group profiles detected, see 'man btrfs(5)'.
WARNING:   Data: single, raid1
WARNING:   Metadata: single, raid1
Data, RAID1: total=832.00MiB, used=0.00B
Data, single: total=1.63GiB, used=0.00B
System, single: total=4.00MiB, used=16.00KiB
Metadata, single: total=8.00MiB, used=112.00KiB
Metadata, RAID1: total=64.00MiB, used=32.00KiB
GlobalReserve, single: total=16.25MiB, used=0.00B

see how there's more than one line for type Data and Metadata, while the profiles are single and RAID1? yeah. that's the tea. 🫖

This state of the filesystem is OK but it most likely needs the user/administrator to take an action and finish the interrupted tasks. This cannot be easily done automatically, also the user knows the expected final profiles. 💁‍♀️

In the example above, the filesystem started as a single device and single block group profile. Then another device was added, followed by balance with convert=raid1 but for some reason hasn't finished. Restarting the balance with convert=raid1 will continue and end up with a filesystem with all block group profiles RAID1. slay, closure. ✨

Note: If you're familiar with balance filters, you can use convert=raid1,profiles=single,soft, which will take only the unconverted single profiles and convert them to raid1. This may speed up the conversion as it would not try to rewrite the already converted raid1 profiles. 💅

Having just one profile is desired because this also clearly defines the profile of newly allocated block groups — otherwise this depends on internal allocation policy. And when there are multiple profiles present, the order of selection is RAID56, RAID10, RAID1, RAID0 as long as the device number constraints are satisfied. 👑

The commands that print that warning were chosen so they're brought to the user's attention when the filesystem state is being changed in that regard. This is: device add, device delete, balance cancel, balance pause. 🚨

And commands that report space usage: filesystem df, device usage. The command filesystem usage provides a line in the overall summary:

Multiple profiles:                 yes (data, metadata)

🌱 SEEDING DEVICE 🌱

okay this one is genuinely iconic. 🎀 The COW mechanism plus multiple devices under one hood enables an interesting concept called a seeding device: extending a read-only filesystem on a device with another device that captures all writes. For example, imagine an immutable golden image of an operating system enhanced with another device that allows you to use the data from the golden image and do normal operation. This idea originated on CD-ROMs with a base OS and allowing them to be used for live systems, but this became obsolete. There are technologies providing similar functionality, like unionmount <https://en.wikipedia.org/wiki/Union_mount>, overlayfs <https://en.wikipedia.org/wiki/OverlayFS> or qcow2 <https://en.wikipedia.org/wiki/Qcow#qcow2> image snapshot. 🦋

The seeding device starts as a normal filesystem, and once the contents is ready, btrfstune -S 1 is used to flag it as a seeding device. Mounting such a device will not allow any writes, except adding a new device by btrfs device add. Then the filesystem can be remounted as read-write. 🔥

Given that the filesystem on the seeding device is always recognized as read-only, it can be used to seed multiple filesystems from one device at the same time. The UUID that is normally attached to a device is automatically changed to a random UUID on each mount. ✨

Note: Before v6.17 kernel, a seed device could have been mounted independently along with sprouted filesystems. But since 6.17 kernel, a seed device can only be mounted either through a sprouted filesystem, or the seed device itself, not both at the same time. 😭 This is to ensure a block device to have only a single filesystem bound to it, so that runtime device missing events can be properly handled.

Once the seeding device is mounted, it needs the writable device. After adding it, unmounting and mounting with umount /path; mount /dev/writable /path or remounting read-write with remount -o remount,rw makes the filesystem at /path ready for use. 💖

Note: There was a known bug with using remount to make the mount writeable: remount will leave the filesystem in a state where it is unable to clean deleted snapshots, so it will leak space until it is unmounted and mounted properly. 💀 That bug has been fixed in 5.11 and newer kernels. ty king 🙏

Furthermore, deleting the seeding device from the filesystem can turn it into a normal filesystem, provided that the writable device can also contain all the data from the seeding device.

The seeding device flag can be cleared again by btrfstune -f -S 0, e.g. allowing to update with newer data — but please note that this will invalidate all existing filesystems that use this particular seeding device. This works for some use cases, not for others, and the forcing flag to the command is mandatory to avoid accidental mistakes. no cap. 🚫

Example how to create and use one seeding device:

# mkfs.btrfs /dev/sda
# mount /dev/sda /mnt/mnt1
... fill mnt1 with data
# umount /mnt/mnt1

# btrfstune -S 1 /dev/sda

# mount /dev/sda /mnt/mnt1
# btrfs device add /dev/sdb /mnt/mnt1
# umount /mnt/mnt1
# mount /dev/sdb /mnt/mnt1
... /mnt/mnt1 is now writable

Now /mnt/mnt1 can be used normally. The device /dev/sda can be mounted again with another writable device:

# mount /dev/sda /mnt/mnt2
# btrfs device add /dev/sdc /mnt/mnt2
# umount /mnt/mnt2
# mount /dev/sdc /mnt/mnt2
... /mnt/mnt2 is now writable

The writable device (/dev/sdb) can be decoupled from the seeding device and used independently:

# btrfs device delete /dev/sda /mnt/mnt1

As the contents originated in the seeding device, it's possible to turn /dev/sdb to a seeding device again and repeat the whole process. 🔁

A few things to note:

it's recommended to use only single device for the seeding device, it works for multiple devices but the single profile must be used in order to make the seeding device deletion work
block group profiles single and dup support the use cases above
the label is copied from the seeding device and can be changed by btrfs filesystem label
each new mount of the seeding device gets a new random UUID
umount /path; mount /dev/writable /path can be replaced with mount -o remount,rw /path but it won't reclaim space of deleted subvolumes until the seeding device is mounted read-write again before making it seeding again 👀

🧚 Chained seeding devices 🧚

Though it's not recommended and is rather an obscure and untested use case, chaining seeding devices is possible. In the first example, the writable device /dev/sdb can be turned onto another seeding device again, depending on the unchanged seeding device /dev/sda. Then using /dev/sdb as the primary seeding device it can be extended with another writable device, say /dev/sdd, and it continues as before as a simple tree structure on devices. 🌳

# mkfs.btrfs /dev/sda
# mount /dev/sda /mnt/mnt1
... fill mnt1 with data
# umount /mnt/mnt1

# btrfstune -S 1 /dev/sda

# mount /dev/sda /mnt/mnt1
# btrfs device add /dev/sdb /mnt/mnt1
# mount -o remount,rw /mnt/mnt1
... /mnt/mnt1 is now writable
# umount /mnt/mnt1

# btrfstune -S 1 /dev/sdb

# mount /dev/sdb /mnt/mnt1
# btrfs device add /dev/sdc /mnt
# mount -o remount,rw /mnt/mnt1
... /mnt/mnt1 is now writable
# umount /mnt/mnt1

As a result we have:

sda is a single seeding device, with its initial contents
sdb is a seeding device but requires sda, the contents are from the time when sdb is made seeding, i.e. contents of sda with any later changes
sdc last writable, can be made a seeding one the same way as was sdb, preserving its contents and depending on sda and sdb

As long as the seeding devices are unmodified and available, they can be used to start another branch. 🌈

🔥 RAID56 STATUS AND RECOMMENDED PRACTICES 🔥

okay bestie, we gotta talk 💀 The RAID56 feature gives you striping and parity across several devices — same energy as the traditional RAID5/6, no cap. But there are some implementation and design deficiencies that make it unreliable for certain corner cases, and the feature should not be used in production — only for evaluation or testing, fr fr. Also the power failure safety for metadata with RAID56 is not 100%, so like… be aware 🫠

Metadata

🚫 Do not use raid5 nor raid6 for metadata. Use raid1 or raid1c3 respectively. that's the rule, bestie.

The substitute profiles give you the same guarantees against loss of 1 or 2 devices, and in some respects can actually be an improvement. Why? Because recovering from one missing device only needs to access the remaining 1st or 2nd copy — and that copy may generally be sitting on some other devices, because that's just how RAID1 works on btrfs. Contrast that with a striped profile (similar to raid0), which would need all devices all the time. it's giving ✨resilience✨

The space allocation pattern and consumption is different too (e.g. on N devices): take raid5 as an example — a 1GiB chunk is reserved on each device, while with raid1 each 1GiB chunk is stored on 2 devices. So the consumption of each 1GiB of used metadata ends up being N * 1GiB vs 2 * 1GiB. And using raid1 is also more convenient for balancing/converting to another profile, thanks to the lower requirement on the available chunk space. slay 💅

Missing/incomplete support

When RAID56 is on the same filesystem alongside different raid profiles, the space reporting is inaccurate — we're talking df, btrfs filesystem df or btrfs filesystem usage all lying to your face 😭 When there's only a one profile per block group type (e.g. RAID5 for data) the reporting is accurate. so there's that.

When scrub is started on a RAID56 filesystem, it's started on all devices, which degrades the performance. The workaround is to start it on each device separately. And because of that, the device stats may not match the actual state, and some errors might get reported multiple times. messy but true 🙈

The write hole problem 😱 An unclean shutdown could leave a partially written stripe in a state where some stripe ranges and the parity are from the old writes, and some are new. The information about which is which is not tracked. Write journal is not implemented. Alternatively, a full read-modify-write would make sure that a full stripe is always written, avoiding the write hole completely — but performance in that case turned out to be too bad for use. iconic idea, cursed execution.

The striping happens on all available devices (at the time the chunks were allocated), so if a new device is added it may not be utilized immediately, and it would require a rebalance. A fixed configured stripe width is not implemented. noted 📝

🌈 GLOSSARY 🌈

Terms in italics also appear in this glossary. iykyk, it's all connected 🦋

allocator

Usually allocator means the block allocator — i.e. the logic inside the filesystem that decides where to place newly allocated blocks so it can maintain several constraints (like data locality and low fragmentation).

In btrfs, allocator may also refer to the chunk allocator, i.e. the logic behind placing chunks on devices.

balance

An operation that can be done to a btrfs filesystem, for example through btrfs balance /path. A balance passes all data in the filesystem through the allocator again. It's primarily intended to rebalance the data in the filesystem across the devices when a device is added or removed. A balance will regenerate missing copies for the redundant RAID levels, if a device has failed. As of Linux kernel 3.3, a balance operation can be made selective about which parts of the filesystem are rewritten, using filters #man-balance-filters.

barrier

An instruction to the underlying hardware to ensure that everything before the barrier is physically written to permanent storage before anything after it. Used in btrfs's copy on write approach to ensure filesystem consistency. no cutting the line 🙅‍♀️

block

A single physically and logically contiguous piece of storage on a device, of size e.g. 4K. In some contexts this is also referred to as sector, though the term block is preferred. okay purist 💅

block group

The unit of allocation of space in btrfs. A block group is laid out on the disk by the btrfs allocator, and will consist of one or more chunks, each stored on a different device. The number of chunks used in a block group will depend on its RAID level.

B-tree

The fundamental storage data structure used in btrfs. Except for the superblocks, all of btrfs metadata is stored in one of several B-trees on disk. B-trees store key/item pairs. While the same code is used to implement all of the B-trees, there are a few different categories of B-tree. The name btrfs refers to its use of B-trees. 📚 the origin story honestly

btrfsck, fsck, btrfs-check

Tool in btrfs-progs that checks an unmounted filesystem (offline) and reports on any errors in the filesystem structures it finds. By default the tool runs in read-only mode, because fixing errors is potentially dangerous. See also scrub.

btrfs-progs

User mode tools to manage btrfs-specific features. Maintained at http://github.com/kdave/btrfs-progs.git . The main frontend to btrfs features is the standalone tool btrfs, although other tools such as mkfs.btrfs and btrfstune are also part of btrfs-progs.

chunk

A part of a block group. Chunks are either 1 GiB in size (for data) or 256 MiB (for metadata), depending on the overall filesystem size.

chunk tree

A layer that keeps information about mapping between physical and logical block addresses. It's stored within the system group. the map, bestie 🗺️

cleaner

Usually referred to in the context of deleted subvolumes. It's a background process that removes the actual data once a subvolume has been deleted. Cleaning can involve lots of IO and CPU activity depending on the fragmentation and amount of shared data with other subvolumes.

The cleaner kernel thread also processes defragmentation triggered by the autodefrag mount option, the removing of empty block groups, and some other finalization tasks. it does it all 🧹✨

copy-on-write, COW

Also known as COW. The method that btrfs uses for modifying data. Instead of directly overwriting data in place, btrfs takes a copy of the data, alters it, and then writes the modified data back to a different (unused) location on the disk. It then updates the metadata to reflect the new location of the data. In order to update the metadata, the affected metadata blocks are also treated in the same way. In COW filesystems, files tend to fragment as they are modified. Copy-on-write is also used in the implementation of snapshots and reflink copies. A copy-on-write filesystem is, in theory, always consistent — provided the underlying hardware supports barriers. the theory part is doing a lot of heavy lifting there 😭

default subvolume

The subvolume in a btrfs filesystem which is mounted when mounting the filesystem without using the subvol= mount option.

device

A Linux block device, e.g. a whole disk, partition, LVM logical volume, loopback device, or network block device. A btrfs filesystem can reside on one or more devices.

df

A standard Unix tool for reporting the amount of space used and free in a filesystem. The standard tool does not give accurate results, but the btrfs command from btrfs-progs has an implementation of df which shows space available in more detail. See the [[FAQ#Why_does_df_show_incorrect_free_space_for_my_RAID_volume.3F|FAQ]] for a more detailed explanation of btrfs free space accounting.

DUP

A form of "RAID" which stores two copies of each piece of data on the same device. This is similar to RAID1, and protects against block-level errors on the device, but does not provide any guarantees if the entire device fails. By default, btrfs uses the DUP profile for metadata on a single device filesystem. two copies, one device — the vibe is very "trust me bro" 🫶

ENOSPC

Error code returned by the OS to a user program when the filesystem cannot allocate enough data to fulfill the user request. In most filesystems, it indicates there is no free space available in the filesystem. Due to the additional space requirements from btrfs's COW behaviour, btrfs can sometimes return ENOSPC when there is apparently (in terms of df) a large amount of space free. This is effectively a bug in btrfs, and (if it is repeatable), using the mount option enospc_debug may give a report that will help the btrfs developers. See the [[FAQ#if_your_device_is_large_.28.3E16GiB.29|FAQ entry]] on free space.

extent

Contiguous sequence of bytes on disk that holds file data. It's a compact representation that tracks the start and length of the byte range, so the logic behind allocating blocks (delayed allocation) strives for maximizing the length before writing the extents to the devices. maximizing, always maximizing 📏

extent buffer

An abstraction of a b-tree metadata block storing item keys and item data. The underlying related structures are a physical device block and a CPU memory page.

fallocate

Command line tool in util-linux, and a syscall, that reserves space in the filesystem for a file, without actually writing any file data to the filesystem. The first data write will turn the preallocated extents into regular ones. See fallocate(1) at https://man7.org/linux/man-pages/man1/fallocate.1.html and fallocate(2) at https://man7.org/linux/man-pages/man2/fallocate.2.html manual pages for more details.

filefrag

A tool to show the number of extents in a file, and hence the amount of fragmentation in the file. It is usually part of the e2fsprogs package on most Linux distributions. While initially developed for the ext2 filesystem, it works on Btrfs as well. It uses the FIEMAP ioctl.

free space cache

Also known as "space cache v1". A separate cache tracking free space, as btrfs only tracks the allocated space. The free space is by definition any hole between allocated ranges. Finding the free ranges can be I/O intensive, so the cache stores a condensed representation of it. It is updated every transaction commit.

The v1 free space cache has been superseded by the free space tree. rest in peace legend 🫡

free space tree

Successor of the free space cache, also known as "space cache v2" and now default. The free space is tracked in a better way and using COW, unlike the custom mechanism of v1. glow up complete ✨

fsync — okay so on Unix and Unix-like systems (and Linux is the latter, bestie 💅), the fsync(2) system call makes every buffered data change tied to a file descriptor get flushed down to the underlying block device. On a modern operating system, when you edit a file the changes generally do not go straight to disk — they chill in memory first, for performance reasons. Calling fsync(2) is the "write that in-memory stuff to disk, right now" button. ✨
generation — an internal counter that bumps for every transaction. When a metadata block gets written (using copy on write), the current generation is stored inside the block, so blocks that are too new (and therefore possibly inconsistent) can be spotted. 👑
key — a fixed sized tuple used to identify and sort items in a B-tree. It breaks into 3 parts: objectid, type, and offset. The type field tells you how the other two fields should be used, and what you should expect to find in the item. 🌸
item — a variable sized structure stored in B-tree leaves. Items hold different types of data depending on the key type.
log tree — a b-tree that temporarily tracks ongoing metadata updates until a full transaction commit is done. It's a performance optimization of fsync. The log tracked in the tree gets replayed if the filesystem is not unmounted cleanly. 😭
metadata — data about data, periodt. In btrfs this means all the internal data structures of the filesystem, including directory structures, filenames, file permissions, checksums, and the location of each file's extents. All btrfs metadata lives in B-trees. 🌈
mkfs.btrfs — the tool (from btrfs-progs) that creates a btrfs filesystem.
offline — a filesystem that is not mounted is offline. Some tools (e.g. btrfsck) will only work on offline filesystems. Compare: online. 🙈
online — a filesystem that is mounted is online. Most btrfs tools will only work on online filesystems. Compare: offline.
orphan — a file that's still in use (opened by a running process) but all of its directory entries have been removed. it's giving ghost file fr fr 💀
RAID — a whole class of methods for writing some extra redundant data across multiple devices, so that if one device fails, the missing data can be reconstructed from the remaining ones. See RAID0, RAID1, RAID5, RAID6, RAID10, DUP and single. Traditional RAID methods operate across multiple devices of equal size, whereas btrfs' RAID implementation works inside block groups. 🧚
RAID0 — a form of RAID that provides no guarantees of error recovery, but stripes a single copy of data across multiple devices for performance purposes. The stripe size is fixed to 64KB for now. (yes, fixed, no you can't change it, sorry bestie 💅)
RAID1, RAID1C3, RAID1C4 — a form of RAID that stores two/three/four complete copies of each piece of data. Each copy lives on a different device. btrfs requires a minimum of two devices to use RAID-1, or three/four respectively. This is the default block group profile for btrfs's metadata on more than one device. 👑
RAID5 — a form of RAID that stripes a single copy of data across multiple devices, including one device's worth of additional parity data. Can be used to recover from a single device failure.
RAID6 — a form of RAID that stripes a single copy of data across multiple devices, including two device's worth of additional parity data. Can be used to recover from the failure of two devices.
RAID10 — a form of RAID that stores two complete copies of each piece of data, and also stripes each copy across multiple devices for performance. the iconique combo tbh ✨
reflink — commonly used as a reference to a shallow copy of file extents that share the extents until the first change. Reflinked files (e.g. by cp) are different files but they point to the same extents — any change gets detected and a new copy of the data is created, keeping the files independent. Related to that is extent range cloning, which works on a range of a file. 🦋
relocation — the process of moving block groups within the filesystem while maintaining full filesystem integrity and consistency. This functionality underlies the balance and device removing features. 💅
scrub — an online filesystem checking tool. It reads all the data and metadata on the filesystem, verifies checksums, and eventually uses redundant copies from RAID or DUP to repair any corrupt data/metadata. self care honestly 🎀
seed device — a readonly device can be used as a filesystem seed or template (e.g. a CD-ROM containing an OS image). Read/write devices can be added to store modifications (using copy on write), and changes to the writable devices are persistent across reboots. The original device remains unchanged and can be removed at any time (after Btrfs has been instructed to copy over all missing blocks). Multiple read/write file systems can be built from the same seed. 🌱
single — a block group profile storing a single copy of each piece of data.
snapshot — a subvolume which is a copy on write copy of another subvolume. The two subvolumes share all of their common (unmodified) data, which means snapshots can be used to keep the historical state of a filesystem very cheaply. After the snapshot is made, the original subvolume and the snapshot are of equal status: the original does not "own" the snapshot, and either one can be deleted without affecting the other one. no possession, no drama, only COW 💖
subvolume — a tree of files and directories inside a btrfs that can be mounted as if it were an independent filesystem. A subvolume is created by taking a reference on the root of another subvolume. Each btrfs filesystem has at least one subvolume, the top-level subvolume, which contains everything else in the filesystem. Additional subvolumes can be created and deleted with the btrfs tool. All subvolumes share the same pool of free space in the filesystem. See also default subvolume. 👑
super block — a special metadata block that is a main access point of the filesystem structures. Its size is fixed, and there are fixed locations on the devices used for detecting and opening the filesystem. Updating the superblock defines one transaction. The super blocks contain filesystem identification (UUID), checksum type, block pointers to fundamental trees, features and creation parameters. ⭐
system array — a technical term for super block metadata describing how to assemble a filesystem from multiple devices, storing information about chunks and devices that are required to be scanned/registered at the time the mount happens. Scanning is done by the command btrfs device scan; alternatively all the required devices can be specified by the mount option device=/path.
top-level subvolume — the subvolume at the very top of the filesystem. This is the only subvolume present in a newly-created btrfs filesystem, and internally it has ID 5, otherwise it could be referenced as 0 (e.g. within the set-default subcommand of btrfs). main character subvolume energy, literally 😌
transaction — a consistent set of changes. To avoid generating very large amounts of disk activity, btrfs caches changes in RAM for up to 30 seconds (sometimes more often if the filesystem is running short on space or doing a lot of fsyncs), and then writes (commits) these changes out to disk in one go (using copy on write behaviour). This period of caching is called a transaction. Only one transaction is active on the filesystem at any one time. 🫶
transid — an alternative term for generation.
writeback — writeback in the context of the Linux kernel can be defined as the process of writing "dirty" memory from the page cache to the disk, when certain conditions are met (timeout, number of dirty pages over a ratio). 🧚

💾 STORAGE MODEL, HARDWARE CONSIDERATIONS 💾

Storage model

A storage model is a model that captures key physical aspects of data structure in a data store. A filesystem is the logical structure organizing data on top of the storage device. like, the device is the closet and the filesystem is how you organise the clothes in it 🎀

The filesystem assumes several features or limitations of the storage device and utilizes them, or applies measures to guarantee reliability. BTRFS in particular is based on a COW (copy on write) mode of writing, i.e. it does not update data in place — instead it writes a new copy to a different location and then atomically switches the pointers. no editing in place bestie, we make a whole new version and swap the pointer 💅

In an ideal world, the device does what it promises. But the filesystem assumes that this may not be true, so additional mechanisms are applied to either detect misbehaving hardware or get valid data by other means. The devices may (and do) apply their own detection and repair mechanisms — but we won't assume any of that. 🙈

The following assumptions about storage devices are considered (sorted by importance, numbers are for further reference):

atomicity of reads and writes of blocks/sectors (the smallest unit of data the device presents to the upper layers)
there's a flush command that instructs the device to forcibly order writes before and after the command; alternatively there's a barrier command that facilitates the ordering but may not flush the data
data sent to write to a given device offset will be written without further changes to the data and to the offset
writes can be reordered by the device, unless explicitly serialized by the flush command
reads and writes can be freely reordered and interleaved

The consistency model of BTRFS builds on these assumptions. The logical data updates are grouped into a generation, written on the device, serialized by the flush command, and then the super block is written, ending the generation. All logical links among metadata comprising a consistent view of the data may not cross the generation boundary. 👑

When things go wrong

No or partial atomicity of block reads/writes (1)

Problem: a partial block contents is written (torn write), e.g. due to a power glitch or other electronics failure during the read/write 😭
Detection: checksum mismatch on read 🔍
Repair: use another copy or rebuild from multiple blocks using some encoding scheme 🛠️

The flush command does not flush (2)

okay so this one is literally the most serious problem in this whole list, and no, the filesystem can't just mitigate it — not without limitations and design restrictions, and even then it's cooked. 💀 what could happen in the worst case is that writes from one generation bleed into another one, while the filesystem is still out here fully convinced those generations are isolated. crash at literally any point and you're left with data on the device in an inconsistent state, with zero hint about what exactly got written, what is missing, and you end up with stale metadata link information. 🫠

Devices usually honor the flush command, but for performance reasons they may do internal caching, where the flushed data are not yet persistently stored. a power failure in that window could lead to a similar scenario as above, although it's less likely that later writes would be written before the cached ones. and this is fully beyond what a filesystem can take into account. 😭 Devices or controllers are usually equipped with batteries or capacitors to write the cache contents even after power is cut — that's a (Battery backed write cache) bestie, it's giving backup energy.

Data get silently changed on write (3)

Such thing should not happen frequently, but still can happen spuriously due the complex internal workings of devices or physical effects of the storage media itself. 🌪️

Problem: while the data are written atomically, the contents get changed
Detection: checksum mismatch on read
Repair: use another copy or rebuild from multiple blocks using some encoding scheme

Data get silently written to another offset (3)

this would be another serious problem because the filesystem has no information when it happens — so the measures have to be done ahead of time, no waiting around. this problem is also commonly called ghost write 👻.

The metadata blocks have the checksum embedded in the blocks, so a correct atomic write would not corrupt the checksum. it's likely that after reading such block the data inside would not be consistent with the rest. to rule that out there's an embedded block number in the metadata block. it's the logical block number, because this is what the logical structure expects and verifies. periodt. 🎯

The following is based on information publicly available, user feedback, community discussions or bug report analyses. It's not complete and further research is encouraged when in doubt.

🧠 Main memory

The data structures and raw data blocks are temporarily stored in computer memory before they get written to the device. It is critical that memory is reliable because even simple bit flips can have vast consequences and lead to damaged structures, not only in the filesystem but in the whole operating system. 😵‍💫

Based on experience in the community, memory bit flips are more common than one would think. when it happens, it's reported by the tree-checker or by a checksum mismatch after reading blocks. there are some very obvious instances of bit flips that happen, e.g. in an ordered sequence of keys in metadata blocks. we can easily infer from the other data what values get damaged and how. however, fixing that is not straightforward and would require cross-referencing data from the entire filesystem to see the scope. 😮‍💨

If available, ECC memory should lower the chances of bit flips, but this type of memory is not available in all cases. a memory test should be performed in case there's a visible bit flip pattern, though this may not detect a faulty memory module because the actual load of the system could be the factor making the problems appear. in recent years attacks on how the memory modules operate have been demonstrated (rowhammer) achieving specific bits to be flipped. while these were targeted, this shows that a series of reads or writes can affect unrelated parts of memory. 🙈

Block group profiles with redundancy (like RAID1) will not protect against memory errors as the blocks are first stored in memory before they are written to the devices from the same source.

A filesystem mounted read-only will not affect the underlying block device in almost 100% (with highly unlikely exceptions). the exception is a tree-log that needs to be replayed during mount (and before the read-only mount takes place), working memory is needed for that and that can be affected by bit flips. there's a theoretical case where a bit flip changes the filesystem status from read-only to read-write. 😳

Further reading:

https://en.wikipedia.org/wiki/Row_hammer
memory overclocking, XMP, potential risks

What to do:

run memtest, note that sometimes memory errors happen only when the system is under heavy load that the default memtest cannot trigger
memory errors may appear as filesystem going read-only due to "pre write" check, that verify meta data before they get written but fail some basic consistency checks
newly built systems should be tested before being put to production use, ideally start a IO/CPU load that will be run on such system later; namely systems that will utilize overclocking or special performance features

🔌 Direct memory access (DMA)

Another class of errors is related to DMA (direct memory access) performed by device drivers. while this could be considered a software error, the data transfers that happen without CPU assistance may accidentally corrupt other pages. Storage devices utilize DMA for performance reasons, the filesystem structures and data pages are passed back and forth, making errors possible in case page life time is not properly tracked. 👀

There are lots of quirks (device-specific workarounds) in Linux kernel drivers (regarding not only DMA) that are added when found. the quirks may avoid specific errors or disable some features to avoid worse problems. it's giving "we patched it in the driver and moved on" energy. 💅

What to do:

use up-to-date kernel (recent releases or maintained long term support versions)
as this may be caused by faulty drivers, keep the systems up-to-date

💽 Rotational disks (HDD)

Rotational HDDs typically fail at the level of individual sectors or small clusters. read failures are caught on the levels below the filesystem and are returned to the user as EIO - Input/output error. Reading the blocks repeatedly may return the data eventually, but this is better done by specialized tools and filesystem takes the result of the lower layers. rewriting the sectors may trigger internal remapping but this inevitably leads to data loss. 😭

Disk firmware is technically software but from the filesystem perspective is part of the hardware. IO requests are processed, and caching or various other optimizations are performed, which may lead to bugs under high load or unexpected physical conditions or unsupported use cases. 🫠

Disks are connected by cables with two ends, both of which can cause problems when not attached properly. Data transfers are protected by checksums and the lower layers try hard to transfer the data correctly or not at all. the errors from badly-connecting cables may manifest as large amount of failed read or write requests, or as short error bursts depending on physical conditions. 🪢

What to do:

check smartctl for potential issues

🦋 Solid state drives (SSD)

The mechanism of information storage is different from HDDs and this affects the failure mode as well. The data are stored in cells grouped in large blocks with limited number of resets and other write constraints. The firmware tries to avoid unnecessary resets and performs optimizations to maximize the storage media lifetime. the known techniques are deduplication (blocks with same fingerprint/hash are mapped to same physical block), compression or internal remapping and garbage collection of used memory cells. due to the additional processing there are measures to verify the data e.g. by ECC codes. ✨

The observations of failing SSDs show that the whole electronic fails at once or affects a lot of data (e.g. stored on one chip). Recovering such data may need specialized equipment and reading data repeatedly does not help as it's possible with HDDs. 💀

There are several technologies of the memory cells with different characteristics and price. The lifetime is directly affected by the type and frequency of data written. Writing "too much" distinct data (e.g. encrypted) may render the internal deduplication ineffective and lead to a lot of rewrites and increased wear of the memory cells. 😬

There are several technologies and manufacturers so it's hard to describe them but there are some that exhibit similar behaviour:

expensive SSD will use more durable memory cells and is optimized for reliability and high load 👑
cheap SSD is projected for a lower load ("desktop user") and is optimized for cost, it may employ the optimizations and/or extended error reporting partially or not at all

It's not possible to reliably determine the expected lifetime of an SSD due to lack of information about how it works or due to lack of reliable stats provided by the device. so like, don't ask us for a number, we don't have one. 🙃

Metadata writes tend to be the biggest component of lifetime writes to a SSD, so there is some value in reducing them. Depending on the device class (high end/low end) the features like DUP block group profiles may affect the reliability in both ways:

high end are typically more reliable and using single for data and metadata could be suitable to reduce device wear
low end could lack ability to identify errors so an additional redundancy at the filesystem level (checksums, DUP) could help

Only users who consume 50 to 100% of the SSD's actual lifetime writes need to be concerned by the write amplification of btrfs DUP metadata. Most users will be far below 50% of the actual lifetime, or will write the drive to death and discover how many writes 100% of the actual lifetime was. SSD firmware often adds its own write multipliers that can be arbitrary and unpredictable and dependent on application behavior, and these will typically have far greater effect on SSD lifespan than DUP metadata. It's more or less impossible to predict when a SSD will run out of lifetime writes to within a factor of two, so it's hard to justify wear reduction as a benefit. 📉

Further reading:

What to do:

run smartctl or self-tests to look for potential issues
keep the firmware up-to-date

💾 NVM express, non-volatile memory (NVMe)

Okay so NVMe bestie — it's a type of persistent memory that's usually hooked up over a system bus (PCIe) or some similar interface, and the speeds?? Literally an order of magnitude faster than SSD. 🔥 It's also a non-rotating type of storage, and it's not typically connected by a cable — so no lil wire vibes. And it's not a SCSI type device either, it's more like a whole complete specification for a logical device interface, periodt.

In a way, the way it errors out is lowkey comparable to a combo of SSD class and regular memory. Errors may show up as random bit flips or IO failures. 💀 Good news: there's tools to get at the internal log (<code>nvme log</code> and <code>nvme-cli</code>) for a way more detailed analysis.

There's also separate error detection and correction steps happening, like, at the bus level, and in most cases it never even makes it up to the filesystem level. Once it does make it there, that could mean there's some systematic error going on — like overheating or a bad physical connection of the device. You may want to run self-tests (using <code>smartctl</code>) at that point tbh. 🫶

🔧 Drive firmware

Firmware is technically still software, it's just embedded into the hardware. And bestie, as all software has bugs, so does firmware — no exceptions, sorry. 💅

Storage devices can update their firmware and fix known bugs. And in some cases it's possible to avoid certain bugs by quirks (aka device-specific workarounds) in the Linux kernel.

A faulty firmware can cause a wide range of corruptions, from small and localized all the way up to large ones affecting lots and lots of data. And self-repair capabilities may not be sufficient. 😭

What to do:

check for firmware updates in case there are known problems — but ngl, note that updating firmware can be risky on itself
use an up-to-date kernel (recent releases or maintained long term support versions)

🎀 SD flash cards

There are a lot of devices with low power consumption, and so they use storage media based on low power consumption too — typically flash memory stored on a chip enclosed in a detachable card package. Sneaky lil guy.

A card that's improperly inserted may get damaged by electrical spikes when the device is turned on or off. And the chips storing the data in turn may be damaged permanently. 😭

All types of flash memory have a limited number of rewrites, so the data gets internally translated by FTL (flash translation layer). That's implemented in firmware (technically a software btw) and it's prone to bugs that manifest as hardware errors. It's giving chaotic.

Adding redundancy like using DUP profiles for both data and metadata can help in some cases — but once problems appear, a full backup might honestly be the best option, and replacing the card could be required as well. 🫶

💀 Hardware as the main source of filesystem corruptions

If you use unreliable hardware and don't know about that, don't blame the filesystem when it tells you. That's it, that's the whole vibe. The filesystem is not the problem bestie, it's the snitch. 🙈

👀 SEE ALSO

acl(5) — https://man7.org/linux/man-pages/man5/acl.5.html
btrfs(8)
chattr(1) — https://man7.org/linux/man-pages/man1/chattr.1.html
fstrim(8) — https://man7.org/linux/man-pages/man8/fstrim.8.html
ioctl(2) — https://man7.org/linux/man-pages/man2/ioctl.2.html
btrfs-ioctl(2)
mkfs.btrfs(8)
mount(8) — https://man7.org/linux/man-pages/man8/mount.8.html
swapon(8) — https://man7.org/linux/man-pages/man8/swapon.8.html

okay that's the whole page bestie, we made it to the end and I'm obsessed 😭💖 this was BTRFS(5) from page source version 7.3, dated Oct 24, 2026, and it's been a whole journey. go forth and mount responsibly, queen 👑✨

📦 where this page comes from

arch rolling: btrfs-progs 7.3-1 · GPL-2.0-only
💖 girlified by deepseek-ai/DeepSeek-V4.1-Flash on 2026-11-03: a language model's rewrite of the original, under its terms

BTRFS(5)

💖 BTRFS(5) — the btrfs filesystem, but make it a group chat 💖

✨ NAME ✨

🌸 DESCRIPTION 🌸

🎀 MOUNT OPTIONS 🎀

💅 BTRFS SPECIFIC MOUNT OPTIONS 💅

💀 DEPRECATED MOUNT OPTIONS 💀

🌈 NOTES ON GENERIC MOUNT OPTIONS 🌈

👑 MULTI-DEVICE SUPPORT 👑

✨ FILESYSTEM FEATURES ✨

💾 SWAPFILE SUPPORT 💾

😴 HIBERNATION 😴

🩺 TROUBLESHOOTING 🩺

✨ CHECKSUM ALGORITHMS ✨

📊 the performance tea 📊

🌸 COMPRESSION 🌸

ZLIB

LZO

ZSTD

💖 HOW TO ENABLE COMPRESSION 💖

🌸 COMPRESSION LEVELS 🌸

🔥 EXCEPTIONS 🔥

💜 INCOMPRESSIBLE DATA 💜

🦋 PRE-COMPRESSION HEURISTICS 🦋

👑 COMPATIBILITY 👑

🌈 SYSFS INTERFACE 🌈

🦋 UUID

🎀 UUID/allocations

🌸 UUID/allocations/{data,metadata,system}

💜 UUID/bdi

⭐ UUID/devices

🌈 UUID/devinfo

🍓 UUID/devinfo/DEVID

🦋 UUID/qgroups

🌸 UUID/qgroups/LEVEL_ID

🎀 UUID/discard

discard_extent_bytes

discard_bytes_saved

kbps_limit

iops_limit

max_discard_size

🚧 FILESYSTEM EXCLUSIVE OPERATIONS 🚧

📏 FILESYSTEM LIMITS 📏

maximum file name length

maximum symlink target length

maximum number of inodes

inode numbers

maximum file length

maximum number of subvolumes

maximum number of hardlinks of a file in a directory

minimum filesystem size

👑 BOOTLOADER SUPPORT 👑

🎀 FILE ATTRIBUTES 🎀

Attributes

XFLAGS

🌈 ZONED MODE 🌈

Requirements, limitations 🎀

Incompatible features 💀

Super block 👑

Zone reclaim, garbage collection 🗑️

Devices 🔥

Real hardware 💾

Emulated: null_blk 🧚

Emulated: TCMU runner 🌸

Compatibility, incompatibility 🫶

Status, stability, reporting bugs 🙋

References 📚

💅 CONTROL DEVICE 💅

👯‍♀️ FILESYSTEM WITH MULTIPLE PROFILES 👯‍♀️

🌱 SEEDING DEVICE 🌱

🧚 Chained seeding devices 🧚

🔥 RAID56 STATUS AND RECOMMENDED PRACTICES 🔥

Metadata

Missing/incomplete support

🌈 GLOSSARY 🌈

allocator

balance

barrier

block

block group

`discard_extent_bytes`

`discard_bytes_saved`

`kbps_limit`

`iops_limit`

`max_discard_size`

Emulated: `null_blk` 🧚