gfs2(5)

📦
version 3.6.1 · 2 versions
, in debian trixie, fedora 43, fedora 44, nixpkgs unstable, ubuntu questing, ubuntu resolute, ubuntu stonking

gfs2 💖

okay bestie, welcome to the GFS2 reference guide — that's the gfs2(5) page, we're in the File Formats Manual, and this is your group chat briefing on the whole clustered-filesystem situation ✨

🦋 SYNOPSIS 🦋

A whole entire overview of the GFS2 filesystem, no cap. That's it, that's the synopsis 💅

✨ DESCRIPTION ✨

So GFS2 is a clustered filesystem — meaning it's literally designed for sharing data between multiple nodes that are all plugged into one common shared storage device 🫶 like, everyone's holding hands around the same disk, iconic behavior.

It can also be used as a local filesystem on a single node, but hear me out — because the whole design is aimed at clusters, doing that will usually result in lower performance than just using a filesystem designed specifically for single-node use. So like, technically valid, lowkey not the vibe. 🙈

GFS2 is a journaling filesystem, and one journal is required for each node that will mount the filesystem. Periodt, one journal per mounting node, no exceptions… well, almost 👀

The one exception? Spectator mounts. These are equivalent to mounting a read-only block device, and as such they can neither recover a journal nor write to the filesystem — so they do not require a journal assigned to them. Slay, a loophole 💅

🌸 The GFS2 docs are split into a bunch of sections, bestie 🌸

ok real talk, that last one says use with caution and I am not being dramatic, that's just what it says, take the hint 💀

more parts incoming, don't go anywhere bestie 🎀

💖 MOUNT OPTIONS 💖

okay bestie, this is the main event, the part everyone actually opens the man page for. every single one of these goes in your mount command or your /etc/fstab and they all hit different. let's get into it ✨

  • lockproto=LockProtoName — this one picks which inter-node lock protocol the GFS2 filesystem uses for this mount, and it straight-up overrides the default lock protocol name sitting in the filesystem's on-disk superblock.

LockProtoName has to be one of the supported locking protocols, and rn those are lock_nolock and lock_dlm. that's it, that's the list.

the default lock protocol name gets written to disk when you first create the filesystem with mkfs.gfs2(8), -p option. you can change it on-disk later with the tunegfs2(8) command.

the lockproto mount option is for special circumstances only, like when you wanna temporarily run a different lock protocol without touching the on-disk default. and fr fr, using the wrong lock protocol on a cluster filesystem mounted from more than one node will almost certainly result in filesystem corruption. do not play with this. 💀
  • locktable=LockTableName — this sets the identity of the cluster and the filesystem for this mount, overriding the default cluster/filesystem identity stored in the on-disk superblock. that cluster/filesystem name is recognized globally across the whole cluster and it establishes a unique namespace for the inter-node locking system, which is what lets you mount multiple GFS2 filesystems. slay.

the format of LockTableName is lock-module-specific. for lock_dlm, the format is clustername:fsname. for lock_nolock, the field is just ignored, no thoughts, head empty.

the default cluster/filesystem name gets written to disk when you first create the filesystem with mkfs.gfs2(8), -t option, and you can change it on-disk with tunegfs2(8).

same energy as lockproto: only use the locktable mount option in special circumstances, like mounting the filesystem in a different cluster or under a different filesystem name without changing the on-disk default.
  • localflocks — this flag tells GFS2 "bestie you're a local (not clustered) filesystem now" so it can let the kernel VFS layer handle all the flock and fcntl file locking. when you're running in cluster mode, those file locks need inter-node locks and they need GFS2's support. when you're running locally, you get better performance by letting VFS do the whole job. iconic delegation tbh.

this one gets turned on automatically by the lock_nolock module. she does it herself. 💅

  • errors=[panic|withdraw] — setting errors=panic makes GFS2 oops when it hits an error that would otherwise make the mount withdraw or print an assertion warning. the default setting is errors=withdraw.
do not use this on a production system. it replaces the earlier debug option on kernel versions 2.6.31 and above.
  • acl — enables POSIX Access Control List acl(5) support inside GFS2. 🎀
  • spectator — mounts this filesystem using a special form of read-only mount. the mount does not use one of the filesystem's journals, and the node is unable to recover journals for other nodes. so like, you're just watching, not participating. main character behavior is off the table.
  • norecovery — a synonym for spectator. same vibe, different name. 🦋
  • suiddir — sets the owner of any newly created file or directory to be that of the parent directory, if the parent directory has the S_ISUID permission attribute bit set. it sets S_ISUID in any new directory, if its parent directory's S_ISUID is set. it strips all execution bits on a new file, if the parent directory owner is different from the owner of the process creating the file.
only set this option if you actually know why you're setting it. no vibes-based filesystem config, please. 😭
  • quota=[off/account/quiet/on] — turns quotas on or off for a filesystem. putting quotas in the account state makes the per UID/GID usage statistics get correctly maintained by the filesystem, and the limit and warn values are ignored. the quiet state is equivalent to on, except it doesn't write warnings about soft and hard quota limits to the system log. the default value is off.
  • discard — makes GFS2 generate discard I/O requests for blocks that have been freed. suitable hardware can use those to implement thin-provisioning and similar schemes. supported in kernel version 2.6.30 and above. ✨
  • barrier — this option, which defaults to on, makes GFS2 send I/O barriers when flushing the journal. it gets automatically turned off if the underlying device doesn't support I/O barriers.
we highly recommend using I/O barriers with GFS2 at all times, unless the block device is designed so it cannot lose its write cache content (e.g. it's on a UPS, or it doesn't have a write cache).
  • commit=secs — similar to the ext3 commit= option, in that it sets the maximum number of seconds between journal commits if there's dirty data in the journal. the default is 60 seconds. this option is only provided in kernel versions 2.6.31 and above.
  • data=[ordered|writeback] — when data=ordered is set, the user data modified by a transaction gets flushed to disk before the transaction is committed to disk. this should prevent you from seeing uninitialized blocks in a file after a crash. data=writeback mode writes the user data to disk at any time after it's dirtied — it doesn't give you the same consistency guarantee as ordered mode, but it should be slightly faster for some workloads. the default is ordered mode.
  • meta — this option selects the meta filesystem root instead of the normal filesystem root. it's normally only used by the GFS2 utility functions.
altering any file on the GFS2 meta filesystem may render the filesystem unusable, so only experts in the GFS2 on-disk layout should use this option. 🔥
  • quota_quantum=secs — sets the number of seconds a change in the quota information can sit on one node before being written to the quota file. this is the preferred way to set this parameter. the value is an integer number of seconds greater than zero, and the default is 60 seconds. shorter settings mean faster updates of the lazy quota information and less likelihood of someone exceeding their quota. longer settings make filesystem operations involving quotas faster and more efficient.
  • statfs_quantum=secs — setting statfs_quantum to 0 is the preferred way to set the slow version of statfs. the default value is 30 secs, which sets the maximum time period before statfs changes get synced to the master statfs file. you can adjust this to allow faster but less accurate statfs values, or slower but more accurate ones. when it's set to 0, statfs always reports the true values.
  • statfs_percent=value — this gives you a bound on the maximum percentage change in the statfs information on a local basis before it gets synced back to the master statfs file, even if the time period hasn't expired. if statfs_quantum is set to 0, then this setting is ignored.
  • rgrplvb — this flag tells gfs2 to look for information about a resource group's free space and unlinked inodes in its glock lock value block. that keeps gfs2 from having to read the resource group data in from disk, which speeds up allocations in some cases. this option was added in the 3.6 Linux kernel. before that kernel, no information was saved to the resource group lvb.
note: to safely turn this on, all nodes mounting the filesystem must be running at least a 3.6 Linux kernel. if any nodes previously mounted the filesystem using older kernels, the filesystem must be unmounted on all nodes before it can be mounted with this option enabled. this option doesn't need to be enabled on all nodes using a filesystem.
  • loccookie — this flag tells gfs2 to use location based readdir cookies instead of its usual filename hash readdir cookies. the filename hash cookies aren't guaranteed to be unique, and as the number of files in a directory increases, so does the likelihood of a collision. NFS requires readdir cookies to be unique, which can cause problems with very large directories (over 100,000 files). with this flag set, gfs2 will try to give out location based cookies. since the cookie is 31 bits, gfs2 will eventually run out of unique cookies and will fail back to using hash cookies. the maximum number of files that could have unique location cookies — assuming perfectly even hashing and names of 8 or fewer characters — is 1,073,741,824. an average directory should be able to give out well over half a billion location based cookies. this option was added in the 4.5 Linux kernel. before that kernel, gfs2 didn't add directory entries in a way that allowed it to use location based readdir cookies.
note: to safely turn this on, all nodes mounting the filesystem must be running at least a 4.5 Linux kernel. if this option is only enabled on some of the nodes mounting a filesystem, the cookies returned by nodes using this option won't be valid on nodes that aren't using it, and vice versa. and finally, when you first enable this option on a filesystem that had been previously mounted without it, you must make sure there are no outstanding cookies being cached by other software, such as NFS. 🙈

that's the whole mount options buffet bestie, every single flag accounted for, no skips, no cuts 💜

🛠️ SETUP 🛠️

Okay bestie, here's the tea ☕ — GFS2 clustering does not run itself. It's driven by the dlm, and the dlm depends on dlm_controld to bring clustering in from userspace. And dlm_controld's clustering? Built on corosync cluster/group membership and messaging. It's a whole ecosystem, fr fr. 🦋

GFS2 also needs clustered lvm, which comes from lvmlockd or, back in the day, clvmd. So go read the docs for every single one of these components and make sure they're configured before you even think about setting up a GFS2 filesystem. Also check your distribution's documentation for any special support requirements it has, because distros love having ✨opinions✨.

💖 Make sure gfs2-utils is installed on every node that mounts the filesystem — it ships the scripts needed for correct withdraw event response. Skip it and you're on your own, no cap.

1️⃣ Create the gfs2 filesystem

mkfs.gfs2 -p lock_dlm -t cluster_name:fs_name -j num /path/to/storage

Breaking it down for you, queen 👑:

  • The cluster_name has to match the name configured in corosync (and therefore in dlm). No matching, no vibes.
  • The fs_name must be a unique name for the filesystem inside the cluster.
  • The -j option is the number of journals to create — and there must be one for each node that will mount the filesystem. Count your nodes, bestie. Count them twice. 🔥

2️⃣ Mount the gfs2 filesystem

If you're using a clustered resource manager, go read its documentation for how to enable a gfs2 filesystem resource. If not, just run:

mount /path/to/storage /mountpoint

Then run dlm_tool ls to verify the nodes that have each fs mounted. Vibe check your cluster before you trust it 💅

3️⃣ Shut down

Using a clustered resource manager? Again, its documentation is the one holding the instructions for disabling a gfs2 filesystem resource. Otherwise, run:

umount -a -t gfs2

And that's a wrap, we love a clean exit 😌

🌈 SEE ALSO 🌈

And that's the whole page, bestie 🫶 this was the gfs2(5) man page from the gfs2-utils package, dated back in the day — go mount something iconic and stay clustered. periodt. 💜✨

📦 where this page comes from