MKFS.LUSTRE(8)

📦 version 2.17.58, in ubuntu stonking

✨ MKFS.LUSTRE ✨

okay so bestie, welcome to mkfs.lustre from the Lustre Configuration Utilities corner of the universe 💖 this is a manual name situation, so consider yourself formally introduced to the eighth section of the manual, Lustre edition.

🌸 NAME 🌸

mkfs.lustre — format a disk for a Lustre service. that's it, that's the whole job description 💅

🎀 SYNOPSIS 🎀

mkfs.lustre {--ost|--mdt|--mgs} --fsname=NAME [OPTIONS] DEVICE
mkfs.lustre {--ost|--mdt|--mgs} --backfstype=zfs --fsname=NAME [OPTIONS]
            POOL_NAME/DATASET_NAME [ZPOOL_SPECIFICATION]

🦋 DESCRIPTION 🦋

so mkfs.lustre is literally the glow-up station for a disk device 💖 you run it to format a device so it can become part of a Lustre filesystem. once you've formatted it, you mount it with mount -t lustre ... and the Lustre service that this command defined just... starts. main character energy fr fr 🔥

👑 OPTIONS 👑

  • --ost — object storage target. the OST era 💫
  • --mdt — metadata storage target. the brain cell of the operation 🧠
  • --mgs — configuration management service. there's exactly one per site or filesystem. fun fact: you can combine it with one --mdt service by specifying both types at once. two jobs, one icon ✨
  • --backfstype=FSTYPE — force a particular format for the backing fs. the options are ldiskfs and zfs. pick your fighter 💅
  • ZPOOL_SPECIFICATION — specify the zpool specifications, formatted like this:
[[VDEV_TYPE] DEVICE [DEVICE ...] [VDEV_TYPE] ...]
  • VDEV_TYPE — can be one of the following: {mirror,raidz,raidz2,raidz3,cache}
  • DEVICE — the linux block device.

if no vdev_type is given, the devices are used in a round-robin (striped) manner. see zpool(8) for more details. iykyk 🌀

  • --comment=COMMENT — set a user comment about this disk. Lustre itself ignores it, so it's just for you, like a little sticky note 🍓
  • --device-size=KB — set the device size for loop devices.
  • --dryrun — only print what would be done; does not affect the disk. a vibe check before you commit, no cap 🫶
  • --failnode=NID,... — set the NID(s) of a failover partner. repeat this option as many times as you want. ⚠️ cannot be used with --servicenode.
  • --servicenode=NID,... — set the NID(s) of all service partner. this option treats all nodes as equal service nodes. ⚠️ cannot be used with --failnode.
  • --fsname=FILESYSTEM_NAME — the Lustre filesystem this target will be part of. valid filesystem_names are between 1 and 8 characters long and must only use upper- and lower-case English letters, numbers, and '-' or '_' (regexp [-_a-zA-Z0-9]). all targets in a single filesystem must specify the same filesystem_name, and it has to be unique between all filesystems mounted by a single client at one time. using "lustre" as the filesystem name is discouraged — it's asking for future problems if a client needs to mount two such filesystems, and it just adds confusion for admins and users between "lustre" as the filesystem_name and "lustre" as the literal string you have to use in places for the filesystem_type. also, the --fsname option is not valid for the MGS, since the MGS may be used for multiple filesystems. ngl that makes sense 💀
  • --index=INDEX — specify a particular OST or MDT index. required for all targets other than the MGS, and it must be unique for all targets in the same filesystem. typically you use sequential values starting from 0. the index parameter may either be a decimal number, or a hexadecimal number starting with '0x'.
  • --mkfsoptions=OPTS — additional formatting options passed straight through to the backing filesystem. for example, options for mke2fs or zpool could be set here. ✨
  • --mountfsoptions=OPTS — set persistent mount options that will be used when mounting Lustre targets.
⚠️ WARNING: unlike earlier versions of mkfs.lustre, this version completely replaces the default mount options with the ones you put on the command line, and it throws a warning on stderr if any of the default mount options are omitted. ⚠️

the defaults for LDISKFS are:

OST: — errors=remount-ro,mballoc,extents

MGS/MDT: — errors=remount-ro,user_xattr

DO NOT alter the default mount options unless you know what you are doing. like seriously, bestie, do not 🙈

  • --backfs-mount-opts=OPTS — use these options for mounting the backing fs while mkfs.lustre is working.
  • --network=NET,... — network(s) to restrict this OST/MDT to. repeat this option as many times as you want 🌐
  • --skipmmp — disable the Multi-Mount Protection (MMP) feature when failover nodes are configured. by default, MMP is automatically enabled when --failnode or --servicenode is specified, so that simultaneous mounts can't happen. use this option only if you have alternative mount protection mechanisms in place — improper use may lead to filesystem corruption. this one is dangerous, handle with care 💀
  • --mgsnode=NID,... — set the NID(s) of the MGS node. required for all targets other than the MGS.
  • --param KEY=VALUE — set permanent parameter KEY to value VALUE. repeat this option as many times as you want. typical options might include:
    • --param sys.timeout=40 — System obd timeout
    • --param lov.stripesize=2M — Default stripe size
    • --param lov.stripecount=2 — Default stripe count
    • --param failover.mode=failout — Return errors instead of waiting for recovery
  • --quiet — print less information. 🤫
  • --reformat — reformat an existing Lustre disk as a new target.
  • --replace — used to initialize a target with the same --index as a previously used target, if the old target was permanently lost for some reason (e.g. multiple disk failure or massive corruption). this avoids having the target try to register as a new target with the MGS. the rescue arc 🫶
  • --stripe-count-hint=STRIPES — specify the expected common number of stripes on a file, so the MDT inode size can be optimized for the typical use case.
  • --force-nohostid — ignore unset hostid for ZFS import protection. to set hostid, either set the spl_hostid parameter for spl.ko or set /etc/hostid, see zgenhostid(8). to populate the spl_hostid parameter, spl.ko must be (re)loaded after /etc/hostid is created. order matters, bestie 🎀
  • --verbose — print more information. the opposite of quiet, we love a chatty king 🗣️
  • -V, --version — output build version of the mkfs.lustre utility.

💜 NID 💜

a Lustre network identifier (NID) is used to uniquely identify a Lustre network endpoint by node ID and network type. the format of the NID is:

network_id@network_type

if a node has multiple network interfaces, it may have multiple NIDs, and all of them must be identified so other nodes can choose the NID that's appropriate for their network interfaces. typically, NIDs are specified in a list delimited by commas (,). however, when failover nodes are specified, the NIDs are delimited by a colon (:) or by repeating a keyword such as --mgsnode= or --servicenode=. 🤝

🔥 EXAMPLES 🔥

combined MGS and MDT for filesystem 'testfs' on node e.g. cfs21:

# mkfs.lustre --fsname=testfs --index=0 --mdt --mgs /dev/sda1

OST for filesystem 'testfs' on any node using the above MGS:

# mkfs.lustre --fsname=testfs --index=0 --ost --mgsnode=cfs21@tcp0 /dev/sdb

combined MGS and MDT for filesystem 'testfs' on failover pair cfs21 and cfs22:

# mkfs.lustre --fsname=testfs --index=0 --mdt --mgs --servicenode=cfs21@tcp0,cfs21ib@o2ib0 --servicenode=cfs22@tcp0,cfs22ib@o2ib0 /dev/sda1

OST for filesystem 'testfs' using the above MGS and having a failover partner cfs24:

# mkfs.lustre --fsname=testfs --index=1 --ost --mgsnode=cfs21@tcp0,cfs21ib@o2ib0:cfs22@tcp0,cfs22ib@o2ib0 --failnode=cfs24@tcp0,cfs24ib@o2ib0 /dev/sdb

standalone MGS on e.g. node cfs22:

# mkfs.lustre --mgs /dev/sda1

MDT for filesystem 'myfs1' on any node, using the above MGS:

# mkfs.lustre --fsname=myfs1 --index=0 --mdt --mgsnode=cfs22@tcp0 /dev/sda2

create zfs pool 'zfspool' on two root vdevs, each a mirror of two disks, and create mdt/mgs on filesystem 'zfspool/mdt1':

# mkfs.lustre --fsname=testfs --index=0 --mdt --mgs zfspool/mdt1 mirror /dev/sdb /dev/sdc mirror /dev/sdd /dev/sde

output build version of the mkfs.lustre utiltiy e.g. 2.15.52_175_ge7aa83d:

# mkfs.lustre --version

🌈 AVAILABILITY 🌈

mkfs.lustre is part of the lustre(7) filesystem package since release 1.4.0. been iconic since day one 👑

⭐ SEE ALSO ⭐

lfs(1), lustre(7), lctl(8), mke2fs(8), mount.lustre(8), tunefs.lustre(8), zpool(8)

and that's the whole page, bestie 💖 this gem was served up by Lustre, dated 2024-08-28, and it lives at the very bottom of the manual like a little signature. go forth and format responsibly, fr fr 🫶✨💅

📦 where this page comes from