LFS-SETDIRSTRIPE(1)

📦 version 2.17.58, in ubuntu stonking

💖 LFS-SETDIRSTRIPE(1) 💖

okay bestie, you found the Lustre Utilities corner of the manual, and specifically the lfs setdirstripe page. Buckle up, this one's a bop. 🦋

✨ NAME ✨

lfs setdirstripe, mkdir — set striping pattern of a directory.

🌸 SYNOPSIS 🌸

lfs setdirstripe [-cCdDhHioTxX] DIR...

💅 DESCRIPTION 💅

So like, this makes a striped directory with the striping pattern you asked for. She's basically lfs setstripe's cousin, but instead of files she's out here creating striped dirs, or setting the default layout that subdirectories will pick up. 💖

She can also make a directory with a foreign (free format) striping pattern — peep --foreign and --xattr for that whole situation.

Also plot twist: lfs mkdir is literally just an alias for this command. Same girl, different name. 👑

🎀 OPTIONS 🎀

  • -c, -T, --mdt-count=COUNT — stripe the new directory over COUNT MDTs. that's the whole vibe.
  • -C, --overstripe-count STRIPE_COUNT — the number of stripes to create, and if STRIPE_COUNT goes over the number of MDTs in the filesystem, she'll make more than 1 stripe per MDT. 0 means "use the filesystem-wide default stripe count" (default is 1). A negative value -N means N stripes per available MDT: -1 gives one stripe per MDT, and -2, -3, -4, -5 give 2, 3, 4, and 5 stripes per MDT respectively. The max multiplier allowed rn is -5.
  • -h, --help — prints the usage message, no thoughts just vibes.
  • -i, --mdt-index=START_MDT_INDEX[,MDT_INDEX...] — use the MDT whose index is START_MDT_INDEX as the master/starting MDT for the directory. If you hand her multiple MDT_INDEX values, then the stripes get allocated on those exact specified MDT indices. If -1 (the default) is used, the client will round-robin subdirectory creation across all MDTs when their free space is within lod.*.mdt_qos_threshold_rr percent of each other — otherwise the client prefers to pick COUNT MDT(s) proportional to the free space and inodes on each. 🧚
⚠️ okay so listen: if you provide multiple MDT_INDEX values while trying to set the default striping pattern of subdirectories with -D, it will not load balance subdirectory creation amongst the MDTs. Instead it makes all new subdirectories striped across all of the specified MDTs. This is discouraged and can lead to filesystem problems if too many striped directories get created 😭. If you're gonna do it anyway, -c N must also be specified, where N matches the number of MDT indices you gave.
  • -H, --mdt-hash=HASH_TYPE — use hash_type for the striped directory. the options are:
    • crush — CRUSH hash algorithm. This is a consistent hash algorithm, so minimum sub files need to relocate during directory restripe.
    • fnv_1a_64 — Fowler-Noll-Vo (FNV-1a) hash algorithm. This provides reasonably uniform, but not cryptographically strong, hashing of the filename. (this is the default, bestie 💅)
    • all_char — sum of ASCII characters modulo number of MDTs. This provides weak hashing of the filename, and is suitable for only testing or when the input is known to have perfectly uniform distribution (e.g. sequential numbers).
  • -d, --delete — delete the default striping layout from the directory. New subdirectories created in this directory will inherit the global default directory layout (by default they will not be striped).
  • -D, --default — set the default striping pattern of subdirectories. Newly created sub-directories will use the new default striping pattern, but existing sub-directories will not be affected. The newly created sub-directories will also inherit the specified default striping pattern.
🫶 note that striping all directories across all MDTs by default is not recommended at this time, because the clients have to do more RPCs to create and access each directory, which hurts performance instead of improving it. Default striped directories are preferred only for parent directories where large subdirectories will be created (e.g. file-per-process job output directories). Instead, using -c 1 -i -1 on top-level directories balances mkdir therein over MDTs automatically without causing all subdirectories to be remote by default. slay.
  • -o, --mode=MODE — set the file access permissions of the new directory to the specified numeric MODE (typically octal), same energy as chmod(1). It is not affected by the current umask(1p).
  • --foreign[=type] — create a directory with a foreign (non-Lustre/free format, see the --xattr option) striping. Where type specifies a known foreign type (like none , symlink , ...) or a 32-bit numeric type.
  • --flags=<hex> — specify a numeric bitmask of type-specific layout flags for the foreign layout.
  • -x, --xattr=STRING — specify a string to be used as a foreign (free format) striping.
  • -X, --max-inherit=MAX_INHERIT — set the inherit depth of default directory layout. If non-zero, then MAX_INHERIT is the number of subdirectory levels for which this default layout is inherited, up to a maximum of 250 levels, and it gets decremented by one when copying the default layout to each new subdirectory, until zero and the default layout is no longer copied. A MAX_INHERIT of -1 means the default layout is inherited for all subdirectories. 🍓
  • --max-inherit-rr=MAX_INHERIT_RR — set the round-robin inherit depth of the default directory layout, only when START_MDT_INDEX is -1. If MAX_INHERIT_RR is zero (unset), then new subdirectories are preferentially created on MDTs with more free space and inodes if the MDTs are imbalanced. If non-zero, then MAX_INHERIT_RR is the number of subdirectory levels, up to a maximum of 250 levels, for which new subdirectories will be created in a round-robin manner across all available MDTs, rather than using MDT space balancing for new subdirectories. Otherwise, MAX_INHERIT_RR is decremented by one when copying the default layout to each new subdirectory, until zero and round-robin is no longer used. It may be useful to set MAX_INHERIT_RR to 2 or 3 when setting the default directory layout on the root of a new filesystem, so that top-level subdirectories are immediately spread across MDTs, rather than waiting for the MDTs to become imbalanced. Round-robin subdirectory creation is unlikely to be useful for existing filesystems that already have many files and imbalanced MDTs, since space balancing will already happen, and this will unnecessarily increase the number of remote subdirectories (increasing overhead) without any benefit. 😭

👑 NOTE 👑

If neither -c nor -i are specified, lfs mkdir will default to -c 1 -i -1 and create the directory on an MDT that is less full than the others.

The lfs setdirstripe command is only usable by root unless the mdt.*.enable_remote_dir_gid is set on the MDS via lctl set_param to be either a non-zero numeric GID to limit it to a single group (e.g. the operator or admin GID), or -1 to allow any user/group to create remote directories. By default, it is 0 to limit remote/striped directories to only the root user.

The root directory of the file system is on MDT0000, and a filesystem-wide default directory layout is set on it by default:

lfs setdirstripe -D -i -1 -c 1 --max-inherit -1 --max-inherit-rr 3

which is a space balanced directory layout ✨ therefore new directories will be created on all MDTs by inode and space usage, and the top three level of new directories will be created in roundrobin mode if system is balanced. If user doesn't want automatic space balance, deleting this default directory layout won't work, because it will be recreated if not set. Instead, user can run

lfs setdirstripe -D -i -1 -c 1 --max-inherit 1 <mountpoint>

to limit space balance in the top level, or run

lfs setdirstripe -D -i 0 -c 1 --max-inherit 1 <mountpoint>

to disable it.

🔥 EXAMPLES 🔥

$ lfs setdirstripe -c 2 -i 1 -H all_char /mnt/lustre/dir1

This creates a directory striped on two MDTs, whose first stripe is on MDT0001 (MDT index 1), and whose hash type is all_char. iconic.

$ lfs mkdir --foreign=symlink --xattr PUUID/CUUID /mnt/lustre/dir1

This creates dir1 with foreign (non-lustre/free format) PUUID/CUUID striping/LMV EA value (symlink type). 💀 she ate fr fr.

⭐ AVAILABILITY ⭐

The lfs setdirstripe command is part of the Lustre filesystem.

🌈 SEE ALSO 🌈

lctl(8), lfs(1), lfs-getdirstripe(1), lfs-setstripe(1), lustre(7)

okay that's the whole page bestie, you're so welcome 💜 this lil number comes to you from Lustre, dated 2017-11-07, and her name is LFS-SETDIRSTRIPE(1) — go forth and stripe responsibly, ily, bye 🫶✨

📦 where this page comes from