mfsmaster.cfg(5)

📦
version 4.59.2 · 6 versions
, in arch rolling, nixos 26.05, nixpkgs unstable, ubuntu stonking

💖 NAME 💖

mfsmaster.cfg — the main configuration file for mfsmaster ✨

so like, this is the mfsmaster.cfg(5) page bestie, and if you're reading it you're about to run the MooseFS master process 👑

🌸 DESCRIPTION 🌸

okay so real talk: the file mfsmaster.cfg contains the configuration of the MooseFS master process. that's it. that's the whole vibe 💅 everything the master needs to know about itself lives in here.

✨ SYNTAX ✨

the syntax is literally just this:

OPTION = VALUE

and any line that starts with the # character? ignored, queen. those are comments. gossip. nothing more 💅

🎀 STARTUP OPTIONS 🎀

heads up bestie!! changes in this section require a process restart. not a reload. a whole restart. no shortcuts 🫶
  • WORKING_USER — the user that the daemon runs as 👤
  • WORKING_GROUP — the group the daemon runs as; this one is optional — if you leave it empty, then the user's default group gets used instead ✨
  • SYSLOG_IDENT — the name of the process that shows up in syslog messages; default is mfsmaster 📝
  • LOCK_MEMORY — whether to perform mlockall() so the mfsmaster process doesn't get swapped out 🧠; default is 0, which means no
  • DISABLE_OOM_KILLER — disable the out of memory killer (Linux only, default is 1) 💀
  • NICE_LEVEL — the nice level to run the daemon with; default is -19; note: the process must be started as root to increase priority, and if setting the priority fails, the process just keeps the nice level it started with 😭
  • FILE_UMASK — set the default umask for group and others (user always has 0); default is 027 — which blocks write for group and blocks everything for others 🔒
  • DATA_PATH — where to store metadata files and the lock file 📁

🦋 RUNTIME OPTIONS 🦋

good news: changes in this section only need a process reload 💅 much chiller
  • LICENCE_FILENAME — (pro only) alternate location/name of the mfslicence.bin file
  • EXPORTS_FILENAME — alternate location/name of the mfsexports.cfg file
  • TOPOLOGY_FILENAME — alternate location/name of the mfstopology.cfg file
  • INSTANCE_NAME — optional name of the MooseFS instance (the cluster, bestie) that gets displayed in the GUI interface; default is the empty string 🌈
  • BACK_LOGS — how many metadata change log files to keep (default is 50) 📚
  • METADATA_SAVE_OFFSET — an offset used for the options METADATA_SAVE_FREQ, (pro only) METADATA_DOWNLOAD_FREQ, and (pro only) METADATA_CRCCHECK_FREQ (in minutes — default is 0); the format X:Y is also accepted and means X hours and Y minutes, and you can slap a suffix "L" on it to mean local (server) time zone instead of UTC; if hours are provided, the modulo value of METADATA_SAVE_FREQ, METADATA_DOWNLOAD_FREQ and METADATA_CRCCHECK_FREQ will be calculated and used — see NOTES 👀
  • METADATA_SAVE_FREQ — how often (in hours) the master (single or follower) will store metadata (default is 1) ⏰
  • METADATA_DOWNLOAD_FREQ — (pro only) how often (in hours) the leader will download metadata from followers (default is 0 — which means never, bestie, never ever)
  • METADATA_CRCCHECK_FREQ — (pro only) how often (in hours) the leader will check metadata consistency with followers (and yes it also saves metadata 😌; default is 24)
  • METADATA_DOWNLOAD_LIMIT — (pro only) the metadata downloading limit in mebibytes per second (0 means no limit, default is 50 — that's 50MiB/s = ~52.4MB/s for the math girlies) 🔥
  • DEPUTY_ELECTION_TIME — (pro only) how many seconds until a master-elect turns into a master-deputy (a.k.a. master-readonly) if there aren't enough chunkservers connected to become master-leader (default is 0 — which means never) 👑
  • KEEP_LEADERSHIP — (pro only) if this option is set to 1, this master will try to assume leadership after it has lost it because of a restart or disconnection; it will not attempt this if a user has switched the leadership on purpose; for the detailed explanation go read NOTES (default is 0 — don't try to assume leadership) 🎀
  • BACK_META_KEEP_PREVIOUS — number of previous metadata files to be kept (default is 1)
  • CHANGELOG_PRESERVE_SECONDS — how many seconds of change logs have to be preserved in memory (default is 5000; this sets the minimum, the actual number may be a bit bigger because logs are kept in 5k blocks; zero disables the extra logs storage) 🧠
  • CHANGELOG_PRESERVE_MB — how much RAM can be used to preserve change logs in memory, in megabytes (default is 500; this also sets the minimum, and the actual number may be a bit bigger because logs are kept in 5k blocks; zero disables the extra logs storage); the master will always preserve the lower number of 5k blocks defined by CHANGELOG_PRESERVE_SECONDS and CHANGELOG_PRESERVE_MB, so bestie please be sure to set both appropriately 🙏
  • CHANGELOG_SAVE_MODE — the changelog save mode (default is 0)
    • 0 — write in background by a different process (less safe, but it doesn't make the master stop in case of heavy hdd load)
    • 1 — write in foreground without syncing data (the master waits for every changelog to be saved to hdd, but without syncing — a little more safe than the background option, but it may cause the master to stop and wait for flushing hdd buffers 😩)
    • 2 — write in foreground with fsync after each write (very safe, but it may make your master very slow unless you have very sophisticated hardware 💀)
  • MISSING_LOG_CAPACITY — how many missing chunks will be stored in the master (up to 100*MISSING_LOG_CAPACITY bytes of memory will be allocated; default value is 100000) 📦
  • CONFIGURATION_TEST_PERIOD — (pro only) how often (in hours) the master leader should run a test that checks if configuration options are consistent between modules (default is 24, max possible value is 168, and 0 means no tests are performed) 🧪
  • SYSLOG_MIN_LEVEL — the minimum level of messages that will be reported by the master; the levels in order of importance are: ERROR, WARNING, NOTICE, INFO, DEBUG (default is INFO) 📢
  • SYSLOG_ELEVATE_TO — reported messages of a level lower than what's set by this option will be elevated to this level (i.e. if SYSLOG_MIN_LEVEL is set to DEBUG and SYSLOG_ELEVATE_TO is set to NOTICE, all INFO and DEBUG messages will be sent to syslog as NOTICE; default is NOTICE) ⬆️

🌈 COMMAND CONNECTION OPTIONS 🌈

changes in this section only need a process reload ✨
  • MATOML_LISTEN_HOST — the IP address to listen on for metalogger, (pro only) masters and supervisors connections (* means any) 🛰️
  • MATOML_LISTEN_PORT — the port to listen on for metalogger, (pro only) masters and supervisors connections 🔌
  • MATOML_TIMEOUT — the default timeout in seconds for the master-metalogger connection (default is 10) ⏳
  • MATOML_FORCE_TIMEOUT — the forced timeout in seconds for the master-metalogger connection (default is 0 — do not force timeouts) 🚫

👑 MASTER-LEADER CONNECTION OPTIONS 👑

changes in this section only need a process reload 💅
  • MASTER_HOST — (pro only) the MooseFS master host (default is mfsmaster) 🏠
  • MASTER_RECONNECTION_DELAY — (pro only) the delay in seconds before the next try to reconnect to the master-leader if not connected (default is 5) 🔁
  • MATOMA_TIMEOUT — (pro only) the timeout in seconds for master-leader connections (default is 10) ⏳
  • BIND_HOST — (pro only) the local address to use for connecting with the master-leader (default is *, i.e. the default local address) 📍

🧚 CHUNKSERVER CONNECTION OPTIONS 🧚

changes in this section only need a process reload 🫶
  • MATOCS_LISTEN_HOST — the IP address to listen on for chunkserver connections (* means any) 🛰️
  • MATOCS_LISTEN_PORT — the port to listen on for chunkserver connections 🔌
  • MATOCS_TIMEOUT — the default timeout in seconds for the master-chunkserver connection (default is 10) ⏳
  • MATOCS_FORCE_TIMEOUT — the forced timeout in seconds for the master-chunkserver connection (default is 0 — do not force timeouts) 🚫
  • AUTH_CODE — an optional authentication string. when it's defined, then <b>only</b> chunkservers with the same AUTH_CODE are allowed to connect to this master. when it's not defined (which is the default), then all chunkservers are allowed 😌 if you want to switch on chunkserver authentication, then first define AUTH_CODE in all your chunkservers (and reload/restart them), then define this option in the master and reload/restart it. and remember bestie, after a reload the currently connected chunkservers are NOT disconnected — the new AUTH_CODE will only be used when chunkservers make a new connection 🔐
  • REMAP_BITS, REMAP_SOURCE_IP_CLASS, REMAP_DESTINATION_IP_CLASS — optional IP class remapping. this remaps chunkserver IP addresses that have the first REMAP_BITS of their IP equal to the first REMAP_BITS of REMAP_SOURCE_IP_CLASS. during remapping the system will set the first REMAP_BITS of the IP address to the first REMAP_BITS of REMAP_DESTINATION_IP_CLASS. all three options need to be defined for the remapping to work 🔄
  • MULTILAN_BITS, MULTILAN_CLASSES — optional LAN IP class remapping. remap chunkservers' IP addresses with the first MULTILAN_BITS of the client's IP for all client IPs matching one of the MULTILAN_CLASSES (the classes must be separated by a comma). all masters and chunkservers must have valid IPs from each of the MULTILAN_CLASSES classes. both options need to be defined for the remapping to work 🌐
  • MULTILAN_IPMAP_FILENAME — alternate location/name of the mfsipmap.cfg file. this file defines an optional custom IP mapping 🗺️

🍓 CHUNKSERVER WORKING OPTIONS 🍓

changes in this section only need a process reload ✨
  • REPLICATIONS_DELAY_INIT — the initial delay in seconds before starting replications (default is 60) ⏱️
  • REPLICATIONS_RESPECT_TOPOLOGY — whether to make undergoal replications respect topology (default is 0)
    • 0 — do not respect topology
    • 1 — pick a destination server at random, but then choose the best source server
    • 2 — try to find a destination server in the same rack as one of the existing copies and then replicate the chunk locally (in the same rack) 🗄️
  • CREATIONS_RESPECT_TOPOLOGY — whether new chunks should be recorded with respect to topology (default is 0)
    • 0 — do not respect topology
    • N (N>0) — first try to create new chunks on servers with a topological distance LOWER than N from the client; if that's not possible — for example because of storage class, chunk servers being busy, or lacking space — then try servers with a distance greater or equal to N 🌸
  • CHUNKS_UNIQUE_MODE — stops the master from putting different copies of the same chunk on the same ip/rack (default is 0) 💅
    • 0 — ignore ip and rackid (standard behaviour)
    • 1 — avoid storing more than one copy on chunkservers using same IP number
    • 2 — avoid storing more than one copy on chunkservers using IP number from the same rack id
NOTICE! This parameter is available for backward compatibility purposes and should not be set to a value other than 0. Instead, distinguish feature from storage classes should be used. However, if it is set to a value other than 0, it will override any distinguish definition in storage classes. For more information about storage classes and distinguish feature refer to mfsscadmin(1) manual.
  • CHUNKS_LOOP_MAX_CPS — the chunks loop shouldn't check more chunks per second than this number, bestie, like calm down (default is 100000) ⏱️
  • CHUNKS_LOOP_MIN_TIME — the chunks loop shouldn't be done in fewer seconds than this number (default is 300)
  • CHUNKS_SOFT_DEL_LIMIT — soft maximum number of chunks to delete on one chunkserver (default is 10)
  • CHUNKS_HARD_DEL_LIMIT — hard maximum number of chunks to delete on one chunkserver (default is 25)
  • CHUNKS_WRITE_REP_LIMIT — maximum number of chunks to replicate to one chunkserver (default is 2,1,1,4,4 — see NOTES) 📥
  • CHUNKS_READ_REP_LIMIT — maximum number of chunks to replicate from one chunkserver (default is 10,5,2,5,10 — see NOTES) 📤
  • CS_HEAVY_LOAD_THRESHOLD — threshold for chunkserver load (default is 150 — see NOTES)
  • CS_HEAVY_LOAD_RATIO_THRESHOLD — threshold ratio for chunkserver load (default is 3.0 — see NOTES)
  • CS_HEAVY_LOAD_GRACE_PERIOD — defines how long chunkservers will remain in 'grace' mode (default is 900 — see NOTES)
  • ACCEPTABLE_PERCENTAGE_DIFFERENCE — maximum percentage difference between space usage of chunkservers (default is 1 = 1%)
  • PRIORITY_QUEUES_LENGTH — length of priority queues (for endangered, undergoal etc. chunks — chunks that should be processed first — default is 1000000)
  • CS_MAINTENANCE_MODE_TIMEOUT — maximum time server can be in maintenance mode (default value is 0 — which means 'forever'); for value formatting see TIME
  • CS_TEMP_MAINTENANCE_MODE_TIMEOUT — maximum time server can be in "temporary" maintenance mode (server is switched to this mode whenever it is stopped gracefully, after reconnection server is switched back to normal mode automatically ; default value: 30m); for value formatting see TIME
  • CS_DAYS_TO_REMOVE_UNUSED — how many days unused (disconnected) chunkserver should be kept in master data structures (valid values: 0 - 365 ; 0 means indefinitely ; default value: 7)

🫶 CLIENTS CONNECTION OPTIONS 🫶

Changes in this section require only process reload. (so chill, no full restart needed 💖)

  • MATOCL_LISTEN_HOST — IP address to listen on for client (mount) connections (* means any)
  • MATOCL_LISTEN_PORT — port to listen on for client (mount) connections
  • MATOCL_TIMEOUT — default timeout in seconds for master-client connection (default is 10)
  • MATOCL_FORCE_TIMEOUT — forced timeout in seconds for master-client connection (default is 0 - do not force timeouts)
  • RESTRICT_INCOMPATIBLE_CLIENT_VERSIONS — whether MooseFS should prevent connections from clients that are unable to read all data (especially erasure encoded data). Default is 1 - prevent connections. If this option is set to 0, clients that try to read data in a format they do not understand will return read errors. Use with caution. 😭

🌈 CLIENTS WORKING OPTIONS 🌈

Changes in this section require only process reload.

  • SESSION_SUSTAIN_TIME — how long to sustain a disconnected client session (default is 1 day); for value formatting see TIME 🕰️

💜 FILE SYSTEM OPTIONS 💜

Changes in this section require only process reload.

  • QUOTA_DEFAULT_GRACE_PERIOD — default grace for soft quota (default is 7 days); for value formatting see TIME
  • ATIME_MODE — set atime modification mode (default is 2 : similar to 'relatime' - see NOTES)
  • KEEP_EMPTY_FILES_IN_TRASH — move empty files to trash after unlink? (default is 0 - delete empty files immediately regardless of trash retention settings)
  • RESERVE_SPACE — set amount of space reserved for superuser (default is 0 = do not reserve space for superuser - see NOTES)
  • MAX_ALLOWED_HARD_LINKS — define limit for number of hardlinks allowed for one object (default is 32767; possible values are from 8 to 65000)
  • INODE_REUSE_DELAY — delay time after which inodes of deleted objects will be reused. BE AWARE if you change this value below 1 day you MUST ensure that this value is higher than any of the following timeouts in all clients: mfsattrcacheto, mfsxattrcacheto, mfsentrycacheto, mfsdirentrycacheto, mfsnegentrycacheto, mfssymlinkcacheto. (default is 1d; possible values are from 300 to 3000000 seconds); for value formatting see TIME 💀
  • DEFAULT_EC_DATA_PARTS — how many data parts should the system use when old style EC definition is in use (@n instead of @8+n or @4+n; default is 8; possible values are 4 or 8)

🌸 NOTES 🌸

⏳ TIME ⏳

For config variables that define time without requiring a single, specific unit, time can be defined as a number of seconds (integer) or a time period in one of two possible formats:

  • first format: #.#T where T is one of: s-seconds, m-minutes, h-hours, d-days or w-weeks; fractions of seconds will be rounded to full seconds
  • second format: #w#d#h#m#s, any number of definitions can be omitted, but the remaining definitions must be in order (so #d#m is still a valid definition, but #m#d is not); ranges: s,m: 0 to 59, h: 0 to 23, d: 0 to 6, w is unlimited and the first definition is also always unlimited (i.e. for #d#h#m d will be unlimited)

Examples, bestie — these are the moment ✨

  • 1.5h is the same as 1h30m, is the same as 90m, is the same as 5400s, is the same as 5400
  • 2.5d is the same as 2d12h, is the same as 60h; 1d36h is not a valid time period (h is not the first definition, so it is bound by range 0 to 23)
  • 1.03m is the same as 62s (61.8 seconds will be rounded up to 62)

💫 GENERAL NOTES 💫

Chunks in master are tested in a loop. Speed (or frequency) is regulated by two options: CHUNKS_LOOP_MIN_TIME and CHUNKS_LOOP_MAX_CPS. First defines minimal time between iterations of the loop and second defines maximal number of chunk tests per second. Typically at the beginning, when the number of chunks is small, time is constant, regulated by CHUNK_LOOP_MIN_TIME, but the when number of chunks becomes bigger, then time of loop can increase according to CHUNKS_LOOP_MAX_CPS.

Example: CHUNKS_LOOP_MIN_TIME is set to 300, CHUNKS_LOOP_MAX_CPS is set to 100000 and there is 1000000 (one million) chunks in the system. 1000000/100000 = 10, which is less than 300, so one loop iteration will take 300 seconds. With 1000000000 (one billion) chunks the system needs 10000 seconds for one iteration of the loop. 😮‍💨

Deletion limits are defined as 'soft' and 'hard' limit. When number of chunks to delete increases from loop to loop, current limit can be temporarily increased above soft limit, but never above hard limit.

Replication limits are divided into five cases:

  • first limit is for endangered chunks (chunks with only one copy)
  • second limit is for undergoal chunks (chunks with number of copies lower than specified goal)
  • third limit is for rebalance between servers with space usage close to arithmetic mean
  • fourth limit is for rebalance between other servers (very low or very high space usage)
  • fifth limit is for recovery replications caused by I/O operations (read/write)

Usually first number should be greater than or equal to second, second greater than or equal to third, and fourth greater than or equal to third ( 1st >= 2nd >= 3rd <= 4th ). Fifth limit should be equal or greater than any of the other limits. If one number is given, then all limits are set to this number (for backward compatibility). If only four numbers are given, the fifth will be set as maximum of the four (also for backward compatibility).

Whenever chunkserver load is higher than CS_HEAVY_LOAD_THRESHOLD and CS_HEAVY_LOAD_RATIO_THRESHOLD times higher than average load, then chunkserver is switched into 'grace' mode. Chunkserver stays in grace mode for CS_HEAVY_LOAD_GRACE_PERIOD seconds. 🧘

There are five possible values for ATIME_MODE (all other values are treated as 0):

  • 0 = Always modify atime for files, folders and symlinks.
  • 1 = Always modify atime but only in case of files (do not modify atime in case of folders and symlinks).
  • 2 = Modify atime only when it is lower than ctime or mtime and when current time is higher than ctime or mtime respectively, also modify atime when current atime is older than 24h. Do it for all objects during access (like "relatime" option in Linux).
  • 3 = Same as above but only in case of files. In case of folders and symlinks do not modify atime.
  • 4 = Never modify atime during access (like "noatime" option).

You can reserve space for superuser using RESERVE_SPACE option. You can define it as number of bytes, percent of total space, capacity of biggest chunkserver, etc. 👑

  • # or #B = number of bytes reserved for superuser. Standard metric prefixes can be used - SI and IEC (k,K,M,Mi,G,Gi etc.)
  • #% or #.#% = percent of total capacity of MooseFS instance
  • #U or #.#U = multiplies of "U" value; U is defined as maximum number of bytes currently used by a single chunkserver
  • #C or #.#C = multiplies of "C" value; C is defined as maximum total capacity of a single chunkserver

When your network has two (or more) IP classes, bestie, you might wanna use one network for standard communication between MFS modules and a separate network purely for I/O. It's giving dedicated-lane energy. 🛣️ You do it by setting REMAP_BITS, REMAP_SOURCE_IP_CLASS and REMAP_DESTINATION_IP_CLASS. When you set these options, the master will change internally the IP addresses of chunkservers and will send them as chunk locations, so clients will make connections with chunkservers using the new (destination) IP for all I/O, but still communicate with the master using the original (source) IP. Also chunkservers will use the original IP to communicate with the master, but they will use the new IPs to communicate between themselves during replication. Beware that all clients and chunkservers must have access to both networks, but masters, metaloggers etc. will need only access to the source network. ✨

When your clients are separated into two or more LAN or VLAN networks, you may want them to connect to masters and chunkservers using IPs from their own network. It can be done by setting MULTILAN_BITS and MULTILAN_CLASSES. Each time a client connects, the master will check whether the connection came from one of the defined LAN classes and if yes, it will remap the first MULTILAN_BITS of any chunkserver IP before it sends the chunkserver's IP to the client. Connections coming from other IP addresses will be treated as usual (i.e. original chunkserver IPs will be sent in response). All masters and chunkservers need to have one IP from each of the defined MULTILAN_CLASSES and one module (chunkserver or master) needs to have the same IP suffix in each class. Proper DNS configuration is also required: clients in each LAN must either get a different IP when querying DNS about master host or must use different master host names that are resolved to IPs in their class. See mfsmount(8), mfsbdev(8) for more info about master host name. 📡

The two above sets of options can be used together — yes, they can be besties too. One purpose would be to create a separate network for replication of data between chunkservers while also maintaining several separate networks (LANs) for separate sets of clients. For example, with the following configuration:

REMAP_BITS = 24
REMAP_SOURCE_IP_CLASS = 10.0.0.0
REMAP_DESTINATION_IP_CLASS = 10.0.1.0
MULTILAN_BITS = 24
MULTILAN_CLASSES = 192.168.1.0, 192.168.2.0, 192.168.3.0, 10.0.1.0

all network traffic from clients matching one of the LAN classes will be handled on that network (so a client with IP 192.168.1.17 will connect to the master and chunkservers using IPs with prefix 192.168.1, and a client with IP 192.168.2.13 will connect to the master and chunkservers using IPs with prefix 192.168.2), all metadata traffic between master and chunkservers will be handled on the network with prefix 10.0.0, and all direct communication between chunkservers (i.e. chunk replications) will be handled on the network with prefix 10.0.1. Be aware that any client from outside of the defined LAN classes will connect to the chunkservers via IPs defined by REMAP_DESTINATION_IP_CLASS. This also assumes proper DNS configuration, that is, if the master server uses IP suffix 1, a client with IP 192.168.1.17 should resolve the master host name as 192.168.1.1 and a client with IP 192.168.2.13 should resolve the master host name as 192.168.2.1. A client from outside of the defined LAN classes may use any of the master server IPs, although preferably 10.0.0.1. 🦋

Masters save metadata to a file on a local disk. The exact times of these operations are regulated by four variables: METADATA_SAVE_OFFSET, METADATA_SAVE_FREQ, (pro only) METADATA_DOWNLOAD_FREQ, (pro only) METADATA_CRCCHECK_FREQ. Scheduling era, honestly. ⏰

A single master will save metadata every METADATA_SAVE_FREQ hours. The first save of the day happens at midnight, every other one is after METADATA_SAVE_FREQ from the previous one. This can be changed with METADATA_SAVE_OFFSET. METADATA_SAVE_OFFSET set to a single value will mean minutes. So settings like this:

METADATA_SAVE_FREQ = 2
METADATA_SAVE_OFFSET = 15

will mean saving at 00:15, 02:15, 04:15 etc., up to 22:15. If an hour is provided, this is also taken into account, but only as modulo, so:

METADATA_SAVE_FREQ = 2
METADATA_SAVE_OFFSET = 1:15

will behave exactly the same as:

METADATA_SAVE_FREQ = 2
METADATA_SAVE_OFFSET = 5:15

and will save metadata at 01:15, 03:15, 05:15 etc. The way modulo just eats those hours? Iconic. 💅

All times mentioned above are calculated in UTC, unless the "L" suffix is used. So, if your servers are in the CET zone (UTC-1), this setting:

METADATA_SAVE_FREQ = 2
METADATA_SAVE_OFFSET = 1:15

will mean the master will save metadata at 01:15, 3:15, etc. UTC, which means 0:15, 2:15, etc. local (CET) time. To save at 01:15 (and every 2 hours from that) local time, you need to write:

METADATA_SAVE_FREQ = 2
METADATA_SAVE_OFFSET = 1:15L

(pro only) With multiple masters, the leader will save metadata according to the setting from METADATA_CRCCHECK_FREQ, but also taking into account the offset. So a setting like:

METADATA_CRCCHECK_FREQ = 12
METADATA_SAVE_FREQ = 2
METADATA_SAVE_OFFSET = 1:15

will mean that the leader saves (and compares against followers) metadata at 01:15 and 13:15, while followers save at 01:15, 03:15, 05:15 etc. 👑

METADATA_DOWNLOAD_FREQ means the leader will download metadata from a follower, and that will also take into account the METADATA_SAVE_OFFSET in the manner identical to that of METADATA_CRCCHECK_FREQ.

Times mentioned above are calculated in UTC. So, if your servers are in the PST zone (UTC-8), this setting:

METADATA_CRCCHECK_FREQ = 12
METADATA_SAVE_FREQ = 2
METADATA_SAVE_OFFSET = 1:15

will mean the leader will save metadata at 01:15 and 13:15 UTC, which means 17:15 and 5:15 (5:15 PM and 5:15 AM) local (PST) time. To save at 01:15 and 13:15 (1:15 AM and 1:15 PM) local time, you need to write:

METADATA_CRCCHECK_FREQ = 12
METADATA_SAVE_FREQ = 2
METADATA_SAVE_OFFSET = 1:15L

Setting:

METADATA_SAVE_OFFSET = 0L

can be used to adjust saving times to the local timezone without changing the default metadata saving schedule. Little hack, big slay. 🎀

The KEEP_LEADERSHIP option can be used to select one master in a Pro installation that will assume the role of LEADER whenever possible. If this master loses leadership for reasons related to some malfunction (a disconnection of too many chunkservers from the cluster, a disconnection of this master from the cluster — due to timeouts or other hardware problems) or due to an (intentional or not) process restart, it will try to assume back the LEADER role as soon as possible (as soon as it reaches a synced FOLLOWER state). If a user switches the leadership using the mfssupervisor tool, this master will not attempt to assume back leadership until such time when it becomes LEADER again (this can happen for one of two reasons: deliberate switching of leadership by a user, or when the current LEADER fails and this server happens to be elected as the new LEADER) or is restarted. It's giving main-character-master energy, fr fr. 👑

Setting KEEP_LEADERSHIP option to 1 for more than one master server in a cluster is NOT recommended and may lead to unexpected behaviour. Like, do not. 🙈

Copyright (C) 2026 Jakub Kruszona-Zawadzki, Saglabs SA

This file is part of MooseFS.

MooseFS is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, version 2 (only).

MooseFS is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, see <https://www.gnu.org/licenses/>.

🌈 SEE ALSO

mfsmaster(8), mfsexports.cfg(5) mfstopology.cfg(5) mfsipmap.cfg(5)

And that's a wrap, bestie! 💖 This page is straight out of MooseFS 4.59.2-1, dated May 2026, and it's the mfsmaster.cfg(5) manual — so go forth, configure your masters, and have the most well-scheduled, perfectly-remapped cluster of your life. Periodt. ✨🦋🎀

📦 where this page comes from