SFSMASTER.CFG(5)

📦
version 5.8.0 · 4 versions
, in ubuntu stonking

💖 NAME 💖

sfsmaster.cfg — bestie, this is the main configuration file for sfsmaster, aka section 5 of the manual. If sfsmaster is the main character, this file is its diary. 🎀

✨ DESCRIPTION ✨

The file sfsmaster.cfg holds the configuration for the SaunaFS metadata server process. That's it, that's the description. Short and iconic. 💅

🌸 SYNTAX 🌸

The vibe is literally just:

OPTION = VALUE

And anything where the line starts with #? Ignored. Not read. Not perceived. Ghosted. 👻

👑 OPTIONS 👑

Configuration options, coming in hot, no skips, fr fr:

  • PERSONALITY — the current personality of this instance of the metadata server. Valid values are master, shadow and ha-cluster-managed. If your installation is managed by an HA cluster, the only valid value is ha-cluster-managed. Otherwise, the only valid values are master and shadow — and in that case, only one metadata server in SaunaFS should have the master personality. What do they mean tho? master = this instance is the main metadata server, governing all the file system metadata modifications, the literal queen bee 👑. shadow = this instance is the backup metadata server, ready for immediate deployment as the new master if the current master has a whole crisis 💀. You can change the metadata server's personality at any moment — but only as long as you're changing from shadow to master. Going the other way around is forbidden, so don't even try it. ha-cluster-managed = this instance is managed by an HA cluster, and the server runs in shadow mode for as long as it hasn't been remotely promoted to master.
  • CLUSTER_ID — uniquely identifies this namespace. Basically it stops you from accidentally connecting to the wrong metadata servers or chunkservers (which would be so embarrassing 🙈). Default is default.
  • DATA_PATH — where to store metadata files and the lock file.
  • WORKING_USER — the user to run the daemon as.
  • WORKING_GROUP — the group to run the daemon as. It's optional — if it's empty, then the default user group gets used.
  • SYSLOG_IDENT — the name of the process to place in syslog messages. Default is sfsmaster.
  • LOCK_MEMORY — whether to perform mlockall() to avoid swapping out the sfsmaster process. Default is 0, i.e. no.
  • LIMIT_GLIBC_MALLOC_ARENAS — Linux only, bestie! Limits glibc malloc arenas to the given value, which prevents it from using a huge amount of virtual memory. Heads up: this can influence performance — it reduces memory fragmentation and improves cache locality, but it may also lead to contention and reduced parallelism in multi-threaded applications. Use it in constrained memory environments; recommended values are 4 or 8. Default is 0: disabled, or let glibc decide.
  • NICE_LEVEL — the nice level to run the daemon with. Default is -19 if possible — note that the process must be started as root to increase priority. 👑
  • EXPORTS_FILENAME — an alternative name of the sfsexports.cfg file.
  • TOPOLOGY_FILENAME — an alternative name of the sfstopology.cfg file.
  • CUSTOM_GOALS_FILENAME — an alternative name of the sfsgoals.cfg file.
  • PREFER_LOCAL_CHUNKSERVER — okay so if a client mountpoint has a local chunkserver and a given chunk happens to reside locally, then sfsmaster will list the local chunkserver first. But plot twist: when the local client mount is issuing many read(s)/write(s) to many local chunks, those requests can overload the local chunkserver and the disk subsystem 😭. Setting this to 0 (the default is 1) means remote chunkservers will be considered as equivalent to the local chunkserver. This is useful when the network is faster than the disk, and when there's a high-IO load on the client mountpoints.
  • BACK_LOGS — number of metadata change log files. Default is 50.
  • BACK_META_KEEP_PREVIOUS — number of previous metadata files to be kept. Default is 1.
  • AUTO_RECOVERY — when this option is set (equals 1), master will try to recover metadata from the changelog when it's being started after a crash. Otherwise it will refuse to start and sfsmetarestore should be used to recover the metadata. Default is 0.
  • OPERATIONS_DELAY_INIT — initial delay in seconds before starting chunk operations. Default is 300.
  • OPERATIONS_DELAY_DISCONNECT — chunk operations delay in seconds after chunkserver disconnection. Default is 3600.
  • MATOML_LISTEN_HOST — IP address to listen on for metalogger connections (* means any).
  • MATOML_LISTEN_PORT — port to listen on for metalogger connections. Default is 9419.
  • MATOML_LOG_PRESERVE_SECONDS — how many seconds of change logs have to be preserved in memory. Default is 600; note that logs are stored in blocks of 5k lines, so sometimes the real number of seconds may be a little bigger. Zero disables the extra logs storage.
  • MATOCS_LISTEN_HOST — IP address to listen on for chunkserver connections (* means any).
  • MATOCS_LISTEN_PORT — port to listen on for chunkserver connections. Default is 9420.
  • MATOCL_LISTEN_HOST — IP address to listen on for client (mount) connections (* means any).
  • MATOCL_LISTEN_PORT — port to listen on for client (mount) connections. Default is 9421.
  • MATOTS_LISTEN_HOST — IP address to listen on for tapeserver connections (* means any).
  • MATOTS_LISTEN_PORT — port to listen on for tapeserver connections. Default is 9424.
  • CHUNKS_LOOP_MAX_CPS — the chunks loop shouldn't check more chunks per second than the given number. Default is 100000.
  • CHUNKS_LOOP_MIN_TIME — the chunks loop will check all chunks in the specified time. Default is 300, unless CHUNKS_LOOP_MAX_CPS forces slower execution.
  • CHUNKS_LOOP_PERIOD — time in milliseconds between chunks loop executions. Default is 1000.
  • CHUNKS_LOOP_MAX_CPU — hard limit on CPU usage by the chunks loop (a percentage value). Default is 60.
  • CHUNKS_SOFT_DEL_LIMIT — soft maximum number of chunks to delete on one chunkserver. Default is 10.
  • CHUNKS_HARD_DEL_LIMIT — hard maximum number of chunks to delete on one chunkserver. Default is 25.
  • CHUNKS_WRITE_REP_LIMIT — maximum number of chunks to replicate to one chunkserver. Default is 2.
  • CHUNKS_READ_REP_LIMIT — maximum number of chunks to replicate from one chunkserver. Default is 10.
  • ENDANGERED_CHUNKS_PRIORITY — the percentage of endangered chunks that should be replicated with high priority. Example: when set to 0.2, up to 20% of chunks served in one turn would be extracted from the endangered priority queue. When set to 1 (max), no other chunks would be processed as long as there are any endangered chunks in the queue (not advised, bestie). Default is 0, i.e. there's no overhead for prioritizing endangered chunks.
  • ENDANGERED_CHUNKS_MAX_CAPACITY — max capacity of the endangered chunks queue. This value can limit the memory usage of the master server if there are lots of endangered chunks in the system. It's ignored if ENDANGERED_CHUNKS_PRIORITY is set to 0. Default is 1Mi, i.e. no more than 1Mi chunks will be kept in a queue.
  • ACCEPTABLE_DIFFERENCE — the maximum difference between disk usage on chunkservers that doesn't trigger chunk rebalancing. Default is 0.1, i.e. 10%.
  • CHUNKS_REBALANCING_BETWEEN_LABELS — when balancing disk usage, allow moving chunks between servers with different labels. Default is 0, i.e. chunks will be moved only between servers with the same label.
  • GLOBALIOLIMITS_FILENAME — configuration of global I/O limits. Default is no I/O limiting.
  • GLOBALIOLIMITS_RENEGOTIATION_PERIOD_SECONDS — how often mountpoints will request bandwidth allocations under constant, predictable load. Default is 0.1.
  • GLOBALIOLIMITS_ACCUMULATE_MS — after inactivity, no waiting is required to transfer the amount of data equivalent to normal data flow over the period of that many milliseconds. Default is 250.
  • METADATA_CHECKSUM_INTERVAL — how often the metadata checksum shall be sent to backup servers. Default is: every 50 metadata updates.
  • METADATA_CHECKSUM_RECALCULATION_SPEED — how fast metadata should be recalculated in the background. Default: 100 objects per function call.
  • DISABLE_METADATA_CHECKSUM_VERIFICATION — should checksum verification be disabled while applying the changelog.
  • NO_ATIME — when this option is set to 1, the inode access time is not updated on every access. Otherwise (when set to 0) it is updated. Default is 0.
  • METADATA_SAVE_REQUEST_MIN_PERIOD — minimal time in seconds between metadata dumps caused by requests from shadow masters. Default is 1800.
  • SESSION_SUSTAIN_TIME — time in seconds for which client session data (e.g. list of open files) should be sustained in the master server after the connection with the client was lost. Values between 60 and 604800 (one week) are accepted. Default is 86400.
  • USE_BDB_FOR_NAME_STORAGE — when this option is set to 1, Berkeley DB is used for storing file/directory names in the file DATA_PATH/name_storage.db. By default all strings are kept in system memory. Default is 0.
  • BDB_NAME_STORAGE_CACHE_SIZE — size of the memory cache (in MB) for file/directory names used by the Berkeley DB storage. Default is 10.
  • AVOID_SAME_IP_CHUNKSERVERS — when this option is set to 1, the process of selecting chunkservers for chunks will try to avoid using those that share the same IP. Default is 0.
  • REDUNDANCY_LEVEL — minimum number of required redundant chunk parts that can be lost before a chunk becomes endangered. Default is 0.
  • SNAPSHOT_INITIAL_BATCH_SIZE — this option can be used to specify the initial number of snapshotted nodes that will be atomically cloned before enqueuing the task for execution in fixed-sized batches. Default is 1000.
  • SNAPSHOT_INITIAL_BATCH_SIZE_LIMIT — this option specifies the maximum initial batch size set for a snapshot request. Default is 10000.
  • FILE_TEST_LOOP_MIN_TIME — the test files loop will try to check all files in the specified time, in seconds. Default is 3600. It's possible for the loop to take more time if the master server is busy, or if the machine doesn't have enough processing power to make all the needed calculations. 😮‍💨

🔒 Options below are mandatory for all Shadow instances 🔒

Like, non-negotiable. You need these:

  • MASTER_HOST — address of the host running the SaunaFS metadata server that currently acts as master.
  • MASTER_PORT — port number where the SaunaFS metadata server currently running as master listens for connections from 'shadow's and metaloggers. Default is 9420.
  • MASTER_RECONNECTION_DELAY — delay in seconds before trying to reconnect to the metadata server after disconnection. Default is 1.
  • MASTER_TIMEOUT — timeout (in seconds) for metadata server connections. Default is 60.

More options, coming right up—stay with me, bestie 💖

  • LOAD_FACTOR_PENALTY — when set, percentage of load will be added to chunkserver disk usage to determine most fitting chunkserver. Heavy loaded chunkservers will be picked for operations less frequently. Default is 0, and correct values are in range from 0 to 0.5. balance queen behavior ✨
  • PRIORITIZE_DATA_PARTS — when set, master server will prioritize data parts in EC goals to land in the chunkservers with higher percentage of available space. Could cause parities landing always in the same chunkservers if the cluster is not well balanced. Default: 1. ⚠️ unbalanced-vibes warning
  • POLL_TIMEOUT_MS — maximum amount of time in milliseconds that the polling operation will wait for events. The value is applied for the polling in the events loop. Smaller values could reduce latency at the cost of CPU usage. Default: 50. latency vs CPU drama 😭
  • ENABLE_PROMETHEUS (EXPERIMENTAL) — whether to enable Prometheus support and metric collection. Note that this requires compiling with Prometheus support. Set to either 1 to enable, or 0 to disable. Default is 0. metrics era 📊
  • PROMETHEUS_HOST (EXPERIMENTAL) — host address where Prometheus metric data can be collected, must be in the format of HOST:PORT. Default 0.0.0.0:9499. address, but make it metrics ✨
  • CREATE_EMPTY_FOLDERS_WHEN_SPACE_DEPLETED (EXPERIMENTAL) — when enabled, this option allows the system to create metadata for empty folders when storage space is depleted. This helps preserving the intended file and directory structure. When this option is disabled, no metadata for folders will be created if there is insufficient storage space. Default: 1. folder structure preservation slay 🫶
  • LOG_LEVEL — setup logging. Uses the environment variable SAUNAFS_LOG_LEVEL or config value LOG_LEVEL to determine logging level. Valid log levels are
    • trace
    • debug
    • info
    • warn or warning
    • err or error
    • crit or critical
    • off

pick your log volume, bestie 🔊

  • TLS_CERT_FILE (EXPERIMENTAL) — path to the TLS certificate file the master/shadow server will use for TLS connections. There is no default value. certificate glam 🔐
  • TLS_KEY_FILE (EXPERIMENTAL) — path to the TLS private key file the master/shadow server will use for TLS connections. There is no default value. keep it secret, keep it safe 🗝️
  • TLS_CA_CERT_FILE (EXPERIMENTAL) — path to the trusted CA certificate which is used to authenticate the TLS connection. There is no default value. trust issues, but make it TLS 🎀
  • USE_CHUNKSERVER_SIDE_CHUNK_LOCK (EXPERIMENTAL) — when set to 1, enables sending chunk part lock messages to the chunkservers. This can be useful to track down which chunk parts are currently being written. Reloadable. Default: 0. lock it down, detective 🔍
  • EMPTY_RESERVED_FILES_PERIOD_MSECONDS (EXPERIMENTAL) — interval for periodic cleaning of reserved files, in milliseconds. If set to 0, the reserved files deletion is disabled. Default: 0. ⚠️ admin-only, read the warning below
⚠️ Warning Administrator-only option. Enabling periodic deletion may remove master-side references to reserved files that are still held by other applications or client instances that have not released them yet. This can disrupt ongoing operations and lead to unexpected errors. Recommended approach: Do not enable automatic deletion unless you fully understand the impact on running workloads. Prefer manually freeing reserved files by locating each reference or process using them and stopping it before releasing the reservation.

🌙 NOTES 🌙

Chunks in master are tested in loop. Speed (or frequency) is regulated by two options CHUNKS_LOOP_MIN_TIME and CHUNKS_LOOP_MAX_CPS. First defines minimal time of the loop and second maximal number of chunk tests per second. Typically at the beginning, when number of chunks is small, time is constant, regulated by CHUNK_LOOP_MIN_TIME, but when number of chunks became bigger then time of loop can increase according to CHUNKS_LOOP_MAX_CPS. chunk loop math, iykyk 🌀

Deletion limits are defined as soft and hard limit. When number of chunks to delete increases from loop to loop then current limit can be temporary increased above soft limit, but never above hard limit. soft limit is a suggestion, hard limit is the law 🚨

🐞 REPORTING BUGS 🐞

Report bugs to the GitHub repository https://github.com/leil-io/saunafs as an issue. be a good citizen and file that issue 💖

Copyright 2008-2009 Gemius SA

Copyright 2013-2019 Skytechnology sp. z o.o.

Copyright 2023-2024 Leil Storage OÜ

SaunaFS is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, version 3.

SaunaFS is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with SaunaFS. If not, see http://www.gnu.org/licenses/.

🔮 SEE ALSO 🔮

This page is SFSMASTER.CFG(5), dated 2026-07-29 — okay love you, bye bestie 💅✨

📦 where this page comes from