LEIL-URAFT(8)

๐Ÿ“ฆ version 5.12.0, in nixpkgs unstable

๐Ÿ’… NAME ๐Ÿ’…

okay so bestie, meet leil-uraft โ€” the LeilFS high availability cluster management tool. yeah. that's the whole vibe. she runs the HA show for your metadata servers (leil-master) and honestly? she's the main character of your cluster. ๐Ÿ‘‘


โœจ SYNOPSIS โœจ

leil-uraft [--help]

leil-uraft [-c CFGFILE]

๐Ÿฆ‹ DESCRIPTION ๐Ÿฆ‹

so leil-uraft is a high availability (HA) management daemon for LeilFS metadata servers (leil-master). think of her as the group-chat admin who makes sure exactly one person is speaking at a time, no chaos, no double-leaders, no drama (well, mostly no drama).

she implements a Raft-inspired leader election protocol across a configured cluster, and she orchestrates the promotion/demotion of the local metadata service. slay.

in a healthy cluster, exactly one node is the active master (Leader). the other metadata-capable nodes run as shadows (Followers), and elector nodes (Followers) participate in voting but never become Leader. like, they're in the group chat, they vote, but they're not getting the crown. periodt.

the leader holds a floating IP (VIP) so clients and chunkservers have a stable endpoint. iconic behavior, honestly.

the election logic lives in the daemon, while backend-specific actions are delegated to leil-uraft-helper:

  • Health checks (isalive)
  • Metadata version checks (metadata-version)
  • Promotion and demotion (promote / demote)
  • Floating IP actions (assign-ip / drop-ip)

see leil-uraft-helper(8) for the helper contract. ๐Ÿซถ

for more configuration info and examples, refer to the leil-uraft.cfg file and the leil-uraft.cfg(5) man entry.


๐Ÿงš CONCEPTS ๐Ÿงš

Node types

  • Metadata nodes: run leil-master and can become Leader. ๐Ÿ‘‘
  • Elector nodes: participate in voting but never become Leader.

Elector mode is controlled by URAFT_ELECTOR_MODE.

Roles (Raft terms)

  • Follower: passive node that receives heartbeats. ๐Ÿ’“
  • Candidate: node requesting votes during an election. ๐Ÿ™‹โ€โ™€๏ธ
  • Leader: node that won an election. ๐Ÿ‘‘

uRaft also reports a president flag. conceptually, president=1 indicates the Leader is active with quorum. iykyk.

Quorum

the daemon requires a majority to elect a Leader and to keep leadership. for N voting nodes, quorum is:

quorum = floor(N/2) + 1

lowkey the most important math in this whole page, ngl. ๐Ÿงฎ


๐ŸŒธ OPERATIONS ๐ŸŒธ

at runtime the daemon loops through these responsibilities, like a little ritual, every single time:

  1. Exchange heartbeats with peers.
  2. Start an election when heartbeats are missing long enough.
  3. Consult the helper for metadata version and health.
  4. When elected, call the helper to promote the node to master.
  5. When stepping down, call the helper to demote the node to shadow.

Health gating:

  • The daemon can temporarily block elections on a node via blocked_promote. this is used to avoid electing a node whose local metadata server is unhealthy. no one wants a sick leader, bestie. ๐Ÿ™ˆ

Floating IP:

  • When configured, the Leader assigns the VIP and periodically checks it.
  • If the VIP disappears unexpectedly, the daemon may attempt to restore it.

๐Ÿ”ฅ PRODUCTION NOTES ๐Ÿ”ฅ

Cluster sizing

  • Use an odd number of voting nodes (3, 5, 7).
  • Keep voters on stable, low-latency links when possible.

odd numbers fr fr โ€” it's giving "avoid ties" energy.

Network requirements

  • uRaft uses UDP messaging. High packet loss and jitter can cause leader churn. ๐Ÿ˜ญ
  • For WAN/VPN deployments, adjust timing parameters (see leil-uraft.cfg(5)).

Metadata freshness

  • Elections consider metadata version. If a node cannot produce a valid metadata version quickly, it may be prevented from becoming Leader.

Failure behavior

  • If isalive reports dead on the current Leader, the safest operational outcome is: drop VIP, demote to shadow, and allow another node to be promoted.
bestie this is the "just let it go" checklist. don't fight it, demote gracefully. ๐Ÿ’…

Cluster Expansion

adding a new node changes the cluster membership and therefore the quorum size. all uRaft nodes must have the exact same URAFT_NODE_ADDRESS list (same order). because the membership list is read at process startup, the change takes effect after restarting leil-uraft on the nodes.

to expand the cluster with one additional node:

  1. Pick the new node's uRaft UDP address and port (for example new-node:9427). Ensure UDP reachability between the new node and all existing voting nodes.
  2. On every node (existing nodes and the new node), append a new URAFT_NODE_ADDRESS = <new-node>:9427 line to leil-uraft.cfg. Keep the ordering identical across the cluster.
  3. On the new node, set URAFT_ID to the index of the newly added URAFT_NODE_ADDRESS entry. (Example: if you added the 6th entry, set URAFT_ID = 5.)
  4. If the new node is intended to be elector-only, set URAFT_ELECTOR_MODE = 1 on that node. Otherwise keep it 0.
  5. Restart leil-uraft across the cluster. A rolling restart is usually acceptable, but expect a short leadership change window during restarts.
  6. Verify the new node has joined: query the uRaft status endpoint (URAFT_STATUS_PORT) on multiple nodes and confirm the node count and leadership look correct.
the "same order on every node" thing is not a suggestion, bestie. same list, same order, or it's not gonna slay. ๐ŸŽ€

after expansion, quorum becomes:

quorum = floor(N/2) + 1

where N is the number of voting nodes.


๐Ÿ‘‘ MONITORING ๐Ÿ‘‘

Status endpoint

the daemon exposes a TCP status endpoint (see URAFT_STATUS_PORT). it reports role, term, leader id, blocked_promote, and vote/heartbeat summaries.

in general, the status endpoint of any node can be fetched with the command:

$ echo | nc <node-hostname> <URAFT_STATUS_PORT>

therefore, the TCP status endpoint of the Leader can be queried with the command:

$ echo | nc leil-master 9428

a typical TCP status of the Leader looks like below:

SERVER ID 2
president=1
state=LEADER
term=171
voted_for=4
leader_id=4
data_version=334
loyalty_agreement=0
local_time=2090338
blocked_promote=0
votes=[       1|       1|       1|       1|       1|       1]
heart=[    0.00|    0.00|    0.00|    0.00|    0.00|    0.00]
recv =[       1|       1|       1|       1|       1|       1]
ver  =[     334|     334|     334|       0|     334|     334]

the TCP status endpoint in an elector node looks like below:

SERVER ID 0
president=0
state=FOLLOWER
term=171
voted_for=4
leader_id=4
data_version=334
loyalty_agreement=1
local_time=2091343
blocked_promote=1

Logs

the daemon logs to syslog under the tag leil-uraft. repeated lines like "Isalive timeout" indicate the helper did not respond within the configured timeout.

so if you see that line over and over? that's your helper ghosting you. ๐Ÿ’€


๐ŸŽ€ OPTIONS ๐ŸŽ€

  • --help โ€” print usage information and exit
  • --config CFGFILE โ€” provide an alternative configuration file path.

๐Ÿ› REPORTING BUGS ๐Ÿ›

report bugs to the GitHub repository https://github.com/leil-io/leilfs as an issue. we love a good bug report, fr fr.


Copyright 2008-2009 Gemius SA

Copyright 2013-2019 Skytechnology sp. z o.o.

Copyright 2023 Leil Storage Oรœ

LeilFS is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, version 3.

LeilFS is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with LeilFS. If not, see http://www.gnu.org/licenses/.


๐ŸŒˆ SEE ALSO ๐ŸŒˆ

leil-uraft-helper(8), leil-uraft.cfg(5)


that's the whole page bestie, and just so you know where it came from: this beauty is the LEIL-URAFT(8) page, dated 1980-01-01, and it's giving vintage in the cutest way possible. go be a good leader now. ๐Ÿฆ‹โœจ periodt.

๐Ÿ“ฆ where this page comes from