CTDB(7)

📦
version 4.24.7 · 18 versions
, in ubuntu stonking

👑 CTDB(7) 👑

okay bestie, buckle UP, because this is the big one: the ctdb page, aka Clustered TDB. This is part 1 of 3, so don't go anywhere, we're just getting started 💖

✨ NAME ✨

ctdb — Clustered TDB 🎀

🌸 DESCRIPTION 🌸

CTDB is a clustered database component sitting inside clustered Samba, and its whole job is giving you a high-availability load-sharing CIFS server cluster. It's giving main character energy in the HA world, no cap.

The main functions of CTDB are, and I need you to appreciate each one:

  • Provide a clustered version of the TDB database, with automatic rebuild/recovery of the databases upon node failures 🛠️
  • Monitor the nodes in the cluster, and the services running on each node 👀
  • Manage a pool of public IP addresses that get used to provide services to clients. Alternatively, CTDB can be used with LVS, if that's your vibe ✨

Combined with a cluster filesystem, CTDB provides a full high-availability (HA) environment for services like clustered Samba, NFS and other services. It's iconic, honestly.

And listen, the manual pages aren't the whole story — there's so much more info at https://wiki.samba.org/index.php/CTDB_and_Clustered_Samba. Go snoop 🦋

🧚 ANATOMY OF A CTDB CLUSTER 🧚

A CTDB cluster is a collection of nodes, and every node has 2 or more network interfaces. All nodes provide network services to clients (usually file/NAS services). The data that gets served by the file services lives on shared storage — usually a cluster filesystem — that every single node can reach.

CTDB gives you an "all active" cluster, where services are load balanced across all the nodes. Nobody's sitting on the bench 💅

🌈 CLUSTER LEADER 🌈

CTDB runs a cluster leader and follower model of cluster management. All the nodes in a cluster elect exactly one node to be the leader. That leader node is the one coordinating the privileged operations — things like database recovery and IP address failover. Leader stuff only, bestie 👑

Heads up: CTDB previously called the leader the recovery master or recmaster. You might still run into those terms in documentation and code, so don't be confused — it's the same thing, just a glow-up rebrand ✨

🔐 CLUSTER LOCK 🔐

CTDB uses a cluster lock to assert its privileged role in the cluster. The node takes the cluster lock when it becomes leader, and holds onto that lock until it is no longer leader. Locked in, literally.

Why do we care? Because the cluster lock helps CTDB avoid a split brain — that's when a cluster becomes partitioned and each partition tries to operate independently. Issues that can result from a split brain include file data corruption, because file locking metadata may not be tracked correctly. Yeah. Data corruption. We do NOT want that 💀

Also a lil history lesson: CTDB previously referred to the cluster lock as the recovery lock. The abbreviation reclock is still used — because just "clock" would be confusing, and honestly, fair.

Now, important: CTDB is unable to configure a default cluster lock, because that would depend on factors like cluster filesystem mountpoints. However, running CTDB without a cluster lock is not recommended, because there will be no split brain protection. Just sayin' 🙈

When a cluster lock is configured, it gets used as the election mechanism. Nodes race to take the cluster lock and the winner becomes the cluster leader. This avoids problems when a node wins an election but is unable to take the lock — which can happen if a cluster becomes partitioned (for example, because of a communication failure) and a different leader gets elected by the nodes in each partition, or if the cluster filesystem has a high failover latency. Messy. We're avoiding the mess 🧹

By default, the cluster lock is implemented using a file — specified by cluster lock in the [cluster] section of ctdb.conf(5) — living in shared storage, usually on a cluster filesystem. To support a cluster lock, the cluster filesystem must support lock coherence. See ping_pong(1) for more details ✨

The cluster lock can also be implemented using an arbitrary cluster mutex helper (or call-out). You indicate this by using an exclamation point (!) as the first character of the cluster lock parameter. For example, a value of !/usr/bin/myhelper cluster would run the given helper with the specified arguments. The helper will keep running for as long as it holds its mutex. Wanna write helpers? See ctdb/doc/cluster_mutex_helper.txt in the source tree, and the related code, for clues 🕵️♀️

When a file is specified for the cluster lock parameter (meaning no leading !), the file lock is implemented by a default helper at /usr/libexec/ctdb/ctdb_mutex_fcntl_helper. That helper takes arguments like this:

ctdb_mutex_fcntl_helper FILE [RECHECK-INTERVAL]

ctdb_mutex_fcntl_helper will take a lock on FILE and then check every RECHECK-INTERVAL seconds to make sure that FILE still exists and that its inode number is unchanged from when the lock was taken. The default value for RECHECK-INTERVAL is 5.

And CTDB does sanity checks to make sure the cluster lock is held as expected. Safety first, always 🫶

🌸 PRIVATE VS PUBLIC ADDRESSES 🌸

Every node in a CTDB cluster has multiple IP addresses assigned to it:

  • A single private IP address that's used for communication between nodes 🔒
  • One or more public IP addresses that get used to provide NAS or other services 🌍

💜 Private address 💜

Each node is configured with a unique, permanently assigned private address, configured by the operating system. This address uniquely identifies a physical node in the cluster, and it's the address the CTDB daemons use to talk to the CTDB daemons on other nodes. It's the group chat, basically 📱

Private addresses are listed in the file /etc/ctdb/nodes. This file contains the list of private addresses for all the nodes in the cluster, one per line, and this file must be the same on all nodes in the cluster. No exceptions, bestie.

Some users like to put this configuration file in their cluster filesystem. You can use a symbolic link, or the nodes list parameter 💅

Private addresses should not be used by clients to connect to services provided by the cluster. Not for clients, periodt.

It is strongly recommended that the private addresses are configured on a private network that is separate from client networks. Why? Because the CTDB protocol is both unauthenticated and unencrypted. If clients share the private network, then steps need to be taken to stop injection of packets to relevant ports on the private addresses. It's also likely that CTDB protocol traffic between nodes could leak sensitive information if it can be intercepted or manipulated 😭

So, in summary: the private network should not be accessible by untrusted sources. We can't say "must not" — because CTDB can't enforce that — so we can only make a strong recommendation. However, for this reason, future bugs in the CTDB protocol used over the private network will be fixed as regular bugs, without following the Samba security process. The same is true of local CTDB protocols used for Unix domain socket and other communication. Yeah. Read that part twice 🫠

Example /etc/ctdb/nodes for a four node cluster:

192.168.1.1
192.168.1.2
192.168.1.3
192.168.1.4

💙 Public addresses 💙

Public addresses are what provide services to clients. Here's the twist: public addresses are not configured at the operating system level, and they're not permanently associated with a particular node. Instead, they're managed by CTDB and assigned to interfaces on physical nodes at runtime. Sneaky? No, iconic ✨

The CTDB cluster will assign/reassign these public addresses across the available healthy nodes in the cluster. When one node fails, its public addresses get taken over by one or more other nodes in the cluster. This makes sure the services provided by all public addresses are always available to clients — as long as there are nodes available that are capable of hosting the address 🫶

The public address configuration is stored in /etc/ctdb/public_addresses on each node. This file contains a list of the public addresses the node is capable of hosting, one per line. Each entry also contains the netmask and the interface to which the address should be assigned. If this file is missing, then no public addresses are configured 🙈

Some users who have the same public addresses on all nodes like to put this configuration file in their cluster filesystem. In that case a symbolic link should be used 🔗

Example /etc/ctdb/public_addresses for a node that can host 4 public addresses, on 2 different interfaces:

10.1.1.1/24 eth1
10.1.1.2/24 eth1
10.1.2.1/24 eth2
10.1.2.2/24 eth2

In many cases the public addresses file will be the same on all nodes. However, it is possible to use different public address configurations on different nodes. Different strokes, same cluster 💅

Example: 4 nodes partitioned into two subgroups:

Node 0:/etc/ctdb/public_addresses
     10.1.1.1/24 eth1
     10.1.1.2/24 eth1

Node 1:/etc/ctdb/public_addresses
     10.1.1.1/24 eth1
     10.1.1.2/24 eth1

Node 2:/etc/ctdb/public_addresses
     10.1.2.1/24 eth2
     10.1.2.2/24 eth2

Node 3:/etc/ctdb/public_addresses
     10.1.2.1/24 eth2
     10.1.2.2/24 eth2

In this example, nodes 0 and 1 host two public addresses on the 10.1.1.x network, while nodes 2 and 3 host two public addresses for the 10.1.2.x network.

Public address 10.1.1.1 can be hosted by either of nodes 0 or 1, and will be available to clients as long as at least one of these two nodes is available 👯♀️

If both nodes 0 and 1 become unavailable, then public address 10.1.1.1 also becomes unavailable. And 10.1.1.1 can not be failed over to nodes 2 or 3, since those nodes do not have this public address configured. Nope, not happening 💀

The ctdb ip command can be used to view the current assignment of public addresses to physical nodes 👀

💖 NODE STATUS 💖

Bestie, you can peep the current status of every single node in the cluster with the ctdb status command. It's giving dashboard, it's giving ✨visibility✨.

A node can be in one of these states, and yes we're going through all of them:

  • OK — this node is healthy and fully functional, living its best life. It hosts public addresses so it can actually serve services. Slay. 👑
  • DISCONNECTED — this node can't be reached by the other nodes over the private network. It's not participating in the cluster rn, it does not host public addresses, and honestly it might just be shut down. Ghosted, but like, by design.
  • DISABLED — someone administratively disabled this node on purpose. It's partially functional and it does participate in the cluster, but it does not host public addresses to provide services. It's in the group chat but on mute.
  • UNHEALTHY — a service on this node failed a health check and needs investigating, bestie. It's partially functional and still participates in the cluster, but nope, it does not host public addresses to provide services. Unhealthy nodes should be investigated and may require an administrative action to fix them up. 🩺
  • BANNED — CTDB is not behaving as designed on this node. Like, maybe it failed too many recovery attempts? 😭 Such nodes get banned from participating in the cluster for a configurable time period before they try to rejoin. A banned node does not host public addresses to provide services. Every banned node should be investigated and may require an administrative action to rectify. Not the iconic behavior we wanted.
  • STOPPED — this node has been administratively excluded from the cluster. A stopped node does not participate in the cluster and does not host public addresses to provide services. This state is perfect for when you're doing maintenance on a node. Self-care era. 🧖
  • PARTIALLYONLINE — a node that's partially online participates in the cluster just like a healthy (OK) node would. Some interfaces that serve public addresses are down, but at least one interface is up. See also ctdb ifaces. Half up, still thriving. 🦋

🌈 CAPABILITIES 🌈

Cluster nodes can have a few different capabilities enabled, and here's the whole rundown:

  • LEADER — says that a node can become the CTDB cluster leader. The current leader gets decided via an election held by all the active nodes that have this capability. Default is YES. Democracy but make it CTDB.
  • LMASTER — says that a node can be the location master (LMASTER) for database records. The LMASTER always knows which node has the latest copy of a record in a volatile database. Default is YES.

Both the LEADER and LMASTER capabilities can be disabled when CTDB is used to create a cluster spanning across WAN links. In that case, CTDB acts as a WAN accelerator. No cap. 🚀

🔥 LVS 🔥

LVS is a mode where CTDB presents one single IP address for the entire cluster. It's the alternative to using public IP addresses and round-robin DNS to loadbalance clients across the cluster. Think less "spread the love around" and more "one door, everyone comes through it."

It's similar to using a layer-4 loadbalancing switch, but with some restrictions. Trade-offs, bestie, you know how it is.

One extra LVS public address gets assigned on the public network to each LVS group. Each LVS group is a set of nodes in the cluster that presents the same LVS address public address to the outside world. Normally there'd just be one LVS group spanning the whole entire cluster, but when one CTDB cluster spans multiple physical sites it might be useful to have one LVS group for each site. You can have multiple LVS groups in a cluster, but each node can only be a member of one LVS group. Pick your lane. 👯

Client access to the cluster is load-balanced across the HEALTHY nodes in an LVS group. If no HEALTHY nodes exist then all nodes in the group get used, regardless of their health status. CTDB will, however, never load-balance LVS traffic to nodes that are BANNED, STOPPED, DISABLED or DISCONNECTED. Periodt. The ctdb lvs command shows which nodes are currently being load-balanced across.

In each LVS group, CTDB selects one of the nodes to be the LVS leader. This node receives all the traffic from clients coming in to the LVS public address and multiplexes it across the internal network to one of the nodes that LVS is using. When responding to the client, that node sends the data back directly to the client, bypassing the LVS leader node entirely. Sneaky but efficient. The command ctdb lvs leader shows which node is the current LVS leader.

The path used for a client I/O is:

  1. Client sends request packet to LVS leader.
  2. LVS leader passes the request on to one node across the internal network.
  3. Selected node processes the request.
  4. Node responds back to client.

So basically all incoming traffic to the cluster passes through one physical node, which limits scalability. You can send more data to the LVS address than one physical node can multiplex. That means you should not use LVS if your I/O pattern is write-intensive, because you'll be limited by the available network bandwidth that node can handle. 😬 LVS does work very well for read-intensive workloads, though — where only smallish READ requests go through the LVS leader bottleneck and the majority of the traffic volume (the data in the read replies) goes straight from the processing node back to the clients. For read-intensive i/o patterns you can achieve very high throughput rates in this mode. Read girlies stay winning. 📚

Note: you can use LVS and public addresses at the same time. Both. At once. Multitasking queen.

If you use LVS, you must have a permanent address configured for the public interface on each node. This address must be routable, and the cluster nodes must be configured so that all traffic back to client hosts is routed through this interface. This is also required in order to allow samba/winbind on the node to talk to the domain controller. This LVS IP address can not be used to initiate outgoing traffic. Important!! 🚨

Make sure that the domain controller and the clients are reachable from a node before you enable LVS. Also ensure that outgoing traffic to these hosts is routed out through the configured public interface.

🌸 Configuration 🌸

To activate LVS on a CTDB node you must specify the CTDB_LVS_PUBLIC_IFACE, CTDB_LVS_PUBLIC_IP and CTDB_LVS_NODES configuration variables. CTDB_LVS_NODES specifies a file containing the private address of all nodes in the current node's LVS group.

Example:

CTDB_LVS_PUBLIC_IFACE=eth1
CTDB_LVS_PUBLIC_IP=10.1.1.237
CTDB_LVS_NODES=/etc/ctdb/lvs_nodes

Example /etc/ctdb/lvs_nodes:

192.168.1.2
192.168.1.3
192.168.1.4

Normally any node in an LVS group can act as the LVS leader. Nodes that are highly loaded due to other demands maybe flagged with the "follower-only" option in the CTDB_LVS_NODES file to limit the LVS functionality of those nodes.

LVS nodes file that excludes 192.168.1.4 from being the LVS leader node:

192.168.1.2
192.168.1.3
192.168.1.4 follower-only

🫶 TRACKING AND RESETTING TCP CONNECTIONS 🫶

CTDB tracks TCP connections from clients to public IP addresses, on known ports. When an IP address moves from one node to another, all existing TCP connections to that IP address get reset. The node taking over this IP address will also send gratuitous ARPs (for IPv4, or neighbour advertisement, for IPv6). This lets clients reconnect quickly instead of waiting for TCP timeouts, which can be very long. Nobody has time for that. 💅

It is important that established TCP connections do not survive a release and take of a public IP address on the same node. Such connections can get out of sync with sequence and ACK numbers, potentially causing a disruptive ACK storm. Yeah. An ACK storm. Sounds fake, is not fake. Stay safe out there. 🌩️

🌸 NAT GATEWAY 🌸

NAT gateway (NATGW, bestie, it's got a nickname) is an optional feature that sets up fallback routing for nodes. Why do we care? Because it lets cluster nodes reach out to external services — think DNS, AD, NIS and LDAP — even when they're not hosting any public addresses (y'know, like when they're unhealthy 🙈).

It also comes in clutch at node startup, because CTDB marks nodes as UNHEALTHY until they've passed a "monitor" event. So NAT gateway dodges the classic "chicken and egg" situation where a node needs external service access just to become healthy. Iconic problem-solving fr fr.

Another way to solve this? Give every node an extra static IP address on a public interface. Simpler, sure — but it burns one extra IP address per node, while NAT gateway generally only uses one extra IP address total. Efficiency ate 💅

🦋 Operation

One extra NATGW public address gets assigned on the public network to each NATGW group. And what's a NATGW group? It's a set of nodes in the cluster that shares the same NATGW address to talk to the outside world. Normally there'd be just one NATGW group covering the whole entire cluster, but when one CTDB cluster spans multiple physical sites, it might be useful to have one NATGW group per site. Multitasking queen behavior.

There can be multiple NATGW groups in a cluster, but each node can only be a member of one NATGW group. One group, one node, no polyamory allowed bestie.

Inside each NATGW group, CTDB picks one node to be the NATGW leader and the rest are NATGW followers. NATGW followers build a fallback default route to the NATGW leader through the private network. When a NATGW follower hosts no public IP addresses, it will use this route for outbound connections. Meanwhile the NATGW leader hosts the NATGW public IP address and routes outgoing connections from follower nodes via this IP address. It also establishes a fallback default route. Leader energy 👑

🎀 Configuration

NATGW is usually configured like this example configuration:

CTDB_NATGW_NODES=/etc/ctdb/natgw_nodes
CTDB_NATGW_PRIVATE_NETWORK=192.168.1.0/24
CTDB_NATGW_PUBLIC_IP=10.0.0.227/24
CTDB_NATGW_PUBLIC_IFACE=eth0
CTDB_NATGW_DEFAULT_GATEWAY=10.0.0.1

Normally any node in a NATGW group can act as the NATGW leader. But some configurations have special nodes that lack connectivity to a public network. In those cases, those nodes can be flagged with the "follower-only" option in the CTDB_NATGW_NODES file to limit the NATGW functionality of those nodes. Setting boundaries, we love to see it ✨

See the NAT GATEWAY section in ctdb-script.options(5) for more details of NATGW configuration.

💖 Implementation details

When NATGW functionality is in use, one of the nodes gets selected to act as a NAT gateway for all the other nodes in the group whenever they need to talk to external services. The NATGW leader is chosen to be a node that is most likely to have usable networks. Reliability check passed, slay.

The NATGW leader hosts the NATGW public IP address CTDB_NATGW_PUBLIC_IP on the configured public interfaces CTDB_NATGW_PUBLIC_IFACE and acts as a router, masquerading outgoing connections from follower nodes via this IP address. If CTDB_NATGW_DEFAULT_GATEWAY is set then it also establishes a fallback default route to the configured this gateway with a metric of 10. Why metric 10 specifically? So it can co-exist with other default routes that may be available. Lowkey genius.

A NATGW follower establishes its fallback default route to the NATGW leader via the private network CTDB_NATGW_PRIVATE_NETWORK with a metric of 10. This route is used for outbound connections when no other default route is available — because the node hosts no public addresses, obviously. A metric 10 routes is used so that it can co-exist with other default routes that may be available when the node is hosting public addresses. Sharing the road, no drama 🫶

CTDB_NATGW_STATIC_ROUTES can be used to have NATGW create more specific routes instead of just default routes.

All of this is implemented in the 11.natgw eventscript. Please see the eventscript file and the NAT GATEWAY section in ctdb-script.options(5) for more details.

📡 POLICY ROUTING 📡

Policy routing is an optional CTDB feature to support complex network topologies. Public addresses may be spread across several different networks (or VLANs) and it may not be possible to route packets from these public addresses via the system's default route. So CTDB has support for policy routing via the 13.per_ip_routing eventscript. This allows routing to be specified for packets sourced from each public address. Routes get added and removed as CTDB moves public addresses between nodes. It's giving ✨traffic choreography✨.

💜 Configuration variables

There are 4 configuration variables related to policy routing: CTDB_PER_IP_ROUTING_CONF, CTDB_PER_IP_ROUTING_RULE_PREF, CTDB_PER_IP_ROUTING_TABLE_ID_LOW, CTDB_PER_IP_ROUTING_TABLE_ID_HIGH. See the POLICY ROUTING section in ctdb-script.options(5) for more details.

🌈 Configuration

The format of each line of CTDB_PER_IP_ROUTING_CONF is:

<public_address> <network> [ <gateway> ]

Leading whitespace is ignored and arbitrary whitespace may be used as a separator. Lines that have a "public address" item that doesn't match an actual public address are ignored. Which means — plot twist — comment lines can be added using a leading character such as '#', since this will never match an IP address. Sneaky, but legal 💅

A line without a gateway indicates a link local route.

For example, consider the configuration line:

192.168.1.99   192.168.1.0/24

If the corresponding public_addresses line is:

192.168.1.99/24     eth2,eth3

CTDB_PER_IP_ROUTING_RULE_PREF is 100, and CTDB adds the address to eth2, then the following routing information is added:

ip rule add from 192.168.1.99 pref 100 table ctdb.192.168.1.99
ip route add 192.168.1.0/24 dev eth2 table ctdb.192.168.1.99

This causes traffic from 192.168.1.99 to 192.168.1.0/24 go via eth2. Path decided, no take-backs 🦋

The ip rule command will show (something like — depending on other public addresses and other routes on the system):

0:        from all lookup local
100:      from 192.168.1.99 lookup ctdb.192.168.1.99
32766:    from all lookup main
32767:    from all lookup default

ip route show table ctdb.192.168.1.99 will show:

192.168.1.0/24 dev eth2 scope link

The usual use for a line containing a gateway is to add a default route corresponding to a particular source address. Consider this line of configuration:

192.168.1.99   0.0.0.0/0 192.168.1.1

In the situation described above this will cause an extra routing command to be executed:

ip route add 0.0.0.0/0 via 192.168.1.1 dev eth2 table ctdb.192.168.1.99

With both configuration lines, ip route show table ctdb.192.168.1.99 will show:

192.168.1.0/24 dev eth2 scope link
default via 192.168.1.1 dev eth2

🌸 Sample configuration

Here is a more complete example configuration. Feast your eyes 😭

/etc/ctdb/public_addresses:

192.168.1.98   eth2,eth3
192.168.1.99   eth2,eth3

/etc/ctdb/policy_routing:

192.168.1.98 192.168.1.0/24
192.168.1.98 192.168.200.0/24 192.168.1.254
192.168.1.98 0.0.0.0/0   192.168.1.1
192.168.1.99 192.168.1.0/24
192.168.1.99 192.168.200.0/24 192.168.1.254
192.168.1.99 0.0.0.0/0   192.168.1.1

The routes local packets as expected, the default route is as previously discussed, but packets to 192.168.200.0/24 are routed via the alternate gateway 192.168.1.254. Detour era 💫

🔔 NOTIFICATIONS 🔔

When certain state changes occur in CTDB, it can be configured to perform arbitrary actions via notifications. For example, sending SNMP traps or emails when a node becomes unhealthy or similar. Your cluster is basically posting status updates on its story 🫶

The notification mechanism runs all executable files ending in ".script" in /etc/ctdb/events/notification/, ignoring any failures and continuing to run all files. No quitter energy here — it runs every single one.

CTDB currently generates notifications after CTDB changes to these states:

init
setup
startup
healthy
unhealthy

📊 LOG LEVELS 📊

Valid log levels, in increasing order of verbosity, are:

ERROR
WARNING
NOTICE
INFO
DEBUG

So ERROR is the quiet one and DEBUG is the one that will NOT stop talking. We've all met a DEBUG 💀

🌐 REMOTE CLUSTER NODES 🌐

It is possible to have a CTDB cluster that spans across a WAN link. For example where you have a CTDB cluster in your datacentre but you also want to have one additional CTDB node located at a remote branch site. This is similar to how a WAN accelerator works but with the difference that while a WAN-accelerator often acts as a Proxy or a MitM, in the ctdb remote cluster node configuration the Samba instance at the remote site IS the genuine server, not a proxy and not a MitM, and thus provides 100% correct CIFS semantics to clients. No impersonators, the real one showed up ✨

See the cluster as one single multihomed samba server where one of the NICs (the remote node) is very far away.

NOTE: This does require that the cluster filesystem you use can cope with WAN-link latencies. Not all cluster filesystems can handle WAN-link latencies! Whether this will provide very good WAN-accelerator performance or it will perform very poorly depends entirely on how optimized your cluster filesystem is in handling high latency for data and metadata operations. Read that again bestie, it's a whole vibe check for your filesystem 😭

To activate a node as being a remote cluster node you need to set the following two parameters in /etc/ctdb/ctdb.conf for the remote node:

[legacy]
  lmaster capability = false
  leader capability = false

Verify with the command "ctdb getcapabilities" that that node no longer has the leader or the lmaster capabilities. Double-check always, no cap 🎀

👀 SEE ALSO 👑

✍️ AUTHOR ✍️

This documentation was written by Ronnie Sahlberg, Amitay Isaacs, Martin Schwenke. Legends, every single one 💜

Copyright © 2007 Andrew Tridgell, Ronnie Sahlberg

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, see http://www.gnu.org/licenses.

and that's the whole page bestie, we made it to the end 🎉 this was ctdb from the CTDB(7) manual, dated 09/13/2026 — go forth and cluster responsibly, mwah 💅💖

📦 where this page comes from