CTDB-SCRIPT.OPTIO(5)

📦
version 4.24.7 · 17 versions
, in ubuntu stonking

✨ NAME ✨

ctdb-script.options(5) — CTDB scripts configuration files 💖 okay bestie, that's literally the NAME section, and it's serving CTDB - clustered TDB database energy. no cap. 🫶

🌸 DESCRIPTION 🌸

📍 Location 📍

okay so each CTDB script has 2 possible locations for its configuration options, and you're gonna want the tea on both:

  • /etc/ctdb/script.options — this is a catch-all global file for general purpose scripts and for options that are used in multiple event scripts. it's giving shared group-chat energy 🎀
  • SCRIPT.options — that is, options for SCRIPT are placed in a file alongside the script, with a .script suffix added. this style is usually recommended for event scripts, main character energy fr fr 👑
bestie, real talk: options in this script-specific file override those in the global file. periodt. 💅

📝 Contents 📝

vibe check: these files should include simple shell-style variable assignments and shell-style comments. keep it cute, keep it shell-y 🐚✨

📈 Monitoring Thresholds 📈

event scripts can monitor resources or services. when a problem is detected, it may be better to warn about a problem rather than to immediately fail monitoring and mark a node as unhealthy. CTDB provides support for event scripts to do threshold-based monitoring. iconic behavior 🌈

a threshold setting looks like WARNING_THRESHOLD[:ERROR_THRESHOLD]. if the number of problems is ≥ WARNING_THRESHOLD, then the script will log a warning and continue. if the number of problems is ≥ ERROR_THRESHOLD, then the script will log an error and exit with failure, causing monitoring to fail. not the monitoring fail 😭 💀

note that ERROR_THRESHOLD is optional, and follows the optional colon (:) separator. the : is literally the gatekeeper 🫶

🌐 NETWORK CONFIGURATION 🌐

10.interface 🎀

okay so this event script is the one doing public IP address release and takeover, plus it keeps an eye on the interfaces that those public IP addresses live on. multitasking queen 💅

CTDB_KILLTCP_USE_SS_KILL=yes|try|no

Do we use ss -K / --kill to reset incoming TCP connections to public IP addresses while releaseip is happening? that's the whole question bestie.

So the classic CTDB way of resetting incoming TCP connections during releaseip is its own custom ctdb_killtcp command. and it's lowkey doing network trickery for every single connection: send a "tickle ACK", capture the reply to extract the TCP sequence number, then send a reset (with the correct sequence number in it). iconic but extra ✨

ss -K has been supported in ss since iproute 4.5 in March 2016, and in the Linux kernel since 4.4 in December 2015. but bestie, the kernel config item it needs, CONFIG_INET_DIAG_DESTROY, is disabled by default. it's been enabled in Debian kernels since ~2017 and in Ubuntu since at least 18.04, and it's only recently gotten enabled in distributions such as RHEL. and there seems to be no way, not even running ss -K, to figure out whether it's supported 💀 so using this feature has to be configurable, periodt. when it is available, it should be the fastest, most reliable way of killing connections.

The supported values are:

  • yes — use ss -K and make no other attempt to kill any remaining connections. this is sane on modern Linux distributions that are guaranteed to have CONFIG_INET_DIAG_DESTROY enabled.
  • try — attempt to use ss -K and fall back to ctdb_killtcp for any remaining connections. this one's a good value when ss supports the -K option but you're not sure whether CONFIG_INET_DIAG_DESTROY is enabled 🙈
  • no — never even attempt ss -K. rely only on ctdb_killtcp.

Default is "no".

CTDB_PARTIALLY_ONLINE_INTERFACES=yes|no

Should one or more offline interfaces make a monitor event fail if there are other interfaces that are up? if this is "yes" and a node has some interfaces that are down, then ctdb status will display the node as "PARTIALLYONLINE" 👑

heads up bestie: CTDB_PARTIALLY_ONLINE_INTERFACES=yes is not generally compatible with NAT gateway or LVS. NAT gateway relies on the interface configured by CTDB_NATGW_PUBLIC_IFACE being up, and LVS relies on CTDB_LVS_PUBLIC_IFACE being up. CTDB does not check whether these options are set in an incompatible way, so you gotta be careful and actually understand the interaction 🫶

Default is "no".

11.natgw 🌈

this one provides CTDB's NAT gateway functionality. slay.

NAT gateway is used to configure fallback routing for nodes when they don't host any public IP addresses. for example, it lets unhealthy nodes still reliably talk to external infrastructure. one node in a NAT gateway group gets designated as the NAT gateway leader node, and the other (follower) nodes get configured with fallback routes via that leader. for more info, see the NAT GATEWAY section in ctdb(7) ✨

CTDB_NATGW_DEFAULT_GATEWAY=IPADDR

IPADDR is an alternate network gateway to use on the NAT gateway leader node. if it's set, a fallback default route is added via this network gateway.

No default. setting this variable is optional — if it's not set then no route is created on the NAT gateway leader node.

CTDB_NATGW_NODES=FILENAME

FILENAME contains the list of nodes that belong to the same NAT gateway group.

File format:

IPADDR [follower-only]

IPADDR is the private IP address of each node in the NAT gateway group.

If "follower-only" is specified then that node can not be the NAT gateway leader node. in that case CTDB_NATGW_PUBLIC_IFACE and CTDB_NATGW_PUBLIC_IP are optional and unused 💅

No default, usually /etc/ctdb/natgw_nodes when enabled.

CTDB_NATGW_PRIVATE_NETWORK=IPADDR/MASK

IPADDR/MASK is the private sub-network that is internally routed via the NAT gateway leader node. this is usually the private network that's used for node addresses.

No default.

CTDB_NATGW_PUBLIC_IFACE=IFACE

IFACE is the network interface on which the CTDB_NATGW_PUBLIC_IP will be configured.

No default.

CTDB_NATGW_PUBLIC_IP=IPADDR/MASK

IPADDR/MASK is the IP address used for outgoing traffic (the traffic originating from CTDB_NATGW_PRIVATE_NETWORK) on the NAT gateway leader node. it must not be a configured public IP address 🚫

No default.

CTDB_NATGW_STATIC_ROUTES=IPADDR/MASK[@GATEWAY] ...

each IPADDR/MASK identifies a network or host that NATGW should create a fallback route to, instead of creating a single default route. this is the move when there's already a default route, via an interface that can't reach required infrastructure, that overrides the NAT gateway default route.

If GATEWAY is specified then the corresponding route on the NATGW leader node will go via GATEWAY. such routes are created even if CTDB_NATGW_DEFAULT_GATEWAY is not specified. if GATEWAY is not specified for some networks, then routes are only created on the NATGW leader node for those networks if CTDB_NATGW_DEFAULT_GATEWAY is specified.

use this with care, bestie — it can cause traffic to unnecessarily double-hop through the NAT gateway leader, even when a node is hosting public IP addresses. each specified network or host should probably have a corresponding automatically created link route or static route to avoid this 🙈

No default.

Example

CTDB_NATGW_NODES=/etc/ctdb/natgw_nodes
CTDB_NATGW_PRIVATE_NETWORK=192.168.1.0/24
CTDB_NATGW_DEFAULT_GATEWAY=10.0.0.1
CTDB_NATGW_PUBLIC_IP=10.0.0.227/24
CTDB_NATGW_PUBLIC_IFACE=eth0

A variation that ensures infrastructure (ADS, DNS, ...) directly attached to the public network (10.0.0.0/24) is always reachable would look like this:

CTDB_NATGW_NODES=/etc/ctdb/natgw_nodes
CTDB_NATGW_PRIVATE_NETWORK=192.168.1.0/24
CTDB_NATGW_PUBLIC_IP=10.0.0.227/24
CTDB_NATGW_PUBLIC_IFACE=eth0
CTDB_NATGW_STATIC_ROUTES=10.0.0.0/24

Note that CTDB_NATGW_DEFAULT_GATEWAY is not specified. 🫶

13.per_ip_routing 🦋

this one gives you CTDB's policy routing functionality.

so a node running CTDB might be a component of a complex network topology. in particular, public addresses can be spread across several different networks (or VLANs) and it may not be possible to route packets from those public addresses via the system's default route 😭 therefore CTDB has support for policy routing via the 13.per_ip_routing eventscript. this lets routing be specified for packets sourced from each public address. the routes get added and removed as CTDB moves public addresses between nodes. main character energy fr fr ✨

For more information, see the POLICY ROUTING section in ctdb(7).

CTDB_PER_IP_ROUTING_CONF=FILENAME

FILENAME contains elements for constructing the desired routes for each source address.

The special FILENAME value __auto_link_local__ indicates that no configuration file is provided and that CTDB should generate reasonable link-local routes for each public IP address.

File format:

              IPADDR DEST-IPADDR/MASK [GATEWAY-IPADDR]

No default, usually /etc/ctdb/policy_routing when enabled.

CTDB_PER_IP_ROUTING_RULE_PREF=NUM

NUM sets the priority (or preference) for the routing rules that CTDB adds.

This should be (strictly) greater than 0 and (strictly) less than 32766. A priority of 100 is recommended, unless that conflicts with a priority already in use on the system. See ip(8) for more details.

CTDB_PER_IP_ROUTING_TABLE_ID_LOW=LOW-NUM, CTDB_PER_IP_ROUTING_TABLE_ID_HIGH=HIGH-NUM

CTDB figures out a unique routing table number to use for the routing related to each public address. LOW-NUM and HIGH-NUM indicate the minimum and maximum routing table numbers that get used.

ip(8) uses some reserved routing table numbers below 255. so CTDB_PER_IP_ROUTING_TABLE_ID_LOW should be (strictly) greater than 255 ⚠️

CTDB uses the standard file /etc/iproute2/rt_tables to maintain a mapping between the routing table numbers and labels. the label for a public address ADDR will look like ctdb.addr. which means the associated rules and routes are easy to read (and manipulate) 💅

No default, usually 1000 and 9000.

Example

CTDB_PER_IP_ROUTING_CONF=/etc/ctdb/policy_routing
CTDB_PER_IP_ROUTING_RULE_PREF=100
CTDB_PER_IP_ROUTING_TABLE_ID_LOW=1000
CTDB_PER_IP_ROUTING_TABLE_ID_HIGH=9000

91.lvs 💖

this one provides CTDB's LVS functionality.

For a general description see the LVS section in ctdb(7).

CTDB_LVS_NODES=FILENAME

FILENAME contains the list of nodes that belong to the same LVS group.

File format:

IPADDR [follower-only]

IPADDR is the private IP address of each node in the LVS group.

If "follower-only" is specified then that node can not be the LVS leader node. in that case CTDB_LVS_PUBLIC_IFACE and CTDB_LVS_PUBLIC_IP are optional and unused ✨

No default, usually /etc/ctdb/lvs_nodes when enabled.

CTDB_LVS_PUBLIC_IFACE=INTERFACE

INTERFACE is the network interface that clients will use to connect to CTDB_LVS_PUBLIC_IP. this is optional for follower-only nodes. No default.

CTDB_LVS_PUBLIC_IP=IPADDR

CTDB_LVS_PUBLIC_IP is the LVS public address. No default. 💅

🎀 SERVICE CONFIGURATION 🎀

CTDB can be configured to manage and/or monitor various NAS (and other) services via its eventscripts.

In the simplest case CTDB will manage a service. That means: the service gets started and stopped right along with CTDB, CTDB monitors the service, and CTDB does any required reconfiguration of the service when public IP addresses are failed over. Like, full-service girlfriend behaviour. 💅

🌈 20.multipathd 🌈

Provides CTDB's Linux multipathd service management.

It can monitor multipath devices to ensure that active paths are available.

  • CTDB_MONITOR_MPDEVICES=MP-DEVICE-LIST — MP-DEVICE-LIST is a list of multipath devices for CTDB to monitor?
    • No default.
not the question mark at the end of that description, bestie, it's literally written like that, i'm just the messenger 🫶

🦋 31.clamd 🦋

This event script provide CTDB's ClamAV anti-virus service management.

This eventscript is not enabled by default. Use ctdb enablescript to enable it.

  • CTDB_CLAMD_SOCKET=FILENAME — FILENAME is the socket to monitor ClamAV.
    • No default.

🍓 40.vsftpd 🍓

Provides CTDB's vsftpd service management.

  • CTDB_VSFTPD_MONITOR_THRESHOLDS=THRESHOLDS — THRESHOLDS indicates how many consecutive monitoring attempts need to report that vsftpd is not listening on TCP port 21 before a warning is logged and before monitoring fails. See the Monitoring Thresholds for a description of how monitoring thresholds work.
    • Default is 1:2.

✨ 48.netbios ✨

Provides CTDB's NetBIOS service management.

  • CTDB_SERVICE_NMB=SERVICE — Distribution specific SERVICE for managing nmbd.
    • Default is distribution-dependant.

💜 49.winbind 💜

Provides CTDB's Samba winbind service management.

  • CTDB_SERVICE_WINBIND=SERVICE — Distribution specific SERVICE for managing winbindd.
    • Default is "winbind".
  • CTDB_SAMBA_INTERFACES_FILE=FILENAME — Generates FILENAME, containing an smb.conf snippet with an interfaces setting that includes interfaces for configured CTDB public IP addresses. This file then needs to be explicitly included in smb.conf.

For example, if public IP addresses are defined on interfaces eth0 and eth1, and this is set to /etc/samba/interfaces.conf, then that file will contain the following before smbd is started:

bind interfaces only = yes
interfaces = lo eth0 eth1

This can be useful for limiting the interfaces used by SMB multichannel.

  • Default is to not generate a file.
  • CTDB_SAMBA_INTERFACES_EXTRA=INTERFACE-LIST — A space separated list to provide additional interfaces to bind.
    • Default is empty — no extra interfaces are added.

🔥 50.samba 🔥

Provides the core of CTDB's Samba file service management.

  • CTDB_SAMBA_CHECK_PORTS=PORT-LIST — When monitoring Samba, check TCP ports in space-separated PORT-LIST.
    • Default is to monitor ports that Samba is configured to listen on.
  • CTDB_SAMBA_SKIP_SHARE_CHECK=yes|no — As part of monitoring, should CTDB skip the check for the existence of each directory configured as share in Samba. This may be desirable if there is a large number of shares.
    • Default is no.
  • CTDB_SERVICE_SMB=SERVICE — Distribution specific SERVICE for managing smbd.
    • Default is distribution-dependant.

🧚 60.nfs 🧚

This event script provides CTDB's NFS service management.

This includes parameters for the kernel NFS server. Alternative NFS subsystems (such as NFS-Ganesha[1]) can be integrated using CTDB_NFS_CALLOUT.

  • CTDB_NFS_CALLOUT=COMMAND — COMMAND specifies the path to a callout to handle interactions with the configured NFS system, including startup, shutdown, monitoring.
    • Default is the included nfs-linux-kernel-callout.
  • CTDB_NFS_CHECKS_DIR=DIRECTORY — Specifies the path to a DIRECTORY containing files that describe how to monitor the responsiveness of NFS RPC services. See the README file for this directory for an explanation of the contents of these "check" files.

CTDB_NFS_CHECKS_DIR can be used to point to different sets of checks for different NFS servers.

One way of using this is to have it point to, say, /etc/ctdb/nfs-checks-enabled.d and populate it with symbolic links to the desired check files. This avoids duplication and is upgrade-safe.

  • Default is /etc/ctdb/nfs-checks.d, which contains NFS RPC checks suitable for Linux kernel NFS.
  • CTDB_NFS_EXPORTS_FILE=FILE — Set FILE as the path of the file containing NFS exports, for use by the NFS callout (see CTDB_NFS_CALLOUT, above). This is used for share checks when CTDB_NFS_SKIP_SHARE_CHECK is not set to "yes". This is most useful with NFS-Ganesha, since it supports configuration include files and exports may be stored in a separate file.
    • Default is /var/lib/nfs/etab for nfs-linux-kernel-callout, /etc/ganesha/ganesha.conf for nfs-ganesha-callout.
  • CTDB_NFS_SHARED_STATE_DIR=DIRECTORY — DIRECTORY where clustered NFS shared state will be located. DIRECTORY should be in a cluster filesystem that is shared between the nodes. No default.
  • CTDB_NFS_SKIP_SHARE_CHECK=yes|no — As part of monitoring, should CTDB skip the check for the existence of each directory exported via NFS. This may be desirable if there is a large number of exports.
    • Default is no.
  • CTDB_RPCINFO_LOCALHOST=IPADDR|HOSTNAME — IPADDR or HOSTNAME indicates the address that rpcinfo should connect to when doing rpcinfo check on IPv4 RPC service during monitoring. Optimally this would be "localhost". However, this can add some performance overheads.
    • Default is "127.0.0.1".
  • CTDB_RPCINFO_LOCALHOST6=IPADDR|HOSTNAME — IPADDR or HOSTNAME indicates the address that rpcinfo should connect to when doing rpcinfo check on IPv6 RPC service during monitoring. Optimally this would be "localhost6" (or similar). However, this can add some performance overheads.
    • Default is "::1".
  • CTDB_STATD_CALLOUT_SHARED_STORAGE=LOCATION — LOCATION where NFSv3 statd state will be stored. Valid values are:
    • persistent_db[:TDB] — Data is queued to local storage and then dequeued to TDB during monitor events. This means there is a window where locking state may be lost. However, this works around performance limitations in CTDB's persistent database handling.

If :TDB is omitted then TDB defaults to ctdb_statd_callout.tdb.

  • shared_dir[:DIRECTORY] — DIRECTORY is a directory in a cluster filesystem that is shared between the nodes. If DIRECTORY is relative (i.e. does not start with /) then it is appended to CTDB_NFS_SHARED_STATE_DIR. If :DIRECTORY is omitted then DIRECTORY defaults to statd.

Using a shared directory may result in performance and/or stability problems. rpc.statd is single-threaded and its HA callout is called synchronously, causing any latency introduced by the callout to be cumulative. Stability issues are most likely if thousands of clients reclaim locks after failover and use of the cluster filesystem introduces too much additional latency. Too much latency in the HA callout may cause rpc.statd to fail health monitoring.

  • none — No cluster-aware handling of NFSv3 statd state is done. NFSv3 lock reclaim will not occur and applications that use locking over NFSv3 are likely to lose or corrupt data. 😭

This should be used with care and only in the case where no applications are using POSIX locks in NFSv3 mounts. It should probably be considered an option to test the latency of statd_callout, without including any storage costs.

👑 70.iscsi 👑

Provides CTDB's Linux iSCSI tgtd service management.

  • CTDB_START_ISCSI_SCRIPTS=DIRECTORY — DIRECTORY on shared storage containing scripts to start tgtd for each public IP address.
    • No default.

⭐ DATABASE HANDLING ⭐

CTDB checks the consistency of databases during startup and provides a facility to backup persistent databases.

🎀 95.database 🎀

  • CTDB_MAX_CORRUPT_DB_BACKUPS=NUM — NUM is the maximum number of volatile TDB database backups to be kept (for each database) when a corrupt database is found during startup. Volatile TDBs are zeroed during startup so backups are needed to debug any corruption that occurs before a restart.
    • Default is 10.
  • CTDB_PERSISTENT_DB_BACKUP_DIR=DIRECTORY — Create a daily backup tarball for all persistent TDBs in DIRECTORY. Note that DIRECTORY must exist or no backups will be created.

Given that persistent databases are fully replicated, duplication is avoid by only creating backups on the current leader node. To maintain a complete, single set of backups, it makes sense for DIRECTORY to be in a cluster filesystem.

This creates the backup from the monitor event, which should be fine because backing up persistent databases is a local operation. Users who do not wish do create backups during the monitor event can choose not to use this option and instead run /etc/ctdb/ctdb-backup-persistent-tdbs.sh -l DIRECTORY on all nodes using a cron(8) job, which will also need to manually manage backup pruning.

  • No default. No daily backups are created.
  • CTDB_PERSISTENT_DB_BACKUP_LIMIT=COUNT — Keep at most COUNT backups in CTDB_PERSISTENT_DB_BACKUP_DIR. Note that if additional manual backups are created in this directory then these will count towards the limit.
    • Default is 14.

💾 SYSTEM RESOURCE MONITORING 💾

🩺 05.system

Okay bestie, this one's CTDB's filesystem and memory usage monitoring. It's literally the vibe check for your machine's resources. 💅

CTDB can experience seemingly random (performance and other) issues if system resources become too constrained. Like, it's not being dramatic — things just start acting up. Options in this section can be enabled to allow certain system resources to be checked. They allow warnings to be logged and nodes to be marked unhealthy when system resource usage reaches the configured thresholds. So basically: early warning system, so your cluster doesn't just collapse mid-party. 🙈

Some checks are enabled by default. It is recommended that these checks remain enabled or are augmented by extra checks. And no cap: there is no supported way of completely disabling the checks. Periodt. You can add more, you can tune them, but you cannot just turn them off. 💀

  • CTDB_MONITOR_FILESYSTEM_USAGE=FS-LIMIT-LIST — FS-LIMIT-LIST is a space-separated list of FILESYSTEM:WARN_LIMIT[:UNHEALTHY_LIMIT] triples indicating that warnings should be logged if the space used on FILESYSTEM reaches WARN_LIMIT%. If usage reaches UNHEALTHY_LIMIT then the node should be flagged unhealthy. Either WARN_LIMIT or UNHEALTHY_LIMIT may be left blank, meaning that check will be omitted. (So you can skip the warning and go straight to unhealthy, or warn and never flag, your call 🎀)

Default is to warn for each filesystem containing a database directory (volatile database directory, persistent database directory, state database directory) with a threshold of 90%.

  • CTDB_MONITOR_MEMORY_USAGE=MEM-LIMITS — MEM-LIMITS takes the form WARN_LIMIT[:UNHEALTHY_LIMIT] indicating that warnings should be logged if memory usage reaches WARN_LIMIT%. If usage reaches UNHEALTHY_LIMIT then the node should be flagged unhealthy. Either WARN_LIMIT or UNHEALTHY_LIMIT may be left blank, meaning that check will be omitted. Same energy as the filesystem one, just for RAM 🧠✨

Default is 80, so warnings will be logged when memory usage reaches 80%.

🐛 EVENT SCRIPT DEBUGGING 🐛

🔍 debug-hung-script.sh

When an eventscript is out here hanging and refusing to move on, this is the little detective that comes looking. 🕵️♀️

  • CTDB_DEBUG_HUNG_SCRIPT_STACKPAT=REGEXP — REGEXP specifies interesting processes for which stack traces should be logged when debugging hung eventscripts and those processes are matched in pstree output. REGEXP is an extended regexp so choices are separated by pipes ('|'). However, REGEXP should not contain parentheses. iykyk — just don't. See also the ctdb.conf(5) [event] "debug script" option.

Default is "exportfs|rpcinfo".

🫶 Tiny heads-up from the manpage gods: no parens in your REGEXP, bestie. Save yourself the confusion.

📁 FILES 📁

  • /etc/ctdb/script.options

👀 SEE ALSO 👀

✍️ AUTHOR ✍️

This documentation was written by Amitay Isaacs, Martin Schwenke. Absolute legends, we stan. 👑

Copyright © 2007 Andrew Tridgell, Ronnie Sahlberg.

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, see http://www.gnu.org/licenses.

📝 NOTES 📝

  1. NFS-Ganesha https://github.com/nfs-ganesha/nfs-ganesha/wiki

And that's a wrap on the whole page, bestie!! 🎉 You made it through every single option, no skimming, main character energy the entire way through. 💖

This lil masterpiece lives on the ctdb page, dated 09/13/2026, and its proper name is CTDB-SCRIPT.OPTIO(5). Go forth and configure responsibly — and remember: leave those default checks ON. 🌸✨🦋

📦 where this page comes from