DIRVISH.CONF(5)

📦
version 1.2.1 · 5 versions
, in debian bookworm, debian trixie, ubuntu jammy, ubuntu noble, ubuntu resolute

✨ dirvish.conf(5) ✨

okay bestie so this is the manual page for dirvish.conf — the dirvish configuration file. it's giving "the file that tells dirvish what to do" and honestly? iconic. let's get into it 💅

🌸 NAME 🌸

dirvish.conf — dirvish configuration file. that's it, that's the name, no cap 💖

💅 DESCRIPTION 💅

the dirvish.conf file is where you keep all the configuration info and the default values for dirvish. it's the main character of your backup setup fr.

the file format is honestly pretty simple. every option wants either a single value or a list of values, and unless it says otherwise you must specify it according to the type it expects.

single-value options go on one line, looking like this:

option: value

options that expect a list have to be written in a multi-line format like the one below, where the lines giving the values are indented by any kind of whitespace — even if you're only giving one single value:

option:
     value1
     value2
     .
     .
     .
     valueN

every value gets its own line. any leading and trailing whitespace just gets thrown out, periodt. options whose names start with a capital letter (like Foo) get discarded by dirvish itself, but support utilities are allowed to use them, iykyk. blank lines? ignored. 🙈

this simplistic format might let you make configuration errors, sure, but it also lets you declare arbitrary options that custom support scripts could use. so it's a tradeoff bestie, and dirvish picked flexibility 💁

a # starts a comment that runs to the end of the line.

on startup the dirvish utilities first load a master dirvish.conf file. /etc/dirvish.conf gets tried first, but if it's not there, /etc/dirvish/master.conf gets tried instead.

during installation dirvish may have been configured to expect the system-wide configuration files in some location other than /etc/dirvish.

multiple configuration files get loaded by the --config, --vault and --branch command-line options, plus the config: and client: configuration parameters. and to prevent looping, each configuration file can only be loaded once. no infinite loops in this house 🔥

🔥 DIRVISH OPTIONS 🔥

just like on the command line, each option may be specified any number of times. options that expect lists will accumulate all of their arguments, and for single-value options each new specification overrides the ones that came before it.

boolean values need to be specified as 1 or 0, or you can use SET or UNSET. heads up: some boolean values are set by default and you have to explicitly unset them if you don't want them.

each option down here is marked with one of:

  • (B) for Boolean
  • (S) for single value
  • (L) for list
  • (0) for other
  • SET option option ... (O) — set one or more boolean options.
  • UNSET option option ... (O) — unset one or more boolean options.
NOTE: the SET and UNSET directives do not use colons <:>. so no SET: bestie, that's not the vibe.
  • RESET option (O) — reset a list option so that it contains no values. this is how you start the specification of the option over from scratch.
NOTE: the RESET directive does not use a colon <:> either. 🫶
  • bank: (L) — specify paths to directories containing vaults.

a bank is a directory containing one or more vaults. the system supports multiple banks so that filesystem mount-points can be managed more effectively.

when a vault is specified, the banks get searched in list order until the vault is found. this way vaults can be moved between banks or added without having to update a master index. lowkey genius ✨

multiple bank: values will accumulate.

  • branch: branch_name (S) — specify a branch to use.

a branch is a sequence of images.

this also specifies a default value for reference:.

setting this in a config file may cause the command line option to be overridden. use branch-default: instead.

  • branch-default: branch_name (S) — specify a default branch to use.
  • client: [username@]client_name (S) — specify a client to back up.

setting this to the same value as hostname will cause dirvish to do a local copy and stay off the network. this automatically invokes whole-file.

the first time this parameter is set, /etc/dirvish/client_name or /etc/dirvish/client_name.conf will be loaded.

  • checksum: (B) — force the checksum comparison of file contents even when the inode fails to indicate that a change has occurred.

default value: 0

  • config: filename (S) — load configuration file.

similar to #include, filename or filename.conf will be loaded immediately.

if filename is a relative path it will be looked for in the vault and then the system-wide configuration directories.

  • devices: (B) — if this is unset, device special files will be excluded from backups.

you may need to unset this when doing backups where the client OS uses device numbers or types unsupported by the server OS, or where the presence of device nodes in the vault presents a security issue. 👀

  • exclude: (L) — specify a filename pattern to exclude.

patterns are based on shell glob with some enhancements.

see rsync(1) for more details.

multiple exclude: values will accumulate.

  • file-exclude: filename (S) — load a set of patterns from a file.

if filename is a relative path it will be looked for in the vault and then the system-wide configuration directories.

  • expire: expire_date (S) — specify a time for the image to expire.

this does not actually expire anything. what it does do is add an Expire: option to the image summary file with the absolute time appended, so that dirvish-expire can automate old image removal for you.

setting this in a config file may cause the command line option to be overridden. use expire-rule: and expire-default: instead.

see Time::ParseDate(3pm) for more details.

  • expire-default: expire_date (S) — specify a default expiration time.

this value will only be used if expire is not set and expire-rule doesn't have a match.

  • expire-rule: (L) — specify rules for expiration.

rules are specified similar to crontab or in Time::Periodformat.

see EXPIRE RULES for more details.

multiple expire-rule: values will accumulate.

  • image: image_name (S) — specify a name for the image.

image_name is passed through POSIX::strftime.

setting this in a config file may cause the command line option to be overridden. use image-default: instead.

see strftime(3) for more details.

  • image-default: image_name (S) — set the default image_name.

this value will only be used if image: is not set.

  • image-perm: octal_mode (S) — set the permissions for the image.

while the image is being created, the image directory permissions will be 0700. after completion it will be changed to octal_mode or 0755.

see chmod(1) and umask(2) for more details.

  • image-time: parsedate_expression (S) — time to use when creating the image name.

if an absolute time without a date is provided it will be forced into the past.

if this isn't set the current time will be used.

see Time::ParseDate(3pm) for more details.

  • image-temp: dirname (S) — temporary directory name to use for the new image. this lets you have images created with the same directory name each run, so that automatic processes can access them.

the next time an image is made on the branch, this option will cause the directory to be renamed to its official name.

  • index: none|text|gzip|bzip2 (S) — create an index file listing all the files in the image.

the index file will be created using find -ls, so the list will be in the same format as ls-dils with paths converted to reflect the source location.

if index is set to bzip2 or gzip (or a path to one), the index file will be compressed accordingly.

this index will be used by dirvish-locate to locate versions of files. see dirvish-locate(8) for more details.

  • init: (B) — create an initial image.

turning this on will prevent backups from being incremental.

  • log: text|gzip|bzip2 (S) — specify the format for the image log file.

if log is set to bzip2 or gzip (or a path to one), the log file will be compressed accordingly.

  • meta-perm: octal-mode (S) — set the permissions for the image meta-data files.

if this value is set, the permissions of the meta-data files in the image will be changed after the image is created. otherwise the active umask will prevail.

SECURITY NOTE: the log, index, and error files contain lists of files. it may be possible that filenames themselves may be, or may contain, confidential information, so uncontrolled access may constitute a security weakness. 🔒 be careful bestie.

see chmod(1) and umask(2) for more details.

  • numeric-ids: (B) — use numeric uid/gid values instead of looking up user/group names for setting permissions.

see rsync(1) for more details.

default value: 1

  • password-file: filepath (S) — specify a file containing the password for connection to an rsync daemon on the backup client.

this is not useful for remote shell passwords.

see --password-file in rsync(1) for more details.

  • permissions: (B) — preserve file permissions. if this is unset, permissions will not be checked or preserved.

with rsync version 2.5.6, not preserving permissions will break the linking. only unset this if you are running a later version of rsync.

see rsync(1) for more details.

default value: 1

  • pre-server: shell_command (S) — 👇 all four of these share the deets below
  • pre-client: shell_command (S) — 👇
  • post-client: shell_command (S) — 👇
  • post-server: shell_command (S) — 👇

execute shell_command on the client or server before or after making a backup.

the client commands are run on the client system using the remote shell command (see the rsh: parameter).

the order of execution is pre-server, pre-client, rsync, post-client, post-server. the shell_command will be passed through strftime(3) to allow date strings to be expanded. 🌈

each pre or post shell_command will be run with these environment variables set: DIRVISH_SERVER, DIRVISH_CLIENT, DIRVISH_SRC, DIRVISH_DEST and DIRVISH_IMAGE. the current directory will be DIRVISH_SRC on the client and DIRVISH_DEST on the server. if there are any exclude patterns defined, the pre-server shell command will also have the exclude file's path in DIRVISH_EXCLUDE, so it may read or modify the exclude list.

STDOUT from each shell_command will be written to the image log file. ✨

part 1 done, bestie — but we're not done yet, keep reading 💖

checked for real — every script's exit status gets a vibe check. Anything non-zero is giving failure and it gets logged. If the pre-server command flops, everything stops dead, no further action, periodt. If the pre-client command flops, the rsync never runs, but the post-server command (if you set one) still goes ahead.

Post shell_commands will also have DIRVISH_STATUS set to success, warning, error, or fatal error.

This is lowkey useful for so many things. Your client shell_commands can stop and start services so their files get backed up safely 🫶. You could use post-server: to schedule replication or a tape backup of the new image. Use your imagination bestie, the world is yours 💅.

  • reference: branch_name|image_name (S) — pick an existing image or a branch to build the new image from.

If you hand it a branch_name, the last existing image from that branch's history file gets used. Heads up: a branch takes precedence over an image of the same name 👑.

If you don't specify this, the branch name becomes the default value.

  • rsh: command (S) — your remote shell utility.

Use it to point at wherever ssh or rsh lives, and/or to pass extra options to that utility — like -p port for ssh when you want an alternate port number.

If you don't set it, ssh gets used.

This remote shell command is the default rsync transport and it's what runs any pre-client and post-client commands. Multipurpose icon 💫.

  • rsync: command (S) — path to the rsync executable on the server.
  • rsync-client: command (S) — path to the rsync executable on the client.
  • rsync-option: (L) — extra options for the rsync command, slay.

Only one option per list item, no bundling bestie.

This is your backstage pass to rsync features dirvish doesn't directly support. That said, where dirvish does support an rsync feature, it's probably better to use the dirvish-supplied mechanism for setting it 👀.

Multiple rsync-option: values accumulate.

  • sparse: (B) — try to handle sparse files efficiently so they eat less space in the vault 🍓.
NOTE: some filesystem types may have problems seeking over null regions. Beware, bestie.
  • speed-limit: Mbps (S) — set a maximum transfer rate.

This caps your network bandwidth usage. The value is in approximate Mega-bits per second, which lines up with how network transport specs are written. An adaptive algorithm is doing its thing, so the actual bandwidth may occasionally exceed Mbps, no cap on the cap sometimes 😭.

See --bwlimit in rsync(1) for more details.

  • stats: (B) — have rsync report transfer statistics 📊.

See rsync(1) for more details.

Default value: 1

  • summary: short|long (S) — pick your summary format.

A short summary only includes the final used values. A long summary includes all configuration values.

With long format, your custom options in the configuration files will show up in the summary ✨.

The default is short.

  • tree: path [alias] (S) — set a directory path on the client to back up.

If path is prefixed with a colon, the transfer happens from an rsync daemon on the client. Otherwise the transfer goes through a remote shell process.

The optional alias sets the path that appears in the index, so dirvish-locate reports paths consistent with common usage 🧚. This helps cut down confusion when you're dealing with users who aren't familiar with the physical topology of their network-provided files.

  • no-run: (B) — don't actually do anything.

Process all configuration files, options and tests, then produce a summary/configuration file on standard output and exit.

I genuinely can't think why you'd do this in a configuration file, but if you want to shoot yourself in the foot, be my guest 💀.

  • vault: vault (S) — pick the vault to store the image in.

Multiple vaults can share a filesystem, but a given vault cannot span filesystems. For filesystem purposes, the vault is the level of atomicity 🔒.

This will seldom be specified in a configuration file.

  • whole-file: (B) — transfer whole files instead of just the parts that changed.

This may be slightly faster for files that have more changed than stayed the same — think compressed or encrypted files. In most cases this is slower over the network but uses less CPU resources. It'll be faster if the transfers aren't over the network, or when the network is faster than the destination disk subsystem ⚡.

  • xdev: (B) — do not cross mount-points when traversing the tree on the client.
  • zxfer: (B) — enable compression on data-transfer 🗜️.

👑 SCHEDULING OPTIONS 👑

  • Dirvish: path (S) — location of the dirvish executable.

If it's not set, it defaults to dirvish.

  • Runall: (L) — specify branches to be scheduled for automated backups. Each value looks like this:
vault:branch [image_time]

If image_time is set here, it will be used.

This option can only be set in the master configuration file, and multiple values will accumulate 🗂️.

⏳ EXPIRE RULES ⏳

Expire rules is a list of rules used to figure out an expiration time for an image.

The last rule that matches is the one that applies, so list order is significant 👀. This lets rules be set in client, vault and branch configuration files to override rules set in the master configuration file without having to use RESET. In most cases it's better to use an expire-default: value than to define a rule that matches every possible time.

Each rule has a pattern expression that the current time gets compared against, followed by a date specifier in Time::ParseDate format. See Time::ParseDate(3pm) for more details.

A matching rule with an empty/missing date specifier, or one specifying never, results in no expiration 💖.

The time pattern expression may be in either crontab or Time::Period format. See crontab(5) and Time::Period(3pm) for more details.

The crontab-formatted patterns are converted to Time::Period format, so the limitations and extensions for the specification of option values of Time::Period apply to the crontab format too. Most notable: the days of the week are numbered 1-7 for sun-sat, so 0 is not a valid wday but sat is 🗓️.

Here are two equivalent examples of an expire-rule list.

expire-default: +5 weeks
expire-rule:

#MIN  HR    DOM   MON         DOW   EXPIRE
*     *     *     *           1     +3 months
*     *     1-7   *           su    +1 year
*     *     1-7   1,4,7,10    1     never
*     10-20 *     *           *     +10 days

or:

wd { sun }                          +3 months
wd { sun } md { 1-7 }               +1 year
wd { 1 } md { 1-7 } mo { 1,4,7,10 } never
hr { 10-20 }                        +10 days

This describes an aggressive retention schedule 🔥. If the nightly backup is made dated the 1st Sunday of each quarter, it's kept forever. The 1st Sunday of any other month is kept for 1 year. All other Sundays are kept for 3 months. The remaining nightlies are kept for 5 weeks. On top of that, if the backup is made between 10AM and 8PM it expires after 10 days. This would be appropriate for someone with a huge backup server who is so paranoid he makes two backups per day 😭. The other possibility for the hour spec would be for ad-hoc special backups to have a default that differs from the normal dailies.

It should be noted that all expiration rules will do is cause dirvish to put an Expire: option in the summary file. The dirvish-expire utility will have to be run to actually delete any expired images. Rules don't delete anything by themselves, bestie 💅.

📁 FILES 📁

  • /etc/dirvish/master.conf — alternate master configuration file.
  • /etc/dirvish.conf — master configuration file.
  • /etc/dirvish/client[.conf] — client configuration file.
  • bank/vault/dirvish/default[.conf] — default vault configuration file.
  • bank/vault/dirvish/branch[.conf] — branch configuration file.
  • bank/vault/dirvish/branch.hist — branch history file.
  • bank/vault/image/summary — image creation summary.
  • bank/vault/image/log — image creation log.
  • bank/vault/image/tree — actual image of source directory tree.
  • bank/vault/image/rsync_error — error output from rsync if errors or warnings were detected.

👀 SEE ALSO 👀

💖 AUTHOR 💖

Dirvish was created by J.W. Schultz of Pegasystems Technologies. Iconic. Legend behaviour fr fr 👑.

💀 BUGS 💀

Rsync version 2.5.6 has a bug so that unsetting the perms option will not disable testing for permissions. Disabling perms will break image linking. Not me warning you twice, this one's real 😭.

Options set in configuration files will override command line options that have been set before the file is read. This behaviour, while consistent, may confuse users. For this reason the more frequently used command line options have options paired with a default option so the order of specification is more forgiving. It's recommended that where such default options exist in configuration files, they should be preferred over the primary option 🫶.

It is possible to specify almost any command line option as an option. Some of them just don't make sense to use here 🙈.

That's a wrap on DIRVISH.CONF(5) bestie — this page is dated and versioned as part of the dirvish manual set, filed under section 5 and tagged DIRVISH.CONF(5) 💅. Go forth and back up responsibly ✨🌈💖

📦 where this page comes from