hdup.conf(5)

๐Ÿฆ‹ hdup.conf(5) ๐Ÿฆ‹

okay so bestie, welcome to the File Formats Manual entry for hdup.conf โ€” yes that's the hdup configuration file, and yes we're gonna get through ALL of it together ๐Ÿ’–


โœจ NAME โœจ

hdup.conf โ€” the hdup configuration file ๐ŸŽ€


๐ŸŒธ DESCRIPTION ๐ŸŒธ

So the config file syntax is borrowed from SaMBa โ€” which, if you didn't know, is what everyone calls an ini-style config file. A # as the first character on a line starts a comment. Blank lines just get skipped, no drama. ๐Ÿ’…

Multiple entries on one line? Gotta be separated by commas: ",". Periodt.

Some options are on/off vibes. Those are the binary options, and all of them default to 'off'. Turn one on with 'yes', 'on' or 'true', and turn it off with 'no', 'off' or 'false'. That's it, that's the whole thing.

The [global] section is required to be the first section in the configuration files. Options you specify under [global] are also used in [host] sections โ€” and they can be overridden when you also specify them under that [host] statement. This works for all options, no cap. On top of that, one [host] statement can inherit from another [host] statement. It's giving family tree ๐ŸŒณ

Config entries may be given multiple times โ€” and in that case the latest one is taken as the final choice. Last one wins, sorry to the earlier ones ๐Ÿ˜ญ

And honestly, the best way to describe it is justโ€ฆ with an example:

#
# backup config for hdup
#
[global]
archive dir = /tmp/storage/
compression = gzip
user = operator
proto = /usr/bin/ssh
proto option = -q -oProtocol=2
overwrite = yes

[host-name-a-conf]
dir = /var/www, /etc/cron.d

[host-name-a-root]
dir = /root/.cpan/Bundle

[host-name-a]
inherit = host-name-a-conf, host-name-a-root
allow remote = yes

๐Ÿ‘‘ [global] ๐Ÿ‘‘

The [global] section is required, bestie, we can't skip it. The keywords specified under it are used for each host (globally ๐ŸŒ).

All keywords specified under [global] are inherited by the other hosts. Ie. if you specify 'compression = gzip' under global, all hosts who do not redefine 'compression' will use 'gzip'. Inheritance slay โœจ


๐Ÿซถ [host-name] ๐Ÿซถ

This is a host statement. For every host you want to back up there should be a host statement. And yes, that's also true when you are restoring an archive. Don't skip it just because you're restoring ๐Ÿ‘€

Host statements can inherit from other host statements. Any keywords initialised for the original host will append to, or overwrite, the current keywords. Depends on the vibe of the keyword ๐Ÿ’ซ

There is a maximum of 255 different hosts in 1 hdup configuration file. Two hundred and fifty five. That's your limit, girly.

Be aware that archive dir must be specified in the configuration file. dir is only needed when performing backups โ€” when restoring it is not needed. ๐ŸŽ€


๐ŸŒˆ Keywords ๐ŸŒˆ

The following keywords are supported: algorithm, allow remote, always backup, archive dir, chunk size, compression, compression level, date spec, dir, exclude, force, free, group, gpg, include, inherit, key, log, mcrypt, no history, nobackup, one filesystem, overwrite, postrun, prerun, proto, proto option, remote hdup, remote hdup option, skip, sparse, tar, tar option and user.

The only mandatory options are archive dir and dir. They must be present for every host. No exceptions, no skipping, no "I'll do it later" ๐Ÿ™ˆ


๐Ÿ’œ algorithm

Optional. What algorithm should hdup use when encrypting an archive. If this is not specified the archive will not be encrypted. Both 'algorithm' and 'key' must be present. For gpg encryption use gpg here. ๐Ÿ”

๐Ÿ’œ allow remote

Optional, binary option. If 'on', remote archives are allowed to be uploaded from this host, otherwise they are denied. Simple as that ๐Ÿšช

๐Ÿ’œ always backup

Optional, binary option. When 'on' hdup will always perform a backup. Normally when an incfile is not found the backup is aborted. What this option does is that if the backup scheme is daily and no weekly incfile is found, hdup performs a weekly backup. And if hdup discovers no monthly incfile when doing a weekly, it performs a monthly dump. It's climbing the ladder of backups ๐Ÿชœ

๐Ÿ’œ archive dir

Mandatory. Specify what directory hdup should use to store the archives and the (incremental) dump information. ๐Ÿ“ฆ

๐Ÿ’œ chunk size

Optional. Give the size of the chunks hdup should create when splitting up an archive. Size can be given with the suffix 'k', 'K' or 'm', 'M'. Chunks of the archive get the suffix '__split__XX', where XX is a two letter sequence starting by 'aa' and ending at 'zz'. To split up archive in CD sized chunks, chunk size = 640m could be used. Iconic ๐Ÿ’ฟ

๐Ÿ’œ compression

Optional. Specify the compression hdup should use. This can be bzip, gzip, lzop or none. Defaults to gzip. Some explanation on the difference might be appropriate here, so: bzip (which uses bzip2) is slow but compresses the best, gzip is faster but offers less compression. lzop is the fastest of them all while offering very good compression. none is of course the fastest. Pick your fighter ๐ŸฅŠ

๐Ÿ’œ compression level

Optional. Specify the compression level, it's an integer between 1 and 9 (inclusive), where 1 equals fast operation, lousy compression, and 9 means best compression, but slow. When omitted it defaults to 6.

Defaults to 6, which for all compression algorithms is the standard default. ๐ŸŽš๏ธ

๐Ÿ’œ date spec

Optional. The following formats are supported:

  • default โ€” format will be 'DD-MM-YYYY'
  • iso โ€” format will be 'YYYY-MM-DD'
  • american โ€” format will be 'MM-DD-YYYYY'

๐Ÿ“…

๐Ÿ’œ dir

Mandatory. Specify which directories or files should be backed up. You can also specify a single file, like /usr/src/linux/.config.

There can be up to 20 different directories specified. There can only be 1 dir statement per host. One. Uno. Don't try it ๐Ÿซต

๐Ÿ’œ exclude

Optional. Specify a list with a regular expressions that should be used to determine which files should not be backed up. See regex(7) for more information about regular expressions. Also see the section PATTERNS. ๐Ÿšซ

๐Ÿ’œ force

Optional, binary option. When 'on' a restore to / will be allowed. That's a big deal, so be careful with this one ๐Ÿ˜ณ

๐Ÿ’œ free

Optional. With free you can specify how much free space must be available on a partition. If this free space requirement is not met, hdup will not perform the backup. Takes an optional size modifier: 'k', 'm' or 'G'. ๐Ÿงฎ

๐Ÿ’œ gpg

Optional. The path to gpg. Defaults to the value of the configure script. ๐Ÿ”‘

๐Ÿ’œ group

Optional. Specify the group under which the archives must be stored. Defaults to whatever group 'user' belongs to. ๐Ÿ‘ฏ

๐Ÿ’œ include

Optional. Specify a list with a regular expressions that should be used to determine which files should be backed up. See regex(7) for more information about regular expressions. Also see the section PATTERNS. Included files take precedence on exclude files. Include girlies win ๐Ÿ†

๐Ÿ’œ inherit

Optional. Specify a list of hosts to inherit from. All keywords specified will either overwrite (for single items) or append (for lists) keywords for the current host. This allows creating specific host configurations out of common parts. Reuse is the moment โœจ

๐Ÿ’œ key

Optional. Which file should be used as the encryption key. Both 'algorithm' and 'key' must be present. In the case where algorithm is gpg the user ID of the key must be specified here. ๐Ÿ”

๐Ÿ’œ log

Optional, binary option. When 'on' hdup will also log to syslog. All message will be logged under LOG_DAEMON with priority LOG_NOTICE. All errors are logged in the following format:

FAILURE, <hostname>, <error condition>

Succes is reported as:

SUCCESS, <hostname>, <archive size>, <archive time>

If the backup is send to a remote system, <archive size> equals "remote". If the operation is restore, then <archive size> equals "restore". ๐Ÿ“

๐Ÿ’œ mcrypt

Optional. The path to mcrypt. Defaults to the value of the configure script. ๐Ÿ”’

๐Ÿ’œ no history

Optional, binary option. When 'on' hdup will store each archive in a directory called 'static' thereby not keeping any history of the archives.

โš ๏ธ WARNING: this option is dangerous to use. When a backup fails and you did not copy the archives to some safe place you are left with no backups at all! A postrun script is provided in the examples directory of the hdup source, which copies the archives to a safe place. It is best to NOT use this option unless you know what you are doing. ๐Ÿ’€

Restoring such an archive can be accomplished by using the word 'static' as the restore date. ๐Ÿ•ฐ๏ธ

๐Ÿ’œ nobackup

Optional. The argument is a filename. When specified hdup looks for this file in the directories it backs up. If this file is found the current directory and all sub-directories are excluded from the backup. Poof, gone ๐Ÿช„

๐Ÿ’œ one filesystem

Optional, binary option. When 'on' hdup will stay in the local file system for each directory specified (with 'dir') when creating a backup. Stay in your lane bestie ๐Ÿ›ฃ๏ธ

๐Ÿ’œ overwrite

Optional, binary option. When 'on' old archives are overwritten. Out with the old ๐Ÿ—‘๏ธ

๐Ÿ’œ postrun

Optional. Specify a command or script that be should run after hdup is finished with the backup. The following variables can be used as arguments:

  • %h expands to the current host.
  • %a expands to the full path of the archivename of the current backup.
  • %s expands to the current scheme.
  • %u expands to the username under which the archives are stored.
  • %e expands to 'yes' when encryption is used, 'no' otherwise.
  • %c expands to 'yes' when chunksize is used, 'no' otherwise.
  • %g expands to the groupname under which the archives are stored.
Note: If the postrun script executes with errors the backup is not aborted.
Note2: Any arguments not defined will be expanded to '-empty', without the quotes.

๐ŸŽฌ

๐Ÿ’œ prerun

Optional. Specify a command or script that should run before hdup begins with the actual backup. The following variables can be used as arguments:

  • %h expands to the current host.
  • %a expands to the full path of the archivename of the current backup.
  • %s expands to the current scheme.
  • %u expands to the username under which the archives are stored.
  • %e expands to 'yes' when encryption is used, 'no' otherwise.
  • %c expands to 'yes' when chunksize is used, 'no' otherwise.
  • %g expands to the groupname under which the archives are stored.
Note: If the prerun script executes with errors the backup IS aborted.
Note2: Any arguments not defined will be expanded to '-empty', without the quotes.

๐Ÿšฆ

๐Ÿ’œ proto

Optional. Specify the path of the program to use when transferring an archive to a remote host. Known to work is ssh. Defaults to the value of the configure script.

These programs must be able to be used as a filter and support the user@remotehost syntax. ๐Ÿ›ฐ๏ธ


๐ŸŒˆ ok bestie, that's not even everything โ€” proto option, remote hdup, remote hdup option, skip, sparse, tar, tar option and user are still waiting for us. Part 2, let's go ๐Ÿ’–
ok so bestie, we're mid-OPTIONS right now โ€” keep scrolling, the list keeps going ๐Ÿ’…

Note: this value also has to be defined in the remote hdup that's receiving the backup, even though it's literally not used over there ๐Ÿ™ˆ. And if you don't wanna set it to 'ssh', you can use '/dev/null' or any other path, whatever floats your boat.

  • proto option โ€” optional. This is where you hand extra options to the proto command in hdup. For example: proto option = -i /home/user/.ssh/identity -oProtocol=2.
  • remote hdup โ€” optional. If you're using the @user@remotehost syntax, this keyword says where the remote hdup actually lives. ๐Ÿ“
  • remote hdup option โ€” optional. If you're using that same @user@remotehost syntax, this one says which options the remote hdup should use (like, say, where its config file is). ๐Ÿซถ
  • skip โ€” optional, binary option. Depricated, it's always 'on' now. When it's 'on', the backup directory automatically gets dropped into the exclude list, so it doesn't get backed up. (Yes, that's the spelling in the original, we don't touch it โœจ)
  • sparse โ€” optional, binary option. Depricated, it's always 'on'. When it's 'on', hdup uses tar's --sparse feature while backing up files.
  • tar โ€” optional. The path to tar. Defaults to whatever value the configure script gave it. Heads up: this tar must support the command line syntax of GNU tar, no exceptions, periodt ๐Ÿ’€
  • tar option โ€” optional. Extra options you wanna pass to the tar that hdup runs. These get handed to both the tar and untar commands. And ngl, hdup does no extra checking on these options, so it's all on you bestie ๐Ÿซฃ
  • user โ€” optional. Says which user the archives must be stored under. Defaults to operator.

๐ŸŒธ PATTERNS ๐ŸŒธ

ok so the include and exclude keywords take a regular expression as their input. iconic. but there's one extra little rule on top:

  • if an expression ends with a slash /, it only applies to directories ๐Ÿ“
  • a / sitting anywhere else is not handled special at all

The whole pathname of a file or directory is what gets used in the pattern matching. And the matching is case sensitive, so don't come at it with vibes instead of the right case ๐Ÿ‘‘

โœจ Examples โœจ

  • wanna match all files ending with .txt? use the pattern .*.txt
  • wanna match everything under opt? use ^/opt
  • wanna match a specific directory in /opt? use ^/opt/bla/ โ€” but note that this excludes all directories which start with this string, so be careful bestie ๐Ÿ˜ญ
  • if you want to match a single directory, you have to supply the full pathname and a leading and closing /

๐Ÿฆ‹ WHAT COMES FIRST include or exclude ๐Ÿฆ‹

(that's it, that's the line, that's the whole vibe check โ€” make of it what you will ๐Ÿ™ˆ)


๐Ÿ’– AUTHOR ๐Ÿ’–

Written by Miek Gieben. Wouter van Gils helped a lot with testing pre-release versions. User feedback is appreciated ๐Ÿซถ so like, tell them nice things.


๐Ÿ› REPORTING BUGS ๐Ÿ›

Found a bug? Report it to <hdup-user@miek.n> or via the bugzilla at the homepage. Do it, bestie, they wanna know ๐Ÿ“ฎ


Copyright (C) 2001-2005 Miek Gieben. This is free software. There is NO warrenty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

fr fr, read that again โ€” no warranty, not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. free software, but you're on your own, no cap ๐Ÿ’…

๐Ÿ” SEE ALSO ๐Ÿ”


and that's the whole page bestie ๐ŸŽ€ this was hdup.conf(5), from the source/version that shipped back on 18 Mar 2003 โ€” so if you're reading this in the group chat era, just know this manual page is lowkey older than us ๐Ÿ˜ญ slay anyway, go configure your backups โœจ๐Ÿ’–