GB-STORAGE-TOOL(1)

📦
version 0.5.0 · 3 versions
, in ubuntu resolute, ubuntu stonking

🌸 NAME 🌸

gb-storage-tool — bestie, this is the girlboss that manages your guerillabackup backup data storages 💅

🦋 SYNOPSIS 🦋

gb-storage-tool --Config file --DryRun

✨ DESCRIPTION ✨

Hiii okay so this is the manual page for the gb-storage-tool command, and she is that girl 💖. This tool does operations on backup file storage locations — the ones that gb-backup-generator or gb-transfer-service use to store your backup data.

Right now, the tool supports checking storage file naming so it can clock incomplete backups — like, the ones that got aborted mid-backup-generation or mid-transfer, say because of a reboot or a crash 😭 (we've all been there).

And if you wanna ignore certain files for a reason — notes, whatever — you add entries to the status file, e.g.

For all files that define valid backup data elements, configurable policies get applied. Peep the POLICIES section below for what's supported, bestie 👇

💅 OPTIONS 💅

  • --Config file — this optional parameter lets you point at an alternative configuration file instead of the default /etc/guerillabackup/storage-tool-config.json. Slay, custom paths 🎀
  • --DryRun — this optional parameter makes gb-storage-tool do policy checks only and not modify the storage — so like, it will not delete files that a retention policy flagged for deletion. It's giving look-don't-touch 👀

👑 POLICIES 👑

gb-storage-tool can apply multiple policies to each backup data source, but here's the tea: you can only have one policy of a given type (policy types are down below, iykyk). Which policies actually get applied is defined by the Policies parameter in the gb-storage-tool configuration file.

A regular expression selects which sources policies should be applied to, and the first matching expression takes precedence ⭐. For each regular expression, you define a list of policies with parameters. Want examples? Go look at /data/etc/storage-tool-config.json.template 🌸

To make policy selection less of a nightmare in large setups, policy inheritance exists. An included configuration (see the Include configuration parameter) may also define policies, which can extend or override policies from the parent configuration(s) — and also policies defined just earlier in the same configuration. The overriding policy definition has to have a higher priority, otherwise it gets ignored, no cap 🙈.

To disable policy inheritance, a subconfiguration can set the Inherit configuration parameter to false (default is true). This will also prevent any policies defined earlier in the very same configuration from being ignored. So if you wanna disable inheritance for all sources in a configuration, the first entry in the policy list should match all sources (.*) and disable inheritance. Periodt. 💅

Each policy defined in the gb-storage-tool configuration file may also keep policy status information in the status file. The status data is usually updated as the policy is applied — unless there's a significant policy violation 😭. That means you either gotta fix the root cause of the violation (e.g. backup data was found to be missing) or update the status to ignore the violation.

The latter can't be done interactively via gb-storage-tool yet — you have to adjust the storage status configuration manually. So the user has to create or update the status configuration with the backup element name as key (that's the filename relative to the data directory, no suffix) and the status information for the policy as described below (and sometimes also given as a hint on the console too, so watch the console).

gb-storage-tool supports following policies:

🕰️ Interval

Verify that all sources generate backups at expected rates and that all backups were transferred successfully. So basically this policy makes spotting system failures in the backup system way easier 🔥. An example policy configuration is:

...
  "Policies": [
    {
      "Sources": "^(.*/)?root$",
      "Inherit": false,
      "List": [
        {
          "Name": "Interval",
          "Priority": 100,
          "FullMin": "6d20H",
          "FullMax": "6d28H",
          "IncMin": "20H",
          "IncMax": "28H"
        }, {
...

This configuration says: to all backups from source with the name "root" (that's the default backup created by the gb-backup-generator), an Interval policy shall be applied ✨. The policy will expect full backups every 7 days ± 4 hours and incremental backups each day ± 4 hours.

When policy validation fails for a given source, the policy configuration may be adjusted — but the violation may also be ignored by updating the check status. That way the validation error won't get reported any more in the next run 🎀. The status data in that case may look like:

...
  "20200102000000-root-full": {
    "Interval": {
      "Ignore": "both"
    }
  },
...

This status indicates that the both interval checks — for the interval from the previous full and incremental backup to the backup named above — should be disabled. To disable only one type of checks, use the "full" or "inc" type keyword instead of "both" 🦋.

While the above is fine for ignoring singular policy violations, the policy itself may also be adjusted. This is useful when, e.g., the backup generation intervals were changed at the source. The status data in that case could look like:

...
  "20200102000000-root-full": {
    "Interval": {
      "Config": {
        "FullMax": "29d28H",
        "FullMin": "29d20H",
        "IncMax": "6d28H",
        "IncMin": "6d20H"
      }
    }
  },
...

🗂️ LevelRetention

This defines a retention policy defined by retention levels — like, on the first level keep each backup for 30 days, next level keep 12 weekly backups, next level keep 12 monthly backups, then 12 every three months, and from that point on only yearly ones 🌈.

...
  "Policies": [
    {
      "Sources": "^(.*/)?root$",
      "List": [
        {
          "Name": "LevelRetention",
          "Levels": [
# Keep weekly backups for 30 days, including incremental backups.
            {
              "KeepCount": 30,
              "Interval": "day",
              "TimeRef": "latest",
              "KeepInc": true
            },
# Keep weekly backups for 3 month, approx. 13 backups.
            {
              "KeepCount": 13,
              "Interval": "day",
              "AlignModulus": 7
            },
...
            {
              "KeepCount": 12,
              "Interval": "month",
              "AlignModulus": 3,
              "AlignValue": 1
            },
...

This configuration defines that, on the finest level, backups for 30 days should be kept, counting from the most recent one ("TimeRef": "latest"), including incremental backups ("KeepInc": true). So for machines that aren't producing backups any more, the most recent ones are kept unconditionally, bestie 💖

On the next level, 13 weekly backups are kept — which may overlap with backups already kept thanks to the first level configuration above. But here only full backups will be kept, ones generated after every 7th day due to "AlignModulus", preferring the one generated on day 0 🌸

At another level, only one backup is kept every three months, preferring the one from the month numbered 1, 4, 7, 10 due to "AlignModulus" and "AlignValue". Hence the first backup in January, April, ... should be kept ✨

📏 Size

This policy checks that backup data sizes are as expected, because size changes may indicate problems — like:

  • a size increase due to archives, database dumps, local file backups ... forgotten by the administrator (thus wasting backup space, but sometimes also causing security issues due to lack of as strict access permissions on those files compared to their source 😬)
  • size increase due to rampant processes filling up database tables or log files in retry loops (also, monitoring should catch that)
  • core dumps accumulating, ...

A Size policy can be defined for both full and incremental backups. For each backup type, the accepted size range can be defined by absolute or relative values. Without providing an expected size, the size of the first backup of that type seen is used. So for servers without accumulating data, following policy could be defined:

...
  "Policies": [
    {
      "Sources": "^(.*/)?root$",
      "List": [
        {
          "Name": "Size",
          "Priority": 0,
          "FullSizeMinRel": 0.9,
          "FullSizeMaxRel": 1.1,
          "IncSizeMin": 100000,
          "IncSizeMaxRel": 10.0
        }, {
...

This configuration will check sizes of "root" backups using the first full and incremental size as reference. Full backups may vary in size between 90% and 110%, while incremental backups have to be at least 100kb large but may vary 10-fold in size 👑

All supported policy parameters are:

FullSizeExpect [size (int)]

Specify the expected full backup size. When missing, the size of the first full backup seen is used as default 🎀

FullSizeMax [size (int)]

Specify the absolute maximum backup size. You cannot use "FullSizeMaxRel" at the same time 🚫

FullSizeMin [size (int)]

Specify the absolute minimum backup size. You cannot use "FullSizeMinRel" at the same time 🚫

IncSizeExpect, IncSizeMax, IncSizeMin [size (int)]

Same as the "Full..." parameters, just for incremental backups. See above 💜

FullSizeMaxRel [ratio (float)]

Specify the maximum backup size in relation to the expected size (see "FullSizeExpect"). You cannot use "FullSizeMax" at the same time 🚫

FullSizeMinRel [ratio (float)]

Specify the minimum backup size in relation to the expected size (see "FullSizeExpect"). You cannot use "FullSizeMin" at the same time 🚫

IncSizeExpectRel [ratio (float)]

Specify the expected incremental backup size in relation to the expected full backup size (see "FullSizeExpect"). You cannot use "IncSizeExpect" at the same time 🚫

  • IncSizeMaxRel, IncSizeMinRel [ratio (float)] — bestie these are literally the same as the "Full..." parameters up there, just for incremental backups instead. Scroll up, it's all there, no cap. ✨

so when a policy validation fails for a given source, you've got options, okay? You can tweak the policy config, OR you can straight up ignore the violation by updating the check status. If you do that, the validation error just… stops being reported in the next run. Iconic of you, honestly. 💅 The status data in that case looks like this:

...
  "20200102000000-root-full": {
    "Size": {
      "Ignore": true
    }
  },
...

And while that ^^ is totally fine for ignoring singular policy violations, you can ALSO adjust the policy itself. Which is kind of the moment when e.g. your backup sizes changed because someone installed new software or services (it happens, the vibes shift 🦋). The updated policy configuration then gets attached to the first element it should apply to:

...
  "20200102000000-root-full": {
    "Size": {
      "Config": {
        "FullSizeExpect": 234567890,
        "FullSizeMinRel": 0.9,
        "FullSizeMaxRel": 1.1,
        "IncSizeMin": 100000,
        "IncSizeMaxRel": 10.0
      }
    }
  },
...

📁 FILES 📁

  • /etc/guerillabackup/storage-tool-config.json — the default config file for the gb-storage-tool tool, bestie. Use storage-tool-config.json.template to make it! That template also has the documentation for every single available parameter, so go look, it's giving "I read the docs" energy. The most relevant parameters for gb-storage-tool are DataDir, Include and Status. 👑
  • /var/lib/guerillabackup/state/storage-tool-status.json — this is the recommended location for the toplevel gb-storage-tool status file. It has to contain valid JSON data, but ALSO comment lines starting with # (yes really, spicy ✨). For the description of policy specific status data, see the POLICIES section up above. 🫶

🐛 BUGS 🐛

If your guerillabackup setup came from packages — like .deb or .rpm files, usually installed via package management software like apt-get, aptitude, rpm, yast — then please report bugs to the package maintainer. They're the ones who handle it, not us. 🙈

If your setup is from unpackaged software trunk, please report at:

https://github.com/halfdog/guerillabackup/issues

(Not me asking you to file detailed issues, but like… file detailed issues, it's main character behavior. 💜)

👀 SEE ALSO 👀

👑 AUTHOR 👑

  • halfdog <me@halfdog.net> — wrote this manual page. Absolute legend, fr fr. 🎀

Copyright © 2022-2023 halfdog

This manual page was written for the guerillabackup system on Linux systems, e.g. Debian.

Permission is granted to copy, distribute and/or modify this document under the terms of the Lesser GNU General Public License, Version 3. Slay, freely. 💅

On Debian systems, the complete text of the Lesser GNU General Public License can be found in /usr/share/common-licenses/LGPL-3.

Okay bestie, that's the whole page!! 💖 Page source guerillabackup, dated 01/21/2026, filed under GB-STORAGE-TOOL(1). Go forth and back up your stuff responsibly. No cap. ✨🌈💜
📦 where this page comes from