HSPACE(1)

📦
version 3.1.0~rc2 · 4 versions
, in debian trixie, ubuntu questing, ubuntu resolute

💅 NAME 💅

okay so bestie, welcome to HSPACE(1) — version 3.1.0~rc2 — and it's literally the Cluster space analyzer for Ganeti. That's the whole vibe. It's giving "how many more instances can we squeeze in?" energy. 🎀

🦋 SYNOPSIS 🦋

Here's the sacred rune, no cap:

hspace {backend options...} [algorithm options...]  [request options...]
[output options...]  [-v...  | -q]

hspace --version

Then the groups, bestie:

Backend options

{ -m cluster | -L[ path ] | -t data-file | --simulate spec | -I path }

Algorithm options

[ --max-cpu *cpu-ratio* ] [ --min-disk *disk-ratio* ] [ -O *name...* ] [
--independent-groups ] [ --no-capacity-checks ]

Request options

[--disk-template template ]

[--standard-alloc disk,ram,cpu ]

[--tiered-alloc disk,ram,cpu ]

Output options

[--machine-readable[=*CHOICE*] ] [-p[fields]]

✨ DESCRIPTION ✨

so like, hspace computes how many additional instances can be fit on a cluster, while maintaining N+1 status. 💖

The program will try to place instances, all of the same size, on the cluster, until the point where we don't have any N+1 possible allocation. It uses the exact same allocation algorithm as the hail iallocator plugin in allocate mode. slay.

The output of the program is designed either for human consumption (the default) or, when enabled with the --machine-readable option (described further below), for machine consumption. In the latter case, it is intended to interpreted as a shell fragment (or parsed as a key=value file). Options which extend the output (e.g. -p, -v) will output the additional information on stderr (such that the stdout is still parseable).

By default, the instance specifications will be read from the cluster; the options --standard-alloc and --tiered-alloc can be used to override them.

The following keys are available in the machine-readable output of the script (all prefixed with HTS_):

  • SPEC_MEM, SPEC_DSK, SPEC_CPU, SPEC_RQN, SPEC_DISK_TEMPLATE, SPEC_SPN — these represent the specifications of the instance model used for allocation (the memory, disk, cpu, requested nodes, disk template, spindles). periodt.
  • TSPEC_INI_MEM, TSPEC_INI_DSK, TSPEC_INI_CPU, ... — only defined when the tiered mode allocation is enabled. These are similar to the above specifications but show the initial starting spec for tiered allocation. 🧚
  • CLUSTER_MEM, CLUSTER_DSK, CLUSTER_CPU, CLUSTER_NODES, CLUSTER_SPN — these represent the total memory, disk, CPU count, total nodes, and total spindles in the cluster.
  • INI_SCORE, FIN_SCORE — these are the initial (current) and final cluster score (see the hbal man page for details about the scoring algorithm). iykyk.
  • INI_INST_CNT, FIN_INST_CNT — the initial and final instance count.
  • INI_MEM_FREE, FIN_MEM_FREE — the initial and final total free memory in the cluster (but this doesn't necessarily mean available for use). 💀
  • INI_MEM_AVAIL, FIN_MEM_AVAIL — the initial and final total available memory for allocation in the cluster. If allocating redundant instances, new instances could increase the reserved memory so it doesn't necessarily mean the entirety of this memory can be used for new instance allocations.
  • INI_MEM_RESVD, FIN_MEM_RESVD — the initial and final reserved memory (for redundancy/N+1 purposes).
  • INI_MEM_INST, FIN_MEM_INST — the initial and final memory used for instances (actual runtime used RAM).
  • INI_MEM_OVERHEAD, FIN_MEM_OVERHEAD — the initial and final memory overhead, i.e. memory used for the node itself and unaccounted memory (e.g. due to hypervisor overhead).
  • INI_MEM_EFF, HTS_INI_MEM_EFF — the initial and final memory efficiency, represented as instance memory divided by total memory.
  • INI_DSK_FREE, INI_DSK_AVAIL, INI_DSK_RESVD, INI_DSK_INST, INI_DSK_EFF — initial disk stats, similar to the memory ones.
  • FIN_DSK_FREE, FIN_DSK_AVAIL, FIN_DSK_RESVD, FIN_DSK_INST, FIN_DSK_EFF — final disk stats, similar to the memory ones.
  • INI_SPN_FREE, ..., FIN_SPN_FREE, .. — initial and final spindles stats, similar to memory ones.
  • INI_CPU_INST, FIN_CPU_INST — initial and final number of virtual CPUs used by instances.
  • INI_CPU_EFF, FIN_CPU_EFF — the initial and final CPU efficiency, represented as the count of virtual instance CPUs divided by the total physical CPU count.
  • INI_MNODE_MEM_AVAIL, FIN_MNODE_MEM_AVAIL — the initial and final maximum per-node available memory. This is not very useful as a metric but can give an impression of the status of the nodes; as an example, this value restricts the maximum instance size that can be still created on the cluster.
  • INI_MNODE_DSK_AVAIL, FIN_MNODE_DSK_AVAIL — like the above but for disk.
  • TSPEC — this parameter holds the pairs of specifications and counts of instances that can be created in the tiered allocation mode. The value of the key is a space-separated list of values; each value is of the form memory,disk,vcpu,spindles=count where the memory, disk and vcpu are the values for the current spec, and count is how many instances of this spec can be created. A complete value for this variable could be: 4096,102400,2,1=225 2560,102400,2,1=20 512,102400,2,1=21.
  • KM_USED_CPU, KM_USED_NPU, KM_USED_MEM, KM_USED_DSK — these represents the metrics of used resources at the start of the computation (only for tiered allocation mode). The NPU value is "normalized" CPU count, i.e. the number of virtual CPUs divided by the maximum ratio of the virtual to physical CPUs.
  • KM_POOL_CPU, KM_POOL_NPU, KM_POOL_MEM, KM_POOL_DSK — these represents the total resources allocated during the tiered allocation process. In effect, they represent how much is readily available for allocation.
  • KM_UNAV_CPU, KM_POOL_NPU, KM_UNAV_MEM, KM_UNAV_DSK — these represents the resources left over (either free as in unallocable or allocable on their own) after the tiered allocation has been completed. They represent better the actual unallocable resources, because some other resource has been exhausted. For example, the cluster might still have 100GiB disk free, but with no memory left for instances, we cannot allocate another instance, so in effect the disk space is unallocable. Note that the CPUs here represent instance virtual CPUs, and in case the --max-cpu option hasn't been specified this will be -1.
  • ALLOC_USAGE — the current usage represented as initial number of instances divided per final number of instances.
  • ALLOC_COUNT — the number of instances allocated (delta between FIN_INST_CNT and INI_INST_CNT).
  • ALLOC_FAIL*_CNT — for the last attempt at allocations (which would have increased FIN_INST_CNT with one, if it had succeeded), this is the count of the failure reasons per failure type; currently defined are FAILMEM, FAILDISK and FAILCPU which represent errors due to not enough memory, disk and CPUs, and FAILN1 which represents a non N+1 compliant cluster on which we can't allocate instances at all.
  • ALLOC_FAIL_REASON — the reason for most of the failures, being one of the above FAIL* strings.
  • OK — a marker representing the successful end of the computation, and having value "1". If this key is not present in the output it means that the computation failed and any values present should not be relied upon. 🚨

Many of the INI_/FIN_ metrics will be also displayed with a TRL_ prefix, and denote the cluster status at the end of the tiered allocation run.

The human output format should be self-explanatory, so it is not described further.

✨ OPTIONS ✨

okay bestie, here's the full lineup of flags you can hand to this program 💅 every single one, no skipping:

  • --disk-template template — overrides the disk template for the instance that got read off the cluster. pass in one of the Ganeti disk templates (like plain, drbd, whatever) and it'll use that instead. 🦋
  • --spindle-use spindles — overrides the spindle use for the instance that came from the cluster. the value can be 0 (yes literally zero — that's for instances doing extremely low I/O), but it can not be negative, periodt. and for shared storage? this value just gets ignored, so don't stress about it. 🙈
  • --max-cpu=*cpu-ratio* — the maximum virtual-to-physical cpu ratio, and it's a floating point number that has to be greater than or equal to one. so like, if you say cpu-ratio is 2.5, then on a 4-cpu machine you're allowing a max of 10 virtual cpus to be in use for primary instances. a value of exactly one = zero over-subscription of CPU (apart from the CPU time the node itself eats). and anything below one? doesn't make sense, because it means other resources (like disk) won't get fully utilised thanks to the CPU restrictions. 😭
  • --min-disk=*disk-ratio* — the minimum amount of free disk space that stays free, as a floating point number. say you pass disk-ratio as 0.25 — that means at least one quarter of disk space has to be left free on nodes. 🍓
  • --independent-groups — treats all groups as independent. so if a node that's not N+1 happy shows up, just ignore its group and keep allocating in the other groups. the default behaviour is way stricter: if any not-N+1-happy node is found, no allocation is attempted at all. 👑
  • --accept-existing-errors — this one is like --independent-groups but on hard mode / strengthened. it tells hspace to ignore the existence of not N+1 happy nodes and just allocate on all the other nodes without creating new N+1 violations. heads up though: this tends to overestimate the capacity, because instances still have to be moved away from those existing not N+1 happy nodes. so like, the vibes are optimistic but the numbers are not fully honest. 💀
  • --no-capacity-checks — normally hspace only considers allocations where every instance on a node could be restarted immediately if that node failed. with this flag given, it only checks N+1 redundancy for DRBD instances. that's the whole check. that's it. 🫶
  • -l rounds, --max-length=*rounds* — caps the number of instance allocations to this length. ngl it's not very useful in practice, but it's there for testing hspace itself, or for capping the runtime on absolutely massive clusters. ⭐
  • -p, --print-nodes — prints the node status before and after, in a format designed so you can actually understand the node's most important parameters. the man page htools(1) has more details on this option. 🌈
  • -O name — this one can be given multiple times, and it marks nodes as being offline. which means a couple of things:
    • instances won't be placed on these nodes, not even temporarily — for example, the replace primary move isn't available if the secondary node is offline, because that move requires a failover. 💅
    • these nodes won't be counted in the score calculation (except for the percentage of instances on offline nodes).

also note the algorithm will mark any node as offline if RAPI reports it that way, or if it has "?" in file-based input in any numeric fields. so it's not just you doing it manually. 👀

  • -S filename, --save-cluster=*filename* — if you give this, the cluster state at the end of allocation gets saved to a file named filename.alloc, and if tiered allocation is enabled, the state after tiered allocation gets saved to filename.tiered. this is so you can re-feed the cluster state back into hspace itself (with different parameters, cute) or into something like hbal, via the -t option. ✨
  • -t datafile, --text-data=*datafile* — backend specification: the name of the file holding node and instance information (for when you're not collecting via RAPI or LUXI). you have to pick this or one of the other backends, no exceptions. the option is described in the man page htools(1). 🎀
  • -m cluster — backend specification: collect data directly from the cluster you pass as the argument, via RAPI. also described in the man page htools(1). 💜
  • -L [path] — backend specification: collect data straight from the master daemon, which gets contacted via LUXI (that's an internal Ganeti protocol). also described in the man page htools(1). 🧚
  • --simulate description — backend specification: kinda like the -t option, this lets you override the cluster data with a simulated cluster. for the details on what the description looks like, go check the man page htools(1). 🌸
  • --standard-alloc disk,ram,cpu — overrides the instance size read from the cluster for standard allocation mode, where we literally just allocate instances of the same fixed size over and over until the cluster runs out of space. the spec you give is similar to the --simulate option, and it holds:
    • the disk size of the instance (units can be used!)
    • the memory size of the instance (units can be used!)
    • the vcpu count for the instance

a cute example description would be 100G,4g,2, describing an instance spec of 100GB disk, 4GiB memory and 2 VCPUs. 🍓

  • --tiered-alloc disk,ram,cpu — overrides the instance size for tiered allocation mode. in this mode the algorithm starts from the spec you gave and allocates until there's no more space; then it decreases the spec and tries allocating again. the decrease happens on the metric that last failed during allocation. the argument should have the same format as --standard-alloc. 🔥

also note: normal allocation and tiered allocation are independent, and both start from the initial cluster state. so the instance counts for these two modes are not related to each other at all. two separate girlies, two separate journeys. 💅

  • --machine-readable[=*choice*] — by default the output is in "human-readable" format, i.e. text descriptions. passing this flag lets you either enable it (--machine-readable or --machine-readable=yes) or explicitly disable it (--machine-readable=no) — referring to the machine readable format described above. ⭐
  • -v, --verbose — increases output verbosity. every use of this option bumps the verbosity up (currently more than 2 doesn't make sense) from the default of one. 🦋
  • -q, --quiet — decreases output verbosity. every use of this option drops the verbosity down (less than zero doesn't make sense) from the default of one. 🙈
  • -V, --version — just shows the program version and exits. that's it. iconic minimalism. ✨

🌈 UNITS 🌈

by default, every unit-accepting option uses mebibytes. if you use the lower-case letters m, g and t (or their longer equivalents mib, gib, tib — case doesn't matter for those) you get explicit binary units. 💖

and if you want units from the SI system, use the upper-case letters M, G and T (or their longer equivalents MB, GB, TB, where case also doesn't matter). 🌸

want more detail on the difference between SI and binary? go read the units(7) man page, bestie. 🫶

🚨 EXIT STATUS 🚨

the exit status of the command will be zero, unless for some reason the algorithm fatally failed (like, say, wrong node or instance data). then it's not zero. 😭

💀 BUGS 💀

the algorithm is highly dependent on the number of nodes — its runtime grows exponentially with that number, so for really big clusters it's just plain impractical. not a vibe, but it is the truth. 🫠

also, the algorithm doesn't rebalance the cluster and it doesn't try to get the optimal fit. it just allocates in the best place for the current step, without taking into consideration the impact on future placements. main character energy, zero forward planning. 💅

📣 REPORTING BUGS 📣

report bugs to the project's issue tracker, or contact the developers using the Ganeti mailing list. they want to hear from you. 🫶

👀 SEE ALSO 👀

Ganeti overview and specifications: ganeti(7) (general overview), ganeti-os-interface(7) (guest OS definitions), ganeti-extstorage-interface(7) (external storage providers). ✨

Ganeti commands: gnt-cluster(8) (cluster-wide commands), gnt-job(8) (job-related commands), gnt-node(8) (node-related commands), gnt-instance(8) (instance commands), gnt-os(8) (guest OS commands), gnt-storage(8) (storage commands), gnt-group(8) (node group commands), gnt-backup(8) (instance import/export commands), gnt-debug(8) (debug commands). 💖

Ganeti daemons: ganeti-watcher(8) (automatic instance restarter), ganeti-cleaner(8) (job queue cleaner), ganeti-noded(8) (node daemon), ganeti-rapi(8) (remote API daemon). 🎀

Ganeti htools: htools(1) (generic binary), hbal(1) (cluster balancer), hspace(1) (capacity calculation), hail(1) (IAllocator plugin), hscan(1) (data gatherer from remote clusters), hinfo(1) (cluster information printer), mon-collector(7) (data collectors interface). 🦋

Copyright (C) 2006-2015 Google Inc. All rights reserved.

redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

  1. redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
  2. redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

and that's the whole page, bestie!! 💖 this one came from the Ganeti project — it's the HSPACE(1) manual, section 1, and the footer down there just says Ganeti / HSPACE(1) with no date attached to it. so yeah, that's a wrap, go allocate some capacity responsibly. mwah. 🫶✨

📦 where this page comes from