COLLECTD.CONF(5)

๐Ÿ’… collectd.conf(5) โ€” the manual page, but make it make sense

okay bestie, so this is collectd.conf(5), the config page for the collectd daemon. it's long, it's real, and we're gonna get through it together. grab a lil snack ๐Ÿ“


โœจ SYNOPSIS โœจ

like, here's the vibe at a glance:

  BaseDir "/var/lib/collectd"
  PIDFile "/run/collectd.pid"
  Interval 10.0

  LoadPlugin cpu
  LoadPlugin load

  <LoadPlugin df>
    Interval 3600
  </LoadPlugin>
  <Plugin df>
    ValuesPercentage true
  </Plugin>

  LoadPlugin ping
  <Plugin ping>
    Host "example.org"
    Host "provider.net"
  </Plugin>

๐ŸŽ€ DESCRIPTION ๐ŸŽ€

so this config file controls how the system statistics collection daemon collectd behaves. the biggest-deal option is LoadPlugin, which literally controls which plugins get loaded โ€” and those plugins ultimately define collectd's behavior, periodt. also: if the AutoLoadPlugin option has been enabled, you can straight up omit the explicit LoadPlugin lines for all plugins that have a configuration block, i.e. a <Plugin ...> block. slay.

the syntax of this config file is similar to the config file of the famous Apache webserver (iykyk). each line contains either an option (a key and a list of one or more values) or a section-start or -end. empty lines and everything after a non-quoted hash-symbol (#) are ignored. keys are unquoted strings, consisting only of alphanumeric characters and the underscore (_) character. keys are handled case insensitive by collectd itself and all plugins included with it. values can either be an unquoted string, a quoted string (enclosed in double-quotes) a number or a boolean expression. unquoted strings consist of only alphanumeric characters and underscores (_) and do not need to be quoted. quoted strings are enclosed in double quotes ("""). you can use the backslash character (\) to include double quotes as part of the string. numbers can be specified in decimal and floating point format (using a dot . as decimal separator), hexadecimal when using the 0x prefix and octal with a leading zero (0). boolean values are either true or false.

lines may be wrapped by using \ as the last character before the newline. this allows long lines to be split into multiple lines. quoted strings may be wrapped as well โ€” however, those are treated special in that whitespace at the beginning of the following lines will be ignored, which allows for nicely indenting the wrapped lines. cute AND functional ๐Ÿ’…

the configuration is read and processed in order, i.e. from top to bottom. so the plugins are loaded in the order listed in this config file. it is a good idea to load any logging plugins first in order to catch messages from plugins during configuration. also, unless AutoLoadPlugin is enabled, the LoadPlugin option must occur before the appropriate <Plugin ...> block. no cap, order matters here ๐Ÿ”ฅ


๐ŸŒธ GLOBAL OPTIONS ๐ŸŒธ

  • BaseDir Directory โ€” sets the base directory. this is the directory beneath which all RRD-files are created. possibly more subdirectories are created. this is also the working directory for the daemon. so like, home base ๐Ÿ 
  • LoadPlugin Plugin โ€” loads the plugin Plugin. this is required to load plugins, unless the AutoLoadPlugin option is enabled (see below). without any loaded plugins, collectd will be mostly useless. like, genuinely useless. a whole daemon doing nothing ๐Ÿ˜ญ

only the first LoadPlugin statement or block for a given plugin name has any effect. this is useful when you want to split up the configuration into smaller files and want each file to be "self contained", i.e. it contains a Plugin block and the appropriate LoadPlugin statement. the downside is that if you have multiple conflicting LoadPlugin blocks, e.g. when they specify different intervals, only one of them (the first one encountered) will take effect and all others will be silently ignored. ๐Ÿ’€ silent. ignored.

LoadPlugin may either be a simple configuration statement or a block with additional options, affecting the behavior of LoadPlugin. a simple statement looks like this:

 LoadPlugin "cpu"

options inside a LoadPlugin block can override default settings and influence the way plugins are loaded, e.g.:

 <LoadPlugin perl>
   Interval 60
 </LoadPlugin>

the following options are valid inside LoadPlugin blocks:

  • Globals true|false โ€” if enabled, collectd will export all global symbols of the plugin (and of all libraries loaded as dependencies of the plugin) and, thus, makes those symbols available for resolving unresolved symbols in subsequently loaded plugins if that is supported by your system.

this is useful (or possibly even required), e.g., when loading a plugin that embeds some scripting language into the daemon (e.g. the Perl and Python plugins). scripting languages usually provide means to load extensions written in C. those extensions require symbols provided by the interpreter, which is loaded as a dependency of the respective collectd plugin. see the documentation of those plugins (e.g., collectd-perl(5) or collectd-python(5)) for details.

by default, this is disabled. as a special exception, if the plugin name is either perl or python, the default is changed to enabled in order to keep the average user from ever having to deal with this low level linking stuff. (they're protecting us from the linking lore ๐Ÿ™ˆ)

  • Interval Seconds โ€” sets a plugin-specific interval for collecting metrics. this overrides the global Interval setting. if a plugin provides its own support for specifying an interval, that setting will take precedence. queen stays winning ๐Ÿ‘‘
  • FlushInterval Seconds โ€” specifies the interval, in seconds, to call the flush callback if it's defined in this plugin. by default, this is disabled.
  • FlushTimeout Seconds โ€” specifies the value of the timeout argument of the flush callback.
  • AutoLoadPlugin false|true โ€” when set to false (the default), each plugin needs to be loaded explicitly, using the LoadPlugin statement documented above. if a <Plugin ...> block is encountered and no configuration handling callback for this plugin has been registered, a warning is logged and the block is ignored.

when set to true, explicit LoadPlugin statements are not required. each <Plugin ...> block acts as if it was immediately preceded by a LoadPlugin statement. LoadPlugin statements are still required for plugins that don't provide any configuration, e.g. the Load plugin. (yes bestie, that one still needs its moment โœจ)

  • CollectInternalStats false|true โ€” when set to true, various statistics about the collectd daemon will be collected, with collectd as the plugin name. defaults to false.

the following metrics are reported:

  • "collectd-write_queue/queue_length" โ€” the number of metrics currently in the write queue. you can limit the queue length with the WriteQueueLimitLow and WriteQueueLimitHigh options.
  • "collectd-write_queue/derive-dropped" โ€” the number of metrics dropped due to a queue length limitation. if this value is non-zero, your system can't handle all incoming metrics and protects itself against overload by dropping metrics. that's the system going "bestie no, too much" ๐Ÿ˜ญ
  • "collectd-cache/cache_size" โ€” the number of elements in the metric cache (the cache you can interact with using collectd-unixsock(5)).
  • Include Path [pattern] โ€” if Path points to a file, includes that file. if Path points to a directory, recursively includes all files within that directory and its subdirectories. if the wordexp function is available on your system, shell-like wildcards are expanded before files are included. this means you can use statements like the following:
  Include "/etc/collectd.d/*.conf"

starting with version 5.3, this may also be a block in which further options affecting the behavior of Include may be specified. the following option is currently allowed:

  <Include "/etc/collectd.d">
    Filter "*.conf"
  </Include>
  • Filter pattern โ€” if the fnmatch function is available on your system, a shell-like wildcard pattern may be specified to filter which files to include. this may be used in combination with recursively including a directory to easily be able to arbitrarily mix configuration files and other documents (e.g. README files). the given example is similar to the first example above but includes all files matching *.conf in any subdirectory of /etc/collectd.d. ๐Ÿฆ‹

if more than one file is included by a single Include option, the files will be included in lexicographical order (as defined by the strcmp function). thus, you can e. g. use numbered prefixes to specify the order in which the files are loaded. (number them, bestie, your future self will thank you ๐Ÿซถ)

to prevent loops and shooting yourself in the foot in interesting ways the nesting is limited to a depth of 8 levels, which should be sufficient for most uses. since symlinks are followed it is still possible to crash the daemon by looping symlinks. in our opinion significant stupidity should result in an appropriate amount of pain. ๐Ÿ’€ okay drag them a little

it is no problem to have a block like <Plugin foo> in more than one file, but you cannot include files from within blocks. (that's the line, don't cross it)

  • PIDFile File โ€” sets where to write the PID file to. this file is overwritten when it exists and deleted when the program is stopped. some init-scripts might override this setting using the -P command-line option.
  • PluginDir Directory โ€” path to the plugins (shared objects) of collectd.
  • TypesDB File [File ...] โ€” set one or more files that contain the data-set descriptions. see types.db(5) for a description of the format of this file.

if this option is not specified, a default file is read. if you need to define custom types in addition to the types defined in the default file, you need to explicitly load both. in other words, if the TypesDB option is encountered the default behavior is disabled and if you need the default types you have to also explicitly load them. โš ๏ธ careful, this one's sneaky

  • Interval Seconds โ€” configures the interval in which to query the read plugins. obviously smaller values lead to a higher system load produced by collectd, while higher values lead to more coarse statistics.
warning: you should set this once and then never touch it again. if you do, you will have to delete all your RRD files or know some serious RRDtool magic! (assuming you're using the RRDtool or RRDCacheD plugin.) like, don't. fr fr. ๐Ÿ”ฅ
  • MaxReadInterval Seconds โ€” a read plugin doubles the interval between queries after each failed attempt to get data.

this options limits the maximum value of the interval. the default value is 86400.


okay that's the first chunk done โ€” more is coming, don't go anywhere ๐Ÿ’–

โฑ๏ธ Timeout Iterations ๐Ÿ’…

okay so this one is about when a value list counts as "missing". A value list is considered missing when no update has been read or received for Iterations iterations. By default? collectd just says a value list is missing when no update has been received for twice the update interval. โœจ

the catch (iykyk): since this setting is measured in iterations, the maximum allowed time without an update depends on the Interval info inside each value list. This is what the Threshold config uses to send out notifications about missing values โ€” see collectd-threshold(5) for the deets. ๐Ÿซถ

๐Ÿงต ReadThreads Num

Number of threads to start for reading plugins. Default is 5, but bestie you might wanna bump it if you've got more than five plugins that take a long-ass time to read. Mostly those are the plugins doing network-IO. ๐ŸŒ

ngl, setting this higher than the number of registered read callbacks is not recommended. ๐Ÿ™ˆ

โœ๏ธ WriteThreads Num

Number of threads to start for dispatching value lists to write plugins. Default is 5 โ€” but you may wanna raise it if you have more than five plugins that can take a relatively long time to write to. ๐Ÿ’พ

๐Ÿšฆ WriteQueueLimitHigh HighNum / WriteQueueLimitLow LowNum

so here's the vibe: metrics get read by the read threads, then dropped into a queue to be handled by the write threads. If one of the write plugins is slow (think network timeouts, I/O saturation of the disk ๐Ÿ’€), that queue just grows. To dodge memory issues in that situation, you can cap the queue size.

By default there's no limit and memory may grow indefinitely. For clients โ€” instances that only handle local metrics โ€” this is most likely not an issue. For servers though? Highly recommend setting this to a non-zero value. ๐Ÿ‘‘

You set the limits with WriteQueueLimitHigh and WriteQueueLimitLow. Each takes a numerical argument = the number of metrics in the queue:

  • if there are HighNum metrics in the queue โ†’ any new metrics get dropped.
  • if there are fewer than LowNum metrics in the queue โ†’ all new metrics get enqueued.
  • if the number currently in the queue is between LowNum and HighNum โ†’ the metric is dropped with a probability proportional to the number of metrics in the queue (it increases linearly until it hits 100%). ๐ŸŽฒ

If WriteQueueLimitHigh is set to non-zero and WriteQueueLimitLow is unset, the latter defaults to half of WriteQueueLimitHigh.

If you don't want random dropping when the queue size is between LowNum and HighNum, just set WriteQueueLimitHigh and WriteQueueLimitLow to the same value. ๐Ÿ’…

turning on the CollectInternalStats option is of great help for figuring out what values to use here, fr fr.

๐Ÿ  Hostname Name

Sets the hostname that identifies a host. If you omit this setting, the hostname gets determined using the gethostname(2) system call. ๐Ÿ–ฅ๏ธ

๐ŸŒ FQDNLookup true|false

If Hostname is determined automatically, this setting controls whether or not the daemon should try to figure out the "fully qualified domain name", aka the FQDN. It does this by using getaddrinfo() to look up the full web address of the first network interface that has one. This option is enabled by default. โœจ

๐Ÿ”— PreCacheChain ChainName / PostCacheChain ChainName

Configure the name of the "pre-cache chain" and the "post-cache chain". Please see "FILTER CONFIGURATION" below for info on chains and how these settings change the daemon's behavior. ๐Ÿฆ‹


๐Ÿงฉ PLUGIN OPTIONS ๐Ÿงฉ

Some plugins may register their own options. These options must be enclosed in a "Plugin"-Section. Which options exist depends on the plugin you're using. Some plugins require external configuration too โ€” the "apache plugin", for example, requires "mod_status" to be configured in the webserver you're gonna collect data from. Those plugins are listed below as well, even if they don't require any configuration inside collectd's own config file. ๐Ÿ’–

A list of all plugins plus a short summary for each one can be found in the README file shipped with the sourcecode and hopefully the binary packets as well. ๐Ÿ“ฆ

๐Ÿงฎ Plugin "aggregation"

The Aggregation plugin makes it possible to aggregate several values into one using aggregation functions like sum, average, min and max. This can be put to a wide variety of uses โ€” e.g. average and total CPU statistics for your entire fleet. ๐Ÿ‘‘

The grouping is powerful but, as with many powerful tools, may be a bit difficult to wrap your head around. ๐Ÿ™ˆ So the grouping gets demonstrated with an example: calculating the average and sum of CPU usage across all CPUs of each host.

To select all the affected values for our example, set "Plugin cpu" and "Type cpu". The other values are left unspecified, meaning "all values". The Host, Plugin, PluginInstance, Type and TypeInstance options work as if they were specified in the "WHERE" clause of a "SELECT" SQL statement. ๐Ÿ—„๏ธ

Plugin "cpu"
Type "cpu"

Although the Host, PluginInstance (CPU number, i.e. 0, 1, 2, ...) and TypeInstance (idle, user, system, ...) fields are left unspecified in the example, the intention is to have a new value for each host / type instance pair. This is achieved by "grouping" the values using the "GroupBy" option. It can be specified multiple times to group by more than one field. โœจ

GroupBy "Host"
GroupBy "TypeInstance"

We do neither specify nor group by plugin instance (the CPU number), so all metrics that differ in the CPU number only will get aggregated. Each aggregation needs at least one such field โ€” otherwise no aggregation would take place at all. ๐Ÿ’…

The full example configuration looks like this:

<Plugin "aggregation">
  <Aggregation>
    Plugin "cpu"
    Type "cpu"

    GroupBy "Host"
    GroupBy "TypeInstance"

    CalculateSum true
    CalculateAverage true
  </Aggregation>
</Plugin>

There are a couple of limitations you should be aware of: โš ๏ธ

  • The Type cannot be left unspecified, because it's not reasonable to add apples to oranges. Also, the internal lookup structure won't work if you try to group by type. ๐ŸŽ๐ŸŠ
  • There must be at least one unspecified, ungrouped field. Otherwise nothing will be aggregated. ๐Ÿ’€

As you can see in the example above, each aggregation has its own Aggregation block. You can have multiple aggregation blocks, and aggregation blocks may match the same values โ€” i.e. one value list can update multiple aggregations. The following options are valid inside Aggregation blocks: ๐Ÿ‘‡

  • Host Host / Plugin Plugin / PluginInstance PluginInstance / Type Type / TypeInstance TypeInstance โ€” selects the value lists to be added to this aggregation. Type must be a valid data set name, see types.db(5) for details.

If the string starts with and ends with a slash ("/"), the string is interpreted as a regular expression. The regex flavor used is POSIX extended regular expressions as described in regex(7). Example usage:

Host "/^db[0-9]\\.example\\.com$/"
  • GroupBy Host|Plugin|PluginInstance|TypeInstance โ€” group values by the specified field. The GroupBy option may be repeated to group by multiple fields. ๐ŸŽ€
  • SetHost Host / SetPlugin Plugin / SetPluginInstance PluginInstance / SetTypeInstance TypeInstance โ€” sets the appropriate part of the identifier to the provided string.

The PluginInstance should include the placeholder "%{aggregation}" which will be replaced with the aggregation function, e.g. "average". Not including the placeholder will result in duplication warnings and/or messed up values if more than one aggregation function is enabled. ๐Ÿ˜ญ

The following example calculates the average usage of all "even" CPUs:

<Plugin "aggregation">
  <Aggregation>
    Plugin "cpu"
    PluginInstance "/[0,2,4,6,8]$/"
    Type "cpu"

    SetPlugin "cpu"
    SetPluginInstance "even-%{aggregation}"

    GroupBy "Host"
    GroupBy "TypeInstance"

    CalculateAverage true
  </Aggregation>
</Plugin>

This will create the files: ๐Ÿ“

  • foo.example.com/cpu-even-average/cpu-idle
  • foo.example.com/cpu-even-average/cpu-system
  • foo.example.com/cpu-even-average/cpu-user
  • ...
  • CalculateNum true|false / CalculateSum true|false / CalculateAverage true|false / CalculateMinimum true|false / CalculateMaximum true|false / CalculateStddev true|false โ€” boolean options for enabling calculation of the number of value lists, their sum, average, minimum, maximum and/or standard deviation. All options are disabled by default. ๐Ÿงฎ

๐Ÿ‡ Plugin "amqp"

The AMQP plugin can be used to communicate with other instances of collectd or third-party applications using an AMQP 0.9.1 message broker. Values are sent to or received from the broker, which handles routing, queueing and possibly filtering out messages. ๐Ÿ“จ

Synopsis:

<Plugin "amqp">
  # Send values to an AMQP broker
  <Publish "some_name">
    Host "localhost"
    Host "fallback-amqp.example.com"
    Port "5672"
    VHost "/"
    User "guest"
    Password "guest"
    Exchange "amq.fanout"
#   ExchangeType "fanout"
#   RoutingKey "collectd"
#   Persistent false
#   ConnectionRetryDelay 0
#   Format "command"
#   StoreRates false
#   TLSEnabled false
#   TLSVerifyPeer true
#   TLSVerifyHostName true
#   TLSCACert "/path/to/ca.pem"
#   TLSClientCert "/path/to/client-cert.pem"
#   TLSClientKey "/path/to/client-key.pem"
#   GraphitePrefix "collectd."
#   GraphiteEscapeChar "_"
#   GraphiteSeparateInstances false
#   GraphiteAlwaysAppendDS false
#   GraphitePreserveSeparator false
  </Publish>

  # Receive values from an AMQP broker
  <Subscribe "some_name">
    Host "localhost"
    Port "5672"
    VHost "/"
    User "guest"
    Password "guest"
    Exchange "amq.fanout"
#   ExchangeType "fanout"
#   Queue "queue_name"
#   QueueDurable false
#   QueueAutoDelete true
#   RoutingKey "collectd.#"
#   ConnectionRetryDelay 0
#   TLSEnabled false
#   TLSVerifyPeer true
#   TLSVerifyHostName true
#   TLSCACert "/path/to/ca.pem"
#   TLSClientCert "/path/to/client-cert.pem"
#   TLSClientKey "/path/to/client-key.pem"
  </Subscribe>
</Plugin>

Like, the plugin's config is literally just a bunch of Publish and Subscribe blocks, and they set up sending and receiving of values respectively. The two blocks are basically twins, so unless I say otherwise, any option works in either one. The name you slap in the block's starting tag is only used for reporting messages โ€” but it may be used to support flushing of certain Publish blocks in the future. ๐Ÿ‘€

  • Host Host [Host ...] โ€” hostname or IP-address of the AMQP broker. Defaults to the default behavior of the underlying communications library, rabbitmq-c, which is "localhost". If you list multiple hosts, then a random one gets chosen at each (re)connection attempt โ€” super useful for failover with a clustered broker. ๐ŸŽฏ
  • Port Port โ€” service name or port number on which the AMQP broker accepts connections. Bestie, this argument must be a string, even if you're using the numeric form. Defaults to "5672".
  • VHost VHost โ€” name of the virtual host on the AMQP broker to use. Defaults to "/".
  • User User and Password Password โ€” the credentials used to authenticate to the AMQP broker. By default "guest"/"guest" is used.
  • Exchange Exchange โ€” in Publish blocks, this option specifies the exchange to send values to. By default, "amq.fanout" will be used. In Subscribe blocks this option is optional โ€” if you give it, a binding between the given exchange and the queue is created, using the routing key if configured. See the Queue and RoutingKey options below. ๐Ÿ”—
  • ExchangeType Type โ€” if given, the plugin will try to create the configured exchange with this type after connecting. When in a Subscribe block, the queue will then be bound to this exchange.
  • Queue Queue (Subscribe only) โ€” configures the queue name to subscribe to. If no queue name was configured explicitly, a unique queue name will be created by the broker. โœจ
  • QueueDurable true|false (Subscribe only) โ€” defines if the queue subscribed to is durable (saved to persistent storage) or transient (will disappear if the AMQP broker is restarted). Defaults to "false". This option should be used in conjunction with the Persistent option on the publish side. ๐Ÿซถ
  • QueueAutoDelete true|false (Subscribe only) โ€” defines if the queue subscribed to will be deleted once the last consumer unsubscribes. Defaults to "true".
  • RoutingKey Key โ€” in Publish blocks, this configures the routing key to set on all outgoing messages. If not given, the routing key will be computed from the identifier of the value. The host, plugin, type and the two instances are concatenated together using dots as the separator and all containing dots replaced with slashes. For example "collectd.host/example/com.cpu.0.cpu.user". This makes it possible to receive only specific values using a "topic" exchange. In Subscribe blocks, it configures the routing key used when creating a binding between an exchange and the queue. The usual wildcards can be used to filter messages when using a "topic" exchange. If you're only interested in CPU statistics, you could use the routing key "collectd.*.cpu.#" for example. ๐Ÿงš
  • Persistent true|false (Publish only) โ€” selects the delivery method to use. If set to true, the persistent mode will be used, i.e. delivery is guaranteed. If set to false (the default), the transient delivery mode will be used, i.e. messages may be lost due to high load, overflowing queues or similar issues. ๐Ÿ’€
  • ConnectionRetryDelay Delay โ€” when the connection to the AMQP broker is lost, defines the time in seconds to wait before attempting to reconnect. Defaults to 0, which implies collectd will attempt to reconnect at each read interval (in Subscribe mode) or each time values are ready for submission (in Publish mode).
  • Format Command|JSON|Graphite (Publish only) โ€” selects the format in which messages are sent to the broker. If set to Command (the default), values are sent as "PUTVAL" commands which are identical to the syntax used by the Exec and UnixSock plugins. In this case, the "Content-Type" header field will be set to "text/collectd". If set to JSON, the values are encoded in the JavaScript Object Notation, an easy and straight forward exchange format. The "Content-Type" header field will be set to "application/json". If set to Graphite, values are encoded in the Graphite format, which is "<metric> <value> <timestamp>\n". The "Content-Type" header field will be set to "text/graphite". A subscribing client should use the "Content-Type" header field to determine how to decode the values. Currently, the AMQP plugin itself can only decode the Command format. ๐Ÿ”ฅ
  • StoreRates true|false (Publish only) โ€” determines whether or not "COUNTER", "DERIVE" and "ABSOLUTE" data sources are converted to a rate (i.e. a "GAUGE" value). If set to false (the default), no conversion is performed. Otherwise the conversion is performed using the internal value cache. Please note that currently this option is only used if the Format option has been set to JSON.
  • GraphitePrefix (Publish and Format=Graphite only) โ€” a prefix can be added in the metric name when outputting in the Graphite format. It's added before the Host name. Metric name will be "<prefix><host><postfix><plugin><type><name>"
  • GraphitePostfix (Publish and Format=Graphite only) โ€” a postfix can be added in the metric name when outputting in the Graphite format. It's added after the Host name. Metric name will be "<prefix><host><postfix><plugin><type><name>"
  • GraphiteEscapeChar (Publish and Format=Graphite only) โ€” specify a character to replace dots (.) in the host part of the metric name. In Graphite metric name, dots are used as separators between different metric parts (host, plugin, type). Default is "_" (Underscore).
  • GraphiteSeparateInstances true|false โ€” if set to true, the plugin instance and type instance will be in their own path component, for example "host.cpu.0.cpu.idle". If set to false (the default), the plugin and plugin instance (and likewise the type and type instance) are put into one component, for example "host.cpu-0.cpu-idle".
  • GraphiteAlwaysAppendDS true|false โ€” if set to true, append the name of the Data Source (DS) to the "metric" identifier. If set to false (the default), this is only done when there is more than one DS.
  • GraphitePreserveSeparator false|true โ€” if set to false (the default) the "." (dot) character is replaced with GraphiteEscapeChar. Otherwise, if set to true, the "." (dot) character is preserved, i.e. passed through. ๐Ÿ’…
  • TLSEnabled true|false โ€” if set to true then connect to the broker using a TLS connection. If set to false (the default), then a plain text connection is used. Requires rabbitmq-c >= 0.4.
  • TLSVerifyPeer true|false โ€” if set to true (the default) then the server certificate chain is verified. Setting this to false will skip verification (insecure). Requires rabbitmq-c >= 0.8. ๐Ÿ™ˆ
  • TLSVerifyHostName true|false โ€” if set to true (the default) then the server host name is verified. Setting this to false will skip verification (insecure). Requires rabbitmq-c >= 0.8.
  • TLSCACert Path โ€” path to the CA cert file in PEM format.
  • TLSClientCert Path โ€” path to the client certificate in PEM format. If this is set, then TLSClientKey must be set as well.
  • TLSClientKey Path โ€” path to the client key in PEM format. If this is set, then TLSClientCert must be set as well. ๐Ÿ”

๐Ÿฆ‹ Plugin "amqp1"

The AMQP1 plugin can be used to communicate with other instances of collectd or third party applications using an AMQP 1.0 message intermediary. Metric values or notifications are sent to the messaging intermediary which may handle direct messaging or queue based transfer. Basically, it's giving messenger-of-the-metrics. ๐Ÿ’Œ

Synopsis:

 <Plugin "amqp1">
   # Send values to an AMQP 1.0 intermediary
  <Transport "name">
    Host "localhost"
    Port "5672"
    User "guest"
    Password "guest"
    Address "collectd"
#    RetryDelay 1
    <Instance "some_name">
        Format "command"
        PreSettle false
        Notify false
 #      StoreRates false
 #      GraphitePrefix "collectd."
 #      GraphiteEscapeChar "_"
 #      GraphiteSeparateInstances false
 #      GraphiteAlwaysAppendDS false
 #      GraphitePreserveSeparator false
    </Instance>
  </Transport>
 </Plugin>

The plugin's configuration consists of a Transport that configures communications to the AMQP 1.0 messaging bus and one or more Instance corresponding to metric or event publishers to the messaging system.

The address in the Transport block concatenated with the name given in the Instance block starting tag will be used as the send-to address for communications over the messaging link. ๐ŸŽ€

The following options are accepted within each Transport block:

  • Host Host โ€” hostname or IP-address of the AMQP 1.0 intermediary. Defaults to the default behavior of the underlying communications library, libqpid-proton, which is "localhost".
  • Port Port โ€” service name or port number on which the AMQP 1.0 intermediary accepts connections. This argument must be a string, even if the numeric form is used. Defaults to "5672".
  • User User and Password Password โ€” credentials used to authenticate to the AMQP 1.0 intermediary. By default "guest"/"guest" is used.
  • Address Address โ€” this option specifies the prefix for the send-to value in the message. By default, "collectd" will be used.
  • RetryDelay RetryDelay โ€” when the AMQP1 connection is lost, defines the time in seconds to wait before attempting to reconnect. Defaults to 1, which implies attempt to reconnect at 1 second intervals. โฐ
  • SendQueueLimit SendQueueLimit โ€” if there is no AMQP1 connection, the plugin will continue to queue messages to send, which could result in unbounded memory consumption. This parameter is used to limit the number of messages in the outbound queue to the specified value. The default value is 0, which disables this feature.

The following options are accepted within each Instance block:

  • Format Command|JSON|Graphite โ€” selects the format in which messages are sent to the intermediary. If set to Command (the default), values are sent as "PUTVAL" commands which are identical to the syntax used by the Exec and UnixSock plugins. In this case, the "Content-Type" header field will be set to "text/collectd".

...bestie if you set it to JSON, the values get encoded in the JavaScript Object Notation (yes, that's the easy, straight-forward exchange format everyone loves), and the "Content-Type" header field gets set to "application/json" โœจ

If you set it to Graphite instead, the values are encoded in the Graphite format, which is literally "<metric> <value> <timestamp>\n", and the "Content-Type" header field becomes "text/graphite".

A subscribing client should be checking that "Content-Type" header field to figure out how to decode the values, periodt โ€” that's the whole vibe check right there.

  • PreSettle true|false โ€” if it's false (the default), the plugin waits for a message acknowledgement from the messaging bus before it sends the next message. That's showing transfer of ownership to the messaging system, iykyk. If it's true, the plugin does not wait for an acknowledgement, and ngl the message may get dropped before ownership even transfers ๐Ÿ’€
  • Notify true|false โ€” if false (the default), the plugin services the instance write call back as a value list. If true, the plugin services the instance as a write notification callback for alert formatting.
  • StoreRates true|false โ€” this decides whether "COUNTER", "DERIVE" and "ABSOLUTE" data sources get converted to a rate (i.e. a "GAUGE" value). Set it to false (the default) and no conversion happens. Otherwise the conversion is done using the internal value cache.
Heads up bestie: right now this option is only used if the Format option has been set to JSON. That's it, that's the rule ๐Ÿ™ˆ
  • GraphitePrefix โ€” a prefix you can add into the metric name when you're outputting in the Graphite format. It gets added before the Host name. So the metric name will be "<prefix><host><postfix><plugin><type><name>" ๐ŸŽ€
  • GraphitePostfix โ€” a postfix you can add into the metric name when outputting in the Graphite format. This one gets added after the Host name. Metric name will be "<prefix><host><postfix><plugin><type><name>" ๐ŸŽ€
  • GraphiteEscapeChar โ€” specify a character to replace dots (.) in the host part of the metric name. In a Graphite metric name, dots are used as separators between the different metric parts (host, plugin, type). Default is "_" (Underscore).
  • GraphiteSeparateInstances true|false โ€” if true, the plugin instance and type instance get their own path component, for example "host.cpu.0.cpu.idle". If false (the default), the plugin and plugin instance (and likewise the type and type instance) are all squished into one component, for example "host.cpu-0.cpu-idle".
  • GraphiteAlwaysAppendDS true|false โ€” if true, append the name of the Data Source (DS) to the "metric" identifier. If false (the default), this is only done when there's more than one DS.
  • GraphitePreserveSeparator false|true โ€” if false (the default), the "." (dot) character is replaced with GraphiteEscapeChar. But if you set it to true, the "." (dot) character is preserved, i.e. it just passes straight through, no touchy ๐Ÿ’…

๐Ÿฆ‹ Plugin "apache"

Okay so to configure the "apache"-plugin you first gotta configure the Apache webserver correctly, no skipping steps bestie. The Apache-plugin "mod_status" needs to be loaded and working, and the "ExtendedStatus" directive needs to be enabled. You can base your Apache config on this little snippet right here:

ExtendedStatus on
<IfModule mod_status.c>
  <Location /mod_status>
    SetHandler server-status
  </Location>
</IfModule>

And since its "mod_status" module is super similar to Apache's, lighttpd is also supported ๐Ÿซถ It introduces a new field called "BusyServers" to count the number of currently connected clients โ€” and yes, that field is supported too.

The configuration of the Apache plugin is made of one or more "<Instance />" blocks. Each block requires one string argument as the instance name, like so:

<Plugin "apache">
  <Instance "www1">
    URL "http://www1.example.com/mod_status?auto"
  </Instance>
  <Instance "www2">
    URL "http://www2.example.com/mod_status?auto"
  </Instance>
</Plugin>

The instance name gets used as the plugin instance. And if you wanna emulate the old (version 4) behavior, you can use an empty string (""). For the plugin to work correctly, every instance name must be unique โ€” the plugin does not enforce this, so it's totally your responsibility to make sure. No pressure lol ๐Ÿ™ˆ

The following options are accepted inside each Instance block:

  • URL http://host/mod_status?auto โ€” sets the URL of the "mod_status" output. This needs to be the output generated by "ExtendedStatus on", and it needs to be the machine readable output you get by appending the "?auto" argument. This option is mandatory, fr fr.
  • User Username โ€” optional user name you need for authentication.
  • Password Password โ€” optional password you need for authentication.
  • VerifyPeer true|false โ€” enable or disable peer SSL certificate verification. See <http://curl.haxx.se/docs/sslcerts.html> for the deets. Enabled by default.
  • VerifyHost true|false โ€” enable or disable peer host name verification. If enabled, the plugin checks whether the "Common Name" or a "Subject Alternate Name" field of the SSL certificate matches the host name provided by the URL option. If this identity check fails, the connection gets aborted, periodt. Obviously this only works when you're connecting to an SSL enabled server. Enabled by default.
  • CACert File โ€” a file that holds one or more SSL certificates. If you wanna use HTTPS you'll possibly need this one. Which CA certificates come bundled with "libcurl" and are checked by default depends on the distribution you use ๐Ÿคทโ€โ™€๏ธ
  • SSLCiphers list of ciphers โ€” specifies which ciphers to use in the connection. That list of ciphers must specify valid ciphers. See <http://www.openssl.org/docs/apps/ciphers.html> for details.
  • Timeout Milliseconds โ€” the Timeout option sets the overall timeout for HTTP requests to URL, in milliseconds. By default, the configured Interval is used to set the timeout.

๐Ÿงš Plugin "apcups"

  • Host Hostname โ€” hostname of the host running apcupsd. Defaults to localhost. Please note that IPv6 support has been disabled unless someone can confirm or decline that apcupsd can actually handle it ๐Ÿ‘€
  • Port Port โ€” TCP-Port to connect to. Defaults to 3551.
  • ReportSeconds true|false โ€” if set to true, the time reported in the "timeleft" metric gets converted to seconds. This is the recommended setting, bestie. If set to false โ€” the default, for backwards compatibility โ€” the time is reported in minutes.
  • PersistentConnection true|false โ€” the plugin is designed to keep the connection to apcupsd open between reads. If the plugin poll interval is greater than 15 seconds (that's the hardcoded socket close timeout in apcupsd NIS), then this option is false by default.

You can tell the plugin to close the connection after each read by setting this option to false, or force it to keep the connection by setting it to true ๐Ÿ’…

If apcupsd appears to close the connection due to inactivity quite quickly, the plugin will try to detect that problem and switch to an open-read-close mode. Resourceful queen behavior honestly.


๐ŸŒธ Plugin "aquaero"

This plugin collects the value of the available sensors in an Aquaero 5 board. Aquaero 5 is a water-cooling controller board, manufactured by Aqua Computer GmbH <http://www.aquacomputer.de/>, with a USB2 connection for monitoring and configuration. The board can handle multiple temperature sensors, fans, water pumps and water level sensors, and it adjusts the output settings โ€” like fan voltage or the power used by the water pump โ€” based on the available inputs, using a configurable controller included in the board. This plugin collects all the available inputs plus some of the output values chosen by that controller. The plugin is based on the libaquaero5 library provided by aquatools-ng ๐ŸงŠ

  • Device DevicePath โ€” device path of the Aquaero 5's USB HID (human interface device), usually in the form "/dev/usb/hiddevX". If this option is not set, the plugin will try to auto-detect the Aquaero 5 USB device based on vendor-ID and product-ID โœจ

๐Ÿ‘‘ Plugin "ascent"

This plugin collects information about an Ascent server โ€” a free server for the "World of Warcraft" game, yes really ๐ŸŽฎ It gathers that info by fetching the XML status page using "libcurl" and parses it using "libxml2".

The configuration options are the same as for the "apache" plugin above:

  • URL http://localhost/ascent/status/ โ€” sets the URL of the XML status output.
  • User Username โ€” optional user name you need for authentication.
  • Password Password โ€” optional password you need for authentication.
  • VerifyPeer true|false โ€” enable or disable peer SSL certificate verification. See <http://curl.haxx.se/docs/sslcerts.html> for the deets. Enabled by default.
  • VerifyHost true|false โ€” enable or disable peer host name verification. If enabled, the plugin checks whether the "Common Name" or a "Subject Alternate Name" field of the SSL certificate matches the host name provided by the URL option. If this identity check fails, the connection is aborted. Obviously, only works when connecting to an SSL enabled server. Enabled by default.
  • CACert File โ€” a file that holds one or more SSL certificates. If you wanna use HTTPS you'll possibly need this option. What CA certificates come bundled with "libcurl" and are checked by default depends on the distribution you use.
  • Timeout Milliseconds โ€” sets the overall timeout for HTTP requests to URL, in milliseconds. By default, the configured Interval is used to set the timeout.

๐Ÿ“ Plugin "barometer"

This plugin reads absolute air pressure using a digital barometer sensor on an I2C bus. The supported sensors are:

The sensor type โ€” one of the above โ€” is detected automatically by the plugin and indicated in the plugin_instance (you'll see the subdirectory "barometer-mpl115" or "barometer-mpl3115", or "barometer-bmp085"). The order of detection is BMP085 -> MPL3115 -> MPL115A2, and the first one found gets used (only one sensor can be used by the plugin, so no collecting them all like Pokรฉmon ๐Ÿ’€).

The plugin provides absolute barometric pressure, air pressure reduced to sea level (with several possible approximations), and as an auxiliary value it also gives you the internal sensor temperature. It uses (expects/provides) typical metric units โ€” pressure in [hPa], temperature in [C], altitude in [m] ๐ŸŒก๏ธ

okay so real talk: it was developed and tested under Linux only. The only platform dependency is the standard Linux i2c-dev interface โ€” and the particular bus driver has to support the SM Bus command subset. ๐Ÿ’…

Reducing or normalizing to mean sea level pressure needs (depending on the selected method/approximation) altitude and a reference to temperature sensor(s) too. When multiple temperature sensors are configured, the minimum of their values is always used โ€” the expectation is that the warmer ones are being affected by e.g. direct sun light at that moment. ๐ŸŒž

Synopsis:

  <Plugin "barometer">
     Device            "/dev/i2c-0";
     Oversampling      512
     PressureOffset    0.0
     TemperatureOffset 0.0
     Normalization     2
     Altitude          238.0
     TemperatureSensor "myserver/onewire-F10FCA000800/temperature"
  </Plugin>

๐Ÿ“ Device device

this is the only mandatory config parameter โ€” no cap. periodt โœจ

Device name of the I2C bus to which the sensor is connected. Note that typically you need to have loaded the i2c-dev module. Using i2c-tools you can check/list i2c buses available on your system by:

i2cdetect -l

Then you can scan for devices on given bus. E.g. to scan the whole bus 0 use:

i2cdetect -y -a 0

This way you should be able to verify that the pressure sensor (either type) is connected and detected on address 0x60. ๐ŸŽฏ

๐ŸŽš๏ธ Oversampling value

Optional parameter controlling the oversampling/accuracy. Default value is 1, providing fastest and least accurate reading.

For MPL115 this is the size of the averaging window. To filter out sensor noise a simple averaging using floating window of this configurable size is used. The plugin will use average of the last "value" measurements (value of 1 means no averaging). Minimal size is 1, maximal 1024.

For MPL3115 this is the oversampling value. The actual oversampling is performed by the sensor and the higher value the higher accuracy and longer conversion time (although nothing to worry about in the collectd context). Supported values are: 1, 2, 4, 8, 16, 32, 64 and 128. Any other value is adjusted by the plugin to the closest supported one.

For BMP085 this is the oversampling value. The actual oversampling is performed by the sensor and the higher value the higher accuracy and longer conversion time (although nothing to worry about in the collectd context). Supported values are: 1, 2, 4, 8. Any other value is adjusted by the plugin to the closest supported one.

๐Ÿงช PressureOffset offset

Optional parameter for MPL3115 only.

You can further calibrate the sensor by supplying pressure and/or temperature offsets. This is added to the measured/calculated value (i.e. if the measured value is too high then use negative offset). In hPa, default is 0.0.

๐ŸŒก๏ธ TemperatureOffset offset

Optional parameter for MPL3115 only.

You can further calibrate the sensor by supplying pressure and/or temperature offsets. This is added to the measured/calculated value (i.e. if the measured value is too high then use negative offset). In C, default is 0.0.

๐Ÿงญ Normalization method

Optional parameter, default value is 0.

Normalization method โ€” what approximation/model is used to compute the mean sea level pressure from the air absolute pressure.

Supported values of the "method" (integer from 0 to 2) are:

โ›ฐ๏ธ Altitude altitude

The altitude (in meters) of the location where you measure the pressure.

๐Ÿซถ TemperatureSensor reference

Temperature sensor(s) which should be used as a reference when normalizing the pressure using "Normalization" method 2. When you specify more sensors, a minimum is found and used each time. The temperature reading directly from this pressure sensor/plugin is typically not suitable as the pressure sensor will be probably inside while we want outside temperature. The collectd reference name is something like <hostname>/<plugin_name>-<plugin_instance>/<type>-<type_instance> (<type_instance> is usually omitted when there is just single value type). Or you can figure it out from the path of the output data files. ๐Ÿ•ต๏ธโ€โ™€๏ธ


๐Ÿ”‹ Plugin "battery"

The battery plugin reports the remaining capacity, power and voltage of laptop batteries. ๐Ÿ’ปโœจ

๐Ÿ”‹ ValuesPercentage false|true

When enabled, remaining capacity is reported as a percentage, e.g. "42% capacity remaining". Otherwise the capacity is stored as reported by the battery, most likely in "Wh". This option does not work with all input methods, in particular when only "/proc/pmu" is available on an old Linux system. Defaults to false.

๐Ÿชซ ReportDegraded false|true

Typical laptop batteries degrade over time, meaning the capacity decreases with recharge cycles. The maximum charge of the previous charge cycle is tracked as "last full capacity" and used to determine that a battery is "fully charged".

When this option is set to false, the default, the battery plugin will only report the remaining capacity. If the ValuesPercentage option is enabled, the relative remaining capacity is calculated as the ratio of the "remaining capacity" and the "last full capacity". This is what most tools, such as the status bar of desktop environments, also do.

When set to true, the battery plugin will report three values: charged (remaining capacity), discharged (difference between "last full capacity" and "remaining capacity") and degraded (difference between "design capacity" and "last full capacity"). ๐Ÿ“Š

๐Ÿงš QueryStateFS false|true

When set to true, the battery plugin will only read statistics related to battery performance as exposed by StateFS at /run/state. StateFS is used in Mer-based Sailfish OS, for example.


๐ŸŒ Plugin "bind"

Starting with BIND 9.5.0, the most widely used DNS server software provides extensive statistics about queries, responses and lots of other information. The bind plugin retrieves this information that's encoded in XML and provided via HTTP and submits the values to collectd. ๐Ÿ”ฅ

To use this plugin, you first need to tell BIND to make this information available. This is done with the "statistics-channels" configuration option:

 statistics-channels {
   inet localhost port 8053;
 };

The configuration follows the grouping that can be seen when looking at the data with an XSLT compatible viewer, such as a modern web browser. It's probably a good idea to make yourself familiar with the provided values, so you can understand what the collected statistics actually mean. ๐Ÿ‘€

Synopsis:

  <Plugin "bind">
    URL "http://localhost:8053/"
    ParseTime       false
    OpCodes         true
    QTypes          true

    ServerStats     true
    ZoneMaintStats  true
    ResolverStats   false
    MemoryStats     true

    <View "_default">
      QTypes        true
      ResolverStats true
      CacheRRSets   true

      Zone "127.in-addr.arpa/IN"
    </View>
  </Plugin>

The bind plugin accepts the following configuration options:

  • URL URL โ€” URL from which to retrieve the XML data. If not specified, "http://localhost:8053/" will be used.
  • ParseTime true|false โ€” When set to true, the time provided by BIND will be parsed and used to dispatch the values. When set to false, the local time source is queried.

This setting is set to true by default for backwards compatibility; setting this to false is recommended to avoid problems with timezones and localization.

  • OpCodes true|false โ€” When enabled, statistics about the "OpCodes", for example the number of "QUERY" packets, are collected.

Default: Enabled.

  • QTypes true|false โ€” When enabled, the number of incoming queries by query types (for example "A", "MX", "AAAA") is collected.

Default: Enabled.

  • ServerStats true|false โ€” Collect global server statistics, such as requests received over IPv4 and IPv6, successful queries, and failed updates.

Default: Enabled.

  • ZoneMaintStats true|false โ€” Collect zone maintenance statistics, mostly information about notifications (zone updates) and zone transfers.

Default: Enabled.

  • ResolverStats true|false โ€” Collect resolver statistics, i.e. statistics about outgoing requests (e.g. queries over IPv4, lame servers). Since the global resolver counters apparently were removed in BIND 9.5.1 and 9.6.0, this is disabled by default. Use the ResolverStats option within a View "_default" block instead for the same functionality.

Default: Disabled.

  • MemoryStats โ€” Collect global memory statistics.

Default: Enabled.

  • Timeout Milliseconds โ€” The Timeout option sets the overall timeout for HTTP requests to URL, in milliseconds. By default, the configured Interval is used to set the timeout.

๐ŸชŸ View Name

Collect statistics about a specific "view". BIND can behave different, mostly depending on the source IP-address of the request. These different configurations are called "views". If you don't use this feature, you most likely are only interested in the "_default" view.

Within a <View name> block, you can specify which information you want to collect about a view. If no View block is configured, no detailed view statistics will be collected.

  • QTypes true|false โ€” If enabled, the number of outgoing queries by query type (e.g. "A", "MX") is collected.

Default: Enabled.

  • ResolverStats true|false โ€” Collect resolver statistics, i.e. statistics about outgoing requests (e.g. queries over IPv4, lame servers).

Default: Enabled.

๐Ÿงฉ Plugin "bind" (the options, cont'd)

  • CacheRRSets true|false โ€” turn this on and the plugin counts up the entries ("RR sets") in the view's cache, broken down by query type. Negative entries โ€” like queries that errored out, e.g. names that straight up don't exist โ€” get reported with a leading exclamation mark, so "!A" and friends. Cute little warning sign, honestly.

Default: Enabled. โœจ

  • *Zone Name* โ€” hand it a zone name and it'll collect the detailed info about that zone inside the view. The stuff it gathers is very similar to the global ServerStats info (peep above ๐Ÿ‘†).

You can repeat this option as many times as you want to get details on multiple zones. Collect them like Pokemon.

By default? No detailed zone info at all, nada. ๐Ÿ’…


๐Ÿถ Plugin "buddyinfo"

The buddyinfo plugin gets its info by reading /proc/buddyinfo. That file holds the deets on how many contiguous physical pages are available at this exact moment.

  • *Zone ZoneName* โ€” which zone to collect info about. Default behavior is to grab all zones. ๐Ÿซถ

๐Ÿ” Plugin "capabilities"

The capabilities plugin collects selected static platform data using dmidecode and exposes it through a micro embedded webserver. The data it hands back is in json format, obviously. ๐Ÿ’ป

Synopsis:

<Plugin capabilities>
  Host "localhost"
  Port "9104"
</Plugin>

Available config options for the capabilities plugin:

  • *Host Hostname* โ€” bind to the hostname / address Host. By default the plugin binds to the "any" address, i.e. it'll accept packets sent to any of the host's addresses. Lowkey generous of it.

Heads up: this option is supported only for libmicrohttpd newer than 0.9.0. โš ๏ธ

  • *Port Port* โ€” the port the embedded webserver should be listening on. Defaults to 9104.

๐Ÿ™ Plugin "ceph"

The ceph plugin collects values from JSON data (parsed by libyajl โ€” https://lloyd.github.io/yajl/) retrieved from ceph daemon admin sockets. So it's literally chatting with your daemons over their lil admin sockets. ๐Ÿ’œ

You need a separate Daemon block for every single ceph daemon you wanna monitor. The example below reads daemon stats from four separate ceph daemons running on the same device โ€” two OSDs, one MON, one MDS:

<Plugin ceph>
  LongRunAvgLatency false
  ConvertSpecialMetricTypes true
  <Daemon "osd.0">
    SocketPath "/var/run/ceph/ceph-osd.0.asok"
  </Daemon>
  <Daemon "osd.1">
    SocketPath "/var/run/ceph/ceph-osd.1.asok"
  </Daemon>
  <Daemon "mon.a">
    SocketPath "/var/run/ceph/ceph-mon.ceph1.asok"
  </Daemon>
  <Daemon "mds.a">
    SocketPath "/var/run/ceph/ceph-mds.ceph1.asok"
  </Daemon>
</Plugin>

The ceph plugin accepts these config options:

  • LongRunAvgLatency true|false โ€” if enabled, latency values (that's sum,count pairs bestie) are calculated as the long run average โ€” average since the ceph daemon was started, so (sum / count). When it's disabled, latency values are calculated as the average since the last collection, i.e. (sum_now - sum_last) / (count_now - count_last). Big difference, pick wisely. ๐Ÿงฎ

Default: Disabled

  • ConvertSpecialMetricTypes true|false โ€” if enabled, special metrics (the ones that differ in type from similar counters) get converted to the type of those similar counters. Right now this only applies to filestore.journal_wr_bytes, which is a counter for OSD daemons. The ceph schema reports this metric type as a sum,count pair, while similar counters are treated as derive types. When converted, the sum is used as the counter value and gets treated as a derive type. When disabled, every metric is treated as whatever type it came in as from the ceph schema. No glow-ups, no conversions.

Default: Enabled

Every Daemon block must have a string argument for the plugin instance name. A SocketPath is also required for each Daemon block:

  • *Daemon DaemonName* โ€” the name used as the instance name for this daemon. ๐Ÿท๏ธ
  • *SocketPath SocketPath* โ€” specifies the path to the UNIX admin socket of the ceph daemon.

๐Ÿง Plugin "cgroups"

This plugin collects the CPU user/system time for each cgroup by reading the cpuacct.stat files in the first cpuacct-mountpoint โ€” typically /sys/fs/cgroup/cpu.cpuacct on machines running systemd. โœจ

  • *CGroup Directory* โ€” select cgroups by name. Whether only the matching cgroups get collected or whether they get ignored is controlled by the IgnoreSelected option, see right below. ๐Ÿ‘‡

See IGNORELISTS for details.

  • IgnoreSelected true|false โ€” inverts the selection. If it's true, all cgroups except the ones matching any one of the criteria get collected. By default only the selected cgroups are collected, but only if a selection is made. If there's no selection configured at all, then all cgroups are selected. Simple as that. ๐Ÿ’…

โฑ๏ธ Plugin "check_uptime"

The check_uptime plugin is designed to check and notify about host or service status based on the uptime metric. It's giving watchdog energy. ๐Ÿ•

When a new metric of uptime type shows up in cache, an OK notification is sent. When the new value for the metric is less than the previous value, a WARNING notification gets sent about a host/service restart. And when no new updates come in for the metric and the cache entry expires, then a FAILURE notification is sent about an unreachable host or service. ๐Ÿ˜ญ

By default (like, when there's no explicit configuration), the plugin checks for the uptime metric.

Synopsis:

 <Plugin "check_uptime">
   Type "uptime"
   Type "my_uptime_type"
 </Plugin>
  • *Type Type* โ€” the metric type to check for status/values. The type should consist of a single GAUGE data source. One. Just the one. ๐Ÿ’ฏ

๐Ÿ•ฐ๏ธ Plugin "chrony"

The chrony plugin collects ntp data from a chronyd server โ€” clock skew, per-peer stratum, all that good stuff. โฐ

To talk to chronyd it mimics what the chronyc control program does on the wire. Copying its fit, honestly.

Available config options for the chrony plugin:

  • *Host Hostname* โ€” hostname of the host running chronyd. Defaults to localhost. ๐Ÿ 
  • *Port Port* โ€” UDP-Port to connect to. Defaults to 323.
  • *Timeout Timeout* โ€” connection timeout in seconds. Defaults to 2.

๐ŸŒ Plugin Connectivity

connectivity โ€” the documentation for collectd's "connectivity plugin".

LoadPlugin connectivity
# ...
<Plugin connectivity>
  Interface eth0
</Plugin>

The "connectivity plugin" queries interface status using netlink (man 7 netlink), which provides info about network interfaces via the NETLINK_ROUTE family (man 7 rtnetlink). The plugin translates whatever value it receives into collectd's internal format and โ€” depending on which write plugins you've got loaded โ€” it may get written to disk or submitted to another instance. It listens to the interfaces enumerated in the plugin configuration (see below). And if no interfaces are listed at all, the default is to monitor all interfaces. Everything. The whole buffet. ๐Ÿ“

This example shows the "connectivity plugin" monitoring all interfaces:

LoadPlugin connectivity <Plugin connectivity> </Plugin>

This example shows the "connectivity plugin" monitoring 2 interfaces, "eth0" and "eth1":

LoadPlugin connectivity <Plugin connectivity>
  Interface eth0
  Interface eth1 </Plugin>

This example shows the "connectivity plugin" monitoring all interfaces except "eth1":

LoadPlugin connectivity <Plugin connectivity>
  Interface eth1
  IgnoreSelected true </Plugin>
  • *Interface interface_name* โ€” interface(s) to monitor connect to. ๐Ÿ”Œ

๐Ÿ•ต๏ธ Plugin "conntrack"

This plugin collects IP conntrack statistics. It's tracking connections, bestie, that's literally the whole job.

  • OldFiles โ€” assume the conntrack_count and conntrack_max files are found in /proc/sys/net/ipv4/netfilter instead of /proc/sys/net/netfilter/. Old school path, old school vibes. ๐Ÿ—‚๏ธ

๐Ÿ–ฅ๏ธ Plugin "cpu"

The CPU plugin collects CPU usage metrics. By default, CPU usage is reported as Jiffies, using the "cpu" type. There are two aggregations available:

  • Sum, per-state, over all CPUs installed in the system; and
  • Sum, per-CPU, over all non-idle states of a CPU, creating an "active" state.

You can combine the two aggregations, which leads to collectd emitting a single "active" metric for the entire system. The second one of these aggregations (or both, honestly) gets enabled and the cpu plugin will report a percentage rather than Jiffies. On top of that, you can request individual, per-state, per-CPU metrics to be reported as percentage. Math class is in session. ๐Ÿง 

The following config options are available:

  • ReportByState true|false โ€” when set to true (the default), it reports per-state metrics, e.g. "system", "user" and "idle". When set to false, it aggregates (sums) all non-idle states into one "active" metric. ๐ŸŒˆ
  • ReportByCpu true|false โ€” when set to true (the default), it reports per-CPU (per-core) metrics. When set to false, instead of reporting metrics for individual CPUs, only a global sum of CPU states gets emitted.
  • ValuesPercentage false|true โ€” this option is only considered when both ReportByCpu and ReportByState are set to true. In that case, metrics default to being reported as Jiffies. By setting this option to true, you can request percentage values in the un-aggregated (per-CPU, per-state) mode as well. ๐Ÿ’ฏ
  • ReportNumCpu false|true โ€” when set to true, reports the number of available CPUs. Defaults to false.
  • ReportGuestState false|true โ€” when set to true, reports the "guest" and "guest_nice" CPU states. Defaults to false. ๐Ÿ™ˆ
  • SubtractGuestState false|true โ€” this option is only considered when ReportGuestState is set to true. "guest" and "guest_nice" are included in "user" and "nice" respectively. If set to true, "guest" gets subtracted from "user" and "guest_nice" gets subtracted from "nice". Defaults to true. Subtraction slay. โž–

โšก Plugin "cpufreq"

This plugin is available on Linux and FreeBSD only. It doesn't have any options. Zero. None. It's iconic like that. ๐Ÿ‘‘

On Linux it reads /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq (for the first CPU installed) to get the current CPU frequency. If that file doesn't exist, make sure cpufreqd (http://cpufreqd.sourceforge.net/) or a similar tool is installed and that a "cpu governor" (that's a kernel module, bestie) is loaded. โš ๏ธ

On Linux, if the system has the cpufreq-stats kernel module loaded, this plugin reports the rate of p-state (cpu frequency) transitions and the percentage of time spent in each p-state. ๐Ÿ“Š

On FreeBSD it does a sysctl dev.cpu.0.freq and submits this as instance 0. At this time FreeBSD only has one frequency setting for all cores. See the BUGS section in the FreeBSD man page for cpufreq(4) for more details. ๐Ÿฆ‹

On FreeBSD the plugin checks the success of sysctl dev.cpu.0.freq and unregisters the plugin when this fails. A message will be logged to indicate this. ๐Ÿ’€

Plugin cpusleep

Okay so this one is literally the chillest plugin ever โ€” it has zero options. None. It just reads CLOCK_BOOTTIME and CLOCK_MONOTONIC and reports the difference between those two clocks. ๐Ÿ’ค

Here's the tea: the BOOTTIME clock keeps incrementing while your device is suspended, but the MONOTONIC clock does not. So the derivative of the difference between them gives you the relative amount of time the device has spent in suspend state. The recorded value is in milliseconds of sleep per seconds of wall clock. Iconic nap-tracking behavior, no cap. ๐Ÿ˜ดโœจ

Plugin csv

  • DataDir Directory โ€” sets the directory where CSV files get stored. Per default, CSV files are generated beneath the daemon's working directory, i.e. the BaseDir. The special strings stdout and stderr can be used to write to the standard output and standard error channels, respectively. This, of course, only makes much sense when collectd is running in foreground- or non-daemon-mode. ๐ŸŽ€
  • StoreRates true|false โ€” if set to true, counter values get converted to rates. If set to false (the default) counter values are stored as is, i.e. as an increasing integer number. ๐Ÿ”ข
  • FileDate true|false โ€” if set to true (the default value), the generated files will include the date. If set to false the date will not be included in the generated files. ๐Ÿ—“๏ธ

cURL Statistics

All cURL-based plugins support collection of generic, request-based statistics. These are disabled by default and can be enabled selectively for each page or URL queried from the curl, curl_json, or curl_xml plugins. See the documentation of those plugins for specific info. This section describes the available metrics that can be configured for each plugin. All options are disabled by default. ๐Ÿ™ˆ

See http://curl.haxx.se/libcurl/c/curl_easy_getinfo.html for more details.

  • TotalTime true|false โ€” total time of the transfer, including name resolving, TCP connect, etc. โฑ๏ธ
  • NamelookupTime true|false โ€” time it took from the start until name resolving was completed. ๐Ÿ”
  • ConnectTime true|false โ€” time it took from the start until the connect to the remote host (or proxy) was completed. ๐Ÿ”Œ
  • AppconnectTime true|false โ€” time it took from the start until the SSL/SSH connect/handshake to the remote host was completed. ๐Ÿ”
  • PretransferTime true|false โ€” time it took from the start until just before the transfer begins. ๐Ÿ
  • StarttransferTime true|false โ€” time it took from the start until the first byte was received. ๐Ÿ“ฅ
  • RedirectTime true|false โ€” time it took for all redirection steps include name lookup, connect, pre-transfer and transfer before final transaction was started. ๐ŸŒ€
  • RedirectCount true|false โ€” the total number of redirections that were actually followed. ๐Ÿ”
  • SizeUpload true|false โ€” the total amount of bytes that were uploaded. โฌ†๏ธ
  • SizeDownload true|false โ€” the total amount of bytes that were downloaded. โฌ‡๏ธ
  • SpeedDownload true|false โ€” the average download speed that curl measured for the complete download. ๐Ÿš€
  • SpeedUpload true|false โ€” the average upload speed that curl measured for the complete upload. ๐Ÿ’จ
  • HeaderSize true|false โ€” the total size of all the headers received. ๐Ÿ“จ
  • RequestSize true|false โ€” the total size of the issued requests. ๐Ÿ“ค
  • ContentLengthDownload true|false โ€” the content-length of the download. ๐Ÿ“
  • ContentLengthUpload true|false โ€” the specified size of the upload. ๐Ÿ“
  • NumConnects true|false โ€” the number of new connections that were created to achieve the transfer. ๐Ÿ”—

Plugin curl

The curl plugin uses the libcurl (http://curl.haxx.se/) to read web pages and the match infrastructure (the same code used by the tail plugin) to use regular expressions with the received data. ๐Ÿ•ธ๏ธโœจ

The following example will read the current value of AMD stock from Google's finance page and dispatch the value to collectd. ๐Ÿ“ˆ๐Ÿ’…

<Plugin curl>
  <Page "stock_quotes">
    Plugin "quotes"
    URL "http://finance.google.com/finance?q=NYSE%3AAMD"
    AddressFamily "any"
    User "foo"
    Password "bar"
    Digest false
    VerifyPeer true
    VerifyHost true
    CACert "/path/to/ca.crt"
    Header "X-Custom-Header: foobar"
    Post "foo=bar"

    MeasureResponseTime false
    MeasureResponseCode false

    <Match>
      Regex "<span +class=\"pr\"[^>]*> *([0-9]*\\.[0-9]+) *</span>"
      DSType "GaugeAverage"
      # Note: `stock_value' is not a standard type.
      Type "stock_value"
      Instance "AMD"
    </Match>
  </Page>
</Plugin>

In the Plugin block, there may be one or more Page blocks, each defining a web page and one or more "matches" to be performed on the returned data. The string argument to the Page block is used as plugin instance. ๐Ÿ—‚๏ธ

The following options are valid within Page blocks:

  • Plugin Plugin โ€” use Plugin as the plugin name when submitting values. Defaults to "curl". ๐Ÿท๏ธ
  • URL URL โ€” URL of the web site to retrieve. Since a regular expression will be used to extract information from this data, non-binary data is a big plus here ;) ๐Ÿ˜‰
  • AddressFamily Type โ€” IP version to resolve URL to. Useful in cases when hostname in URL resolves to both IPv4 and IPv6 addresses, and you are interested in using one of them specifically. Use "ipv4" to enforce IPv4, "ipv6" to enforce IPv6, or "any" to keep the default behavior of resolving addresses to all IP versions your system allows. If "libcurl" is compiled without IPv6 support, using "ipv6" will result in a warning and fallback to "any". If Type cannot be parsed, a warning will be printed and the whole Page block will be ignored. โš ๏ธ๐ŸŒ
  • User Name โ€” username to use if authorization is required to read the page. ๐Ÿ™‹โ€โ™€๏ธ
  • Password Password โ€” password to use if authorization is required to read the page. ๐Ÿคซ๐Ÿ”‘
  • Digest true|false โ€” enable HTTP digest authentication. ๐Ÿงพ
  • VerifyPeer true|false โ€” enable or disable peer SSL certificate verification. See http://curl.haxx.se/docs/sslcerts.html for details. Enabled by default. ๐Ÿ›ก๏ธ
  • VerifyHost true|false โ€” enable or disable peer host name verification. If enabled, the plugin checks if the "Common Name" or a "Subject Alternate Name" field of the SSL certificate matches the host name provided by the URL option. If this identity check fails, the connection is aborted. Obviously, only works when connecting to a SSL enabled server. Enabled by default. ๐Ÿชช
  • CACert file โ€” file that holds one or more SSL certificates. If you want to use HTTPS you will possibly need this option. What CA certificates come bundled with "libcurl" and are checked by default depends on the distribution you use. ๐Ÿ“œ
  • Header Header โ€” a HTTP header to add to the request. Multiple headers are added if this option is specified more than once. ๐Ÿ“‹
  • Post Body โ€” specifies that the HTTP operation should be a POST instead of a GET. The complete data to be posted is given as the argument. This option will usually need to be accompanied by a Header option to set an appropriate "Content-Type" for the post body (e.g. to "application/x-www-form-urlencoded"). ๐Ÿ“ฎ
  • MeasureResponseTime true|false โ€” measure response time for the request. If this setting is enabled, Match blocks (see below) are optional. Disabled by default. โณ

Beware that requests will get aborted if they take too long to complete. Adjust Timeout accordingly if you expect MeasureResponseTime to report such slow requests. ๐Ÿ’€

This option is similar to enabling the TotalTime statistic but it's measured by collectd instead of cURL.

  • MeasureResponseCode true|false โ€” measure response code for the request. If this setting is enabled, Match blocks (see below) are optional. Disabled by default. ๐Ÿ”ข
  • <Statistics> โ€” one Statistics block can be used to specify cURL statistics to be collected for each request to the remote web site. See the section "cURL Statistics" above for details. If this setting is enabled, Match blocks (see below) are optional. ๐Ÿ“Š
  • <Match> โ€” one or more Match blocks that define how to match information in the data returned by "libcurl". The "curl" plugin uses the same infrastructure that's used by the "tail" plugin, so please see the documentation of the "tail" plugin below on how matches are defined. If the MeasureResponseTime or MeasureResponseCode options are set to true, Match blocks are optional. ๐ŸŽฏ
  • Interval Interval โ€” sets the interval (in seconds) in which the values will be collected from this URL. By default the global Interval setting will be used. ๐Ÿ”„
  • Timeout Milliseconds โ€” the Timeout option sets the overall timeout for HTTP requests to URL, in milliseconds. By default, the configured Interval is used to set the timeout. Prior to version 5.5.0, there was no timeout and requests could hang indefinitely. This legacy behaviour can be achieved by setting the value of Timeout to 0. โฐ

If Timeout is 0 or bigger than the Interval, keep in mind that each slow network connection will stall one read thread. Adjust the ReadThreads global setting accordingly to prevent this from blocking other plugins. ๐Ÿงต๐Ÿšจ


Plugin curl_json

The curl_json plugin collects values from JSON data to be parsed by libyajl (https://lloyd.github.io/yajl/) retrieved via either libcurl (http://curl.haxx.se/) or read directly from a unix socket. The former can be used, for example, to collect values from CouchDB documents (which are stored JSON notation), and the latter to collect values from a uWSGI stats socket. ๐Ÿ๐Ÿฆ‹

The following example will collect several values from the built-in "_stats" runtime statistics module of CouchDB (http://wiki.apache.org/couchdb/Runtime_Statistics). ๐Ÿ’พ

<Plugin curl_json>
  <URL "http://localhost:5984/_stats">
    AddressFamily "any"
    Instance "httpd"
    <Key "httpd/requests/count">
      Type "http_requests"
    </Key>

    <Key "httpd_request_methods/*/count">
      Type "http_request_methods"
    </Key>

    <Key "httpd_status_codes/*/count">
      Type "http_response_codes"
    </Key>
  </URL>
</Plugin>

This example will collect data directly from a uWSGI "Stats Server" socket. ๐Ÿงฆ

<Plugin curl_json>
  <Sock "/var/run/uwsgi.stats.sock">
    Instance "uwsgi"
    <Key "workers/*/requests">
      Type "http_requests"
    </Key>

    <Key "workers/*/apps/*/requests">
      Type "http_requests"
    </Key>
  </Sock>
</Plugin>

Okay so inside the Plugin block you can have one or more URL blocks โ€” each one defines a URL to be fetched via HTTP (using libcurl, obviously) โ€” or Sock blocks, which define a unix socket to read JSON from directly. And each of those blocks can have one or more Key blocks. ๐Ÿ’…

The Key string argument has to be in a path format, bestie. Each component is used to match the key from a JSON map or the index of a JSON array. And if a path component of a Key is a * wildcard, then the values for all map keys or array indices will be collected. Iconic.

The following options are valid within URL blocks:

  • AddressFamily Type โ€” which IP version to resolve the URL to. Super useful when the hostname in the URL resolves to both IPv4 and IPv6 addresses and you're interested in using one of them specifically. Use "ipv4" to enforce IPv4, "ipv6" to enforce IPv6, or "any" to keep the default behavior of resolving addresses to all IP versions your system allows. If "libcurl" is compiled without IPv6 support, using "ipv6" will result in a warning and fallback to "any". And if Type can't be parsed, a warning will be printed and the whole URL block will be ignored. ๐Ÿ˜ญ
  • Host Name โ€” use Name as the host name when submitting values. Defaults to the global host name setting.
  • Plugin Plugin โ€” use Plugin as the plugin name when submitting values. Defaults to "curl_json".
  • Instance Instance โ€” sets the plugin instance to Instance.
  • Interval Interval โ€” sets the interval (in seconds) in which the values will be collected from this URL. By default the global Interval setting will be used.
  • User Name
  • Password Password
  • Digest true|false
  • VerifyPeer true|false
  • VerifyHost true|false
  • CACert file
  • Header Header
  • Post Body
  • Timeout Milliseconds

These options behave exactly equivalent to the appropriate options of the cURL plugin โ€” please see there for a detailed description. ๐Ÿซถ

  • <Statistics> โ€” one Statistics block can be used to specify cURL statistics to be collected for each request to the remote URL. See the section "cURL Statistics" above for details. โœจ

And the following options are valid within Key blocks:

  • Type Type โ€” sets the type used to dispatch the values to the daemon. Detailed information about types and their configuration can be found in types.db(5). This option is mandatory. ๐Ÿ‘‘
  • Instance Instance โ€” type-instance to use. Defaults to the current map key or current string array element value.

๐ŸŒธ Plugin "curl_jolokia" ๐ŸŒธ

The curl_jolokia plugin collects values from MBeanServer โ€” servlet engines equipped with the jolokia (<https://jolokia.org>) MBean. It sends a pre-configured JSON-Postbody to the servlet via HTTP commanding the jolokia Bean to reply with a single JSON equipped with all JMX counters requested. By reducing TCP roundtrips in comparison to conventional JMX clients that query one value via tcp at a time, it can return hundreds of values in one roundtrip. Moreof โ€” no java binding is required in collectd to do so. ๐Ÿคฏ

It uses libyajl (<https://lloyd.github.io/yajl/>) to parse the Jolokia JSON reply retrieved via libcurl (<http://curl.haxx.se/>).

 <Plugin curl_jolokia>
   <URL "http://10.10.10.10:7101/jolokia-war-1.2.0/?ignoreErrors=true&canonicalNaming=false";>
     Host "_APPPERF_JMX"
     User "webloginname"
     Password "passvoid"
     Post <JOLOKIA json post data>

   <BeanName "PS_Scavenge">
        MBean "java.lang:name=PS Scavenge,type=GarbageCollector"
        BeanNameSpace "java_lang"
        <AttributeName "collectiontime" >
               Attribute "CollectionTime"
               type "gauge"
        </AttributeName>
        <AttributeName "collectioncount" >
               Attribute "CollectionCount"
               type "gauge"
        </AttributeName>
   </BeanName>
  </Plugin>

The plugin is intended to be written in a simple manner. Thus it doesn't try to solve the task of generating the jolokia post data, or automatically map the values, but rather leans on a verbose config containing the prepared flat JSON post data and a config section per gauge transformed (as one sample shown above). However, Jolokia can output all available gauges, and we have a python script to filter them, and generate a configuration for you:

 jolokia_2_collectcfg.py

It can gather all interesting gauges, write a simple one value per line config for itself and subsequent calls. You can remove lines from this file manually, or create filter lists. You then use the script to generate a collectd config. The script can then inspect data files from some testruns, and remove all gauges, that don't contain any movement. ๐Ÿ’–

The base config looks like this:

The following options are valid within URL blocks:

  • Host Name โ€” use Name as the host name when submitting values. Defaults to the global host name setting.
  • Plugin Plugin โ€” use Plugin as the plugin name when submitting values. Defaults to "curl_jolokia".
  • Instance Instance โ€” sets the plugin instance to Instance.
  • Interval Interval โ€” sets the interval (in seconds) in which the values will be collected from this URL. By default the global Interval setting will be used.
  • User Name
  • Password Password
  • Digest true|false
  • VerifyPeer true|false
  • VerifyHost true|false
  • CACert file
  • Header Header
  • Post Body
  • Timeout Milliseconds

These options behave exactly equivalent to the appropriate options of the cURL plugin โ€” please see there for a detailed description. ๐ŸŒˆ

  • <BeanName> โ€” one BeanName block configures the translation of the gauges of one bean to their respective collectd names, where BeanName sets the main name.
  • MBean MBean โ€” the name of the Bean on the server.
  • BeanNameSpace BeanNameSpace โ€” the name space the Bean resides under.
    • AttributeName AttributeName โ€” a bean can contain several Attributes with gauges. Each one can be matched by an AttributeName section or be ignored.
    • Attribute Attribute โ€” how should this attribute be called under the BeanName in the collectd hierarchy?
    • Type Type โ€” sets the type used to dispatch the values to the daemon. Detailed information about types and their configuration can be found in types.db(5). This option is mandatory. ๐Ÿ’…

๐Ÿฆ‹ Plugin "curl_xml" ๐Ÿฆ‹

The curl_xml plugin uses libcurl (<http://curl.haxx.se/>) and libxml2 (<http://xmlsoft.org/>) to retrieve XML data via cURL.

 <Plugin "curl_xml">
   <URL "http://localhost/stats.xml">
     AddressFamily "any"
     Host "my_host"
     #Plugin "curl_xml"
     Instance "some_instance"
     User "collectd"
     Password "thaiNg0I"
     VerifyPeer true
     VerifyHost true
     CACert "/path/to/ca.crt"
     Header "X-Custom-Header: foobar"
     Post "foo=bar"

     <XPath "table[@id=\"magic_level\"]/tr">
       Type "magic_level"
       #InstancePrefix "prefix-"
       InstanceFrom "td[1]"
       #PluginInstanceFrom "td[1]"
       ValuesFrom "td[2]/span[@class=\"level\"]"
     </XPath>
   </URL>
 </Plugin>

In the Plugin block, there may be one or more URL blocks, each defining a URL to be fetched using libcurl. Within each URL block there are options which specify the connection parameters โ€” for example authentication information โ€” and one or more XPath blocks. ๐ŸŽ€

Each XPath block specifies how to get one type of information. The string argument must be a valid XPath expression which returns a list of "base elements". One value is dispatched for each "base element". The type instance and values are looked up using further XPath expressions that should be relative to the base element. โœจ

Within the URL block the following options are accepted:

  • AddressFamily Type โ€” IP version to resolve the URL to. Useful in cases when the hostname in the URL resolves to both IPv4 and IPv6 addresses, and you're interested in using one of them specifically. Use "ipv4" to enforce IPv4, "ipv6" to enforce IPv6, or "any" to keep the default behavior of resolving addresses to all IP versions your system allows. If "libcurl" is compiled without IPv6 support, using "ipv6" will result in a warning and fallback to "any". If Type cannot be parsed, a warning will be printed and the whole URL block will be ignored.
  • Host Name โ€” use Name as the host name when submitting values. Defaults to the global host name setting.
  • Plugin Plugin โ€” use Plugin as the plugin name when submitting values. Defaults to 'curl_xml'.
  • Instance Instance โ€” use Instance as the plugin instance when submitting values. May be overridden by the PluginInstanceFrom option inside XPath blocks. Defaults to an empty string (no plugin instance).
  • Interval Interval โ€” sets the interval (in seconds) in which the values will be collected from this URL. By default the global Interval setting will be used.
  • Namespace Prefix URL โ€” if an XPath expression references namespaces, they must be specified with this option. Prefix is the "namespace prefix" used in the XML document. URL is the "namespace name", an URI reference uniquely identifying the namespace. The option can be repeated to register multiple namespaces.

Examples:

 Namespace "s" "http://schemas.xmlsoap.org/soap/envelope/"
 Namespace "m" "http://www.w3.org/1998/Math/MathML"
  • User User
  • Password Password
  • Digest true|false
  • VerifyPeer true|false
  • VerifyHost true|false
  • CACert CA Cert File
  • Header Header
  • Post Body
  • Timeout Milliseconds

These options behave exactly equivalent to the appropriate options of the cURL plugin โ€” please see there for a detailed description. ๐Ÿ’œ

  • <Statistics> โ€” one Statistics block can be used to specify cURL statistics to be collected for each request to the remote URL. See the section "cURL Statistics" above for details.
  • <XPath XPath-expression> โ€” within each URL block, there must be one or more XPath blocks. Each XPath block specifies how to get one type of information. The string argument must be a valid XPath expression which returns a list of "base elements". One value is dispatched for each "base element". ๐ŸŒŸ

Within the XPath block the following options are accepted:

  • Type Type โ€” specifies the Type used for submitting patches. This determines the number of values that are required / expected and whether the strings are parsed as signed or unsigned integer or as double values. See types.db(5) for details. This option is required. ๐Ÿ‘‘
  • InstancePrefix InstancePrefix โ€” slaps your InstancePrefix onto the front of the type instance ๐ŸŽ€ the values just get squished together with no separator at all, jsyk. Totally optional, babe.
  • InstanceFrom InstanceFrom โ€” gives you a XPath expression that decides the type instance. Rule: it has to return exactly one element, no more no less ๐Ÿ˜ค that element's value becomes the type instance, possibly with InstancePrefix stuck on the front (see above, bestie).
  • PluginInstanceFrom PluginInstanceFrom โ€” same deal but for the plugin instance โœจ XPath expression, must return exactly one element, and that element's value is your plugin instance. Periodt.

So here's the vibe check on those two ๐Ÿ‘‘ if the "base XPath expression" โ€” aka the argument you hand to the XPath block โ€” returns exactly one argument, then InstanceFrom and PluginInstanceFrom can both dip, they're optional. But if it doesn't? Then you need at least one of InstanceFrom or PluginInstanceFrom. No arguing with the math ๐Ÿ’…

  • ValuesFrom ValuesFrom [ValuesFrom ...] โ€” one or more XPath expressions for reading the actual values ๐Ÿ“Š the number of XPath expressions must match the number of data sources in the type you set with Type (see above, iykyk). Each XPath expression has to return exactly one element, and that element's value gets parsed as a number and used as the value for the matching spot in the value list that gets dispatched to the daemon. This one is required, no skipping ๐Ÿ™ˆ

๐Ÿงš Plugin "dbi" ๐Ÿงš

Okay so this one's the database girlie ๐Ÿ’œ it uses the dbi library (<http://libdbi.sourceforge.net/>) to connect to a whole bunch of databases, run SQL statements, and read the results back. And yes, dbi is short for "database interface" โ€” in case the name was keeping you up at night, you're welcome ๐Ÿ˜Œ

You get to configure how each column should be interpreted, and the plugin will generate one or more data sets from every row that comes back, according to those rules. Iconic behaviour honestly.

Because this plugin is so generic, the config is a lil more complex than the other plugins. Usually it looks something like this:

<Plugin dbi>
  <Query "out_of_stock">
    Statement "SELECT category, COUNT(*) AS value FROM products WHERE in_stock = 0 GROUP BY category"
    # Use with MySQL 5.0.0 or later
    MinVersion 50000
    <Result>
      Type "gauge"
      InstancePrefix "out_of_stock"
      InstancesFrom "category"
      ValuesFrom "value"
    </Result>
  </Query>
  <Database "product_information">
    #Plugin "warehouse"
    Driver "mysql"
    Interval 120
    DriverOption "host" "localhost"
    DriverOption "username" "collectd"
    DriverOption "password" "aZo6daiw"
    DriverOption "dbname" "prod_info"
    SelectDB "prod_info"
    Query "out_of_stock"
  </Database>
</Plugin>

So what's happening up there ๐Ÿฆ‹ that config defines one query with one result and one database. The query then gets linked to the database using the Query option inside the <Database> block. You can have as many queries and databases as you want, and you can also use the Include statement to split your config file into multiple smaller files ๐ŸŽ€

โš ๏ธ BUT โ€” the <Query> block has to come before the <Database> blocks, because the file is read from top to bottom! Do not test her on this ๐Ÿ’€

Here's the complete list of options, no cap:


๐ŸŒธ Query blocks ๐ŸŒธ

Query blocks define SQL statements plus how the returned data should be interpreted. They're identified by the name you give in the opening line of the block โ€” so that name needs to be unique โญ other than that, collectd doesn't use the name at all.

Inside each Query block there's one or more Result blocks. Result blocks define which column holds which value or instance information. You can use multiple Result blocks to make multiple values out of one returned row โ€” which is especially useful when your queries take forever and sending basically the same query again and again is just not the move ๐Ÿ˜ญ

Example:

<Query "environment">
  Statement "select station, temperature, humidity from environment"
  <Result>
    Type "temperature"
    # InstancePrefix "foo"
    InstancesFrom "station"
    ValuesFrom "temperature"
  </Result>
  <Result>
    Type "humidity"
    InstancesFrom "station"
    ValuesFrom "humidity"
  </Result>
</Query>

The following options are accepted:

  • Statement SQL โ€” sets the statement that gets executed on the server. collectd does not interpret this, it just hands it straight to the database server ๐Ÿซถ so the SQL dialect you're using depends entirely on which server collectd is connected to.

The query has to return at least two columns โ€” one for the instance and one for the value. You cannot omit the instance, even if the statement is guaranteed to always return exactly one line ๐Ÿ™ƒ in that case you can usually do something like this:

Statement "SELECT \"instance\", COUNT(*) AS value FROM table"

(That works with MySQL but may not be valid SQL according to the spec. If you're using a more strict database server, you may have to select from a dummy table or something. Annoying but true ๐Ÿ’€)

Also please note: some databases โ€” Oracle, for example โ€” will literally fail if you put a semicolon at the end of the statement. So don't โœ‹

  • MinVersion Version
  • MaxVersion Value โ€” only use this query for the specified database version ๐ŸŽฏ you can use these two to provide multiple queries with the same name but slightly different syntax. The plugin will only use the queries where the specified min and max versions fit the version of the database currently in use.

The database version comes from dbi_conn_get_engine_version โ€” see the libdbi docs at <http://libdbi.sourceforge.net/docs/programmers-guide/reference-conn.html#DBI-CONN-GET-ENGINE-VERSION> for the deets. Basically each part of the version is assumed to be in the range from 00 to 99, and all the dots get removed ๐ŸŽ€ so version "4.1.2" becomes "40102", and version "5.0.42" becomes "50042".

โš ๏ธ Warning: the plugin will use all matching queries! So if you specify multiple queries with the same name and overlapping ranges, weird stuff will happen ๐Ÿ˜ญ don't do it! A valid example would be something along these lines:
MinVersion 40000
MaxVersion 49999
...
MinVersion 50000
MaxVersion 50099
...
MinVersion 50100
# No maximum

In the example above there are three ranges that don't overlap ๐Ÿ‘‘ the last one goes from version "5.1.0" to infinity, meaning "all later versions". Versions before "4.0.0" are not specified.

  • Type Type โ€” the type used for each line returned. See types.db(5) for more details on how types are defined. In short: a type is a predefined layout of data, and the number of values and the type of values has to match the type definition ๐Ÿงš

If you specify "temperature" here, you need exactly one gauge column. If you specify "if_octets", you'll need two counter columns. See the ValuesFrom setting below ๐Ÿ‘‡

There must be exactly one Type option inside each Result block. Not zero, not two ๐Ÿ’…

  • InstancePrefix prefix โ€” prepends prefix to the type instance. If InstancesFrom (see below) isn't given, the string is just copied over. If InstancesFrom is given, then prefix and all the strings returned in the appropriate columns get concatenated together, separated by dashes ("-") ๐ŸŒˆ
  • InstancesFrom column0 [column1 ...] โ€” names the columns whose values will be used to create the "type-instance" for each row. If you give more than one column, all their values get joined together with dashes ("-") between them โœจ

The plugin itself does not check whether all the instances it builds are different ๐Ÿ™ˆ that's your responsibility, bestie โ€” you have to make sure each one is unique. This goes double if you don't specify InstancesFrom: in that case you have to make sure only one row is returned.

If neither InstancePrefix nor InstancesFrom is given, the type-instance will be empty ๐Ÿซฅ

  • ValuesFrom column0 [column1 ...] โ€” names the columns whose content is used as the actual data for the data sets dispatched to the daemon ๐Ÿ“Š how many of these columns you need is decided by the Type setting above. If you specify too many or not enough columns, the plugin will complain about it and no data will be submitted to the daemon ๐Ÿ˜ญ

The actual data type in the columns isn't that important โ€” the plugin will automatically cast the values to the right type if it knows how ๐Ÿ’– so it should be able to handle integer and floating point types, as well as strings (if they include a number at the beginning).

There must be at least one ValuesFrom option inside each Result block. Required, periodt.

  • MetadataFrom [column0 column1 ...] โ€” names the columns whose content is used as metadata for the data sets that get dispatched to the daemon ๐Ÿท๏ธ

Same as above, the actual data type in the columns isn't that important โ€” the plugin will automatically cast the values to the right type if it knows how. So it should be able to handle integer and floating point types, as well as strings (if they include a number at the beginning) ๐Ÿ’œ


๐Ÿ’… Database blocks ๐Ÿ’…

Database blocks define a connection to a database and which queries should be sent to that database. Since the "dbi" library can handle a whole wide variety of databases, the config is super generic ๐Ÿฆ‹ if in doubt, go read libdbi's documentation โ€” we stick as close as we can to the terminology used there.

Each database needs a "name" as a string argument in the starting tag of the block. That name gets used as the "PluginInstance" in the values submitted to the daemon โœจ other than that, the name isn't used for anything.

  • Plugin Plugin โ€” use Plugin as the plugin name when submitting query results from this Database. Defaults to "dbi" ๐ŸŽ€
  • Interval Interval โ€” sets the interval (in seconds) in which values get collected from this database โฑ๏ธ by default the global Interval setting is used.
  • Driver Driver โ€” specifies the driver used to connect to the database. In a lot of cases these drivers are named after the database they connect to, but that's not a technical necessity ๐Ÿ‘‘ these drivers are sometimes called "DBD", DataBase Driver, and some distributions ship them in separate packages. Drivers for the "dbi" library are developed by the libdbi-drivers project at <http://libdbi-drivers.sourceforge.net/> ๐ŸŒŸ

You need to give the driver name as expected by the dbi library here โ€” like, exactly what the library calls it, okurrr? Bestie, you should be able to find that in the docs for each individual driver. And if you typo the driver name? ๐Ÿ˜ญ The plugin will dump a whole list of every known driver name into the log so you can go "oh. oops." and fix it.

  • DriverOption Key Value โ€” sets driver-specific options. Which options a driver actually supports lives in that driver's docs, somewhere over at <http://libdbi-drivers.sourceforge.net/>. That said, the options "host", "username", "password", and "dbname" are giving de facto standards energy โ€” everybody uses them. โœจ

DBDs can register two types of options: string options and numeric options. The plugin goes dbi_conn_set_option when your config hands it a string, and dbi_conn_require_option_numeric when your config hands it a number. So these two lines literally result in different calls being used โ€” the quotes MATTER, bestie:

DriverOption "Port" 1234      # numeric
DriverOption "Port" "1234"    # string

Ngl, drivers are lowkey not that keen to report errors when you pass them an unknown option, so invalid settings here may just go totally unnoticed ๐Ÿ™ˆ. That's not the plugin's fault โ€” it will report errors if it gets them from the library / the driver. And if a driver does complain about an option, the plugin dumps a complete list of all options that driver understands into the log. Unfortunately there's no way to programmatically find out whether an option wants a string or a numeric argument, so you're gonna have to refer to the appropriate DBD's documentation to find out. Sorry ๐Ÿ’€.

  • SelectDB Database โ€” sometimes the database name you connected with isn't the database name you actually wanna query data from. If this option is set, the plugin will "select" (switch to) that database right after the connection is established. Slay, context switch.
  • Query QueryName โ€” associates the query named QueryName with this database connection. The query needs to be defined before this statement, i.e. all the query blocks you want to refer to must be placed above the database block you're referring to them from. Order matters, queen ๐Ÿ‘‘.
  • Host Hostname โ€” sets the host field of value lists to Hostname when dispatching values. Defaults to the global hostname setting.

โœจ Plugin "dcpmm" โœจ

The dcpmm plugin collects Intel(R) Optane(TM) DC Persistent Memory related performance statistics. ๐Ÿ’พ The plugin requires root privileges to perform the statistics collection โ€” so yeah, main character energy but with sudo.

Synopsis:

<Plugin "dcpmm">
  Interval 10.0
  CollectHealth false
  CollectPerfMetrics true
  EnableDispatchAll false
</Plugin>
  • Interval time in seconds โ€” sets the interval (in seconds) in which the values get collected. Defaults to the "global Interval" value. This overrides the global Interval for the dcpmm plugin only โ€” none of the other plugins will be affected. No collateral damage ๐Ÿซถ.
  • CollectHealth true|false โ€” collects health information. CollectHealth and CollectPerfMetrics cannot be true at the same time. Defaults to "false".

The health information metrics are literally these:

  • health_status โ€” overall health summary (0: normal | 1: non-critical | 2: critical | 3: fatal).
  • lifespan_remaining โ€” the module's remaining life as a percentage value of factory expected life span.
  • lifespan_used โ€” the module's used life as a percentage value of factory expected life span.
  • power_on_time โ€” the lifetime the DIMM has been powered on in seconds.
  • uptime โ€” the current uptime of the DIMM for the current power cycle in seconds.
  • last_shutdown_time โ€” the time the system was last shutdown. The time is represented in epoch (seconds).
  • media_temperature โ€” the media's current temperature in degree Celsius.
  • controller_temperature โ€” the controller's current temperature in degree Celsius.
  • max_media_temperature โ€” the media's the highest temperature reported in degree Celsius.
  • max_controller_temperature โ€” the controller's highest temperature reported in degree Celsius.
  • tsc_cycles โ€” the number of tsc cycles during each interval.
  • epoch โ€” the timestamp in seconds at which the metrics are collected from DCPMM DIMMs.
  • CollectPerfMetrics true|false โ€” collects memory performance metrics. CollectHealth and CollectPerfMetrics cannot be true at the same time. Defaults to "true".

The memory performance metrics are these icons:

  • total_bytes_read โ€” number of bytes transacted by the read operations.
  • total_bytes_written โ€” number of bytes transacted by the write operations.
  • read_64B_ops_rcvd โ€” number of read operations performed to the physical media in 64 bytes granularity.
  • write_64B_ops_rcvd โ€” number of write operations performed to the physical media in 64 bytes granularity.
  • media_read_ops โ€” number of read operations performed to the physical media.
  • media_write_ops โ€” number of write operations performed to the physical media.
  • host_reads โ€” number of read operations received from the CPU (memory controller).
  • host_writes โ€” number of write operations received from the CPU (memory controller).
  • read_hit_ratio โ€” measures the efficiency of the buffer in the read path. Range of 0.0 - 1.0.
  • write_hit_ratio โ€” measures the efficiency of the buffer in the write path. Range of 0.0 - 1.0.
  • tsc_cycles โ€” the number of tsc cycles during each interval.
  • epoch โ€” the timestamp in seconds at which the metrics are collected from DCPMM DIMMs.
  • EnableDispatchAll false โ€” this parameter helps to seamlessly enable simultaneous health and memory perf metrics collection in the future ๐Ÿ”ฎ. It's unused at the moment and must always be false. Like, don't touch it, bestie.

๐ŸŽ€ Plugin "df" ๐ŸŽ€

  • Device Device โ€” select partitions based on the devicename.

See /"IGNORELISTS" for details.

  • MountPoint Directory โ€” select partitions based on the mountpoint.

See /"IGNORELISTS" for details.

  • FSType FSType โ€” select partitions based on the filesystem type.

See /"IGNORELISTS" for details.

  • IgnoreSelected true|false โ€” inverts the selection: if set to true, all partitions except the ones matching any one of the criteria get collected. By default only selected partitions are collected if a selection is made. If no selection is configured at all, all partitions are selected. Easy ๐Ÿ’….
  • LogOnce false|false โ€” only log stat() errors once.
  • ReportByDevice true|false โ€” report using the device name rather than the mountpoint. Like, with this false (the default), it reports a disk as "root", but with it true, it's "sda1" (or whichever). Pick your fighter.
  • ReportInodes true|false โ€” enables or disables reporting of free, reserved and used inodes. Defaults to inode collection being disabled.

Enable this option if inodes are a scarce resource for you โ€” usually because many small files are stored on the disk. This is a usual scenario for mail transfer agents and web caches. ๐Ÿ“ฌ

  • ValuesAbsolute true|false โ€” enables or disables reporting of free and used disk space in 1K-blocks. Defaults to true.
  • ValuesPercentage false|true โ€” enables or disables reporting of free and used disk space in percentage. Defaults to false.

This is useful for deploying collectd on the cloud โ˜๏ธ, where machines with different disk size may exist. Then it's way more practical to configure thresholds based on relative disk size. Big brain.


๐Ÿฆ‹ Plugin "disk" ๐Ÿฆ‹

The disk plugin collects information about the usage of physical disks and logical disks (partitions). The values collected are the number of octets written to and read from a disk or partition, the number of read/write operations issued to the disk, and a rather complex "time" it took for these commands to be issued. It's giving so much data ๐Ÿ”ฅ.

Using the following two options you can ignore some disks or configure the collection of only specific disks.

  • Disk Name โ€” select the disk Name. Whether it's collected or ignored depends on the IgnoreSelected setting, see below. As with other plugins that use the daemon's ignorelist functionality, a string that starts and ends with a slash is interpreted as a regular expression. Examples:
Disk "sdd"
Disk "/hda[34]/"

See /"IGNORELISTS" for details.

  • IgnoreSelected true|false โ€” sets whether selected disks, i.e. the ones matched by any of the Disk statements, are ignored, or if all the other disks are ignored. The behavior (hopefully) is intuitive: if no Disk option is configured, all disks are collected. If at least one Disk option is given and IgnoreSelected is absent or set to false, only matching disks will be collected. If IgnoreSelected is set to true, all disks are collected except the ones matched. See? Intuitive. ๐Ÿ˜Œ
  • UseBSDName true|false โ€” whether to use the device's "BSD Name", on Mac OS X, instead of the default major/minor numbers. Requires collectd to be built with Apple's IOKitLib support. ๐ŸŽ
  • UdevNameAttr Attribute โ€” attempt to override the disk instance name with the value of a specified udev attribute when built with libudev. If the attribute isn't defined for the given device, the default name is used. Example:
UdevNameAttr "DM_NAME"
Please note that using an attribute that does not differentiate between the whole disk and its particular partitions (like ID_SERIAL) will result in data about the whole disk and each partition being mixed together incorrectly. ๐Ÿ’€ In this case, you can use the ID_COLLECTD attribute that is provided by contrib/99-storage-collectd.rules udev rule file instead.
  • ReportDiscard true|false โ€” whether to submit discard metrics (/proc/diskstats 15-18), on Linux only. Requires Linux Kernel version > 4.18+.
  • ReportFlush true|false โ€” whether to submit flush metrics (/proc/diskstats 19-20), on Linux only. Requires Linux Kernel version > 5.5+.

๐ŸŒˆ Plugin "dns" ๐ŸŒˆ

  • Interface Interface โ€” the dns plugin uses libpcap to capture dns traffic and analyzes it. This option sets the interface that should be used. If this option is not set, or set to "any", the plugin will try to get packets from all interfaces. This may not work on certain platforms, such as Mac OS X. ๐Ÿ™ˆ
  • IgnoreSource IP-address โ€” ignore packets that originate from this address. Bye, blocked. โœŒ๏ธ
  • SelectNumericQueryTypes true|false โ€” on by default โœจ it collects the query types it doesn't recognise (which is why they show up as numeric only, bestie, don't panic).

๐Ÿฆ‹ Plugin "dpdkevents"

The dpdkevents plugin is the one that collects events straight out of DPDK โ€” think link status of your network ports and the Keep Alive status of DPDK logical cores. Iconic, honestly ๐Ÿ’…

Heads up though: to get Keep Alive events you gotta meet these requirements โ€” DPDK >= 16.07, and support for Keep Alive has to actually be implemented in the DPDK application. No implementation, no events, iykyk ๐Ÿ˜ญ

More details live here: http://dpdk.org/doc/guides/sample_app_ug/keep_alive.html

Synopsis:

 <Plugin "dpdkevents">
   <EAL>
     Coremask "0x1"
     MemoryChannels "4"
     FilePrefix "rte"
   </EAL>
   <Event "link_status">
     SendEventsOnUpdate true
     EnabledPortMask 0xffff
     PortName "interface1"
     PortName "interface2"
     SendNotification false
   </Event>
   <Event "keep_alive">
     SendEventsOnUpdate true
     LCoreMask "0xf"
     KeepAliveShmName "/dpdk_keepalive_shm_name"
     SendNotification false
   </Event>
 </Plugin>

Options:

The EAL block ๐Ÿงš

  • Coremask Mask โ€” the mask of cores to run on.
  • Memorychannels Channels โ€” number of memory channels per processor socket.
  • FilePrefix File โ€” the prefix text used for hugepage filenames. The filename gets set to /var/run/.<prefix>_config where prefix is whatever you passed in. Cute, right? ๐ŸŽ€

The Event block โญ

The Event block is where you configure one specific event. It takes a single argument โ€” the name of the event. That's it, that's the whole vibe.

Link Status event ๐Ÿ”ฅ

  • SendEventOnUpdate true|false โ€” if you set this to true, the link status value is only dispatched when it's actually different from the previously read value. Optional argument, and the default value is true.
  • EnabledPortMask Mask โ€” a hexadecimal bit mask of the DPDK ports that should be enabled. A mask of 0x0 means all ports get disabled (๐Ÿ˜ญ why), and a bitmask of all F's means all ports are enabled. Optional โ€” by default all ports are enabled.
  • PortName Name โ€” a string containing an optional name for the enabled DPDK ports. Each PortName option should contain only one port name, so just stack up as many PortName options as you want ๐Ÿ’– Default naming convention is used if PortName is blank. And if you've got fewer PortName options than there are enabled ports, the default naming convention covers the extra ports.
  • SendNotification true|false โ€” if true, link status notifications get sent instead of the link status being collected as a statistic. Optional, default value is false.

Keep Alive event ๐Ÿ‘‘

  • SendEventOnUpdate true|false โ€” if true, the keep alive value is only dispatched when it differs from the previously read value. Optional, default value is true.
  • LCoreMask Mask โ€” a hexadecimal bit mask of the logical cores to monitor keep alive state on.
  • KeepAliveShmName Name โ€” the shared memory name identifier that the secondary process uses to monitor the keep alive cores state.
  • SendNotification true|false โ€” if true, keep alive notifications are sent, instead of keep alive information being collected as a statistic. Optional, default value is false.

๐ŸŒธ Plugin "dpdkstat"

The dpdkstat plugin collects information about DPDK interfaces using the extended NIC stats API in DPDK. She's thorough ๐Ÿ’…

Synopsis:

 <Plugin "dpdkstat">
   <EAL>
     Coremask "0x4"
     MemoryChannels "4"
     FilePrefix "rte"
     SocketMemory "1024"
     LogLevel "7"
     RteDriverLibPath "/usr/lib/dpdk-pmd"
   </EAL>
   SharedMemObj "dpdk_collectd_stats_0"
   EnabledPortMask 0xffff
   PortName "interface1"
   PortName "interface2"
 </Plugin>

Options:

The EAL block ๐Ÿงš

  • Coremask Mask โ€” a string containing a hexadecimal bit mask of the cores to run on. Fair warning bestie: core numbering can change between platforms, so figure that out beforehand, no cap.
  • Memorychannels Channels โ€” a string containing a number of memory channels per processor socket.
  • FilePrefix File โ€” the prefix text used for hugepage filenames. The filename is set to /var/run/.<prefix>_config where prefix is what you pass in.
  • SocketMemory MB โ€” a string containing the amount of Memory to allocate from hugepages on specific sockets, in MB. Optional value โœจ
  • LogLevel LogLevel_number โ€” a string containing the log level number. Optional. If you don't include it, the default value "7" - (INFO) is used. Set it to "8" - (DEBUG) to enable debug traces ๐Ÿ”
  • RteDriverLibPath Path โ€” a string containing the path to a shared pmd driver lib, or the path to a directory where shared pmd driver libs live. Optional. This one lets you load shared pmd driver libs from the defined path. E.g.: "/usr/lib/dpdk-pmd/librte_pmd_i40e.so" or "/usr/lib/dpdk-pmd"
  • SharedMemObj Mask โ€” a string containing the name of the shared memory object that should be used to share stats from the DPDK secondary process over to the collectd dpdkstat plugin. Defaults to dpdk_collectd_stats if you don't configure anything else.
  • EnabledPortMask Mask โ€” a hexadecimal bit mask of the DPDK ports that should be enabled. 0x0 = all ports disabled, all Fs = all ports enabled. Optional โ€” the default is all ports enabled.
  • PortName Name โ€” a string containing an optional name for the enabled DPDK ports. Each PortName option holds only one port name; add as many PortName options as you like ๐Ÿซถ Default naming convention kicks in if PortName is blank, and if there are fewer PortName options than enabled ports, the default naming convention handles the rest.

๐ŸŒˆ Plugin "dpdk_telemetry"

The dpdk_telemetry plugin collects DPDK ethernet device metrics via the dpdk_telemetry library. It's giving main character energy ๐ŸŽ€

Here's the whole situation: the plugin grabs metrics from a DPDK packet forwarding application by sending a JSON formatted message over a UNIX domain socket. The DPDK telemetry component replies with a JSON formatted answer delivering the metrics you asked for. Then the plugin parses that JSON data and publishes the metric values to collectd for further use. Literally the full pipeline โœจ

Synopsis:

  <Plugin dpdk_telemetry>
    ClientSocketPath "/var/run/.client"
    DpdkSocketPath "/var/run/dpdk/rte/telemetry"
  </Plugin>

Options:

  • ClientSocketPath Client_Path โ€” the UNIX domain client socket at Client_Path used to receive messages from the DPDK telemetry library. Defaults to "/var/run/.client".
  • DpdkSocketPath Dpdk_Path โ€” the UNIX domain DPDK telemetry socket to connect to at Dpdk_Path to send messages. Defaults to "/var/run/dpdk/rte/telemetry".

๐Ÿ’Œ Plugin "email"

  • SocketFile Path โ€” sets the socket-file that's going to be created.
  • SocketGroup Group โ€” if running as root, this changes the group of the UNIX-socket after it's been created. Defaults to collectd.
  • SocketPerms Permissions โ€” changes the file permissions of the UNIX-socket after it's been created. Permissions have to be given as a numeric, octal value, exactly like you'd pass to chmod(1). Defaults to 0770.
  • MaxConns Number โ€” sets the maximum number of connections that can be handled in parallel. And because this many threads get started immediately, setting this super high will waste your precious resources ๐Ÿ˜ญ Defaults to 5 and gets forced to be at most 16384 to prevent typos and dumb mistakes. We love a plugin with boundaries ๐Ÿ’…

๐Ÿงช Plugin "epics"

The epics plugin collects data from the EPICS (Experimental Physics and Industrial Control System) message bus. Science besties rise ๐Ÿ™Œ

Real talk: on top of the plugin configuration, the host OS has to be configured to be part of the message bus โ€” the caRepeater daemon has to be up and running, and the appropriate EPICS environment variables have to be set, if required. Consult the EPICS documentation for the details, okurrr ๐Ÿ’œ

Synopsis:

 <Plugin epics>
   <Variable "Cavity1:Temp">
     Type "gauge"
   </Variable>
   Label "Cavity1:Name"
 </Plugin>

Options:

  • <Variable PVName> โ€” the plugin config is made of multiple <Variable> blocks, one per monitored EPICS Process Variable (PV). These variables are constantly monitored, and their values are latched once per global Interval โฑ๏ธ
  • Type Type โ€” this is a mandatory option that specifies the internal collectd type for this particular PV. The PV data type announced by EPICS and the Type you specify here must be in a reasonable agreement โ€” fr fr, don't mix and match.
  • Label PVName โ€” bonus round: a string-typed PV can also be monitored as a Label โœจ For instance, this can potentially be useful for tracking an experiment ID. Note that in addition to configuring Label, the appropriate label-aware collectd writing plugin has to be enabled. For instance, the write_prometheus plugin, the write_influxdb_udp plugin, or the write_http plugin with InfluxDB Format ๐Ÿ‘‘

๐Ÿฆ‹ Plugin "ethstat"

The ethstat plugin collects information about network interface cards (NICs) by talking directly to the underlying kernel driver using ioctl(2). No middleman, no small talk, just vibes ๐Ÿซถ

Synopsis:

 <Plugin "ethstat">
   Interface "eth0"
   Map "rx_csum_offload_errors" "if_rx_errors" "checksum_offload"
   Map "multicast" "if_multicast"
 </Plugin>

Options:

  • Interface Name โ€” collect statistical information about interface Name.
  • Map Name Type [TypeInstance] โ€” by default the plugin submits values as type "derive" with the type instance set to Name, which is the name of the metric as reported by the driver. But if a matching Map option exists, then the given Type and, optionally, TypeInstance get used instead ๐Ÿ’…
  • MappedOnly true|false โ€” when set to true, only metrics that can be mapped to a type get collected, and every other metric is ignored. Defaults to false.

๐ŸŽ€ Plugin "exec"

Ok bestie, please please please make sure to read collectd-exec(5) before you use this plugin. It's got the valuable info on when the executable actually gets executed and what output is expected from it. Skipping it is not the move ๐Ÿ™ˆ
Exec User[:[Group]] Executable [<arg> [<arg> ...]]
NotificationExec User[:[Group]] Executable [<arg> [<arg> ...]]

Execute the executable Executable as user User. If the user name is followed by a colon and a group name, the effective group is set to that group. The real group and the saved-set group both get set to the default group of that user. And if no group is given, the effective group ID will be the same as the real group ID. That's the whole rule, periodt ๐ŸŒŸ

๐Ÿ’€ ok real talk, bestie: switching the user and/or group means the daemon needs superuser privileges. Periodt. If the daemon is chugging along as an unprivileged user, you literally have to put that same user/group right here in the config. And if it's running with superuser privileges, then you must supply a non-root user here. No vibes-based guessing, ok?

After the executable you can drop optional arguments, and those get passed straight to the program. But heads up ๐Ÿ‘€ โ€” because of how the config parser does its thing, numbers and boolean values may get changed on you. So if you absolutely need something passed as-is, wrap it in quotes. Quote it. Slay. ๐ŸŽ€

Also, Exec and NotificationExec change the semantics of the programs they execute โ€” meaning the data passed to them and the response expected back from them. It's all documented in great detail in collectd-exec(5). Go read it, iykyk.


๐Ÿฆ‹ Plugin "fhcount"

The "fhcount" plugin hands you stats about used, unused, and total file handles on Linux. That's it, that's the whole vibe.

The fhcount plugin supports these configuration options:

  • ValuesAbsolute true|false โ€” Turns reporting of file handle usage in absolute numbers on or off, like, "file handles used" style. Defaults to true. โœ…
  • ValuesPercentage false|true โ€” Turns reporting of file handle usage in percentages on or off, e.g. percent of file handles used. Defaults to false.

๐Ÿ“ Plugin "filecount"

The "filecount" plugin counts how many files live in a certain directory (and its subdirectories) plus their combined size. The config is genuinely so straightforward:

<Plugin "filecount">
  <Directory "/var/qmail/queue/mess">
    Instance "qmail-message"
  </Directory>
  <Directory "/var/qmail/queue/todo">
    Instance "qmail-todo"
  </Directory>
  <Directory "/var/lib/php5">
    Instance "php5-sessions"
    Name "sess_*"
  </Directory>
</Plugin>

The example above counts the number of files in QMail's queue directories and the number of PHP5 sessions. FYI ๐Ÿ’ก: the "todo" queue holds the messages QMail hasn't looked at yet, and the "message" queue holds the messages that got classified into "local" and "remote".

So, as you can see, the config is made of one or more "Directory" blocks, each one specifying a directory to count files in. Inside those blocks, these options are recognized:

  • Plugin Plugin โ€” Use Plugin as the plugin name when submitting values. Defaults to filecount.
  • Instance Instance โ€” Sets the plugin instance to Instance. If you don't give it, the instance gets set to the directory name with all slashes replaced by underscores and all leading underscores removed. An empty value is allowed. ๐Ÿซถ
  • Name Pattern โ€” Only counts files matching Pattern, where Pattern is a shell-like wildcard as understood by fnmatch(3). Only the filename gets checked against the pattern, not the entire path. And in case this makes it easier for you: this option was named after the -name parameter of find(1). iykyk.
  • MTime Age โ€” Counts only files of a specific age. If Age is greater than zero, only files that haven't been touched in the last Age seconds get counted. If Age is a negative number, it's inversed ๐Ÿ”„ โ€” so if -60 is specified, only files modified in the last minute get counted.

The number can also be followed by a "multiplier" to easily specify a larger timespan. When you use this notation, the argument must be quoted, i.e. must be passed as a string. So that -60 could also be written as "-1m" (one minute). Valid multipliers are "s" (second), "m" (minute), "h" (hour), "d" (day), "w" (week), and "y" (year). There is no "month" multiplier. You can also use fractional numbers โ€” e.g. "0.5d" is identical to "12h". Math slay โœจ.

  • Size Size โ€” Counts only files of a specific size. When Size is a positive number, only files at least this big get counted. If Size is negative, it's inversed, i.e. only files smaller than the absolute value of Size get counted.

Same as the MTime option, a "multiplier" may be added โ€” see above for the full description. Valid multipliers here are "b" (byte), "k" (kilobyte), "m" (megabyte), "g" (gigabyte), "t" (terabyte), and "p" (petabyte). โš ๏ธ Please note that there are 1000 bytes in a kilobyte, not 1024.

  • Recursive true|false โ€” Controls whether or not to recurse into subdirectories. Enabled by default.
  • IncludeHidden true|false โ€” Controls whether or not to include "hidden" files and directories in the count. "Hidden" files and directories are those whose name begins with a dot. Defaults to false, i.e. by default hidden files and directories are ignored. ๐Ÿ™ˆ
  • RegularOnly true|false โ€” Controls whether or not to include only regular files in the count. Defaults to true, i.e. by default non-regular files are ignored.
  • FilesSizeType Type โ€” Sets the type used to dispatch files' combined size. An empty value ("") disables reporting. Defaults to bytes.
  • FilesCountType Type โ€” Sets the type used to dispatch the number of files. An empty value ("") disables reporting. Defaults to files.
  • TypeInstance Instance โ€” Sets the type instance used to dispatch values. Defaults to an empty string (no plugin instance).

๐ŸŒˆ Plugin "GenericJMX"

The GenericJMX plugin is written in Java, so it's documented in collectd-java(5). Not here, bestie โ€” different page, different vibe. ๐Ÿงš


โญ Plugin "gmond"

The gmond plugin receives the multicast traffic sent by gmond, the statistics collection daemon of Ganglia. Mappings for the standard "metrics" are built-in, and custom mappings may be added via Metric blocks โ€” see below ๐Ÿ‘‡.

Synopsis:

<Plugin "gmond">
  MCReceiveFrom "239.2.11.71" "8649"
  <Metric "swap_total">
    Type "swap"
    TypeInstance "total"
    DataSource "value"
  </Metric>
  <Metric "swap_free">
    Type "swap"
    TypeInstance "free"
    DataSource "value"
  </Metric>
</Plugin>

The following metrics are built-in:

  • load_one, load_five, load_fifteen
  • cpu_user, cpu_system, cpu_idle, cpu_nice, cpu_wio
  • mem_free, mem_shared, mem_buffers, mem_cached, mem_total
  • bytes_in, bytes_out
  • pkts_in, pkts_out

Available configuration options:

  • MCReceiveFrom MCGroup [Port] โ€” Sets the multicast group and UDP port to which to subscribe.

Default: 239.2.11.71 / 8649

  • <Metric Name> โ€” These blocks add a new metric conversion to the internal table. Name, the string argument to the Metric block, is the metric name as used by Ganglia.
    • Type Type โ€” Type to map this metric to. Required.
    • TypeInstance Instance โ€” Type-instance to use. Optional.
    • DataSource Name โ€” Data source to map this metric to. If the configured type has exactly one data source, this is optional. Otherwise the option is required.

๐Ÿ›ฐ๏ธ Plugin "gps"

The "gps plugin" connects to gpsd on the host machine. The host, port, timeout and pause are all configurable. Obsessed with the flexibility tbh.

This is useful if you run an NTP server using a GPS for source and you want to monitor it. ๐Ÿ‘‘

โš ๏ธ Mind your GPS must send $--GSA for having the data reported! No GSA, no data, no cap.

The following elements are collected:

  • satellites โ€” Number of satellites used for fix (type instance "used") and in view (type instance "visible"). 0 means no GPS satellites are visible. ๐Ÿ˜ญ
  • dilution_of_precision โ€” Vertical and horizontal dilution (type instance "horizontal" or "vertical"). It should be between 0 and 3. Look at the documentation of your GPS to know more.

Synopsis:

LoadPlugin gps
<Plugin "gps">
  # Connect to localhost on gpsd regular port:
  Host "127.0.0.1"
  Port "2947"
  # 15 ms timeout
  Timeout 0.015
  # PauseConnect of 5 sec. between connection attempts.
  PauseConnect 5
</Plugin>

Available configuration options:

  • Host Host โ€” The host on which the gpsd daemon runs. Defaults to localhost.
  • Port Port โ€” Port to connect to gpsd on the host machine. Defaults to 2947.
  • Timeout Seconds โ€” Timeout in seconds (default 0.015 sec).

The GPS data stream gets fetched by the plugin from the daemon. It waits for data to be available, and if none arrives it times out and loops for another reading. Mind putting a low value โ€” gpsd expects the value in the micro-seconds area (recommended is 500 us) since the waiting function is blocking. The value must be between 500 us and 5 sec.; if it's outside that range, the default value is applied.

This only applies from gpsd release-2.95. ๐Ÿ“Œ

  • PauseConnect Seconds โ€” Pause to apply between attempts of connection to gpsd, in seconds (default 5 sec).

๐ŸŽฎ Plugin "gpu_nvidia"

Efficiently collects various statistics from the system's NVIDIA GPUs using the NVML library. Currently collected: fan speed, core temperature, percent load, percent memory used, compute and memory frequencies, and power consumption. It's giving main character energy for your graphics card, fr fr. ๐Ÿ”ฅ

  • GPUIndex โ€” If one or more of these options is specified, only GPUs at that index (as determined by nvidia-utils through nvidia-smi) have statistics collected. If no instance of this option is specified, all GPUs are monitored.
  • IgnoreSelected โ€” If set to true, all detected GPUs except the ones at indices specified by GPUIndex entries are collected. For greater clarity: setting IgnoreSelected without any GPUIndex directives will result in no statistics being collected. Yikes, careful. ๐Ÿ’€
  • InstanceByGPUIndex โ€” If set to false, the GPU ID will not be part of the plugin instance. The default is 'GPU ID'-'GPU name'.
  • InstanceByGPUName โ€” If set to false, the GPU name will not be part of the plugin instance. The default is 'GPU ID'-'GPU name'.

๐Ÿ’… Plugin "grpc"

The grpc plugin provides an RPC interface to submit values to or query values from collectd, based on the open source gRPC framework. It exposes an end-point for dispatching values to the daemon. Very network, very demure. โœจ

The gRPC homepage can be found at https://grpc.io/.

  • Server Host Port โ€” The Server statement sets the address of a server to which to send metrics via the "DispatchValues" function.

The argument Host may be a hostname, an IPv4 address, or an IPv6 address. ๐ŸŒˆ

Optionally, Server may be specified as a configuration block which supports the following options:

  • EnableSSL false|true โ€” Whether to require SSL for outgoing connections. Default: false.
  • SSLCACertificateFile Filename
  • SSLCertificateFile Filename
  • SSLCertificateKeyFile Filename โ€” Filenames specifying SSL certificate and key material to be used with SSL connections. ๐Ÿ”

๐Ÿ’… Listen Host Port

Okay bestie, the Listen statement is literally the one that tells the daemon which network address to bind to. And the way it works? If you give it multiple Listen statements, it'll bind to all of them โ€” no drama, no picking favorites. And if you specify none at all, it just defaults to 0.0.0.0:50051 like it's fine with whatever. ๐Ÿซถ

The argument Host can be a hostname, an IPv4 address, or an IPv6 address. Iconic, flexible, we love that for us.

Optionally (iykyk), Listen can also be written as a configuration block that supports these options:

  • EnableSSL true|false โ€” whether to enable SSL for incoming connections. Default: false.
  • SSLCACertificateFile Filename
  • SSLCertificateFile Filename
  • SSLCertificateKeyFile Filename

These three are the filenames specifying the SSL certificate and key material to be used with SSL connections. ๐Ÿ”

  • VerifyPeer true|false โ€” when it's enabled, a valid client certificate is required to connect to the server. When it's disabled, a client certificate is not requested and any unsolicited client certificate is accepted. Enabled by default. โœจ

๐ŸŽง Plugin "hddtemp"

So to get values from hddtemp, collectd connects to localhost (127.0.0.1), port 7634/tcp. You can change those defaults with the Host and Port options โ€” scroll down, they're right there. But heads up: "hddtemp" has to actually be running for this to work correctly. If it's not running, timeouts may appear, and those timeouts may interfere with other statistics. ๐Ÿ˜ญ Not the vibe.

The hddtemp homepage can be found at http://www.guzu.net/linux/hddtemp.php.

  • Host Hostname โ€” hostname to connect to. Defaults to 127.0.0.1.
  • Port Port โ€” TCP port to connect to. Defaults to 7634.

๐Ÿ˜ Plugin "hugepages"

To collect hugepages info, collectd reads the directories /sys/devices/system/node/*/hugepages and /sys/kernel/mm/hugepages. Reading either of those can be disabled with the options below (default is enabled โ€” slay).

  • ReportPerNodeHP true|false โ€” if enabled, information gets collected from the hugepage counters in /sys/devices/system/node/*/hugepages. This one's for checking the per-node hugepage statistics on a NUMA system. ๐Ÿงš
  • ReportRootHP true|false โ€” if enabled, information gets collected from the hugepage counters in /sys/kernel/mm/hugepages. Works on both NUMA and non-NUMA systems to check the overall hugepage statistics.
  • ValuesPages true|false โ€” whether to report hugepages metrics in number of pages. Defaults to true.
  • ValuesBytes false|true โ€” whether to report hugepages metrics in bytes. Defaults to false.
  • ValuesPercentage false|true โ€” whether to report hugepages metrics as percentage. Defaults to false.

๐ŸŒˆ Plugin "infiniband"

The "infiniband" plugin collects information about IB ports, periodt. Metrics are gathered from /sys/class/infiniband/DEVICE/port/PORTNUM/*, and the port names get formatted like DEVICE:PORTNUM (there's examples below, don't worry ๐Ÿ’–).

Options

  • Port Port โ€” select the port Port. Whether it gets collected or ignored depends on the IgnoreSelected setting (see below, bestie). And just like other plugins that use the daemon's ignorelist functionality, a string that starts and ends with a slash is interpreted as a regular expression. Examples:
Port "mlx5_0:1"
Port "/mthca0:[0-9]/"

See the "IGNORELISTS" section for details. ๐Ÿ‘‘

  • IgnoreSelected true|false โ€” sets whether the selected ports are ignored, or whether all the other ports are ignored. The behavior is (hopefully ๐Ÿ˜…) intuitive:
    • If no Port option is configured โ†’ all ports are collected.
    • If at least one Port option is given and IgnoreSelected is not given or set to false โ†’ only the matching ports are collected.
    • If IgnoreSelected is set to true โ†’ all ports are collected except the ones matched.

๐Ÿ”ฅ Plugin "intel_pmu"

The intel_pmu plugin collects performance counters data on Intel CPUs using the Linux perf interface. All events are reported on a per core basis. Main character energy fr fr.

Note: when you're using the intel_pmu plugin, the number of reading threads in collectd should be adjusted accordingly. The value should be more than a half of the configured cores โ€” so for 60 monitored cores, the recommendation is to set ReadThreads > 30. The optimal number of WriteThreads depends on the volume of metrics from read plugins, the interval, and the number of enabled write plugins. The above adjustments can help with performance scaling when you're monitoring a high number of events on many cores. ๐Ÿ’€ no cap, plan for it.

Synopsis

<Plugin intel_pmu>
  EventList "/var/cache/pmu/GenuineIntel-6-55-4-core.json"
  HardwareEvents "L2_RQSTS.CODE_RD_HIT,L2_RQSTS.CODE_RD_MISS" "L2_RQSTS.ALL_CODE_RD"
  Cores "0-3" "4,6" "[12-15]"
  HardwareEvents "L2_RQSTS.PF_MISS"
  Cores "[1,2]"
  HardwareEvents "INST_RETIRED.ANY" "CPU_CLK_UNHALTED.THREAD"
  Cores ""
  AggregateUncorePMUs true
</Plugin>

Options

EventList filename

JSON performance counter event list file name. To be able to monitor all Intel CPU specific events, the JSON event list file should be downloaded. Use the pmu-tools event_download.py script to download an event list for the current CPU. ๐ŸŽ€

HardwareEvents events

This field is a list of event names, or groups of comma separated event names. This option requires the EventList option to be configured. If "All" is provided, all events from EventList are going to be loaded. This option can be used multiple times in pair with the Cores option, as shown in the example above. โœจ

Cores cores groups

All events are reported on a per core basis. Monitoring of the events can be configured for a group of cores (aggregated statistics). This field defines groups of cores on which to monitor supported events. The field is represented as a list of strings with core group values. Each string represents a list of cores in a group. If a group is enclosed in square brackets, each core is added individually to a separate group (that is, the statistics are not aggregated). Allowed formats are:

0,1,2,3
0-10,20-18
1,3,5-8,10,0x10-12
[4-15,32-63]

If an empty string is provided as the value for this field, the default cores configuration is applied โ€” that is, a separate group is created for each core. This option can be used once for every HardwareEvents set. ๐Ÿฆ‹

AggregateUncorePMUs false|true

This option toggles the event value reporting from all the uncore PMUs, either dispatching as an aggregated value in a single metric, or dispatching as individual values. If AggregateUncorePMUs is set to 'true', uncore events from the various PMU subsystems across uncore are reported as a single metric, usually reported as single 'Core0' events. The total value is obtained by summing all counters across all the units (e.g. CHAs). If AggregateUncorePMUs is set to 'false', values from the individual PMU subsystems across uncore are dispatched separately with the PMU name/number added to collectd's plugin_instance. ๐Ÿ‘‘


๐Ÿ‘‘ Plugin "intel_rdt"

The intel_rdt plugin collects information provided by monitoring features of Intel Resource Director Technology (Intel(R) RDT) like Cache Monitoring Technology (CMT) and Memory Bandwidth Monitoring (MBM). These features provide information about utilization of shared resources. CMT monitors last level cache occupancy (LLC). MBM supports two types of events reporting local and remote memory bandwidth: local memory bandwidth (MBL) reports the bandwidth of accessing memory associated with the local socket, while remote memory bandwidth (MBR) reports the bandwidth of accessing the remote socket. Also, this technology allows monitoring instructions per clock (IPC). Monitor events are hardware dependent, bestie โ€” monitoring capabilities are detected on plugin initialization and only supported events are monitored. โœจ

Note: the intel_rdt plugin is using model-specific registers (MSRs), which require an additional capability to be enabled if collectd is run as a service. Please refer to the contrib/systemd.collectd.service file for more details. ๐Ÿ™ˆ

Synopsis

<Plugin "intel_rdt">
  MonIPCEnabled true
  MonLLCRefEnabled false
  MonLLCMissEnabled false
  Cores "0-2" "3,4,6" "8-10,15"
  Processes "sshd,qemu-system-x86" "bash"
</Plugin>

Options

Interval seconds

The interval within which to retrieve statistics on monitored events, in seconds. For milliseconds, divide the time by 1000 โ€” for example, if the desired interval is 50ms, set interval to 0.05. Due to limited capacity of counters, it is not recommended to set interval higher than 1 sec. โš ๏ธ

MonIPCEnabled true|false

Determines whether or not to enable IPC monitoring. If set to true (the default), IPC monitoring statistics will be collected by the intel_rdt plugin. ๐Ÿ’–

MonLLCRefEnabled true|false

Determines whether or not to enable LLC references monitoring. If set to false (the default), LLC references monitoring statistics will not be collected by the intel_rdt plugin.

MonLLCMissEnabled true|false

Determines whether or not to enable LLC misses monitoring. If set to false (the default), LLC misses monitoring statistics will not be collected by the intel_rdt plugin.

Cores cores groups

Monitoring of the events can be configured for a group of cores (aggregated statistics). This field defines groups of cores on which to monitor supported events. The field is represented as a list of strings with core group values. Each string represents a list of cores in a group. Allowed formats are:

0,1,2,3
0-10,20-18
1,3,5-8,10,0x10-12

If an empty string is provided as the value for this field, the default cores configuration is applied โ€” a separate group is created for each core. ๐Ÿงš

Processes process names groups

Monitoring of the events can be configured for a group of processes (aggregated statistics). This field defines groups of processes on which to monitor supported events. The field is represented as a list of strings with process names group values. Each string represents a list of processes in a group. Allowed format is:

sshd,bash,qemu
Note: by default, the global interval is used to retrieve statistics on monitored events. To configure a plugin specific interval, use the Interval option of the intel_rdt <LoadPlugin> block. For milliseconds, divide the time by 1000 โ€” for example, if the desired interval is 50ms, set interval to 0.05. Due to limited capacity of counters, it is not recommended to set interval higher than 1 sec. ๐Ÿซถ

๐ŸŒธ Plugin "interface"

  • Interface Interface โ€” select this interface. By default, these interfaces will then be collected. For a more detailed description, see IgnoreSelected below. ๐Ÿ‘‡

See the "IGNORELISTS" section for details. ๐Ÿ’…

  • IgnoreSelected true|false โ€” okay so if no config is given, the interface-plugin will collect data from all interfaces. That may not be practical, especially for loopback- and similar interfaces ๐Ÿ˜ญ. So you can use the Interface-option to pick the interfaces you actually care about. Sometimes, tho, itโ€™s easier/preferred to collect all interfaces except a few ones. This option lets you do exactly that: by setting IgnoreSelected to true, the effect of Interface is inverted โ€” all selected interfaces are ignored and all other interfaces are collected. ๐Ÿ’…

It is possible to use regular expressions to match interface names, if the name is surrounded by /.../ and collectd was compiled with support for regexps. This is useful if thereโ€™s a need to collect (or ignore) data for a group of interfaces that are similarly named, without needing to explicitly list all of them (especially useful if the list is dynamic). Example:

Interface "lo"
Interface "/^veth/"
Interface "/^tun[0-9]+/"
IgnoreSelected "true"

This will ignore the loopback interface, all interfaces with names starting with veth and all interfaces with names starting with tun followed by at least one digit. โœจ

  • ReportInactive true|false โ€” when set to false, only interfaces with non-zero traffic will be reported. Note that the check is done by looking into whether a package was sent at any time from boot and the corresponding counter is non-zero. So, if the interface has been sending data in the past since boot, but not during the reported time-interval, it will still be reported. ๐Ÿซถ

The default value is true and results in collection of the data from all interfaces that are selected by Interface and IgnoreSelected options.

  • UniqueName true|false โ€” interface name is not unique on Solaris (KSTAT); interface name is unique only within a module/instance. Following tuple is considered unique:
(ks_module, ks_instance, ks_name)

If this option is set to true, interface name contains above three fields separated by an underscore. For more info on KSTAT, visit http://docs.oracle.com/cd/E23824_01/html/821-1468/kstat-3kstat.html#REFMAN3Ekstat-3kstat

This option is only available on Solaris. ๐ŸŒž


๐Ÿ”ฅ Plugin "ipmi"

The ipmi plugin allows you to monitor server platform status using the Intelligent Platform Management Interface (IPMI). Local and remote interfaces are supported. ๐Ÿ‘‘

The plugin configuration consists of one or more Instance blocks which specify one ipmi connection each. Each block requires one unique string argument as the instance name. If instances are not configured, an instance with the default option values will be created.

For backwards compatibility, any option other than Instance block will trigger legacy config handling and it will be treated as an option within Instance block. This support will go away in the next major version of Collectd. โš ๏ธ

Within the Instance blocks, the following options are allowed:

  • Address Address โ€” hostname or IP to connect to. If not specified, plugin will try to connect to local management controller (BMC).
  • Username Username โ€” the username to use for the connection to remote BMC.
  • Password Password โ€” the password to use for the connection to remote BMC.
  • AuthType MD5|rmcp+ โ€” forces the authentication type to use for the connection to remote BMC. By default the most secure type is selected. ๐Ÿ”
  • Host Hostname โ€” sets the host field of dispatched values. Defaults to the global hostname setting.
  • Sensor Sensor โ€” selects sensors to collect or to ignore, depending on IgnoreSelected.

See IGNORELISTS for details.

  • IgnoreSelected true|false โ€” if no configuration is given, the ipmi plugin will collect data from all sensors found of type "temperature", "voltage", "current" and "fanspeed". This option enables you to do that: by setting IgnoreSelected to true the effect of Sensor is inverted โ€” all selected sensors are ignored and all other sensors are collected. ๐ŸŒ€
  • NotifySensorAdd true|false โ€” if a sensor appears after initialization time of a minute, a notification is sent.
  • NotifySensorRemove true|false โ€” if a sensor disappears, a notification is sent.
  • NotifySensorNotPresent true|false โ€” if you have, for example, dual power supply and one of them is (un)plugged, then a notification is sent. ๐Ÿ”Œ
  • NotifyIPMIConnectionState true|false โ€” if a IPMI connection state changes after initialization time of a minute, a notification is sent. Defaults to false.
  • SELEnabled true|false โ€” if system event log (SEL) is enabled, plugin will listen for sensor threshold and discrete events. When event is received the notification is sent. SEL event filtering can be configured using SELSensor and SELIgnoreSelected config options. Defaults to false.
  • SELSensor SELSensor โ€” selects sensors to get events from or to ignore, depending on SELIgnoreSelected.

See IGNORELISTS for details.

  • SELIgnoreSelected true|false โ€” if no configuration is given, the ipmi plugin will pass events from all sensors. This option enables you to do that: by setting SELIgnoreSelected to true the effect of SELSensor is inverted โ€” all events from selected sensors are ignored and all events from other sensors are passed.
  • SELClearEvent true|false โ€” if SEL clear event is enabled, plugin will delete event from SEL list after it is received and successfully handled. In this case other tools that are subscribed for SEL events will receive an empty event. Defaults to false.

๐Ÿฆ‹ Plugin "ipstats"

This plugin collects counts for ipv4 and ipv6 various types of packets passing through the system in total. At the moment itโ€™s only supported on FreeBSD. ๐ŸงŠ

The full list of options available to include in the counted statistics is:

IPv4 options, get into it ๐Ÿ’œ

  • ip4receive โ€” IPv4 total packets received.
  • ip4badsum โ€” IPv4 checksum bad.
  • ip4tooshort โ€” IPv4 packet too short.
  • ip4toosmall โ€” IPv4 not enough data.
  • ip4badhlen โ€” IPv4 ip header length < data size.
  • ip4badlen โ€” IPv4 ip length < ip header length.
  • ip4fragment โ€” IPv4 fragments received.
  • ip4fragdrop โ€” IPv4 frags dropped (dups, out of space).
  • ip4fragtimeout โ€” IPv4 fragments timed out.
  • ip4forward โ€” IPv4 packets forwarded.
  • ip4fastforward โ€” IPv4 packets fast forwarded.
  • ip4cantforward โ€” IPv4 packets rcvd for unreachable dest.
  • ip4redirectsent โ€” IPv4 packets forwarded on same net.
  • ip4noproto โ€” IPv4 unknown or unsupported protocol.
  • ip4deliver โ€” IPv4 datagrams delivered to upper level.
  • ip4transmit โ€” IPv4 total ip packets generated here.
  • ip4odrop โ€” IPv4 lost packets due to nobufs, etc.
  • ip4reassemble โ€” IPv4 total packets reassembled ok.
  • ip4fragmented โ€” IPv4 datagrams successfully fragmented.
  • ip4ofragment โ€” IPv4 output fragments created.
  • ip4cantfrag โ€” IPv4 donโ€™t fragment flag was set, etc.
  • ip4badoptions โ€” IPv4 error in option processing.
  • ip4noroute โ€” IPv4 packets discarded due to no route.
  • ip4badvers โ€” IPv4 ip version != 4.
  • ip4rawout โ€” IPv4 total raw ip packets generated.
  • ip4toolong โ€” IPv4 ip length > max ip packet size.
  • ip4notmember โ€” IPv4 multicasts for unregistered grps.
  • ip4nogif โ€” IPv4 no match gif found.
  • ip4badaddr โ€” IPv4 invalid address on header.

IPv6 options, same main character energy ๐Ÿ’™

  • ip6receive โ€” IPv6 total packets received.
  • ip6tooshort โ€” IPv6 packet too short.
  • ip6toosmall โ€” IPv6 not enough data.
  • ip6fragment โ€” IPv6 fragments received.
  • ip6fragdrop โ€” IPv6 frags dropped(dups, out of space).
  • ip6fragtimeout โ€” IPv6 fragments timed out.
  • ip6fragoverflow โ€” IPv6 fragments that exceeded limit.
  • ip6forward โ€” IPv6 packets forwarded.
  • ip6cantforward โ€” IPv6 packets rcvd for unreachable dest.
  • ip6redirectsent โ€” IPv6 packets forwarded on same net.
  • ip6deliver โ€” IPv6 datagrams delivered to upper level.
  • ip6transmit โ€” IPv6 total ip packets generated here.
  • ip6odrop โ€” IPv6 lost packets due to nobufs, etc.
  • ip6reassemble โ€” IPv6 total packets reassembled ok.
  • ip6fragmented โ€” IPv6 datagrams successfully fragmented.
  • ip6ofragment โ€” IPv6 output fragments created.
  • ip6cantfrag โ€” IPv6 donโ€™t fragment flag was set, etc.
  • ip6badoptions โ€” IPv6 error in option processing.
  • ip6noroute โ€” IPv6 packets discarded due to no route.
  • ip6badvers โ€” IPv6 ip6 version != 6.
  • ip6rawout โ€” IPv6 total raw ip packets generated.
  • ip6badscope โ€” IPv6 scope error.
  • ip6notmember โ€” IPv6 donโ€™t join this multicast group.
  • ip6nogif โ€” IPv6 no match gif found.
  • ip6toomanyhdr โ€” IPv6 discarded due to too many headers.

By default the following options are included in the counted packets:

  • ip4receive, ip4forward, ip4transmit
  • ip6receive, ip6forward, ip6transmit

For example, to also count IPv4 and IPv6 fragments received, include the following configuration:

<Plugin ipstats>
  ip4fragment true
  ip6fragment true
</Plugin>

๐ŸŒˆ Plugin "iptables"

  • Chain Table Chain [Comment|Number [Name]] โ€” select the iptables/ip6tables filter rules to count packets and bytes from.
  • Chain6 Table Chain [Comment|Number [Name]] โ€” select the iptables/ip6tables filter rules to count packets and bytes from. โœจ

If only Table and Chain are given, this plugin will collect the counters of all rules which have a comment-match. The comment is then used as type-instance.

If Comment or Number is given, only the rule with the matching comment or the nth rule will be collected. Again, the comment (or the number) will be used as the type-instance.

If Name is supplied, it will be used as the type-instance instead of the comment or the number.


๐ŸŒธ Plugin "irq"

  • Irq Irq โ€” select this irq. By default these irqs will then be collected. For a more detailed description see IgnoreSelected below.

See IGNORELISTS for details.

  • IgnoreSelected true|false โ€” if no configuration is given, the irq-plugin will collect data from all irqs. This may not be practical, especially if no interrupts happen. Thus, you can use the Irq-option to pick the interrupt youโ€™re interested in. Sometimes, however, itโ€™s easier/preferred to collect all interrupts except a few ones. This option enables you to do that: by setting IgnoreSelected to true the effect of Irq is inverted โ€” all selected interrupts are ignored and all other interrupts are collected. ๐Ÿ’…

Plugin java โ˜•๐Ÿ’œ

ok so the java plugin is the whole reason you can write collectd extensions in Java โ€” yes, Java, in this economy. This section is only about the syntax + semantics of the config options, fr. For the deep lore on the Java plugin itself, go read collectd-java(5), bestie.

Synopsis:

<Plugin "java">
  JVMArg "-verbose:jni"
  JVMArg "-Djava.class.path=/opt/collectd/lib/collectd/bindings/java"
  LoadPlugin "org.collectd.java.Foobar"
  <Plugin "org.collectd.java.Foobar">
    # To be parsed by the plugin
  </Plugin>
</Plugin>

Available configuration options:

  • JVMArg Argument โ€” the Argument gets passed straight to the Java Virtual Machine (JVM). It works exactly the way arguments to the java binary on the command line work โ€” run java --help for the deets.
โš ๏ธ bestie, listen: all of these options have to appear before (i.e. above) any other options! The moment another option shows up, the JVM starts and every later option just has to be ignored. order matters, no cap. ๐Ÿ˜ญ
  • LoadPlugin JavaClass โ€” instantiates a new JavaClass object. The constructor of that object very likely then registers one or more callback methods with the server. See collectd-java(5) for details. When the first one of these options is found, the virtual machine (JVM) is created โ€” which means every JVMArg option must come before (i.e. above) all the LoadPlugin options! ๐Ÿ’…
  • Plugin Name โ€” the entire block gets passed to the Java plugin as an org.collectd.api.OConfigItem object. For this to actually work, the plugin has to register a configuration callback first โ€” see "config callback" in collectd-java(5). Which means the Plugin block has to appear after the appropriate LoadPlugin block. Also note that Name depends on the (Java) plugin registering the callback and is completely independent from the JavaClass argument passed to LoadPlugin. ๐Ÿซถ

Plugin load ๐Ÿ“ˆ๐ŸŒธ

the load plugin collects the system load. These numbers give you a rough overview of how utilized a machine is. The system load is defined as the number of runnable tasks in the run-queue, and lots of operating systems hand it to you as a one, five, or fifteen minute average. iconic, honestly.

The following configuration options are available:

  • ReportRelative false|true โ€” when enabled, the system load divided by the number of available CPU cores is reported, for the 1 min, 5 min, and 15 min intervals. Defaults to false. โญ

Plugin logfile ๐Ÿ“๐ŸŽ€

  • LogLevel debug|info|notice|warning|err โ€” sets the log-level. So if, for example, it's set to notice, then all events with severity notice, warning, or err get written to the logfile.
๐ŸŒธ heads up: debug is only available if collectd has been compiled with debugging support!
  • File File โ€” sets the file to write log messages to. The special strings stdout and stderr can be used to write to the standard output and standard error channels, respectively. This, of course, only makes much sense when collectd is running in foreground- or non-daemon-mode. ๐Ÿ’…
  • Timestamp true|false โ€” prefixes all lines with the current time. Defaults to true. โฐ
  • PrintSeverity true|false โ€” when enabled, all lines get prefixed by the severity of the log message, for example "warning". Defaults to false.
โœจ Note: there's no need to notify the daemon after moving or removing the log file (e.g. when rotating the logs). The plugin reopens the file for each line it writes. self-sufficient queen behavior ๐Ÿ‘‘

Plugin logparser ๐Ÿ”๐Ÿงš

the logparser plugin is used to parse different kinds of logs. Set the right options and you can choose which strings to collect. The plugin searches the log file for messages that contain several matches (two or more). When all the mandatory matches are found, it sends a proper notification containing all the fetched values. it's giving detective energy ๐Ÿ”

Synopsis:

<Plugin logparser>
  <Logfile "/var/log/syslog">
    FirstFullRead false
    <Message "pcie_errors">
      DefaultType "pcie_error"
      DefaultSeverity "warning"
      <Match "aer error">
        Regex "AER:.*error received"
        SubmatchIdx -1
      </Match>
      <Match "incident time">
        Regex "(... .. ..:..:..) .* pcieport.*AER"
        SubmatchIdx 1
        IsMandatory false
      </Match>
      <Match "root port">
        Regex "pcieport (.*): AER:"
        SubmatchIdx 1
        IsMandatory true
      </Match>
      <Match "device">
        PluginInstance true
        Regex " ([0-9a-fA-F:\\.]*): PCIe Bus Error"
        SubmatchIdx 1
        IsMandatory false
      </Match>
      <Match "severity_mandatory">
        Regex "severity="
        SubMatchIdx -1
      </Match>
      <Match "nonfatal">
        Regex "severity=.*\\([nN]on-[fF]atal"
        TypeInstance "non_fatal"
        IsMandatory false
      </Match>
      <Match "fatal">
        Regex "severity=.*\\([fF]atal"
        Severity "failure"
        TypeInstance "fatal"
        IsMandatory false
      </Match>
      <Match "corrected">
        Regex "severity=Corrected"
        TypeInstance "correctable"
        IsMandatory false
      </Match>
      <Match "error type">
        Regex "type=(.*),"
        SubmatchIdx 1
        IsMandatory false
      </Match>
      <Match "id">
        Regex ", id=(.*)"
        SubmatchIdx 1
      </Match>
    </Message>
  </Logfile>
</Plugin>

Options:

  • Logfile File โ€” the Logfile block defines the file to search. It may contain one or more Message blocks, which are defined below. ๐Ÿ“‚
  • FirstFullRead true|false โ€” set to true if the file has to be parsed from the beginning on the first read. If false, only subsequent writes to the log file will be parsed. ๐Ÿ™ˆ
  • Message Name โ€” a Message block contains the matches to search the log file for. Each Message block builds a notification message using the matched elements, if its mandatory Match blocks are matched. ๐Ÿ’Œ
  • DefaultPluginInstance String โ€” sets the default value for the plugin instance of the notification.
  • DefaultType String โ€” sets the default value for the type of the notification.
  • DefaultTypeInstance String โ€” sets the default value for the type instance of the notification.
  • DefaultSeverity String โ€” sets the default severity. Must be set to "OK", "WARNING", or "FAILURE". Default value is "OK". โœ…
  • Match Name โ€” multiple Match blocks define regular expression patterns for extracting or excluding specific string patterns from parsing. The first and last Match items in the same Message set the boundaries of a multiline message and are mandatory. If these matches aren't found, then the whole message is discarded. ๐Ÿ’€
  • Regex Regex โ€” a regular expression with a pattern matching string. It may contain subexpressions, so the next option, SubmatchIdx, specifies which subexpression should be stored. ๐Ÿงต
  • SubmatchIdx Integer โ€” the index of the subexpression to be used for the notification. Multiple subexpressions are allowed. An index value of 0 takes the whole regular expression match as the result. An index value of -1 does not add the result to the message item. Can be omitted โ€” the default value is 0. ๐Ÿ”ข
  • Excluderegex Regex โ€” a regular expression for excluding lines containing specific matching strings. This gets processed before checking the Regex pattern. It's optional and can be omitted. ๐Ÿšซ
  • IsMandatory true|false โ€” flag indicating whether the Match item is mandatory for message validation. If set to true, the whole message is discarded if it's missing. For false, its presence is optional. Default value is set to true. ๐Ÿ‘‘
  • PluginInstance true|String โ€” if set to true, it sets the plugin instance to the string returned by the regex. It can be overridden by a user string. ๐Ÿ’…
  • Type true|String โ€” sets the notification type using rules like PluginInstance.
  • TypeInstance true|String โ€” sets the notification type instance using rules like above.
  • Severity String โ€” sets the notification severity to one of the options: "OK", "WARNING", "FAILURE". ๐Ÿ”ฅ

Plugin log_logstash ๐Ÿชต๐Ÿฆ‹

the log_logstash plugin behaves like the logfile plugin but formats messages as JSON events for logstash to parse and input. same vibes, new fit. โœจ

  • LogLevel debug|info|notice|warning|err โ€” sets the log-level. So if, for example, it's set to notice, then all events with severity notice, warning, or err get written to the logfile.
๐ŸŒธ reminder: debug is only available if collectd has been compiled with debugging support!
  • File File โ€” sets the file to write log messages to. The special strings stdout and stderr can be used to write to the standard output and standard error channels, respectively. This, of course, only makes much sense when collectd is running in foreground- or non-daemon-mode. ๐Ÿ’œ
โœจ Note: there's no need to notify the daemon after moving or removing the log file (e.g. when rotating the logs). The plugin reopens the file for each line it writes. again, self-reliant icon behavior ๐ŸŽ€

Plugin lpar ๐Ÿ–ฅ๏ธ๐Ÿ‘‘

the lpar plugin reads CPU statistics of Logical Partitions โ€” a virtualization technique for IBM POWER processors, iykyk. It takes into account CPU time stolen from or donated to a partition, in addition to the usual user, system, and I/O statistics. generous and petty at the same time, kinda iconic. ๐Ÿ’…

The following configuration options are available:

  • CpuPoolStats false|true โ€” when enabled, statistics about the processor pool are read too. The partition needs to have pool authority in order to be able to acquire this information. Defaults to false. ๐Ÿ”
  • ReportBySerial false|true โ€” if enabled, the serial of the physical machine the partition is currently running on is reported as the hostname, and the logical hostname of the machine is reported in the plugin instance. Otherwise, the logical hostname will be used (just like other plugins) and the plugin instance will be empty. Defaults to false. ๐ŸŒˆ

Plugin lua ๐Ÿง๐Ÿ’š

this plugin embeds a Lua interpreter into collectd and provides an interface to collectd's plugin system. See collectd-lua(5) for its documentation. short and sweet, we love a mysterious queen ๐Ÿซถ


Plugin mbmon ๐ŸŒก๏ธ๐Ÿ“

the mbmon plugin uses mbmon to retrieve temperature, voltage, etc. ๐ŸŒก๏ธ

By default collectd connects to localhost (127.0.0.1), port 411/tcp. The Host and Port options can be used to change these values, see below. "mbmon" has to be running for this to work correctly. If "mbmon" is not running, timeouts may appear, which may interfere with other statistics. ๐Ÿ˜ญ

"mbmon" must be run with the -r option ("print TAG and Value format"); Debian's /etc/init.d/mbmon script already does this โ€” other people will need to ensure that this is the case. no cap, go check. ๐Ÿ”

  • Host Hostname โ€” the hostname you're connecting to. Defaults to 127.0.0.1.
    • Port Port โ€” the TCP port you're connecting to. Defaults to 411.

๐Ÿงš Plugin "mdevents "

The mdevents plugin is the one that keeps an eye on your md devices (that's Linux software RAID, bestie) and scoops up their status changes. ๐Ÿ”ฅ

Real talk: RAID arrays exist so your system stays up and running when the usual hardware drama happens (looking at you, disk failure ๐Ÿ’€). Mdadm is the standard software RAID management tool on Linux, and it's honestly iconic โ€” it can monitor "metadata events" like disk failures, clean-to-dirty transitions, and etc. The kernel reports those actions to userspace through sysfs, and mdadm takes action accordingly with its monitoring capability. The mdmon polls /sys looking for changes in the entries array_state, sync_action, and per-disk state attribute files. Heads up though: this is meaningful for RAID1, 5 and 10 only.

So how does mdevents do it? It gathers RAID array events that get written to syslog by mdadm. Once it registers an event, it can send a collectd notification carrying the mdadm event's data. An event consists of the event type, the raid array name, and โ€” for particular events โ€” the name of the component device. ๐ŸŽ€

Example message:

"Jan 17 05:24:27 pc1 mdadm[188]: NewArray event detected on md device
/dev/md0"

The plugin also classifies the events it gathers, which means a notification gets a different severity โ€” {OKAY, WARNING, FAILURE} โ€” depending on the particular mdadm event. Slay.

For proper work, the mdevents plugin needs syslog and mdadm utilities to be present on the running system. Otherwise it will not be compiled as a part of collectd. ๐Ÿ˜ญ

Synopsis:

<Plugin mdevents>
  Event ""
  IgnoreEvent False
  Array ""
  IgnoreArray False
</Plugin>

Plugin configuration time! The mdevents plugin's configuration is mostly based on IgnoreList, which is a collectd's utility. You get to specify which particular events / RAID arrays are actually in your interest. ๐Ÿ’… Setting the IgnoreEvent/IgnoreArray booleans won't take effect if the Event/Array config lists are empty โ€” the plugin will accept the entry anyway. Noted.

Options:

  • Event "EventName" โ€” names of events to be monitored, separated by spaces. Possible events include:

Event Name | Class of event:

  • DeviceDisappeared โ€” FAILURE
  • RebuildStarted โ€” OKAY
  • RebuildNN โ€” OKAY
  • RebuildFinished โ€” WARNING
  • Fail โ€” FAILURE
  • FailSpare โ€” WARNING
  • SpareActive โ€” OKAY
  • NewArray โ€” OKAY
  • DegradedArray โ€” FAILURE
  • MoveSpare โ€” WARNING
  • SparesMissing โ€” WARNING
  • TestMessage โ€” OKAY

You should set the events that should be monitored as strings separated by spaces โ€” for example: Events "DeviceDisappeared Fail DegradedArray".

  • IgnoreEvent false|true โ€” if IgnoreEvent is set to true, the events specified in Events will be ignored. If it's false, only the specified events will be monitored. ๐Ÿ™ˆ
  • Array arrays โ€” you can specify an array or a group of arrays using a regexp. The plugin will accept only RAID array names that start with /dev/md. ๐Ÿ’–
  • IgnoreArray false|true โ€” if IgnoreArray is set to true, the arrays specified in Array will be ignored. If it's false, only the specified events will be monitored.

๐Ÿฉบ Plugin "mcelog"

The mcelog plugin uses mcelog to retrieve machine check exceptions. Fancy.

By default the plugin connects to /var/run/mcelog-client to check if the mcelog server is running. When the server is running, the plugin will tail the specified logfile to retrieve machine check exception information and send a notification with the details from the logfile. It also uses the mcelog client protocol to retrieve memory-related machine check exceptions. Quick note: for memory exceptions, notifications are only sent when there is a change in the number of corrected/uncorrected memory errors. ๐Ÿ‘‘

The Memory block

Note: these options cannot be used in conjunction with the logfile options โ€” they are mutually exclusive. Pick a lane, bestie. โœจ
  • McelogClientSocket Path โ€” connect to the mcelog client socket using the UNIX domain socket at Path. Defaults to /var/run/mcelog-client.
  • PersistentNotification true|false โ€” override the default configuration so notifications are only sent when there is a change in the number of corrected/uncorrected memory errors. When set to true, notifications will be sent for every read cycle. Default is false. Does not affect the stats being dispatched.
  • McelogLogfile Path โ€” the mcelog file to parse. Defaults to /var/log/mcelog. Note: this option cannot be used in conjunction with the memory block options โ€” they are mutually exclusive. ๐Ÿซถ

๐Ÿ’ฟ Plugin "md"

The md plugin collects information from Linux Software-RAID devices (md). We love a RAID moment.

All reported values are of the type md_disks. The reported type instances are active, failed (present but not operational), spare (hot stand-by) and missing (physically absent) disks.

  • Device Device โ€” select md devices based on device name. The device name is the basename of the device, i.e. the name of the block device without the leading /dev/. See IgnoreSelected for more details.

See /"IGNORELISTS" for details.

  • IgnoreSelected true|false โ€” invert device selection. If set to true, all md devices except those listed using Device are collected. If false (the default), only those listed are collected. If no configuration is given, the md plugin will collect data from all md devices. Simple as that. ๐Ÿฆ‹

๐Ÿ“ Plugin "memcachec"

The memcachec plugin connects to a memcached server, queries one or more given pages, and parses the returned data according to your specification. The matches used are the same as the matches used in the "curl" and "tail" plugins. iykyk.

To talk to the memcached server, this plugin uses the libmemcached library. Please note that there is another library with a very similar name, libmemcache (notice the missing d), which is not applicable. Don't get it twisted. ๐Ÿ’€

Synopsis of the configuration:

<Plugin "memcachec">
  <Page "plugin_instance">
    Server "localhost"
    Key "page_key"
    Plugin "plugin_name"
    <Match>
      Regex "(\\d+) bytes sent"
      DSType CounterAdd
      Type "ipt_octets"
      Instance "type_instance"
    </Match>
  </Page>
</Plugin>

The configuration options are:

  • <Page Name> โ€” each Page block defines one page to be queried from the memcached server. The block requires one string argument, which is used as plugin instance.
  • Server Address โ€” sets the server address to connect to when querying the page. Must be inside a Page block.
  • Key Key โ€” when connected to the memcached server, asks for the page Key.
  • Plugin Plugin โ€” use Plugin as the plugin name when submitting values. Defaults to "memcachec".
  • <Match> โ€” Match blocks define which strings to look for and how the matched substrings are interpreted. For a description of match blocks, please see "Plugin tail". ๐ŸŒธ

๐ŸงŠ Plugin "memcached"

The memcached plugin connects to a memcached server and queries statistics about cache utilization, memory and bandwidth used. Obsessed. <http://memcached.org/>

<Plugin "memcached">
  <Instance "name">
    #Host "memcache.example.com"
    Address "127.0.0.1"
    Port 11211
  </Instance>
</Plugin>

The plugin configuration consists of one or more Instance blocks, which specify one memcached connection each. Within the Instance blocks, the following options are allowed:

  • Host Hostname โ€” sets the host field of dispatched values. Defaults to the global hostname setting. For backwards compatibility, values are also dispatched with the global hostname when Host is set to 127.0.0.1 or localhost and Address is not set.
  • Address Address โ€” hostname or IP to connect to. For backwards compatibility, defaults to the value of Host or 127.0.0.1 if Host is unset.
  • Port Port โ€” TCP port to connect to. Defaults to 11211.
  • Socket Path โ€” connect to memcached using the UNIX domain socket at Path. If this setting is given, the Address and Port settings are ignored. Periodt. ๐Ÿ’…

๐Ÿ‘‘ Plugin "mic"

The mic plugin gathers CPU statistics, memory usage and temperatures from Intel's Many Integrated Core (MIC) systems. So specific, so iconic.

Synopsis:

<Plugin mic>
  ShowCPU true
  ShowCPUCores true
  ShowMemory true

  ShowTemperatures true
  Temperature vddg
  Temperature vddq
  IgnoreSelectedTemperature true

  ShowPower true
  Power total0
  Power total1
  IgnoreSelectedPower true
</Plugin>

The following options are valid inside the Plugin mic block:

  • ShowCPU true|false โ€” if enabled (the default) a sum of the CPU usage across all cores is reported.
  • ShowCPUCores true|false โ€” if enabled (the default) per-core CPU usage is reported.
  • ShowMemory true|false โ€” if enabled (the default) the physical memory usage of the MIC system is reported.
  • ShowTemperatures true|false โ€” if enabled (the default) various temperatures of the MIC system are reported.
  • Temperature Name โ€” this option controls which temperatures are being reported. Whether matching temperatures are being ignored or only matching temperatures are reported depends on the IgnoreSelectedTemperature setting below. By default all temperatures are reported.
  • IgnoreSelectedTemperature false|true โ€” controls the behavior of the Temperature setting above. If set to false (the default) only temperatures matching a Temperature option are reported or, if no Temperature option is specified, all temperatures are reported. If set to true, matching temperatures are ignored and all other temperatures are reported.

Known temperature names are:

  • die โ€” Die of the CPU
  • devmem โ€” Device Memory
  • fin โ€” Fan In
  • fout โ€” Fan Out
  • vccp โ€” Voltage ccp
  • vddg โ€” Voltage ddg
  • vddq โ€” Voltage ddq
  • ShowPower true|false โ€” if enabled (the default) various temperatures of the MIC system are reported.
  • Power Name โ€” this option controls which power readings are being reported. Whether matching power readings are being ignored or only matching power readings are reported depends on the IgnoreSelectedPower setting below. By default all power readings are reported.
  • IgnoreSelectedPower false|true โ€” this is the lil switch that flips the meaning of the Power setting right above it ๐Ÿ’…. Set it to false (the default) and only power readings that match a Power option get reported โ€” or, if you didn't specify any Power option at all, all the power readings get reported. Set it to true and it inverts: matching power readings get ignored, everything else gets reported. Slay, but backwards ๐Ÿฆ‹.

The power names you can use are, like, these:

  • total0 โ€” total power utilization averaged over Time Window 0 (uWatts).
  • total1 โ€” total power utilization averaged over Time Window 0 (uWatts).
  • inst โ€” instantaneous power (uWatts).
  • imax โ€” max instantaneous power (uWatts).
  • pcie โ€” PCI-E connector power (uWatts).
  • c2x3 โ€” 2x3 connector power (uWatts).
  • c2x4 โ€” 2x4 connector power (uWatts).
  • vccp โ€” core rail (uVolts).
  • vddg โ€” uncore rail (uVolts).
  • vddq โ€” memory subsystem rail (uVolts).

๐ŸŒธ Plugin "memory" ๐ŸŒธ

The memory plugin is the one that's like "bestie, here's how much RAM you're using" and it brings these config options with it:

  • ValuesAbsolute true|false โ€” turns reporting of physical memory usage in absolute numbers (aka bytes, the real numbers) on or off. Defaults to true. ๐Ÿ‘‘
  • ValuesPercentage false|true โ€” turns reporting of physical memory usage in percentages on or off, like "percent of physical memory used". Defaults to false.
Fr fr this percentage one is the moment: it's genuinely useful when you're deploying collectd in a heterogeneous environment where the sizes of physical memory vary ๐Ÿ˜ญ โ€” because a percentage still means something when the boxes are all different sizes.

๐Ÿฆ‹ Plugin "modbus" ๐Ÿฆ‹

Okay so the modbus plugin connects to a Modbus "slave" over Modbus/TCP or Modbus/RTU and reads register values โœจ. It supports reading single registers (unsigned 16 bit values), large integer values (unsigned 32 bit and 64 bit values) and floating point values (two registers interpreted as IEEE floats in big endian notation). It's giving "speaks fluent industrial protocol", iykyk.

Synopsis, bestie:

<Data "voltage-input-1">
  RegisterBase 0
  RegisterType float
  RegisterCmd ReadHolding
  Type voltage
  Instance "input-1"
  #Scale 1.0
  #Shift 0.0
</Data>

<Data "voltage-input-2">
  RegisterBase 2
  RegisterType float
  RegisterCmd ReadHolding
  Type voltage
  Instance "input-2"
</Data>

<Data "supply-temperature-1">
  RegisterBase 0
  RegisterType Int16
  RegisterCmd ReadHolding
  Type temperature
  Instance "temp-1"
</Data>

<Host "modbus.example.com">
  Address "192.168.0.42"
  Port    "502"
  Interval 60

  <Slave 1>
    Instance "power-supply"
    Collect  "voltage-input-1"
    Collect  "voltage-input-2"
  </Slave>
</Host>

<Host "localhost">
  Device "/dev/ttyUSB0"
  Baudrate 38400
  Interval 20

  <Slave 1>
    Instance "temperature"
    Collect  "supply-temperature-1"
  </Slave>
</Host>

โœจ <Data Name> blocks โœจ

Data blocks are literally the matchmaker between register numbers and the "types" collectd uses ๐ŸŽ€.

Inside <Data /> blocks, these options are allowed:

  • RegisterBase Number โ€” configures the base register to read from the device. If RegisterType has been set to Uint32 or Float, this register and the next one get read (the register number is increased by one). Two-for-one, iconic.
  • RegisterType Int16|Int32|Int64|Uint16|Uint32|UInt64|Float|Int32LE|Uint32LE|FloatLE|Double โ€” specifies what kind of data the device returns. Defaults to Uint16. If the type is Int32, Int32LE, Uint32, Uint32LE, Float or FloatLE, two 16 bit registers at RegisterBase and RegisterBase+1 will be read and the data is combined into one 32 value. For Int32, Uint32 and Float the most significant 16 bits are in the register at RegisterBase and the least significant 16 bits are in the register at RegisterBase+1. For Int32LE, Uint32LE, or Float32LE, the high and low order registers are swapped, with the most significant 16 bits in RegisterBase+1 and the least significant 16 bits in RegisterBase. If the type is Int64, UInt64 or Double, four 16 bit registers at RegisterBase, RegisterBase+1, RegisterBase+2 and RegisterBase+3 will be read and the data combined into one 64 bit value. Big brain behavior ngl ๐Ÿง โœจ
  • RegisterCmd ReadHolding|ReadInput โ€” specifies the register type to be collected from the device. Works only with libmodbus 2.9.2 or higher. Defaults to ReadHolding.
  • Type Type โ€” specifies the "type" (data set) to use when dispatching the value to collectd. Currently, only data sets with exactly one data source are supported. One source only, no polyamory ๐Ÿ’€.
  • Instance Instance โ€” sets the type instance to use when dispatching the value to Instance. If unset, an empty string (no type instance) is used.
  • Scale Value โ€” the values taken from the device are multiplied by Value. This field is optional and the default is 1.0.
  • Shift Value โ€” Value is added to values from the device after they've been multiplied by the Scale value. This field is optional and the default value is 0.0.

๐Ÿ’œ <Host Name> blocks ๐Ÿ’œ

Host blocks are how you say which hosts to connect to and what data to read from their "slaves" ๐Ÿซถ. The string argument Name is used as the hostname when dispatching the values to collectd.

Within <Host /> blocks, these options are allowed:

  • Address Hostname โ€” for Modbus/TCP, specifies the node name (the actual network address) used to connect to the host. This may be an IP address or a hostname. Please note that the libmodbus library in use only supports IPv4 at the moment. IPv4 only, no cap ๐Ÿฅฒ.
  • Port Service โ€” for Modbus/TCP, specifies the port used to connect to the host. The port can either be given as a number or as a service name. Please note that the Service argument must be a string, even if ports are given in their numerical form. Defaults to "502". Yes, string even when it's a number โ€” the quotes stay on, periodt ๐Ÿ’….
  • Device Devicenode โ€” for Modbus/RTU, specifies the path to the serial device being used.
  • Baudrate Baudrate โ€” for Modbus/RTU, specifies the baud rate of the serial device. Note, connections currently support only 8/N/1.
  • UARTType UARTType โ€” for Modbus/RTU, specifies the type of the serial device. RS232, RS422 and RS485 are supported. Defaults to RS232. Available only on Linux systems with libmodbus>=2.9.4. Linux girlies stay winning ๐ŸŒˆ.
  • Interval Interval โ€” sets the interval (in seconds) in which the values will be collected from this host. By default the global Interval setting will be used.

๐ŸŒŸ <Slave ID> ๐ŸŒŸ

Over each connection, multiple Modbus devices may be reached, so the slave ID is what tells everyone which device should be addressed ๐Ÿ‘‘. For each device you want to query, one Slave block must be given.

Within <Slave /> blocks, these options are allowed:

  • Instance Instance โ€” specifies the plugin instance to use when dispatching the values to collectd. By default "slave_ID" is used.
  • Collect DataName โ€” specifies which data to retrieve from the device. DataName must be the same string as the Name argument passed to a Data block. You can specify this option multiple times to collect more than one value from a slave. At least one Collect option is mandatory โ€” it's not optional, bestie, it's required ๐Ÿšจ.

๐ŸŽ€ Plugin "mqtt" ๐ŸŽ€

The MQTT plugin is the social butterfly: it can send metrics to MQTT (Publish blocks) and receive values from MQTT (Subscribe blocks). Both directions, main character energy ๐Ÿ’…โœจ.

Synopsis:

<Plugin mqtt>
  <Publish "name">
    Host "mqtt.example.com"
    Prefix "collectd"
  </Publish>
  <Subscribe "name">
    Host "mqtt.example.com"
    Topic "collectd/#"
  </Subscribe>
</Plugin>

The plugin's configuration lives in Publish and/or Subscribe blocks, configuring the sending and receiving direction respectively. The plugin will register a write callback named "mqtt/name", where name is the string argument given to the Publish block. Both types of blocks share many but not all of the following options โ€” if an option is valid in only one of the blocks, it will be mentioned explicitly. No sneaky surprises ๐Ÿฆ‹.

Options:

  • Host Hostname โ€” hostname of the MQTT broker to connect to.
  • Port Service โ€” port number or service name of the MQTT broker to connect to.
  • User UserName โ€” username used when authenticating to the MQTT broker.
  • Password Password โ€” password used when authenticating to the MQTT broker.
  • ClientId ClientId โ€” MQTT client ID to use. Defaults to the hostname used by collectd.
  • QoS [0-2] โ€” sets the Quality of Service, with the values 0, 1 and 2 meaning:
    • 0 โ€” At most once
    • 1 โ€” At least once
    • 2 โ€” Exactly once

In Publish blocks, this option determines the QoS flag set on outgoing messages and defaults to 0. In Subscribe blocks, it determines the maximum QoS setting the client is going to accept and defaults to 2. If the QoS flag on a message is larger than the maximum accepted QoS of a subscriber, the message's QoS will be downgraded. Downgraded, not deleted โ€” important distinction ๐Ÿ˜Œ.

  • Prefix Prefix (Publish only) โ€” this plugin will use one topic per value list which will look like a path. Prefix is used as the first path element and defaults to collectd. An example topic name would be:
collectd/cpu-0/cpu-user
  • Retain false|true (Publish only) โ€” controls whether the MQTT broker will retain (keep a copy of) the last message sent to each topic and deliver it to new subscribers. Defaults to false.
  • StoreRates true|false (Publish only) โ€” controls whether "DERIVE" and "COUNTER" metrics are converted to a rate before sending. Defaults to true.
  • CleanSession true|false (Subscribe only) โ€” controls whether the MQTT "cleans" the session up after the subscriber disconnects, or whether it maintains the subscriber's subscriptions and all messages that arrive while the subscriber is disconnected. Defaults to true. The "do you ghost me or do you keep my texts" setting, honestly ๐Ÿ™ˆ.
  • Topic TopicName (Subscribe only) โ€” configures the topic(s) to subscribe to. You can use the single level "+" and multi level "#" wildcards. Defaults to collectd/#, i.e. all topics beneath the collectd branch.
  • CACert file โ€” path to the PEM-encoded CA certificate file. Setting this option enables TLS communication with the MQTT broker, and as such, Port should be the TLS-enabled port of the MQTT broker. This option enables the use of TLS. Secure the bag ๐Ÿ”ฅ๐Ÿ”.
  • CertificateFile file โ€” the path to the PEM-encoded certificate file you wanna use as your client certificate when connecting to the MQTT broker. โš ๏ธ Only valid if CACert and CertificateKeyFile are also set, iykyk โ€” no solo acts here.
  • CertificateKeyFile file โ€” path to the unencrypted PEM-encoded key file that goes with CertificateFile (they're a matching set, bestie ๐Ÿ”). Only valid if CACert and CertificateFile are also set.
  • TLSProtocol protocol โ€” if you configure this, it sets the string protocol version (like "tlsv1" or "tlsv1.2", quoted, yes) to use for the TLS connection to the broker. If you don't set it, a default version is used, and that default literally depends on the version of OpenSSL the Mosquitto library was linked against. Only valid if CACert is set.
  • CipherSuite ciphersuite โ€” a string describing which ciphers are available for use. Peep ciphers(1) and the openssl ciphers utility for more info ๐Ÿ‘€. If unset, the default ciphers get used. Only valid if CACert is set.

๐Ÿฌ Plugin "mysql" ๐Ÿฌ

The "mysql plugin" needs mysqlclient installed, no way around it. It connects to one or more databases when it starts and keeps that connection up for as long as it possibly can. If the connection gets interrupted for whatever reason (rude), it will try to re-connect. And it will complain loudly if anything goes wrong โ€” main character energy, honestly ๐Ÿ’….

This plugin issues the MySQL SHOW STATUS / SHOW GLOBAL STATUS command and collects info about MySQL network traffic, executed statements, requests, the query cache and threads by evaluating the Bytes_{received,sent}, Com_*, Handler_*, Qcache_* and Threads_* return values. Wanna know what those values actually mean? Refer to the MySQL reference manual, 5.1.6. Server Status Variables, okay? ๐Ÿซถ

Optionally, primary and replica statistics may be collected in a MySQL replication setup. In that case, info about the synchronization state of the nodes is collected by evaluating the Position return value of the SHOW MASTER STATUS command and the Seconds_Behind_Master, Read_Master_Log_Pos and Exec_Master_Log_Pos return values of the SHOW SLAVE STATUS command. See the MySQL reference manual, 12.5.5.21 SHOW MASTER STATUS Syntax and 12.5.5.31 SHOW SLAVE STATUS Syntax for the details โœจ.

Synopsis:

<Plugin mysql>
  <Database foo>
    Host "hostname"
    User "username"
    Password "password"
    Port "3306"
    MasterStats true
    ConnectTimeout 10
    SSLKey "/path/to/key.pem"
    SSLCert "/path/to/cert.pem"
    SSLCA "/path/to/ca.pem"
    SSLCAPath "/path/to/cas/"
    SSLCipher "DHE-RSA-AES256-SHA"
  </Database>

  <Database bar>
    Alias "squeeze"
    Host "localhost"
    Socket "/var/run/mysql/mysqld.sock"
    SlaveStats true
    SlaveNotifications true
  </Database>

 <Database galera>
    Alias "galera"
    Host "localhost"
    Socket "/var/run/mysql/mysqld.sock"
    WsrepStats true
 </Database>
</Plugin>

A Database block defines one connection to a MySQL database. It accepts a single argument which specifies the name of the database. Literally none of the other options are required ๐Ÿ’–. MySQL will use default values as documented in the mysql_real_connect() and mysql_ssl_set() sections in the MySQL reference manual โ€” go look, it's all there.

  • Alias Alias โ€” the alias to use as the sender instead of the hostname when reporting. Super useful when your hostnames are cryptic nonsense, ngl.
  • Host Hostname โ€” hostname of the database server. Defaults to localhost.
  • User Username โ€” username to use when connecting to the database. The user does not have to be granted any privileges (which is a synonym for granting the USAGE privilege), unless you wanna collect replication statistics (see MasterStats and SlaveStats below). In that case, the user needs the REPLICATION CLIENT (or SUPER) privileges. Otherwise, literally any existing MySQL user will do ๐ŸŽ€.
  • Password Password โ€” the password needed to log into the database.
  • Database Database โ€” select this database. Defaults to no database, which is a perfectly reasonable option for what this plugin does. Like, it's fine. Don't stress.
  • Port Port โ€” TCP port to connect to. The port must be specified in its numeric form, but it must be passed as a string nonetheless. For example:
Port "3306"

If Host is set to localhost (the default), this setting has no effect. See the documentation for the mysql_real_connect function for details ๐Ÿ’….

  • Socket Socket โ€” specifies the path to the UNIX domain socket of the MySQL server. This option only has any effect if Host is set to localhost (the default). Otherwise, use the Port option above. See the documentation for the mysql_real_connect function for details.
  • InnodbStats true|false โ€” if enabled, metrics about the InnoDB storage engine are collected. Disabled by default.
  • MasterStats true|false, SlaveStats true|false โ€” enable the collection of primary / replica statistics in a replication setup ๐Ÿฆ‹. To be able to get access to these statistics, the user needs special privileges โ€” see the User documentation above. Defaults to false.
  • SlaveNotifications true|false โ€” if enabled, the plugin sends a notification if the replication slave I/O and / or SQL threads are not running. Defaults to false. (It's giving "hey bestie, something's wrong" energy ๐Ÿ””.)
  • WsrepStats true|false โ€” enable the collection of wsrep plugin statistics, used in Master-Master replication setups like in MySQL Galera/Percona XtraDB Cluster. The user needs only privileges to execute 'SHOW GLOBAL STATUS'. Defaults to false.
  • ConnectTimeout Seconds โ€” sets the connect timeout for the MySQL client.
  • SSLKey Path โ€” if provided, the X509 key in PEM format.
  • SSLCert Path โ€” if provided, the X509 cert in PEM format.
  • SSLCA Path โ€” if provided, the CA file in PEM format (check the OpenSSL docs, babe).
  • SSLCAPath Path โ€” if provided, the CA directory (check the OpenSSL docs).
  • SSLCipher String โ€” if provided, the SSL cipher to use.

๐Ÿ“ Plugin "netapp" ๐Ÿ“

The netapp plugin can collect a whole bunch of performance and capacity info from a NetApp filer using the NetApp API. Slay โœจ.

Okay so heads up: NetApp has a wide line of products and a lot of different software versions for each of these products. This plugin was developed for a NetApp FAS3040 running OnTap 7.2.3P8 and tested on FAS2050 7.3.1.1L1, FAS3140 7.2.5.1 and FAS3020 7.2.4P9. It should work for most combinations of model and software version, but it is very hard to test this. If you've used this plugin with other models and/or software versions, feel free to send us a mail and tell us about the results โ€” even if it's just a short "It works" ๐Ÿซถ no gatekeeping.

To collect these data, collectd will log in to the NetApp via HTTP(S) and HTTP basic authentication.

๐Ÿšจ Do not use a regular user for this! Create a special collectd user with just the minimum of capabilities needed. The user only needs the login-http-admin capability as well as a few more depending on which data will be collected. Required capabilities are documented below.

Synopsis

 <Plugin "netapp">
   <Host "netapp1.example.com">
    Protocol      "https"
    Address       "10.0.0.1"
    Port          443
    User          "username"
    Password      "aef4Aebe"
    Interval      30

    <WAFL>
      Interval 30
      GetNameCache   true
      GetDirCache    true
      GetBufferCache true
      GetInodeCache  true
    </WAFL>

    <Disks>
      Interval 30
      GetBusy true
    </Disks>

    <VolumePerf>
      Interval 30
      GetIO      "volume0"
      IgnoreSelectedIO      false
      GetOps     "volume0"
      IgnoreSelectedOps     false
      GetLatency "volume0"
      IgnoreSelectedLatency false
    </VolumePerf>

    <VolumeUsage>
      Interval 30
      GetCapacity "vol0"
      GetCapacity "vol1"
      IgnoreSelectedCapacity false
      GetSnapshot "vol1"
      GetSnapshot "vol3"
      IgnoreSelectedSnapshot false
    </VolumeUsage>

    <Quota>
      Interval 60
    </Quota>

    <Snapvault>
      Interval 30
    </Snapvault>

    <System>
      Interval 30
      GetCPULoad     true
      GetInterfaces  true
      GetDiskOps     true
      GetDiskIO      true
    </System>

    <VFiler vfilerA>
      Interval 60

      SnapVault true
      # ...
    </VFiler>
   </Host>
 </Plugin>

The netapp plugin accepts the following configuration options ๐Ÿ‘‡

  • Host Name โ€” a host block defines one NetApp filer. It will show up in collectd with the name you specify here, which doesn't have to be its real name nor its hostname (see the Address option below). Call it whatever, we don't judge ๐Ÿ’œ.
  • VFiler Name โ€” a VFiler block may only be used inside a host block. It accepts all the same options as the Host block (except for cascaded VFiler blocks) and will execute all NetApp API commands in the context of the specified VFiler(R). It will appear in collectd with the name you specify here, which doesn't have to be its real name either. The VFiler name may be specified using the VFilerName option โ€” if that's not specified, it defaults to the name you specify here.

The VFiler block inherits all connection-related settings from the surrounding Host block (which appear before the VFiler block) but they may be overwritten inside the VFiler block.

This feature is useful, for example, when using a VFiler as a SnapVault target (supported since OnTap 8.1). In that case, the SnapVault statistics are not available in the host filer (vfiler0) but only in the respective VFiler context. Iconic, honestly.

  • Protocol httpd|http โ€” the protocol collectd will use to query this host.

Optional

Type: string

Default: https

Valid options: http, https

  • Address Address โ€” the hostname or IP address of the host.

Optional

Type: string

Default: the "host" block's name.

  • Port Port โ€” the TCP port to connect to on the host.

Optional

Type: integer

Default: 80 for protocol "http", 443 for protocol "https"

  • User User, Password Password โ€” the username and password to use to log in to the NetApp.

Mandatory

Type: string

  • VFilerName Name โ€” the name of the VFiler in which context to execute API commands. If not specified, the name provided to the VFiler block will be used instead.

Optional

Type: string

Default: name of the VFiler block

๐Ÿงš Note: this option may only be used inside VFiler blocks.
  • Interval Interval โ€” TODO

These options decide what data gets collected, bestie. You can use them as a whole block and fine-tune all the little knobs inside it, use them as one single statement to just vibe with all the defaults, or leave them out entirely to collect absolutely nothing. Your call, no pressure. ๐Ÿ’…

The following options are valid inside all the blocks:

  • Interval Seconds โ€” Collect the respective statistics every Seconds seconds. Defaults to the host specific setting.

The System block

This one collects all kinds of performance data about the whole system, like, the entire thing. ๐Ÿ–ฅ๏ธ

Note: to get this data the collectd user needs the api-perf-object-get-instances capability.
  • Interval Seconds โ€” Collect disk statistics every Seconds seconds.
  • GetCPULoad true|false โ€” Set this to true and the current CPU usage gets read. This is the average usage across all CPUs in your NetApp, with zero info about individual CPUs. So it's giving "overall vibes" not "per-core lore".
Note: these are the same values that the NetApp CLI command sysstat returns in the CPU field.

Optional. Type: boolean. Default: true. Result: two value lists of type cpu, and type instances idle and system.

  • GetInterfaces true|false โ€” Set this to true and the current traffic of the network interfaces gets read. This is the total traffic over all interfaces of your NetApp, no individual interface tea. โ˜•
Note: this is the same values that the NetApp CLI command sysstat returns in the Net kB/s field.

Or is it? ๐Ÿ™ˆ

Optional. Type: boolean. Default: true. Result: one value list of type if_octects.

  • GetDiskIO true|false โ€” Set this to true and the current IO throughput gets read. This is the total IO of your NetApp, no info about individual disks, volumes or aggregates. Just the whole picture, main character energy. โœจ
Note: this is the same values that the NetApp CLI command sysstat returns in the Disk kB/s field.

Optional. Type: boolean. Default: true. Result: one value list of type disk_octets.

  • GetDiskOps true|false โ€” Set this to true and the current number of HTTP, NFS, CIFS, FCP, iSCSI, etc. operations gets read. That's the total number of operations on your NetApp, no individual volume or aggregate breakdown.
Note: these are the same values that the NetApp CLI command sysstat returns in the NFS, CIFS, HTTP, FCP and iSCSI fields.

Optional. Type: boolean. Default: true. Result: a variable number of value lists of type disk_ops_complex. Each type of operation results in one value list, with the name of the operation as the type instance.


The WAFL block

This collects all sorts of performance data about the WAFL file system. Right now that just means cache performance, which, honestly? Iconic. ๐ŸŽ€

Note: to get this data the collectd user needs the api-perf-object-get-instances capability.
Note: the interface to get these values is classified as Diagnostics by NetApp. Which means it is not guaranteed to be stable, not even between minor releases. So like... proceed with awareness, bestie. ๐Ÿ˜ญ
  • Interval Seconds โ€” Collect disk statistics every Seconds seconds.
  • GetNameCache true|false โ€” Optional. Type: boolean. Default: true. Result: one value list of type cache_ratio and type instance name_cache_hit.
  • GetDirCache true|false โ€” Optional. Type: boolean. Default: true. Result: one value list of type cache_ratio and type instance find_dir_hit.
  • GetInodeCache true|false โ€” Optional. Type: boolean. Default: true. Result: one value list of type cache_ratio and type instance inode_cache_hit.
  • GetBufferCache true|false โ€”
Note: this is the same value that the NetApp CLI command sysstat returns in the Cache hit field.

Optional. Type: boolean. Default: true. Result: one value list of type cache_ratio and type instance buf_hash_hit.


The Disks block

This one collects performance data about the individual disks in the NetApp. Per-disk gossip, basically. ๐Ÿ’ฟ

Note: to get this data the collectd user needs the api-perf-object-get-instances capability.
  • Interval Seconds โ€” Collect disk statistics every Seconds seconds.
  • GetBusy true|false โ€” Set this to true and the busy time of all disks gets calculated, and the value of the busiest disk in the system gets written. Only the winner gets reported. ๐Ÿ‘‘
Note: this is the same values that the NetApp CLI command sysstat returns in the Disk util field. Probably. ๐Ÿ’€

Optional. Type: boolean. Default: true. Result: one value list of type percent and type instance disk_busy.


The VolumePerf block

This collects all the performance data about the individual volumes. ๐Ÿ—‚๏ธ

You can pick which data to collect about which volume using the options below. They follow the standard ignorelist semantic, iykyk.

Note: to get this data the collectd user needs the api-perf-object-get-instances capability.
  • Interval Seconds โ€” Collect volume performance data every Seconds seconds.
  • GetIO Volume
  • GetOps Volume
  • GetLatency Volume

Select the given volume for IO, operations or latency statistics collection. The argument is the name of the volume without the /vol/ prefix.

Since the standard ignorelist functionality is used here, you can use a string starting and ending with a slash to specify regular expression matching. To match the volumes vol0, vol2 and vol7, you can use this regular expression:

GetIO "/^vol[027]$/"

If no regular expression is specified, an exact match is required. Both regular and exact matching are case sensitive.

If no volume was specified at all for either of the three options, that data gets collected for all available volumes.

See IGNORELISTS for details.

  • IgnoreSelectedIO true|false
  • IgnoreSelectedOps true|false
  • IgnoreSelectedLatency true|false

When set to true, the volumes selected for IO, operations or latency statistics collection get ignored, and the data is collected for all the other volumes instead. Flip the script. ๐Ÿ”„

When set to false, data is only collected for the specified volumes, and all other volumes get ignored.

If no volumes have been specified with the above Get* options, all volumes get collected regardless of the IgnoreSelected* option.

Defaults to false.


The VolumeUsage block

This collects capacity data about the individual volumes. How much room do they have, how much are they using, the whole storage situation. ๐Ÿ“ฆ

Note: to get this data the collectd user needs the api-volume-list-info capability.
  • Interval Seconds โ€” Collect volume usage statistics every Seconds seconds.
  • GetCapacity VolumeName โ€” The current capacity of the volume gets collected. This results in two to four value lists, depending on the configuration of the volume. All data sources are of type df_complex with the name of the volume as plugin_instance.

There will be type_instances used and free for the number of used and available bytes on the volume. If the volume has some space reserved for snapshots, a type_instance snap_reserved will be available. If the volume has SIS enabled, a type_instance sis_saved will be available โ€” that's the number of bytes saved by the SIS feature. Frugal queen. ๐Ÿ’ธ

Note: the current NetApp API has a bug that results in this value being reported as a 32 bit number. This plugin tries to guess the correct number, which works most of the time. If you see strange values here, bug NetApp support to fix this. ๐Ÿ™ƒ

Repeat this option to specify multiple volumes.

  • IgnoreSelectedCapacity true|false โ€” Specify whether to collect only the volumes selected by the GetCapacity option, or to ignore those volumes. IgnoreSelectedCapacity defaults to false. However, if no GetCapacity option is specified at all, all capacities get selected anyway. Make it make sense. ๐Ÿคทโ€โ™€๏ธ
  • GetSnapshot VolumeName โ€” Select volumes from which to collect snapshot information.

Usually, the space used for snapshots is included in the space reported as used. If snapshot information is collected as well, the space used for snapshots gets subtracted from the used space.

And to make things even more interesting, it's possible to reserve space to be used for snapshots. If the space required for snapshots is less than that reserved space, there's reserved free and reserved used space in addition to free and used. If the space required for snapshots exceeds the reserved space, that part allocated in the normal space gets subtracted from the used space again. Math is mathing. ๐Ÿงฎ

Repeat this option to specify multiple volumes.

  • IgnoreSelectedSnapshot โ€” Specify whether to collect only the volumes selected by the GetSnapshot option, or to ignore those volumes. IgnoreSelectedSnapshot defaults to false. However, if no GetSnapshot option is specified at all, all capacities get selected anyway.

The Quota block

This collects (tree) quota statistics โ€” used disk space and number of used files. It's a whole thing. This mechanism is useful for getting usage info for single qtrees. In case the quotas aren't used for any other purpose, an entry similar to the following in /etc/quotas would be sufficient:

/vol/volA/some_qtree tree - - - - -

After adding the entry, issue quota on -w volA on the NetApp filer. ๐Ÿซถ

  • Interval Seconds โ€” Collect SnapVault(R) statistics every Seconds seconds.

The SnapVault block

This collects statistics about the time and traffic of SnapVault(R) transfers. Snapshot backup lore. ๐ŸŒธ

  • Interval Seconds โ€” Collect SnapVault(R) statistics every Seconds seconds.

The netlink plugin uses a netlink socket to query the Linux kernel about statistics of various interface and routing aspects. Direct line to the kernel, no middleman. ๐Ÿ“ž

  • Interface Interface
  • VerboseInterface Interface

Instruct the plugin to collect interface statistics. This is basically the same as the statistics provided by the interface plugin (see above), but potentially much more detailed. More lore per packet, honestly. ๐Ÿ”ฅ

When configuring with Interface, only the basic statistics get collected, namely octets, packets, and errors. These statistics are collected by the interface plugin too, so using both at the same time is no benefit.

  • VerboseInterface โ€” collects literally everything except the basic counters, so if you're also running the "interface" plugin, nothing gets collected twice. We're talking dropped packets, received multicast packets, collisions and a whole zoo of differentiated RX and TX errors ๐Ÿฆ‹. Wanna peek at what you're signing up for? Run this:
ip -s -s link list
  • If Interface is All, every single interface gets selected. Slay.

You can also flex with regular expressions to match interface names โ€” just wrap the name in /.../ and make sure collectd was compiled with regexp support. Super useful when you've got a squad of similarly-named interfaces you want to collect (or ignore) without listing every one by hand (especially when the list is dynamic, iykyk). Examples:

Interface "/^eth/"
Interface "/^ens[1-4]$|^enp[0-3]$/"
VerboseInterface "/^eno[0-9]+/"

That matches every interface whose name starts with eth, every interface in the range ens1 โ€“ ens4 and enp0 โ€“ enp3, and for verbose metrics, every interface whose name starts with eno followed by at least one digit. Chef's kiss ๐Ÿ’…

  • QDisc Interface [QDisc]
  • Class Interface [Class]
  • Filter Interface [Filter] โ€” collect the octets and packets that pass a certain qdisc, class or filter.

QDiscs and classes are identified by their type and handle (or classid). Filters don't necessarily have a handle, so the parent's handle is what gets used instead. Ngl, the notation here is a little different from tc(1): collectd doesn't skip the major or minor number when it's zero, and it doesn't print special ids by their name. So a qdisc might show up as "pfifo_fast-1:0" even though the minor number of all qdiscs is zero and thus not displayed by tc(1). That's just how it is ๐Ÿซถ

If QDisc, Class, or Filter is given without the second argument โ€” i.e. without an identifier โ€” then all qdiscs, classes, or filters associated with that interface get collected.

Since a filter itself doesn't necessarily have a handle, the parent's handle is used. Lowkey this may lead to problems when more than one filter is attached to a qdisc or class. Not nice, but we genuinely don't know how this could be done any better. If you have an idea, please don't hesitate to tell us ๐Ÿ˜ญ

Just like with the Interface option, you can specify All as the interface, meaning all interfaces.

Here are some examples to help the above text make sense:

<Plugin netlink>
  VerboseInterface "All"
  QDisc "eth0" "pfifo_fast-1:0"
  QDisc "ppp0"
  Class "ppp0" "htb-1:10"
  Filter "ppp0" "u32-1:0"
</Plugin>

See IGNORELISTS for details.

  • IgnoreSelected โ€” behaves like every other similar plugin: if nothing is selected at all, everything gets collected. If some things are selected using the options above, only those statistics get collected. Set IgnoreSelected to true and the behavior flips โ€” the specified statistics will not be collected. Classic inversion โœจ
  • CollectVFStats true|false โ€” lets the plugin collect VF's statistics if there are Virtual Functions available for the interfaces specified in Interface or VerboseInterface. All available stats are collected no matter whether the parent interface is set by Interface or VerboseInterface. No cap ๐Ÿงš

๐Ÿ›ฐ๏ธ Plugin "network"

The Network plugin sends data to a remote instance of collectd, receives data from a remote instance, or does both at the same time. Data that's been received from the network is usually not transmitted again โ€” but that can be turned on, see the Forward option below.

The default IPv6 multicast group is "ff18::efc0:4a42". The default IPv4 multicast group is 239.192.74.66. The default UDP port is 25826. Write those down bestie ๐Ÿ“

Both Server and Listen can be used as a single option or as a block. When used as a block, the given options are valid for that socket only. The following example exports the metrics twice: once to an "internal" server (no encryption, no signing) and once to an external server (with cryptographic signature). It's giving main character energy:

<Plugin "network">
  # Export to an internal server
  # (demonstrates usage without additional options)
  Server "collectd.internal.tld"

  # Export to an external server
  # (demonstrates usage with signature options)
  <Server "collectd.external.tld">
    SecurityLevel "sign"
    Username "myhostname"
    Password "ohl0eQue"
  </Server>
</Plugin>
  • <Server Host [Port]> โ€” the Server statement/block sets the server to send datagrams to. It may occur multiple times so each datagram goes to multiple destinations. Multitasking queen ๐Ÿ‘‘

The argument Host may be a hostname, an IPv4 address or an IPv6 address. The optional second argument specifies a port number or a service name. If it's not given, the default, 25826, is used.

The following options are recognized within Server blocks:

  • SecurityLevel Encrypt|Sign|None โ€” sets the security you require for network communication. When it's set to Encrypt, data sent over the network gets encrypted using AES-256, and the integrity of encrypted packets is ensured using SHA-1. When set to Sign, transmitted data is signed using the HMAC-SHA-256 message authentication code. When set to None, data is sent with no security at all (bold choice).

This feature is only available if the network plugin was linked with libgcrypt.

  • Username Username โ€” sets the username to transmit. The server uses it to look up the password. See AuthFile below. All security levels except None require this setting.

This feature is only available if the network plugin was linked with libgcrypt.

  • Password Password โ€” sets a password (shared secret) for this socket. All security levels except None require this setting.

This feature is only available if the network plugin was linked with libgcrypt.

  • Interface Interface name โ€” set the outgoing interface for IP packets. This applies at least to IPv6 packets and, if possible, to IPv4. If this option isn't applicable, is undefined, or a non-existent interface name is specified, the default behavior is to let the kernel choose the appropriate interface. Heads up though: manually selecting an interface for unicast traffic is only necessary in rare cases ๐Ÿ™ˆ
  • BindAddress IP Address โ€” set the outgoing IP address for IP packets. You can use this instead of the Interface option to explicitly define the IP address used to send packets to the remote server.
  • ResolveInterval Seconds โ€” sets the interval at which to re-resolve the DNS for the Host. Handy for forcing a regular DNS lookup to support a high availability setup. If not specified, re-resolves are never attempted. Periodt.
  • <Listen Host [Port]> โ€” the Listen statement sets the interfaces to bind to. When multiple statements are found, the daemon binds to multiple interfaces.

The argument Host may be a hostname, an IPv4 address or an IPv6 address. If the argument is a multicast address, the daemon will join that multicast group. The optional second argument specifies a port number or a service name. If it's not given, the default, 25826, is used.

The following options are recognized within <Listen> blocks:

  • SecurityLevel Encrypt|Sign|None โ€” sets the security you require for network communication. When it's set to Encrypt, only encrypted data will be accepted, and the integrity of encrypted packets is ensured using SHA-1. When set to Sign, only signed and encrypted data is accepted. When set to None, all data will be accepted. If an AuthFile option was given (see below), encrypted data is decrypted if possible.

This feature is only available if the network plugin was linked with libgcrypt.

  • AuthFile Filename โ€” sets a file in which usernames are mapped to passwords. These passwords are used to verify signatures and to decrypt encrypted network packets. If SecurityLevel is set to None, this is optional. If given, signed data gets verified and encrypted packets get decrypted. Otherwise, signed data is accepted without checking the signature, and encrypted data can't be decrypted at all. For the other security levels this option is mandatory.

The file format is super simple: each line consists of a username followed by a colon and any number of spaces followed by the password. To demonstrate, an example file could look like this:

user0: foo
user1: bar

Each time a packet is received, the modification time of the file is checked using stat(2). If the file has been changed, the contents get re-read. While the file is being read, it's locked using fcntl(2). Safe and sound ๐Ÿ”’

  • Interface Interface name โ€” set the incoming interface for IP packets explicitly. This applies at least to IPv6 packets and, if possible, to IPv4. If this option isn't applicable, is undefined, or a non-existent interface name is specified, the default behavior is to let the kernel choose the appropriate interface. So incoming traffic only gets accepted if it arrives on the given interface. Vibe check passed โœ…
  • TimeToLive 1-255 โ€” set the time-to-live of sent packets. This applies to all packets: unicast and multicast, IPv4 and IPv6. The default is to not change this value, which means multicast packets will be sent with a TTL of 1 (one) on most operating systems.
  • MaxPacketSize 1024-65535 โ€” set the maximum size for datagrams received over the network. Packets larger than this get truncated. Defaults to 1452 bytes, which is the maximum payload size that can be transmitted in one Ethernet frame using IPv6 / UDP.

On the server side, this limit should be set to the largest value used on any client. Likewise, the value on the client must not be larger than the value on the server, or data will be lost. Don't do that to yourself ๐Ÿ˜ญ

Compatibility: versions prior to version 4.8 used a fixed sized buffer of 1024 bytes. Versions 4.8, 4.9 and 4.10 used a default value of 1024 bytes to avoid problems when sending data to an older server. History lesson over ๐ŸŽ€

  • Forward true|false โ€” okay so if you set this to true, the plugin writes packets it received over the network back out to the sending sockets โœจ. Only turn this on when your Listen and Server statements actually differ โ€” otherwise packets may get sent multiple times to the same multicast group ๐Ÿ™ˆ. It's not a huge problem though, because the plugin has duplicate detection, so the values won't loop ๐Ÿ’…. Just means extra network traffic than strictly needed.
  • ReportStats true|false โ€” the network plugin isn't just a receive-and-send girlie, she can also make stats about herself ๐Ÿ‘‘. Collectd data includes the number of received and sent octets and packets, the length of the receive queue, and the number of values handled. When set to true, the Network plugin makes all these stats available. Defaults to false.

๐Ÿงš Plugin "nfs"

The nfs plugin collects info about how the Network File System (NFS) is being used ๐Ÿฆ‹. It counts the number of procedure calls for each procedure, grouped by version and by whether the system is running as server or client.

It's possible to omit metrics for a specific NFS version by setting one or more of these options to false (they all default to true btw):

  • ReportV2 true|false
  • ReportV3 true|false
  • ReportV4 true|false

๐ŸŒธ Plugin "nginx"

This plugin collects the number of connections and requests handled by the "nginx daemon" (speak: engine X ๐Ÿ˜Œ), a HTTP and mail server/proxy. It queries the page provided by the ngx_http_stub_status_module module โ€” and heads up, that module isn't compiled by default ๐Ÿ’€. Please refer to <http://wiki.codemongers.com/NginxStubStatusModule> for the info on how to compile and configure nginx and this module.

The following options are accepted by the "nginx plugin":

  • URL http://host/nginx_status โ€” sets the URL of the ngx_http_stub_status_module output.
  • User Username โ€” optional user name needed for authentication.
  • Password Password โ€” optional password needed for authentication.
  • VerifyPeer true|false โ€” enable or disable peer SSL certificate verification. See <http://curl.haxx.se/docs/sslcerts.html> for the details. Enabled by default.
  • VerifyHost true|false โ€” enable or disable peer host name verification ๐Ÿ‘€. If enabled, the plugin checks whether the "Common Name" or a "Subject Alternate Name" field of the SSL certificate matches the host name provided by the URL option. If this identity check fails, the connection is aborted. Obviously this only works when connecting to a SSL enabled server. Enabled by default.
  • CACert File โ€” file that holds one or more SSL certificates. If you want to use HTTPS you'll possibly need this option ๐ŸŽ€. What CA certificates come bundled with libcurl and are checked by default depends on the distribution you use.
  • Timeout Milliseconds โ€” the Timeout option sets the overall timeout for HTTP requests to URL, in milliseconds. By default the configured Interval is used to set the timeout.
  • Socket Path โ€” the Socket option sets the UNIX domain socket to use, if NGINX listens on a UNIX domain socket instead ๐Ÿซถ. Note that you still need to provide the URL option.

๐ŸŽ€ Plugin "notify_desktop"

This plugin sends a desktop notification to a notification daemon, as defined in the Desktop Notification Specification โœจ. To actually display the notifications, notification-daemon is required and collectd has to be able to access the X server โ€” i.e. the DISPLAY and XAUTHORITY environment variables have to be set correctly โ€” plus the D-Bus message bus ๐Ÿ’œ.

The Desktop Notification Specification can be found at <http://www.galago-project.org/specs/notification/>.

  • OkayTimeout timeout
  • WarningTimeout timeout
  • FailureTimeout timeout

These set the timeout, in milliseconds, after which to expire the notification for OKAY, WARNING and FAILURE severities respectively โฐ. If zero has been specified, the displayed notification will not be closed at all โ€” the user has to do so herself ๐Ÿ’…. These options default to 5000. If a negative number has been specified, the default is used as well.

๐Ÿ’Œ Plugin "notify_email"

The notify_email plugin uses the ESMTP library to send notifications to a configured email address ๐Ÿ“ง.

libESMTP is available from <http://www.stafford.uklinux.net/libesmtp/>.

Available configuration options:

  • From Address โ€” email address from which the emails should appear to come from.

Default: "root@localhost"

  • Recipient Address โ€” configures the email address(es) to which the notifications should be mailed. May be repeated to send notifications to multiple addresses ๐Ÿ“ฌ.
At least one Recipient must be present for the plugin to work correctly. No cap.
  • SMTPServer Hostname โ€” hostname of the SMTP server to connect to.

Default: "localhost"

  • SMTPPort Port โ€” TCP port to connect to.

Default: 25

  • SMTPUser Username โ€” username for ASMTP authentication. Optional.
  • SMTPPassword Password โ€” password for ASMTP authentication. Optional.
  • Subject Subject โ€” subject-template to use when sending emails ๐ŸŽ€. There must be exactly two string-placeholders in the subject, given in the standard printf(3) syntax, i.e. %s. The first will be replaced with the severity, the second with the hostname.

Default: "Collectd notify: %s@%s"

๐Ÿฅ€ Plugin "notify_nagios"

The notify_nagios plugin writes notifications to Nagios' command file as a passive service check result ๐Ÿ“.

Available configuration options:

  • CommandFile Path โ€” sets the command file to write to. Defaults to /usr/local/nagios/var/rw/nagios.cmd.

โฑ๏ธ Plugin "ntpd"

The ntpd plugin collects per-peer ntp data such as time offset and time dispersion ๐Ÿ•ฐ๏ธ.

For talking to ntpd, it mimics what the ntpdc control program does on the wire โ€” using mode 7 specific requests. This mode is deprecated with newer ntpd releases (4.2.7p230 and later) ๐Ÿ’€. For the ntpd plugin to work correctly with them, the ntp daemon must be explicitly configured to enable mode 7 (which is disabled by default). Refer to the ntp.conf(5) manual page for details.

Available configuration options for the ntpd plugin:

  • Host Hostname โ€” hostname of the host running ntpd. Defaults to localhost.
  • Port Port โ€” UDP port to connect to. Defaults to 123.
  • ReverseLookups true|false โ€” sets whether or not to perform reverse lookups on peers. Since the name or IP-address may be used in a filename, it's recommended to disable reverse lookups ๐Ÿ™ˆ. The default is to do reverse lookups to preserve backwards compatibility, though.
  • IncludeUnitID true|false โ€” when a peer is a refclock, include the unit ID in the type instance. Defaults to false for backward compatibility.
If two refclock peers use the same driver and this is false, the plugin will try to write simultaneous measurements from both to the same type instance โ€” which results in error messages in the log and only one set of measurements making it through ๐Ÿ˜ญ.

๐Ÿ”‹ Plugin "nut"

  • UPS upsname@hostname[:port] โ€” add a UPS to collect data from. The format is identical to the one accepted by upsc(8).
  • ForceSSL true|false โ€” stops connections from falling back to unsecured if an SSL connection cannot be established ๐Ÿ”’. Defaults to false if undeclared.
  • VerifyPeer true|false โ€” if set to true, requires a CAPath be provided. Will use the CAPath to find certificates to use as Trusted Certificates to validate a upsd server certificate. If validation of the upsd server certificate fails, the connection will not be established ๐Ÿšซ. If ForceSSL is undeclared or set to false, setting VerifyPeer to true will override and set ForceSSL to true.
  • CAPath I/path/to/certs/folder โ€” if VerifyPeer is set to true, this is required. Otherwise it's ignored. The folder pointed at must contain certificate(s) named according to their hash ๐Ÿ’…. Ex: XXXXXXXX.Y where X is the hash value of a cert and Y is 0. If name collisions occur because two different certs have the same hash value, Y can be incremented to avoid conflict. To create a symbolic link to a certificate, the following command can be used from within the directory where the cert resides:
"ln -s some.crt ./$(openssl x509 -hash -noout -in some.crt).0"

Alternatively, the package openssl-perl provides a command c_rehash that will generate links like the one described above for ALL certs in a given folder ๐ŸŒˆ. Example usage: "c_rehash /path/to/certs/folder"

  • ConnectTimeout Milliseconds โ€” the ConnectTimeout option sets the connect timeout, in milliseconds. By default the configured Interval is used to set the timeout.

๐Ÿ›ฐ๏ธ Plugin "olsrd"

The olsrd plugin connects to the TCP port opened by the txtinfo plugin of the Optimized Link State Routing daemon and reads info about the current state of the meshed network ๐Ÿ•ธ๏ธ.

The following configuration options are understood:

  • Host Host โ€” connect to Host. Defaults to "localhost".
  • Port Port โ€” specifies the port to connect to. This must be a string, even if you give the port as a number rather than a service name ๐Ÿ‘€. Defaults to "2006".
  • CollectLinks No|Summary|Detail โ€” specifies what info to collect about links, i.e. direct connections of the daemon queried. If set to No, no info is collected. If set to Summary, the number of links and the average of all link quality (LQ) and neighbor link quality (NLQ) values is calculated. If set to Detail, LQ and NLQ are collected per link.

Defaults to Detail.

  • CollectRoutes No|Summary|Detail โ€” specifies what info to collect about routes of the daemon queried. If set to No, no info is collected. If set to Summary, the number of routes and the average metric and ETX is calculated. If set to Detail, metric and ETX are collected per route.

Defaults to Summary.

  • CollectTopology No|Summary|Detail โ€” specifies what info to collect about the global topology. If set to No, no info is collected. If set to Summary, the number of links in the entire topology and the average link quality (LQ) is calculated. If set to Detail, LQ and NLQ are collected for each link in the entire topology.

Defaults to Summary.

๐Ÿงช Plugin "onewire"

EXPERIMENTAL! See notes below ๐Ÿ˜ณ.

The onewire plugin uses the owcapi library from the owfs project <http://owfs.org/> to read sensors connected via the onewire bus ๐Ÿฆ‹.

It can be used in two possible modes โ€” standard or advanced โœจ.

Okay so in the standard mode only temperature sensors โ€” the ones with family code 10, 22 and 28 (e.g. DS1820, DS18S20, DS1920) โ€” can be read. If youโ€™ve got other sensors youโ€™d like included, please send a sort request to the mailing list. You can select sensors to be read or ignored depending on the IgnoreSelected option. When no list is provided, the whole bus is walked and all sensors are read. ๐Ÿฆ‹

Hubs (the DS2409 chips) are working, but read the note below about why this plugin is experimental. ๐Ÿ’…

In the advanced mode you can configure any sensor to be read (only numerical value) using a full OWFS path (e.g. /uncached/10.F10FCA000800/temperature). In this mode you have to list all the sensors. Neither default bus walk nor IgnoreSelected are used here. Address and type (file) is extracted from the path automatically and should produce compatible structure with the "standard" mode (basically the path is expected as for example /uncached/10.F10FCA000800/temperature, where it would extract address part F10FCA000800 and the rest after the slash is considered the type โ€” here temperature). There are two advantages to this mode โ€” you can access virtually any sensor (not just temperature), select whether to use cached or directly read values, and it is slighlty faster. The downside is more complex configuration. ๐Ÿ˜ญ

The two modes are distinguished automatically by the format of the address. It is not possible to mix the two modes. Once a full path is detected in any Sensor then the whole addressing (all sensors) is considered to be this way (and as standard addresses will fail parsing they will be ignored). No mixing, bestie โ€” the plugin commits to one vibe. โœจ

  • Device Device โ€” Sets the device to read the values from. This can either be a "real" hardware device, such as a serial port or an USB port, or the address of the owserver(1) socket, usually localhost:4304. This directive is required and does not have a default value.

Though the documentation claims to automatically recognize the given address format, with version 2.7p4 we had to specify the type explicitly. So with that version, the following configuration worked for us:

  <Plugin onewire>
    Device "-s localhost:4304"
  </Plugin>
  • Sensor Sensor โ€” In the standard mode selects sensors to collect or to ignore (depending on IgnoreSelected, see below). Sensors are specified without the family byte at the beginning, so you have to use for example F10FCA000800, and not include the leading 10. family byte and point. When no Sensor is configured the whole Onewire bus is walked and all supported sensors (see above) are read.

In the advanced mode the Sensor specifies a full OWFS path โ€” e.g. /uncached/10.F10FCA000800/temperature (or when cached values are OK /10.F10FCA000800/temperature). IgnoreSelected is not used.

As there can be multiple devices on the bus you can list multiple sensor (use multiple Sensor elements).

See /"IGNORELISTS" for details.

  • IgnoreSelected true|false โ€” If no configuration is given, the onewire plugin will collect data from all sensors found. This may not be practical, especially if sensors are added and removed regularly. Sometimes, however, itโ€™s easier/preferred to collect only specific sensors or all sensors except a few specified ones. This option enables you to do that: By setting IgnoreSelected to true the effect of Sensor is inverted: All selected interfaces are ignored and all other interfaces are collected.

Used only in the standard mode โ€” see above.

  • Interval Seconds โ€” Sets the interval in which all sensors should be read. If not specified, the global Interval setting is used.
EXPERIMENTAL! The "onewire" plugin is experimental, because it doesnโ€™t yet work with big setups. It works with one sensor being attached to one controller, but as soon as you throw in a couple more senors and maybe a hub or two, reading all values will take more than ten seconds (the default interval). We will probably add some separate thread for reading the sensors and some cache or something like that, but itโ€™s not done yet. We will try to maintain backwards compatibility in the future, but we canโ€™t promise. So in short: If it works for you: Great! But keep in mind that the config might change, though this is unlikely. Oh, and if you want to help improving this plugin, just send a short notice to the mailing list. Thanks :)

๐Ÿš Plugin "openldap" ๐Ÿš

To use the "openldap" plugin you first need to configure the OpenLDAP server correctly. The backend database "monitor" needs to be loaded and working. See slapd-monitor(5) for the details. ๐Ÿ‘‘

The configuration of the "openldap" plugin consists of one or more Instance blocks. Each block requires one string argument as the instance name. For example:

<Plugin "openldap">
  <Instance "foo">
    URL "ldap://localhost/"
  </Instance>
  <Instance "bar">
    URL "ldaps://localhost/"
  </Instance>
</Plugin>

The instance name will be used as the plugin instance. To emulate the old (version 4) behavior, you can use an empty string (""). In order for the plugin to work correctly, each instance name must be unique. This is not enforced by the plugin and it is your responsibility to ensure it is. ๐Ÿ’…

The following options are accepted within each Instance block:

  • URL ldap://host/binddn โ€” Sets the URL to use to connect to the OpenLDAP server. This option is mandatory.
  • BindDN BindDN โ€” Name in the form of an LDAP distinguished name intended to be used for authentication. Defaults to empty string to establish an anonymous authorization.
  • Password Password โ€” Password for simple bind authentication. If this option is not set, unauthenticated bind operation is used.
  • StartTLS true|false โ€” Defines whether TLS must be used when connecting to the OpenLDAP server. Disabled by default.
  • VerifyHost true|false โ€” Enables or disables peer host name verification. If enabled, the plugin checks if the "Common Name" or a "Subject Alternate Name" field of the SSL certificate matches the host name provided by the URL option. If this identity check fails, the connection is aborted. Enabled by default.
  • CACert File โ€” File that holds one or more SSL certificates. If you want to use TLS/SSL you may possibly need this option. What CA certificates are checked by default depends on the distribution you use and can be changed with the usual ldap client configuration mechanisms. See ldap.conf(5) for the details.
  • Timeout Seconds โ€” Sets the timeout value for ldap operations, in seconds. By default, the configured Interval is used to set the timeout. Use -1 to disable (infinite timeout).
  • Version Version โ€” An integer which sets the LDAP protocol version number to use when connecting to the OpenLDAP server. Defaults to 3 for using LDAPv3.

๐ŸŒˆ Plugin "openvpn" ๐ŸŒˆ

The OpenVPN plugin reads a status file maintained by OpenVPN and gathers traffic statistics about connected clients. ๐Ÿš€

To set up OpenVPN to write to the status file periodically, use the --status option of OpenVPN.

So, in a nutshell you need:

openvpn $OTHER_OPTIONS \
  --status "/var/run/openvpn-status" 10

Available options:

  • StatusFile File โ€” Specifies the location of the status file.
  • ImprovedNamingSchema true|false โ€” When enabled, the filename of the status file will be used as plugin instance and the clientโ€™s "common name" will be used as type instance. This is required when reading multiple status files. Enabling this option is recommended, but to maintain backwards compatibility this option is disabled by default.
  • CollectCompression true|false โ€” Sets whether or not statistics about the compression used by OpenVPN should be collected. This information is only available in single mode. Enabled by default.
  • CollectIndividualUsers true|false โ€” Sets whether or not traffic information is collected for each connected client individually. If set to false, currently no traffic data is collected at all because aggregating this data in a save manner is tricky. Defaults to true.
  • CollectUserCount true|false โ€” When enabled, the number of currently connected clients or users is collected. This is especially interesting when CollectIndividualUsers is disabled, but can be configured independently from that option. Defaults to false.

๐Ÿ”ฎ Plugin "oracle" ๐Ÿ”ฎ

The "oracle" plugin uses the Oracleยฎ Call Interface (OCI) to connect to an Oracleยฎ Database and lets you execute SQL statements there. It is very similar to the "dbi" plugin, because it was written around the same time. See the "dbi" pluginโ€™s documentation above for details. ๐Ÿ’Ž

<Plugin oracle>
  <Query "out_of_stock">
    Statement "SELECT category, COUNT(*) AS value FROM products WHERE in_stock = 0 GROUP BY category"
    <Result>
      Type "gauge"
      # InstancePrefix "foo"
      InstancesFrom "category"
      ValuesFrom "value"
    </Result>
  </Query>
  <Database "product_information">
    #Plugin "warehouse"
    ConnectID "db01"
    Username "oracle"
    Password "secret"
    Query "out_of_stock"
  </Database>
</Plugin>

๐Ÿงš Query blocks ๐Ÿงš

The Query blocks are handled identically to the Query blocks of the "dbi" plugin. Please see its documentation above for details on how to specify queries. ๐Ÿซถ

๐ŸŒธ Database blocks ๐ŸŒธ

Database blocks define a connection to a database and which queries should be sent to that database. Each database needs a "name" as string argument in the starting tag of the block. This name will be used as "PluginInstance" in the values submitted to the daemon. Other than that, that name is not used. ๐Ÿ‘‘

  • Plugin Plugin โ€” Use Plugin as the plugin name when submitting query results from this Database. Defaults to "oracle".
  • ConnectID ID โ€” Defines the "database alias" or "service name" to connect to. Usually, these names are defined in the file named "$ORACLE_HOME/network/admin/tnsnames.ora".
  • Host Host โ€” Hostname to use when dispatching values for this database. Defaults to using the global hostname of the collectd instance.
  • Username Username โ€” Username used for authentication.
  • Password Password โ€” Password used for authentication.
  • Query QueryName โ€” ties the query called QueryName to this database connection ๐Ÿซถ. And bestie, order matters: the query has to be defined before this statement, meaning every Query/query block you wanna reference has to sit above the database block that references it. No time travel in config files, sorry ๐Ÿ’….

๐Ÿฆ‹ Plugin "ovs_events" ๐Ÿฆ‹

So the ovs_events plugin is the one watching the link status of your Open vSwitch (OVS) connected interfaces, right? It dispatches those values to collectd and fires off a notification the second a link state change happens โœจ. It gets its link-state-change gossip straight from the OVS database.

Synopsis, babes:

 <Plugin "ovs_events">
   Port 6640
   Address "127.0.0.1"
   Socket "/var/run/openvswitch/db.sock"
   Interfaces "br0" "veth0"
   SendNotification true
   DispatchValues false
 </Plugin>

Here's the config options it serves you ๐Ÿ‘‘:

  • Address node โ€” the address of the OVS DB server JSON-RPC interface this plugin talks to ๐Ÿ’–. To actually turn that interface on, the OVS DB daemon needs to be running with the --remote=ptcp: option. Peep ovsdb-server(1) for the deets. The value can be a network hostname, IPv4 numbers-and-dots notation, or IPv6 hexadecimal string format. Defaults to "localhost".
  • Port service โ€” the TCP port to connect to ๐Ÿ”ฅ. You can hand it either a service name or a port number. Defaults to 6640.
  • Socket path โ€” the UNIX domain socket path of the OVS DB server JSON-RPC interface used by the plugin ๐ŸŒธ. To enable the interface, the OVS DB daemon should be running with the --remote=punix: option. See ovsdb-server(1) for more. Heads up: if this option is set, the Address and Port options are ignored ๐Ÿ’€.
  • Interfaces [ifname ...] โ€” the list of interface names this plugin should monitor ๐ŸŽ€. If you don't specify it, or it's empty, then literally all OVS connected interfaces on all bridges get monitored. Default: empty (all interfaces on all bridges are monitored)
  • SendNotification true|false โ€” set to true and OVS link notifications (interface status and OVS DB connection terminate) get sent to collectd ๐ŸŒˆ. Default value is true.
  • DispatchValues true|false โ€” dispatches the OVS DB interface link status value using the configured plugin interval โœจ. Defaults to false. Oh and btw: if both SendNotification and DispatchValues are false, the plugin will provide you with no OVS information at all ๐Ÿ™ˆ.
Note, bestie: by default the global Interval setting is what's used to retrieve the OVS link status โญ. Wanna set a plugin-specific interval? Use the Interval option of the OVS LoadPlugin block settings. For milliseconds, just divide the time by 1000 โ€” like, if your dream interval is 50ms, set interval to 0.05 ๐Ÿ’œ.

๐ŸŒŠ Plugin "ovs_stats" ๐ŸŒŠ

The ovs_stats plugin collects statistics of OVS connected interfaces ๐Ÿ“Š. It uses the OVSDB management protocol (RFC7047) monitor mechanism to pull those stats out of OVSDB, no cap.

Synopsis:

 <Plugin "ovs_stats">
   Port 6640
   Address "127.0.0.1"
   Socket "/var/run/openvswitch/db.sock"
   Bridges "br0" "br_ext"
   InterfaceStats false
 </Plugin>

The options, served fresh ๐Ÿ“:

  • Address node โ€” the address of the OVS DB server JSON-RPC interface used by the plugin ๐Ÿ’–. To enable the interface, the OVS DB daemon should be running with the --remote=ptcp: option. See ovsdb-server(1) for the fine print. The value may be a network hostname, IPv4 numbers-and-dots notation, or IPv6 hexadecimal string format. Defaults to "localhost".
  • Port service โ€” the TCP port to connect to ๐Ÿ”ฅ. Either a service name or a port number is fine. Defaults to 6640.
  • Socket path โ€” the UNIX domain socket path of the OVS DB server JSON-RPC interface used by the plugin ๐ŸŒธ. To enable the interface, the OVS DB daemon should be running with the --remote=punix: option. See ovsdb-server(1) for more details. If this option is set, Address and Port are ignored ๐Ÿ’€.
  • Bridges [brname ...] โ€” the list of OVS bridge names to monitor ๐ŸŽ€. If you omit it or leave it empty, then all OVS bridges get monitored. Default: empty (monitor all bridges)
  • InterfaceStats false|true โ€” tells the plugin it should also gather statistics for individual interfaces in addition to ports ๐Ÿ‘‘. Super useful when you're monitoring an OVS setup with bond ports and you lowkey wanna know the individual stats for the interfaces inside those bonds. Defaults to false.

๐Ÿšจ Plugin "pcie_errors" ๐Ÿšจ

The pcie_errors plugin collects PCI Express errors from the Device Status in the Capability structure, and from the Advanced Error Reporting Extended Capability where that's available ๐Ÿ˜ค. Every single read it polls the config space of PCI Express devices and dispatches a notification for every error that's set โ€” so yeah, it's checking for new errors at every read, always watching ๐Ÿ‘€. The device shows up in plugin_instance in the format "domain:bus:dev.fn". Errors get split into categories indicated by type_instance: "correctable", and for uncorrectable errors "non_fatal" or "fatal" ๐Ÿ’…. Fatal errors are reported as NOTIF_FAILURE and all the others as NOTIF_WARNING.

Synopsis:

  <Plugin "pcie_errors">
    Source "sysfs"
    AccessDir "/sys/bus/pci"
    ReportMasked false
    PersistentNotifications false
  </Plugin>

Options, bestie:

  • Source sysfs|proc โ€” use sysfs or proc to read data from /sysfs or /proc ๐Ÿฆ‹. The default value is sysfs.
  • AccessDir dir โ€” the directory used to access device config space โœจ. It's optional, and defaults to /sys/bus/pci for sysfs and to /proc/bus/pci for proc.
  • ReportMasked false|true โ€” if true, the plugin will notify you about errors that are set to masked in the Error Mask register ๐Ÿ™ˆ. Those errors aren't reported to the PCI Express Root Complex, jsyk. Defaults to false.
  • PersistentNotifications false|true โ€” if false, the plugin only dispatches a notification on set/clear of an error ๐Ÿ’€. The ones it already reported get ignored. Defaults to false.

๐Ÿงš Plugin "perl" ๐Ÿงš

This plugin embeds a Perl interpreter right into collectd and gives you an interface to collectd's plugin system ๐Ÿชโœจ. Its documentation lives in collectd-perl(5) โ€” go read that, iykyk.


๐ŸŒธ Plugin "pinba" ๐ŸŒธ

The Pinba plugin receives profiling information from Pinba, which is an extension for the PHP interpreter ๐Ÿ˜. At the end of executing a script โ€” like, right after a PHP-based webpage has been delivered โ€” the extension sends a UDP packet stuffed with timing information, peak memory usage, and so on ๐Ÿ“ฆ. The plugin waits for those packets, parses them, accounts for all the info provided, and then it gets dispatched to the daemon once per interval โฑ๏ธ.

Synopsis:

 <Plugin pinba>
   Address "::0"
   Port "30002"
   # Overall statistics for the website.
   <View "www-total">
     Server "www.example.com"
   </View>
   # Statistics for www-a only
   <View "www-a">
     Host "www-a.example.com"
     Server "www.example.com"
   </View>
   # Statistics for www-b only
   <View "www-b">
     Host "www-b.example.com"
     Server "www.example.com"
   </View>
 </Plugin>

The plugin provides these config options ๐Ÿ’–:

  • Address Node โ€” configures the address used to open a listening socket ๐ŸŽ€. By default the plugin binds to the any address "::0".
  • Port Service โ€” configures the port (service) to bind to ๐Ÿ”ฅ. By default the default Pinba port "30002" is used. This option accepts service names in addition to port numbers, which is why it needs a string argument, periodt.
  • <View Name> block โ€” the packets sent by the Pinba extension include the hostname of the server, the server name (the name of the virtual host), and the script that was executed ๐Ÿ‘‘. With View blocks you can separate the data into multiple groups so your statistics actually mean something. Important: every packet gets added to all matching groups, so a single packet may be accounted for more than once ๐Ÿ’€.
    • Host Host โ€” matches the hostname of the system the webserver / script is running on ๐Ÿฆ‹. This contains the result of the gethostname(2) system call. If you don't configure it, all hostnames will be accepted.
    • Server Server โ€” matches the name of the virtual host, i.e. the contents of the $_SERVER["SERVER_NAME"] variable when you're within PHP โœจ. If not configured, all server names will be accepted.
    • Script Script โ€” matches the name of the script, i.e. the contents of the $_SERVER["SCRIPT_NAME"] variable when within PHP ๐ŸŒˆ. If not configured, all script names will be accepted.

๐Ÿ“ Plugin "ping" ๐Ÿ“

The Ping plugin spins up a whole new thread that sends ICMP "ping" packets to the hosts you configured, periodically, and measures the network latency ๐Ÿ“ก. Every time the plugin's "read" function is called, it submits the average latency, the standard deviation, and the drop rate for each host ๐Ÿ’–.

Available config options, go off:

  • Host IP-address โ€” the host to ping periodically ๐ŸŽฏ. This option may be repeated several times so you can ping multiple hosts at once.
  • Interval Seconds โ€” sets the interval in which ICMP echo packets get sent to the configured hosts โฑ๏ธ. And no, this is not the interval in which metrics are read from the plugin โ€” it's the interval in which the hosts are actually "pinged". So this setting should be smaller than or equal to the global Interval setting. Fractional times, like "1.24", are allowed ๐Ÿ’…. Default: 1.0
  • Timeout Seconds โ€” how long to wait for a response from the host you sent an ICMP packet to ๐Ÿ™ˆ. If no reply arrives after Seconds seconds, the host is assumed to be down or the packet assumed dropped. This setting must be smaller than the Interval setting above for the plugin to work correctly, fr fr. Fractional arguments are accepted. Default: 0.9
  • TTL 0-255 โ€” sets the Time-To-Live of the generated ICMP packets ๐Ÿฆ‹.
  • Size size โ€” sets the size of the data payload in the ICMP packet to the specified size (it gets filled with a regular ASCII pattern) ๐Ÿ“ฆ. If it's not set, a default 56 byte long string is used, so that the packet size of an ICMPv4 packet is exactly 64 bytes โ€” similar to what the normal ping(1) command does โœจ.
  • SourceAddress host โ€” sets the source address to use ๐ŸŒธ. host may be either a numerical network address or a network hostname.
  • AddressFamily af โ€” sets the address family to use. af may be "any", "ipv4" or "ipv6" ๐ŸŒˆ. Fair warning: this option is ignored if you set a SourceAddress ๐Ÿ’€.
  • Device name โ€” sets the outgoing network device to be used. name has to specify an interface name (e. g. "eth0") ๐Ÿ”Œ. Heads up though: this might not be supported by all operating systems, so vibe check your kernel first ๐Ÿ˜ญ.

๐Ÿง MaxMissed Packets

okay so this one is a little โœจtriggerโœจ โ€” once the host has ghosted you for Packets packets (like, not replied at all ๐Ÿ˜ญ), it fires off a DNS resolve. why do we care? because that's what lets you use dynamic DNS services (think dyndns.org, yes that's a real example from the page ๐Ÿ’–) together with the ping plugin. it's giving "if you're not gonna answer me, I'm updating my address book" energy, honestly.

Default: -1 (disabled)


๐Ÿ˜ Plugin "postgresql"

okay bestie, this plugin queries statistics from PostgreSQL databases. It keeps a persistent connection to all the configured databases and tries to reconnect if that connection gets interrupted โ€” resilient queen ๐Ÿ’…. You set up a database by specifying a Database block, as described below.

The default statistics come from PostgreSQL's statistics collector, which therefore has to be enabled for this plugin to work correctly. Usually it is, by default โ€” so like, don't panic ๐Ÿซถ. See the section "The Statistics Collector" of the PostgreSQL Documentation for the deets.

And here's the fun part โœจ โ€” you can specify custom database queries using a Query block (again, described below), and then you may collect any data that's available from some PostgreSQL database. That means you can:

  • access statistics of external daemons which are available in a PostgreSQL database, or
  • use future or special statistics provided by PostgreSQL without needing to upgrade your collectd installation.

Obsessed. No upgrade required, we love to see it. ๐Ÿ‘‘

Starting with version 5.2, the "postgresql" plugin also supports writing data to PostgreSQL databases. It's implemented in a generic way: you specify an SQL statement, and collectd executes it to write the data (details below). The benefit of that approach is there's no fixed database layout โ€” instead, the layout may be optimized for the current setup. main character energy for your schema fr fr ๐ŸŒˆ.

The PostgreSQL Documentation manual can be found at http://www.postgresql.org/docs/manuals/.

<Plugin postgresql>
  <Query magic>
    Statement "SELECT magic FROM wizard WHERE host = $1;"
    Param hostname
    <Result>
      Type gauge
      InstancePrefix "magic"
      ValuesFrom magic
    </Result>
  </Query>

  <Query rt36_tickets>
    Statement "SELECT COUNT(type) AS count, type \
                      FROM (SELECT CASE \
                                   WHEN resolved = 'epoch' THEN 'open' \
                                   ELSE 'resolved' END AS type \
                                   FROM tickets) type \
                      GROUP BY type;"
    <Result>
      Type counter
      InstancePrefix "rt36_tickets"
      InstancesFrom "type"
      ValuesFrom "count"
    </Result>
  </Query>

  <Writer sqlstore>
    Statement "SELECT collectd_insert($1, $2, $3, $4, $5, $6, $7, $8, $9);"
    StoreRates true
  </Writer>

  <Database foo>
    Plugin "kingdom"
    Host "hostname"
    Port "5432"
    User "username"
    Password "secret"
    SSLMode "prefer"
    KRBSrvName "kerberos_service_name"
    Query magic
  </Database>

  <Database bar>
    Interval 300
    Service "service_name"
    Query backends # predefined
    Query rt36_tickets
  </Database>

  <Database qux>
    # ...
    Writer sqlstore
    CommitInterval 10
  </Database>
</Plugin>

The Query block defines one database query, which may later be used by a database definition. It accepts a single mandatory argument that specifies the name of the query. The names of all queries have to be unique โ€” see the MinVersion and MaxVersion options below for the one exception to that rule ๐Ÿ‘€.

In each Query block, there's one or more Result blocks. Multiple Result blocks may be used to extract multiple values from a single query. Stack 'em up ๐ŸŽ€.

The following configuration options are available to define the query:

๐Ÿ“ Statement sql query statement

Specify the sql query statement which the plugin should execute. The string may contain the tokens $1, $2, etc., which are used to reference the first, second, etc. parameter. The value of the parameters is specified by the Param configuration option โ€” see below for details. To include a literal $ character followed by a number, surround it with single quotes (').

Any SQL command which may return data (such as "SELECT" or "SHOW") is allowed. Note, however, that only a single command may be used. Semicolons are allowed as long as a single non-empty command has been specified only.

The returned lines will be handled separately, one after another.

๐Ÿ”ข Param hostname|database|instance|username|interval

Specify the parameters which should be passed to the SQL query. The parameters are referred to in the SQL query as $1, $2, etc., in the same order as they appear in the configuration file. The value of the parameter is determined depending on the value of the Param option, as follows:

  • hostname โ€” the configured hostname of the database connection. If a UNIX domain socket is used, the parameter expands to "localhost".
  • database โ€” the name of the database of the current connection.
  • instance โ€” the name of the database plugin instance. See the Instance option of the database specification below for details.
  • username โ€” the username used to connect to the database.
  • interval โ€” the interval with which this database is queried (as specified by the database specific or global Interval options).
Please note that parameters are only supported by PostgreSQL's protocol version 3 and above, which was introduced in version 7.4 of PostgreSQL. so if you're on something ancient, this part is not for you ๐Ÿ˜ญ

๐Ÿท๏ธ PluginInstanceFrom column

Specify how to create the "PluginInstance" for reporting this query's results. Only one column is supported. You may concatenate fields and string values in the query statement to get the required results โ€” get creative bestie โœจ.

๐Ÿ“… MinVersion version / MaxVersion version

Specify the minimum or maximum version of PostgreSQL that this query should be used with. Some statistics might only be available with certain versions of PostgreSQL. This lets you specify multiple queries with the same name that apply to different versions โ€” so the same configuration can be used in a heterogeneous environment. slay, one config to rule them all ๐Ÿ‘‘.

The version has to be specified as the concatenation of the major, minor and patch-level versions, each represented as two-decimal-digit numbers. For example, version 8.2.3 will become 80203.


The Result block defines how to handle the values returned from the query. It defines which column holds which value and how to dispatch that value to the daemon. iykyk ๐Ÿ’…

๐Ÿ’– Type type

The type name to be used when dispatching the values. The type describes how to handle the data and where to store it. See types.db(5) for more details on types and their configuration. The number and type of values (as selected by the ValuesFrom option) has to match the type of the given name.

This option is mandatory โ€” no skipping ๐Ÿ”ฅ.

๐Ÿงต InstancePrefix prefix / InstancesFrom column0 [column1 ...]

Specify how to create the "TypeInstance" for each data set (i.e. line). InstancePrefix defines a static prefix that will be prepended to all type instances. InstancesFrom defines the column names whose values will be used to create the type instance. Multiple values will be joined together using the hyphen ("-") as separation character.

The plugin itself does not check whether or not all built instances are different. It is your responsibility to assure that each is unique. no cap, that's on you ๐Ÿซต

Both options are optional. If none is specified, the type instance will be empty.

๐Ÿ“Š ValuesFrom column0 [column1 ...]

Names the columns whose content is used as the actual data for the data sets that are dispatched to the daemon. How many such columns you need is determined by the Type setting, as explained above. If you specify too many or not enough columns, the plugin will complain about that and no data will be submitted to the daemon. ๐Ÿ’€

The actual data type, as seen by PostgreSQL, is not that important as long as it represents numbers. The plugin will automatically cast the values to the right type if it knows how to do that. For that, it uses the strtoll(3) and strtod(3) functions, so anything supported by those functions is supported by the plugin as well. iconic behavior honestly ๐ŸŽ€.

This option is required inside a Result block and may be specified multiple times. If multiple ValuesFrom options are specified, the columns are read in the given order.


The following predefined queries are available โ€” the definitions can be found in the postgresql_default.conf file, which by default is available at "prefix/share/collectd/" ๐Ÿฆ‹:

  • backends โ€” this query collects the number of backends, i.e. the number of connected clients.
  • transactions โ€” this query collects the numbers of committed and rolled-back transactions of the user tables.
  • queries โ€” this query collects the numbers of various table modifications (i.e. insertions, updates, deletions) of the user tables.
  • query_plans โ€” this query collects the numbers of various table scans and returned tuples of the user tables.
  • table_states โ€” this query collects the numbers of live and dead rows in the user tables.
  • disk_io โ€” this query collects disk block access counts for user tables.
  • disk_usage โ€” this query collects the on-disk size of the database in bytes.

In addition, the following detailed queries are available by default. Please note that each of those queries collects information by table, thus potentially producing a lot of data โ€” like, a LOT, be warned bestie ๐Ÿ™ˆ. For details, see the description of the non-by_table queries above.

  • queries_by_table
  • query_plans_by_table
  • table_states_by_table
  • disk_io_by_table

The Writer block defines a PostgreSQL writer backend. It accepts a single mandatory argument specifying the name of the writer. This will then be used in the Database specification in order to activate the writer instance. The names of all writers have to be unique. The following options may be specified:

โœ๏ธ Statement sql statement

This mandatory option specifies the SQL statement that will be executed for each submitted value. A single SQL statement is allowed only. Anything after the first semicolon will be ignored. ๐Ÿšฎ

Nine parameters will be passed to the statement and should be specified as tokens $1, $2, through $9 in the statement string. The following values are made available through those parameters:

  • $1 โ€” the timestamp of the queried value as an RFC 3339-formatted local time.
  • $2 โ€” the hostname of the queried value.
  • $3 โ€” the plugin name of the queried value ๐Ÿ’…
  • $4 โ€” the plugin instance of the queried value. This one can be NULL if there's, like, no plugin instance.
  • $5 โ€” the type of the queried value (cf. types.db(5)).
  • $6 โ€” the type instance of the queried value. Also possibly NULL if there's no type instance, no cap.
  • $7 โ€” an array of names for the submitted values (i.e. the name of the data sources of the submitted value-list) โœจ
  • $8 โ€” an array of types for the submitted values (i.e. the type of the data sources of the submitted value-list; "counter", "gauge", โ€ฆ). Heads up bestie: if StoreRates is enabled (which it is by default, see below), all types will be "gauge".
  • $9 โ€” an array of the submitted values. The dimensions of the value name and value arrays match, fr fr.

In general, it is advisable to create and call a custom function in the PostgreSQL database for this purpose. Any procedural language supported by PostgreSQL will do (see chapter "Server Programming" in the PostgreSQL manual for details).


  • StoreRates false|true โ€” if set to true (the default), convert counter values to rates. If set to false, counter values are stored as is, i.e. as an increasing integer number. ๐Ÿงš

The Database block defines one PostgreSQL database for which to collect statistics. It accepts a single mandatory argument which specifies the database name. None of the other options are required. PostgreSQL will use default values as documented in the section "CONNECTING TO A DATABASE" in the psql(1) manpage. However โ€” and listen, this matters โ€” be aware that those defaults may be influenced by the user collectd is run as and special environment variables. See the manpage for details. ๐Ÿ‘€

  • Interval seconds โ€” specify the interval with which the database should be queried. The default is to use the global Interval setting.
  • CommitInterval seconds โ€” this option may be used for database connections which have "writers" assigned (see above). If specified, it causes a writer to put several updates into a single transaction. This transaction will last for the specified amount of time. By default, each update will be executed in a separate transaction.

Each transaction generates a fair amount of overhead which can, thus, be reduced by activating this option. The draw-back is that data covering the specified amount of time will be lost, for example, if a single statement within the transaction fails or if the database server crashes. ๐Ÿ˜ญ So like, trade-off city.

  • Plugin Plugin โ€” use Plugin as the plugin name when submitting query results from this Database. Defaults to "postgresql".
  • Instance name โ€” specify the plugin instance name that should be used instead of the database name (which is the default, if this option has not been specified). This allows one to query multiple databases of the same name on the same host (e.g. when running multiple database server versions in parallel). The plugin instance name can also be set from the query result using the PluginInstanceFrom option in the Query block. ๐Ÿ’ซ
  • Host hostname โ€” specify the hostname or IP of the PostgreSQL server to connect to. If the value begins with a slash, it is interpreted as the directory name in which to look for the UNIX domain socket.

This option is also used to determine the hostname that is associated with a collected data set. If it has been omitted or either begins with with a slash or equals localhost it will be replaced with the global hostname definition of collectd. Any other value will be passed literally to collectd when dispatching values. Also see the global Hostname and FQDNLookup options. ๐Ÿซถ

  • Port port โ€” specify the TCP port or the local UNIX domain socket file extension of the server.
  • User username โ€” specify the username to be used when connecting to the server.
  • Password password โ€” specify the password to be used when connecting to the server.
  • ExpireDelay delay โ€” skip expired values in query output.
  • SSLMode disable|allow|prefer|require โ€” specify whether to use an SSL connection when contacting the server. The following modes are supported:
    • disable โ€” do not use SSL at all.
    • allow โ€” first, try to connect without using SSL. If that fails, try using SSL.
    • prefer (default) โ€” first, try to connect using SSL. If that fails, try without using SSL.
    • require โ€” use SSL only. No exceptions, bestie. ๐Ÿ”’
  • Instance name โ€” specify the plugin instance name that should be used instead of the database name (which is the default, if this option has not been specified). This allows one to query multiple databases of the same name on the same host (e.g. when running multiple database server versions in parallel).
Yes, this option shows up twice in the original docs. Iconic of them honestly. Both are listed here, both count. ๐Ÿ’…
  • KRBSrvName kerberos_service_name โ€” specify the Kerberos service name to use when authenticating with Kerberos 5 or GSSAPI. See the sections "Kerberos authentication" and "GSSAPI" of the PostgreSQL Documentation for details.
  • Service service_name โ€” specify the PostgreSQL service name to use for additional parameters. That service has to be defined in pg_service.conf and holds additional connection parameters. See the section "The Connection Service File" in the PostgreSQL Documentation for details.
  • Query query โ€” specifies a query which should be executed in the context of the database connection. This may be any of the predefined or user-defined queries. If no such option is given, it defaults to "backends", "transactions", "queries", "query_plans", "table_states", "disk_io" and "disk_usage" (unless a Writer has been specified). Else, the specified queries are used only.
  • Writer writer โ€” assigns the specified writer backend to the database connection. This causes all collected data to be send to the database using the settings defined in the writer configuration (see the section "FILTER CONFIGURATION" below for details on how to selectively send data to certain plugins).

Each writer will register a flush callback which may be used when having long transactions enabled (see the CommitInterval option above). When issuing the FLUSH command (see collectd-unixsock(5) for details) the current transaction will be committed right away. Two different kinds of flush callbacks are available with the "postgresql" plugin:

  • postgresql โ€” flush all writer backends. ๐ŸŒŠ
  • postgresql-database โ€” flush all writers of the specified database only.

๐ŸŒธ Plugin "powerdns" ๐ŸŒธ

The "powerdns" plugin queries statistics from an authoritative PowerDNS nameserver and/or a PowerDNS recursor. Since both offer a wide variety of values, many of which are probably meaningless to most users, but may be useful for some. So you may chose which values to collect, but if you don't, some reasonable defaults will be collected. ๐Ÿฆ‹

  <Plugin "powerdns">
    <Server "server_name">
      Collect "latency"
      Collect "udp-answers" "udp-queries"
      Socket "/var/run/pdns.controlsocket"
    </Server>
    <Recursor "recursor_name">
      Collect "questions"
      Collect "cache-hits" "cache-misses"
      Socket "/var/run/pdns_recursor.controlsocket"
    </Recursor>
    LocalSocket "/opt/collectd/var/run/collectd-powerdns"
  </Plugin>

Server and Recursor block

The Server block defines one authoritative server to query, the Recursor does the same for an recursing server. The possible options in both blocks are the same, though. The argument defines a name for the server / recursor and is required. โœจ

  • Collect Field โ€” using the Collect statement you can select which values to collect. Here, you specify the name of the values as used by the PowerDNS servers, e.g. "dlg-only-drops", "answers10-100".

The method of getting the values differs for Server and Recursor blocks: when querying the server a "SHOW *" command is issued in any case, because that's the only way of getting multiple values out of the server at once. collectd then picks out the values you have selected. When querying the recursor, a command is generated to query exactly these values. So if you specify invalid fields when querying the recursor, a syntax error may be returned by the daemon and collectd may not collect any values at all. ๐Ÿ’€ Yeah. Be careful.

If no Collect statement is given, the following Server values will be collected:

latency
packetcache-hit
packetcache-miss
packetcache-size
query-cache-hit
query-cache-miss
recursing-answers
recursing-questions
tcp-answers
tcp-queries
udp-answers
udp-queries

The following Recursor values will be collected by default:

noerror-answers
nxdomain-answers
servfail-answers
sys-msec
user-msec
qa-latency
cache-entries
cache-hits
cache-misses
questions

Please note that up to that point collectd doesn't know what values are available on the server and values that are added do not need a change of the mechanism so far. However, the values must be mapped to collectd's naming scheme, which is done using a lookup table that lists all known values. If values are added in the future and collectd does not know about them, you will get an error much like this:

  powerdns plugin: submit: Not found in lookup table: foobar = 42

In this case please file a bug report with the collectd team. ๐Ÿ›๐Ÿ™

  • Socket Path โ€” configures the path to the UNIX domain socket to be used when connecting to the daemon. By default "${localstatedir}/run/pdns.controlsocket" will be used for an authoritative server and "${localstatedir}/run/pdns_recursor.controlsocket" will be used for the recursor.

LocalSocket Path

Querying the recursor is done using UDP. When using UDP over UNIX domain sockets, the client socket needs a name in the file system, too. You can set this local name to Path using the LocalSocket option. The default is "prefix/var/run/collectd-powerdns". ๐ŸŽ€


๐ŸŒˆ Plugin "processes" ๐ŸŒˆ

Collects information about processes of local system. Literally that's it, she's simple, she's effective, we stan. ๐Ÿ‘‘

okay so out of the box โ€” like, zero process matches configured โ€” the plugin is in its low-effort era and only collects general statistics: the number of processes in each state and the fork rate. that's it. that's the vibe.

if you want the good stuff, you set up process matches with the Process and ProcessMatch options. and plot twist: these can also be a block, so you can specify further options inside them ๐ŸŽ€

the statistics collected for matched processes are:

  • size of the resident segment size (RSS)
  • user- and system-time used
  • number of processes
  • number of threads
  • number of open files (under Linux)
  • number of memory mapped files (under Linux)
  • io data (where available)
  • context switches (under Linux)
  • minor and major pagefaults
  • Delay Accounting information (Linux only, requires libmnl)

synopsis, bestie โœจ

<Plugin processes>
  CollectFileDescriptor  true
  CollectContextSwitch   true
  CollectDelayAccounting false
  CollectSystemContextSwitch false
  Process "name"
  ProcessMatch "name" "regex"
  <Process "collectd">
    CollectFileDescriptor  false
    CollectContextSwitch   false
    CollectDelayAccounting true
  </Process>
  <ProcessMatch "name" "regex">
    CollectFileDescriptor false
    CollectContextSwitch true
  </ProcessMatch>
</Plugin>

the options, one by one ๐Ÿ’…

  • Process Name โ€” selects more detailed statistics for processes matching this name. heads up though: some platforms have a limit on the length of process names, and Name must stay below this limit. no cap, just a cap on characters ๐Ÿซถ
  • ProcessMatch name regex โ€” selects more detailed statistics for processes matching the specified regex (see regex(7) for details). the statistics of all matching processes get summed up and dispatched to the daemon using the specified name as an identifier โ€” which is literally how you "group" several processes together, iconic behavior tbh. name must not contain slashes. periodt.
  • CollectContextSwitch Boolean โ€” collects the number of context switches for matched processes. disabled by default.
  • CollectDelayAccounting Boolean โ€” if enabled, collects Linux Delay Accounding information for matching processes. Delay Accounting provides the time processes wait for the CPU to become available, for I/O operations to finish, for pages to be swapped in and for freed pages to be reclaimed. the metrics are reported as "seconds per second" using the delay_rate type, e.g. delay_rate-delay-cpu. disabled by default. also: this option is only available on Linux, requires the "libmnl" library and requires the "CAP_NET_ADMIN" capability at runtime โ€” so like, check your privileges before you come for her ๐Ÿ’€
  • CollectFileDescriptor Boolean โ€” collects number of file descriptors of matched processes. disabled by default.
  • CollectMemoryMaps Boolean โ€” collects the number of memory mapped files of the process. the limit for this number is configured via /proc/sys/vm/max_map_count in the Linux kernel. knowledge is power, bestie ๐Ÿง 
  • CollectSystemContextSwitch Boolean โ€” collects the number of context switches at the system level. collects ctxt fields from /proc/stat in linux systems. can be configured only outside the Process and ProcessMatch blocks โ€” so don't try to sneak it inside, it's not that girl ๐Ÿ™ˆ
little nuance drop: the CollectContextSwitch, CollectDelayAccounting, CollectFileDescriptor and CollectMemoryMaps options may be used inside Process and ProcessMatch blocks. when used there, these options affect reporting the corresponding processes only. outside of Process and ProcessMatch blocks, these options set the default value for subsequent matches. iykyk ๐Ÿฆ‹

Plugin "procevent"

this one is a whole stalker era but make it wholesome: the procevent plugin monitors when processes start (EXEC) and stop (EXIT). notifications queen ๐Ÿ‘‘

synopsis:

<Plugin procevent>
  BufferLength 10
  Process "name"
  ProcessRegex "regex"
</Plugin>

options:

  • BufferLength length โ€” maximum number of process events that can be stored in the plugin's ring buffer. by default, this is set to 10. once an event has been read, its location becomes available for storing a new event. recycling, we love to see it โ™ป๏ธ
  • Process name โ€” enumerate a process name to monitor. all processes that match this exact name will be monitored for EXECs and EXITs.
  • ProcessRegex regex โ€” enumerate a process pattern to monitor. all processes that match this regular expression will be monitored for EXECs and EXITs.

Plugin "protocols"

this plugin collects a lot of information about various network protocols, such as IP, TCP, UDP, etc. she's thorough, she does her research ๐Ÿ”

available configuration options:

  • Value Selector โ€” selects whether or not to select a specific value. the string being matched is of the form "Protocol:ValueName", where Protocol will be used as the plugin instance and ValueName will be used as type instance. an example of the string being used would be "Tcp:RetransSegs".

you can use regular expressions to match a large number of values with just one configuration option. to select all "extended" TCP values, you could use the following statement:

Value "/^TcpExt:/"

whether only matched values are selected or all matched values are ignored depends on the IgnoreSelected. by default, only matched values are selected. if no value is configured at all, all values will be selected. see /"IGNORELISTS" for details ๐Ÿ“š

  • IgnoreSelected true|false โ€” if set to true, inverts the selection made by Value, i. e. all matching values will be ignored. gaslight gatekeep girlboss โœจ

Plugin "python"

this plugin embeds a Python-interpreter into collectd and provides an interface to collectd's plugin system. see collectd-python(5) for its documentation. short and sweet, she said what she said ๐Ÿ


Plugin "ras"

the ras plugin gathers and counts errors provided by [RASDaemon](https://github.com/mchehab/rasdaemon). this plugin requires access to SQLite3 database from RASDaemon.

metrics:

  • type: ras_errors
  • plugin_instance: CPU_(number CPU) for metrics per CPU Core metric. for metrics per Server metrics this value is empty.
  • type_instance: โ€” split into two moods:
    • per CPU Core:
      • memory_read_corrected_errors
      • memory_read_uncorrectable_errors
      • memory_write_corrected_errors
      • memory_write_uncorrectable_errors
      • cache_l0_l1_errors
      • tlb_instruction_errors
      • processor_base_errors
      • processor_bus_errors
      • internal_timer_errors
      • smm_handler_code_access_violation_errors
      • internal_parity_errors
      • frc_errors
      • external_mce_errors
      • microcode_rom_parity_errors
      • unclassified_mce_errors
    • per Server:
      • cache_l2_errors
      • upi_errors

please note that processor_base_errors is an aggregate counter measuring the following MCE events:

  • internal_timer_errors
  • smm_handler_code_access_violation_errors
  • internal_parity_errors
  • frc_errors
  • external_mce_errors
  • microcode_rom_parity_errors
  • unclassified_mce_errors

in addition, RASDaemon runs, by default, with --enable-sqlite3 flag. in case of problems with the SQLite3 database, please verify this is still a default option. like, don't come for the plugin if the flag got changed, mmkay ๐Ÿซถ

  • DB_Path Path โ€” path to the RASDemon database (sqlite3). please make sure that user has read permissions to this database. example and default setting:
DB_Path "/var/lib/rasdaemon/ras-mc_event.db"

Plugin "redfish"

the redfish plugin collects sensor data using a REST protocol called Redfish. fancy, networked, sensor-collecting icon ๐Ÿ’…

sample configuration:

<Plugin redfish>
  <Query "fans">
    Endpoint "/redfish/v1/Chassis/Chassis-1/Thermal"
    <Resource "Fans">
      <Property "ReadingRPM">
        PluginInstance "chassis-1"
        Type "rpm"
      </Property>
    </Resource>
  </Query>
  <Query "temperatures">
    Endpoint "/redfish/v1/Chassis/Chassis-1/Thermal"
    <Resource "Temperatures">
      <Property "ReadingCelsius">
        PluginInstance "chassis-1"
        Type "degrees"
      </Property>
    </Resource>
  </Query>
  <Query "voltages">
    Endpoint "/redfish/v1/Chassis/Chassis-1/Power"
    <Resource "Voltages">
      <Property "ReadingVolts">
        PluginInstance "chassis-1"
        Type "volts"
      </Property>
    </Resource>
  </Query>
  <Service "local">
    Host "127.0.0.1:5000"
    User "user"
    Passwd "passwd"
    Queries "fans" "voltages" "temperatures"
  </Service>
</Plugin>

and here's what every piece of that means:

  • Query โ€” section defining a query performed on Redfish interface
  • Endpoint โ€” URI of the REST API Endpoint for accessing the BMC
  • Resource โ€” selects single resource or array to collect data.
  • Property โ€” selects property from which data is gathered
  • PluginInstance โ€” plugin instance of dispatched collectd metric
  • Type โ€” type of dispatched collectd metric
  • TypeInstance โ€” type instance of collectd metric
  • Service โ€” section defining service to be sent requests
  • Username โ€” BMC username
  • Password โ€” BMC password
  • Queries โ€” queries to run

Plugin "routeros"

the routeros plugin connects to a device running RouterOS, the Linux-based operating system for routers by MikroTik. the plugin uses librouteros to connect and reads information about the interfaces and wireless connections of the device. and the config supports querying multiple routers, because she's polyamorous with her network gear ๐ŸŽ€

<Plugin "routeros">
  <Router>
    Host "router0.example.com"
    User "collectd"
    Password "secr3t"
    CollectInterface true
    CollectCPULoad true
    CollectMemory true
  </Router>
  <Router>
    Host "router1.example.com"
    User "collectd"
    Password "5ecret"
    CollectInterface true
    CollectRegistrationTable true
    CollectDF true
    CollectDisk true
    CollectHealth true
  </Router>
</Plugin>

as you can see above, the configuration of the routeros plugin consists of one or more <Router> blocks. within each block, the following options are understood:

  • Host Host โ€” hostname or IP-address of the router to connect to.
  • Port Port โ€” port name or port number used when connecting. if left unspecified, the default will be chosen by librouteros, currently "8728". this option expects a string argument, even when a numeric port number is given. yes, even then. string. don't argue with her ๐Ÿ’€
  • User User โ€” use the user name User to authenticate. defaults to "admin".
  • Password Password โ€” set the password used to authenticate.
  • CollectInterface true|false โ€” when set to true, interface statistics will be collected for all interfaces present on the device. defaults to false.
  • CollectRegistrationTable true|false โ€” when set to true, information about wireless LAN connections will be collected. defaults to false.
  • CollectCPULoad true|false โ€” flip this to true and boom, your CPU usage is getting collected ๐Ÿ’… The number is a dimensionless value where zero means no CPU usage at all, zero, nada. Defaults to false.
  • CollectMemory true|false โ€” turn this on and you're collecting how much memory is used and how much is free. Real talk though โ€” how "used memory" actually gets calculated is unknown, like, nobody's saying whether caches count as used space or not ๐Ÿ˜ญ. Defaults to false.
  • CollectDF true|false โ€” enable it and you get the amount of used and free disk space collected. Defaults to false.
  • CollectDisk true|false โ€” enable it and it collects the number of sectors written plus bad blocks ๐Ÿ”ฅ. Defaults to false.
  • CollectHealth true|false โ€” enable this and health statistics get collected, which includes the voltage and temperature on supported hardware ๐ŸŒก๏ธ. Defaults to false.

๐Ÿ“ Plugin "redis" ๐Ÿ“

Okay so the Redis plugin connects to one or more Redis servers, gathers info about each server's state, and runs your user-defined queries โœจ For every server there's a Node block that configures the connection parameters and the set of user-defined queries for that node.

<Plugin redis>
  <Node "example">
      Host "localhost"
      Port "6379"
      #Socket "/var/run/redis/redis.sock"
      Timeout 2000
      ReportCommandStats false
      ReportCpuUsage true
      <Query "LLEN myqueue">
        #Database 0
        Type "queue_length"
        Instance "myqueue"
      </Query>
  </Node>
</Plugin>

Node Nodename

The Node block identifies a new Redis node โ€” meaning a new Redis instance running on a specified host and port ๐ŸŽ€ The name for the node is a canonical identifier that gets used as the plugin instance. It's limited to 128 characters in length, so no essay-length names bestie.

When no Node is configured explicitly, the plugin connects to "localhost:6379".

Host Hostname

The Host option is the hostname or IP-address where the Redis instance is running. Simple as that ๐Ÿ’œ

Port Port

The Port option is the TCP port where the Redis instance accepts connections. You can give either a service name or a port number. But please note โ€” numerical port numbers must be given as a string too ๐Ÿซถ.

Socket Path

Connect to Redis using the UNIX domain socket at Path. If this setting is given, the Hostname and Port settings are ignored, periodt.

Password Password

Use Password to authenticate when connecting to Redis ๐Ÿ”

Timeout Milliseconds

The Timeout option sets the socket timeout for node response. Since the Redis read function is blocking, you should keep this value as low as possible. It is expected what Timeout values should be lower than Interval defined globally.

Defaults to 2000 (2 seconds).

ReportCommandStats false|true

Enables or disables reporting of statistics based on the command type, including the rate of command calls and the average CPU time consumed by command processing ๐Ÿ“Š. Defaults to false.

ReportCpuUsage true|false

Enables or disables reporting of CPU consumption statistics. Defaults to true.

Query Querystring

The Query block identifies a query to execute against the redis server. There may be an arbitrary number of queries to execute, go off ๐Ÿ‘‘. Each query should return a single string or integer.

Type Collectd type

Within a query definition, a valid collectd type to use when submitting the result of the query. When it's not supplied, it'll default to gauge.

Currently only types with one datasource are supported. See types.db(5) for more details on types and their configuration.

Instance Type instance

Within a query definition, an optional type instance to use when submitting the result of the query. When not supplied it will default to the escaped command, up to 128 chars.

Database Index

This index selects the Redis logical database to use for the query. Defaults to 0.

๐Ÿ’€ Plugin "rrdcached" ๐Ÿ’€

The "rrdcached" plugin uses the RRDtool accelerator daemon, rrdcached(1), to store values to RRD files in an efficient manner โœจ The combo of the "rrdcached" plugin and the "rrdcached" daemon is very similar to how the "rrdtool" plugin works (see below). The added abstraction layer gives you a bunch of benefits though: because the cache isn't within "collectd" anymore, it doesn't need to be flushed when "collectd" is restarting. That means much shorter (if any) gaps in graphs, especially under heavy load ๐Ÿ”ฅ Also, the "rrdtool" command line utility is aware of the daemon, so it can flush values to disk automatically when needed. That lets you integrate automated flushing of values into graphing solutions way more easily.

There are disadvantages though, ngl: the daemon may reside on a different host, so it may not be possible for "collectd" to create the appropriate RRD files anymore. And even if "rrdcached" runs on the same host, it may run in a different base directory, so relative paths may do weird stuff if you're not careful ๐Ÿ˜ญ.

So the recommended configuration is to let "collectd" and "rrdcached" run on the same host, communicating via a UNIX domain socket ๐Ÿซถ The DataDir setting should be set to an absolute path, so that a changed base directory doesn't result in RRD files being created / expected in the wrong place.

DaemonAddress Address

Address of the daemon as understood by the "rrdc_connect" function of the RRD library. See rrdcached(1) for details. Example:

<Plugin "rrdcached">
  DaemonAddress "unix:/var/run/rrdcached.sock"
</Plugin>

DataDir Directory

Set the base directory where the RRD files live. If this is a relative path, it's relative to the working base directory of the "rrdcached" daemon! Use of an absolute path is recommended ๐Ÿ’…

CreateFiles true|false

Enables or disables the creation of RRD files. If the daemon isn't running locally, or DataDir is set to a relative path, this will not work as expected. Default is true.

CreateFilesAsync false|true

When enabled, new RRD files are enabled asynchronously, using a separate thread that runs in the background ๐ŸŒˆ This prevents writes from blocking, which is a problem especially when many hundreds of files need to be created at once. However โ€” since the whole point of creating files asynchronously is not to block until the file is available, values before the file is available will be discarded ๐Ÿ’€. When disabled (the default) files are created synchronously, blocking for a short while, while the file is being written.

StepSize Seconds

Force the stepsize of newly created RRD-files. Ideally (and per default) this setting is unset and the stepsize is set to the interval in which the data is collected. Do not use this option unless you absolutely have to for some reason ๐Ÿ™ˆ Setting this option may cause problems with the "snmp plugin", the "exec plugin", or when the daemon is set up to receive data from other hosts.

HeartBeat Seconds

Force the heartbeat of newly created RRD-files. This setting should be unset, in which case the heartbeat is set to twice the StepSize, which should equal the interval in which data is collected. Do not set this option unless you have a very good reason to do so.

RRARows NumRows

The "rrdtool plugin" calculates the number of PDPs per CDP based on the StepSize, this setting and a timespan. This plugin creates RRD-files with three times five RRAs, i. e. five RRAs with the CFs MIN, AVERAGE, and MAX. The five RRAs are optimized for graphs covering one hour, one day, one week, one month, and one year โญ

So for each timespan, it calculates how many PDPs need to be consolidated into one CDP by calculating:

number of PDPs = timespan / (stepsize * rrarows)

Bottom line is, set this no smaller than the width of your graphs in pixels ๐Ÿ“ The default is 1200.

RRATimespan Seconds

Adds an RRA-timespan, given in seconds. Use this option multiple times to have more then one RRA. If this option is never used, the built-in default of (3600, 86400, 604800, 2678400, 31622400) is used.

For more information on how RRA-sizes are calculated see RRARows above.

XFF Factor

Set the "XFiles Factor". The default is 0.1. If unsure, don't set this option. Factor must be in the range "[0.0-1.0)", i.e. between zero (inclusive) and one (exclusive) ๐ŸŽฏ

CollectStatistics false|true

When set to true, various statistics about the rrdcached daemon will be collected, with "rrdcached" as the plugin name. Defaults to false.

Statistics are read via rrdcacheds socket using the STATS command. See rrdcached(1) for details ๐Ÿ‘‘

๐Ÿฆ‹ Plugin "rrdtool" ๐Ÿฆ‹

You can use the settings StepSize, HeartBeat, RRARows, and XFF to fine-tune your RRD-files. Please read rrdcreate(1) if you encounter problems using these settings. If you don't want to dive into the depths of RRDtool, you can safely ignore these settings ๐Ÿ’–

DataDir Directory

Set the directory to store RRD files under. By default RRD files are generated beneath the daemon's working directory, i.e. the BaseDir.

CreateFilesAsync false|true

When enabled, new RRD files are enabled asynchronously, using a separate thread that runs in the background ๐ŸŒธ This prevents writes from blocking, which is a problem especially when many hundreds of files need to be created at once. However โ€” since the purpose of creating the files asynchronously is not to block until the file is available, values before the file is available will be discarded ๐Ÿ˜ญ. When disabled (the default) files are created synchronously, blocking for a short while, while the file is being written.

StepSize Seconds

Force the stepsize of newly created RRD-files. Ideally (and per default) this setting is unset and the stepsize is set to the interval in which the data is collected. Do not use this option unless you absolutely have to for some reason. Setting this option may cause problems with the "snmp plugin", the "exec plugin", or when the daemon is set up to receive data from other hosts ๐Ÿ’€

HeartBeat Seconds

Force the heartbeat of newly created RRD-files. This setting should be unset, in which case the heartbeat is set to twice the StepSize, which should equal the interval in which data is collected. Do not set this option unless you have a very good reason to do so ๐Ÿซถ

  • RRARows NumRows โ€” so the rrdtool plugin figures out how many PDPs go into one CDP using the StepSize, this setting, and a timespan. It builds RRD files with three times five RRAs, i.e. five RRAs using the CFs MIN, AVERAGE, and MAX. Those five RRAs are tuned for graphs covering one hour, one day, one week, one month, and one year โœจ

So for each timespan it works out how many PDPs need consolidating into one CDP like this:

number of PDPs = timespan / (stepsize * rrarows)

Bottom line, bestie: set this no smaller than the width of your graphs in pixels. The default is 1200.

  • RRATimespan Seconds โ€” adds an RRA-timespan, given in seconds. Use this option multiple times if you want more than one RRA. If you never use it, the built-in default of (3600, 86400, 604800, 2678400, 31622400) is used ๐Ÿ’ซ

For more on how RRA-sizes are calculated, look at RRARows above.

  • XFF Factor โ€” sets the "XFiles Factor". Default is 0.1. If you're unsure, don't set this one, fr. Factor must be inside the range "[0.0-1.0)", meaning between zero (inclusive) and one (exclusive) โญ
  • CacheFlush Seconds โ€” when the rrdtool plugin is using a cache (that's by setting CacheTimeout, see below) it writes all values for a given RRD file once the oldest value is older than (or equal to) the number of seconds set by CacheTimeout. That check happens on new values arriwal. If some RRD file stops being updated for whatever reason (computer got shut down, network went sideways, etc.), some values may still be chilling in the cache ๐Ÿ™ˆ If CacheFlush is set, then every Seconds seconds the entire cache gets searched for entries older than CacheTimeout + RandomTimeout seconds, and whatever it finds gets written to disk. Since scanning the whole cache is kind of expensive and does absolutely nothing under normal circumstances, don't make this value too small. 900 seconds is a solid pick, though setting it to 7200 seconds doesn't normally do much harm either.

Defaults to 10x CacheTimeout. CacheFlush must be larger than or equal to CacheTimeout, otherwise that default above is used instead.

  • CacheTimeout Seconds โ€” if you set this to anything greater than zero, the rrdtool plugin saves values in a cache, exactly like described above. Writing multiple values at once cuts down IO operations and therefore lessens the load from updating files ๐Ÿ’… The trade-off is that the graphs kind of "drag behind" and more memory gets used.
  • WritesPerSecond Updates โ€” when you're collecting a ton of statistics with collectd and the rrdtool plugin, you will run into serious performance problems. The CacheFlush setting and the internal update queue make sure collectd keeps working just fine even under heavy load, but the system may become very unresponsive and slow ๐Ÿ˜ญ This is especially a problem if you create graphs from the RRD files on the same machine, like with the graph.cgi script that ships in the contrib/collection3/ directory.

This setting is designed for very large setups. Setting it to a value between 25 and 80 updates per second, depending on your hardware, leaves the server responsive enough to draw graphs even while all the cached values are being written to disk. Flushed values โ€” i.e. values forced to disk by the FLUSH command โ€” are not effected by this limit. They still get written as fast as possible, so web frontends have up-to-date data when generating graphs ๐Ÿ”ฅ

For example: if you have 100,000 RRD files and set WritesPerSecond to 30 updates per second, writing all the values to disk takes approximately 56 minutes. Together with the flushing ability that's integrated into collection3, you end up with a responsive and fast system, up-to-date graphs, and basically a "backup" of your values every hour โœจ

  • RandomTimeout Seconds โ€” when set, the actual timeout for each value gets chosen randomly between CacheTimeout-RandomTimeout and CacheTimeout+RandomTimeout. The intention is to dodge the high-load situations that show up when a bunch of values time out at the exact same moment ๐ŸŒ€ This is especially a problem shortly after the daemon starts, because all values were added to the internal cache at roughly the same time.

๐ŸŽ€ Plugin "sensors" ๐ŸŽ€

The Sensors plugin uses lm_sensors to grab sensor-values. Which means all the needed modules have to be loaded and lm_sensors has to be configured (most likely by editing /etc/sensors.conf). Read sensors.conf(5) for details ๐Ÿ’œ

The lm_sensors homepage can be found at http://secure.netroedge.com/~lm78/.

  • SensorConfigFile File โ€” read the lm_sensors configuration from File. When unset (recommended), the library's default gets used.
  • Sensor chip-bus-address/type-feature โ€” selects the name of the sensor you want to collect or ignore, depending on the IgnoreSelected below. For example, the option Sensor it8712-isa-0290/voltage-in1 causes collectd to gather data for the voltage sensor in1 of the it8712 on the isa bus at the address 0290.

The value passed to this option has the format "plugin_instance/type-type_instance".

See /"IGNORELISTS" for details ๐Ÿซถ

  • IgnoreSelected true|false โ€” if no configuration is given, the sensors plugin collects data from all sensors. That may not be practical, especially for uninteresting sensors. So you can use the Sensor option to pick the sensors you're actually interested in. Sometimes, though, it's easier/preferred to collect everything except a few. This option lets you do that: by setting IgnoreSelected to true the effect of Sensor flips โ€” all selected sensors are ignored and all the other sensors get collected ๐Ÿ’ซ
  • UseLabels true|false โ€” configures how sensor readings get reported. When set to true, sensor readings are reported using their descriptive label (e.g. "VCore"). When set to false (the default) the sensor name is used ("in0") โœจ

๐ŸŒˆ Plugin "sigrok" ๐ŸŒˆ

The sigrok plugin uses libsigrok to retrieve measurements from any device supported by the sigrok http://sigrok.org/ project ๐Ÿ‘‘

Synopsis

<Plugin sigrok>
  LogLevel 3
  <Device "AC Voltage">
     Driver "fluke-dmm"
     MinimumInterval 10
     Conn "/dev/ttyUSB2"
  </Device>
  <Device "Sound Level">
     Driver "cem-dt-885x"
     Conn "/dev/ttyUSB1"
  </Device>
</Plugin>
  • LogLevel 0-5 โ€” the sigrok logging level to pass on to the collectd log, as a number between 0 and 5 (inclusive). These levels correspond to "None", "Errors", "Warnings", "Informational", "Debug "and "Spew", respectively. The default is 2 ("Warnings"). The sigrok log messages, regardless of their level, are always submitted to collectd at its INFO log level โญ
  • <Device Name> โ€” a sigrok-supported device, uniquely identified by this section's options. The Name is passed to collectd as the plugin instance ๐ŸŽ€
  • Driver DriverName โ€” the sigrok driver to use for this device.
  • Conn ConnectionSpec โ€” if the device can't be auto-discovered, or more than one might get discovered by the driver, ConnectionSpec specifies the connection string to the device. It can be the form of a device path (e.g. "/dev/ttyUSB2"), or, in the case of a non-serial USB-connected device, the USB VendorID.ProductID separated by a period (e.g. 0403.6001). A USB device can also be specified as Bus.Address (e.g. 1.41) ๐Ÿ’…
  • SerialComm SerialSpec โ€” for serial devices with non-standard port settings, this option lets you specify them in a form sigrok understands, e.g. "9600/8n1". This shouldn't be necessary; drivers know how to talk to the devices they support ๐Ÿฆ‹
  • MinimumInterval Seconds โ€” specifies the minimum time between measurement dispatches to collectd, in seconds. Since some sigrok-supported devices can acquire measurements many times per second, it may be necessary to throttle them. For example, the RRD plugin can't process writes more than once per second.

The default MinimumInterval is 0, meaning measurements received from the device are always dispatched to collectd. When throttled, unused measurements are discarded ๐Ÿ’€


๐ŸŒธ Plugin "slurm" ๐ŸŒธ

This plugin collects per-partition SLURM node and job state information, plus internal health statistics. It takes no options. It should run on a node that's capable of running the sinfo and squeue commands, i.e. it has a running slurmd and a valid slurm.conf. Note that this plugin needs the Globals option set to true in order to function properly ๐Ÿ‘‘


๐Ÿ’– Plugin "smart" ๐Ÿ’–

The "smart" plugin collects SMART information from physical disks. Values collectd grabs include temperature, power cycle count, poweron time and bad sectors. Also, all SMART attributes are collected along with the normalized current value, the worst value, the threshold and a human readable value. The plugin can also collect SMART attributes for NVMe disks (present in accordance with NVMe 1.4 spec) and Additional SMART Attributes from Intelยฎ NVMe disks ๐Ÿงš

Using the following two options you can ignore some disks or configure collection to only specific disks.

  • Disk Name โ€” select the disk Name. Whether it gets collected or ignored depends on the IgnoreSelected setting, see below. As with other plugins that use the daemon's ignorelist functionality, a string that starts and ends with a slash is interpreted as a regular expression. Examples:
Disk "sdd"
Disk "/hda[34]/"
Disk "nvme0n1"

See /"IGNORELISTS" for details ๐Ÿ’œ

  • IgnoreSelected true|false โ€” sets whether selected disks, i. e. the ones matched by any of the Disk statements, are ignored or if all other disks are ignored. The behavior (hopefully) is intuitive: if no Disk option is configured, all disks get collected. If at least one Disk option is given and no IgnoreSelected or set to false, only matching disks get collected. If IgnoreSelected is set to true, all disks get collected except the ones matched โœจ
  • IgnoreSleepMode true|false โ€” normally, the "smart" plugin ignores disks that are reported to be asleep. This option disables the sleep mode check and lets the plugin collect data from those disks anyway. This is useful when libatasmart mistakenly reports disks as asleep because it hasn't been updated to incorporate support for newer idle states in the ATA spec ๐ŸŒ€
  • UseSerial true|false โ€” okay so a disk's kernel name (like sda) can literally change from one boot to the next, which is chaos ๐Ÿ’€. Turn this on and the "smart" plugin uses the disk's serial number instead (like HGST_HUH728080ALE600_2EJ8VH8X) as the key for storing data, instead of the kernel name. Translation: your data for a given disk stays together even when the kernel decides to rename it. Slay.

โœจ Plugin snmp โœจ

So the "snmp plugin" config is a little more complicated than the other plugins (an understatement, honestly ๐Ÿ˜ญ), so its docs got their own whole manpage: collectd-snmp(5). Go read that one for the deets, bestie ๐Ÿซถ

๐Ÿฆ‹ Plugin snmp_agent ๐Ÿฆ‹

The snmp_agent plugin is an AgentX subagent โ€” it receives and handles queries from the SNMP master agent and hands back the data collected by read plugins. It only handles requests for the OIDs you actually specified in the config file, no cap. To answer SNMP queries, the plugin grabs data from collectd and translates the requested values from collectd's internal format into SNMP format. This one's a generic plugin and cannot work without configuration โ€” it needs you, bestie ๐Ÿ’…

For more on AgentX subagents, see http://www.net-snmp.org/tutorial/tutorial-5/toolkit/demon/

Synopsis (iconic, copy it exactly โœจ):

<Plugin snmp_agent>
  <Data "memAvailReal">
    Plugin "memory"
    #PluginInstance "some"
    Type "memory"
    TypeInstance "free"
    OIDs "1.3.6.1.4.1.2021.4.6.0"
  </Data>
  <Table "ifTable">
    IndexOID "IF-MIB::ifIndex"
    SizeOID "IF-MIB::ifNumber"
    <Data "ifDescr">
      <IndexKey>
        Source "PluginInstance"
      </IndexKey>
      Plugin "interface"
      OIDs "IF-MIB::ifDescr"
    </Data>
    <Data "ifOctets">
      Plugin "interface"
      Type "if_octets"
      TypeInstance ""
      OIDs "IF-MIB::ifInOctets" "IF-MIB::ifOutOctets"
    </Data>
  </Table>
  <Table "CPUAffinityTable">
    <Data "DomainName">
      <IndexKey>
        Source "PluginInstance"
      </IndexKey>
      Plugin "virt"
      OIDs "LIBVIRT-HYPERVISOR-MIB::lvhAffinityDomainName"
    </Data>
    <Data "VCPU">
      Plugin "virt"
      <IndexKey>
        Source "TypeInstance"
        Regex "^vcpu_([0-9]{1,3})-cpu_[0-9]{1,3}$"
        Group 1
      </IndexKey>
      OIDs "LIBVIRT-HYPERVISOR-MIB::lvhVCPUIndex"
    </Data>
    <Data "CPU">
      Plugin "virt"
      <IndexKey>
        Source "TypeInstance"
        Regex "^vcpu_[0-9]{1,3}-cpu_([0-9]{1,3})$"
        Group 1
      </IndexKey>
      OIDs "LIBVIRT-HYPERVISOR-MIB::lvhCPUIndex"
    </Data>
    <Data "CPUAffinity">
      Plugin "virt"
      Type "cpu_affinity"
      OIDs "LIBVIRT-HYPERVISOR-MIB::lvhCPUAffinity"
    </Data>
  </Table>
</Plugin>

Inside the "<Plugin snmp_agent>" block you can have two flavors of block: Data and Table. Let's get into it ๐ŸŽ€

๐ŸŒˆ Data block ๐ŸŒˆ

The Data block defines a list of OIDs that are gonna be handled. It can define scalar or table OIDs. If a Data block lives inside a Table block, it represents table OIDs. Here's what you can set:

  • IndexKey block โ€” this block holds all the data needed to build the index of an SNMP table properly. If more than one Data block in a table has an IndexKey block, then a multiple key index gets built. And if the Data block defines a scalar data type, IndexKey has literally no effect and you can just skip it ๐Ÿ™ˆ
    • Source String โ€” set this to one of: "Hostname", "Plugin", "PluginInstance", "Type", "TypeInstance". It tells the plugin which field of the corresponding collectd metric gets used as the SNMP table index.
    • Regex String โ€” you can also use this to parse strings or numbers out of a specific field. Example: a type-instance field of "vcpu1-cpu2" can be parsed into two numeric fields, CPU = 2 and VCPU = 1, and then used as a table index later. Obsessed.
    • Group Number โ€” specify the group number in case groups are used in the regex.
  • Plugin String โ€” the name of the read plugin whose collected data gets mapped to the specified OIDs.
  • PluginInstance String โ€” the read plugin instance whose collected data gets mapped to the specified OIDs. Optional, and by default there's no plugin instance check. Only allowed if the Data block defines a scalar data type.
  • Type String โ€” collectd's type used for the specified OID, e.g. "if_octets". Types are read from the TypesDB (see collectd.conf(5)).
  • TypeInstance String โ€” collectd's type-instance used for the specified OID.
  • OIDs OID [OID ...] โ€” configures the OIDs the snmp_agent plugin handles. Values for these OIDs come from the collectd data type specified by the Plugin, PluginInstance, Type and TypeInstance fields of this Data block. The number of OIDs you configure should match the number of values in the specified Type. Like, two OIDs "IF-MIB::ifInOctets" "IF-MIB::ifOutOctets" can be mapped to the "rx" and "tx" values of the "if_octets" type ๐Ÿ’–
  • Scale Value โ€” values taken from collectd get multiplied by Value. Optional, default is 1.0.
  • Shift Value โ€” Value gets added to the values from collectd after they've been multiplied by the Scale value. Optional, default is 0.0.

๐Ÿ’œ The Table block ๐Ÿ’œ

The Table block defines a collection of Data blocks that all belong to one SNMP table. On top of having multiple Data blocks, you can also set these:

  • IndexOID OID โ€” an OID handled by the plugin, mapped to the numerical index value that the plugin generates for each table record.
  • SizeOID OID โ€” an OID handled by the plugin. The returned value is the number of records in the table. Optional.

๐Ÿ“ Plugin statsd ๐Ÿ“

The statsd plugin listens on a UDP socket, reads "events" in the statsd protocol, and periodically dispatches rates or other aggregates of those numbers. It's giving little listener that could โœจ

It implements the Counter, Timer, Gauge and Set types, which get dispatched as the collectd types "derive", "latency", "gauge" and "objects" respectively.

These config options are valid:

  • Host Host โ€” bind to the hostname / address Host. By default the plugin binds to the "any" address, i.e. it accepts packets sent to any of the host's addresses.
  • Port Port โ€” the UDP port to listen to. Can be a service name or a port number. Defaults to 8125.
  • DeleteCounters false|true
  • DeleteTimers false|true
  • DeleteGauges false|true
  • DeleteSets false|true

These four control what happens if metrics don't get updated within an interval. Set to False (the default) and metrics are dispatched unchanged โ€” i.e. the rate of counters and size of sets will be zero, timers report "NaN", and gauges are unchanged. Set to True and those metrics aren't dispatched at all and get removed from the internal cache. Bye bestie ๐Ÿ‘‹

  • CounterSum false|true โ€” when enabled, creates a "count" metric which reports the change since the last read. This option primarily exists for compatibility with the statsd implementation by Etsy.
  • CounterSum false|true โ€” when enabled, creates a "gauge" metric which reports counters as a "gauge" of the differential, resetting the counter between flush intervals. This option primarily exists for compatibility with the statsd implementation in GitHub Enterprise Server.
no cap, CounterSum really is spelled twice in the original page โ€” the second one is the GitHub Enterprise Server one. Carry on ๐Ÿซถ
  • TimerPercentile Percent โ€” calculate and dispatch the configured percentile, i.e. compute the latency so that Percent of all reported timers are smaller than or equal to the computed latency. Super useful for chopping off that long tail latency, like they do in Service Level Agreements (SLAs). You can calculate different percentiles by setting this option several times. If you don't specify any, no percentiles get calculated / dispatched.
  • TimerLower false|true
  • TimerUpper false|true
  • TimerSum false|true
  • TimerCount false|true

These calculate and dispatch various values out of the Timer metrics received during an interval. Set to False (the default) and these values aren't calculated / dispatched. Quick heads up: reported timer values less than 0.001 are ignored in all Timer* reports ๐Ÿ’€

๐ŸŒธ Plugin swap ๐ŸŒธ

The Swap plugin collects info about used and available swap space. On Linux and Solaris, these options are available:

  • ReportByDevice false|true โ€” configures how physical swap devices get reported. Set to false (the default) and you only get the summary over all swap devices, i.e. the globally used and available space across all of them. Set to true and the used and available space of each device gets reported separately. This option is only available if the Swap plugin can read "/proc/swaps" (under Linux) or use the swapctl(2) mechanism (under Solaris).
  • ReportBytes false|true โ€” when enabled, swap I/O is reported in bytes. When disabled (the default), swap I/O is reported in pages. This option is available under Linux only.
  • ValuesAbsolute true|false โ€” enables or disables reporting of absolute swap metrics, i.e. number of bytes available and used. Defaults to true.
  • ValuesPercentage false|true โ€” enables or disables reporting of relative swap metrics, i.e. percent available and free. Defaults to false. This is useful for deploying collectd in a heterogeneous environment where swap sizes differ and you want to specify generic thresholds or similar. Main character deployment energy ๐Ÿ‘‘
  • ReportIO true|false โ€” enables or disables reporting swap IO. Defaults to true. Useful for when swap IO isn't necessary, isn't available, or isn't reliable.

โญ Plugin sysevent โญ

The sysevent plugin monitors rsyslog messages. That's the whole vibe.

Synopsis:

<Plugin sysevent>
  Listen "192.168.0.2" "6666"
  BufferSize 1024
  BufferLength 10
  RegexFilter "regex"
</Plugin>

rsyslog should be configured so it sends data to the IP and port you included in the plugin configuration. For example, given the configuration above, something like this would be set in /etc/rsyslog.conf:

if $programname != 'collectd' then
*.* @192.168.0.2:6666
$template ls_json,"{%timestamp:::date-rfc3339,jsonf:@timestamp%, \
%source:::jsonf:@source_host%,\"@source\":\"syslog://%fromhost-ip:::json%\", \
\"@message\":\"%timestamp% %app-name%:%msg:::json%\",\"@fields\": \
{%syslogfacility-text:::jsonf:facility%,%syslogseverity:::jsonf:severity-num%, \
%syslogseverity-text:::jsonf:severity%,%programname:::jsonf:program%, \
%procid:::jsonf:processid%}}"

if $programname != 'collectd' then
*.* @192.168.0.2:6666;ls_json

Okay so real talk: these rsyslog.conf examples are not complete, bestie ๐Ÿ˜ญ rsyslog needs way more than just these options in its config file. These are literally just here to show you the proper remote logging and JSON format syntax. Don't copy-paste them into a fresh conf and expect magic, iykyk ๐Ÿ’…

โœจ Options:

  • Listen host port โ€” listen on this IP and this port for incoming rsyslog messages ๐ŸŽง
  • BufferSize length โ€” the maximum allowed size for incoming rsyslog messages. Messages bigger than this get truncated down to this size. Default is 4096 bytes ๐Ÿ“
  • BufferLength length โ€” maximum number of rsyslog events that can live in the plugin's ring buffer. Default is 10. Once an event has been read, its slot frees up for a new event to move in ๐Ÿ 
  • RegexFilter regex โ€” hand over a regex filter to apply to all incoming rsyslog messages. If a message matches, it gets published. slay ๐Ÿ”ฅ

๐ŸŒˆ Plugin "syslog"

  • LogLevel debug|info|notice|warning|err โ€” sets the log level. So like, if you set it to notice, then every event with severity notice, warning, or err gets submitted to the syslog-daemon ๐Ÿ“จ
heads up bestie: debug is only available if collectd was compiled with debugging support. no debug build, no debug logs, periodt ๐Ÿ’€
  • NotifyLevel OKAY|WARNING|FAILURE โ€” controls which notifications get sent to syslog. Default behaviour is to send none ๐Ÿ™ˆ Less severe notifications always imply logging the more severe ones, so: set it to OKAY and all notifications get sent to syslog; set it to WARNING and you get WARNING + FAILURE notifications but OKAY ones get dismissed; set it to FAILURE and only failures make it to syslog. the way that escalation works is iconic honestly ๐Ÿ‘‘

๐ŸŽ€ Plugin "table"

The "table plugin" gives you a generic way to parse tabular data and dispatch user-specified values. Values are picked based on column numbers. For example, this plugin can be used to grab values from Linux proc(5) filesystem files, or from CSV (comma separated values) files ๐Ÿงฎ

<Plugin table>
  <Table "/proc/slabinfo">
    #Plugin "slab"
    Instance "slabinfo"
    Separator " "
    <Result>
      Type gauge
      InstancePrefix "active_objs"
      InstancesFrom 0
      ValuesFrom 1
    </Result>
    <Result>
      Type gauge
      InstancePrefix "objperslab"
      InstancesFrom 0
      ValuesFrom 4
    </Result>
  </Table>
</Plugin>

So the config is made of one or more Table blocks, and each one configures a single file to parse. Inside every Table block there's one or more Result blocks, which configure which data to select and how to interpret it ๐Ÿ’–

The following options are available inside a Table block:

  • Plugin Plugin โ€” if specified, Plugin is used as the plugin name when submitting values. Defaults to table ๐Ÿท๏ธ
  • Instance instance โ€” if specified, instance is used as the plugin instance. If you leave it out, the filename of the table gets used instead, with all special characters swapped for an underscore ("_") ๐Ÿ”„
  • Separator string โ€” any character of string is treated as a delimiter between the different columns of the table. A sequence of two or more contiguous delimiters counts as a single delimiter, i.e. there cannot be any empty columns. The plugin uses strtok_r(3) to parse the lines of a table โ€” go read its docs for the full deets. This option is mandatory โš ๏ธ
A horizontal tab, newline, and carriage return can be specified as "\\t", "\\n", and "\\r" respectively. And yes, the double backslashes are required because of collectd's config parsing โ€” sorry, that's just how it is ๐Ÿ˜ญ

The following options are available inside a Result block:

  • Type type โ€” sets the type used to dispatch values to the daemon. All the detailed info about types and their configuration lives in types.db(5). This option is mandatory โœจ
  • InstancePrefix prefix โ€” if specified, prepend prefix to the type instance. If omitted, only the InstancesFrom option is considered for the type instance ๐Ÿงต
  • InstancesFrom column0 [column1 ...] โ€” if specified, the content of the given columns (identified by the column number, starting at zero) is used to create the type instance for each row. Multiple values (plus the instance prefix) get joined together with dashes (-) as the separator. If omitted, only the InstancePrefix option is considered for the type instance ๐ŸŽฏ
ngl, the plugin itself does not check whether all the built instances are different. It's your responsibility to make sure each one is unique. That goes double if you don't specify InstancesFrom โ€” then you have to make sure the table only contains one row ๐Ÿ˜ฌ

If neither InstancePrefix nor InstancesFrom is given, the type instance will be empty. that's it, that's the vibe ๐Ÿซถ

  • ValuesFrom column0 [column1 ...] โ€” specifies the columns (identified by column numbers starting at zero) whose content is used as the actual data for the data sets dispatched to the daemon. How many of those columns you need is decided by the Type setting above. If you specify too many or not enough columns, the plugin will complain about it and no data will be submitted to the daemon ๐Ÿ’€ The plugin uses strtoll(3) and strtod(3) to parse counter and gauge values respectively, so anything those functions support is supported by the plugin too. This option is mandatory ๐Ÿ”ฅ

๐Ÿฆ‹ Plugin "tail"

The "tail plugin" follows logfiles, just like tail(1) does, parses every line, and dispatches whatever values it finds. What gets matched is up to you, configured with (extended) regular expressions, as described in regex(7) ๐Ÿ‘€

<Plugin "tail">
  <File "/var/log/exim4/mainlog">
    Plugin "mail"
    Instance "exim"
    Interval 60
    <Match>
      Regex "S=([1-9][0-9]*)"
      DSType "CounterAdd"
      Type "ipt_bytes"
      Instance "total"
    </Match>
    <Match>
      Regex "\\<R=local_user\\>"
      ExcludeRegex "\\<R=local_user\\>.*mail_spool defer"
      DSType "CounterInc"
      Type "counter"
      Instance "local_user"
    </Match>
    <Match>
      Regex "l=([0-9]*\\.[0-9]*)"
      <DSType "Distribution">
        Percentile 99
        Bucket 0 100
        #BucketType "bucket"
      </DSType>
      Type "latency"
      Instance "foo"
    </Match>
  </File>
</Plugin>

The config consists of one or more File blocks, each configuring one logfile to parse. Inside each File block there's one or more Match blocks, which configure a regular expression to search for ๐Ÿ”

The Plugin and Instance options in the File block can be used to set the plugin name and instance respectively. So in the example above, the plugin name "mail-exim" would be used ๐Ÿ’Œ

These options apply to all the Match blocks that follow, right up until the next Plugin or Instance option. That way you can extract several plugin instances out of one logfile โ€” super handy when parsing syslog and the like ๐Ÿ˜Œ

The Interval option lets you define the length of time between reads. If it's not set, the default Interval gets used โฑ๏ธ

Every Match block has these options to describe how the match should be performed:

  • Regex regex โ€” sets the regular expression used for matching against a line. The first subexpression has to match something that can be turned into a number by strtoll(3) or strtod(3), depending on the value of "CounterAdd" (see below). Because extended regular expressions are used, you do not need backslashes for subexpressions! If in doubt, please consult regex(7) ๐Ÿ“– But because of collectd's config parsing you do need to escape backslashes. So if you want to match literal parentheses, you do this:
Regex "SPAM \\(Score: (-?[0-9]+\\.[0-9]+)\\)"
  • ExcludeRegex regex โ€” sets an optional regular expression used for excluding lines from the match. Here's an example that excludes all connections from localhost from the match:
ExcludeRegex "127\\.0\\.0\\.1"
  • DSType Type โ€” sets how the values get cumulated. Type is one of these bad boys:
    • GaugeAverage โ€” calculate the average of all values matched during the interval ๐Ÿ“Š
    • GaugeMin โ€” report the smallest value matched during the interval ๐Ÿ”ฝ
    • GaugeMax โ€” report the greatest value matched during the interval ๐Ÿ”ผ
    • GaugeLast โ€” report the last value matched during the interval ๐Ÿ•
    • GaugePersist โ€” report the last matching value. The metric is not reset to "NaN" at the end of an interval. It's continuously reported until another value is matched. This is intended for cases where only state changes get reported โ€” like, say, a thermometer that only reports the temperature when it changes ๐ŸŒก๏ธ
    • CounterSet, DeriveSet, AbsoluteSet โ€” the matched number is a counter. This simply sets the internal counter to this value. Variants exist for "COUNTER", "DERIVE", and "ABSOLUTE" data sources ๐ŸŽš๏ธ
    • GaugeAdd, CounterAdd, DeriveAdd โ€” add the matched value to the internal counter. In the case of DeriveAdd, the matched number may be negative, which will effectively subtract from the internal counter โž•โž–
    • GaugeInc, CounterInc, DeriveInc โ€” increase the internal counter by one. These DSTypes are the only ones that don't use the matched subexpression โ€” they simply count the number of matched lines. So you can totally use a regular expression without a submatch here ๐Ÿงฎ
GaugeInc is reset to zero after every read, unlike the other Gauge* metrics which get reset to "NaN". don't let that catch you off guard bestie ๐Ÿ’…

Distribution ๐Ÿ’ซ

Okay bestie, so Distribution is the type for doing calculations based on the distribution of values โ€” primarily calculating percentiles. Itโ€™s giving latency tracking, but it can be used for other metrics too. The range of values tracked with this setting must be in the range (0โ€“2^34) and can be fractional. Huge caveat though: neither zero nor 2^34 are inclusive bounds, i.e. zero literally cannot be handled by a distribution. ๐Ÿ˜ญ

This option must be used together with the Percentile and/or Bucket options. No solo runs, periodt.

Synopsis:

<DSType "Distribution">
  Percentile 99
  Bucket 0 100
  BucketType "bucket"
</DSType>
  • Percentile Percent โ€” calculate and dispatch the configured percentile, i.e. compute the value so that Percent of all matched values are smaller than or equal to the computed latency. Metrics are reported with the type Type (the value of the above option) and the type instance "[<Instance>-]<Percent>". This option may be repeated to calculate more than one percentile. ๐ŸŽ€
  • Bucket lower_bound upper_bound โ€” export the number of values (a "DERIVE") falling within the given range. Both lower_bound and upper_bound may be a fractional number, such as 0.5. Each Bucket option specifies an interval "(lower_bound, upper_bound]", i.e. the range excludes the lower bound and includes the upper bound. lower_bound and upper_bound may be zero, meaning no lower/upper bound. To export the entire (0โ€“inf) range without overlap, use the upper bound of the previous range as the lower bound of the following range. In other words, use the following schema:
Bucket   0   1
Bucket   1   2
Bucket   2   5
Bucket   5  10
Bucket  10  20
Bucket  20  50
Bucket  50   0

Metrics are reported with the type set by the BucketType option ("bucket" by default) and the type instance "<Type>[-<Instance>]-<lower_bound>_<upper_bound>". This option may be repeated to calculate more than one rate. ๐Ÿ”ฅ

  • BucketType Type โ€” sets the type used to dispatch Bucket metrics. Optional, by default "bucket" will be used. ๐Ÿ‘‘

The Gauge* and Distribution types interpret the submatch as a floating point number, using strtod(3). The Counter* and AbsoluteSet types interpret the submatch as an unsigned integer using strtoull(3). The Derive* types interpret the submatch as a signed integer using strtoll(3). CounterInc, DeriveInc and GaugeInc do not use the submatch at all and it may be omitted in this case.

The Gauge* types, unless noted otherwise, are reset to "NaN" after being reported. In other words, GaugeAverage reports the average of all values matched since the last metric was reported (or "NaN" if there was no match). ๐Ÿ’€

  • Type Type โ€” sets the type used to dispatch this value. Detailed information about types and their configuration can be found in types.db(5). โœจ
  • Instance TypeInstance โ€” this optional setting sets the type instance to use. ๐Ÿฆ‹

Plugin "tail_csv" ๐Ÿ

The tail_csv plugin reads files in the CSV format, e.g. the statistics file written by Snort. Itโ€™s lowkey iconic for slurping up CSV stats.

Synopsis:

<Plugin "tail_csv">
  <Metric "snort-dropped">
      Type "percent"
      Instance "dropped"
      ValueFrom 1
  </Metric>
  <File "/var/log/snort/snort.stats">
      Plugin "snortstats"
      Instance "eth0"
      Interval 600
      Collect "snort-dropped"
      FieldSeparator ","
      #TimeFrom 0
  </File>
</Plugin>

The configuration consists of one or more Metric blocks that define an index into the line of the CSV file and how this value is mapped to collectdโ€™s internal representation. These are followed by one or more Instance blocks which configure which file to read, in which interval and which metrics to extract.

  • <Metric Name> โ€” the Metric block configures a new metric to be extracted from the statistics file and how it is mapped on collectdโ€™s data model. The string Name is only used inside the Instance blocks to refer to this block, so you can use one Metric block for multiple CSV files. ๐ŸŽ€
    • Type Type โ€” configures which Type to use when dispatching this metric. Types are defined in the types.db(5) file, see the appropriate manual page for more information on specifying types. Only types with a single data source are supported by the tail_csv plugin. The information whether the value is an absolute value (i.e. a "GAUGE") or a rate (i.e. a "DERIVE") is taken from the Typeโ€™s definition.
    • Instance TypeInstance โ€” if set, TypeInstance is used to populate the type instance field of the created value lists. Otherwise, no type instance is used.
    • ValueFrom Index โ€” configure to read the value from the field with the zero-based index Index. If the value is parsed as signed integer, unsigned integer or double depends on the Type setting, see above.
  • <File Path> โ€” each File block represents one CSV file to read. There must be at least one File block but there can be multiple if you have multiple CSV files. ๐Ÿ—‚๏ธ
    • Plugin Plugin โ€” use Plugin as the plugin name when submitting values. Defaults to "tail_csv".
    • Instance PluginInstance โ€” sets the plugin instance used when dispatching the values.
    • Collect Metric โ€” specifies which Metric to collect. This option must be specified at least once, and you can use this option multiple times to specify more than one metric to be extracted from this statistic file.
    • Interval Seconds โ€” configures the interval in which to read values from this instance / file. Defaults to the pluginโ€™s default interval.
    • TimeFrom Index โ€” rather than using the local time when dispatching a value, read the timestamp from the field with the zero-based index Index. The value is interpreted as seconds since epoch. The value is parsed as a double and may be factional. ๐Ÿ˜ญ
    • FieldSeparator Character โ€” specify the character to use as field separator while parsing the CSV. Defaults to ',' if not specified. The value can only be a single character.

Plugin "teamspeak2" ๐ŸŽง

The "teamspeak2 plugin" connects to the query port of a teamspeak2 server and polls interesting global and virtual server data. The plugin can query only one physical server but unlimited virtual servers. You can use the following options to configure it:

  • Host hostname/ip โ€” the hostname or ip which identifies the physical server. Default: 127.0.0.1
  • Port port โ€” the query port of the physical server. This needs to be a string. Default: "51234"
  • Server port โ€” this option has to be added once for every virtual server the plugin should query. If you want to query the virtual server on port 8767 this is what the option would look like:
Server "8767"

This option, although numeric, needs to be a string, i. e. you must use quotes around it! If no such statement is given only global information will be collected. ๐Ÿ’…

Plugin "ted" โšก

The TED plugin connects to a device of "The Energy Detective", a device to measure power consumption. These devices are usually connected to a serial (RS232) or USB port. The plugin opens a configured device and tries to read the current energy readings. For more information on TED, visit http://www.theenergydetective.com/.

Available configuration options:

  • Device Path โ€” path to the device on which TED is connected. collectd will need read and write permissions on that file. Default: /dev/ttyUSB0
  • Retries Num โ€” apparently reading from TED is not that reliable. You can therefore configure a number of retries here. You only configure the retries here โ€” if you specify zero, one reading will be performed (but no retries if that fails); if you specify three, a maximum of four readings are performed. Negative values are illegal. Default: 0 ๐Ÿ˜ญ

Plugin "tcpconns" ๐Ÿ”Œ

The "tcpconns plugin" counts the number of currently established TCP connections based on the local port and/or the remote port. Since there may be a lot of connections the default is to count all connections with a local port, for which a listening socket is opened. You can use the following options to fine-tune the ports you are interested in:

  • ListeningPorts true|false โ€” if this option is set to true, statistics for all local ports for which a listening socket exists are collected. The default depends on LocalPort and RemotePort (see below): If no port at all is specifically selected, the default is to collect listening ports. If specific ports (no matter if local or remote ports) are selected, this option defaults to false, i. e. only the selected ports will be collected unless this option is set to true specifically. ๐Ÿ‘‘
  • LocalPort Port โ€” count the connections to a specific local port. This can be used to see how many connections are handled by a specific daemon, e. g. the mailserver. You have to specify the port in numeric form, so for the mailserver example youโ€™d need to set 25.
  • RemotePort Port โ€” count the connections to a specific remote port. This is useful to see how much a remote service is used. This is most useful if you want to know how many connections a local service has opened to remote services, e. g. how many connections a mail server or news server has to other mail or news servers, or how many connections a web proxy holds to web servers. You have to give the port in numeric form.
  • AllPortsSummary true|false โ€” if this option is set to true a summary of statistics from all connections are collected. This option defaults to false.

Plugin "thermal" ๐ŸŒก๏ธ

  • ForceUseProcfs true|false โ€” by default, the Thermal plugin tries to read the statistics from the Linux "sysfs" interface. If that is not available, the plugin falls back to the "procfs" interface. By setting this option to true, you can force the plugin to use the latter. This option defaults to false.
  • Device Device โ€” picks the name of the thermal device you wanna collect (or ignore, depends on IgnoreSelected, bestie ๐Ÿ‘€). You can drop this option multiple times to build a whole list of devices.

Peep /"IGNORELISTS" for the full deets.

  • IgnoreSelected true|false โ€” flips the whole selection vibe ๐Ÿ”„ if you set it to true, everything gets collected except the devices you named with Device. By default, if you made a selection, only the selected devices get collected. And if you configured no selection at all? Every device is selected, no cap. ๐Ÿ’…

Plugin "threshold" ๐ŸŒก๏ธ

The Threshold plugin is the vibe-checker: it takes every value collectd collects or receives, compares it against a threshold you configure, and fires off notifications if a value goes out of bounds. Basically it's the friend who yells "girl are you okay??" when the numbers look off. ๐Ÿ˜ญ

The full docs for this plugin live in the collectd-threshold(5) manual page โ€” go read that one, iykyk.

Plugin "tokyotyrant" ๐Ÿœ

The TokyoTyrant plugin connects to a TokyoTyrant server and collects a couple of metrics: the number of records, and the database size on disk. Two metrics, minimal, iconic.

  • Host Hostname/IP โ€” the hostname or IP that identifies the server. Default: 127.0.0.1
  • Port Service/Port โ€” the query port of the server. Heads up: this needs to be a string, even if you're writing the port in its numeric form. Default: 1978

Plugin "turbostat" โšก

The Turbostat plugin reads CPU frequency and C-state residency on modern Intel processors by hitting up Model Specific Registers. Very "I talk directly to the silicon" of it. ๐Ÿ”ฅ

  • CoreCstates Bitmask(Integer) โ€” a bit mask of the list of core C-states your processor supports. Only use this if the automated detection fails, fr. The default value is extracted from the CPU model and family.

Currently supported C-states (by this plugin): 3, 6, 7

Example:

All states (3, 6 and 7):
(1<<3) + (1<<6) + (1<<7) = 392
  • PackageCstates Bitmask(Integer) โ€” a bit mask of the list of package C-states your processor supports. Again: only bust this out if automated detection fails. Default value comes from the CPU model and family.

Currently supported C-states (by this plugin): 2, 3, 6, 7, 8, 9, 10

Example:

States 2, 3, 6 and 7:
(1<<2) + (1<<3) + (1<<6) + (1<<7) = 396
  • SystemManagementInterrupt true|false โ€” boolean that turns on collection of the I/O System-Management Interrupt counter. Only use this if automated detection fails, or if you actively wanna disable the feature. ๐Ÿšซ
  • DigitalTemperatureSensor true|false โ€” boolean that turns on collection of the temperature of each core. Same deal: only use if automated detection fails, or if you wanna switch it off.
  • TCCActivationTemp Temperature โ€” the Thermal Control Circuit Activation Temperature of the installed CPU. This is the temp used when collecting the temperature of cores or packages. Only use it if automated detection fails; the default value is extracted from MSR_IA32_TEMPERATURE_TARGET. ๐ŸŒก๏ธ
  • RunningAveragePowerLimit Bitmask(Integer) โ€” a bit mask of the list of elements to be thermally monitored. Only use if automated detection fails, or if you wanna disable some collections. The bits this plugin accepts are:
    • 0 ('1'): Package
    • 1 ('2'): DRAM
    • 2 ('4'): Cores
    • 3 ('8'): Embedded graphic device
  • LogicalCoreNames true|false โ€” boolean that turns on logical core numbering for per-core stats. When it's enabled, "cpu<n>" gets used as the plugin instance, where n is a dynamic number the kernel hands out. Otherwise it's "core<n>" if there's only one package, and "pkg<n>-core<m>" if there's more than one, where n is the n-th core of package m.
  • RestoreAffinityPolicy AllCPUs|Restore โ€” okay so reading data from the CPU has a side-effect: collectd's process CPU affinity mask changes. ๐Ÿ˜ฌ Once the reading is done, that affinity mask needs restoring. This option lets you pick the restore policy.

AllCPUs (the default): restore the affinity by setting affinity to any/all CPUs.

Restore: save the affinity using sched_getaffinity() before reading data, then restore it after.

On some systems sched_getaffinity() will fail because the CPU set size doesn't line up between userspace and kernel. When that happens the plugin detects the unsuccessful call and fails with an error, which prevents data collection entirely. Most configs don't need to save affinity anyway, since the collectd process is allowed to run on any/all available CPUs. ๐Ÿซถ

If you genuinely need to save and restore affinity and you're getting errors like 'Unable to save the CPU affinity', setting the 'possible_cpus' kernel boot option might help too.

See these links for the details:

<https://github.com/collectd/collectd/issues/1593> <https://sourceware.org/bugzilla/show_bug.cgi?id=15630> <https://bugzilla.kernel.org/show_bug.cgi?id=151821>

Plugin "ubi" ๐Ÿงฑ

The Ubi plugin collects some stats about the UBI (Unsorted Block Image). The values it collects are the number of bad physical eraseblocks on the underlying MTD (Memory Technology Device), plus the maximum erase counter value concerning one volume. Niche? Yes. Iconic? Also yes. ๐Ÿ‘‘

See these links for the details:

<http://www.linux-mtd.infradead.org/doc/ubi.html> <http://www.linux-mtd.infradead.org/doc/ubifs.html> <https://www.kernel.org/doc/Documentation/ABI/stable/sysfs-class-ubi>

  • Device Name โ€” select the device Name of the UBI volume. Whether it gets collected or ignored depends on the IgnoreSelected setting, see below. ๐Ÿ‘‡

Peep /"IGNORELISTS" for the deets.

  • IgnoreSelected true|false โ€” sets whether the selected devices (i.e. the ones matched by any of the Device statements) get ignored, or if all the other devices get ignored instead. Let's break it down, bestie:
    • If no Device option is configured, all devices are collected.
    • If at least one Device is given and IgnoreSelected is absent or set to false, only matching disks get collected.
    • If IgnoreSelected is set to true, all devices get collected except the ones matched.

Periodt. ๐Ÿ’…

Plugin "unixsock" ๐Ÿงฆ

  • SocketFile Path โ€” sets the socket-file that's gonna get created.
  • SocketGroup Group โ€” if running as root, this changes the group of the UNIX-socket after it's been created. Defaults to collectd.
  • SocketPerms Permissions โ€” changes the file permissions of the UNIX-socket after creation. The permissions must be given as a numeric, octal value exactly like you'd pass to chmod(1). Defaults to 0770. ๐Ÿ”
  • DeleteSocket false|true โ€” if set to true, it deletes the socket file before calling bind(2), in case a file with the given name already exists. Picture this: collectd crashes, a socket file gets left behind, and now the daemon can't open a new socket when it restarts. ๐Ÿ˜ญ Since this is potentially dangerous, it defaults to false.
โš ๏ธ Fr fr, keep DeleteSocket at false unless you know what you're doing โ€” deleting files before bind(2) is genuinely risky territory.

Plugin "uuid" ๐Ÿชช

This plugin, if loaded, causes the Hostname to be taken from the machine's UUID. The UUID is a universally unique designation for the machine, usually grabbed from the machine's BIOS. It's most useful when the machine runs in a virtual environment like Xen, in which case the UUID is preserved across shutdowns and migration. Slay, persistent identity. โœจ

These are the methods used to find the machine's UUID, in order:

  1. Check /etc/uuid (or UUIDFile).
  2. Check for UUID from HAL (<http://www.freedesktop.org/wiki/Software/hal>) if present.
  3. Check for UUID from "dmidecode" / SMBIOS.
  4. Check for UUID from Xen hypervisor.

If no UUID can be found then the hostname is not modified. (Sorry babe, no ID, no change. ๐Ÿ™ˆ)

  • UUIDFile Path โ€” takes the UUID from the given file (default /etc/uuid).

Plugin "varnish" ๐ŸŒˆ

The varnish plugin collects info about Varnish, an HTTP accelerator. It grabs a subset of the values varnishstat(1) displays and organizes them into categories you can enable or disable. Currently only the metrics shown in varnishstat(1)'s MAIN section get collected. The exact meaning of each metric lives in varnish-counters(7). Go read that if you want the lore. ๐Ÿ“–

Synopsis:

<Plugin "varnish">
  <Instance "example">
    CollectBackend     true
    CollectBan         false
    CollectCache       true
    CollectConnections true
    CollectDirectorDNS false
    CollectESI         false
    CollectFetch       false
    CollectHCB         false
    CollectObjects     false
    CollectPurge       false
    CollectSession     false
    CollectSHM         true
    CollectSMA         false
    CollectSMS         false
    CollectSM          false
    CollectStruct      false
    CollectTotals      false
    CollectUptime      false
    CollectVCL         false
    CollectVSM         false
    CollectWorkers     false
    CollectLock        false
    CollectMempool     false
    CollectManagement  false
    CollectSMF         false
    CollectVBE         false
    CollectMSE         false
  </Instance>
</Plugin>

The config consists of one or more <Instance Name> blocks. Name is the parameter passed to "varnishd -n". If you leave it empty, it'll collectd statistics from the default "varnishd" instance โ€” and honestly that should work fine in most cases. ๐Ÿซถ

Inside each <Instance> block, these options are recognized:

  • CollectBackend true|false โ€” back-end connection stats, like successful, reused, and closed connections. true by default. โœ…
  • CollectBan true|false โ€” stats about ban operations, like number of bans added, retired, and number of objects tested against ban operations. Only available with Varnish 3.x and above. false by default.
  • CollectCache true|false โ€” cache hits and misses. true by default. โœ…
  • CollectConnections true|false โ€” number of client connections received, accepted and dropped. true by default. โœ…
  • CollectDirectorDNS true|false โ€” DNS director lookup cache stats. Only available with Varnish 3.x. false by default.
  • CollectESI true|false โ€” Edge Side Includes (ESI) parse stats. false by default.
  • CollectFetch true|false โ€” stats about fetches (HTTP requests sent to the backend). false by default.
  • CollectHCB true|false โ€” inserts and look-ups in the crit bit tree based hash. Look-ups get split into locked and unlocked ones. false by default.
  • CollectObjects true|false โ€” stats on cached objects: number of objects expired, nuked (prematurely expired), saved, moved, etc. false by default. ๐Ÿ’€
  • CollectPurge true|false โ€” stats about purge operations: how many purges got added, how many retired, how many objects got tested against purge ops. Only a Varnish 2.x thing. Defaults to false.
  • CollectSession true|false โ€” client session stats. Past + current session counts, session herd and linger counters, that whole vibe. Defaults to false. Heads up: if you're on Varnish 4.x, some metrics that used to live in the Connections and Threads sections in older Varnish versions have moved here. Moving day, ig. ๐Ÿ“ฆ
  • CollectSHM true|false โ€” stats about the shared memory log, aka that memory region where log messages hang out until it fills up and gets flushed to disk. Defaults to true (yes bestie, an actual true default, cherish it).
  • CollectSMA true|false โ€” malloc or umem (that's umem_alloc(3MALLOC)-based) storage statistics. The umem storage component is Solaris-specific, very niche of it. Note: SMA, SMF and MSE share counters, so enable only the one your Varnish instance actually uses. Available with Varnish 2.x, varnish 4.x and above (Not available in varnish 3.x). Defaults to false.
  • CollectSMS true|false โ€” synth (synthetic content) storage statistics. This storage component is used internally only. Defaults to false.
  • CollectSM true|false โ€” file (memory mapped file) storage statistics. Only available with Varnish 2.x โ€” on varnish 4.x and above use CollectSMF instead. Defaults to false.
  • CollectStruct true|false โ€” current varnish internal state statistics. Number of current sessions, objects in the cache store, open connections to backends (with Varnish 2.x), etc. Defaults to false.
  • CollectTotals true|false โ€” the overview counters: number of sessions created, number of requests, bytes transferred. Defaults to false.
  • CollectUptime true|false โ€” Varnish uptime. Only available with Varnish 3.x and above. Defaults to false.
  • CollectVCL true|false โ€” number of total (available + discarded) VCL (config files). Defaults to false.
  • CollectVSM true|false โ€” stats about Varnish's shared memory usage (the logging and statistics subsystems use it). Only available with Varnish 4.x. Defaults to false.
  • CollectWorkers true|false โ€” stats about worker threads. Defaults to false.
  • CollectVBE true|false โ€” backend counters. Only available with Varnish 4.x and above. Defaults to false.
  • CollectSMF true|false โ€” file (memory mapped file) storage statistics. Only available with Varnish 4.x and above. Note: SMA, SMF and MSE share counters, so enable only the one used by the Varnish instance. Used to be called SM in Varnish 2.x, glow-up era. Defaults to false.
  • CollectManagement true|false โ€” management process counters. Only available with Varnish 4.x and above. Defaults to false.
  • CollectLock true|false โ€” lock counters. Only available with Varnish 4.x and above. Defaults to false.
  • CollectMempool true|false โ€” memory pool counters. Only available with Varnish 4.x and above. Defaults to false.
  • CollectMSE true|false โ€” Varnish Massive Storage Engine 2.0 (MSE2) is an improved storage backend for Varnish, replacing the traditional malloc and file storages. Only available with Varnish-Plus 4.x and above. Note: SMA, SMF and MSE share counters, enable only the one used by the Varnish instance. Defaults to false.
  • CollectGOTO true|false โ€” vmod-goto counters. Only available with Varnish Plus 6.x. Defaults to false.

๐Ÿงš Plugin "virt"

Okay this one is genuinely iconic. This plugin lets you collect CPU, disk, network load and other metrics for virtualized guests on the machine. Everything comes through the libvirt API (http://libvirt.org/). The slay part? The majority of metrics can be gathered without installing any additional software on the guests โ€” especially collectd, which runs only on the host system. Host-side only, no guest chores, we love to see it. โœจ

Only Connection is required. That's it. One thing. Required.

Consider the following example config:

<Plugin "virt">
  Connection "qemu:///system"
  HostnameFormat "hostname"
  InterfaceFormat "address"
  PluginInstanceFormat "name"
</Plugin>

It will generate the following values:

node42.example.com/virt-instance-0006f26c/disk_octets-vda
node42.example.com/virt-instance-0006f26c/disk_ops-vda
node42.example.com/virt-instance-0006f26c/if_dropped-ca:fe:ca:fe:ca:fe
node42.example.com/virt-instance-0006f26c/if_errors-ca:fe:ca:fe:ca:fe
node42.example.com/virt-instance-0006f26c/if_octets-ca:fe:ca:fe:ca:fe
node42.example.com/virt-instance-0006f26c/if_packets-ca:fe:ca:fe:ca:fe
node42.example.com/virt-instance-0006f26c/memory-actual_balloon
node42.example.com/virt-instance-0006f26c/memory-available
node42.example.com/virt-instance-0006f26c/memory-last_update
node42.example.com/virt-instance-0006f26c/memory-major_fault
node42.example.com/virt-instance-0006f26c/memory-minor_fault
node42.example.com/virt-instance-0006f26c/memory-rss
node42.example.com/virt-instance-0006f26c/memory-swap_in
node42.example.com/virt-instance-0006f26c/memory-swap_out
node42.example.com/virt-instance-0006f26c/memory-total
node42.example.com/virt-instance-0006f26c/memory-unused
node42.example.com/virt-instance-0006f26c/memory-usable
node42.example.com/virt-instance-0006f26c/virt_cpu_total
node42.example.com/virt-instance-0006f26c/virt_vcpu-0

You can get info on the metric's units from the online libvirt documentation. For instance, virt_cpu_total is in nanoseconds. (nanoseconds!! tiny!! ๐Ÿซถ)

Connection uri

Connect to the hypervisor given by uri. For example if using Xen use:

Connection "xen:///"

Details on which URIs are allowed live at http://libvirt.org/uri.html.

RefreshInterval seconds

Refresh the list of domains and devices every seconds. The default is 60 seconds. Setting this to be the same or smaller than the Interval will cause the list of domains and devices to be refreshed on every iteration.

Fair warning: refreshing the devices in particular is quite a costly operation, so if your virtualization setup is static you might consider increasing this. If this option is set to 0, refreshing is disabled completely.

Domain name ยท BlockDevice name:dev ยท InterfaceDevice name:dev ยท IgnoreSelected true|false

These four hold hands โ€” they decide which domains and devices get collected. ๐ŸŽ€

If IgnoreSelected is not given or false, then only the listed domains and disk/network devices are collected.

If IgnoreSelected is true, then the test is reversed: the listed domains and disk/network devices are ignored, while the rest are collected. Flip era.

The domain name and device names may use a regular expression, if the name is surrounded by /.../ and collectd was compiled with support for regexps.

The default is to collect statistics for all domains and all their devices. Everything, everywhere, all at once.

Note: BlockDevice and InterfaceDevice options are related to corresponding *Format options. Specifically, BlockDevice filtering depends on the BlockDeviceFormat setting โ€” if the user wants to filter block devices by 'target' name, then BlockDeviceFormat has to be set to 'target' and BlockDevice must be set to a valid block device target name ("/:hdb/"). Mixing formats and filter values from different worlds (i.e., using 'target' name as a BlockDevice value with BlockDeviceFormat set to 'source') may lead to unexpected results (all devices filtered out or all visible, depending on the value of the IgnoreSelected option). Similarly, InterfaceDevice is related to the InterfaceFormat setting (i.e., when the user wants to use MAC address as a filter then InterfaceFormat has to be set to 'address' โ€” using the wrong type here may filter out all of the interfaces). Do not mix your universes, bestie. ๐Ÿ’€

Example 1:

Ignore all hdb devices on any domain, but other block devices (eg. hda) will be collected:

BlockDevice "/:hdb/"
IgnoreSelected "true"
BlockDeviceFormat "target"

Example 2:

Collect metrics only for block device on 'baremetal0' domain when its 'source' matches given path:

BlockDevice "baremetal0:/var/lib/libvirt/images/baremetal0.qcow2"
BlockDeviceFormat source

As you can see, it's possible to filter devices/interfaces using various formats โ€” for block devices, 'target' or 'source' name can be used. Interfaces can be filtered using 'name', 'address' or 'number'. Pick your fighter. ๐Ÿ‘‘

Example 3:

Collect metrics only for domains 'baremetal0' and 'baremetal1' and ignore any other domain:

Domain "baremetal0"
Domain "baremetal1"

It is possible to filter multiple block devices/domains/interfaces by adding multiple filtering entries in separate lines. Stack 'em. ๐Ÿฅž

BlockDeviceFormat target|source

If BlockDeviceFormat is set to target, the default, then the device name seen by the guest will be used for reporting metrics. This corresponds to the "<target>" node in the XML definition of the domain.

If BlockDeviceFormat is set to source, then metrics will be reported using the path of the source, e.g. an image file. This corresponds to the "<source>" node in the XML definition of the domain.

Example:

If the domain XML has the following device defined:

<disk type='block' device='disk'>
  <driver name='qemu' type='raw' cache='none' io='native' discard='unmap'/>
  <source dev='/var/lib/libvirt/images/image1.qcow2'/>
  <target dev='sda' bus='scsi'/>
  <boot order='2'/>
  <address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>

Setting "BlockDeviceFormat target" will cause the type instance to be set to "sda". Setting "BlockDeviceFormat source" will cause the type instance to be set to "var_lib_libvirt_images_image1.qcow2".

Note: this option determines also what field will be used for filtering over block devices (the filter value in BlockDevice will be applied to target or source). More info about filtering block devices can be found in the description of BlockDevice. ๐Ÿฆ‹

BlockDeviceFormatBasename false|true

BlockDeviceFormatBasename controls whether the full path or the basename(1) of the source is being used as the type instance when BlockDeviceFormat is set to source. Defaults to false.

Example:

Assume the device path (source tag) is "/var/lib/libvirt/images/image1.qcow2". Setting "BlockDeviceFormatBasename false" will cause the type instance to be set to "var_lib_libvirt_images_image1.qcow2". Setting "BlockDeviceFormatBasename true" will cause the type instance to be set to "image1.qcow2". Short and cute vs. long and thorough โ€” your call. ๐ŸŒธ

  • HostnameFormat name|uuid|hostname|metadata... โ€” okay so when the virt plugin logs data, this setting decides what it sets as the hostname of the collected data ๐Ÿ’…. The default is to use the guest name as provided by the hypervisor, which is literally the same as setting name.

Setting it to uuid means use the guest's UUID โ€” super useful if you wanna track the same guest across migrations โœจ.

hostname means to use the global Hostname setting, which is probably not useful on its own because all guests will appear to have the same name ๐Ÿ˜ญ. It IS useful in conjunction with PluginInstanceFormat though.

metadata means use information from the guest's metadata. Use HostnameMetadataNS and HostnameMetadataXPath to localize this information.

You can also specify combinations of these fields, bestie! For example name uuid means to concatenate the guest name and UUID (with a literal colon character between, thus "foo:1234-1234-1234-1234") ๐Ÿซถ.

At the moment of writing (collectd-5.5), hostname string is limited to 62 characters. In case when combination of fields exceeds 62 characters, hostname will be truncated without a warning ๐Ÿ’€.
  • InterfaceFormat name|address|number โ€” when the virt plugin logs interface data, it sets the name of the collected data according to this setting. The default is to use the path as provided by the hypervisor (the "dev" property of the target node), which is equal to setting name.

address means use the interface's mac address โ€” this is iconic because the interface path might change between reboots of a guest or across migrations ๐Ÿฆ‹.

number means use the interface's number in guest.

Note: this option determines also what field will be used for filtering over interface device (filter value in InterfaceDevice will be applied to name, address or number). More info about filtering interfaces can be found in the description of InterfaceDevice.
  • PluginInstanceFormat name|uuid|metadata|none โ€” when the virt plugin logs data, it sets the plugin_instance of the collected data according to this setting. The default is to not set the plugin_instance.

name means use the guest's name as provided by the hypervisor. uuid means use the guest's UUID. metadata means use information from guest's metadata.

You can also specify combinations of the name and uuid fields. For example name uuid means to concatenate the guest name and UUID (with a literal colon character between, thus "foo:1234-1234-1234-1234") ๐Ÿ’–.

  • HostnameMetadataNS string โ€” when metadata is used in HostnameFormat or PluginInstanceFormat, this selects in which metadata namespace we will pick the hostname. The default is http://openstack.org/xmlns/libvirt/nova/1.0 โœจ.
  • HostnameMetadataXPath string โ€” when metadata is used in HostnameFormat or PluginInstanceFormat, this describes where the hostname is located in the libvirt metadata. The default is /instance/name/text() ๐ŸŽ€.
  • ReportBlockDevices true|false โ€” enabled by default. Allows to disable stats reporting of block devices for whole plugin ๐Ÿ‘‘
  • ReportNetworkInterfaces true|false โ€” enabled by default. Allows to disable stats reporting of network interfaces for whole plugin ๐ŸŒธ.
  • ExtraStats string โ€” report additional extra statistics. The default is no extra statistics, preserving the previous behaviour of the plugin. If unsure, leave the default โ€” fr fr. If enabled, allows the plugin to reported more detailed statistics about the behaviour of Virtual Machines. The argument is a space-separated list of selectors โœจ.

Currently supported selectors are:

  • cpu_util: report CPU utilization per domain in percentage ๐Ÿ”ฅ.
  • disk: report extra statistics like number of flush operations and total service time for read, write and flush operations. Requires libvirt API version 0.9.5 or later.
  • disk_err: report disk errors if any occured. Requires libvirt API version 0.9.10 or later.
  • domain_state: report domain state and reason as 'domain_state' metric.
  • fs_info: report file system information as a notification. Requires libvirt API version 1.2.11 or later. Can be collected only if Guest Agent is installed and configured inside VM. Make sure that installed Guest Agent version supports retrieving file system information ๐Ÿ‘€.
  • job_stats_background: report statistics about progress of a background job on a domain. Only one type of job statistics can be collected at the same time. Requires libvirt API version 1.2.9 or later.
  • job_stats_completed: report statistics about a recently completed job on a domain. Only one type of job statistics can be collected at the same time. Requires libvirt API version 1.2.9 or later.
  • memory: report statistics about memory usage details, provided by libvirt virDomainMemoryStats() function.
  • pcpu: report the physical user/system cpu time consumed by the hypervisor, per-vm. Requires libvirt API version 0.9.11 or later.
  • perf: report performance monitoring events. To collect performance metrics they must be enabled for domain and supported by the platform. Requires libvirt API version 1.3.3 or later. Note: perf metrics can't be collected if intel_rdt plugin is enabled ๐Ÿ˜ฌ.
  • vcpu: report domain virtual CPUs utilisation.
  • vcpupin: report pinning of domain VCPUs to host physical CPUs.
  • disk_physical: report 'disk_physical' statistic for disk device. Note: This statistic is only reported for disk devices with 'source' property available.
  • disk_allocation: report 'disk_allocation' statistic for disk device. Note: This statistic is only reported for disk devices with 'source' property available.
  • disk_capacity: report 'disk_capacity' statistic for disk device. Note: This statistic is only reported for disk devices with 'source' property available.
  • PersistentNotification true|false โ€” override default configuration to only send notifications when there is a change in the lifecycle state of a domain. When set to true notifications will be sent for every read cycle. Default is false. Does not affect the stats being dispatched ๐ŸŒˆ.
  • Instances integer โ€” how many read instances you want to use for this plugin. The default is one, and the sensible setting is a multiple of the ReadThreads value.

This option is only useful when domains are specially tagged ๐Ÿท๏ธ. If you are not sure, just use the default setting.

The reader instance will only query the domains with attached matching tag. Tags should have the form of 'virt-X' where X is the reader instance number, starting from 0.

The special-purpose reader instance #0, guaranteed to be always present, will query all the domains with missing or unrecognized tag, so no domain will ever be left out ๐Ÿซถ.

Domain tagging is done with a custom attribute in the libvirt domain metadata section. Value is selected by an XPath /domain/metadata/ovirtmap/tag/text() expression in the http://ovirt.org/ovirtmap/tag/1.0 namespace. (XPath and namespace values are not configurable yet ๐Ÿ˜ญ).

Tagging could be used by management applications to evenly spread the load among the reader threads, or to pin on the same threads all the libvirt domains which use the same shared storage, to minimize the disruption in presence of storage outages. Slay ๐Ÿ’….


โœจ Plugin "vmem" โœจ

The "vmem" plugin collects information about the usage of virtual memory. Since the statistics provided by the Linux kernel are very detailed, they are collected very detailed. However, to get all the details, you have to switch them on manually. Most people just want an overview over, such as the number of pages read from swap space ๐Ÿฆ‹.

  • Verbose true|false โ€” enables verbose collection of information. This will start collecting page "actions", e. g. page allocations, (de)activations, steals and so on. Part of these statistics are collected on a "per zone" basis ๐Ÿ‘‘.

๐ŸŒธ Plugin "vserver" ๐ŸŒธ

This plugin doesn't have any options. VServer support is only available for Linux. It cannot yet be found in a vanilla kernel, though. To make use of this plugin you need a kernel that has VServer support built in, i. e. you need to apply the patches and compile your own kernel, which will then provide the /proc/virtual filesystem that is required by this plugin ๐Ÿ˜ญ.

The VServer homepage can be found at http://linux-vserver.org/ โœจ.

Note: The traffic collected by this plugin accounts for the amount of traffic passing a socket which might be a lot less than the actual on-wire traffic (e. g. due to headers and retransmission). If you want to collect on-wire traffic you could, for example, use the logging facilities of iptables to feed data for the guest IPs into the iptables plugin ๐Ÿ’€.

๐Ÿ’œ Plugin "write_graphite" ๐Ÿ’œ

The "write_graphite" plugin writes data to Graphite, an open-source metrics storage and graphing project. The plugin connects to Carbon, the data layer of Graphite, via TCP or UDP and sends data via the "line based" protocol (per default using port 2003). The data will be sent in blocks of at most 1428 bytes to minimize the number of network packets ๐Ÿ”ฅ.

Synopsis:

<Plugin write_graphite>
  <Node "example">
    Host "localhost"
    Port "2003"
    Protocol "tcp"
    LogSendErrors true
    Prefix "collectd"
    UseTags false
    ReverseHost false
  </Node>
</Plugin>

The configuration consists of one or more <Node Name> blocks. Inside the Node blocks, the following options are recognized:

  • Host Address โ€” hostname or address to connect to. Defaults to "localhost".
  • Port Service โ€” service name or port number to connect to. Defaults to 2003.
  • Protocol String โ€” protocol to use when connecting to Graphite. Defaults to "tcp".
  • ReconnectInterval Seconds โ€” when set to non-zero, forces the connection to the Graphite backend to be closed and re-opend periodically. This behavior is desirable in environments where the connection to the Graphite backend is done through load balancers, for example. When set to zero, the default, the connetion is kept open for as long as possible ๐Ÿซถ.
  • LogSendErrors false|true โ€” if set to true (the default), logs errors when sending data to Graphite. If set to false, it will not log the errors. This is especially useful when using Protocol UDP since many times we want to use the "fire-and-forget" approach and logging errors fills syslog with unneeded messages ๐Ÿ˜Œ.

๐ŸŒธ Prefix String ๐ŸŒธ

When UseTags is false, the Prefix value gets dropped right in front of the host name. When UseTags is true, Prefix slides in front of the series name instead.

Fr fr though: dots and whitespace are not escaped in this string โ€” peep EscapeCharacter below for that whole situation ๐Ÿ’…

๐ŸŒธ Postfix String ๐ŸŒธ

When UseTags is false, the Postfix value gets appended to the host name. When UseTags is true, the Postgix value gets appended to the end of the series name โ€” and that's before the first ; that separates the name from the tags, iykyk.

Same deal: dots and whitespace are not escaped in this string (again, see EscapeCharacter below) ๐Ÿซถ

๐ŸŒธ EscapeCharacter Char ๐ŸŒธ

Carbon uses the dot (.) as its escape character and straight up does not allow whitespace in the identifier. So the EscapeCharacter option is the vibe check that decides what character dots, whitespace and control characters get replaced with. Defaults to underscore (_).

๐ŸŒธ StoreRates false|true ๐ŸŒธ

Set it to true (which is the default, bestie) and counter values get converted to rates โœจ. Set it to false and counter values are stored as is, i.e. as an increasing integer number. That's it, that's the whole thing.

๐ŸŒธ SeparateInstances false|true ๐ŸŒธ

If you set this to true, the plugin instance and type instance each get their own path component โ€” like host.cpu.0.cpu.idle, gorge. If you set it to false (the default), then the plugin and plugin instance (and likewise the type and type instance) get mushed into one component, like host.cpu-0.cpu-idle.

Heads up: this option's value is not used when UseTags is true. Not me warning you twice. ๐Ÿ’€

๐ŸŒธ AlwaysAppendDS false|true ๐ŸŒธ

Set to true and the name of the Data Source (DS) gets appended to the metric identifier. Set to false (the default) and this only happens when there's more than one DS. Lowkey sensible tbh.

๐ŸŒธ PreserveSeparator false|true ๐ŸŒธ

If false (the default!), the . (dot) character gets replaced with EscapeCharacter. But if you set it to true, the . (dot) character is preserved โ€” i.e. passed through, untouched, main character energy โญ

Again: this option's value is not used when UseTags is true. ๐Ÿ™ˆ

๐ŸŒธ DropDuplicateFields false|true ๐ŸŒธ

Set to true and it detects + removes duplicate components in Graphite metric names. Example time: the metric name host.load.load.shortterm gets shortened to host.load.shortterm. Slay, less clutter.

๐ŸŒธ UseTags false|true ๐ŸŒธ

Set to true and Graphite metric names get generated as tagged series. This is so much more flexible than the traditional hierarchical layout, no cap ๐Ÿ’…

Example:

test.single;host=example.com;plugin=test;plugin_instance=foo;type=single;type_instance=bar

You can use the Postfix option to add more tags by specifying it like ;tag1=value1;tag2=value2. Note that tagging support was added since Graphite version 1.1.x ๐Ÿ“Œ

If set to true, the SeparateInstances and PreserveSeparator settings just don't get used.

Default value: false.

๐ŸŒธ ReverseHost false|true ๐ŸŒธ

Set to true and the (dot separated) parts of the host field of the value list get rewritten in reverse order ๐Ÿ”„. The rewrite happens before special characters are replaced with the EscapeCharacter.

This option might be convenient if the metrics are presented with Graphite in a DNS like tree structure (probably without replacing dots in hostnames).

Example:

 Hostname "node3.cluster1.example.com"
 LoadPlugin "cpu"
 LoadPlugin "write_graphite"
 <Plugin "write_graphite">
  <Node "graphite.example.com">
   EscapeCharacter "."
   ReverseHost true
  </Node>
 </Plugin>

 result on the wire: com.example.cluster1.node3.cpu-0.cpu-idle 99.900993 1543010932

Default value: false.


๐Ÿฆ‹ Plugin "write_log" ๐Ÿฆ‹

The write_log plugin writes metrics as INFO log messages. It supports two output formats: Graphite and JSON. Two whole formats, we love options ๐ŸŽ€

Synopsis:

<Plugin write_log>
  Format Graphite
</Plugin>

๐Ÿ’œ Format Format ๐Ÿ’œ

The output format to use. Can be one of Graphite or JSON. Pick your fighter.


๐Ÿฆ‹ Plugin "write_tsdb" ๐Ÿฆ‹

The write_tsdb plugin writes data to OpenTSDB โ€” a scalable open-source time series database, iconic behavior honestly ๐Ÿ‘‘. The plugin connects to a TSD, a leaderless, no shared state daemon that ingests metrics and stores them in HBase. It uses TCP over the line based protocol with a default port 4242. And the data gets sent in blocks of at most 1428 bytes to minimize the number of network packets. Efficient queen ๐Ÿ’…

Synopsis:

<Plugin write_tsdb>
  ResolveInterval 60
  ResolveJitter 60
  <Node "example">
    Host "tsd-1.my.domain"
    Port "4242"
    HostTags "status=production"
  </Node>
</Plugin>

The configuration consists of one or more <Node Name> blocks plus some global directives.

Global directives are:

๐Ÿ’œ ResolveInterval seconds

๐Ÿ’œ ResolveJitter seconds

When collectd connects to a TSDB node, it requests the hostname from DNS. That can become a whole problem if the TSDB node is unavailable or badly configured, because collectd will request DNS in order to reconnect for every metric, which can flood your DNS ๐Ÿ˜ญ. So you can cache the last value for ResolveInterval seconds. Defaults to the Interval of the write_tsdb plugin, e.g. 10 seconds.

You can also define a jitter โ€” a random interval to wait in addition to ResolveInterval. This prevents all your collectd servers from resolving the hostname at the same time when the connection fails. Defaults to the Interval of the write_tsdb plugin, e.g. 10 seconds.

Note: if the DNS resolution has already been successful when the socket closes, the plugin will try to reconnect immediately with the cached information. DNS is queried only when the socket is closed for longer than ResolveInterval + ResolveJitter seconds. Receipts! ๐Ÿงพ

Inside the Node blocks, these options are recognized:

๐Ÿ’œ Host Address ๐Ÿ’œ

Hostname or address to connect to. Defaults to localhost.

๐Ÿ’œ Port Service ๐Ÿ’œ

Service name or port number to connect to. Defaults to 4242.

๐Ÿ’œ HostTags String ๐Ÿ’œ

When set, HostTags gets added to the end of the metric. It's intended to be used for name=value pairs that the TSD will tag the metric with. Dots and whitespace are not escaped in this string. ๐ŸŒธ

๐Ÿ’œ StoreRates false|true ๐Ÿ’œ

If set to true, convert counter values to rates. If set to false (the default), counter values are stored as is, as an increasing integer number.

๐Ÿ’œ AlwaysAppendDS false|true ๐Ÿ’œ

If set to true, append the name of the Data Source (DS) to the metric identifier. If set to false (the default), this is only done when there is more than one DS.


๐Ÿฆ‹ Plugin "write_mongodb" ๐Ÿฆ‹

The write_mongodb plugin sends values to MongoDB, a schema-less NoSQL database. Fancy โœจ

Synopsis:

<Plugin "write_mongodb">
  <Node "default">
    Host "localhost"
    Port "27017"
    Timeout 1000
    StoreRates true
  </Node>
</Plugin>

The plugin can send values to multiple instances of MongoDB by specifying one Node block for each instance. Within the Node blocks, the following options are available:

๐Ÿ’œ Host Address ๐Ÿ’œ

Hostname or address to connect to. Defaults to localhost.

๐Ÿ’œ Port Service ๐Ÿ’œ

Service name or port number to connect to. Defaults to 27017.

๐Ÿ’œ Timeout Milliseconds ๐Ÿ’œ

Set the timeout for each operation on MongoDB to Timeout milliseconds. Setting this option to zero means no timeout, which is the default. (Zero = chaos, but okay, live your life ๐Ÿซถ)

๐Ÿ’œ StoreRates false|true ๐Ÿ’œ

If set to true (the default), convert counter values to rates. If set to false, counter values are stored as is, i.e. as an increasing integer number.

๐Ÿ’œ Database Database

๐Ÿ’œ User User

๐Ÿ’œ Password Password

Sets the information used when authenticating to a MongoDB database. The fields are optional (in which case no authentication is attempted), but if you want to use authentication, all three fields must be set. No skipping, bestie, all three or nothing ๐Ÿ’€


๐Ÿฆ‹ Plugin "write_prometheus" ๐Ÿฆ‹

The write_prometheus plugin implements a tiny webserver that can be scraped using Prometheus. Tiny! Webserver! It's giving small but mighty ๐Ÿงš

Options:

๐Ÿ’œ Host Host ๐Ÿ’œ

Bind to the hostname / address Host. By default, the plugin binds to the any address, i.e. it accepts packets sent to any of the host's addresses.

This option is supported only for libmicrohttpd newer than 0.9.0. Old libmicrohttpd girlies, I'm sorry ๐Ÿ˜ญ

๐Ÿ’œ Port Port ๐Ÿ’œ

Port the embedded webserver should listen on. Defaults to 9103.

๐Ÿ’œ StalenessDelta Seconds ๐Ÿ’œ

Time in seconds after which Prometheus considers a metric stale if it hasn't seen any update for it. This value must match the setting in Prometheus. It defaults to 300 seconds (that's 5 minutes), same as Prometheus. โฐ

Background:

Prometheus has a global setting, StalenessDelta, which controls after which time a metric without updates is considered stale. This setting effectively puts an upper limit on the interval in which metrics are reported.

When the write_prometheus plugin encounters a metric with an interval exceeding this limit, it will inform you, the user, and provide the metric to Prometheus without a timestamp. That causes Prometheus to consider the metric fresh each time it's scraped, with the time of the scrape being considered the time of the update. The result? There appear more datapoints in Prometheus than were actually created โ€” but at least the metric doesn't disappear periodically. Trade-offs, babes ๐ŸŒˆ


๐Ÿฆ‹ Plugin "write_http" ๐Ÿฆ‹

This output plugin submits values to an HTTP server using POST requests, encoding metrics with JSON or using the PUTVAL command described in collectd-unixsock(5). ๐ŸŒ

Synopsis:

<Plugin "write_http">
  <Node "example">
    URL "http://example.com/post-collectd"
    User "collectd"
    Password "weCh3ik0"
    Format JSON
  </Node>
</Plugin>

The plugin can send values to multiple HTTP servers by specifying one <Node Name> block for each server. Within each Node block, the following options are available:

๐Ÿ’œ URL URL ๐Ÿ’œ

URL to which the values are submitted. Mandatory, bestie โ€” you literally cannot skip this one ๐Ÿšจ

๐Ÿ’œ User Username ๐Ÿ’œ

Optional user name needed for authentication.

๐Ÿ’œ Password Password ๐Ÿ’œ

Optional password needed for authentication.

๐Ÿ’œ VerifyPeer true|false ๐Ÿ’œ

Enable or disable peer SSL certificate verification. See http://curl.haxx.se/docs/sslcerts.html for details. Enabled by default. ๐Ÿ”’

๐Ÿ’œ VerifyHost true|false ๐Ÿ’œ

Enable or disable peer host name verification. If enabled, the plugin checks whether the Common Name or a Subject Alternate Name field of the SSL certificate matches the host name provided by the URL option. If this identity check fails, the connection gets aborted. Obviously, this only works when connecting to a SSL enabled server. Enabled by default. ๐Ÿ›ก๏ธ

CACert File

Okay so this is the file that holds one or more SSL certificates. If you're trying to use HTTPS you'll possibly need this one. And the whole "which CA certificates ship bundled with libcurl and get checked by default" situation? That's entirely up to whichever distribution you're running, bestie. No universal answer, sorry. ๐Ÿ’…

CAPath Directory

A directory that holds one or more CA certificate files. You reach for this when, for some reason, <italic>all* the CA certs you need aren't chilling in the same file and you can't just point at them with the CACert option. Heads up though: this one requires libcurl to be built against OpenSSL. ๐Ÿฆ‹

ClientKey File

The file holding the private key in PEM format, used for certificate-based authentication. That's it, that's the whole vibe. ๐Ÿ”‘

ClientCert File

The file holding the SSL certificate used for certificate-based authentication. Pairs with ClientKey, obviously. ๐ŸŽ€

ClientKeyPass Password

The password needed to actually load the private key in ClientKey. Don't lose it, don't commit it, iykyk. ๐Ÿ™ˆ

Header Header

A HTTP header to add to the request. And if you specify this option more than once, multiple headers get added. Example:

Header "X-Custom-Header: custom_value"

custom header era โœจ

SSLVersion SSLv2|SSLv3|TLSv1|TLSv1_0|TLSv1_1|TLSv1_2

Defines which SSL protocol version must be used. By default, libcurl will attempt to figure out the remote SSL protocol version all on its own. Check curl_easy_setopt(3) for the nitty-gritty details. ๐Ÿ‘‘

Format Command|JSON|KAIROSDB

The format of the output to generate. Watch:

  • Command โ€” creates output that's understood by the Exec and UnixSock plugins.
  • JSON โ€” creates output in the JavaScript Object Notation (JSON).
  • KAIROSDB โ€” creates output in the KairosDB format.

Defaults to Command. periodt ๐Ÿ’…

Attribute String String

Only available for the KAIROSDB output format. ๐Ÿซถ

Basically you take the two given strings and consider them the key and value of an additional tag for every metric being sent out.

And yeah, you can add multiple Attribute lines. Stack 'em up. ๐Ÿ“

TTL Int

Only available for the KAIROSDB output format.

Sets the Cassandra ttl for the data points.

Please refer to <http://kairosdb.github.io/docs/build/html/restapi/AddDataPoints.html?highlight=ttl>

Prefix String

Only available for the KAIROSDB output format.

Sets the metrics prefix string. Defaults to collectd. โœจ

Metrics true|false

Controls whether metrics are POSTed to this location. Defaults to true. ๐Ÿ“ฌ

Notifications false|true

Controls whether notifications are POSTed to this location. Defaults to false. ๐Ÿ””

StoreRates true|false

If set to true, counter values get converted to rates. If set to false (that's the default, btw) counter values are stored as is โ€” i.e. as an increasing integer number. No conversion, no drama. ๐Ÿ“ˆ

BufferSize Bytes

Sets the send buffer size to Bytes. And okay, the tradeoff is real: by increasing this buffer, fewer HTTP requests get generated, but more metrics get batched / cached for longer before being sent โ€” which introduces additional delay until they're actually available on the server side. So like, be intentional about it. ๐Ÿ’€

Bytes must be at least 1024 and cannot exceed the size of an int, i.e. 2 GByte. Defaults to 4096.

LowSpeedLimit Bytes per Second

Sets the minimal transfer rate in Bytes per Second below which the connection with the HTTP server gets considered too slow and gets aborted. And, uh, all the data submitted over this connection will probably be lost. Defaults to 0, which means no minimum transfer rate is enforced. ๐Ÿ™ˆ

Timeout Timeout

Sets the maximum time in milliseconds given for HTTP POST operations to complete. When this limit is reached, the POST operation gets aborted, and all the data in the current send buffer will probably be lost. Defaults to 0, which means the connection never times out. main character energy: zero patience, infinite time. ๐Ÿ˜Œ

LogHttpError false|true

Enables printing of the HTTP error code to the log. Turned off by default. ๐Ÿ”ฅ

<Statistics Name>

One Statistics block can be used to specify cURL statistics to be collected for each request to the remote URL. See the cURL Statistics section above for the details, bestie. ๐Ÿงš

The write_http plugin regularly submits the collected values to the HTTP server. How frequently that happens depends on how much data you're collecting and the size of BufferSize. And the optimal value to set Timeout to is slightly below this interval โ€” which you can estimate by monitoring the network traffic between collectd and the HTTP server. โœจ


๐Ÿ’ซ Plugin "write_influxdb_udp" ๐Ÿ’ซ

The write_influxdb_udp plugin sends data to instances of InfluxDB using the "Line Protocol". Each plugin gets sent as a measurement with a time precision of miliseconds, while plugin instance, type and type instance get sent as tags. slay. ๐ŸŒˆ

 <Plugin "write_influxdb_udp">
   Server "influxdb.fqdn"
   Server "influxdb2.fqdn"
   TimePrecision "ms"
   StoreRates "yes"
   WriteMetadata "no"
 </Plugin>

<Server Host [Port]>

The Server statement sets a server to send datagrams to. And yes, this statement can appear multiple times โ€” once for each unique destination you're sending to. ๐Ÿ“ฎ

The argument Host may be a hostname, an IPv4 address or an IPv6 address. The optional second argument specifies a port number or a service name. If you don't give one, the default, 8089, is used. The arguments Host and Port should be enclosed in "quotes". ๐Ÿซถ

TimePrecision ms|us|ns

The TimePrecision option sets the precision of the timestamps sent to InfluxDB. It must match the precision set in the InfluxDB line protocol configuration. No cap, they have to line up.

The defaut value is ms. Note that the InfluxDB default may differ. ๐Ÿ™ˆ

TimeToLive 1-255

Set the time-to-live of sent packets. This applies to all packets โ€” unicast and multicast, and IPv4 and IPv6. The default is to not change this value. That means multicast packets will be sent with a TTL of 1 (one) on most operating systems. ๐Ÿฆ‹

MaxPacketSize 1024-65535

Set the maximum size for datagrams received over the network. Packets larger than this get truncated. Defaults to 1452 bytes, which is the maximum payload size that can be transmitted in one Ethernet frame using IPv6 / UDP. very demure, very precise. ๐Ÿ“

StoreRates true|false

If set to true, convert absolute, counter and derive values to rates. If set to false (the default), absolute, counter and derive values are sent as is. ๐Ÿ”ฅ

WriteMetadata true|false

Defaults to false. If set to true, send aditional tags to influxdb with collectd value metadata. ๐Ÿท๏ธ


๐Ÿ‘‘ Plugin "write_kafka" ๐Ÿ‘‘

The write_kafka plugin will send values to a Kafka topic, a distributed queue. Synopsis time:

 <Plugin "write_kafka">
   Property "metadata.broker.list" "broker1:9092,broker2:9092"
   <Topic "collectd">
     Format JSON
   </Topic>
 </Plugin>

The following options are understood by the write_kafka plugin: ๐Ÿ’…

<Topic Name>

The plugin's configuration consists of one or more Topic blocks. Each block gets a unique Name and specifies one kafka producer. Inside the Topic block, the following per-topic options are understood:

Property String String

Configure the named property for the current topic. Properties are forwarded to the kafka producer library librdkafka. pass it along, bestie. ๐Ÿซถ

Key String

Use the specified string as a partitioning key for the topic. Kafka breaks topics into partitions and guarantees that for a given topology, the same consumer will be used for a specific key. Oh and โ€” the special (case insensitive) string Random can be used to specify that an arbitrary partition should be used. ๐ŸŽฒ

Format Command|JSON|Graphite

Selects the format in which messages are sent to the broker. The lineup:

  • Command (the default) โ€” values are sent as "PUTVAL" commands, identical to the syntax used by the Exec and UnixSock plugins.
  • JSON โ€” the values are encoded in the JavaScript Object Notation, an easy and straight forward exchange format.
  • Graphite โ€” values are encoded in the Graphite format, which is "<metric> <value> <timestamp>\n".

pick your fighter โœจ

StoreRates true|false

Determines whether or not "COUNTER", "DERIVE" and "ABSOLUTE" data sources get converted to a rate (i.e. a "GAUGE" value). If set to false (the default), no conversion is performed. Otherwise the conversion is performed using the internal value cache.

Please note that currently this option is only used if the Format option has been set to JSON.

GraphitePrefix (Format=Graphite only)

A prefix can be added in the metric name when outputting in the Graphite format.

When GraphiteUseTags is false, the prefix gets added before the Host name. Metric name will be "<prefix><host><postfix><plugin><type><name>"

When GraphiteUseTags is true, the prefix gets added in front of the series name. ๐ŸŽ€

GraphitePostfix (Format=Graphite only)

A postfix can be added in the metric name when outputting in the Graphite format.

When GraphiteUseTags is false, the postfix gets added after the Host name. Metric name will be "<prefix><host><postfix><plugin><type><name>"

When GraphiteUseTags is true, the prefix value gets appended to the end of the series name (before the first ; that separates the name from the tags). ๐Ÿ’ซ

GraphiteEscapeChar (Format=Graphite only)

Specify a character to replace dots (.) in the host part of the metric name. In a Graphite metric name, dots are used as separators between different metric parts (host, plugin, type). Default is "_" (Underscore). ๐Ÿฆ‹

GraphiteSeparateInstances false|true

If set to true, the plugin instance and type instance will be in their own path component โ€” for example "host.cpu.0.cpu.idle". If set to false (the default), the plugin and plugin instance (and likewise the type and type instance) get put into one component โ€” for example "host.cpu-0.cpu-idle".

Option value is not used when GraphiteUseTags is true.

GraphiteAlwaysAppendDS true|false

If set to true, append the name of the Data Source (DS) to the "metric" identifier. If set to false (the default), this is only done when there's more than one DS. ๐Ÿ‘‘

GraphitePreserveSeparator false|true

If set to false (the default) the "." (dot) character gets replaced with GraphiteEscapeChar. Otherwise, if set to true, the "." (dot) character is preserved โ€” i.e. passed through. untouched. iconic. โœจ

Option value is not used when GraphiteUseTags is true.
  • GraphiteUseTags false|true โ€” set this to true and your Graphite metric names get generated as tagged series. ๐Ÿ’…

Default value: false.

  • StoreRates true|false โ€” if true (the default, bestie), counter values get converted into rates. If false, counter values are stored as is, i.e. as an increasing integer number. ๐Ÿฆ‹

This will be reflected in the "ds_type" tag: if StoreRates is enabled, converted values will have "rate" appended to the data source type, e.g. "ds_type:derive:rate". โœจ

  • Property String String โ€” this is how you configure the kafka producer through properties, and you almost always will want to set metadata.broker.list to your Kafka broker list. ๐ŸŽ€

๐ŸŒธ Plugin write_redis

The write_redis plugin submits values to Redis, a data structure server. (iconic pairing ngl ๐Ÿ”ฅ)

Synopsis:

  <Plugin "write_redis">
    <Node "example">
        Host "localhost"
        Port "6379"
        Timeout 1000
        Prefix "collectd/"
        Database 1
        MaxSetSize -1
        MaxSetDuration -1
        StoreRates true
    </Node>
  </Plugin>

Values are submitted to Sorted Sets, using the metric name as the key and the timestamp as the score. Retrieving a date range can then be done using the "ZRANGEBYSCORE" Redis command. ๐Ÿ‘‘

Also โ€” and this is the cute part โ€” all the identifiers of these Sorted Sets are kept in a Set called "collectd/values" (or "${prefix}/values" if the Prefix option was specified) and can be retrieved using the "SMEMBERS" Redis command. You can specify the database to use with the Database parameter (default is 0). See <http://redis.io/commands#sorted_set> and <http://redis.io/commands#set> for details. ๐Ÿ“š

The information shown in the synopsis above is the default configuration which is used by the plugin if no configuration is present. (yes bestie, it just works out of the box ๐Ÿ’…)

The plugin can send values to multiple instances of Redis by specifying one Node block for each instance. Within the Node blocks, the following options are available:

  • Node Nodename โ€” the Node block identifies a new Redis node, that is a new Redis instance running on a specified host and port. The node name is a canonical identifier which is used as plugin instance. It is limited to 51 characters in length. ๐ŸŽ€
  • Host Hostname โ€” the hostname or IP-address where the Redis instance is running on. ๐Ÿ“
  • Port Port โ€” the TCP port on which the Redis instance accepts connections. Either a service name of a port number may be given. Please note that numerical port numbers must be given as a string, too. (yes, as a string, don't @ me ๐Ÿ™ˆ)
  • Timeout Milliseconds โ€” sets the socket connection timeout, in milliseconds. โฑ๏ธ
  • Prefix Prefix โ€” the prefix used when constructing the name of the Sorted Sets and the Set containing all metrics. Defaults to "collectd/", so metrics will have names like "collectd/cpu-0/cpu-user". When setting this to something different, it is recommended but not required to include a trailing slash in Prefix. โœจ
  • Database Index โ€” this index selects the redis database to use for writing operations. Defaults to 0. ๐Ÿ—‚๏ธ
  • MaxSetSize Items โ€” limits the number of items that the Sorted Sets can hold. Negative values for Items sets no limit, which is the default behavior. ๐Ÿ“
  • MaxSetDuration Seconds โ€” limits the duration of items that the Sorted Sets can hold. Negative values for Items sets no duration, which is the default behavior. โณ
  • StoreRates true|false โ€” if set to true (the default), convert counter values to rates. If set to false counter values are stored as is, i.e. as an increasing integer number. ๐Ÿ’œ

๐Ÿ‘‘ Plugin write_riemann

The write_riemann plugin will send values to Riemann, a powerful stream aggregation and monitoring system. The plugin sends Protobuf encoded data to Riemann using UDP packets. ๐Ÿ›ฐ๏ธ

Synopsis:

 <Plugin "write_riemann">
   <Node "example">
     Host "localhost"
     Port "5555"
     Protocol UDP
     StoreRates true
     AlwaysAppendDS false
     TTLFactor 2.0
   </Node>
   Tag "foobar"
   Attribute "foo" "bar"
 </Plugin>

The following options are understood by the write_riemann plugin:

  • <Node Name> โ€” the plugin's configuration consists of one or more Node blocks. Each block is given a unique Name and specifies one connection to an instance of Riemann. Inside the Node block, the following per-connection options are understood: ๐Ÿซถ
    • Host Address โ€” hostname or address to connect to. Defaults to "localhost". ๐Ÿ“
    • Port Service โ€” service name or port number to connect to. Defaults to 5555. ๐Ÿ”Œ
    • Protocol UDP|TCP|TLS โ€” specify the protocol to use when communicating with Riemann. Defaults to TCP. (even though the synopsis above shows UDP, bestie ๐Ÿ‘€)
    • TLSCertFile Path โ€” when using the TLS protocol, path to a PEM certificate to present to remote host. ๐ŸŽซ
    • TLSCAFile Path โ€” when using the TLS protocol, path to a PEM CA certificate to use to validate the remote hosts's identity. ๐Ÿ›ก๏ธ
    • TLSKeyFile Path โ€” when using the TLS protocol, path to a PEM private key associated with the certificate defined by TLSCertFile. ๐Ÿ”‘
    • Batch true|false โ€” if set to true and Protocol is set to TCP, events will be batched in memory and flushed at regular intervals or when BatchMaxSize is exceeded. ๐Ÿ“ฆ

Notifications are not batched and sent as soon as possible.

When enabled, it can occur that events get processed by the Riemann server close to or after their expiration time. Tune the TTLFactor and BatchMaxSize settings according to the amount of values collected, if this is an issue. โš ๏ธ

Defaults to true

  • BatchMaxSize size โ€” maximum payload size for a riemann packet. Defaults to 8192. ๐Ÿ“
  • BatchFlushTimeout seconds โ€” maximum amount of seconds to wait in between to batch flushes. No timeout by default. โฒ๏ธ
  • StoreRates true|false โ€” if set to true (the default), convert counter values to rates. If set to false counter values are stored as is, i.e. as an increasing integer number. ๐Ÿ’œ

This will be reflected in the "ds_type" tag: if StoreRates is enabled, converted values will have "rate" appended to the data source type, e.g. "ds_type:derive:rate". โœจ

  • AlwaysAppendDS false|true โ€” if set to true, append the name of the Data Source (DS) to the "service", i.e. the field that, together with the "host" field, uniquely identifies a metric in Riemann. If set to false (the default), this is only done when there is more than one DS. ๐Ÿท๏ธ
  • TTLFactor Factor โ€” Riemann events have a Time to Live (TTL) which specifies how long each event is considered active. collectd populates this field based on the metrics interval setting. This setting controls the factor with which the interval is multiplied to set the TTL. The default value is 2.0. Unless you know exactly what you're doing, you should only increase this setting from its default value. โณ
  • Notifications false|true โ€” if set to true, create riemann events for notifications. This is true by default. When processing thresholds from write_riemann, it might prove useful to avoid getting notification events. ๐Ÿ””
  • CheckThresholds false|true โ€” if set to true, attach state to events based on thresholds defined in the Threshold plugin. Defaults to false. โš–๏ธ
  • EventServicePrefix String โ€” add the given string as a prefix to the event service name. If EventServicePrefix not set or set to an empty string (""), no prefix will be used. ๐ŸŽ€
  • Tag String โ€” add the given string as an additional tag to the metric being sent to Riemann. ๐Ÿท๏ธ
  • Attribute String String โ€” consider the two given strings to be the key and value of an additional attribute for each metric being sent out to Riemann. ๐Ÿซถ

๐Ÿฆ‹ Plugin write_sensu

The write_sensu plugin will send values to Sensu, a powerful stream aggregation and monitoring system. The plugin sends JSON encoded data to a local Sensu client using a TCP socket. ๐Ÿ“ก

Synopsis:

 <Plugin "write_sensu">
   <Node "example">
     Host "localhost"
     Port "3030"
     StoreRates true
     AlwaysAppendDS false
     IncludeSource false
     MetricHandler "influx"
     MetricHandler "default"
     NotificationHandler "flapjack"
     NotificationHandler "howling_monkey"
     Notifications true
   </Node>
   Tag "foobar"
   Attribute "foo" "bar"
 </Plugin>

The following options are understood by the write_sensu plugin:

  • <Node Name> โ€” the plugin's configuration consists of one or more Node blocks. Each block is given a unique Name and specifies one connection to an instance of Sensu. Inside the Node block, the following per-connection options are understood: ๐Ÿซถ
    • Host Address โ€” hostname or address to connect to. Defaults to "localhost". ๐Ÿ“
    • Port Service โ€” service name or port number to connect to. Defaults to 3030. ๐Ÿ”Œ
    • StoreRates true|false โ€” if set to true (the default), convert counter values to rates. If set to false counter values are stored as is, i.e. as an increasing integer number. ๐Ÿ’œ

This will be reflected in the "collectd_data_source_type" tag: if StoreRates is enabled, converted values will have "rate" appended to the data source type, e.g. "collectd_data_source_type:derive:rate". โœจ

  • AlwaysAppendDS false|true โ€” if set the true, append the name of the Data Source (DS) to the "service", i.e. the field that, together with the "host" field, uniquely identifies a metric in Sensu. If set to false (the default), this is only done when there is more than one DS. ๐Ÿท๏ธ
  • Notifications false|true โ€” if set to true, create Sensu events for notifications. This is false by default. At least one of Notifications or Metrics should be enabled. ๐Ÿ””
  • Metrics false|true โ€” if set to true, create Sensu events for metrics. This is false by default. At least one of Notifications or Metrics should be enabled. ๐Ÿ“Š
  • Separator String โ€” sets the separator for Sensu metrics name or checks. Defaults to "/". โœ‚๏ธ
  • MetricHandler String โ€” adds a handler that gets set whenever metrics are shipped off to Sensu. You can stack several of these, one per line. Defaults to no handler at all.
  • NotificationHandler String โ€” same vibe but for notifications: a handler that gets set when notifications go out to Sensu. Several allowed, one per line. Defaults to no handler.
  • EventServicePrefix String โ€” sticks the given string on the front of the event service name. If it's not set, or it's set to an empty string (""), then no prefix, periodt.
  • Tag String โ€” adds the given string as one more tag on the metric heading to Sensu.
  • Attribute String String โ€” treats those two strings as the key and the value of an extra attribute for every metric being sent out to Sensu.
  • IncludeSource false|true โ€” set it to true and the source host of the metrics/notification gets passed along to sensu via the source attribute. Heads up though: this may register the host as a proxy client in sensu ๐Ÿ‘€

Set it to false (which is the default) and the hostname gets discarded, so it looks like the event came from the connected sensu agent. iykyk.


๐Ÿ”ฅ Plugin "write_stackdriver" ๐Ÿ”ฅ

The "write_stackdriver" plugin writes metrics to the Google Stackdriver Monitoring service. Cloud girlies rise ๐Ÿ’…

It supports two authentication methods: when you configure it, credentials get read from the JSON credentials file you point at with CredentialFile. Or, when you're running on Google Compute Engine (GCE), an OAuth token is pulled from the metadata server and used to authenticate to GCM. Slay, two paths, same destination โœจ

Synopsis:

<Plugin write_stackdriver>
  CredentialFile "/path/to/service_account.json"
  <Resource "global">
    Label "project_id" "monitored_project"
  </Resource>
</Plugin>
  • CredentialFile file โ€” path to a JSON credentials file holding the credentials for a GCP service account.

If CredentialFile is not specified, the plugin falls back to Application Default Credentials. Which literally means the credentials used depend on the environment, bestie:

  • The environment variable "GOOGLE_APPLICATION_CREDENTIALS" gets checked first. If it's set, it should point to a JSON file that defines the credentials.
  • Then the path "${HOME}/.config/gcloud/application_default_credentials.json" gets checked. That's where the credentials used by the gcloud command line utility live. You can run gcloud auth application-default login to create them.
โš ๏ธ Real talk: these credentials are often tied to your personal account, not a service account โ€” which makes them genuinely unfit for production. Don't do it, bestie ๐Ÿ˜ญ
  • When running on GCE, the built-in service account associated with the virtual machine instance is used. See also the Email option below. ๐Ÿ‘‘
  • Project Project โ€” the Project ID or the Project Number of the Stackdriver Account. The Project ID is a string identifying the GCP project, and you can choose it freely when creating a new project. The Project Number is a 12-digit decimal number. You can look up both on the Developer Console.

This setting is optional, no pressure. If you don't set it, the project ID is read from the credentials file or determined from the GCE's metadata service. ๐Ÿ’ซ

  • Email Email (GCE only) โ€” picks the GCE Service Account used for authentication.

Every GCE instance has a "default" Service Account but may also be associated with additional Service Accounts. This is often used to restrict the permissions of services running on the GCE instance down to the required minimum. The write_stackdriver plugin requires the "https://www.googleapis.com/auth/monitoring" scope. So when multiple Service Accounts are available, this option selects which one the write_stackdriver plugin uses. Choose your fighter ๐Ÿซถ

This block takes one string argument, the ResourceType. Inside the block are one or more Label options which configure the resource labels.

This block is optional. The default depends on the runtime environment: on GCE, the "gce_instance" resource type is used, otherwise the "global" resource type is used:

  • On GCE, defaults to the equivalent of this config:
<Resource "gce_instance">
  Label "project_id" "<project_id>"
  Label "instance_id" "<instance_id>"
  Label "zone" "<zone>"
</Resource>

The values for project_id, instance_id and zone are read from the GCE metadata service.

  • Elsewhere (i.e. not on GCE), defaults to the equivalent of this config:
<Resource "global">
  Label "project_id" "<Project>"
</Resource>

Where Project refers to the value of the Project option or the project ID inferred from the CredentialFile.


๐ŸŒˆ Plugin "write_syslog" ๐ŸŒˆ

The "write_syslog" plugin writes data in syslog format log messages. It implements the basic syslog protocol, RFC 5424, extends it with content-based filtering, rich filtering capabilities, flexible configuration options, and throws in features like using TCP for transport. Iconic behaviour, honestly.

The plugin can connect to a Syslog daemon โ€” like syslog-ng and rsyslog โ€” that will ingest metrics, transform them, and ship them to the specified output. It uses TCP over the "line based" protocol with a default port 44514. The data gets sent in blocks of at most 1428 bytes so we minimize the number of network packets. Efficiency queen ๐Ÿ’…

Synopsis:

<Plugin write_syslog>
  ResolveInterval 60
  ResolveJitter 60
  <Node "example">
    Host "syslog-1.my.domain"
    Port "44514"
    Prefix "collectd"
    MessageFormat "human"
    HostTags ""
  </Node>
</Plugin>

The configuration consists of one or more <Node Name> blocks and global directives.

Global directives are:

  • ResolveInterval seconds
  • ResolveJitter seconds

When collectd connects to a syslog node, it will request the hostname from DNS. That can become a whole problem if the syslog node is unavailable or badly configured, because collectd will request DNS in order to reconnect for every metric, which can literally flood your DNS. So you can cache the last value for ResolveInterval seconds. Defaults to the Interval of the write_syslog plugin, e.g. 10 seconds.

You can also define a jitter โ€” a random interval to wait in addition to ResolveInterval. This prevents all your collectd servers from resolving the hostname at the same exact time when the connection fails. Defaults to the Interval of the write_syslog plugin, e.g. 10 seconds.

๐Ÿ“Œ Note: if the DNS resolution has already been successful when the socket closes, the plugin will try to reconnect immediately using the cached information. DNS is queried only when the socket has been closed for longer than ResolveInterval + ResolveJitter seconds. no cap.

Inside the Node blocks, the following options are recognized:

  • Host Address โ€” hostname or address to connect to. Defaults to "localhost".
  • Port Service โ€” service name or port number to connect to. Defaults to 44514.
  • HostTags String โ€” when set, HostTags is added to the end of the metric. It's intended for adding extra metadata to tag the metric with. Dots and whitespace are not escaped in this string. ๐Ÿ™ˆ

Examples:

When MessageFormat is set to "human".

["prefix1" "example1"="example1_v"]["prefix2" "example2"="example2_v"]"

When MessageFormat is set to "JSON", the text should be in JSON format. Escaping the quotation marks is required.

HostTags "\"prefix1\": {\"example1\":\"example1_v\",\"example2\":\"example2_v\"}"
  • MessageFormat String โ€” MessageFormat selects the format in which messages are sent to the syslog daemon, human or JSON. Defaults to human.

Syslog message format:

<priority>VERSION ISOTIMESTAMP HOSTNAME APPLICATION PID MESSAGEID STRUCTURED-DATA MSG

The difference between the message formats is in the STRUCTURED-DATA and MSG parts.

Human format:

<166>1 ISOTIMESTAMP HOSTNAME collectd PID MESSAGEID
["collectd" "value": "v1" "plugin"="plugin_v" "plugin_instance"="plugin_instance_v"
"type_instance"="type_instance_v" "type"="type_v" "ds_name"="ds_name_v" "interval"="interval_v" ]
"host_tag_example"="host_tag_example_v" plugin_v.type_v.ds_name_v="v1"

JSON format:

<166>1 ISOTIMESTAMP HOSTNAME collectd PID MESSAGEID STRUCTURED-DATA
{
  "collectd": {
  "time": time_as_epoch, "interval": interval_v, "plugin": "plugin_v",
  "plugin_instance": "plugin_instance_v", "type":"type_v",
  "type_instance": "type_instance_v", "plugin_v": {"type_v": v1}
  } , "host":"host_v", "host_tag_example": "host_tag_example_v"
}
  • StoreRates false|true โ€” set to true and counter values get converted to rates. Set to false (the default) and counter values are stored as is, as an increasing integer number.
  • AlwaysAppendDS false|true โ€” set to true and the name of the Data Source (DS) gets appended to the "metric" identifier. Set to false (the default) and this only happens when there's more than one DS.
  • Prefix String โ€” when set, Prefix gets added to all metric names as a prefix. It's intended for when you want to be able to define the source of the specific metric. Dots and whitespace are not escaped in this string. ๐ŸŽ€

โญ Plugin "xencpu" โญ

This plugin collects metrics of hardware CPU load for a machine running the Xen hypervisor. Load is calculated from the 'idle time' value, provided by Xen. The result is reported using the "percent" type, for each CPU (core). ๐ŸŒธ

This plugin doesn't have any options (yet). Short and sweet, we love a low-maintenance icon.


๐Ÿฆ‹ Plugin "zookeeper" ๐Ÿฆ‹

The zookeeper plugin will collect statistics from a Zookeeper server using the mntr command. It requires Zookeeper 3.4.0+ and access to the client port.

Synopsis:

<Plugin "zookeeper">
  Host "127.0.0.1"
  Port "2181"
</Plugin>
  • Host Address โ€” hostname or address to connect to. Defaults to "localhost".
  • Port Service โ€” service name or port number to connect to. Defaults to 2181. ๐Ÿซถ

๐Ÿ’… THRESHOLD CONFIGURATION ๐Ÿ’…

Okay bestie, so starting with version 4.3.0 collectd has support for monitoring. And by "monitoring" we mean the values aren't just being stored or yeeted off somewhere โ€” they're actually being judged. And if a problem is recognized, collectd acts on it. The only action collectd takes itself is to generate and dispatch a "notification". Plugins can register to receive notifications and do their own appropriate further actions from there. Slay.

Since systems (and what you expect them to do) differ a lot, you can configure thresholds for your values freely. This gives you a lot of flexibility but also a lot of responsibility. ๐Ÿซถ

Every time a value is out of range a notification is dispatched. Which means the idle percentage of your CPU needs to be less than the configured threshold only once for a notification to be generated. There's no such thing as a moving average or similar โ€” at least not now. ๐Ÿ˜ญ

Also, all values that match a threshold are considered relevant or "interesting". As a consequence collectd will issue a notification if they are not received for Timeout iterations. The Timeout configuration option is explained in section "GLOBAL OPTIONS". If, for example, Timeout is set to "2" (the default) and some host sends its CPU statistics to the server every 60 seconds, a notification will be dispatched after about 120 seconds. It may take a little longer because the timeout is checked only once each Interval on the server. โฐ

When a value comes within range again or is received after it was missing, an "OKAY-notification" is dispatched. ๐ŸŽ€

Here's a configuration example to get you started. Read below for more info:

 <Plugin threshold>
   <Type "foo">
     WarningMin    0.00
     WarningMax 1000.00
     FailureMin    0.00
     FailureMax 1200.00
     Invert false
     Instance "bar"
   </Type>

   <Plugin "interface">
     Instance "eth0"
     <Type "if_octets">
       FailureMax 10000000
       DataSource "rx"
     </Type>
   </Plugin>

   <Host "hostname">
     <Type "cpu">
       Instance "idle"
       FailureMin 10
     </Type>

     <Plugin "memory">
       <Type "memory">
         Instance "cached"
         WarningMin 100000000
       </Type>
     </Plugin>
   </Host>
 </Plugin>

There are basically two types of configuration statements, and lowkey they do two very different jobs:

  • The "Host", "Plugin", and "Type" blocks select the value for which a threshold should be configured. The "Plugin" and "Type" blocks may be specified further using the "Instance" option.
  • The other statements specify the threshold to configure.

You can combine the blocks by nesting them, though they must be nested in the above order, i.e. "Host" may contain either "Plugin" and "Type" blocks, "Plugin" may only contain "Type" blocks, and "Type" may not contain other blocks. If multiple blocks apply to the same value, the most specific block is used. No cap. ๐Ÿ‘‘

The threshold statements must be included in a "Type" block. Currently the following statements are recognized:

  • FailureMax Value, WarningMax Value โ€” sets the upper bound of acceptable values. If unset it defaults to positive infinity. If a value is greater than FailureMax a FAILURE notification will be created. If the value is greater than WarningMax but less than (or equal to) FailureMax, a WARNING notification will be created.
  • FailureMin Value, WarningMin Value โ€” sets the lower bound of acceptable values. If unset it defaults to negative infinity. If a value is less than FailureMin a FAILURE notification will be created. If the value is less than WarningMin but greater than (or equal to) FailureMin, a WARNING notification will be created.
  • DataSource DSName โ€” some data sets have more than one "data source". Iconic examples: the "if_octets" data set, which has received ("rx") and sent ("tx") bytes, and the "disk_ops" data set, which holds "read" and "write" operations. The system load data set, "load", even has three data sources: "shortterm", "midterm", and "longterm". โœจ

Normally, all data sources are checked against a configured threshold. If that's not it, or if you want different limits for each data source, you can use the DataSource option to have a threshold apply only to one data source.

  • Invert true|false โ€” if set to true the range of acceptable values is inverted, i.e. values between FailureMin and FailureMax (WarningMin and WarningMax) are not okay. Defaults to false. ๐Ÿ™ˆ
  • Persist true|false โ€” sets how often notifications are generated. If set to true, one notification will be generated for each value that is out of the acceptable range. If set to false (the default), then a notification is only generated if a value is out of range but the previous value was okay.

This applies to missing values too: if set to true, a notification about a missing value is generated once every Interval seconds. If set to false, only one such notification is generated until the value appears again.

  • Percentage true|false โ€” if set to true, the minimum and maximum values given are interpreted as percentage values, relative to the other data sources. This is helpful for example for the "df" type, where you may want to issue a warning when less than 5 % of the total space is available. Defaults to false. ๐Ÿ’–
  • Hits Number โ€” delays creating the notification until the threshold has been passed Number times. When a notification has been generated, or when a subsequent value is inside the threshold, the counter is reset. If, for example, a value is collected once every 10 seconds and Hits is set to 3, a notification will be dispatched at most once every 30 seconds.

This is useful when short bursts are not a problem. If, for example, 100% CPU usage for up to a minute is normal (and data is collected every 10 seconds), you could set Hits to 6 to account for this. ๐Ÿ”ฅ

  • Hysteresis Number โ€” when set to non-zero, a hysteresis value is applied when checking minimum and maximum bounds. This is useful for values that increase slowly and fluctuate a bit while doing so. When these values come close to the threshold, they may "flap", i.e. switch between failure/warning case and okay case repeatedly. ๐Ÿ˜ฉ

If, for example, the threshold is configured as

WarningMax 100.0
Hysteresis 1.0

then a Warning notification is created when the value exceeds 101, and the corresponding Okay notification is only created once the value falls below 99, thus avoiding the "flapping". Ate. ๐Ÿ’…


๐Ÿฆ‹ FILTER CONFIGURATION ๐Ÿฆ‹

Starting with collectd 4.6 there's a powerful filtering infrastructure implemented right in the daemon. The concept has mostly been copied from ip_tables, the packet filter infrastructure for Linux. We're using similar terminology so that users who are familiar with iptables feel right at home. ๐Ÿ 

๐ŸŽ€ Terminology ๐ŸŽ€

The following are the terms used in the remainder of the filter configuration documentation. For an ASCII-art schema of the mechanism, see "General structure" below.

Match

A match is a criteria to select specific values. Examples are, of course, the name of the value or its current value.

Matches are implemented in plugins which you have to load prior to using the match. The name of such plugins starts with the "match_" prefix. ๐ŸŒธ

Target

A target is some action that is to be performed with data. Such actions could, for example, be to change part of the value's identifier or to ignore the value completely.

Some of these targets are built into the daemon, see "Built-in targets" below. Other targets are implemented in plugins which you have to load prior to using the target. The name of such plugins starts with the "target_" prefix. ๐ŸŽฏ

Rule

The combination of any number of matches and at least one target is called a rule. The target actions will be performed for all values for which all matches apply. If the rule does not have any matches associated with it, the target action will be performed for all values. โœจ

Chain

A chain is a list of rules and possibly default targets. The rules are tried in order and if one matches, the associated target will be called. If a value is handled by a rule, it depends on the target whether or not any subsequent rules are considered, or if traversal of the chain is aborted โ€” see "Flow control" below. After all rules have been checked, the default targets will be executed. ๐Ÿฆ‹

๐ŸŒˆ General structure ๐ŸŒˆ

The following shows the resulting structure:

 +---------+
 ! Chain   !
 +---------+
      !
      V
 +---------+  +---------+  +---------+  +---------+
 ! Rule    !->! Match   !->! Match   !->! Target  !
 +---------+  +---------+  +---------+  +---------+
      !
      V
 +---------+  +---------+  +---------+
 ! Rule    !->! Target  !->! Target  !
 +---------+  +---------+  +---------+
      !
      V
      :
      :
      !
      V
 +---------+  +---------+  +---------+
 ! Rule    !->! Match   !->! Target  !
 +---------+  +---------+  +---------+
      !
      V
 +---------+
 ! Default !
 ! Target  !
 +---------+

โญ Flow control โญ

There are four ways to control which way a value takes through the filter mechanism:

  • jump โ€” the built-in jump target can be used to "call" another chain, i.e. process the value with another chain. When the called chain finishes, usually the next target or rule after the jump is executed.
  • stop โ€” the stop condition, signaled for example by the built-in target stop, causes all processing of the value to be stopped immediately. ๐Ÿ’€
  • return โ€” causes processing in the current chain to be aborted, but processing of the value generally will continue. This means that if the chain was called via Jump, the next target or rule after the jump will be executed. If the chain was not called by another chain, control will be returned to the daemon and it may pass the value to another chain.
  • continue โ€” most targets will signal the continue condition, meaning that processing should continue normally. There is no special built-in target for this condition. ๐Ÿ‘‘

๐Ÿ’œ Synopsis ๐Ÿ’œ

The configuration reflects this structure directly:

        PostCacheChain "PostCache"
        <Chain "PostCache">
          <Rule "ignore_mysql_show">
            <Match "regex">
              Plugin "^mysql$"
              Type "^mysql_command$"
              TypeInstance "^show_"
            </Match>
            <Target "stop">
            </Target>
          </Rule>
          <Target "write">
            Plugin "rrdtool"
          </Target>
        </Chain>

okay so bestie, here's what that example is actually doing ๐Ÿ’… โ€” it ignores all values where the plugin field is "mysql", the type is "mysql_command" and the type instance begins with "show_". Every other value gets sent to the "rrdtool" write plugin via the default target of the chain. And because this chain runs after the value has been added to the cache, the MySQL "show_*" command statistics are still gonna be available via the "unixsock" plugin. iconic behaviour, no notes โœจ

๐ŸŽ€ List of configuration options ๐ŸŽ€

PreCacheChain ChainName &nbsp;ยท&nbsp; PostCacheChain ChainName

These two set the name of the "pre-cache chain" and the "post-cache chain". The argument is the name of a chain that should be executed before and/or after the values have been added to the cache. ๐Ÿซถ

So to actually get the implications here, you gotta know what's going on inside collectd, fr fr. This diagram shows how values travel from the read-plugins to the write-plugins:

   +---------------+
   !  Read-Plugin  !
   +-------+-------+
           !
 + - - - - V - - - - +
 : +---------------+ :
 : !   Pre-Cache   ! :
 : !     Chain     ! :
 : +-------+-------+ :
 :         !         :
 :         V         :
 : +-------+-------+ :  +---------------+
 : !     Cache     !--->!  Value Cache  !
 : !     insert    ! :  +---+---+-------+
 : +-------+-------+ :      !   !
 :         !   ,------------'   !
 :         V   V     :          V
 : +-------+---+---+ :  +-------+-------+
 : !  Post-Cache   +--->! Write-Plugins !
 : !     Chain     ! :  +---------------+
 : +---------------+ :
 :                   :
 :  dispatch values  :
 + - - - - - - - - - +

So after the values are passed from the "read" plugins to the dispatch functions, the pre-cache chain runs first. The values get added to the internal cache after that. The post-cache chain runs after the values have been added to the cache. And likeโ€ฆ why is it such a huge deal whether chains run before or after the values get added to this cache? great question, glad you asked ๐Ÿ‘‘

Targets that change the identifier of a value list should be executed before the values are added to the cache, so that the name in the cache matches the name used in the "write" plugins. The "unixsock" plugin also uses this cache to receive a list of all available values. If you change the identifier after the value list has been added to the cache, that may easily lead to confusion โ€” but it's not forbidden, of course ๐Ÿ™ˆ

The cache is also used to convert counter values to rates. These rates are, for example, used by the "value" match (see below). If you use the rate stored in the cache before the new value is added, you'll be using the old, previous rate. Write plugins may use this rate too โ€” see the "csv" plugin, for example. The "unixsock" plugin uses these rates too, to implement the "GETVAL" command.

And last thing (but not last ๐Ÿ˜…), the stop target makes a difference: if the pre-cache chain returns the stop condition, the value will not be added to the cache, and the post-cache chain will not be run. periodt.


Chain Name

Adds a new chain with a certain name. This name can be used to refer to a specific chain, for example to jump to it. ๐Ÿฆ‹

Inside the Chain block, there can be Rule blocks and Target blocks.

Rule [Name]

Adds a new rule to the current chain. The name of the rule is optional and currently has no meaning for the daemon โ€” it's just there to vibe โœจ

Inside the Rule block there may be any number of Match blocks, and there must be at least one Target block.

Match Name

Adds a match to a Rule block. The name says what kind of match should be performed. Which matches are available depends on the plugins that have been loaded.

The arguments inside the Match block get passed to the plugin implementing the match, so which arguments are valid here depends on the plugin being used. If you don't need to pass any arguments to a match, you can use the shorter syntax:

Match "foobar"

Which is equivalent to:

<Match "foobar">
</Match>

Target Name

Adds a target to a rule, or a default target to a chain. The name says what kind of target is to be added. Which targets are available depends on the plugins being loaded.

The arguments inside the Target block get passed to the plugin implementing the target, so which arguments are valid here depends on the plugin being used. If you don't need to pass any arguments to a target, you can use the shorter syntax:

Target "stop"

This is the same as writing:

<Target "stop">
</Target>

๐Ÿ”ฅ Built-in targets ๐Ÿ”ฅ

The following targets are built into the core daemon, so they need no plugins to be loaded:

return

Signals the "return" condition, see the "Flow control" section above. This causes the current chain to stop processing the value and returns control to the calling chain. The calling chain will continue processing targets and rules just after the jump target (see below). This is very similar to the RETURN target of iptables, see iptables(8). ๐Ÿ‘‘

This target does not have any options.

Example:

Target "return"

stop

Signals the "stop" condition, see the "Flow control" section above. This causes processing of the value to be aborted immediately. This is similar to the DROP target of iptables, see iptables(8). ๐Ÿ’€

This target does not have any options.

Example:

Target "stop"

write

Sends the value to "write" plugins. ๐Ÿซถ

Available options:

  • Plugin Name โ€” name of the write plugin to which the data should be sent. This option may be given multiple times to send the data to more than one write plugin. If the plugin supports multiple instances, the plugin's instance(s) must also be specified.

If no plugin is explicitly specified, the values will be sent to all available write plugins.

Single-instance plugin example:

<Target "write">
  Plugin "rrdtool"
</Target>

Multi-instance plugin example:

<Plugin "write_graphite">
  <Node "foo">
  ...
  </Node>
  <Node "bar">
  ...
  </Node>
</Plugin>
 ...
<Target "write">
  Plugin "write_graphite/foo"
</Target>

jump

Starts processing the rules of another chain, see "Flow control" above. If the end of that chain is reached, or a stop condition is encountered, processing will continue right after the jump target, i. e. with the next target or the next rule. This is similar to the -j command line option of iptables, see iptables(8). ๐Ÿงš

Available options:

  • Chain Name โ€” jumps to the chain Name. This argument is required and may appear only once.

Example:

<Target "jump">
  Chain "foobar"
</Target>

๐ŸŒธ Available matches ๐ŸŒธ

regex

Matches a value using regular expressions. ๐ŸŽ€

Available options:

  • Host Regex
  • Plugin Regex
  • PluginInstance Regex
  • Type Regex
  • TypeInstance Regex
  • MetaData String Regex โ€” match values where the given regular expressions match the various fields of the identifier of a value. If multiple regular expressions are given, all regexen must match for a value to match.
  • Invert false|true โ€” when set to true, the result of the match is inverted, i. e. all value lists where all regular expressions apply are not matched, and all other value lists are matched. Defaults to false.

Example:

<Match "regex">
  Host "customer[0-9]+"
  Plugin "^foobar$"
</Match>

timediff

Matches values that have a time which differs from the time on the server. โฐ

This match is mainly intended for servers that receive values over the "network" plugin and write them to disk using the "rrdtool" plugin. RRDtool is very sensitive to the timestamp used when updating the RRD files. In particular, the time must be ever increasing. If a misbehaving client sends one packet with a timestamp far in the future, all further packets with a correct time will be ignored because of that one packet. And what's worse? Such corrupted RRD files are hard to fix. ๐Ÿ˜ญ

This match lets you match all values outside a specified time range (relative to the server's time), so you can use the stop target (see below) to ignore the value, for example.

Available options:

  • Future Seconds โ€” matches all values that are ahead of the server's time by Seconds or more seconds. Set to zero for no limit. Either Future or Past must be non-zero.
  • Past Seconds โ€” matches all values that are behind of the server's time by Seconds or more seconds. Set to zero for no limit. Either Future or Past must be non-zero.

Example:

<Match "timediff">
  Future  300
  Past   3600
</Match>

This example matches all values that are five minutes or more ahead of the server, or one hour (or more) lagging behind. ๐Ÿ•

value

Matches the actual value of data sources against given minimum / maximum values. If a data-set consists of more than one data-source, all data-sources must match the specified ranges for a positive match. โœจ

Available options:

  • Min Value โ€” sets the smallest value which still results in a match. If unset, behaves like negative infinity.
  • Max Value โ€” sets the largest value which still results in a match. If unset, behaves like positive infinity.
  • Invert true|false โ€” inverts the selection. If the Min and Max settings result in a match, no-match is returned and vice versa. Please note that the Invert setting only effects how Min and Max are applied to a specific value. Especially the DataSource and Satisfy settings (see below) are not inverted.
  • DataSource DSName [DSName ...] โ€” pick one or more data sources. If you don't configure any, all data sources get checked. And if the match's type doesn't even have a data source with the name(s) you gave, that's always a no-match (even if Invert is set, which is wild).
  • Satisfy Any|All โ€” tells collectd how to combine checks across several data sources. Set it to Any and the match succeeds if one data source is in the range. Set it to All and the match only succeeds if every data source is in the range. Default is All.

Usually All is for positive matches, Any is for negative matches. Like, with All you're checking that all values are "good", while with Any you're checking if any value is "bad" (or outside the "good" range).

Either Min or Max, but not both, may be unset.

Example:

# Match all values smaller than or equal to 100. Matches only if all data
# sources are below 100.
<Match "value">
  Max 100
  Satisfy "All"
</Match>

# Match if the value of any data source is outside the range of 0 - 100.
<Match "value">
  Min   0
  Max 100
  Invert true
  Satisfy "Any"
</Match>

empty_counter

Matches all values that have one or more data sources of type COUNTER and where all counter values are zero. These counters usually never increased since they started existing (so they're kinda uninteresting), or they got reset recently or overflowed and you had really, really bad luck.

bestie, keep in mind that ignoring these counters can cause confusing behavior: counters that hardly ever increase will be zero for long periods. If the counter gets reset for some reason (machine or service restarted, usually), the graph will be empty (NAN) for a long time. People may not understand why. ๐Ÿ˜ญ

hashed

Calculates a hash value of the host name and matches values according to that hash. This lets you divide all hosts into groups and only match values in a specific group. The intended use is load balancing โ€” you want to handle only part of all data and leave the rest for other servers. ๐Ÿ’…

The hashing function tries to distribute hosts evenly. First, it calculates a 32-bit hash value using the characters of the hostname:

hash_value = 0;
for (i = 0; host[i] != 0; i++)
  hash_value = (hash_value * 251) + host[i];

The constant 251 is a prime number that's supposed to make this hash more random. Then the code checks the group for this host according to the Total and Match arguments:

if ((hash_value % Total) == Match)
  matches;
else
  does not match;
Please note: when you set Total to two (i.e. you have only two groups), then the least significant bit of the hash value will be the XOR of all least significant bits in the host name. One consequence: if you have two hosts, "server0.example.com" and "server1.example.com", where the host name differs in one digit only and the digits differ by one, those hosts will never end up in the same group. wild, right? ๐Ÿซ 

Available options:

  • Match Match Total โ€” divide the data into Total groups and match all hosts in group Match as described above. Groups are numbered from zero, so Match must be smaller than Total. Total must be at least one, although only values greater than one really make sense.

You can repeat this option to match multiple groups, like:

Match 3 7
Match 5 7

The above config divides the data into seven groups and matches groups three and five. One use case: keep every value on two hosts so that if one fails, the missing data can later be reconstructed from the second host. big brain energy ๐Ÿง โœจ

Example:

# Operate on the pre-cache chain, so that ignored values are not even in the
# global cache.
<Chain "PreCache">
  <Rule>
    <Match "hashed">
      # Divide all received hosts in seven groups and accept all hosts in
      # group three.
      Match 3 7
    </Match>
    # If matched: Return and continue.
    Target "return"
  </Rule>
  # If not matched: Return and stop.
  Target "stop"
</Chain>

Available targets

notification

Creates and dispatches a notification. ๐Ÿ“ฃ

Available options:

  • Message String โ€” this required option sets the message of the notification. The following placeholders get replaced with an appropriate value:
    • %{host}
    • %{plugin}
    • %{plugin_instance}
    • %{type}
    • %{type_instance}

These placeholders are replaced by the identifier field of the same name.

  • %{ds:name}

These placeholders are replaced by a (hopefully) human-readable representation of the current rate of this data source. If you changed the instance name (using the set or replace targets, see below), it may not be possible to convert counter values to rates.

Please note that these placeholders are case sensitive! ๐Ÿ™ˆ
  • Severity "FAILURE"|"WARNING"|"OKAY" โ€” sets the severity of the message. If omitted, the severity "WARNING" is used.

Example:

<Target "notification">
  Message "Oops, the %{type_instance} temperature is currently %{ds:value}!"
  Severity "WARNING"
</Target>

replace

Replaces parts of the identifier using regular expressions. โœ‚๏ธ

Available options:

  • Host Regex Replacement
  • Plugin Regex Replacement
  • PluginInstance Regex Replacement
  • TypeInstance Regex Replacement
  • MetaData String Regex Replacement
  • DeleteMetaData String Regex

Match the appropriate field with the given regular expression Regex. If the regex matches, the part that matches is replaced with Replacement. If multiple places of the input buffer match a given regex, only the first occurrence will be replaced.

You can specify each option multiple times to use multiple regular expressions one after another. slay ๐Ÿ’…

Example:

<Target "replace">
  # Replace "example.net" with "example.com"
  Host "\\<example.net\\>" "example.com"

  # Strip "www." from hostnames
  Host "\\<www\\." ""
</Target>

set

Sets part of the identifier of a value to a given string. ๐Ÿ”ง

Available options:

  • Host String
  • Plugin String
  • PluginInstance String
  • TypeInstance String
  • MetaData String String

Set the appropriate field to the given string. The strings for plugin instance, type instance, and meta data may be empty; the strings for host and plugin may not be empty. It's currently not possible to set the type of a value this way.

The following placeholders get replaced with an appropriate value:

  • %{host}
  • %{plugin}
  • %{plugin_instance}
  • %{type}
  • %{type_instance}

These placeholders are replaced by the identifier field of the same name.

  • %{meta:name}

These placeholders are replaced by the meta data value with the given name.

Please note that these placeholders are case sensitive! ๐Ÿ™ˆ
  • DeleteMetaData String โ€” delete the named meta data field.

Example:

<Target "set">
  PluginInstance "coretemp"
  TypeInstance "core3"
</Target>

Backwards compatibility

If you use collectd with an old configuration โ€” i.e. one without a Chain block โ€” it will behave as it used to. That's equivalent to this config:

<Chain "PostCache">
  Target "write"
</Chain>

If you specify a PostCacheChain, the write target will not be added anywhere, and you'll have to make sure it's called where appropriate. We suggest adding the above snippet as the default target to your "PostCache" chain. ๐Ÿ’–

Examples

Ignore all values where the hostname does not contain a dot, i.e. can't be an FQDN.

<Chain "PreCache">
  <Rule "no_fqdn">
    <Match "regex">
      Host "^[^\.]*$"
    </Match>
    Target "stop"
  </Rule>
  Target "write"
</Chain>

IGNORELISTS

Ignorelists are a generic framework to either ignore some metrics or report only specific metrics. Plugins usually provide one or more options to specify the items (mount points, devices, ...) and the boolean option "IgnoreSelected". ๐Ÿšซ

  • Select String โ€” selects the item String. This option often has a plugin-specific name, e.g. Sensor in the "sensors" plugin. It's also plugin-specific what this string is compared to. For example, the "df" plugin's MountPoint compares it to a mount point, and the "sensors" plugin's Sensor compares it to a sensor name.

By default, this option does a case-sensitive full-string match. The following config will match "foo", but not "Foo":

Select "foo"

If String starts and ends with / (a slash), the string is compiled as a regular expression. For example, to match all items starting with "foo", you could use:

Select "/^foo/"

The regular expression is not anchored, i.e. the following config will match "foobar", "barfoo" and "AfooZ":

Select "/foo/"

The Select option may be repeated to select multiple items. ๐Ÿ’…

  • IgnoreSelected true|false โ€” if set to true, matching metrics are ignored and all other metrics are collected. If set to false, matching metrics are collected and all other metrics are ignored.

SEE ALSO

collectd(1), collectd-exec(5), collectd-perl(5), collectd-unixsock(5), types.db(5), hddtemp(8), iptables(8), kstat(3KSTAT), mbmon(1), psql(1), regex(7), rrdtool(1), sensors(1)

AUTHOR

Florian Forster <octo@collectd.org>


and that's a wrap on COLLECTD.CONF(5)! ๐ŸŽ€โœจ this page is from version 5.12.0.348.g93f9bdcb dated 2024-12-18. now go forth and configure your collectd like the main character you are ๐Ÿ’…๐Ÿ‘‘

๐Ÿ“ฆ where this page comes from