COLLECTD-PYTHON(5)

📦
version 5.12.0 · 10 versions
, in ubuntu stonking

💅 COLLECTD-PYTHON(5) — the collectd python plugin, explained by your favorite gremlin 🐍✨

okay bestie, sit down, grab a strawberry, because this is the manual page for collectd-python — yes, that's COLLECTD-PYTHON(5) from the collectd section, and it's lowkey iconic. it's the doc for collectd's python plugin, and i'm gonna walk you through every single bit of it. no cap, nothing skipped. 🫶

✨ NAME ✨

collectd-python — the documentation of collectd's python plugin. that's it. that's the name. we love a concise queen. 👑

🌸 SYNOPSIS 🌸

here's the vibe in code form, verbatim, do not touch:

LoadPlugin python
# ...
<Plugin python>
  ModulePath "/path/to/your/python/modules"
  LogTraces true
  Interactive false
  Import "spam"

  <Module spam>
    spam "wonderful" "lovely"
  </Module>
</Plugin>

🦋 DESCRIPTION 🦋

so the python plugin literally embeds a whole Python interpreter inside collectd, and it gives you an interface into collectd's plugin system. translation: you can write collectd plugins in Python now. it's giving convenience. 💖

and it's so much more efficient than running a Python script every single time you wanna read a value with the exec plugin (peep collectd-exec(5) for that one, iykyk). plus it gives you way more functionality too. main character behaviour honestly. 🔥

the minimum required Python version is 2.6. like, that's the floor. periodt.

🎀 CONFIGURATION 🎀

these are the config options, and yes, we're doing all of them. 💅

  • LoadPlugin Plugin — loads the Python plugin Plugin. that's the whole thing. slay.
  • Encoding Name — this sets the default encoding for Unicode objects you pass to collectd. if you just… don't set it, it defaults to ascii on Python 2. on Python 3 it will always be utf-8, because that function was removed, so this option just gets silently ignored. 🙈 and bestie, these defaults are hardcoded in Python, so they ignore everything else — including your locale. yes, even that.
  • ModulePath Name — prepends Name to sys.path. real talk: you literally will not be able to import any scripts you wrote unless they sit in one of the directories in this list. also note it only has effect on plugins loaded after this option. you can stack multiple ModulePath lines to add more than one directory. 🧚
  • LogTraces bool — if a Python script throws an exception, collectd logs it with the name of the exception and the message. if you set this option to true, it'll also log the full stacktrace, exactly like the default output of an interactive Python interpreter. this does not apply to the CollectError exception — that one never logs a stacktrace, no matter what. probably should be false most of the time, but it's genuinely so useful for developing and debugging new modules. ⭐
  • Interactive bool — this one makes the module launch an interactive Python interpreter that reads from and writes to the terminal. ⚠️ huge caveat: collectd will terminate right after starting up if you try to run it as a daemon while this option is enabled, so make sure you start collectd with the -f option. the collectd module is not imported into the interpreter's globals — you gotta do that manually. be sure to read the help text of the module, it can be used as a reference guide while you're coding.

this interactive session behaves slightly differently from a daemonized collectd script and from a normal Python interpreter:

  1. collectd will try to import the readline module to give you a decent way of entering your commands. the daemonized collectd won't do that. 🎀
  2. Python will be handling SIGINT. pressing Ctrl+C usually causes collectd to shut down — which would be a whole problem in an interactive session, so Python handles it here instead. this lets you use Ctrl+C to interrupt Python code without killing collectd. it also means you can catch KeyboardInterrupt exceptions, which does not work during normal operation. to quit collectd, send EOF (press Ctrl+D at the beginning of a new line). 💀
  3. collectd handles SIGCHLD. this means Python won't be able to determine the return code of spawned processes with system(), popen() and subprocess. the result? Python won't use external programs like less to display help texts. you can override this behavior with the PAGER environment variable, e.g. export PAGER=less before starting collectd. depending on your version of Python, this might or might not result in an OSError exception, which can be ignored. 🙈

if you really need to spawn new processes from Python, you can register an init callback and reset the action for SIGCHLD to the default behavior. ⚠️ but please note: this will break the exec plugin. do not even load the exec plugin if you intend to do this. like, seriously.

there's an example script located in contrib/python/getsigchld.py to do exactly this. if you import this from collectd.conf, SIGCHLD will be handled normally and spawning processes from Python will work as intended. 🦋

  • Import Name — imports the python script Name and loads it into the collectd python process. if your python script is not found, be sure its directory exists in python's sys.path — you can prepend to sys.path using the ModulePath configuration option. 💖
  • <Module Name> block — this block may be used to pass on configuration settings to a Python module. the configuration is converted into an instance of the Config class, which is passed to the registered configuration callback. see below for details about the Config class and how to register callbacks. the name identifies the callback. ✨

🌈 STRINGS 🌈

okay this section is important, pay attention. there are a lot of places where strings get sent from collectd to Python and from Python to collectd. how exactly that works depends on whether byte or unicode strings are used, and whether it's Python2 or Python3. 😭

quick recap of the lore: Python2 has str, which is just bytes, and unicode. Python3 has str, which is a unicode object, and bytes.

Python → collectd: all of these objects are supported in all places, however str should be used if possible. these strings must not contain a NUL byte — ignoring this will result in a TypeError exception. if a byte string was used, it'll be used as-is by collectd. if a unicode object was used, it'll be encoded using the default encoding (see above). if that's not possible, Python raises a UnicodeEncodeError exception. 💅

collectd → Python: the behavior depends on the Python version you're using. Python2 will always receive a str object. Python3 will usually receive a str object as well — however, the original string will be decoded to unicode using the default encoding. if that fails because the string isn't a valid sequence for this encoding, a bytes object will be returned instead. 🦋

👑 WRITING YOUR OWN PLUGINS 👑

writing your own plugins is, like, genuinely simple. collectd manages plugins by means of dispatch functions, which call the appropriate callback functions registered by the plugins. any plugin basically consists of the implementation of these callback functions plus initializing code that registers the functions with collectd. see the EXAMPLES section below for a really basic example. 🔥

these are the types of callback functions collectd knows about — and bestie, all of them are optional:

  • configuration functions — these are called during configuration if an appropriate Module block has been encountered. it's called once for each Module block that matches the name of the callback as provided with the register_config method — see below. ⚠️ Python thread support has not been initialized at this point, so do not use any threading functions here!
  • init functions — these are called once after loading the module and before any calls to the read and write functions. it should be used to initialize the internal state of the plugin (e.g. open sockets, …). this is the earliest point where you may use threads. 🧵
  • read functions — these are used to collect the actual data. it's called once per interval (see the Interval configuration option of collectd). usually it'll call plugin_dispatch_values to dispatch the values to collectd, which will pass them on to all registered write functions. if this function throws any kind of exception, the plugin will be skipped for an increasing amount of time until it returns normally again. 😭
  • write functions — these are used to write the dispatched values. called once for every value that was dispatched by any plugin. 💖
  • flush functions — these are used to flush internal caches of plugins. usually triggered by the user only. any plugin which caches data before writing it to disk should provide this kind of callback function. 🎀
  • log functions — these are used to pass messages of plugins or the daemon itself to the user. 🌸
  • notification function — these are used to act upon notifications. in general, a notification is a status message that may be associated with a data instance. usually a notification is generated by the daemon if a configured threshold has been exceeded (see the section THRESHOLD CONFIGURATION in collectd.conf(5) for more details), but any plugin may dispatch notifications as well. ⭐
  • shutdown functions — these are called once before the daemon shuts down. should be used to clean up the plugin (e.g. close sockets, …). 💅

any function (except log functions) may throw an exception in case of errors. the exception will be passed on to the user using collectd's logging mechanism. if a log callback throws an exception, it'll be printed to standard error instead. 🫶

see the documentation of the various register_ methods in the FUNCTIONS section below for the number and types of arguments passed to each callback function. that section also explains how to register callback functions with collectd. 📚

to enable a module: copy it to a place where Python can find it (i.e. a directory listed in sys.path) just as any other Python plugin, and add an appropriate Import option to the configuration file. after restarting collectd, you're done. slay. 🦋✨

✨ CLASSES ✨

okay bestie, these are the complex types used to pass values back and forth between the Python plugin and collectd. they're literally giving bridge-between-two-worlds energy. 🌉💖

💖 CollectdError

This one is an exception. If any Python script raises this exception, collectd will still treat it like an error, but it gets logged as a warning instead of an error, and it will never generate a stacktrace. no cap, that's the whole point. 🫶

class CollectdError(Exception)

Basic exception for collectd Python scripts. Throwing this exception will not cause a stacktrace to be logged, even if LogTraces is enabled in the config. periodt.

🎀 Signed

The Signed class is just a long. It has all its methods and behaves exactly like any other long object. It is used to indicate if an integer was or should be stored as a signed or unsigned integer object. lowkey iconic. ✨

class Signed(long)

This is a long by another name. Use it in meta data dicts to choose the way it is stored in the meta data. 💅

🦋 Unsigned

The Unsigned class is just a long. It has all its methods and behaves exactly like any other long object. It is used to indicate if an integer was or should be stored as a signed or unsigned integer object. same energy, different signedness. 🌈

class Unsigned(long)

This is a long by another name. Use it in meta data dicts to choose the way it is stored in the meta data. 💜

🎀 Config

The Config class is an object which keeps the information provided in the configuration file. The sequence of children keeps one entry for each configuration option. Each such entry is another Config instance, which may nest further if nested blocks are used. it's giving config-tree realness. 🌳✨

class Config(object)

This represents a piece of collectd's config file. It is passed to scripts with config callbacks (see register_config) and is of little use if created somewhere else. bestie, don't just make one in the wild, okay? 🙈

It has no methods beyond the bare minimum and only exists for its data members.

🎀 Data descriptors defined here:

  • parent — This represents the parent of this node. On the root node of the config tree it will be None.
  • key — This is the keyword of this item, i.e. the first word of any given line in the config file. It will always be a string.
  • values — This is a tuple (which might be empty) of all value, i.e. words following the keyword in any given line in the config file. Every item in this tuple will be either a string, a float or a boolean, depending on the contents of the configuration file.
  • children — This is a tuple of child nodes. For most nodes this will be empty. If this node represents a block instead of a single line of the config file it will contain all nodes in this block.

🫶 PluginData

This should not be used directly, but it is the base class for both Values and Notification. It is used to identify the source of a value or notification. it's the backstage pass, not the main character. 👑

class PluginData(object)

This is an internal class that is the base for Values and Notification. It is pretty useless by itself and was therefore not exported to the collectd module.

🎀 Data descriptors defined here:

  • host — The hostname of the host this value was read from. For dispatching this can be set to an empty string which means the local hostname as defined in collectd.conf.
  • plugin — The name of the plugin that read the data. Setting this member to an empty string will insert "python" upon dispatching.
  • plugin_instance — Plugin instance string. May be empty.
  • time — This is the Unix timestamp of the time this value was read. For dispatching values this can be set to zero which means "now". This means the time the value is actually dispatched, not the time it was set to 0.
  • type — The type of this value. This type has to be defined in your types.db. Attempting to set it to any other value will raise a TypeError exception. Assigning a type is mandatory, calling dispatch without doing so will raise a RuntimeError exception.
  • type_instance — Type instance string. May be empty.

🌟 Values

A Value is an object which features a sequence of values. It is based on the PluginData type and uses its members to identify the values. it's literally the data-delivery bestie. 🚚💨

class Values(PluginData)

A Values object used for dispatching values to collectd and receiving values from write callbacks.

Method resolution order:

  • Values
  • PluginData
  • object

🛠️ Methods defined here:

dispatch([type][, values][, plugin_instance][, type_instance][, plugin][, host][, time][, interval]) -> None.

Dispatch this instance to the collectd process. The object has members for each of the possible arguments for this method. For a detailed explanation of these parameters see the member of the same same.

If you do not submit a parameter the value saved in its member will be submitted. If you do provide a parameter it will be used instead, without altering the member. slay, no side effects. ✨

write([destination][, type][, values][, plugin_instance][, type_instance][, plugin][, host][, time][, interval]) -> None.

Write this instance to a single plugin or all plugins if destination is omitted. This will bypass the main collectd process and all filtering and caching. Other than that it works similar to dispatch. In most cases dispatch should be used instead of write. fr fr, dispatch is usually the move. 💅

🎀 Data descriptors defined here:

  • interval — The interval is the timespan in seconds between two submits for the same data source. This value has to be a positive integer, so you can't submit more than one value per second. If this member is set to a non-positive value, the default value as specified in the config file will be used (default: 10). If you submit values more often than the specified interval, the average will be used. If you submit less values, your graphs will have gaps. 😭📉
  • values — These are the actual values that get dispatched to collectd. It has to be a sequence (a tuple or list) of numbers. The size of the sequence and the type of its content depend on the type member your types.db file. For more information on this read the types.db(5) manual page. If the sequence does not have the correct size upon dispatch a RuntimeError exception will be raised. If the content of the sequence is not a number, a TypeError exception will be raised. bestie, match the shape. 🧩
  • meta — These are the meta data for this Value object. It has to be a dictionary of numbers, strings or bools. All keys must be strings. int and <long> objects will be dispatched as signed integers unless they are between 2**63 and 2**64-1, which will result in a unsigned integer. You can force one of these storage classes by using the classes collectd.Signed and collectd.Unsigned. A meta object received by a write callback will always contain Signed or Unsigned objects. 🏷️✨

🔥 Notification

A notification is an object defining the severity and message of the status message as well as an identification of a data instance by means of the members of PluginData on which it is based. it's giving "hey bestie, something happened" energy. 🚨💬

class Notification(PluginData)

The Notification class is a wrapper around the collectd notification. It can be used to notify other plugins about bad stuff happening. It works similar to Values but has a severity and a message instead of interval and time. Notifications can be dispatched at any time and can be received with register_notification. 👑📣

Method resolution order:

  • Notification
  • PluginData
  • object

🛠️ Methods defined here:

dispatch([type][, message][, plugin_instance][, type_instance][, plugin][, host][, time][, severity][, meta]) -> None.

Dispatch a notification. Dispatch this instance to the collectd process. The object has members for each of the possible arguments for this method. For a detailed explanation of these parameters see the member of the same same.

If you do not submit a parameter the value saved in its member will be submitted. If you do provide a parameter it will be used instead, without altering the member. same rules, iconic consistency. 💅

🎀 Data descriptors defined here:

  • message — Some kind of description of what's going on and why this Notification was generated. spill the tea. ☕
  • severity — The severity of this notification. Assign or compare to NOTIF_FAILURE, NOTIF_WARNING or NOTIF_OKAY. choose your vibe check wisely. ⚠️✅
  • meta — These are the meta data for the Notification object. It has to be a dictionary of numbers, strings or bools. All keys must be strings. int and long objects will be dispatched as signed integers unless they are between 2**63 and 2**64-1, which will result in a unsigned integer. One of these storage classes can be forced by using the classes collectd.Signed and collectd.Unsigned. A meta object received by a notification callback will always contain Signed or Unsigned objects. 🏷️🔥

🫶 FUNCTIONS 🫶

Okay bestie, these are the actual C-to-Python handshake functions. This is the whole interface the daemon gives you, so pay attention 💅

✨ register_*(callback[, data][, name]) -> identifier

So there are eight different register functions, one for each event you can hook into. With exactly one exception (we'll get to it, it's register_read, stay tuned), they all get called like the line above.

  • callback — a callable object 💖 it gets called every single time the event fires.
  • data — an optional object that gets passed right back to your callback every time it's called. Ngl this one's a little sneaky: if you omit this parameter, no object is passed back to your callback at all — not even None. So don't go assuming you'll get a None handed to you 🫠
  • name — an optional identifier for this callback. The default name is python.module, and module gets pulled from the __module__ attribute of your callback function. Every callback needs a unique identifier, so if you wanna register the same callback multiple times in the same module, you have to pass a name here. Otherwise you can safely ignore this parameter, fr fr.
  • identifier — the full identifier that got assigned to this callback.

These functions get called at the various stages of the daemon's life (peep the "WRITING YOUR OWN PLUGINS" section above 👀) and here's exactly what arguments each one hands over:

register_config

The only argument passed is a Config object. Scroll up for the layout of this data type. Important caveat: you cannot receive the whole config files this way — only Module blocks inside the Python configuration block. And on top of that, you'll only receive blocks where your callback identifier matches python.blockname. Sorry babe, that's just how it is 🎀

register_init

The callback gets called with no arguments.

register_read(callback[, interval][, data][, name]) -> identifier

Ahh here's the exception I mentioned 👑 this one takes an extra parameter: interval. It specifies the time between calls to the callback function.

The callback itself gets called with no arguments.

register_shutdown

The callback gets called with no arguments.

register_write

The callback function gets called with one argument passed, and that argument is a Values object. For the layout of Values, see above 👆 And heads up: if this callback function throws an exception, the next call will be delayed by an increasing interval. So like, don't just yeet errors into the void 💀

register_flush

Like register_config, this one's important for register_flush because it determines what flush requests the plugin will receive.

The arguments passed are timeout and identifier. timeout indicates that only data older than timeout seconds is to be flushed. identifier specifies which values are to be flushed.

register_log

The arguments are severity and message. severity is an integer — small for important messages and high for less important messages (yes, backwards from what your brain wants, I know 😭). The least important level is LOG_DEBUG, the most important level is LOG_ERR. In between, from least to most important, you've got: LOG_INFO, LOG_NOTICE, and LOG_WARNING. And message is simply a string with no newline at the end.

🫠 If this callback throws an exception, it will not be logged. It just gets printed to sys.stderr, which usually means it's silently ignored. So like... your error handling could literally vanish into the void. Be aware.

register_notification

The only argument passed is a Notification object. See above for the layout of this data type ✨

🦋 unregister_*(identifier) -> None

Removes a callback or a data-set from collectd's internal list of callback functions. Every single register_* function has a matching unregister_* function — cute symmetry, love that for us 💅

identifier is either the string that was returned by the register function or a callback function. The identifier gets constructed the same way as it does for the register functions.

🌸 get_dataset(name) -> definition

Returns the definition of a dataset specified by name. definition is a list of tuples, and each tuple represents one data source. Every tuple has 4 values:

name

A string — the name of the data source.

type

A string equal to either of these variables: DS_TYPE_COUNTER, DS_TYPE_GAUGE, DS_TYPE_DERIVE, or DS_TYPE_ABSOLUTE. Iconic quartet honestly ⭐

min

A float or None — the minimum value.

max

A float or None — the maximum value.

💜 flush(plugin[, timeout][, identifier]) -> None

Flush one plugin or all the plugins. timeout and the specified identifiers get passed on to the registered flush-callbacks. If you omit them, timeout defaults to -1 and identifier defaults to None. And if you do specify the plugin argument, only that named plugin gets flushed 🫶

🔥 error, warning, notice, info, debug(message)

Log a message with the specified severity. That's it, that's the whole function. Log away bestie 📝

🌈 EXAMPLES 🌈

Every Python module starts off kinda like this:

import collectd

A very simple read function might look like:

import random

def read(data=None):
  vl = collectd.Values(type='gauge')
  vl.plugin='python.spam'
  vl.dispatch(values=[random.random() * 100])

And a very simple write function might look like:

def write(vl, data=None):
  for i in vl.values:
    print "%s (%s): %f" % (vl.plugin, vl.type, i)

To register those functions with collectd:

collectd.register_read(read)
collectd.register_write(write)

And for the complete documentation of the data types used by the read, write and match functions, go check out the "CLASSES" section above 👑✨

💀 CAVEATS 💀

Okay this part is important, don't scroll past it 🙈

  • collectd is heavily multi-threaded. Every single collectd thread that touches the Python plugin gets mapped to a Python interpreter thread. Any such thread gets created and destroyed transparently and on-the-fly.

So like, any plugin has to be thread-safe if it provides several entry points from collectd — that means if it registers more than one callback, or if a registered callback might get called more than once in parallel. No cap, this will bite you 🫠

  • The Python thread module is initialized just before the init callbacks get called. Which means you must not use Python's threading module prior to this point. And that includes all config callbacks and possibly other callbacks too. Yes, really. All of them 😭
  • The python plugin exports the internal API of collectd, and that API is considered unstable and subject to change at any time. We try really hard not to break backwards compatibility in the Python API during the life cycle of one major release — but ngl, this cannot be guaranteed at all times. So watch out for warnings dispatched by the python plugin after upgrades 👀🔔

🎀 KNOWN BUGS 🎀

  • Not all aspects of the collectd API are accessible from Python. This includes but is not limited to filters. Yeah. Filters. We know. 💔

⭐ SEE ALSO ⭐

collectd(1), collectd.conf(5), collectd-perl(5), collectd-exec(5), types.db(5), python(1)

👑 AUTHOR 👑

The "python plugin" has been written by Sven Trenkel <collectd at semidefinite.de>.

This manpage has been written by Sven Trenkel <collectd at semidefinite.de>. It is based on the collectd-perl(5) manual page by Florian Forster <octo at collectd.org> and Sebastian Harl <sh at tokkee.org>.

and that's a wrap bestie!! 🦋💖✨ this lil masterpiece is version 5.12.0.git, dated 2026-08-22, and it lives in section COLLECTD-PYTHON(5) of the manual. go forth and write some thread-safe plugins, ilysm 🫶🎀

📦 where this page comes from