PMLOGGER(1)

📦
version 7.2.2 · 8 versions
, in arch rolling, fedora 43, fedora 44

🌸 PMLOGGER(1) — General Commands Manual 🌸

okay bestie, sit down, hydrate, and let me tell you about pmlogger — the PCP tool that makes performance metric archives. yes, it's a General Commands Manual page. yes, it's long. no, we are not skipping anything. 💅

✨ NAME ✨

pmlogger — create an archive for performance metrics

✨ SYNOPSIS ✨

pmlogger [-CLNoPruy?]  [-c conffile] [-D debug] [-d directory] [-h host]
[-H hostname] [-I version] [-K spec] [-l logfile] [-m note] [-n
pmnsfile] [-p pid] [-s endsize] [-t interval] [-T endtime] [-U username]
[-v volsize] [-V version] [-x fd] [archive]

✨ DESCRIPTION ✨

so like, pmlogger creates the archives of performance metric values that other Performance Co-Pilot tools can later ``play back'' (yes, that's the actual vibe — see PCPIntro(1)). These logs are literally the whole foundation of the VCR paradigm and the retrospective performance analysis services that the entire PCP toolkit is known for. iconic behavior tbh. 📼✨

The mandatory argument archive is either:

  • the base name for the physical files that make up an archive on the local filesystem, or
  • the remote HTTP server that's gonna receive the logged data.

The local archive argument is allowed to contain strftime(3) meta-characters, and those get substituted before the archive files are created. When pmlogger runs as a service (peep pmlogger_daily(1)), the standard archive base name template is %Y%m%d.%H.%M.

The remote archive form tells you where the pmproxy(1) server is — that's the server that all the archive components get sent to as they're being generated by pmlogger. The value you give must contain protocol, hostname, and an optional port, e.g. http://example.com:44322. You can use an https:// prefix instead to encrypt the connection with TLS; in that case the receiving pmproxy(1) must have a TLS certificate configured, and client-side TLS parameters get read from $PCP_TLSCONF_PATH (typically /etc/pcp/tls.conf).

🚨 bestie listen: the server is always authenticated before any data is sent. The pmproxy certificate must chain to a certificate authority trusted via $PCP_TLSCONF_PATH, and its subject (or a subject alternative name) must match the hostname in the https:// connection string. Otherwise the connection is refused — it does not fall back to an unauthenticated or cleartext transfer. no cap. 🙅♀️

That pmproxy server will pick an archive name based on the start time, kinda like the model used by default in pmlogger_check(1).

When you're pushing to a remote pmproxy(1) that wants per-client authentication, the --push-user option names the user to authenticate as with HTTP Basic authentication — this is your alternative to client-certificate (mutual TLS) authentication. The password gets read from the file named by --push-password-file or, if that's not given, from the $PCP_PUSH_PASSWORD environment variable. It is never accepted on the command line. And because HTTP Basic credentials are only base64 encoded and not encrypted, --push-user requires an https:// connection — sending credentials over a cleartext or unauthenticated connection is refused. 💀

The -V option specifies the version for the archive that gets generated. By default the archive version $PCP_ARCHIVE_VERSION is used (set to 3 in current PCP releases), and the only values currently supported for version are 2 or 3.

Unless you direct it to another host with the -h option, or you're directly using PMDAs via the -o option, pmlogger will contact the Performance Metrics Collector Daemon (PMCD) on the local host and use that as the source of the metric values to be logged.

🧠 the two-level logging state situation (iykyk)

To support all the flexibility and control over what gets logged and when, pmlogger keeps an independent two level logging state for each instance of each performance metric. Let's break it down:

  • First (mandatory) level: logging is allowed to be on (with an associated interval between samples), or off, or maybe.
  • Second (advisory) level: in the maybe case, logging is allowed to be on (with an associated interval between samples), or off.

The mandatory level lets you universally specify that some metrics must be logged, or must not be logged. The default state for all instances of all metrics when pmlogger starts is mandatory maybe and advisory off.

Use pmlc(1) to interrogate and change the logging state once pmlogger is running. 🎛️

Now the ✨rules✨:

  • If a metric's state is mandatory (on or off) and a request is made to change it to mandatory maybe, the new state is mandatory maybe and advisory off.
  • If a metric's state is already advisory (on or off) and a request is made to change it to mandatory maybe, the current state is retained.

And the big one — it is not possible for pmlogger to log specific instances of a metric and all instances of the same metric concurrently:

  • If specific instances are being logged and a request to log all instances comes in, then all instances of the metric get logged according to the new request, superseding any prior logging request for that metric.
  • A request to log all instances of a metric will supersede any previous request to log all instances.
  • A request to log specific instances of a metric when all instances are already being logged is refused. To do that, you must turn off logging for all instances of the metric first.

In each case, the validity of the request is checked first. For example, a request to change a metric's logging state to advisory on when it's currently mandatory off is never permitted — you have to change the state to mandatory maybe first. 😤

👑 the "primary" pmlogger situation

Optionally, each system running pmcd(1) may also be configured to run a ``primary'' pmlogger instance. This instance is launched by $PCP_RC_DIR/pmlogger, and it's affected by these files:

  • $PCP_SYSCONF_DIR/pmlogger/control
  • $PCP_SYSCONF_DIR/pmlogger/control.d/*
  • $PCP_SYSCONFIG_DIR/pmlogger — environment variable settings for the primary pmlogger
  • $PCP_SYSCONF_DIR/pmlogger/pmlogger.options — command line options passed to the primary pmlogger
  • $PCP_VAR_DIR/config/pmlogger/config.default — the default initial configuration file for the primary pmlogger

(and psst, use chkconfig(8), systemctl(1) or similar platform-specific commands to activate or disable the primary pmlogger instance)

The primary pmlogger instance is identified by the -P option. There may be at most one ``primary'' pmlogger instance on each system. The primary instance (if there is one) must be running on the same host as the pmcd(1) it connects to (if any), so the -h and -P options are mutually exclusive. 🫶

🧚 local context mode (the -o moment)

Logging some metrics is possible even when there's no local pmcd(1) — that's the "local context" mode of operation. It's activated with the -o option, and it makes pmlogger use local DSO PMDAs instead of talking to pmcd(1).

When operating with a local context, the -K option can control which DSO PMDAs should be made accessible. The spec argument conforms to the syntax described in pmSpecLocalPMDA(3). And yes, you can use more than one -K option. 🌈

🦋 lingering, non-primary behavior

When launched as a non-primary instance, pmlogger will exit immediately if the configuration file causes no metric logging to be scheduled.

The -L option overrides this behavior and causes a non-primary pmlogger instance to ``linger'', presumably pending some future dynamic re-configuration and state change via pmlc(1).

pmlogger will also linger without the -L option being used if all the metrics to be logged are logged as once only metrics. And when those once only metrics have been logged, a warning message gets generated stating that the event queue is empty and no more events will be scheduled.

📝 log files and where your diagnostics go

By default, all diagnostics and errors from pmlogger are written to the file pmlogger.log in the directory where pmlogger is launched. The -l option overrides that default behavior.

  • If the log file cannot be created or is not writable, output goes to standard error instead.
  • If the logfile for the -l option is - (i.e. -l-), then log messages are written to the standard output stream. This is particularly useful when running pmlogger manually rather than as a service daemon. 🙈

🔔 the -N option (service manager gossip)

The -N option directs pmlogger to notify a service manager — typically systemd(1) — when it has started and is about to begin writing PCP archives. You'd normally only use this when pmlogger is run as a daemon service under the control of a service manager. For more details, see __pmServerNotifyServiceManagerReady(3) and systemd(1).

On platforms that don't use a service manager supporting notifications, the -N option is basically a no-op. 🤷♀️

⏰ the -s option: how big before we dip

If specified, the -s option tells pmlogger to terminate after a certain size in records, bytes, or time units has been accumulated.

  • If endsize is an integer, then endsize records will be written to the archive.
  • If endsize is an integer suffixed by b or bytes, then endsize bytes of archive data will be written out. (note, however, that archive record boundaries will not be broken, so this limit may be slightly surpassed — it's giving "close enough" 💅)
  • Other viable file size units include: K, Kb, KiB, Kbyte, Kilobyte for kilobytes, and M, Mb, MiB, Mbyte, Megabyte for megabytes, and G, Gb, GiB, Gbyte, Gigabyte for gigabytes. These units may be optionally suffixed by an s and may be of mixed case.
  • Alternatively endsize may be an integer or a floating point number suffixed using a time unit as described in PCPIntro(1) for the interval argument (to the standard PCP -t command line option).

Some examples of different formats:

-s 100
-s 100bytes
-s 100K
-s 100Mb
-s 10Gbyte
-s 10mins
-s 1.5hours

The default is for pmlogger to run forever. ⭐

📊 the -r option: receipts, basically

The -r option causes the size of the physical record(s) for each group of metrics, and the expected contribution of the group to the size of the PCP archive for one full day of collection, to be reported in the log file. This info gets reported the first time each group is successfully written to the archive. ✨

🎀 the -U option: who are you running as

The -U option specifies the user account under which to run pmlogger. The default is the current user account for interactive use.

When run as a daemon, the unprivileged pcp account is used in current versions of PCP — but in older versions the superuser account (root) was used by default. (yes, really. we've grown. 🌱)

💾 the -v option: multi-volume era

The archive is potentially a multi-volume data set, and the -v option causes pmlogger to start a new volume after a certain size in records, bytes, or time units has been accumulated for the current volume. The format of this size specification is identical to that of the -s option (scroll up bestie 👆).

The default is for pmlogger to create a single volume archive.

Additional volume switches can also be forced asynchronously by either using pmlc(1) or by sending pmlogger a SIGHUP signal (see below).

⚠️ note: if a scheduled volume switch is in operation due to the -v option, then its counters will be reset after an asynchronous switch. so don't come for the numbers after a manual switch, okay? 😭

Independent of any -v option, each volume of an archive is limited to no more than 2^31 bytes, so pmlogger will automatically create a new volume for the archive before this limit is reached. 👑

(part 1 done, keep scrolling bestie, there's more 💖)

Okay so quick vibe check before we keep going: normally pmlogger lives in the distributed Performance Metrics Name Space (PMNS) — the big shared one. But if you pass the -n option, bestie, it loads an alternative local PMNS from the file pmnsfile instead. Your own little private namespace. Iconic.

Under normal circumstances pmlogger runs forever ✨ — literally forever — unless you gave it a -s option or you hit it with a termination signal. If you do want it to stop on its own, the -T option limits the execution time, using the time format from PCPIntro(1). Now here's the tea: that time is interpreted inside the time zone of the PMCD server, unless you also give -y, in which case it uses the time zone of this logger host. Pick your vibes.

Some examples of the different formats, because seeing it is believing:

-T 10mins
-T '@ 11:30'

So from this you can see that -T 10mins and -s 10mins do literally the exact same thing. Same actions, no difference, periodt. 💅

Alternatively, you can tie pmlogger's runtime to the lifetime of another process — use -p or --PID to nominate the PID of the process you care about. Then pmlogger just... exits when that other process stops existing. It's giving codependent in the most useful way. 🫶

When pmlogger receives a SIGHUP signal, the current volume of the archive gets closed and a brand new volume is opened. This mechanism — or the alternative mechanism via pmlc(1) — is how you manage the growth of archive files. Once a log volume is closed, you can archive that file without any ill-effect on the continued operation of pmlogger. See also the -v option above, bestie, it's relevant. 💖

When pmlogger receives a SIGUSR2 signal, the current archive is closed and a new archive is opened. BUT. For this to actually succeed, the original archive argument must include strftime(3) meta characters — like %Y%m%d.%H.%M for example. If it doesn't? pmlogger will just exit, because the archive files will already exist and pmlogger will not over-write existing archive files. No cap.

Also note: SIGUSR2 triggers pmlogger to re-exec itself and re-parse all the original arguments. Which means any relative time limits on its termination time or sampling limit get reset and start over. This only affects relative termination times, not absolute ones — so -T 5s is affected, but -T 5pm is not. Got it? 🎀

Historically, the buffers for the current archive could be flushed to disk using the flush command of pmlc(1), or by using the -u option. But the current version of pmlogger and the libpcp routines that underpin it unconditionally use unbuffered writes and a single fwrite(3) for each logical record written. So "flushing" does not force any additional data to be written to the file system. The -u option and the pmlc(1) flush command are just retained for backwards compatibility. They're the vintage of this manpage. 🦋

When launched with the -x option, pmlogger will accept asynchronous control requests on the file descriptor fd. This option is only expected to be used internally by PCP applications that support "live record mode". It's giving behind the scenes, not for us.

The -m option lets the string note be appended to the port map file for this instance of pmlogger, in the $PCP_TMP_DIR/pmlogger directory. Currently it's used internally to document the file descriptor (fd) when the -x option is used, or to indicate that this pmlogger instance was started under the control of pmlogger_check(1) (-m pmlogger_check), or that it was re-exec'd (see execvp(3)) because of a SIGUSR2 signal as described above (-m reexec). Little breadcrumbs. 🍓

The -H option lets you override the hostname written into the archive label. This mirrors the -H option of pmcd(1), but it lets you specify it on the pmlogger process itself. Without this option, the value returned from the logged pmcd(1) is what you get. 💜

The -C option makes the configuration file get parsed, and then pmlogger exits without creating an output archive. So when -C is specified, the archive command line argument is not required. Any errors in the configuration file get reported. It's basically the dry run of your dreams. ✨

The -d or --directory option may be used to specify the directory where the archive should be created. And directory may include sh(1) metacharacters — like $(...) or `...` or $var — and pmlogger will expand these to produce a final directory path. The resultant path must be absolute, not relative, okay? The -d option is intended primarily for use by pmlogger_check(1), and normal users would not typically need to use the option, because the directory for the archive is either implied by archive if it contains directory components, or else it's the current directory by default. 👑

📝 CONFIGURATION FILE SYNTAX 📝

The configuration file may be specified with the -c option. If it isn't, then configuration specifications are read from standard input. No file, no problem. 🎀

If conffile does not exist, then a search is made in the directory $PCP_VAR_DIR/config/pmlogger for a file of the same name, and if it's found, that file is used. So for example, if config.mumble does not exist in the current directory and the file $PCP_VAR_DIR/config/pmlogger/config.mumble does exist, then -c config.mumble and -c $PCP_VAR_DIR/config/pmlogger/config.mumble are equivalent. Same thing, different address. 🦋

The syntax for the configuration file is as follows.

  1. Words are separated by white space (space, tab or newline).
  2. The symbol # (hash) introduces a comment, and all text up to the next newline is ignored. Silence the haters with a #. 💅
  3. Keywords (shown in bold below) must appear literally (i.e. in lower case). Lowercase only, bestie, it's not flexible about this.
  4. Each specification begins with the optional keyword log, followed by one of the states mandatory on, mandatory off, mandatory maybe, advisory on or advisory off.
  5. For the on states, a logging interval must follow, using the syntax once, or default, or every N timeunits, or simply N timeunits — where N is an unsigned integer, and timeunits is one of the keywords msec, millisecond, sec, second, min, minute, hour, or the plural form of one of the above.

Internal limitations require the interval to be smaller than (approximately) 74 hours. An interval value of zero is a synonym for once. An interval of default means to use the default logging interval of 60 seconds; this default value may be changed to interval with the -t command line option.

The interval argument follows the syntax described in PCPIntro(1), and in the simplest form may be an unsigned integer (the implied units in this case are seconds).

  1. Following the state and possible interval specifications comes a {, followed by a list of one or more metric specifications and a closing }. The list is white space (or comma) separated. If there is only one metric specification in the list, the braces are optional. 🫶
  2. A metric specification consists of a metric name optionally followed by a set of instance names. The metric name follows the standard PCP naming conventions, see PMNS(5), and if the metric name is a non-leaf node in the PMNS (see PMNS(5)), then pmlogger will recursively descend the PMNS and apply the logging specification to all descendent metric names that are leaf nodes in the PMNS. The set of instance names is a [, followed by a list of one or more space (or comma) separated names, numbers or strings, and a closing ]. Elements in the list that are numbers are assumed to be internal instance identifiers, other elements are assumed to be external instance identifiers — see pmGetInDom(3) for more information.

If no instances are given, then the logging specification is applied to all instances of the associated metric. All of them. Every single one. 😌

  1. There may be an arbitrary number of logging specifications.
  2. As of PCP version 4.0 and later, any metric name specification that does not resolve to a leaf node in the PMNS is added to an internal list of possible dynamic subtree roots. PMDAs can dynamically create new metrics below a dynamic root node in their PMNS, and send a notification to clients that the PMNS has changed — see pmdaExtSetFlags(3) and in particular the METRIC CHANGES section for API details. This mechanism is currently supported by pmdaopenmetrics(1) and pmdammv(1). When a fetch issued by pmlogger returns with the PMDA_EXT_NAMES_CHANGE flag set, pmlogger will traverse the internal list of possible dynamic subtree nodes and dynamically discover any new metrics. In effect, as of PCP version 4.0 and later, pmlogger can be configured to dynamically log new metrics that appear in the future, after the configuration file is initially parsed. It's giving clairvoyant and honestly? Obsessed. 🔥
  3. Following all of the logging specifications, there may be an optional access control section, introduced by the literal token [access]. Thereafter come access control rules that allow or disallow operations from particular hosts or groups of hosts.

The operations may be used to interrogate or control a running pmlogger using pmlc(1) and fall into the following classes:

  • enquire — interrogate the status of pmlogger and the metrics it is logging
  • advisory — Change advisory logging.
  • mandatory — Change mandatory logging.
  • all — All of the above.

Access control rules are of the form allow hostlist : operationslist ;'' and disallow hostlist : operationslist ;''.

The hostlist follows the syntax and semantics for the access control mechanisms used by PMCD and are fully documented in pmcd(1). An operationslist is a comma separated list of the operations advisory, mandatory, enquire and all.

A missing [access] section allows all access and is equivalent to allow * : all;. 🎀

The configuration (either from standard input or conffile) is initially scanned by pmcpp(1) with the options -rs and -I $PCP_VAR_DIR/config/pmlogger. This extends the configuration file syntax with:

  • include file processing (%include)
  • a common location to search for include files ($PCP_VAR_DIR/config/pmlogger)
  • macro definitions (%define)
  • macro expansion (%name and %{name})
  • conditional inclusion of lines (%ifdef name ... %else ... %endif and %ifndef name ... %else ... %endif)

Preprocessor era, but make it cute. 💖

✨ OPTIONS ✨

okay bestie, here's the whole menu of command line flags, no skips, no skimming 💅

  • -c conffile, --config=conffile — pick which conffile to use. that's it, that's the whole vibe 🎀
  • -C, --check — just parse the config and bounce. no logging, just a vibe check on your syntax ✅
  • -h host, --host=host — grab your performance metrics from pmcd(1) on host instead of the default localhost. go get that data from somewhere else, queen 🌍
  • -H hostname, --labelhost=hostname — tell pmlogger what hostname to use instead of whatever pmcd(1) hands back. like a name tag but for your archive 🏷️
  • -I version, --pmlc-ipc-version=version — normally pmlogger and pmlc(1) autonegotiate a mutually acceptable version for their private IPC channel, like two besties agreeing on a hangout spot 🫶 but -I forces pmlogger to offer (at most) version as the version to be used. you lowkey need this when pmlogger is being managed by an older pmlc(1) that just can't autonegotiate correctly. awkward. fix it with -I.
  • -K spec, --spec-local=spec — when you're fetching metrics from a local context (peep -o), -K controls which DSO PMDAs get to be accessible. the spec argument has to match the syntax described in pmSpecLocalPMDA(3). and yes, you can stack more than one -K option, go off ✨
  • -l logfile, --log=logfile — send ALL your diagnostics to logfile instead of the default pmlogger.log 📝
  • -L, --linger — run even if you're not the primary logger instance and there's literally nothing to log. main character energy, refusing to leave the party 💅
  • -m note, --note=note — append note to the port map file for this instance. little annotation, little signature 🖊️
  • -n pmnsfile, --namespace=pmnsfile — load an alternative Performance Metrics Name Space (PMNS(5)) from the file pmnsfile. swap the whole namespace, why not 🌈
  • -N, --notify — notify your service manager like systemd(1) whenever it's needed 📣
  • -o, --local-PMDA — use a local context to collect metrics from DSO PMDAs on the local host, with no PMCD involved at all. also go look at -K 👀
  • -p PID, --PID=PID — log the specified metrics for the entire lifetime of the process identified by PID. ride or die with that process 🫡
  • -P, --primary — run as the primary logger instance. there's a more detailed description of what that means further up the page, bestie, scroll ⬆️
  • -r, --report — report record sizes and the archive growth rate. numbers, we love numbers 📊
  • -s endsize, --size=endsize — terminate once the archive size exceeds endsize. bye bye when you get too big 🚪
  • -t interval, --interval=interval — set the logging interval. the default value is 60 seconds. and fr fr go read the ENVIRONMENT and FILES sections below about the PMLOGGER_INTERVAL environment variable and how it messes with that default logging interval ⏰
  • -T endtime, --finish=endtime — specify the endtime, aka when to stop logging. no more logging after this, periodt 🛑
  • -u — use unbuffered output. this is the default, so this option literally does nothing 😭 iconic useless queen
  • -U username, --username=username — when you're in daemon mode, run as the user username 🧚
  • -v volsize, --volsize=volsize — switch volumes after reaching archive volume size volsize 📦
  • -V version, --version=version — specify the archive version. the default and the only accepted value is 2. don't try anything else 💀
  • -x fd — allow asynchronous control requests on the file descriptor fd 🎛️
  • -y — use the local timezone instead of the timezone from the pmcd(1) host 🕰️
  • --push-user=user — authenticate to the remote pmproxy(1) server as user using HTTP Basic authentication when pushing to an https:// remote archive. the password gets read from --push-password-file or the $PCP_PUSH_PASSWORD environment variable — never from the command line — and it's only ever sent over an https:// connection 🔐
  • --push-password-file=file — read the HTTP Basic authentication password from the first line of file. only meaningful together with --push-user 🤝
  • -?, --help — display the usage message and exit. the classic 🙋‍♀️

🦋 EXAMPLES 🦋

so here's a cute lil fact: for every PCP utility there's a sample pmlogger configuration file you could use to build an archive that's perfect for replaying with that tool — meaning it includes all of the performance metrics that tool uses. for a tool named foo, this config file lives at $PCP_VAR_DIR/config/pmlogger/config.foo 🍓

and here's a simple default configuration file for a primary pmlogger instance — it shows off most of what the configuration specification language can do:

log mandatory on once { hinv.ncpu hinv.ndisk }
log mandatory on every 10 minutes {
    disk.all.write
    disk.all.read
    network.interface.in.packets [ "et0" ]
    network.interface.out.packets [ "et0" ]
    nfs.server.reqs [ "lookup" "getattr" "read" "write" ]
}

log advisory on every 30 minutes {
    environ.temp
    pmcd.pdu_in.total
    pmcd.pdu_out.total
}

%include "macros.default"

%ifdef %disk_detail
log mandatory on %disk_detail_freq {
    disk.dev
}
%endif

[access]
disallow * : all except enquire;
allow localhost : mandatory, advisory;

🌸 HOSTNAME CHANGES 🌸

the hostname of the PMCD that's feeding metrics to pmlogger gets used in a bunch of different ways. let's break it down 💜

PMCD's hostname (or whatever you passed to -H) gets baked into the archive label, and that can be used to make sure later operations across multiple archives — like pmlogextract(1) — only get applied to archives from the same host. no mixing, no crossover 🚫

for pmlogger instances using a local PMCD that are launched and managed by pmlogger_check(1) and pmlogger_daily(1) (or the systemd(1) or cron(8) services that use those scripts), the local hostname may also get used to construct the name of a directory where all the archives for that one host are stored, like $PCP_ARCHIVE_DIR/<hostname> 📁

and PMCD's hostname also gets stored in the archive via the pmcd.pmlogger.hostname metric, which is used as part of the checks that make sure there's continuity of performance data after the nightly archive rotation by pmlogger_daily(1) 🔄

now here's the messy part: the hostname of the PMCD host can change during boot time when the system transitions from a temporary hostname to a persistent hostname, or by explicit administrative action anytime after the system has booted. when that happens, pmlogger may need to take special action — specifically, if the pmlogger instance was launched from pmlogger_check(1) or pmlogger_daily(1), then pmlogger must exit. under normal circumstances systemd(1) or cron(8) will launch a new pmlogger shortly after, and this new instance will be operating in the context of the new hostname for the host where PMCD is running. glow up, new name, new me ✨

🧮 DERIVED METRICS 🧮

as pmRegisterDerived(3) explains, derived metrics are a whole mechanism for extending the PMNS with new metrics defined on the PCP client-side using expressions over the existing performance metrics. you can put derived metrics in a pmlogger configuration file, but they come with some special handling rules, bestie, listen up 👂

one: the values of a derived metric get stored in the archive — not the operand metrics from the derived metric's expression. why? because the definition of a derived metric exists independently of pmlogger and the archive, and it might be different, or not exist at all, when the archive gets replayed. so we store the results, not the recipe 🍰

two: to avoid any ambiguity during archive replay, each derived metric's Performance Metric Identifier (PMID) gets rewritten (or ``remapped'') to show that this was, but now is not, a derived metric. go read pmRegisterDerived(3) for the detailed description of that PMID remapping 🔁

💀 DIAGNOSTICS 💀

the archives are, like, sufficiently precious, so pmlogger will not truncate an existing physical file. nope. never. if you see a message shaped like this:

 ...: "foo.index" already exists, not over-written
 ...: File exists

that means this exact situation has arisen. you have to explicitly remove the files and launch pmlogger again. do it yourself, pmlogger's not your cleanup crew 🧹

there may be at most one primary pmlogger instance per monitored host. try to bend that rule and you get this error:

 pmlogger: there is already a primary pmlogger running

one primary. that's the rule. 😤

various other messages about the creation and/or deletion of files in $PCP_TMP_DIR/pmlogger mean either there's a permission problem on that directory, or some feral files have appeared in there 🙈

📁 FILES 📁

  • archive.meta — metadata (metric descriptions, instance domains, etc.) for the archive
  • archive.0 — the initial volume of metrics values (subsequent volumes have suffixes 1, 2, ...)
  • archive.index — temporal index to support rapid random access to the other files in the archive
  • $PCP_TMP_DIR/pmlogger — pmlogger maintains the files in this directory as the port map between the process id of the pmlogger instance, the IPC port that may be used to control each pmlogger instance (as used by pmlc(1)), the name of the host where pmcd(1) is providing the metrics for pmlogger, the remote connection string for the remote HTTP server that is receiving the logged data or the full path to the archive base name for a local logger, and any annotation from the -m or -x options; all of this information is also available via the pmcd.pmlogger.* metrics, e.g. $ pminfo -f pmcd.pmlogger
  • $PCP_VAR_DIR/config/pmlogger/config.default — default configuration file for the primary logger instance launched from $PCP_RC_DIR/pmlogger
  • $PCP_VAR_DIR/config/pmlogger/config.* — assorted configuration files suitable for creating logs that may be subsequently replayed with the PCP visualization and monitoring tools
  • $PCP_ARCHIVE_DIR/<hostname> — default directory for PCP archive files for performance metric values collected from the host <hostname>
  • $PCP_SYSCONFIG_DIR/pmlogger — additional environment variables that will be set when the primary pmlogger instance executes. only settings of the form "PMLOGGER_VARIABLE=value" will be honoured 👑
  • ./pmlogger.log — (or $PCP_ARCHIVE_DIR/<hostname>/pmlogger.log when started automatically by either $PCP_RC_DIR/pmlogger or one of the pmlogger(1) monitoring scripts such as pmlogger_check(1)) all messages and diagnostics are directed here

🌿 ENVIRONMENT 🌿

okay so bestie, pmlogger is a social butterfly and normally it creates a socket to receive control messages from pmlc(1) on the first available TCP/IP port numbered 4330 or higher. that's just its thing.

  • the environment variable PMLOGGER_PORT can be used to specify an alternative starting port number ✨

if you set PMLOGGER_LOCAL to the value 1, pmlogger will run in a localhost-only mode of operation, where it binds only to the loopback interface. shy girlie behavior, we respect it 🫶

the PMLOGGER_REQUEST_TIMEOUT variable may be set by applications such as pmlc(1) to specify a timeout in seconds for connection requests to the pmlogger control port. if it's not set, connections may block indefinitely 😭 (this variable would not normally be set by pmlogger itself).

the PMLOGGER_MAXPENDING variable can be set to indicate the maximum length to which the queue of pending pmlc connections may grow. the queue said "no more besties allowed" 💅

the default sampling interval used by pmlogger can be set using the PMLOGGER_INTERVAL variable (if not set, 60 seconds will be used). both the command line and directives in the configuration file will override this value. it's an integer in units of seconds.

on platforms using systemd(1), and when the -N option is given, the NOTIFY_SOCKET environment variable would normally be set by the service manager prior to launching pmlogger.

two additional environment variables are used as part of the hand-off from one pmlogger instance to the next during a re-exec operation, namely __PMLOGGER_REEXEC and __PMLOGGER_TZ. the details of how these are used is an internal implementation detail, but end-users should avoid using variables of these names in their shell's environment 🙈

🌈 PCP ENVIRONMENT 🌈

environment variables with the prefix PCP_ are used to parameterize the file and directory names used by PCP. on each installation, the file /etc/pcp.conf contains the local values for these variables.

the $PCP_CONF variable may be used to specify an alternative configuration file, as described in pcp.conf(5).

💀 DEBUGGING OPTIONS 💀

the -D or --debug option enables the output of additional diagnostics on stderr to help triage problems, although the information is sometimes cryptic and primarily intended to provide guidance for developers rather than for end-users. debug is a comma separated list of debugging options; use pmdbg(1) with the -l option to obtain a list of the available debugging options and their meaning.

debugging options specific to pmlogger are as follows:

  • appl0 — control request PDUs to and from pmlc(1)
  • appl1 — credentials exchange
  • appl2 — alarm (timing) loop diagnostics, callback work and record control mode messages
  • appl3 — signal callbacks and exit logging
  • appl4 — record timestamps in logfile as milestones are reached
  • appl5 — PDU stats after conffile file processed
  • appl6 — building the metadata cache from conffile to boost start-up performance, pass0() work
  • appl7 — suppress building of the metadata cache, pass0() becomes a no-op
  • appl8 — hist_hash updates and queries to support query command from pmlc(1)

🌸 SEE ALSO 🌸

PCPIntro(1), pmcd(1), pmlc(1), pmlogdump(1), pmlogger_check(1), pmlogger_daily(1), pmproxy(1), systemctl(1), systemd(1), PMAPI(3), __pmServerNotifyServiceManagerReady(3), execvp(3), pmRegisterDerived(3), pmSpecLocalPMDA(3), strftime(3), PMNS(5), pcp.conf(5), pcp.env(5), LOGARCHIVE(5), chkconfig(8) and cron(8).

and that's the whole page bestie, we ate and left no crumbs 💅✨ this icon of a manpage comes from Performance Co-Pilot (that's the PCP girlies 👑) and it's dated PMLOGGER(1) — go forth and log your metrics, no cap 🎀🦋💖

📦 where this page comes from