PMREGISTERDERIVED(3)

📦
version 7.2.2 · 7 versions
, in arch rolling, fedora 43, fedora 44, ubuntu stonking

💖 PMREGISTERDERIVED(3) 💖

okay so bestie, welcome to pmRegisterDerived(3) from the Library Functions Manual — it's giving "I want new metrics and I want them now" energy and honestly? iconic. let's get into it 💅

✨ NAME ✨

pmRegisterDerived, pmRegisterDerivedMetric — register a global derived metric name and definition 🎀

🌸 C SYNOPSIS 🌸

#include <pcp/pmapi.h>

char *pmRegisterDerived(char *name, char *expr);
int pmRegisterDerivedMetric(char *name, char *expr, char **errmsg);

cc ... -lpcp

🦋 DESCRIPTION 🦋

so. derived metrics are literally the glow-up of the Performance Metrics Name Space (PMNS) 💖 — they let you bolt brand-new metrics onto the PMNS, defined on the PCP client-side, using expressions built out of the performance metrics that already exist. no cap.

like, the classic use case is aggregating a whole bunch of similar metrics into one higher-level summary metric 🌈, or supporting the ``delta value-a over delta value-b'' class of metrics — which, btw, are just not possible with the base data semantics of PCP. the girlies that need the second one? average I/O size, defined as:

delta(disk.dev.total_bytes) / delta(disk.dev.total)

where both of the disk.dev metrics are counters, and what you actually want is to sample both metrics, compute the difference between the current and previous values, and then calculate the ratio of those differences. math queen behavior ✨

the args 🫶

the arguments to pmRegisterDerived are:

  • name — the name of the new derived metric 💜
  • expr — an expression defining how the values of name should be computed 🧮

pmRegisterDerivedMetric is the exact functional equivalent to pmRegisterDerived — the only difference is it gives you a simplified model of error handling, where a formatted message comes back via the errmsg parameter. same brain, cuter delivery 💅

when stuff gets checked 👀

  • Syntactic checking happens at the time pmRegisterDerived is called. like, right then. immediately.
  • Semantic checking is deferred 😭 — it waits until each new PMAPI context is created with pmNewContext(3), or re-established with pmReconnectContext(3). that's the moment when the PMNS and metadata are finally available, so semantic checking can happen and the metadata of the derived metrics can be determined.
if you call pmRegisterDerived after one or more PMAPI contexts has already been opened, then the newly registered metrics will be available in those contexts — but the more normal use is to make all your calls to pmRegisterDerived (possibly via pmLoadDerivedConfig(3)) or pmRegisterDerivedMetric before calling pmNewContext(3). bestie. do it in that order. trust.

all of the defined global derived metrics are available in all PMAPI contexts. periodt. 👑

per-context girlies 💅

it's also possible to define per-context derived metrics once a PMAPI context has been established. these ones are private to the context they were defined in, and you make them with the allied routines pmAddDerived(3) and pmAddDerivedMetric(3). it's giving "this stays between us" 🙈

to undefine a derived metric by name, go refer to pmUnregisterDerived(3) — which removes all instances of the derived metric in all contexts, irrespective of how the derived metric was defined. it does not care how you got here, it is removing everything 💀

naming rules 🎀

name should follow the syntactic rules for the names of performance metrics, namely:

  • one or more components separated with a dot (.)
  • each component must begin with an alphabetic, followed by zero or more characters drawn from the alphabetics, numerics and underscore (_)

for more details, refer to PCPIntro(1) and PMNS(5) 📚

name must be unique across all derived metrics, and should not match the name of any regular metric in the PMNS. it is acceptable for name to share some part of its prefix with an existing subtree of the PMNS 🌸 — e.g. the average I/O size metric above could be named disk.dev.avgsz, which would place it amongst the other disk.dev metrics in the PMNS. alternatively, derived metrics could populate their own subtree of the PMNS, e.g. that same average I/O size metric could be named my.summary.disk.avgsz. main character energy, your own little neighborhood 🏡

the expression rules (the important part, bestie) 🌈

the expression expr follows these syntactic rules:

  • Terminal elements are either names of existing metrics or numeric constants. recursive definitions are not allowed, so only the names of regular metrics (not other derived metrics) may be used. numeric constants are either integers constrained to the precision of 32-bit unsigned integers, or double precision floating point numbers. 🔢
  • the usual binary arithmetic operators are supported, namely addition (+), subtraction (-), multiplication (*) and division (/) with the normal precedence rules where multiplication and division have higher precedence than addition and subtraction, so a+b*c is evaluated as a+(b*c). ✨
  • unary negation may be used, e.g. -3*some.metric. ➖
  • C-style relational operators are supported, namely <, <=, ==, >=, > and !=. relational expressions return a value as a 32-bit unsigned number, being 0 for false and 1 for true. the expected operator precedence rules apply, so arithmetic operators have higher precedence than relational operators, and a-b>c+d is evaluated as (a-b)>(c+d). all the relational operators have equal precedence, so the (slightly odd) expression involving consecutive relational operators a>b!=c is evaluated as (a>b)!=c. 🤓
  • C-style boolean operators are supported, namely and (&&) and or (||). boolean expressions return a value as a 32-bit unsigned number, being 0 for false and 1 for true. the expected operator precedence rules apply, so relational operators have higher precedence than boolean operators, and a>b*c&&d<=e+f is evaluated as (a>(b*c))&&(d<=(e+f)). both the boolean operators have equal precedence, so the expression involving consecutive boolean operators a>=b||b>c&&d!=e||f>g is evaluated as (((a>=b)||(b>c))&&(d!=e))||(f>g). 🔥
  • additionally, the ! operator may be used to negate a boolean or relational expression, returning a value as a 32-bit unsigned number, being 0 for false and 1 for true. the expected operator precedence rules apply, so boolean (and relational) operators have higher precedence than boolean negation, and !a>b||c<d is evaluated as !((a>b)||(c<d)), while !a<b+c is evaluated as !(a<(b+c)). 💅
  • C-style ternary conditional expressions are supported. in general terms the expression check ? foo : bar is evaluated as foo (the true'' operand) if `check` (the guard'') is true, else the expression evaluates to bar (the false'' operand). some **special semantic rules** apply to the guard'' expression and the other two operand expressions:

(a) each expression may involve a singular value or a set of values (when the expression involves one or more metrics with an instance domain). 🌸

(b) all expressions with a set of values must be defined over the same instance domain.

(c) both operand expressions must have the same metadata, so the same metric type, semantics and units (dimension and scale). 💜

(d) the ``guard'' expression must have an arithmetic or relational or boolean value, so that it can be evaluated as 0 for false, else true.

(e) if the guard'' expression has a singular value and one or more of the other operand expressions involves an instance domain, the guard'' applies to all instances.

(f) if the guard'' expression has a set of values and one or more of the other operand expressions involves an instance domain, the guard'' is evaluated once for each instance (there must be exactly one instance domain as per rule (b) above).

(g) if one of the operand expressions has a singular value and the other has a set of values, and the singular value is selected based on the evaluation of the ``guard'', then the result is a set of values (all the same) with instance enumeration being taken from the other operand expression. for example in the expression: check ? foo : bar, if foo is an expression with a singular value and bar is a set-valued expression, then if check is true, then the result is a set of values (all having the same value, foo) over the instance domain of bar.

(h) as a special case, either (but not both) of the expressions foo or bar may be the special constructor novalue(). if novalue() needs to be evaluated, it will return no values. the most common use of novalue() is in conjunction with defined() as in the following example:

mumble = defined(abc.def) ? count(abc.def) : novalue()

the metadata for novalue() is taken from the peer operand of the : operator, making conditions (b) and (c) above trivially true; if the peer operand cannot be evaluated, then the default metadata for novalue() is a singular discrete integer with no units, and conditions (b) and (c) are waived. 🫶

an extension of novalue() is novalue(<metadata>) where the <metadata> can be specified using one or more parameters of the form tag=value or tag="value" as described for the mkconst() constructor below, for example:

novalue(type=float, semantics=instant, units="Kbytes/sec")

(i) as an additional further case, if check is of the form defined(somemetric) then references to undefined metrics are allowed in whichever of foo or bar is not required once the existence of somemetric has been established. this allows uses of the form:

fumble = defined(new.metric) ? new.metric : old.metric

which is valid when new.metric is defined and when new.metric is not defined, although this does mean rules (b) and (c) are relaxed in this case, which further means novalue() may have no peer operand to provide metadata. 😭

a generalization of this construct is supported for any check that can be evaluated statically, so a boolean expression involving defined() predicates, for example:

bar = !defined(a) || !defined(b) ? novalue() : a + b
  • selection of a single instance can be specified by the construct [instance_name] which may be appended to a metric name or a parenthesized expression. for example: 🎯
fw.bytes = network.interface.in.bytes[eth1] + \
           network.interface.out.bytes[eth1]

or (equivalently):

fw.bytes = (network.interface.in.bytes + \
            network.interface.out.bytes)[eth1]

all characters between the ['' and ]'' are considered to be part of the (external) instance name, so be careful to avoid any spurious white space. 😤 a backslash may be used as an escape prefix in the (unlikely) event that the external instance name contains a ``]''. 🦋

  • Numeric constants can also be built with the mkconst() constructor, which is lowkey iconic because it takes a bunch of arguments: the first one is a numeric constant (integer or floating point, dealer's choice 💅), and then you get one or more parameters shaped like tag=value or tag="value", where the allowed tags and values are, bestie, exactly this:
    • type — one of the numeric metric types from <pcp/pmapi.h>, with the PM_TYPE_ prefix stripped off, so: 32, U32, 64, U64, FLOAT or DOUBLE. ✨
    • semantics — one of the semantic types from <pcp/pmapi.h>, with the PM_SEM_ prefix stripped off, so: COUNTER, INSTANT or DISCRETE. 🦋
    • units — a spec of dimension and scale (together they make up the units, periodt) written in the syntax that pmParseUnitsStr(3) accepts. 🌸
    • indom — an instance domain number, given as NULL for PM_INDOM_NULL (which is the default) or as domain.serial where domain sits in the range [0,510] and serial sits in the range [0,4194303]. 👑
    • meta — a metric name, and that metric is the one supplying the base metadata that other parameters are allowed to modify. 🎀

The value can optionally be wrapped in double quotes, and it can show up in any mix of UPPER and/or lower case. The tag, though, must be lower case exactly like the table above — no cap, it's giving lowercase or nothing. 💀

This whole thing is most useful when the expression semantics require matching type and/or semantics and/or units for the operands, like:

idle = mem.util.free > mkconst(10485760, units=Kbyte)
avg_io_size = delta(disk.dev.total) == 0 ? \
    mkconst(1.0, semantics=instant, units="kbyte / count") : \
    delta(disk.dev.total_bytes) / delta(disk.dev.total)
  • Rescaling — expressions can be rescaled with the rescale function, which takes two arguments. The first is an arithmetic expression to be rescaled, and the second is the desired units after rescaling, given as a string value in the syntax accepted by pmParseUnitsStr(3). For example:
rescale(network.interface.total.bytes, "Mbytes/hour")

The expression and the desired units must both have the same dimension, e.g Space=1, Time=-1 and Count=0 in the example above. No dimension-mismatch shenanigans, okay? 🫶

  • The unary functions below operate on a single performance metric and return one or more values. For all of them — except count() and defined() — the type of the operand metric must be arithmetic (integer of various sizes and signedness, float or double). 📐
    • avg(x) — a singular instance that is the average value across all instances for the metric x. 💖
    • count(x) — a singular instance that is the count of the number of instances for the metric x. Special case alert 🚨: if fetching the metric x returns an error, then count(x) will be 0.
    • defined(x) — a boolean value that is true (1) if the metric x is defined in the PMNS, else false (0). The function is evaluated when a new PMAPI context is created with pmNewContext(3) or re-established with pmReconnectContext(3). So any subsequent changes to the PMNS after the PMAPI context has been established will not change the value of this function in the expression evaluation. It's frozen in time, bestie. 🧊
    • max(x) — a singular instance that is the maximum value across all instances for the metric x. 🔥
    • min(x) — a singular instance that is the minimum value across all instances for the metric x. 🌸
    • sum(x) — a singular instance that is the sum of the values across all instances for the metric x. ➕
  • One more unary function here, and it returns the instantaneous value of an expression — not the rate-converted value that is the default for expressions with the semantics of PM_SEM_COUNTER. 😤
    • instant(expr) — returns the current value of the expression, even if it has the semantics of a counter, i.e. PM_SEM_COUNTER. The semantics of the derived metric are based on the semantics of the expression expr; if expr has semantics PM_SEM_COUNTER, the semantics of instant(expr) is PM_SEM_INSTANT, otherwise the semantics of the derived metric is the same as the semantics of expr. ✨
  • And the consecutive-sample girlies — the following unary functions return values computed from the value of an expression on consecutive samples, or pmFetch(3) calls. The expression (expr below) may involve one or more metrics but must have an arithmetic value (integer of various sizes and signedness, float or double) for all instances. No exceptions, fr fr. 💅

okay so if expr is a set-valued expression (iykyk, it hands you a whole little squad of instances instead of one single value), then only the instances that show up in both samples make it into the result. no newcomers, no strays. 💅

✨ the two glow-up functions: delta and rate ✨

  • delta(expr) — bestie, this one gives you the difference in values for the expression between one call to pmFetch(3) and the next. there's one value in the result for every instance that appears in both the current and the previous sample. 🫶

and here's the type tea: if the expression is unsigned, the type of the result gets converted so as much precision as possible can be retained. so if the expression has type PM_TYPE_U32 then the result is of type PM_TYPE_64, else if the expression has type PM_TYPE_U64 then the result is of type PM_TYPE_DOUBLE. otherwise the type of the result is the same as the type of the expression. no cap. ✨

  • rate(expr) — this one returns the difference in values for the expression between one call to pmFetch(3) and the next, divided by the elapsed time between the calls to pmFetch(3). it's giving speed. 🔥

the semantics of the derived metric are based on the semantics of the expression with the dimension in the time domain decreased by one, plus scaling if required in the time utilization case where the operand is in units of time and the derived metric is unitless.

there's one value in the result for each instance that appears in both the current and the previous sample — except in the case where the expression has the semantics of a counter, i.e. PM_SEM_COUNTER, and the current value of an instance is smaller than the previous value of the same instance. in that case, no value is returned for this instance (this corresponds to a counter wrap'' or a counter reset''). 😭 these rules mimic the rate conversion applied to counter metrics by tools such as pmval(1), pmie(1) and pmchart(1).

  • the matchinst function may be used to select a subset of the instances from an instance domain for a metric or expression. it takes two arguments:
    • (a) an instance filter that consists of an optional negation operator !'' followed by a regular expression delimited by /'' characters. the regular expression follows the POSIX Extended Regular Expression syntax as described in regex(3). a single backslash may be used to escape the regular expression delimiter /'', but double backslashes are required to escape any regular expression special characters, e.g. for the (extremely unlikely) case of wanting to match instance names like some*text/other[text]'' a regular expression of the form /some\\*text\/other\\[text]/ would be required. and if present, the negation operator reverses the sense of the filtering, so all instances not matching the regular expression will be selected. 🙈
    • (b) a metric or expression that must be defined over an instance domain.

for example, the following expression will have values for the metric network.interface.in.bytes for all network interfaces except the loopback and virtual bridge devices:

matchinst(!/^(lo)|(vbir)/, network.interface.in.bytes)
  • the scalar function may be used to convert a metric or expression defined over an instance domain into a scalar value that can be used in other expressions. for example:
net.in.bytes = scalar(network.interface.in.bytes[eth0]) + \
           scalar(network.interface.in.bytes[eth1])

the instance domain is removed from the metadata for the result, and the instance identifier is removed from the value during fetching.

⚠️ okay so listen: if the metric or expression involves more than one instance, then the result is formed by picking the first instance — this is arbitrary, which implies the scalar function should only be used for metrics or expressions that are expected to contain zero or one instances, e.g. the construct ``[instance_name]'' or the matchinst function with a pattern that matches at most one instance. don't come crying to me when you get a random instance, bestie. 💀
  • parenthesis may be used for explicit grouping. 🎀
  • a line ending with \'' is treated as to be continued'' and the following line is appended after stripping the ``'' and the embedded newline. the way that little backslash keeps the vibe going... iconic. ✨
  • control lines begin with #control followed by settings for one or more derived metric controls or options as described in pmSetDerivedControl(3). 👑
  • other lines beginning with ``#'' are treated as comments and ignored. (not me ignoring things that start with a # 🔥)
  • white space is ignored. it's just there for the aesthetic, and honestly? same. 🌸

🧠 SEMANTIC CHECKS AND RULES 🧠

okay bestie so here's the thing — before any derived metric can actually exist, there's a whole lil conversion pipeline that has to figure out its metadata, plus a vibe check to make sure the expression even makes sense. It's giving "math teacher who cares about your feelings." 💖

✨ the semantics tea ☕

In an arithmetic expression or a relational expression, if the semantics of both operands is not a counter (i.e. PM_SEM_INSTANT or PM_SEM_DISCRETE), then the result gets semantics PM_SEM_INSTANT — unless both operands are PM_SEM_DISCRETE, in which case the result is also PM_SEM_DISCRETE. slay, the discrete stays discrete. 💅

For an arithmetic expression, the dimension of each operand must be the same. For a relational expression, the dimension of each operand must also be the same — except numeric constants (with no dimension) are allowed to sneak in, e.g. in the expression network.interface.in.drops > 0. 🫶

🚫 no chaotic combinations pls

To prevent arbitrary and non-sensical combinations, some restrictions apply to expressions that combine metrics with counter semantics to produce a result with counter semantics. For an arithmetic expression:

  • if both operands have the semantics of a counter, then only addition or subtraction is allowed
  • or if the left operand is a counter and the right operand is not, then only multiplication or division are allowed
  • or if the left operand is not a counter and the right operand is a counter, then only multiplication is allowed

Metrics with extra units, like temperature or voltage, are subject to some additional constraints, specifically:

  • no automatic scale conversion (although the rescale function works as expected)
  • extra units cannot be used for both operands in multiplication (*)
  • extra units are only allowed for both operands of division (/) if the units and scale are identical

Because relational expressions use the current value only and produce a result that is not a counter, either or both operands of a relational expression may be counters. iconic behaviour honestly. 👑

📏 mapping the pmUnits of the metadata

The rules go like this:

  • If both operands have a dimension of Count and the scales are not the same, use the larger scale and convert the values of the operand with the smaller scale.
  • If both operands have a dimension of Time and the scales are not the same, use the larger scale and convert the values of the operand with the smaller scale.
  • If both operands have a dimension of Space and the scales are not the same, use the larger scale and convert the values of the operand with the smaller scale.
  • For addition and subtraction all dimensions for each of the operands and result are identical.
  • For multiplication, the dimensions of the result are the sum of the dimensions of the operands.
  • For division, the dimensions of the result are the difference of the dimensions of the operands.

Scale conversion involves division if the dimension is positive, else multiplication if the dimension is negative. And heads up — if scale conversion is applied to either of the operands, the result is promoted to type PM_TYPE_DOUBLE. glow up achieved ✨

🧮 putting it all together (example time)

Consider the derived metric defined as follows:

x = network.interface.speed - delta(network.interface.in.bytes) /
                    delta(sample.milliseconds)

The type, dimension and scale settings would propagate up the expression tree as follows.

┌─────────────────────────┬────────┬───────────────┬─────────────────┐
│       Expression        │  Type  │  Dimension &  │ Scale Factor(s) │
│                         │        │  Scale        │                 │
├─────────────────────────┼────────┼───────────────┼─────────────────┤
│ sample.milliseconds     │ DOUBLE │ millisec      │                 │
│ delta(...)              │ DOUBLE │ millisec      │                 │
│ network...bytes         │ U64    │ byte          │                 │
│ delta(...)              │ U64    │ byte          │                 │
│ delta(...) / delta(...) │ DOUBLE │ byte/millisec │ /1048576 and    │
│                         │        │               │ *1000           │
│ network...speed         │ FLOAT  │ Mbyte/sec     │                 │
│ x                       │ DOUBLE │ Mbyte/sec     │                 │
└─────────────────────────┴────────┴───────────────┴─────────────────┘

🫧 instance domains & the semantic check timing drama

Expressions involving single instance selection or the matchinst function must be associated with underlying metrics that have an instance domain. These constructors make no sense for singular metrics, fr fr. 🙈

Because semantic checking cannot be done at the time pmRegisterDerived is called, errors found during semantic checking (when any subsequent calls to pmNewContext(3) or pmReconnectContext(3) succeed) are reported using pmprintf(3). These include:

  • Error: derived metric <name1>: operand: <name2>: <reason> There was a problem calling pmLookupName(3) to identify the operand metric <name2> used in the definition of the derived metric <name1>.
  • Error: derived metric <name1>: operand (<name2> [<pmid2>]): <reason> There was a problem calling pmLookupDesc(3) to identify the operand metric <name2> with PMID <pmid2> used in the definition of the derived metric <name1>.
  • Semantic error: derived metric <name>: <operand> : <operand> Different <metadata> for ternary operands For a ternary expression, the "true" operand and the "false" operand must have exactly the same metadata, so type, semantics, instance domain, and units (dimension and scale).
  • Semantic error: derived metric <name>: <operand> <op> <operand>: Dimensions are not the same Operands must have the same units (dimension and scale) for each of addition, subtraction, the relational operators and the boolean "and" or "or" operators.
  • Semantic error: derived metric <name>: <operand> <op> <operand>: Illegal operator for counter and non-counter Only multiplication or division are allowed if the left operand has the semantics of a counter and the right operand is not a counter.
  • Semantic error: derived metric <name>: <operand> <op> <operand>: Illegal operator for counters If both operands have the semantics of counter, only addition or subtraction make sense, so multiplication and division are not allowed.
  • Semantic error: derived metric <name>: <operand> <op> <operand>: Illegal operator for non-counter and counter Only multiplication is allowed if the right operand has the semantics of a counter and the left operand is not a counter.
  • Semantic error: derived metric <metric> <expr> RESCALE <units>: Incompatible dimensions The parameters <expr> and <units> to the rescale function must have the same dimension along the axes of Time, Space and Count.
  • Semantic error: derived metric <name>: Incorrect time dimension for operand Rate conversion using the rate() function is only possible for operand metrics with a Time dimension of 0 or 1 (see pmLookupDesc(3)). If the operand metric's Time dimension is 0, then the derived metrics has a value "per second" (Time dimension of -1). If the operand metric's Time dimension is 1, then the derived metrics has a value of time utilization (Time dimension of 0).
  • Semantic error: derived metric <name>: <function>(<operand>): Non-arithmetic operand for function The unary functions are only defined if the operand has arithmetic type. Similarly the first argument to the rescale function must be of arithmetic type.
  • Semantic error: derived metric <name>: <expr> ? ...: Non-arithmetic operand for ternary guard The first expression for a ternary operator must have an arithmetic type.
  • Semantic error: derived metric <name>: ... - ...: Non-arithmetic operand for unary negation Unary negation only makes sense if the following expression has an arithmetic type.
  • Semantic error: derived metric <name>: <operand> <op> <operand>: Non-arithmetic type for <left-or-right> operand The binary arithmetic operators are only allowed with operands with an arithmetic type (integer of various sizes and signedness, float or double).
  • Semantic error: derived metric <name>: <operand> <op> <operand>: Non-counter and not dimensionless for <left-or-right> operand For multiplication or division or any of the relational operators, if one of the operands has the semantics of a counter and the other has the semantics of a non-counter (instantaneous or discrete) then the non-counter operand must have no units (dimension and scale).
  • Semantic error: derived metric <name>: <expr> ? <expr> : <expr>: Non-scalar ternary guard with scalar expressions If the "true" and "false" operands of a ternary expression have a scalar value, then the "guard" expression must also have a scalar value.
  • Semantic error: derived metric <name>: <expr> <op> <expr>: Operands should have the same instance domain For all of the binary operators (arithmetic and relational), if both operands have non-scalar values, then they must be defined over the same instance domain.
  • Semantic error: derived metric <name>: operand <dname>: Illegal nested derived metric A derived metric (<dname>) cannot be nested (i.e. used) in the definition of another derived metric (<name>).
  • Semantic error: derived metric <name>: <expr> <op> <expr>: Illegal operator for non-counters Metrics with counter semantics may be added or subtracted, but <op> (division or multiplication) is not allowed.
  • Semantic error: derived metric <name>: operand <bname>: Unknown metric for ternary expression When a new context was established, the metric <bname> was not in the PMNS of the new context and <bname> is a required operand in the definition of the derived metric <name>.
fr fr bestie, that error list is LONG but every single one is a real thing that can happen, so keep it in your back pocket 💖

✨ EXPRESSION EVALUATION ✨

okay bestie, so when we're talking about the binary arithmetic operators, there's this whole type-promotion situation going on and honestly it's giving ✨hierarchy✨ 💅

Here's the tea: if either operand has to be scaled (like, think convert bytes to Kbytes energy), then the result gets promoted to PM_TYPE_DOUBLE. no negotiations. 💀

Otherwise? The type of the result is decided by the types of the operands, and you go through this list from top to bottom until you find a match — first one wins, periodt 👑

  1. either is PM_TYPE_DOUBLE → operator: any → result type: PM_TYPE_DOUBLE
  2. any → operator: division → result type: PM_TYPE_DOUBLE
  3. either is PM_TYPE_FLOAT → operator: any → result type: PM_TYPE_FLOAT
  4. either is PM_TYPE_U64 → operator: any → result type: PM_TYPE_U64
  5. either is PM_TYPE_64 → operator: any → result type: PM_TYPE_64
  6. either is PM_TYPE_U32 → operator: any → result type: PM_TYPE_U32
  7. otherwise (both are PM_TYPE_32) → operator: any → result type: PM_TYPE_32
🫶 pro tip from me to you: the order is NOT optional, bestie. it literally says evaluated from top to bottom until a match is found. so like, don't go jumping around the list, okay? 😭

🎀 PMIDs AND MASKING 🎀

ok so bestie, before we get into the messy part, let's talk IDs 💖

Inside PCP, every single metric gets its very own Performance Metric Identifier (that's a PMID, iykyk) and internally that PMID is built from 3 fields:

  • the domain number — that's the Performance Metrics Domain Agent, aka the PMDA
  • the cluster number
  • the item number

Derived metrics use the reserved domain number 511 plus some special PMIDs, and the whole vibe is that a PMID is written out as domain.cluster.item. Here's the tea 🍓:

  • global — starting PMID is 511.0.1. Metrics get assigned ascending PMIDs as they're registered via pmRegisterDerived or pmRegisterDerivedMetric, or pmLoadDerivedConfig(3).
  • per-context — starting PMID is 511.2047.1023. Metrics get assigned descending PMIDs as they're registered via pmAddDerived(3) or pmAddDerivedMetric(3).
  • remapped — starting PMID is 511.c.i. When a derived metric gets recorded into a PCP archive by pmlogger(1) or one of the related archive creation tools, the PMID of the derived metric gets remapped in the archive so that c is the cluster of the derived metric plus 2048, and i is the item of the derived metric. For example, a derived metric with PMID 511.0.13 will have the remapped PMID 511.2048.13 in an archive. See pmlogger(1) for the whole discussion about adding derived metrics to a PCP archive. 👑

Now for base metrics from a host or archive context, there's a requirement that every metric name is unique. But derived metrics need that restriction to be relaxed, so the same metric name could be associated with a base metric, and/or a per-context derived metric, and/or a global metric. Which means... one or more of the metrics with the same name may be masked by other metrics of the same name 🙈

Here's exactly which metric gets used when one of these multiply-defined names gets presented to the PMAPI:

  • base metric: no, remapped: no, per-context: no, global: no → none; the metric is undefined
  • base metric: no, remapped: no, per-context: no, global: yes → global derived
  • base metric: no, remapped: no, per-context: yes, global: no → per-context derived
  • base metric: no, remapped: no, per-context: yes, global: yes → per-context derived
  • base metric: no, remapped: yes, per-context: no, global: no → remapped
  • base metric: no, remapped: yes, per-context: no, global: yes → remapped
  • base metric: no, remapped: yes, per-context: yes, global: no → remapped
  • base metric: no, remapped: yes, per-context: yes, global: yes → remapped
  • base metric: yes, remapped: no, per-context: no, global: no → base metric
  • base metric: yes, remapped: no, per-context: no, global: yes → base metric
  • base metric: yes, remapped: no, per-context: yes, global: no → base metric
  • base metric: yes, remapped: no, per-context: yes, global: yes → base metric
  • base metric: yes, remapped: yes, per-context: -, global: - → cannot happen; for any name there can be at most one of a base metric or a remapped metric

When masking happens, it happens silently. Like, no drama, no notification, it just... does it 💅

Most PCP applications support -Dderive on the command line, and this pmdbg(1) debug option will report derived metric operations (including masking) on stderr. So if you're like "where did my metric GO" — that's your girl 🫶

💀 CAVEATS 💀

Okay bestie, listen up because this one matters:

Derived metrics are not available when using pmFetchArchive(3), because this routine does not use a target list of PMIDs that could be remapped — which is exactly what is done for pmFetch(3). 😭

💅 DIAGNOSTICS 💅

ok bestie, vibe check on how this thing tells you whether it ate or flopped:

  • Success? pmRegisterDerived hands you back NULL. That's the good ending 🎀
  • Found a syntactic error at registration time? Then the value pmRegisterDerived returns is a pointer into expr showing exactly where the error is lurking. To figure out what the error actually was, your app should call pmDerivedErrStr(3) and pull the matching parser error message. iykyk ✨
  • pmRegisterDerivedMetric? On success it returns 0, and errmsg is undefined — don't go reading it, it's not your moment 🙈
  • If the given expr does not conform to the required syntax, pmRegisterDerivedMetric returns -1 and drops a dynamically allocated error message string into errmsg. That string is terminated with a newline and includes both the input name and expr, along with an indicator of the position where the error got caught. e.g.
Error: pmRegisterDerivedMetric("my.disk.rates", ...) syntax error
4rat(disk.dev.read)
    ^
  • Sometimes that position indicator line is followed by an extra diagnostic line describing the nature of the error, when that info is available 🦋
  • > ⚠️ real talk: if there's an error, you — the pmRegisterDerivedMetric caller — are the one responsible for calling free(3) to release the space allocated for errmsg. Nobody else is coming to do it for you. periodt 💅

The debugging option derive (see pmdbg(1)) can be used to give you extra diagnostic info about the parsing, binding and evaluation of derived metric expressions. Stack more debugging options on top of derive for even more info, as laid out below. Alternatively, an application can use pmSetDerivedControl(3) to achieve the same result ✨

  • appl0 → PCP_DERIVED_DEBUG_SYNTAX — covers pmRegisterDerived, pmAddDerived(3), the lexical scanner, the syntactic checks, and pmUnregisterDerived(3).
  • appl1 → PCP_DERIVED_DEBUG_SEMANTICS — covers the actions when a new PMAPI context is created, the semantic checks, detailed error reports for binding failures, and the updates that catch global derived metrics registered after the PMAPI context was created.
  • appl2 → PCP_DERIVED_DEBUG_EVAL — covers PMNS operations, pmFetch(3) and pmLookupDesc(3).

And for even more detail, add the desperate debugging option to any of the above, e.g. -Dderive,appl1,desperate shows the binding of every expression node for every derived metric when a new PMAPI context is created 👑

✨ OPTIONS ✨

On top of the diagnostic control options, pmSetDerivedControl(3) can also set this one option, which controls the parsing and evaluation of derived metric expressions:

  • PCP_DERIVED_OPTION_NOVALUE — if a metric operand inside a derived metric expression isn't available in the current PMAPI context, the default behaviour (that's when this option has the value 0) is to mark the expression as invalid and the associated derived metric name is not visible in the PMNS — so neither metadata nor values can be fetched. Setting this option to 1 switches things up: the unavailable operand gets replaced by the novalue() constructor, with the default metadata of a singular discrete integer metric with no units. The derived metric stays in the PMNS, but attempts to retrieve values via pmFetch(3) will likely return PM_ERR_VALUE (Missing metric value(s)).

Refer to pmSetDerivedControl(3) for the alternative ways controls and options can be set — via the PCP_DERIVED_CONTROL environment variable and/or #control lines in a the definition of a derived metric 🌸

🫶 SEE ALSO 🫶

and that's a wrap on the whole page bestie, you made it all the way through 💖 this one's straight from Performance Co-Pilot, filed under PMREGISTERDERIVED(3) — go forth and register those derived metrics, ur literally the main character of the PMNS now 🌈👑💅

📦 where this page comes from