PMFETCHGROUP(3)

PMFETCHGROUP(3) 💅✨

Library Functions Manual — like, the whole girlies squad of fetchgroup functions, no cap 🫶

---

✨ NAME ✨

pmCreateFetchGroup, pmExtendFetchGroup_item, pmExtendFetchGroup_indom, pmExtendFetchGroup_event, pmExtendFetchGroup_timespec, pmExtendFetchGroup_timeval, pmFetchGroup, pmGetFetchGroupContext, pmClearFetchGroup, pmDestroyFetchGroup — simplified performance metrics value fetch and conversion 🌸

okay so basically it's giving "get your metrics, but make it easy" 💖

---

✨ C SYNOPSIS ✨

#include <pcp/pmapi.h>

int pmCreateFetchGroup(pmFG *ptr, int type, const char *name);
int pmExtendFetchGroup_item(pmFG pmfg, const char *metric,
                            const char *instance, const char *scale,
                            pmAtomValue *out_value, int out_type,
                            int out_sts);
int pmExtendFetchGroup_indom(pmFG pmfg, const char *metric,
                             const char *scale, int out_inst_codes[],
                             char *out_inst_names[],
                             pmAtomValue out_values[], int out_type,
                             int out_stss[], unsigned int out_maxnum,
                             unsigned int *out_num, int *out_sts);
int pmExtendFetchGroup_event(pmFG pmfg, const char *metric,
                             const char *instance, const char *field,
                             const char *scale,
                             struct timespec out_times[],
                             pmAtomValue out_values[], int out_type,
                             int out_stss[], unsigned int out_maxnum,
                             unsigned int *out_num, int *out_sts);
int pmExtendFetchGroup_timespec(pmFG pmfg, struct timespec *out_value);
int pmExtendFetchGroup_timeval(pmFG pmfg, struct timeval *out_value);
int pmGetFetchGroupContext(pmFG pmfg);
int pmFetchGroup(pmFG pmfg);
int pmClearFetchGroup(pmFG pmfg);
int pmDestroyFetchGroup(pmFG pmfg);

cc ... -lpcp

---

✨ DESCRIPTION ✨

bestie. listen. the fetchgroup functions are a registration-based mechanism for fetching groups of performance metrics, and they come with automation for general unit, rate, and type conversions, plus convenient instance and value encodings. 🦋

they are a powerful and compact alternative to the classic Performance Metrics Application Programming Interface (PMAPI(3)) sequence of separate lookup, check, fetch, iterate, extract, and convert functions. like, the whole six-step routine? gone. iconic behavior honestly. 💅

the general idea is literally two stages:

1. the setup stage — the application identifies metrics of interest by name and with the desired conversions, and registers a unique pmAtomValue output location where the fetchgroup system is to later deposit the result. it's also possible to identify a metric with an instance domain, and register a unique vector of pmAtomValue objects for them. 👑
2. the operation stage — one single, simple pmFetchGroup function fetches, decodes, converts, and stores all the metrics to their destinations, where the application can read them directly. ✨

that function may be called repeatedly, and each time new pmAtomValue values will be stored in the same destinations. rate conversions between consecutive samples may be requested. slay 🔥

each fetchgroup is associated with a private PMAPI context, so it can manipulate instance profiles and other such state without disrupting other contexts. the instance profile is manipulated to optimize fetches of individual items, even if some are derived metrics. this private PMAPI context belongs to the fetchgroup, is used for all of its internal operations, and will be destroyed. 💀

multiple fetchgroups may be used concurrently, independently. an opaque type pmFG is used to identify a fetchgroup, which is passed to all related function calls. iykyk 🎀

---

🌸 Creating a fetchgroup 🌸

int pmCreateFetchGroup(pmFG *ptr, int type, const char *name);

this function creates a new fetchgroup, associated with a new PMAPI context. 🫶

the type and name parameters are relayed to pmNewContext(3) for creation of the context. the fetchgroup identifier is returned upon success through the ptr pointer. this object is later used as a parameter to all other fetchgroup functions. the private PMAPI context may be accessed with pmGetFetchGroupContext, if required. ✨

the normal function return code is zero, and ptr is set. this function may fail in case of pmNewContext or memory allocation errors. those are indicated with a negative return code and a cleared ptr value. 😭

---

🌈 Getting the private PMAPI context 🌈

int pmGetFetchGroupContext(pmFG pmfg);

this function returns the private PMAPI context used by the given fetchgroup. it may be safely used to adjust some configuration parameters of the context, such as via pmSetMode(3), before fetchgroup extension and fetching begins. 🦋

however — and this is the important part bestie — mutation of this context by PMAPI functions after this time may disrupt fetchgroup functionality. for example, a pmSetMode call could invalidate one rate-conversion time-step. 🙈

the normal function return code is the context number.

---

💖 Extending a fetchgroup with a metric instance of interest 💖

int pmExtendFetchGroup_item(pmFG pmfg, const char *metric,
                            const char *instance, const char *scale,
                            pmAtomValue *out_value, int out_type,
                            int *out_sts);

this function registers interest in a single metric and an optional instance. ✨

the metric name is given in the mandatory metric parameter, which is checked immediately via pmLookupName(3) and other calls. if and only if the metric has an instance domain, the specific instance of interest may be named by the instance parameter, which is checked immediately via pmNameInDom(3); otherwise pass NULL.

if the fetchgroup context is a set of archives, it is possible that the metric / instance pair is not yet defined at the current time origin. therefore, this function may attempt to seek to the end of the current set of archives temporarily to retry the metric / instance lookup. 👑

the optional scale parameter specifies desired unit/scale/rate conversions for the metric value. it can take the following values:

• NULL — no unit/scale conversion. if the metric has PM_SEM_COUNTER semantics, perform rate conversion.
• rate — perform rate conversion regardless of semantics, and no unit/scale conversion.
• instant — perform no rate conversion regardless of semantics, and no unit/scale conversion.
• EXPRESSION — perform unit/scale/rate conversion as specified by the EXPRESSION, which is parsed by pmParseUnitsStr(3). this may be useful to assert a canonical scaling for the resulting metric value, independent of PCP version or configuration. dimensionality must match the metric, except if rate conversion is requested, in which case the time dimension must be one smaller than the metric's time dimension. note that the type of rate conversion performed here matches the rate(x) function in derived metric expressions, in that it is calculated as the naive difference between previous and current values of a metric, divided by elapsed time. for example, if a counter wraps around, or a non-counter value decreases, a negative output rate may be computed. 💀

the optional but usual out_value parameter specifies the pmAtomValue where the converted result should later be stored. if the value is NULL, fetching and conversions will be attempted, and possible errors reported, but the result tossed away. the mandatory out_type parameter specifies the PM_TYPE_* requested for the output value. it need not match the metric's native type, as the fetchgroup facility is capable of casting between all supported types (including to and from strings). 🌈

any errors subsequently encountered during fetching, unit/scale/rate conversion, or casting, will result in the assignment of a sentinel value to the output pmAtomValue (see the ``UNUSUAL SITUATIONS'' section below). in addition, if the optional out_sts parameter is specified, an appropriate PMAPI error code will be stored there. 😭

as a review: only the pmfg, metric, and out_type parameters are mandatory. the others may be NULL to indicate application disinterest. 🫶

the normal function return code is zero. this function may fail in case of various lookup, type- and conversion-checking errors. those are indicated with a negative return code.

---

🦋 Extending a fetchgroup with a metric instance domain of interest 🦋

int pmExtendFetchGroup_indom(pmFG pmfg, const char* metric,
                             const char *scale, int out_inst_codes[],
                             char *out_inst_names[],
                             pmAtomValue out_values[], int out_type,
                             int out_stss[], unsigned int out_maxnum,
                             unsigned int *out_num, int *out_sts);

this function generalizes the pmExtendFetchGroup_item function by registering interest in a whole instance domain. therefore, the function registers preallocated vectors for output variables (instead of a singleton). instances will be stored in sorted order in elements of those vectors. the concepts are otherwise the same. ✨

the metric name is specified by the mandatory metric parameter. note that it may refer to a metric without an instance domain, in which case the single output value will appear as one unnamed instance. 💅

the optional scale parameter specifies desired unit/scale/rate conversions for the metric value, same as above. 👑

the optional out_inst_codes parameter specifies a vector of integers, where the raw instance number of the fetched metrics should later be stored. 🎀

the optional out_inst_names parameter specifies a vector of strings, where the instance names of the fetched metrics should later be stored. if an instance does not have a corresponding name, a NULL pointer is stored instead. the application must not modify or free(3) strings in that vector. 🙈

the optional out_values parameter specifies a vector of pmAtomValue objects where the converted result should later be stored. the mandatory out_type parameter specifies the PM_TYPE_* requested for the all output values, same as above. 🌸

the optional out_stss parameter specifies a vector of integers where per-instance error codes should be stored. 😭

the mandatory out_maxnum parameter specifies the number of elements of the vectors above. in other words, it tells the fetchgroup the maximum number of instances which are expected. the optional out_num parameter specifies an integer where the actual number of instances should later be stored. it will range between 0 and out_maxnum. it is initialized to 0 by this function. 🫶

Okay so the optional out_sts parameter is like, one single spot where you park the integer status code for the whole entire fetch of this metric. Normally it's just zero, bestie. But other than a severe fetch error, you might see a PM_ERR_TOOBIG chilling in there if the number of instances you actually ran into was bigger than out_maxnum. 💀

Any errors that show up later — during fetching, unit/scale/rate conversion, or casting — will slap a sentinel value onto the appropriate output pmAtomValue (scroll down to the UNUSUAL SITUATIONS section for the tea 🫖). And on top of that, if the optional out_stss parameter was specified, a PMAPI error code gets stored in the matching position.

Quick review moment ✨ — only pmfg, metric, out_type, and out_maxnum are mandatory. Everything else can be NULL to say "nah, not interested, ty though."

The normal return code is zero. This function can fail from all sorts of lookup, type- and conversion-checking errors, and those come back as a negative return code.

---

🦋 Extending a fetchgroup with an event field 🦋

int pmExtendFetchGroup_event(pmFG pmfg, const char *metric,
                             const char *instance, const char *field,
                             const char *scale,
                             struct timespec out_times[],
                             pmAtomValue out_values[], int out_type,
                             int out_stss[], unsigned int out_maxnum,
                             unsigned int *out_num, int *out_sts);

This one registers interest in all instances of one field across all the records of an event metric. And like — because event metrics can return multiple records per fetch, and each record can have multiple fields of a given field metric type, this function registers preallocated vectors for the output variables, same energy as pmExtendFetchGroup_indom. They get filled in temporal/sequential order. No cap. 🌈

The metric name goes in the mandatory metric parameter. It must be of PM_TYPE_EVENT. And if that metric has an instance domain, then the instance parameter becomes mandatory to identify which instance you're actually about.

The field you wanna pull out of the event records is the mandatory field parameter — which is a metric name of normal scalar type. As is typical for event field metrics, it should not have an instance domain. The optional scale parameter specifies the unit/scale conversions you want on this metric value. Rate conversions are not available, because there's straight-up ambiguity about which previous value you'd even compute rates from. 🤷‍♀️

The optional out_times parameter is a vector of timespec structs, and it receives a copy of the timestamp of the event record where each particular field was found. 🕰️

The optional out_values parameter is a vector of pmAtomValue objects where the converted result should later land. The mandatory out_type parameter specifies the PM_TYPE_* you're requesting for the output values.

The optional out_stss parameter is a vector of integers where the per-field error codes get stored.

The mandatory out_maxnum parameter says how many elements are in the vectors above — aka, it tells the fetchgroup the max number of instances expected. The optional out_num parameter is an integer where the actual number of instances should later be stored. It'll range between zero and out_maxnum, and it gets initialized to zero by this function. ✨

Finally, the optional out_sts parameter is a single location where an integer status code for the overall fetch of this metric gets stored. Normally that's zero — even if no event field values were found (in which case out_num would be zero). Other than a severe fetch error, you might see a PM_ERR_TOOBIG here if the number of fields actually encountered was larger than out_maxnum. 💀

Any errors encountered later during fetching, unit/scale conversion, or casting will drop a sentinel value into the appropriate output pmAtomValue (see UNUSUAL SITUATIONS below, bestie). And if the optional out_stss parameter was specified, a PMAPI error code lands in the appropriate position.

Review time again 🎀 — only pmfg, metric, field, out_type, and out_maxnum are mandatory. The rest can be NULL for "not my problem, app's not interested."

Normal return code: zero. It can fail on lookup, type- and conversion-checking errors, and those show up as a negative return code.

---

⭐ Extending a fetchgroup with the fetch timestamp ⭐

int pmExtendFetchGroup_timespec(pmFG pmfg, struct timespec *out_value);
int pmExtendFetchGroup_timeval(pmFG pmfg, struct timeval *out_value);

These functions register interest in the pmHighResResult timestamp. If the out_value pointer is non-NULL, then on every future pmFetchGroup call, the matching result timestamp gets copied right there. Iconic. 🫶

---

🔥 Fetching all metrics in a fetchgroup 🔥

int pmFetchGroup(pmFG pmfg);

This does one pmFetch on its private PMAPI context, including every metric that was registered via prior pmExtendFetchGroup_* calls. It runs all the data extraction and conversion operations needed to populate all the output variables you asked for. Slay. 💅

The normal return code is zero or positive, same as the underlying pmFetch function. This function can fail on severe fetch errors, and those come back negative.

If you get per-metric availability or conversion errors, or severe fetch errors, the output variables get reset to sentinel values and individual error codes get set. And PM_ERR_AGAIN signals a rate-conversion failure because there's no previous value to work with. 😭

BUT — temporarily absent metrics with discrete semantics are exempt from some of that sentinel/error processing: if a pmFetchGroup fails to collect a result for a discrete metric (pmHighResResult pmValueSet.numval==0), then the last seen valid value (if there is one) is retained. This exists to make life easier when you're processing sets of archives that mix once-sampled and repeatedly-sampled metrics. The environment variable PCP_DISCRETE_ONCE can be set to disable this feature.

---

🌸 Clearing a fetchgroup 🌸

int pmClearFetchGroup(pmFG pmfg);

When the current fetch state of a fetchgroup isn't needed anymore, you can explicitly reset it with this. It releases any dynamically stored state but keeps the private PMAPI context intact for later use — like, no change to the context at all, and the context stays at the current fetch offset. It also frees any pointers such as indom instance names or strings that may have been stored in the output variables. 🧚

---

💜 Destroying a fetchgroup 💜

int pmDestroyFetchGroup(pmFG pmfg);

When the fetchgroup is done-done, you can explicitly free it with this function. It releases any dynamically stored state, plus the private PMAPI context. It also frees any pointers such as indom instance names or strings that may have been stored in the output variables. Bye bestie 🦋

---

🎀 EXAMPLE 🎀

The following program demonstrates fetchgroup usage. Run it with different $PCP_DISK_UNITS environment variables to see different unit/rate conversion in effect. ✨

#include <pcp/pmapi.h>
#include <stdio.h>

#define pcpassert(sts) \
    while (sts<0) { fprintf(stderr, "%s\n", pmErrStr(sts)); exit(42); }

int main()
{
    pmFG fg;
    pmAtomValue v, v2;
    enum { v3_maxnum = 100 };
    pmAtomValue v3_values[v3_maxnum];
    char *v3_names[v3_maxnum];
    int v3_stss[v3_maxnum];
    unsigned int v3_num;
    int sts, i;
    char *diskunits = getenv("PCP_DISK_UNITS");
    struct timeval t;

    sts = pmCreateFetchGroup(&fg, PM_CONTEXT_HOST, "local:");
    pcpassert(sts);
    sts = pmExtendFetchGroup_item(fg, "kernel.all.load", "1 minute",
                                  NULL, &v, PM_TYPE_FLOAT, NULL);
    pcpassert(sts);
    sts = pmExtendFetchGroup_item(fg, "kernel.all.idletime", NULL,
                                  "hour", &v2, PM_TYPE_DOUBLE, NULL);
    pcpassert(sts);
    sts = pmExtendFetchGroup_indom(fg, "disk.dev.total", diskunits,
                                   NULL, v3_names,
                                   v3_values, PM_TYPE_STRING,
                                   v3_stss, v3_maxnum, &v3_num, NULL);
    pcpassert(sts);
    sts = pmExtendFetchGroup_timeval(fg, &t);
    pcpassert(sts);

    for (i=0; i < 10; i++) {
        unsigned int j;
        char stamp[28];

        sts = pmFetchGroup(fg);
        pcpassert(sts);
        printf("%s", pmCtime(&t.tv_sec, stamp));
        printf("1-minute load: %f; idletime: %f h\n", v.f, v2.d);
        for (j=0; j < v3_num; j++) {
            if (v3_stss[j] == 0)
                 printf("disk %s i/o operations (%s): %s\n",
                        v3_names[j] ? v3_names[j] : "?",
                        diskunits ? diskunits : "-",
                        v3_values[j].cp);
        }
        sleep(1);
    }

    sts = pmDestroyFetchGroup(fg);
    pcpassert(sts);
    return 0;
}

---

💀 UNUSUAL SITUATIONS 💀

The fetchgroup API supports only the numeric, string and event metric types. Aggregates get rejected during pmExtendFetchGroup_*.

Any strings the fetchgroup API hands to the application are "owned" by the API. Treat them as read-only, bestie — do not modify them and do not free them. 🫶

Error codes are always negative integers, whether they come back as a return value from fetchgroup functions or get stored in out_sts-type variables. Normal result codes are always zero.

Threading warning, and it's a big one: because of the unique ways extracted data is shared between the application and a fetchgroup, the functions in this API are not protected by the multi-threading mutexes conventional elsewhere in PMAPI. Specifically, for any given pmFG, it is not safe to concurrently call two or more fetchgroup API functions, and it is not safe to traverse the registered output variables while calling one of the functions. The calling application must make sure only one thread at a time uses these calls and the registered output variables. On the flip side, concurrency between different pmFG instances is unrestricted, because they share no global data. 🌈

Any pointers passed to a successful pmFetchGroupExtent_* call must stay valid throughout the lifetime of the fetchgroup, since future pmFetchGroup calls may write into them. 👑

✨ DIAGNOSTICS ✨

okay so bestie, when stuff goes wrong you're gonna want the receipts, and the fetchgroup API has several options for collecting those diagnostics. 💅

• Negative integer error codes might come back from each function when something is seriously serious. 😭
• On top of that, every output pmAtomValue can have a matching integer variable hanging out next to it — that's where pmFetchGroup can stash per-metric per-instance error codes for you. 🎀

---

🦋 the sentinel value situation

As an alternative, per-metric per-instance error conditions are also signalled by setting the corresponding pmAtomValue to a sentinel value. If you don't need unambiguous and precise error detection, honestly? this may be sufficient. lowkey enough.

The sentinel values, fr fr:

• negative one — for all integers, including unsigned integers (i.e. all bits are set) 👀
• NaN — for floating point types
• NULL pointer — for strings
• 0.0s — for the timestamp

🌸 and here's the guarantee, no cap: the fetchgroup API promises that once an output pmAtomValue is registered (during a successful pmExtendFetchGroup_* call), it will be cleared to the sentinel value or to a valid converted metric value — from the moment of registration all the way until the pmDestroyFetchGroup call. iconic behaviour honestly.

---

💀 COMPATIBILITY 💀

okay so history lesson, bestie:

• Prior to PCP 7.0, pmExtendFetchGroup_timestamp was just an alias for pmExtendFetchGroup_timeval. the way that it was literally the same thing… 🫶
• To support the PMAPI transition, the old interface and semantics can still be used if applications are recompiled with:

-DPMAPI_VERSION=2

• pmExtendFetchGroup_timestamp — although this is now deprecated and compile-time support will be removed in a future release. 😭

🚨 so if you're out here relying on the old behaviour, pls be aware: this is a limited-time offer, and the clock is ticking. deprecated means deprecated, periodt.

---

👑 SEE ALSO 👑

the girlies you should also be reading:

• PMAPI(3)
• pmLookupName(3)
• pmParseUnitsStr(3)
• pmUseContext(3)
• pmRegisterDerived(3)
• pmExtractValue(3)

---

🌈 aaaand that's the whole page, bestie!! go forth and fetch your groups with confidence 💖

source: Performance Co-Pilot · PCP · PMFETCHGROUP(3) — that's the collection and the page number, and if you need the date just go look at the actual footer bc I'm not making one up 💅✨