PMDANIXL(1)

πŸ“¦ version 7.2.2, in arch rolling, fedora 43, fedora 44, ubuntu stonking

πŸ’– PMDANIXL(1) β€” the nixl PMDA, but make it make sense πŸ’–

okay bestie so this is the manual page for pmdanixl, living in section 1 of the General Commands Manual. it's a PCP thing. it's a whole vibe. let's go ✨


🌸 NAME 🌸

pmdanixl β€” the NVIDIA NIXL Performance Metrics Domain Agent πŸ¦‹


✨ SYNOPSIS ✨

$PCP_PMDAS_DIR/nixl/pmdanixl

like, that's it. that's the whole synopsis. she's low maintenance πŸ’…


πŸŽ€ DESCRIPTION πŸŽ€

so pmdanixl is a Performance Co-Pilot (PCP) Performance Metrics Domain Agent (PMDA), and her whole job is collecting performance telemetry from one or more running NIXL (NVIDIA Inference Xfer Library) agents. she's basically the group chat admin for your metrics 🫢

here's the tea: NIXL agents write per-event telemetry into shared-memory ring buffers, and those live in the directory that the NIXL_TELEMETRY_DIR environment variable points at β€” and if you don't set it, it defaults to /var/tmp. our girl pmdanixl discovers the agent files in that directory, drains the new events on every fetch, and accumulates them into per-agent running counters. every discovered agent shows up as its own separate instance in the nixl.agents instance domain, so you can monitor metrics from different agents totally independently. no cap, it's giving main character energy for each agent πŸ‘‘

the metrics are organised into four groups:

  • nixl.transfer.* β€” data transfer byte and request counts (TX and RX). the "how much stuff moved" girlies.
  • nixl.memory.* β€” memory registration and deregistration sizes.
  • nixl.performance.* β€” accumulated transfer time measurements (in microseconds).
  • nixl.errors.* β€” error event counts, broken down by error type. 😭 the sad group but we still stan her.

🌈 NIXL CONFIGURATION 🌈

⚠️ listen bestie: NIXL telemetry is disabled by default. nothing happens until you turn it on, fr fr.

these environment variables must be set in each NIXL application process before it creates its agent, otherwise telemetry data won't get written to shared memory and this PMDA literally cannot discover it. periodt.

  • NIXL_TELEMETRY_ENABLE=1 β€” activates telemetry collection. accepted values are 1, yes, on, true, or enable (case-insensitive). pick your fighter πŸ’…
  • NIXL_TELEMETRY_DIR=dir β€” the directory where NIXL writes its shared-memory telemetry files, one file per agent, named after the agent name supplied at agent creation time. this must match the telemetry_dir setting in $PCP_SYSCONF_DIR/nixl/nixl.conf. defaults to /var/tmp.
  • NIXL_TELEMETRY_BUFFER_SIZE=N β€” how many events the ring buffer can hold before older events get silently overwritten πŸ™ˆ. if your app produces bursts of telemetry faster than the PMDA drains them, bump this number up. defaults to 4096.
  • NIXL_TELEMETRY_RUN_INTERVAL=ms β€” the flush interval in milliseconds used by the NIXL exporter thread. defaults to 100.

a minimal setup for a single application is, like:

export NIXL_TELEMETRY_ENABLE=1
export NIXL_TELEMETRY_DIR=/var/tmp
./my_nixl_application

with multiple applications writing to the same directory, each agent file will be discovered automatically and reported as its own instance. πŸŽ‰ and the PMDA user (set via user in nixl.conf) should match the user running the NIXL applications, so that file permissions align without requiring world-writable telemetry files. safety first, bestie 🧚


πŸ’… INSTALLATION πŸ’…

once your NIXL applications are configured to export telemetry, install the PMDA:

# cd $PCP_PMDAS_DIR/nixl
# ./Install

if NIXL_TELEMETRY_DIR is not the default /var/tmp, set it in the environment before running Install, so it gets recorded in the PMDA's startup configuration. don't skip this, i'm begging 😭

to verify the PMDA is running and instances are visible:

$ pminfo -t nixl
$ pminfo -f nixl.transfer.tx_bytes

vibe check passed βœ…


πŸ¦‹ REMOVAL πŸ¦‹

# cd $PCP_PMDAS_DIR/nixl
# ./Remove

she's gone, she's logged off, it's fine, we're fine πŸ’€


⭐ ENVIRONMENT ⭐

  • NIXL_TELEMETRY_DIR β€” overrides telemetry_dir from nixl.conf at runtime. defaults to /var/tmp.

πŸ“ FILES πŸ“

  • $PCP_SYSCONF_DIR/nixl/nixl.conf β€” PMDA configuration file.
  • $PCP_PMDAS_DIR/nixl/Install β€” installation script.
  • $PCP_PMDAS_DIR/nixl/Remove β€” removal script.
  • $PCP_LOG_DIR/pmcd/nixl.log β€” PMDA log file.

πŸ’œ PCP ENVIRONMENT πŸ’œ

environment variables with the prefix PCP_ are used to parameterize the file and directory names used by PCP. on each installation, the file /etc/pcp.conf contains the local values for these variables. and the $PCP_CONF variable may be used to specify an alternative configuration file, as described in pcp.conf(5). iykyk πŸ“–


πŸ‘‘ SEE ALSO πŸ‘‘

PCPIntro(1), pmcd(1), pminfo(1), pmval(1) and PMDA(3).


okay that's the whole thing, go forth and monitor responsibly 🌸

this was Performance Co-Pilot / PCP β€” mwah, kisses, stan pmdanixl πŸŽ€πŸ’–βœ¨

πŸ“¦ where this page comes from