PMC.CMN-600(3)

📦
version 15.1 · 2 versions
, in freebsd 15.1

🌸 pmc.cmn-600 — library functions manual, but make it cute 🌸

okay so bestie, welcome to pmc.cmn-600 (3): the library for accessing the Arm CoreLink CMN-600 Coherent Mesh Network Controller performance counter events. yes, that whole name. iconic, honestly.

🫶 LIBRARY 🫶

Performance Counters Library (libpmc, -lpmc).

📚 SYNOPSIS 📚

#include <pmc.h>

🌈 DESCRIPTION 🌈

okay so real talk: CMN-600 PMU counters may be configured to count any one of a defined set of hardware events. Unlike other performance counters, counters for the CMN-600 require the node ID to set up. no node ID, no counting, periodt.

Node ID information currently can be obtained one of two ways. Using bootverbose, for example set sysctl debug.bootverbose=1 and then load the hwpmc(4) KLD module. The cmn600 module will be loaded automatically as a dependency:

$ sysctl debug.bootverbose=1
$ kldload hwpmc

Another way is to use sysctl to trigger dump of nodes tree to system console:

$ sysctl dev.cmn600.0.dump_nodes=1
💡 heads up, bestie: some BIOS versions of dual-socket machines have no NUMA domain information in ACPI. In such cases, to get more accurate events statistics, set the kernel environment variable hint.cmn600.{unit}.domain={value}. Where {unit} is a cmn600 device unit number and {value} is the NUMA domain of the CPU package containing that CMN-600 controller. Example:
$ kenv hint.cmn600.0.domain=0
$ kenv hint.cmn600.1.domain=1
$ kldunload hwpmc cmn600
$ kldload hwpmc

Arm CoreLink CMN-600 Coherent Mesh Network Controller performance counters are documented in “Revision: r3p2”, Arm CoreLink CMN-600 Coherent Mesh Network Technical Reference Manual, ARM Limited, 2020.

💅 PMC Capabilities 💅

CMN-600 PMU counters support the following capabilities:

  • PMC_CAP_CASCADE — No
  • PMC_CAP_EDGE — No
  • PMC_CAP_INTERRUPT — Yes
  • PMC_CAP_INVERT — No
  • PMC_CAP_READ — Yes
  • PMC_CAP_PRECISE — No
  • PMC_CAP_SYSTEM — Yes
  • PMC_CAP_TAGGING — No
  • PMC_CAP_THRESHOLD — Yes
  • PMC_CAP_USER — No
  • PMC_CAP_WRITE — Yes

✨ Event Qualifiers ✨

Event specifiers for these PMCs support the following common qualifiers:

  • nodeid=nodeid — Request counting for specific event at node nodeid.
  • occupancy=value — Filtering by occupancy type.
  • xpport=port — Count only events matched by port. (East, West, North, South, devport0, devport1 or numeric 0 to 5)
  • xpchannel=channel — Filter events by XP node channel. (REQ, RSP, SNP, DAT or 0, 1, 2, 3)

🦋 Class Name Prefix 🦋

These PMCs are named using a class name prefix of CMN600_PMU_. iykyk.

🌟 Event Specifiers 🌟

The following list of PMC events are available:

🧚 DVM node events 🧚

  • dn_rxreq_dvmop — Number of DVMOP requests received. This includes all the sub-types include TLB invalidate, Branch predictor invalidate, instruction cache (physical and virtual) invalidate.
  • dn_rxreq_dvmsync — Number of DVM Sync requests received.
  • dn_rxreq_dvmop_vmid_filtered — Number of incoming DVMOP requests that are subject to VMID based filtering. This is a measure of the effectiveness of VMID based filtering and potential reduction in DVM snoops.
  • dn_rxreq_retried — Number of incoming requests that are retried. This is a measure of the retry rate.
  • dn_rxreq_trk_occupancy — Counts the tracker occupancy in DN. "occupancy": All, dvmop, dvmsync
  • dn_rxreq_tlbi_dvmop — Number of DVMOP TLB invalidate requests received.
  • dn_rxreq_bpi_dvmop — Number of DVMOP Branch predictor invalidate requests received.
  • dn_rxreq_pici_dvmop — Number of DVMOP physical instruction cache invalidate requests received.
  • dn_rxreq_vivi_dvmop — Number of DVMOP virtual instruction cache invalidate requests received.
  • dn_rxreq_dvmop_other_filtered — Number of DVM op requests to RNDs, BPI or PICI/VICI, that were filtered
  • dn_rxreq_snp_sent — Number of SNPs sent to RNs
  • dn_rxreq_snp_stalled — Number of SNPs stalled to RNs due to lack of Crds
  • dn_rxreq_trk_full — DVM tracker full counter

💖 HN-F node events 💖

  • hnf_cache_miss — Counts total cache misses in first lookup result (high priority)
  • hnf_slc_sf_cache_access — Counts number of cache accesses in first access (high priority)
  • hnf_cache_fill — Counts total allocations in HN SLC (all cache line allocations to SLC)
  • hnf_pocq_retry — Counts number of retried requests
  • hnf_pocq_reqs_recvd — Counts number of requests received by HN
  • hnf_sf_hit — Counts number of SF hits
  • hnf_sf_evictions — Counts number of SF eviction cache invalidations initiated
  • hnf_dir_snoops_sent — Counts number of directed snoops sent (not including SF back invalidation)
  • hnf_brd_snoops_sent — Counts number of multicast snoops sent (not including SF back invalidation)
  • hnf_slc_eviction — Counts number of SLC evictions (dirty only)
  • hnf_slc_fill_invalid_way — Counts number of SLC fills to an invalid way
  • hnf_mc_retries — Counts number of retried transactions by the MC
  • hnf_mc_reqs — Counts number of requests sent to MC
  • hnf_qos_hh_retry — Counts number of times a HighHigh priority request is protocol-retried at the HN-F
  • hnf_qos_pocq — Counts the POCQ occupancy in HN-F. Support argument "occupancy". Accept: All, Read, Write, Atomic, Stash. Default: All.
  • hnf_pocq_addrhaz — Counts number of POCQ address hazards upon allocation
  • hnf_pocq_atomic_addrhaz — Counts number of POCQ address hazards upon allocation for atomic operations
  • hnf_ld_st_swp_adq_full — Counts number of times ADQ is full for Ld/St/SWP type atomic operations while POCQ has pending operations
  • hnf_cmp_adq_full — Counts number of times ADQ is full for CMP type atomic operations while POCQ has pending operations
  • hnf_txdat_stall — Counts number of times HN-F has a pending TXDAT flit but no credits to upload
  • hnf_txrsp_stall — Counts number of times HN-F has a pending TXRSP flit but no credits to upload
  • hnf_seq_full — Counts number of times requests are replayed in SLC pipe due to SEQ being full
  • hnf_seq_hit — Counts number of times a request in SLC hit a pending SF eviction in SEQ
  • hnf_snp_sent — Counts number of snoops sent including directed, multicast, and SF back invalidation
  • hnf_sfbi_dir_snp_sent — Counts number of times directed snoops were sent due to SF back invalidation
  • hnf_sfbi_brd_snp_sent — Counts number of times multicast snoops were sent due to SF back invalidation
  • hnf_snp_sent_untrk — Counts number of times snooped were sent due to untracked RNF's
  • hnf_intv_dirty — Counts number of times SF back invalidation resulted in dirty line intervention from the RN
  • hnf_stash_snp_sent — Counts number of times stash snoops were sent
  • hnf_stash_data_pull — Counts number of times stash snoops resulted in data pull from the RN
  • hnf_snp_fwded — Counts number of times data forward snoops were sent

🔥 HN-I node events 🔥

  • hni_rrt_rd_occ_cnt_ovfl — RRT read occupancy count overflow
  • hni_rrt_wr_occ_cnt_ovfl — RRT write occupancy count overflow
  • hni_rdt_rd_occ_cnt_ovfl — RDT read occupancy count overflow
  • hni_rdt_wr_occ_cnt_ovfl — RDT write occupancy count overflow
  • hni_wdb_occ_cnt_ovfl — WDB occupancy count overflow
  • hni_rrt_rd_alloc — RRT read allocation
  • hni_rrt_wr_alloc — RRT write allocation
  • hni_rdt_rd_alloc — RDT read allocation
  • hni_rdt_wr_alloc — RDT write allocation
  • hni_wdb_alloc — WDB allocation
  • hni_txrsp_retryack — RETRYACK TXRSP flit sent
  • hni_arvalid_no_arready — ARVALID set without ARREADY event
  • hni_arready_no_arvalid — ARREADY set without ARVALID event
  • hni_awvalid_no_awready — AWVALID set without AWREADY event
  • hni_awready_no_awvalid — AWREADY set without AWVALID event
  • hni_wvalid_no_wready — WVALID set without WREADY event
  • hni_txdat_stall — TXDAT stall (TXDAT valid but no link credit available)
  • hni_nonpcie_serialization — Non-PCIe serialization event
  • hni_pcie_serialization — PCIe serialization event

⭐ XP node events ⭐

  • xp_txflit_valid — Number of flits transmitted on a specified port and CHI channel. This is a measure of the flit transfer bandwidth from an XP. Note: On device ports, this event also includes link flit transfers.
  • xp_txflit_stall — Number of cycles when a flit is stalled at an XP waiting for link credits at a specified port and CHI channel. This is a measure of the flit traffic congestion on the mesh and at the flit download ports.
  • xp_partial_dat_flit — Number of times when a partial DAT flit is uploaded onto the mesh from a RN-F_CHIA port. Partial DAT flit transmission occurs when XP is not able to combine two 128b DAT flits and send them over the 256b DAT channel. This can happen under 2 circumstances: 1. Only one 128b DAT flit is received within a transmission time window. 2. Two 128b DAT flits are received but they are not two halves of a single 256b word.

🍓 SBSX node events 🍓

  • sbsx_rd_req — Read request
  • sbsx_wr_req — Write request
  • sbsx_cmo_req — CMO request
  • sbsx_txrsp_retryack — RETRYACK TXRSP flit sent
  • sbsx_txdat_flitv — TXDAT flit seen
  • sbsx_txrsp_flitv — TXRSP flit seen
  • sbsx_rd_req_trkr_occ_cnt_ovfl — Read request tracker occupancy count overflow
  • sbsx_wr_req_trkr_occ_cnt_ovfl — Write request tracker occupancy count overflow
  • sbsx_cmo_req_trkr_occ_cnt_ovfl — CMO request tracker occupancy count overflow
  • sbsx_wdb_occ_cnt_ovfl — WDB occupancy count overflow
  • sbsx_rd_axi_trkr_occ_cnt_ovfl — Read AXI pending tracker occupancy count overflow
  • sbsx_cmo_axi_trkr_occ_cnt_ovfl — CMO AXI pending tracker occupancy count overflow
  • sbsx_arvalid_no_arready — ARVALID set without ARREADY
  • sbsx_awvalid_no_awready — AWVALID set without AWREADY
  • sbsx_wvalid_no_wready — WVALID got set but there was, like, zero WREADY. Awkward. 💅
  • sbsx_txdat_stall — TXDAT stall, aka TXDAT is valid but there's no link credit available 😭
  • sbsx_txrsp_stall — TXRSP stall, fr the same deal: TXRSP is valid but no link credit available

🦋 RN-D node events

  • rnd_s0_rdata_beats — Number of RData beats, RVALID and RREADY, dispatched on port 0. It's a measure of the read bandwidth, and yes it includes CMO responses 💖
  • rnd_s1_rdata_beats — Number of RData beats, RVALID and RREADY, dispatched on port 1. Measure of the read bandwidth, CMO responses included ✨
  • rnd_s2_rdata_beats — Number of RData beats, RVALID and RREADY, dispatched on port 2. Measure of the read bandwidth, CMO responses included 🎀
  • rnd_rxdat_flits — Number of RXDAT flits received. This one is a measure of the true read data bandwidth, and it excludes CMOs 👑
  • rnd_txdat_flits — Number of TXDAT flits dispatched. Measure of the write bandwidth, periodt ✍️
  • rnd_txreq_flits_total — Number of TXREQ flits dispatched. Measure of the total request bandwidth 🌈
  • rnd_txreq_flits_retried — Number of retried TXREQ flits dispatched. Measure of the retry rate, iykyk 🔁
  • rnd_rrt_occ_ovfl — Every single entry in the read request tracker is occupied. It's a measure of oversubscription in the read request tracker 😳
  • rnd_wrt_occ_ovfl — All entries in the write request tracker are occupied. Measure of oversubscription in the write request tracker 💀
  • rnd_txreq_flits_replayed — Number of replayed TXREQ flits. This is the measure of replay rate 🌀
  • rnd_wrcancel_sent — Number of write data cancels sent. The measure of write cancel rate 🙈
  • rnd_s0_wdata_beats — Number of WData beats, WVALID and WREADY, dispatched on port 0. Measure of write bandwidth on AXI port 0 💅
  • rnd_s1_wdata_beats — Number of WData beats, WVALID and WREADY, dispatched on port 1. Measure of write bandwidth on AXI port 1 💅
  • rnd_s2_wdata_beats — Number of WData beats, WVALID and WREADY, dispatched on port 2. Measure of write bandwidth on AXI port 2 💅
  • rnd_rrt_alloc — Number of allocations in the read request tracker. Measure of read transaction count 🧚
  • rnd_wrt_alloc — Number of allocations in the write request tracker. Measure of write transaction count 🧚
  • rnd_rdb_unord — Number of cycles for which Read Data Buffer state machine is in Unordered Mode 🌪️
  • rnd_rdb_replay — Number of cycles for which Read Data Buffer state machine is in Replay mode 🔄
  • rnd_rdb_hybrid — Number of cycles for which Read Data Buffer state machine is in hybrid mode. Hybrid mode is where there is a mix of ordered/unordered traffic, bestie — a little of both 🍓
  • rnd_rdb_ord — Number of cycles for which Read Data Buffer state machine is in ordered Mode 🧘

🌸 RN-I node events

  • rni_s0_rdata_beats — Number of RData beats, RVALID and RREADY, dispatched on port 0. Measure of the read bandwidth, including CMO responses 💖
  • rni_s1_rdata_beats — Number of RData beats, RVALID and RREADY, dispatched on port 1. Measure of the read bandwidth, including CMO responses ✨
  • rni_s2_rdata_beats — Number of RData beats, RVALID and RREADY, dispatched on port 2. Measure of the read bandwidth, including CMO responses 🎀
  • rni_rxdat_flits — Number of RXDAT flits received. Measure of the true read data bandwidth, excluding CMOs 👑
  • rni_txdat_flits — Number of TXDAT flits dispatched. Measure of the write bandwidth 🌈
  • rni_txreq_flits_total — Number of TXREQ flits dispatched. Measure of the total request bandwidth 🔥
  • rni_txreq_flits_retried — Number of retried TXREQ flits dispatched. Measure of the retry rate 🔁
  • rni_rrt_occ_ovfl — All entries in the read request tracker are occupied. Measure of oversubscription in the read request tracker 😳
  • rni_wrt_occ_ovfl — All entries in the write request tracker are occupied. Measure of oversubscription in the write request tracker 💀
  • rni_txreq_flits_replayed — Number of replayed TXREQ flits. This is the measure of replay rate 🌀
  • rni_wrcancel_sent — Number of write data cancels sent. This is the measure of write cancel rate 🙈
  • rni_s0_wdata_beats — Number of WData beats, WVALID and WREADY, dispatched on port 0. Measure of write bandwidth on AXI port 0 💅
  • rni_s1_wdata_beats — Number of WData beats, WVALID and WREADY, dispatched on port 1. Measure of write bandwidth on AXI port 1 💅
  • rni_s2_wdata_beats — Number of WData beats, WVALID and WREADY, dispatched on port 2. Measure of write bandwidth on AXI port 2 💅
  • rni_rrt_alloc — Number of allocations in the read request tracker. Measure of read transaction count 🧚
  • rni_wrt_alloc — Number of allocations in the write request tracker. Measure of write transaction count 🧚
  • rni_rdb_unord — Number of cycles for which Read Data Buffer state machine is in Unordered Mode 🌪️
  • rni_rdb_replay — Number of cycles for which Read Data Buffer state machine is in Replay mode 🔄
  • rni_rdb_hybrid — Number of cycles for which Read Data Buffer state machine is in hybrid mode. Hybrid mode is where there is a mix of ordered/unordered traffic 🍓
  • rni_rdb_ord — Number of cycles for which Read Data Buffer state machine is in ordered Mode 🧘

⭐ CXHA node events

Heads up, bestie: the CXHA events descriptions are guessed. Not me making things up — the page literally says so 🙈
  • cxha_rddatbyp — Number of Read DAT Bypass
  • cxha_chirsp_up_stall — Number of CHI RSP up Stall
  • cxha_chidat_up_stall — Number of CHI DAT up Stall
  • cxha_snppcrd_lnk0_stall — Number of Snoop Pcrd Stall on Link 0
  • cxha_snppcrd_lnk1_stall — Number of Snoop Pcrd Stall on Link 1
  • cxha_snppcrd_lnk2_stall — Number of Snoop Pcrd Stall on Link 2
  • cxha_reqtrk_occ — Request Tracker Occupancy
  • cxha_rdb_occ — Read Data Buffer Occupancy
  • cxha_rdbbyp_occ — Read Data Buffer Bypass Occupancy
  • cxha_wdb_occ — Write Data Buffer Occupancy
  • cxha_snptrk_occ — Snoop Tracker Occupancy
  • cxha_sdb_occ — SDB Occupancy
  • cxha_snphaz_occ — Snoop Hazard Occupancy

💜 CXRA node events

  • cxra_req_trk_occ — Request tracker occupancy
  • cxra_snp_trk_occ — Snoop tracker occupancy
  • cxra_rd_dat_buf_occ — Read data buffer occupancy
  • cxra_wr_dat_buf_occ — Write data buffer occupancy
  • cxra_snp_sink_buf_occ — Snoop sink buffer occupancy
  • cxra_snp_bcasts — Snoop broadcasts
  • cxra_req_chains — Number of request chains formed larger than one
  • cxra_req_chain_avg_len — Average size of request chains, only for chain sizes larger than one
  • cxra_chi_rsp_upload_stalls — Local RA upload stalls to CHI because of contention with HA
  • cxra_chi_dat_upload_stalls — Local RA upload stalls to CHI because of contention with HA
  • cxra_dat_pcrd_stalls_lnk0 — Memory Data Request available, but no DAT Pcrd to send over CCIX per LinkEnd 0
  • cxra_dat_pcrd_stalls_lnk1 — Memory Data Request available, but no DAT Pcrd to send over CCIX per LinkEnd 1
  • cxra_dat_pcrd_stalls_lnk2 — Memory Data Request available, but no DAT Pcrd to send over CCIX per LinkEnd 2
  • cxra_req_pcrd_stalls_lnk0 — Memory Data Request available but no Req Pcrd to send over CCIX per LinkEnd 0
  • cxra_req_pcrd_stalls_lnk1 — Memory Data Request available but no Req Pcrd to send over CCIX per LinkEnd 1
  • cxra_req_pcrd_stalls_lnk2 — Memory Data Request available but no Req Pcrd to send over CCIX per LinkEnd 2
  • cxra_ext_rsp_stall — CHI external RSP stall
  • cxra_ext_dat_stall — CHI external DAT stall

💖 CXLA node events

  • cxla_rx_tlp_link0 — RX TLP for Link 0
  • cxla_rx_tlp_link1 — RX TLP for Link 1
  • cxla_rx_tlp_link2 — RX TLP for Link 2
  • cxla_tx_tlp_link0 — TX TLP for Link 0
  • cxla_tx_tlp_link1 — TX TLP for Link 1
  • cxla_tx_tlp_link2 — TX TLP for Link 2
  • cxla_rx_cxs_link0 — RX CXS for Link 0
  • cxla_rx_cxs_link1 — RX CXS for Link 1
  • cxla_rx_cxs_link2 — RX CXS for Link 2
  • cxla_tx_cxs_link0 — TX CXS for Link 0
  • cxla_tx_cxs_link1 — TX CXS for Link 1
  • cxla_tx_cxs_link2 — TX CXS for Link 2
  • cxla_avg_rx_tlp_sz_dws — Average RX TLP size in DWs
  • cxla_avg_tx_tlp_sz_dws — Average TX TLP size in DWs
  • cxla_avg_rx_tlp_sz_ccix_msg — Average RX TLP size in CCIX messages
  • cxla_avg_tx_tlp_sz_ccix_msg — Average TX TLP size in CCIX messages
  • cxla_avg_sz_rx_cxs_dw_beat — Average size of RX CXS in DWs within a beat
  • cxla_avg_sz_tx_cxs_dw_beat — Average size of TX CXS in DWs within a beat
  • cxla_tx_cxs_link_credit_backpressure — TX CXS link credit backpressure
  • cxla_rx_tlp_buffer_full — RX TLP buffer full and backpressured
  • cxla_tx_tlp_buffer_full — TX TLP buffer full and backpressured
  • cxla_avg_latency_process_rx_tlp — Average latency to process an RX TLP
  • cxla_avg_latency_form_tx_tlp — Average latency to form a TX TLP

👀 SEE ALSO

pmc(3), pmc.amd(3), pmc.atom(3), pmc.core(3), pmc.core2(3), pmc.corei7(3), pmc.corei7uc(3), pmc.iaf(3), pmc.soft(3), pmc.tsc(3), pmc.westmere(3), pmc.westmereuc(3), pmc_cpuinfo(3), pmclog(3), hwpmc(4)

Go read the whole friend group, bestie 🫶

🎀 HISTORY

  • The pmc library first appeared in FreeBSD 6.0. Vintage icon behavior 💅
  • The pmc.cmn-600 driver was added in FreeBSD 14.0. Newer era, still slaying ✨

🧚 AUTHORS

  • The Performance Counters Library (libpmc, -lpmc) library was written by Joseph Koshy <jkoshy@FreeBSD.org>, Oleksandr Rybalko <ray@FreeBSD.org>. Legends, honestly 👑
  • The CMN-600 PMU driver was sponsored by ARM Ltd. Shoutout 🫶
  • This manual page was written by Oleksandr Rybalko <ray@FreeBSD.org>. Not all heroes wear capes 🌈

and that's the whole page, bestie — Debian, December 19, 2021, PMC.CMN-600(3), ate and left no crumbs 💅✨

📦 where this page comes from
  • freebsd 15.1: base 15.1
  • Copyright (c) 2021 ARM Ltd. All rights reserved
  • 💖 girlified by deepseek-ai/DeepSeek-V4.1-Flash-1017 on : a language model's rewrite of the original, under its terms