RFC 8911

Registry for Performance Metrics

📅 published November 2021, from ietf

RFC 8911: Registry for Performance Metrics 🎀

Okay bestie, this is RFC 8911, and it's called Registry for Performance Metrics 💖 It came out of the Internet Engineering Task Force (IETF) in November 2021 and it's a Standards Track doc, so this is the real deal and not just someone's opinion ✨ The authors are M. Bagnulo (UC3M), B. Claise (Huawei), P. Eardley (BT), A. Morton (AT&T Labs), and A. Akhter (Consultant). It carries ISSN 2070-1721 🦋 The title block doesn't list any Obsoletes or Updates lines, so it isn't replacing or changing any other RFC 🙈

Abstract 💅

Okay so this doc defines the format for the IANA Registry of Performance Metrics 🧚 It also gives a set of guidelines for Registered Performance Metric requesters and reviewers, so both the people asking for a metric and the people checking it are covered 🫶

Status of This Memo 📜

This is an Internet Standards Track document.

This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 7841.

Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at https://www.rfc-editor.org/info/rfc8911.

Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.

Table of Contents 🗂️

   1.  Introduction
   2.  Terminology
   3.  Scope
   4.  Motivations for the Performance Metrics Registry
     4.1.  Interoperability
     4.2.  Single Point of Reference for Performance Metrics
     4.3.  Side Benefits
   5.  Criteria for Performance Metrics Registration
   6.  Performance Metrics Registry: Prior Attempt
     6.1.  Why This Attempt Should Succeed
   7.  Definition of the Performance Metrics Registry
     7.1.  Summary Category
       7.1.1.  Identifier
       7.1.2.  Name
       7.1.3.  URI
       7.1.4.  Description
       7.1.5.  Reference
       7.1.6.  Change Controller
       7.1.7.  Version (of Registry Format)
     7.2.  Metric Definition Category
       7.2.1.  Reference Definition
       7.2.2.  Fixed Parameters
     7.3.  Method of Measurement Category
       7.3.1.  Reference Method
       7.3.2.  Packet Stream Generation
       7.3.3.  Traffic Filter
       7.3.4.  Sampling Distribution
       7.3.5.  Runtime Parameters
       7.3.6.  Role
     7.4.  Output Category
       7.4.1.  Type
       7.4.2.  Reference Definition
       7.4.3.  Metric Units
       7.4.4.  Calibration
     7.5.  Administrative Information
       7.5.1.  Status
       7.5.2.  Requester
       7.5.3.  Revision
       7.5.4.  Revision Date
     7.6.  Comments and Remarks
   8.  Processes for Managing the Performance Metrics Registry Group
     8.1.  Adding New Performance Metrics to the Performance Metrics
           Registry
     8.2.  Backward-Compatible Revision of Registered Performance
           Metrics
     8.3.  Non-Backward-Compatible Deprecation of Registered
           Performance Metrics
     8.4.  Obsolete Registry Entries
     8.5.  Registry Format Version and Future Changes/Extensions
   9.  Security Considerations
   10. IANA Considerations
     10.1.  Registry Group
     10.2.  Performance Metrics Name Elements
     10.3.  New Performance Metrics Registry
   11. Blank Registry Template
     11.1.  Summary
       11.1.1.  ID (Identifier)
       11.1.2.  Name
       11.1.3.  URI
       11.1.4.  Description
       11.1.5.  Reference
       11.1.6.  Change Controller
       11.1.7.  Version (of Registry Format)
     11.2.  Metric Definition
       11.2.1.  Reference Definition
       11.2.2.  Fixed Parameters
     11.3.  Method of Measurement
       11.3.1.  Reference Method
       11.3.2.  Packet Stream Generation
       11.3.3.  Traffic Filtering (Observation) Details
       11.3.4.  Sampling Distribution
       11.3.5.  Runtime Parameters and Data Format
       11.3.6.  Roles
     11.4.  Output
       11.4.1.  Type
       11.4.2.  Reference Definition
       11.4.3.  Metric Units
       11.4.4.  Calibration
     11.5.  Administrative Items
       11.5.1.  Status
       11.5.2.  Requester
       11.5.3.  Revision
       11.5.4.  Revision Date
     11.6.  Comments and Remarks
   12. References
     12.1.  Normative References
     12.2.  Informative References
   Acknowledgments
   Authors' Addresses

1. Introduction 🌸

Okay so the IETF specifies and uses Performance Metrics of protocols and applications transported over its protocols 🫶 Performance Metrics are a huge deal for network operations that use IETF protocols, and [RFC6390] specifies guidelines for their development.

The definition and use of Performance Metrics in the IETF have been fostered in various working groups (WGs), and the big ones are:

  • The "IP Performance Metrics" (IPPM) WG is the WG primarily focusing on Performance Metrics definition at the IETF. It's the main one, no cap 💖
  • The "Benchmarking Methodology" WG (BMWG) defines many Performance Metrics for use in laboratory benchmarking of internetworking technologies. 🧪
  • The "Metric Blocks for use with RTCP's Extended Report Framework" (XRBLOCK) WG (concluded) specified many Performance Metrics related to "RTP Control Protocol Extended Reports (RTCP XR)" [RFC3611], which establishes a framework to allow new information to be conveyed in RTCP, supplementing the original report blocks defined in "RTP: A Transport Protocol for Real-Time Applications" [RFC3550]. Iconic work, honestly ✨
  • The "IP Flow Information eXport" (IPFIX) WG (concluded) specified an Internet Assigned Numbers Authority (IANA) process for new Information Elements. Some Information Elements related to Performance Metrics are proposed on a regular basis. 📬
  • The "Performance Metrics for Other Layers" (PMOL) WG (concluded) defined some Performance Metrics related to Session Initiation Protocol (SIP) voice quality [RFC6035]. 📞

It is expected that more Performance Metrics will be defined in the future -- not only IP-based metrics but also metrics that are protocol specific and application specific. The future is literally so bright 🌈

Despite the importance of Performance Metrics, there are two related problems for the industry 😭

  • First, ensuring that when one party requests that another party measure (or report or in some way act on) a particular Performance Metric, both parties have exactly the same understanding of what Performance Metric is being referred to. Same page, every time 📖
  • Second, discovering which Performance Metrics have been specified, to avoid developing a new Performance Metric that is very similar but not quite interoperable. Nobody wants a near-duplicate that almost works 🙃

These problems can be addressed by creating a Registry for Performance Metrics with the Internet Assigned Numbers Authority (IANA). As such, this document defines the new IANA Registry for Performance Metrics. This is the fix, bestie 💅

Per this document, IANA has created and now maintains the Performance Metrics Registry, according to the maintenance procedures and the format defined in the sections below. The resulting Performance Metrics Registry is for use by the IETF and others. Although the Registry formatting specifications herein are primarily for Registry creation by IANA, any other organization that wishes to create a Performance Metrics Registry may use the same formatting specifications for their purposes. The authors make no guarantee of the Registry format's applicability to any possible set of Performance Metrics envisaged by other organizations, but we encourage others to apply it. In the remainder of this document, unless we explicitly say otherwise, we will refer to the IANA-maintained Performance Metrics Registry as simply the Performance Metrics Registry. 🦋

2. Terminology 📚

okay so first things first, the capital words in this section are a whole thing 💅 When "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" show up in ALL CAPS, they're reading as BCP 14 [RFC2119] [RFC8174] key words, meaning they're actual requirement levels. Lowercase ones are just regular English, no cap 🙈

Performance Metric: A quantitative measure of performance, aimed at an IETF-specified protocol or at an application running over an IETF-specified protocol. Examples are the FTP response time for a complete file download, the DNS Response time to resolve the IP address(es), a database logging time, etc. This matches the definition of a metric in [RFC2330] and is broader than the definition of a Performance Metric in [RFC6390] ✨

Registered Performance Metric: A Performance Metric that's been entered into the Performance Metrics Registry, which IANA runs. It only gets in after clearing every Registry review criterion defined in this document 🎀

Performance Metrics Registry: The IANA Registry that holds the Registered Performance Metrics 🦋

Proprietary Registry: A set of metrics kept in a proprietary Registry, instead of the Performance Metrics Registry 🔒

Performance Metrics Experts: A group of designated experts [RFC8126] picked by the IESG to validate Performance Metrics before the Performance Metrics Registry gets updated. They work super closely with IANA 🧚

Parameter: An input factor that's defined as a variable in a Performance Metric's definition. It's a numerical or other specified factor, one of a set that either defines a metric or sets the conditions it runs under. All Parameters must be known before you can make a measurement with a metric and interpret the results 🍓 There are two kinds: Fixed and Runtime. For Fixed Parameters, the value is specified in the Performance Metrics Registry Entry, and different Fixed values give you different Registered Performance Metrics. For Runtime Parameters, the value gets defined when the Metric Measurement Method runs, and a given Registered Performance Metric can support multiple values for it. Runtime Parameters don't change what the Performance Metric fundamentally is, but some of them have a big influence on the network property being measured and on how you read the results 👑

📝 Note: okay this one is my favorite example 🥹 Think about packet loss with two different Active Measurement Method cases. Case one is packet loss as background loss, where the Runtime Parameter set has a very sparse Poisson stream and only tells you when packets were lost. Real user streams probably see way more loss at those times, because of tail drop or radio errors. Case two is packet loss ratio, which is the complementary probability of delivery ratio. Here the Runtime Parameter set has a very dense, bursty stream that acts like a real user stream, so it shows the loss a stream like that actually experiences. Both are "Loss metrics," but how you interpret the results depends heavily on the Runtime Parameters (at least!). It gets so extreme that you're basically using loss ratio to infer its complementary, delivery ratio 🙃

Active Measurement Methods: Measurement Methods run on traffic that exists only for measurement, gets generated for that reason alone, and whose traffic characteristics are known a priori. The full definition of Active Methods is in Section 3.4 of [RFC7799]. Examples are the Measurement Methods for the one-way delay metric in [RFC7679] and the round-trip delay metric in [RFC2681] ⚡

Passive Measurement Methods: Measurement Methods run on network traffic that's generated either by (1) the end users or (2) network elements that would exist whether or not any measurement was happening. The full definition of Passive Methods is in Section 3.6 of [RFC7799]. One thing to know is that sensitive information may be observed and, as a result, stored in the measurement system 👀

Hybrid Measurement Methods: Measurement Methods that mix Active Methods and Passive Methods to assess Active Metrics, Passive Metrics, or new metrics derived from a priori knowledge and observations of the stream of interest. The full definition of Hybrid Methods is in Section 3.8 of [RFC7799] 🌈

3. Scope 🔭

This document is meant for two different audiences, bestie 🫶

  1. For people preparing a candidate Performance Metric, it gives criteria that the proposal SHOULD meet (see Section 5). It also gives instructions for writing the text for each column of the candidate Performance Metric, plus the references needed for the new Performance Metrics Registry Entry (up to and including the publication of one or more immutable documents, such as an RFC) (see Section 7) 💌
  2. For the appointed Performance Metrics Experts and the IANA personnel running the new IANA Performance Metrics Registry, it defines a set of acceptance criteria that a candidate Registered Performance Metric should be evaluated against, along with requirements for what a candidate Performance Metrics Registry Entry is made of 🧐

Other organizations that standardize performance metrics are encouraged to use the process defined in this memo to propose a candidate Registered Performance Metric. Also, this document may be useful for other organizations defining their own Performance Metrics Registry, and they may reuse the features of the Performance Metrics Registry defined here 🤝

This Performance Metrics Registry applies to Performance Metrics derived from Active Measurement, Passive Measurement, and any other form of Performance Metric. It's built to cover Performance Metrics developed across the IETF, especially for the technologies specified in the following working groups: IPPM, XRBLOCK, IPFIX, and BMWG. This document also analyzes a prior attempt at a Performance Metrics Registry and explains why that design wasn't good enough [RFC6248] 🙅‍♀️

[RFC8912] populates the new Registry with the initial set of entries 🎉

4. Motivations for the Performance Metrics Registry 💫

Here we go through a few reasons why the Performance Metrics Registry is a big deal 🌸

4.1. Interoperability 🔗

Like any IETF Registry, the main goal is managing the registration of Identifiers for use within one or more protocols. For the Performance Metrics Registry specifically, there are two types of protocols that will use the Performance Metrics in it during their operation (by referring to the index values):

Control Protocol: This protocol lets one entity ask another entity to perform a measurement using a specific metric from the Performance Metrics Registry. A big example is the Large-scale Measurement of Broadband Performance (LMAP) framework [RFC7594]. In LMAP terms, the Performance Metrics Registry is used in the LMAP Control Protocol so a Controller can schedule a Measurement Task for one or more Measurement Agents. To make that work, entries in the Performance Metrics Registry must be sufficiently defined so a Measurement Agent implementation can trigger a specific Measurement Task when it gets a Control Protocol message. This requirement really narrows down what kinds of entries are acceptable for the Registry 🎯

Report Protocol: This protocol lets an entity report Measurement Results to another entity. By referencing a specific Registered Performance Metric, you can properly characterize the Measurement Result data being reported. In LMAP terms, the Performance Metrics Registry is used in the LMAP Report Protocol so a Measurement Agent can report Measurement Results to a Collector 📊

Something to note: the LMAP framework explicitly allows using not only the IANA-maintained Performance Metrics Registry, but also other registries containing Performance Metrics. Those could be (1) registries defined by other organizations or (2) private registries. That said, people creating registries for use in an LMAP framework are encouraged to use the Registry format defined here, because it makes life way easier for developers of LMAP Measurement Agents who want to programmatically use the info in those other registries' entries 💖

4.2. Single Point of Reference for Performance Metrics 📍

The Performance Metrics Registry works as a single point of reference for Performance Metrics defined in different IETF working groups. Like we said earlier, several working groups define Performance Metrics in the IETF, and it's honestly hard to keep track of all of them 😭 That leads to multiple definitions of similar Performance Metrics that try to measure the same phenomena in slightly different (and incompatible) ways. A Registry would let the IETF community and others have one list of relevant Performance Metrics defined by the IETF (and by others, where it makes sense) ✨ That single list is also essential for communicating about Performance Metrics: entities that request measurements, run them, and report the results can all benefit from a shared understanding of the referenced Performance Metric 🫶

4.3. Side Benefits 🎁

Having a Registry like this comes with a couple of side benefits. First, the Performance Metrics Registry could act as an inventory of useful, widely used Performance Metrics that different Measurement Agent implementations normally support 🗂️ Second, results from measurements using the Performance Metrics should be comparable even when they're done by different implementations and on different networks, as long as the Performance Metric is properly defined 🔁 BCP 176 [RFC6576] looks at whether results from independent implementations are equivalent, in the context of checking whether metric specifications are complete and clear 🧠 [RFC6576] is a BCP [RFC2026] that defines the Standards Track advancement testing for (Active) IPPM Metrics, and the same process will likely be enough to figure out whether Registered Performance Metrics are specified well enough to give comparable (or equivalent) results 💜 If a Registered Performance Metric has gone through that kind of testing, this SHOULD be noted in "Comments and Remarks" (see Section 7.6), with a reference to the test results 📎

5. Criteria for Performance Metrics Registration ✨

okay bestie, we are NOT stuffing the Performance Metrics Registry with every single combo of Parameters for every metric 💀. It's not possible, and honestly it's not even desirable. A Registered Performance Metric SHOULD be:

  1. Interpretable by the human user. 🧠
  2. Implementable by the software or hardware designer. 🛠️
  3. Deployable by network operators. 🚀
  4. Accurate in terms of producing equivalent results, and for interoperability and deployment across vendors. 🤝
  5. Operationally useful, so that it has significant industry interest and/or has seen deployment. 🔥
  6. Sufficiently tightly defined, so that different values for the Runtime Parameters do not change the fundamental nature of the measurement or change the practicality of its implementation. 💅

so basically, there needs to be evidence that (1) a candidate Registered Performance Metric has serious industry interest or has actually been deployed, and (2) everyone agrees the candidate does its intended job. no notes 🫶

6. Performance Metrics Registry: Prior Attempt 🕰️

there was an earlier try at a Metrics Registry [RFC4148], and it's giving "we tried and it flopped." [RFC6248] obsoleted it because it was "found to be insufficiently detailed to uniquely identify IPPM metrics... [there was too much] variability possible when characterizing a metric exactly", and that led to the IPPM Metrics Registry defined in [RFC4148] having "very few users, if any." 😭

three extra quotes from [RFC6248] are lowkey the tea if you want to understand what went wrong:

  1. "It is not believed to be feasible or even useful to register every possible combination of Type P, metric parameters, and Stream parameters using the current structure of the IPPM Metrics Registry."
  2. "The current registry structure has been found to be insufficiently detailed to uniquely identify IPPM metrics."
  3. "Despite apparent efforts to find current or even future users, no one responded to the call for interest in the RFC 4148 registry during the second half of 2010."

the current approach learns from that drama 🧠 by tightly defining each Registered Performance Metric with only a few variable (Runtime) Parameters, if any, that the measurement designer gets to set. the vibe is that entries stem from different Measurement Methods that need input (Runtime) Parameters to set stuff like Source and Destination addresses, which don't change the fundamental nature of the measurement. the catch is that this could still lead to a big pile of entries in the Performance Metrics Registry. but we're team "less is more" here 🍓: a reduced set of useful metrics beats a huge set with questionable usefulness. 💖

6.1. Why This Attempt Should Succeed ✨

as we said in the last bit, the big problem with the old Registry was that its metrics were way too generic to be useful 😩. this document has stricter criteria for Performance Metric registration (see Section 5) and sets up a group of Performance Metrics Experts who give guidelines for checking whether a Performance Metric is properly specified. 🎀

another big difference is that this time there's at least one clear user for the Performance Metrics Registry: the LMAP framework and protocol. the LMAP protocol is going to use the Performance Metrics Registry values in its operation, and that actually helps show whether a metric is properly defined. we expect the LMAP Control Protocol to let a Controller ask a Measurement Agent to run a measurement using a given metric by embedding the Performance Metrics Registry Identifier in the protocol. a metric and method count as properly specified if they're defined well enough that it's possible (and practical) to implement them in the Measurement Agent. that's exactly where the old attempt went wrong: a Registry Entry with an undefined Type-P (Section 13 of [RFC2330]) lets measurement results vary a lot. 🙈

7. Definition of the Performance Metrics Registry 📋

this Performance Metrics Registry applies to Performance Metrics used for Active Measurement, Passive Measurement, and any other form of Performance Measurement. every category of measurement has its own vibe, so some of the columns below won't apply to every metric category. when a column doesn't apply, it SHOULD be filled with the "N/A" value (Not Applicable). however, the "N/A" value MUST NOT be used by any metric in these columns: Identifier, Name, URI, Status, Requester, Revision, Revision Date, Description. later on, a new category of metrics might need extra columns, and adding new columns is an officially recognized way to extend the Registry 🦋. the specification that defines the new column(s) MUST give general guidelines for filling them in for existing entries. 🫶

the columns are grouped into "Categories" so the Registry is easier to use 🧚. Categories get described at the "Section 7.x" heading level, and columns at the "Section 7.x.y" level. the figure below shows how it's organized, and each entry (row) gives you a complete description of one Registered Performance Metric. ✨

every column is basically a checklist item 💅 that helps stop people from forgetting stuff during registration and Expert Review [RFC8126].

here's the layout of Registry Categories and Columns:

       Category
       ------------------...
       Column |  Column |...

   Summary
   ----------------------------------------------------------------
   Identifier | Name | URI | Desc. | Reference | Change     | Ver |
              |      |     |       |           | Controller |

   Metric Definition
   -----------------------------------------
   Reference Definition | Fixed Parameters |

   Method of Measurement
   ---------------------------------------------------------------------
   Reference | Packet     | Traffic | Sampling     | Runtime    | Role |
   Method    | Stream     | Filter  | Distribution | Parameters |      |
             | Generation |
   Output
   -----------------------------------------
   Type | Reference  | Units | Calibration |
        | Definition |       |             |

   Administrative Information
   -------------------------------------
   Status |Requester | Rev | Rev. Date |

   Comments and Remarks
   --------------------

there's a blank template of the Registry in Section 11 of this memo if you want to copy it 📝.

7.1. Summary Category 🌸

7.1.1. Identifier 🔢

this column gives a numeric Identifier for the Registered Performance Metric. the Identifier of each Registered Performance Metric MUST be unique 🔑. heads up: revising a Metric through the process in Section 8.2 creates a new entry in the Performance Metrics Registry that keeps the same identifier. 👀

the unique Identifier for a Registered Performance Metric is an unbounded integer (range 0 to infinity) ♾️.

the Identifier 0 should be Reserved. the Identifier values from 64512 to 65535 are reserved for private or experimental use, and users may run into overlapping uses 🌈.

when new Registered Performance Metrics get added to the Performance Metrics Registry, IANA SHOULD assign the lowest available Identifier to the new one 🧁.

if a Performance Metrics Expert providing review decides there's a good reason to assign a specific numeric Identifier, maybe leaving a temporary gap in the numbering, then that Performance Metrics Expert SHALL inform IANA of the decision. 🚨

7.1.2. Name ✨

okay so the Name is literally the first thing a human implementer sees when they're deciding if a metric is right for their measurement study, so it has to be super precise and descriptive 💅 later on, users will scroll through the Names to check if the metric they want is already registered, or if there's a similar entry they can use as a starting point for a new one 🦋

Names are built from these elements, separated by an underscore "_" 🎀

   MetricType_Method_SubTypeMethod_... Spec_Units_Output

MetricType: a mix of the directional properties and the metric being measured, such as (but not limited to) these 💖

           +-----------+--------------------------------------+
           | RTDelay   | Round-Trip Delay                     |
           +-----------+--------------------------------------+
           | RTDNS     | Response Time Domain Name Service    |
           +-----------+--------------------------------------+
           | RLDNS     | Response Loss Domain Name Service    |
           +-----------+--------------------------------------+
           | OWDelay   | One-Way Delay                        |
           +-----------+--------------------------------------+
           | RTLoss    | Round-Trip Loss                      |
           +-----------+--------------------------------------+
           | OWLoss    | One-Way Loss                         |
           +-----------+--------------------------------------+
           | OWPDV     | One-Way Packet Delay Variation       |
           +-----------+--------------------------------------+
           | OWIPDV    | One-Way Inter-Packet Delay Variation |
           +-----------+--------------------------------------+
           | OWReorder | One-Way Packet Reordering            |
           +-----------+--------------------------------------+
           | OWDuplic  | One-Way Packet Duplication           |
           +-----------+--------------------------------------+
           | OWBTC     | One-Way Bulk Transport Capacity      |
           +-----------+--------------------------------------+
           | OWMBM     | One-Way Model-Based Metric           |
           +-----------+--------------------------------------+
           | SPMonitor | Single-Point Monitor                 |
           +-----------+--------------------------------------+
           | MPMonitor | Multi-Point Monitor                  |
           +-----------+--------------------------------------+

                                 Table 1

Method: one of the methods defined in [RFC7799], such as (but not limited to) these 🌸

      +-------------+----------------------------------------------+
      | Active      | depends on a dedicated measurement packet    |
      |             | stream and observations of the stream as     |
      |             | described in [RFC7799]                       |
      +-------------+----------------------------------------------+
      | Passive     | depends *solely* on observation of one or    |
      |             | more existing packet streams as described in |
      |             | [RFC7799]                                    |
      +-------------+----------------------------------------------+
      | HybridType1 | Hybrid Type I observations on one stream     |
      |             | that combine both Active Methods and Passive |
      |             | Methods as described in [RFC7799]            |
      +-------------+----------------------------------------------+
      | HybridType2 | Hybrid Type II observations on two or more   |
      |             | streams that combine both Active Methods and |
      |             | Passive Methods as described in [RFC7799]    |
      +-------------+----------------------------------------------+
      | Spatial     | spatial metric as described in [RFC5644]     |
      +-------------+----------------------------------------------+

                                 Table 2

SubTypeMethod: one or more subtypes that describe the features of the entry in more detail, such as (but not limited to) these 🧚

    +----------------+------------------------------------------------+
    | ICMP           | Internet Control Message Protocol              |
    +----------------+------------------------------------------------+
    | IP             | Internet Protocol                              |
    +----------------+------------------------------------------------+
    | DSCPxx         | where xx is replaced by a Diffserv code point  |
    +----------------+------------------------------------------------+
    | UDP            | User Datagram Protocol                         |
    +----------------+------------------------------------------------+
    | TCP            | Transport Control Protocol                     |
    +----------------+------------------------------------------------+
    | QUIC           | QUIC transport protocol                        |
    +----------------+------------------------------------------------+
    | HS             | Hand-Shake, such as TCP's 3-way HS             |
    +----------------+------------------------------------------------+
    | Poisson        | packet generation using Poisson distribution   |
    +----------------+------------------------------------------------+
    | Periodic       | periodic packet generation                     |
    +----------------+------------------------------------------------+
    | SendOnRcv      | sender keeps one packet in transit by sending  |
    |                | when previous packet arrives                   |
    +----------------+------------------------------------------------+
    | PayloadxxxxB   | where xxxx is replaced by an integer, the      |
    |                | number of octets or 8-bit Bytes in the Payload |
    +----------------+------------------------------------------------+
    | SustainedBurst | capacity test, worst case                      |
    +----------------+------------------------------------------------+
    | StandingQueue  | test of bottleneck queue behavior              |
    +----------------+------------------------------------------------+

                                  Table 3

SubTypeMethod values are joined with a hyphen "-" 🔗 that tells you they all belong to this element, and their order doesn't matter when it comes to checking Name uniqueness.

Spec: an immutable document Identifier combined with a document section Identifier 📜 for RFCs, this is the RFC number plus the major section number that specifies this Registry Entry, in the form "RFCXXXXsecY", e.g., RFC7799sec3. Note: the RFC number is not the primary reference specification for the metric definition (for example, [RFC7679] is the primary reference specification for one-way delay metrics) 🙈 until the RFC number gets assigned to the specifying document, it keeps the placeholder "RFCXXXXsecY", and it stays blank in Private Registry Entries that don't have a matching RFC. Anticipating the "RFC10K" problem, the RFC number keeps replacing "RFCXXXX" no matter how many digits it has 🔥 and anticipating Registry Entries from other standards bodies, the form of this Name Element MUST be proposed and reviewed for consistency and uniqueness by the Expert Reviewer 👑

Units: the units of measurement for the output, such as (but not limited to) these 📏

                +------------+----------------------------+
                | Seconds    |                            |
                +------------+----------------------------+
                | Ratio      | unitless                   |
                +------------+----------------------------+
                | Percent    | value multiplied by 100%   |
                +------------+----------------------------+
                | Logical    | 1 or 0                     |
                +------------+----------------------------+
                | Packets    |                            |
                +------------+----------------------------+
                | BPS        | bits per second            |
                +------------+----------------------------+
                | PPS        | packets per second         |
                +------------+----------------------------+
                | EventTotal | for unitless counts        |
                +------------+----------------------------+
                | Multiple   | more than one type of unit |
                +------------+----------------------------+
                | Enumerated | a list of outcomes         |
                +------------+----------------------------+
                | Unitless   |                            |
                +------------+----------------------------+

                                  Table 4

Output: the type of output that comes out of the measurement, such as (but not limited to) these 🍓

           +--------------+------------------------------------+
           | Singleton    |                                    |
           +--------------+------------------------------------+
           | Raw          | multiple singletons                |
           +--------------+------------------------------------+
           | Count        |                                    |
           +--------------+------------------------------------+
           | Minimum      |                                    |
           +--------------+------------------------------------+
           | Maximum      |                                    |
           +--------------+------------------------------------+
           | Median       |                                    |
           +--------------+------------------------------------+
           | Mean         |                                    |
           +--------------+------------------------------------+
           | 95Percentile | 95th percentile                    |
           +--------------+------------------------------------+
           | 99Percentile | 99th percentile                    |
           +--------------+------------------------------------+
           | StdDev       | standard deviation                 |
           +--------------+------------------------------------+
           | Variance     |                                    |
           +--------------+------------------------------------+
           | PFI          | pass, fail, inconclusive           |
           +--------------+------------------------------------+
           | FlowRecords  | descriptions of flows observed     |
           +--------------+------------------------------------+
           | LossRatio    | lost packets to total packets, <=1 |
           +--------------+------------------------------------+

                                  Table 5

Here's an example, as described in Section 4 of [RFC8912] 🌈

      RTDelay_Active_IP-UDP-Periodic_RFC8912sec4_Seconds_95Percentile

Heads up: private registries that follow the format described here SHOULD use the prefix "Priv_" on any Name, so nobody accidentally collides with a public entry 🙈 (more considerations are described in Section 10). Private Registry Entries usually don't have a specifying RFC, so the Spec: element has no clear interpretation for them 🥺

7.1.3. URI 🔗

okay so the URI column is the metric's home address 🏡 The URI column MUST contain a URL [RFC3986] that uniquely identifies and locates the Metric Entry so it is accessible through the Internet. That URL points to a file holding all the human-readable info for one Registry Entry 💅 The URL SHALL reference a target file that is preferably HTML-formatted and contains URLs to referenced sections of HTMLized RFCs, or other reference specifications. These target files are super easy to edit and reuse when you're prepping new entries, which is so smart 🧠 The exact form of each target file's URL, and the target file itself, will be decided by IANA and live on https://www.iana.org/ ✨ Section 4 of [RFC8912], plus the major sections after it, has an example of a target file in HTML form, so go peek if you want to see how it looks 👀

7.1.4. Description 📝

A Registered Performance Metric description is a written version of a particular Performance Metrics Registry Entry 🎀 It's the extra context that goes alongside the Registered Performance Metric Name, so Performance Metrics Registry users can pick the metrics that actually fit what they need 🫶

7.1.5. Reference 📚

This entry points to the specification that holds the candidate Registry Entry that got reviewed and agreed upon, if one exists. An RFC or other specification is the receipt, basically 🧾

7.1.6. Change Controller 👑

This entry names the entity in charge of approving revisions to the Registry Entry, and it SHALL provide contact info (for an individual, where that makes sense) 📞 Somebody has to be the boss of the changes, and this is how you find them 💅

7.1.7. Version (of Registry Format) 🔢

This column holds the version number of the Registry format that was used when the Performance Metric got registered ✨ The format complying with this memo MUST use 1.0. If a new RFC ever changes the Registry format, it'll bring its own new version number to match that format 🌈 The version number of Registry Entries SHALL NOT change unless the Registry Entry is updated to reflect the Registry format (following the procedures in Section 8). So basically, don't bump the number just because you feel like it 🙅‍♀️

7.2. Metric Definition Category 🧪

This category has the columns that spill all the necessary details about the metric definition 💖 That includes the immutable document reference and the values of the input factors called "Fixed Parameters". Those are things the immutable document leaves open, but the Performance Metric gives them one particular value to lock in 🔒

7.2.1. Reference Definition 📖

This entry gives a reference (or references) to the relevant sections of the document or documents that define the metric, plus any supplemental info needed so implementations can read it without any confusion 🤓 A given reference needs to be an immutable document, such as an RFC. For other standards bodies, you'll probably need to point at a specific, dated version of a specification 📅

7.2.2. Fixed Parameters 🔒

Fixed Parameters are Parameters whose values must be specified in the Performance Metrics Registry. The measurement system uses these values, so they're locked in and not up for debate 💅

Where referenced metrics supply a list of Parameters as part of their descriptive template, a subset of those Parameters gets designated as Fixed Parameters. For example, for Active Metrics, Fixed Parameters pin down most or all of the IPPM framework convention "packets of Type-P" from [RFC2330], like the transport protocol, payload length, TTL, and so on 🚀 For Passive Metrics, think about an RTP packet loss calculation. It depends on validating a packet as RTP, which is a multi-packet validation controlled by the MIN_SEQUENTIAL variable defined in [RFC3550] 🧐 Changing MIN_SEQUENTIAL can change the loss report, so this variable could be set as a Fixed Parameter 🎯

Parameters MUST have well-defined names. For human readers, the hanging-indent style is preferred. Any Parameter names and definitions that do not appear in the Reference Method Specification MUST appear in this column (or the Runtime Parameters column) 📋

Parameters MUST have a well-specified data format. No vague vibes allowed 🙅‍♀️

A Parameter that is a Fixed Parameter for one Performance Metrics Registry Entry may be designated as a Runtime Parameter for another Performance Metrics Registry Entry. Same parameter, different role depending on the entry, iconic 🦋

7.3. Method of Measurement Category 🛠️

This category has columns for references to the relevant sections of the immutable document(s), plus any supplemental info needed so the method is unambiguous for implementations 🌸

7.3.1. Reference Method 📐

This entry points to relevant sections of immutable documents, such as RFC(s), that describe the Method of Measurement. For other standards bodies, you'll likely need a specific, dated version of a specification. It also covers any supplemental info needed so implementations that refer to the immutable document text can interpret it unambiguously 🧩

Specifically, this section should include pointers to pseudocode or actual code that could be used for an unambiguous implementation. Receipts for the win 🧾✨

7.3.2. Packet Stream Generation 🌊

This column applies to Performance Metrics that generate traffic as part of their Measurement Method, including, but not necessarily limited to, Active Metrics. That generated traffic is called a "stream", and this column describes its characteristics 💫

Each entry for this column contains the following information:

  • Value: The name of the packet stream scheduling discipline
  • Reference: The specification where the Parameters of the stream are defined

The packet generation stream may require Parameters such as the average packet rate and distribution truncation value for streams with Poisson-distributed inter-packet sending times. If such Parameters are needed, they should be included in either the Fixed Parameters column or the Runtime Parameters column, depending on whether they will be fixed or will be an input for the metric 🎚️

The simplest example of stream specification is singleton scheduling (see [RFC2330]), where a single atomic measurement is conducted. Each atomic measurement could consist of sending a single packet (such as a DNS request) or sending several packets (for example, to request a web page) 🌐 Other streams support a series of atomic measurements using pairs of packets. The packet stream follows a schedule that defines the timing between transmitted packets, and an atomic measurement assesses the reception time between successive packets (e.g., a measurement of Inter-Packet Delay Variation) ⏱️ More complex streams and measurement relationships are possible, so the sky's the limit 🌌

Principally, two different streams are used in IPPM Metrics: (1) Poisson, distributed as described in [RFC2330], and (2) periodic, as described in [RFC3432]. Both Poisson and periodic have their own unique Parameters, and the relevant set of Parameter names and values should be included in either the Fixed Parameters column or the Runtime Parameters column 💜

7.3.3. Traffic Filter 🕵️‍♀️

This column applies to Performance Metrics that observe packets flowing through (the device with) the Measurement Agent, meaning packets that aren't necessarily addressed to the Measurement Agent. That includes, but isn't limited to, Passive Metrics 👀 The filter specifies the traffic that gets measured. This covers protocol field values and ranges, like address ranges, along with flow or session Identifiers 🎀

The Traffic Filter itself depends on the metric's needs and on balancing an operator's measurement needs against a user's need for privacy 🔐 For mechanics, the filter criteria might use BPF (Berkeley Packet Filter), or PSAMP (Packet Sampling) [RFC5475] Property Match Filtering, which reuses IPFIX [RFC7012]. An example BPF string for matching TCP/80 traffic to remote Destination net 192.0.2.0/24 would be dst net 192.0.2.0/24 and tcp dst port 80 🌷 More complex filter engines may allow matching with Deep Packet Inspection (DPI) technology, which is a whole other level 😤

The Traffic Filter includes the following information:

  • Type: The type of Traffic Filter used, e.g., BPF, PSAMP, OpenFlow rule, etc., as defined by a normative reference
  • Value: The actual set of rules expressed

7.3.4. Sampling Distribution 🎲

The sampling distribution decides, out of all the packets that match the Traffic Filter, which one or more of those packets actually get used for the measurement 🎯 One possibility is "all", which means every packet matching the Traffic Filter is considered, but there can be other sampling strategies too 🧁 It includes the following information:

  • Value: The name of the sampling distribution
  • Reference definition: Pointer to the specification where the sampling distribution is properly defined

The sampling distribution may require Parameters. If such Parameters are needed, they should be included in either the Fixed Parameters column or the Runtime Parameters column, depending on whether they will be fixed or will be an input for the metric 🧮

PSAMP is documented in "Sampling and Filtering Techniques for IP Packet Selection" [RFC5475], while "A Framework for Packet Selection and Reporting" [RFC5474] gives more background info 📖 The sampling distribution Parameters might be expressed in terms of the model described in "Information Model for Packet Sampling Exports" [RFC5477] and the process provided in "Flow Selection Techniques" [RFC7014] 🦢

7.3.5. Runtime Parameters 🎛️

In contrast to the Fixed Parameters, Runtime Parameters are Parameters that don't change the fundamental nature of the measurement, and their values aren't specified in the Performance Metrics Registry 🙌 They're left as variables in the Registry to help the measurement system implementer or user. Their values get supplied on execution, configured into the measurement system, and reported with the Measurement Results so the context stays complete 🧠✨

Where metrics supply a list of Parameters as part of their descriptive template, a subset of those Parameters gets designated as Runtime Parameters 💕

Parameters MUST have well-defined names. For human readers, the hanging-indent style is preferred. The names and definitions that do not appear in the Reference Method Specification MUST appear in this column 📝

A data format for each Runtime Parameter MUST be specified in this column, to simplify the control and implementation of measurement devices 🛠️ For example, Parameters that include an IPv4 address can be encoded as a 32-bit integer (i.e., a binary base64-encoded value) or as "ip-address" defined in [RFC6991]. The actual encoding(s) used must be explicitly defined for each Runtime Parameter 🔍 IPv6 addresses and options MUST be accommodated, allowing Registered Performance Metrics to be used in that address family. Other address families are permissible too 🌍

Examples of Runtime Parameters include IP addresses, measurement point designations, start times and end times for measurement, and other info essential to the Method of Measurement 🦋

7.3.6. Role 🎭

okay so some Methods of Measurement have more than one Role, and it's giving multitasking 💅. Like, for a one-way packet delay Active Measurement, there's one Measurement Agent that generates the packets and another Agent that receives them 📦. This column holds the name of the Role(s) for that particular entry. In the one-way delay example, the Registry's Role column should have two entries, one for each Role (Source and Destination) 💖. When a Measurement Agent is told to perform the "Source" Role for the one-way delay metric, it knows it's required to generate packets ✨. The values for this field are defined in the Reference Method of Measurement, and this often means the Role names get shortened to stuff like "Src" 🫶.

When the Role column of a Registry Entry defines more than one Role, the Role SHALL be treated as a Runtime Parameter and supplied for execution 🔥. It should be noted that the LMAP framework [RFC7594] distinguishes the Role from other Runtime Parameters 👑.

7.4. Output Category 🎀

For entries that involve a stream and many singleton measurements, you can specify a statistic in this column to summarize the results into one single value 🌸. If the whole set of measured singletons is output, that gets specified here too 🧚.

Some metrics bake one specific statistic right into the reference metric definition, while others let you pick from several output types or statistics 🦋.

7.4.1. Type 💜

This column has the name of the output type 🍓. The output type defines one single kind of result the metric produces. It can be the raw results (packet send times and singleton metrics), or it can be a summary statistic ✨. The specification of the output type MUST define the format of the output 🔥. In some systems, format specifications make life way easier for both measurement implementation and collection/storage tasks 🫶. Heads up: if you need two different statistics from one measurement (like both "Xth percentile mean" and "Raw"), you have to define a brand new output type, called "Xth percentile mean AND Raw" 🙈. Check out Section 7.1.2 above for a list of output types 💅.

7.4.2. Reference Definition 🌈

This column is a pointer to the specification(s) where the output type and format are defined 📚.

7.4.3. Metric Units 📏

The measured results have to be expressed using some standard dimension or unit of measure, and this column is where you put those units 💖.

When a sample of singletons (see Section 11 of [RFC2330] for definitions of these terms) is collected, this entry specifies the units for each measured value 🧮.

7.4.4. Calibration 🛠️

Some specifications for Methods of Measurement can do an error calibration. Section 3.7.3 of [RFC7679] is one example 🔍. In the Registry Entry, this field identifies a calibration method for the metric. When one is available, the measurement system SHOULD perform the calibration when requested and produce output that shows it's the result of a calibration method 🫧. In-situ calibration can be enabled with an internal loopback that covers as much of the measurement system as possible, does address manipulation as needed, and provides some form of isolation (like deterministic delay) so the send and receive interfaces don't fight each other 😤. This way, some portion of the random and systematic error can be characterized 🎯.

For one-way delay measurements, the error calibration MUST include an assessment of the internal clock synchronization with its external reference, because that internal clock is the thing supplying the timestamps 🕰️. In practice, the time offsets of clocks at both the Source and Destination are needed to estimate the systematic error caused by imperfect clock synchronization. (The time offsets get smoothed, so the random variation usually doesn't show up in the results 🙈.)

Both internal loopback calibration and clock synchronization can be used to estimate the available accuracy of the Output Metric Units 📐. For example, repeated loopback delay measurements will reveal the part of the output result resolution that comes from system noise and is therefore inaccurate 😭.

7.5. Administrative Information 📋

7.5.1. Status 🚦

This entry shows the status of the specification of this Registered Performance Metric. The allowed values are 'Current', 'Deprecated', and 'Obsolete' 💅. Every newly defined Registered Performance Metric starts out with 'Current' Status 🌟.

7.5.2. Requester 🙋

This entry names the requester for the Registered Performance Metric. The requester MAY be a document (like an RFC) or a person 💁.

7.5.3. Revision 🔢

This entry gives the revision number of a Registered Performance Metric. It starts at 0 when the metric is defined and goes up by one for each revision 📈. The exception is a non-backward-compatible revision, so check out Section 8.3 for that one 👀.

7.5.4. Revision Date 📅

This entry shows the date of acceptance of the most recent revision for the Registered Performance Metric. The date SHALL be determined by IANA and the reviewing Performance Metrics Expert 🗓️.

7.6. Comments and Remarks 💬

Besides giving extra details that don't fit anywhere else, this open category (it's just one column) lets unforeseen issues get handled by simply updating this informational entry 🌷.

8. Processes for Managing the Performance Metrics Registry Group ✨

Once a Performance Metric (or a set of them) has been identified for a given application, candidate Performance Metrics Registry Entry specifications written according to Section 7 should be submitted to IANA so they can go through review by the Performance Metrics Experts, as described below 🦋. This same process is used for other changes to a Performance Metrics Registry Entry, like deprecation or revision, which we'll get into later in this section 🔄.

It's desirable for the author(s) of a candidate Performance Metrics Registry Entry to seek review in the relevant IETF working group, or to offer the chance for review on the working group mailing list 📨.

8.1. Adding New Performance Metrics to the Performance Metrics Registry 🆕

Requests to add Registered Performance Metrics to the Performance Metrics Registry SHALL be submitted to IANA. IANA then forwards the request to a designated group of experts (the Performance Metrics Experts) appointed by the IESG 🧑‍🔬. These are the reviewers called for by the Specification Required policy [RFC8126] that's defined for the Performance Metrics Registry. The Performance Metrics Experts check things like compliance with this document, compliance with other applicable Performance Metrics-related RFCs, and consistency with the currently defined set of Registered Performance Metrics 🔎. The smoothest path starts with preparing an Internet-Draft that contains the proposed Performance Metrics Registry Entry, using the template in Section 11. That way the submission formatting benefits from the normal IETF Internet-Draft submission processing (including HTMLization) 💻.

Submission to IANA may happen during IESG review (leading to IETF Standards Action). In that case, an Internet-Draft proposes one or more Registered Performance Metrics to be added to the Performance Metrics Registry, and includes the text of the proposed Registered Performance Metric(s) 📝.

If an RFC-to-be includes a Performance Metric and a proposed Performance Metrics Registry Entry, but the Performance Metrics Expert's review finds that one or more of the criteria listed in Section 5 haven't been met, then the proposed Performance Metrics Registry Entry MUST be removed from the text 🚫. Once there's evidence that the Performance Metric meets the criteria in Section 5, the proposed Performance Metrics Registry Entry SHOULD be submitted to IANA to be evaluated in consultation with the Performance Metrics Experts for registration at that point 🔁.

Authors of proposed Registered Performance Metrics SHOULD review compliance with the specifications in this document before sending their submissions to IANA 🧐.

At least one Performance Metrics Expert should endeavor to complete referred reviews in a timely manner ⏰. If the request is acceptable, the Performance Metrics Experts signify their approval to IANA, and IANA updates the Performance Metrics Registry 🎉. If the request isn't acceptable, the Performance Metrics Experts MAY coordinate with the requester to change it so that it complies. Otherwise, IANA SHALL coordinate resolution of issues on behalf of the expert 🤝. The Performance Metrics Experts MAY choose to reject clearly frivolous or inappropriate change requests outright, but those exceptional circumstances should be rare 🙅‍♀️.

If the proposed Metric is unique in a significant way, it may be necessary to propose a new Name Element Registry, or (more likely) a new Entry in an existing Name Element Registry, in order to describe the Metric properly 🏷️. This proposal is part of the request for the new Metric, so it goes through the same IANA review and approval process 💫.

Decisions by the Performance Metrics Experts may be appealed per Section 10 of [RFC8126] ⚖️.

8.2. Backward-Compatible Revision of Registered Performance Metrics ✨

okay so a revision request is ONLY allowed when the changes keep backward compatibility with implementations of the older Performance Metrics Registry Entry for that Registered Performance Metric 🎀 (so entries with lower revision numbers that share the same Identifier and Name).

The Status field in the Performance Metrics Registry is there to say whether an entry is 'Current', 'Deprecated', or 'Obsolete' 💅. The word 'deprecated' gets used when an entry is replaced, either with a backward-compatible revision (this bit right here) or with a non-backward-compatible revision (see Section 8.3, bestie 🦋).

Also, no policy is defined for revising the Performance Metric Entries in the IANA Registry or for fixing errors in them 🙈. To be super clear, changes and deprecations in the Registry are not encouraged and should be avoided as much as possible 💀. BUT change is inevitable, so this section handles the need for revisions anyway 🫶.

Revisions start when someone sends a candidate Registered Performance Metric definition to IANA, following Section 8.1 🌸. That submission has to name the existing Performance Metrics Registry Entry and explain how and why it should be revised ✨.

The #1 rule for changing existing Registered Performance Metrics is avoiding measurement interoperability problems 🔥. The Performance Metrics Experts have to keep interoperability above all else, no exceptions 👑. Changes to Registered Performance Metrics may only happen in an interoperable way. If a necessary change can't be done in a way that works with unchanged implementations, it MUST result in a brand-new Registered Performance Metric (with a new Name, replacing the RFCXXXXsecY portion of the Name) and possibly the deprecation of the older metric 😭.

A change to a Registered Performance Metric SHALL be treated as backward compatible when:

  1. it fixes an error that is obviously only editorial, or
  2. it fixes an ambiguity in the Registered Performance Metric's definition, and that ambiguity leads to issues severe enough to stop the metric from being used as originally defined, or
  3. it fills in missing information in the metric definition without changing its meaning (for example, explicitly defining 'quantity' semantics for numeric fields that have no Data Type Semantics value), or
  4. it lines up with an external reference that was itself corrected, or
  5. the current Registry format got a new column that isn't relevant to an existing Registered Performance Metric (meaning that new column can be safely filled in with "Not Applicable") 🌈.

If the Performance Metrics Experts decide a revision is allowed and backward compatible under the rules in this document, IANA SHOULD carry out the change(s) in the Performance Metrics Registry 💖. The requester of the change gets added onto the original requester in the Registry 🧚. The Name of the revised Registered Performance Metric, including the RFCXXXXsecY portion, SHALL stay exactly the same, even if the change came from an IETF Standards Action ✨. The revised Registry Entry SHOULD reference the new immutable document, like an RFC 📚. Other standards bodies will probably need to reference a specific, dated version of a specification, in an appropriate category and column 🎀.

Each Registered Performance Metric has a revision number that starts at zero 🔢. Every change that goes through this process bumps the revision number up by one ⭐.

When a revised Registered Performance Metric gets accepted into the Registry, the acceptance date of the most recent revision goes into the Revision Date column for that metric 📅.

If it makes sense, additions to Registered Performance Metrics written in the Comments or Remarks column should include the date. But those additions do NOT count as a revision under this process 🙈.

Older versions of updated Metric Entries stay in the Registry for archival purposes 🗂️. They're kept with every field unmodified (Revision Date included) EXCEPT the Status field, which SHALL be changed to 'Deprecated' 🥀.

Nothing in this process should ever be read as letting the Performance Metrics Experts overrule IETF consensus 🚫. Specifically, any Registered Performance Metrics that were added to the Registry with IETF consensus need IETF consensus to be revised or deprecated 👑.

8.3. Non-Backward-Compatible Deprecation of Registered Performance Metrics

This section explains how to make a non-backward-compatible update to a Registered Performance Metric 💅. A Registered Performance Metric MAY be deprecated and replaced when:

  1. the Registered Performance Metric definition has an error or shortcoming that can't be permissibly changed under Section 8.2 ("Revising Registered Performance Metrics"), or
  2. the deprecation lines up with an external reference that was itself deprecated through that reference's accepted deprecation method.

A request for deprecation goes to IANA, which passes it on to the Performance Metrics Experts for review 🔍. When a Performance Metric gets deprecated, the Performance Metric Description in the Registry MUST be updated to explain the deprecation, AND point to the new Performance Metric created to replace the deprecated one 🫶.

When a new, non-backward-compatible Performance Metric replaces a (now) deprecated metric, the revision number of the new Registered Performance Metric goes up over the value in the deprecated version, and today's date gets entered as the Revision Date of the new metric 📆.

Heads up: using a deprecated Registered Performance Metric on purpose should result in a log entry or a human-readable warning from the application doing it 🚨.

Names and Metric IDs of deprecated Registered Performance Metrics must not be reused 🙅‍♀️.

The deprecated entries stay in the Registry with all Administrative columns unmodified, EXCEPT the Status field (which gets changed to 'Deprecated') 🦋.

8.4. Obsolete Registry Entries

Existing Registry Entries can become obsolete over time because of:

  1. the Registered Performance Metric is found to have considerable errors (and nobody sees the value in the effort to fix it), or
  2. one or more critical References (or sections of them) have been designated obsolete by the SDO, or
  3. other reasons brought to the attention of IANA and the Registry Experts 🌷.

When a Performance Metric Registry Entry gets declared obsolete, the Performance Metric Description in the Registry gets updated to explain why the Entry is now obsolete and hasn't been replaced (because deprecation always involves replacement) 💀.

Obsolete entries are kept with all Administrative columns unmodified, EXCEPT the Status field (which gets changed to 'Obsolete') 🥀.

8.5. Registry Format Version and Future Changes/Extensions

The Registry Format Version defined in this memo is 1.0, and candidate Registry Entries that comply with this memo MUST use 1.0 🔖.

The Registry Format can only be updated by publishing a new RFC with the new format (a Standards Action) 📜.

When a Registered Performance Metric is created or revised, it uses the most recent Registry Format Version ✨.

Only one form of Registry extension is envisaged 🔮:

Adding columns, or both categories and columns, to accommodate unanticipated aspects of new measurements and metric categories.

If the Performance Metrics Registry gets extended this way, the version number of future entries that comply with the extension SHALL be incremented (in either the unit or the tenths digit, depending on how big the extension is) 🔢💜.

9. Security Considerations 🔐

This document defines a Registry structure and does not itself introduce any new security considerations for the Internet 🛡️. The definition of Performance Metrics for this Registry may introduce some security concerns, but the mandatory references should have their own considerations for security, and such definitions should be reviewed with security in mind if the security considerations are not covered by one or more reference standards 😤.

The aggregated results of the Performance Metrics described in this Registry might reveal network topology information that may be considered sensitive 👀. If cases like that are found, access control mechanisms should be applied 🔒😭.

10. IANA Considerations 🏛️

With the background and processes described in earlier sections, IANA has taken the actions described below 👑.

10.1. Registry Group

The new Registry group is named Performance Metrics 🎀. This document refers to it as the "Performance Metrics Group" or "Registry Group", meaning all registrations appearing on <https://www.iana.org/assignments/performance-metrics> (https://www.iana.org/assignments/performance-metrics) 🌐.

For clarity, this document and [RFC8912] use the following conventions to refer to the various IANA registries related to Performance Metrics 📋:

 +===============+===========================+=====================+
 |               | RFC 8911 and RFC 8912     | IANA Web page       |
 +===============+===========================+=====================+
 | Page Title    | Performance Metrics Group | Performance Metrics |
 +---------------+---------------------------+---------------------+
 | Main Registry | Performance Metrics       | Performance Metrics |
 |               | Registry                  | Registry            |
 +---------------+---------------------------+---------------------+
 | Registry Row  | Performance Metrics       | registration (also  |
 |               | Registry Entry            | template)           |
 +---------------+---------------------------+---------------------+

                              Table 6

Registration Procedure: Specification Required 📝

Reference: RFC 8911 📚

Experts: Performance Metrics Experts 🧚‍♀️💖

10.2. Performance Metrics Name Elements

okay bestie, this part sets up the registries for the Performance Metric Name Elements 🧚✨ A Name for a Performance Metric Registry Entry is built from several Elements joined by an "_" (underscore), in the order from Section 7.1.2. IANA created these registries to hold the current options for each Element:

MetricType

Method

SubTypeMethod

Spec

Units

Output

When IANA first set everything up, they filled these in using the value lists for each Name Element in Section 7.1.2 💅 Heads up: the Name Elements in each Registry are case sensitive 👀

When you're getting a Metric Entry ready to register, the developer SHOULD pick Name Elements from the ones already registered 🌸 But if your metric is unique in a significant way, you might need to propose a brand new Name Element so it's described properly, as explained below. ✨

A candidate Metric Entry proposes a set of values for its Name Elements. IANA and an Expert Reviewer look them over. A candidate can even propose a new value for a Name Element (one that isn't already on the list), or a whole new Name Element 😭 These new assignments are handled by IANA under the Specification Required policy [RFC8126], which includes Expert Review. That means a group of Performance Metrics Experts reviews them, and those experts are appointed by the IESG on the recommendation of the Transport Area Directors 🫶

10.3. New Performance Metrics Registry

This document specifies the Performance Metrics Registry 🦋 The Registry has these columns in the Summary category:

Identifier

Name

URI

Description

Reference

Change Controller

Version

The details for these columns, plus extra info from the template for Registry Entries (categories and columns), are defined further in Section 7 💖

Here's the Identifier situation. The Identifier 0 should be Reserved. A Registered Performance Metric's unique Identifier is an unbounded integer (anything from 0 up to infinity). Identifier values from 64512 to 65535 are set aside for private or experimental use, so you might bump into overlapping uses there 👀 When new Registered Performance Metrics get added, IANA SHOULD give each one the lowest Identifier that's still available 🍓 If a Performance Metrics Expert doing the review decides there's a good reason to pick a specific numeric Identifier (even if it leaves a temporary gap in the numbering), the Performance Metrics Expert SHALL tell IANA about that decision ✨

Names that start with the prefix "Priv_" are reserved for private use and are never considered for registration 🙈 The Name column entries are further defined in Section 7.

The URI column holds a URL pointing to each completed Registry Entry. The Registry Entry text SHALL be HTMLized so it's easier to read. (The same tool that HTMLizes Internet-Drafts can do this too.) It also links to the referenced section(s) of an RFC or another immutable document 🔥

The Reference column includes an RFC number, an approved specification designator from another standards body, or some other immutable document 💜

New assignments for the Performance Metrics Registry are handled by IANA in one of two ways: the Specification Required policy [RFC8126] (which includes Expert Review, meaning a group of experts reviews them, and in this case those are the Performance Metrics Experts, appointed by the IESG on the recommendation of the Transport Area Directors), or Standards Action. The experts can start out drawn from the Working Group Chairs, document editors, and members of the Performance Metrics Directorate, plus other sources of experts 🌈

Extensions to the Performance Metrics Registry need an IETF Standards Action 👑 Only one kind of Registry extension is on the table:

  • Adding columns, or both categories and columns, to handle unanticipated aspects of new measurements and metric categories. 🧚

If the Performance Metrics Registry gets extended this way, the version number of future entries that follow the extension SHALL be incremented (in either the unit or the tenths digit, depending on how big the extension is) 💅

11. Blank Registry Template

This section is a blank template to help IANA and Registry Entry writers 🎀

11.1. Summary

This category has multiple indexes into the Registry Entry: the element ID and the Metric Name 🌸

11.1.1. ID (Identifier)

<insert a numeric Identifier, an integer, TBD>

11.1.2. Name

<insert the Name, according to the metric naming convention>

11.1.3. URI

URL: https://www.iana.org/assignments/performance-metrics/ ... <Name>

11.1.4. Description

<provide a description>

11.1.5. Reference

<provide the RFC or other specification that contains the approved candidate Registry Entry>

11.1.6. Change Controller

<provide information regarding the entity responsible for approving
revisions to the Registry Entry (including contact information for an
individual, where appropriate)>

11.1.7. Version (of Registry Format)

11.2. Metric Definition

This category has columns that prompt you to enter all the necessary details about the metric definition, including the immutable document reference and the values of input factors, called "Fixed Parameters" 🦋

11.2.1. Reference Definition

<provide a full bibliographic reference to an immutable document>

<provide a specific section reference and additional clarifications,
if needed>

11.2.2. Fixed Parameters

<list and specify Fixed Parameters, input factors that must be
determined and embedded in the measurement system for use when
needed>

11.3. Method of Measurement

This category has columns for references to the relevant sections of the immutable document(s), plus any supplemental info needed to make the method unambiguous for implementations 🔥

11.3.1. Reference Method

<for the metric, insert relevant section references and supplemental info>

11.3.2. Packet Stream Generation

<provide a list of generation Parameters and section/spec references if needed>

11.3.3. Traffic Filtering (Observation) Details

This category gives the filter details (when there are any), which narrow down the set of packets that count toward the measured results out of all the packets observed 👀

<provide a section reference>

11.3.4. Sampling Distribution

<insert time distribution details, or how this is different from the filter>

11.3.5. Runtime Parameters and Data Format

Runtime Parameters are input factors that have to be figured out, configured into the measurement system, and reported with the results so the context is complete 💖

<provide a list of Runtime Parameters and their data formats>

11.3.6. Roles

<list the names of the different Roles from the Measurement Method>

11.4. Output

This category spells out all the details of the measurement output for the metric 🍓

11.4.1. Type

<insert the name of the output type -- raw results or a selected summary statistic>

11.4.2. Reference Definition

<describe the reference data format for each type of result>

11.4.3. Metric Units

<insert units for the measured results, and provide the reference specification>

11.4.4. Calibration

<insert information on calibration>

11.5. Administrative Items

This category has the administrative info 🌈

11.5.1. Status

<provide status: 'Current' or 'Deprecated'>

11.5.2. Requester

<provide a person's name, an RFC number, etc.>

11.5.3. Revision

<provide the revision number: starts at 0>

11.5.4. Revision Date

<provide the date, in YYYY-MM-DD format>

11.6. Comments and Remarks

<list any additional (informational) details for this entry>

12. References

12.1. Normative References

12.2. Informative References 📚

okay so these are the "good to know" reads, not the must-reads 💅 Nothing here is required to implement the thing, but they're the background that makes the whole doc make sense 🧚

Acknowledgments 🫶

okay the shoutouts are so deserved 😭 Big thanks to Brian Trammell and Bill Cerveny, the IPPM co-chairs while this memo was being developed, for leading several brainstorming sessions on this topic. 👑 Thanks to Barbara Stark and Juergen Schoenwaelder for the detailed feedback and suggestions. 💖 Thanks to Andrew McGregor for suggestions on metric naming, which is literally the hardest part 💀 Thanks to Michelle Cotton for her early IANA review, and to Amanda Baber for answering questions about how the Registry is presented and about getting the complete template through a URL 🧚 Thanks to Roni Even for his review and suggestions to generalize the procedures 🦋 And thanks to all of the Area Directors for their reviews. 🌈 Everyone who showed up for this one is iconic ✨

Authors' Addresses 💜

Marcelo Bagnulo
Universidad Carlos III de Madrid
Av. Universidad 30
28911 Leganes Madrid
Spain

Phone: 34 91 6249500
Email: marcelo@it.uc3m.es
URI:   http://www.it.uc3m.es


Benoit Claise
Huawei

Email: benoit.claise@huawei.com


Philip Eardley
BT
Adastral Park, Martlesham Heath
Ipswich
United Kingdom

Email: philip.eardley@bt.com


Al Morton
AT&T Labs
200 Laurel Avenue South
Middletown, NJ 07748
United States of America

Email: acmorton@att.com


Aamer Akhter
Consultant
118 Timber Hitch
Cary, NC
United States of America

Email: aakhter@gmail.com
📦 where this page comes from