RFC 9544

Precision Availability Metrics (PAMs) for Services Governed by Service Level Objectives (SLOs)

📅 published March 2024, from ietf

RFC 9544: Precision Availability Metrics (PAMs) for Services Governed by Service Level Objectives (SLOs) 🦋

okay bestie, this RFC is literally about how to check whether a network service is keeping its promises 💅 It came out in March 2024 as an Informational document (not a Standards Track spec), with ISSN 2070-1721. Its header doesn't list anything it obsoletes or updates, so it's standing on its own 🌸

The authors are G. Mirsky, J. Halpern, X. Min, A. Clemm, J. Strassner, and J. Francois. The header lists Halpern at Ericsson, Min at ZTE Corp., Strassner at Futurewei, and Francois at Inria and University of Luxembourg. The header doesn't give affiliations for Mirsky or Clemm, so I'm not guessing 🙈

Abstract

okay so this doc defines a set of metrics for networking services that have performance requirements written as Service Level Objectives (SLOs). these get called "Precision Availability Metrics (PAMs)" and they're super useful for defining and monitoring SLOs ✨ for example, providers and/or customers of an RFC 9543 Network Slice Service can use PAMs to check whether the service is actually delivered in compliance with its defined SLOs 💜

Status of This Memo

this doc is not an Internet Standards Track specification. it's published for informational purposes 🧚

it's an IETF product 🌈 which means it represents the consensus of the IETF community. it got public review and was approved for publication by the Internet Engineering Steering Group (IESG). not every document the IESG approves is a candidate for any level of Internet Standard, so see Section 2 of RFC 7841 for that 💅

info about the current status, any errata, and how to give feedback is at https://www.rfc-editor.org/info/rfc9544 🎀

Copyright (c) 2024 IETF Trust and the persons identified as the document authors. All rights reserved. 🔥

this document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication. please actually read these, bestie, because they describe your rights and restrictions for this document 🫶 Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions, and they're provided without warranty as described in the Revised BSD License 💀

Table of Contents

1.  Introduction
2.  Conventions
  2.1.  Terminology
  2.2.  Acronyms
3.  Precision Availability Metrics
  3.1.  Introducing Violated Intervals
  3.2.  Derived Precision Availability Metrics
  3.3.  PAM Configuration Settings and Service Availability
4.  Statistical SLO
5.  Other Expected PAM Benefits
6.  Extensions and Future Work
7.  IANA Considerations
8.  Security Considerations
9.  Informative References
Acknowledgments
Contributors
Authors' Addresses

1. Introduction ✨

okay so service providers and users constantly need to check how good their network services are 👑 when service-level guarantees are written down (including how they're measured) as part of a contract between the customer and the provider, and SLOs are defined, you NEED a way to verify that what got delivered matches what was negotiated and contractually agreed 💅 Examples of SLOs would be target values for the maximum packet delay (one-way and/or round-trip) or the maximum packet loss ratio that would be considered acceptable 🌸

SLOs can describe how well a particular set of nodes can communicate, based on measurable expectations 🦋 Those expectations can include, but aren't limited to, things like latency, delay variation, loss, capacity/throughput, ordering, and fragmentation 🍓 No matter which SLO parameters you pick or how you measure the service-level parameters, Precision Availability Metrics tell you whether a given service has been available according to expectations at all times ⭐

there are a bunch of metrics that can characterize service quality (lots are documented in the IANA "Performance Metrics" registry [IANA-PM-Registry] according to [RFC8911] and [RFC8912]) and they show how the quality of delivered networking services compares to their SLOs 🧚 The real question isn't so much the absolute service level (like what latency someone actually experienced), it's whether the service is provided in compliance with the negotiated and contracted service levels 💖 For instance, this could include whether the experienced packet delay falls inside an acceptable range that was contracted for the service ✨ The specific quality of service depends on the SLO, or set of SLOs, in effect for a given service 🎀 When an SLO isn't met, the quality of experience can drop for gamers, or even put the safety of a large geographical area at risk 😭

the same service level can be fine for one application and totally unacceptable for another, depending on what that application needs 💅 So it's not enough to just measure service levels over time. the quality of the service being delivered in context (for example, with the applicable SLO in mind) has to be assessed too 🫶 BUT, right now, there are no standard metrics for how well services are delivered relative to their SLOs, or for determining whether SLOs are being met at all times 💀 Metrics like these, plus the instrumentation to support them, are essential for monitoring (making sure networking services perform according to their objectives) and for accounting (keeping a record of the service levels delivered, which matters for monetizing these services and for triaging problems) 👑

the current state of the art includes metrics like interface metrics, which give statistical data on traffic volume and behavior observed at an interface [RFC2863] [RFC8343] 🌈 But they're agnostic about actual service levels and aren't specific to distinct flows 🙈 Flow records [RFC7011] [RFC7012] keep statistics about flows, including flow volume and flow duration, but again, they hold very little info about service levels, let alone whether delivered service levels meet their targets (their associated SLOs) 😭

this specification introduces a new set of metrics, Precision Availability Metrics (PAMs), which capture service levels for a flow, specifically how closely the flow complies with the SLOs in effect ✨ PAMs can be used to assess whether a service is provided in compliance with its defined SLOs 💜 That info can be used in multiple ways, like optimizing service delivery, taking timely action when a service degrades, or accounting for the quality of services delivered 🦋

Availability is discussed in Section 3.4 of [RFC7297] 🌸 In this document, "availability" means that a service characterized by its SLOs is considered unavailable whenever those SLOs are violated, even if basic connectivity is still working 💅 "Precision" refers to services whose service levels are governed by SLOs and must be delivered precisely according to the associated quality and performance requirements 🔥 It's worth noting that precision refers to what is being assessed, NOT the mechanism used to measure it. in other words, it doesn't refer to the precision of the mechanism that measures the actual service levels 🙈 Also, precision with respect to delivering an SLO especially applies when a metric value approaches the threshold levels specified in the SLO 🍓

the methods that provide accurate measurements are separate topics, independent of the definition of the metrics that the results get expressed in 🧚 Likewise, Service Level Expectations (SLEs), as defined in Section 5.1 of [RFC9543], are out of scope for this document 🫶

2. Conventions 🎀

2.1. Terminology 💖

In this document, SLA and SLO are used as defined in [RFC3198]. If you want an applicability example of these concepts in the context of RFC 9543 Network Slice Services, check out Section 5.1 of [RFC9543] 🌸

2.2. Acronyms 🌈

IPFIX  IP Flow Information Export

PAM    Precision Availability Metric

SLA    Service Level Agreement

SLE    Service Level Expectation

SLO    Service Level Objective

SVI    Severely Violated Interval

SVIR   Severely Violated Interval Ratio

SVPC   Severely Violated Packets Count

VFI    Violation-Free Interval

VI     Violated Interval

VIR    Violated Interval Ratio

VPC    Violated Packets Count

3. Precision Availability Metrics 🦋

part 2 drops next, don't go anywhere 🫶

3.1. Introducing Violated Intervals ✨

Okay so bestie, when you're analyzing the availability metrics of a service between two measurement points, you have to pick a time interval to be the unit of PAMs 🎀. [ITU.G.826] uses one second, and honestly? That's reasonable 💅. But some services need a different granularity (e.g., decamillisecond), so the time interval in PAMs is a variable parameter. It's still constant for a particular measurement session 🦋.

Then, for PAMs, every time interval gets sorted into one of three categories: Violated Interval (VI), Severely Violated Interval (SVI), or Violation-Free Interval (VFI) 🫶. Here's the tea:

  • VI is a time interval during which at least one of the performance parameters degraded below its configurable optimal threshold.
  • SVI is a time interval during which at least one of the performance parameters degraded below its configurable critical threshold.
  • Consequently, VFI is a time interval during which all performance parameters are at or better than their respective pre-defined optimal levels.

The monitoring of performance parameters to decide the quality of an interval happens between the elements of the network that are identified in the SLO corresponding to the performance parameter 👀. Mechanisms for setting levels of a threshold of an SLO are outside the scope of this document, so we're not going there today 🙈.

From those definitions, you get a set of basic metrics that count how many time intervals land in each category 🍓:

  • VI count
  • SVI count
  • VFI count

These count metrics are essential for calculating the respective ratios (see Section 3.2) that can be used to assess the instability of a service. Main character energy 👑.

Beyond accounting for violated intervals, it's sometimes super useful to keep counts of packets that violate a performance threshold ✨. For example, this lets you tell apart isolated violation occurrences (like a sporadic issue caused by a temporary spike in a queue depth along the packet's path) from broad violations across multiple packets (like a problem with slow route convergence across the network, or more foundational issues like insufficient network resources) 😭. Keeping those counts and comparing them with the overall amount of traffic also helps assess compliance with statistical SLOs (see Section 4). So the following additional metrics are defined:

  • VPC (Violated Packets Count)
  • SVPC (Severely Violated Packets Count)

3.2. Derived Precision Availability Metrics 🧚

You can make a whole new set of metrics based on the PAMs we just introduced 💜. In this document, these are called "derived PAMs". Some of them are modeled after Mean Time Between Failure (MTBF) metrics, where a "failure" just means a failure to deliver a service according to its SLO 💀.

  • Time since the last violated interval (e.g., since last violated ms or since last violated second). This one is perfect for monitoring the current compliance status of the service, e.g., for trending analysis 📈.
  • Number of packets since the last violated packet. This one is also great for monitoring the current compliance status of the service 🌸.
  • Mean time between VIs (e.g., between violated milliseconds or between violated seconds). This is the arithmetic mean of time between consecutive VIs 🦋.
  • Mean packets between VIs. This is the arithmetic mean of the number of SLO-compliant packets between consecutive VIs. It's another variation of MTBF in a service setting 🫶.

There's an analogous set of metrics for SVI too 🔥:

  • Time since the last SVI (e.g., since last violated ms or since last violated second). This one is useful for monitoring the current compliance status of the service.
  • Number of packets since the last severely violated packet. This one is useful for monitoring the current compliance status of the service too.
  • Mean time between SVIs (e.g., between severely violated milliseconds or between severely violated seconds). This is the arithmetic mean of time between consecutive SVIs.
  • Mean packets between SVIs. This is the arithmetic mean of the number of SLO-compliant packets between consecutive SVIs. It's another variation of "MTBF" in a service setting.

To show a historic degree of precision availability, you can define even more derived PAMs 🌈:

  • Violated Interval Ratio (VIR) is the ratio of the summed numbers of VIs and SVIs to the total number of time unit intervals in a time of the availability periods during a fixed measurement session.
  • Severely Violated Interval Ratio (SVIR) is the ratio of SVIs to the total number of time unit intervals in a time of the availability periods during a fixed measurement session.

3.3. PAM Configuration Settings and Service Availability 🎀

A service provider might want to figure out the current condition of the service its PAMs are maintained for 👀. To make that easier, it's conceivable to pair PAMs with a state model. That state model can tell you whether a service is currently available or unavailable, depending on the network's recent ability to provide service without intervals where violations occur ✨. It's also conceivable to define a state model where transitions happen according to some predefined PAM settings.

Defining a service state model is outside the scope of this document, but this section gives some considerations for how such a state model and its configuration settings could be defined 💅.

For example, a state model could be a Finite State Machine with two states: "available" and "unavailable". The initial state could be "available". A service could then be deemed "unavailable" based on the number of successive interval violations experienced up to a particular observation time moment 😭. To go back to "available", a number of intervals without violations would need to be observed 🫶.

The number of successive intervals with violations, and the number of successive intervals free of violations, that are required for a state to transition to another state is set by a configuration setting. Specifically, these configuration parameters are defined:

Unavailability threshold: The number of successive intervals during which a violation occurs to transition to an unavailable state.

Availability threshold: The number of successive intervals during which no violations must occur to allow transition to an available state from a previously unavailable state.

Additional configuration parameters could be defined to account for the severity of violations. And it's also conceivable to define configuration settings that take VIR and SVIR into account 🧚.

4. Statistical SLO 🌈

Here's something to keep in mind: certain SLAs may be statistical, which means they require the service levels of packets in a flow to adhere to specific distributions 🤓. For example, an SLA might say that a given SLO applies to at least a certain percentage of packets, allowing for a certain level of, say, packet loss and/or exceeding a packet delay threshold 🙈. In that case, each such event doesn't necessarily count as an SLO violation. Even so, it's still useful to keep those statistics, because the number of out-of-SLO packets still matters when you look at it in proportion to the total number of packets 💜.

Along those lines, an SLA might set a multi-tiered SLO for, say, end-to-end latency (from the lowest tier to the highest) like this 🍓:

  • not to exceed 30 ms for any packet;
  • not to exceed 25 ms for 99.999% of packets; and
  • not to exceed 20 ms for 99% of packets.

So in that case, any individual packet with a latency greater than 20 ms and lower than 30 ms can't be considered an SLO violation by itself. Compliance with the SLO might need to be assessed after the fact ⏰.

To support statistical SLOs more directly, you need additional metrics, like histogram metrics for service-level parameters with buckets that match individual SLOs 📊. The definition of histogram metrics is outside the scope of this document and could be considered for future work (see Section 6). But for the example we just looked at, a histogram for a particular flow could be kept with four buckets 🎀:

  • one containing the count of packets within 20 ms;
  • a second with the count of packets between 20 and 25 ms (or simply all within 25 ms);
  • a third with the count of packets between 25 and 30 ms (or merely all packets within 30 ms); and
  • a fourth with the count of anything beyond (or simply a total count).

Of course, the number of buckets and the boundaries between them should correspond to the needs of the SLA associated with the application, i.e., to the specific guarantees and SLOs that were provided 🫶.

5. Other Expected PAM Benefits 💖

PAMs come with several benefits next to other, more conventional performance metrics ✨. Without PAMs, you could run ongoing measurements of service levels, keep a time series of service-level records, and then assess compliance with specific SLOs after the fact 🕰️. But that would mean collecting vast amounts of data that would need to be generated, exported, transmitted, collected, and stored 😩. On top of that, extensive post-processing would be needed to compare that data against SLOs and analyze its compliance. Doing all of this at scale and in real time would be a serious challenge 💀.

Adding PAMs gives a more compact expression of service-level compliance. In that sense, PAMs don't just represent raw data; they express actionable information 🦋. Paired with proper instrumentation, PAMs can help you avoid expensive post-processing 💅.

6. Extensions and Future Work 🔮

Here's a list of items that are outside the scope of this specification but would be super useful extensions and future work opportunities 🌸:

  • A YANG data model will allow PAMs to be incorporated into monitoring applications based on the YANG, NETCONF, and RESTCONF frameworks. It will also enable the configuration and retrieval of PAM-related settings 🧚.
  • A set of IPFIX Information Elements will allow PAMs to be associated with flow records and exported as part of flow data, for example, for processing by accounting applications that assess compliance of delivered services with quality guarantees 📦.
  • Additional second-order metrics, such as "longest disruption of service time" (measuring consecutive time units with SVIs), can be defined and would be deemed useful by some users. At the same time, such metrics can be computed in a straightforward manner and will be application specific in many cases. For this reason, they're omitted here so this specification doesn't get overburdened 🙈.
  • Metrics can be defined to represent histograms for service-level parameters with buckets corresponding to individual SLOs 📊.

7. IANA Considerations 🎀

This document has no IANA actions 🌷.

8. Security Considerations 🔐

Okay bestie, this section is where we get serious, so grab your snacks 🍓. Instrumentation for the metrics used to check SLO compliance is an extremely attractive target for an attacker 😭. If someone messes with how those metrics get maintained, a service could be falsely marked as complying when it isn't, or the other way around, where it gets flagged as non-compliant even though it's actually fine 💀. This document doesn't say how networks should be instrumented to maintain the metrics we talked about, but that instrumentation needs to be properly secured so the measurements stay accurate and nobody can tamper with the metrics being kept 🔒✨.

Where metrics are defined relative to an SLO, the configuration of those SLOs needs to be properly secured too 💅. And where SLOs can be adjusted, the link between any metric instance and a particular SLO must be unambiguous 🎀. The same service levels that count as SLO violations for one flow and should be kept as part of the "violated time units" and related metrics might be totally compliant for another flow 👀. So if it's impossible to tie SLOs and PAMs together, it's better to just keep statistics about the service levels delivered (for example, overall histograms of end-to-end latency) without deciding which ones count as violations 🌸.

By the same token, whether something counts as a "severe" or a "significant" violation depends on configuration settings or context 🤓. Those settings or that context need to be specially secured 🫶. Also, the configuration must be bound to the metrics being maintained, so it's clear which configuration setting was in effect when those metrics were being assessed 🦋. An attacker who can tamper with these configuration settings will make the metrics useless at best, or misleading at worst 😱. Not a vibe, bestie.

9. Informative References 📚

[IANA-PM-Registry] IANA, "Performance Metrics", <https://www.iana.org/assignments/performance-metrics>.

[ITU.G.826] ITU-T, "End-to-end error performance parameters and objectives for international, constant bit-rate digital paths and connections", ITU-T G.826, December 2002.

[RFC2863] McCloghrie, K. and F. Kastenholz, "The Interfaces Group MIB", RFC 2863, DOI 10.17487/RFC2863, June 2000, <https://www.rfc-editor.org/info/rfc2863>.

[RFC3198] Westerinen, A., Schnizlein, J., Strassner, J., Scherling, M., Quinn, B., Herzog, S., Huynh, A., Carlson, M., Perry, J., and S. Waldbusser, "Terminology for Policy-Based Management", RFC 3198, DOI 10.17487/RFC3198, November 2001, <https://www.rfc-editor.org/info/rfc3198>.

[RFC7011] Claise, B., Ed., Trammell, B., Ed., and P. Aitken, "Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of Flow Information", STD 77, RFC 7011, DOI 10.17487/RFC7011, September 2013, <https://www.rfc-editor.org/info/rfc7011>.

[RFC7012] Claise, B., Ed. and B. Trammell, Ed., "Information Model for IP Flow Information Export (IPFIX)", RFC 7012, DOI 10.17487/RFC7012, September 2013, <https://www.rfc-editor.org/info/rfc7012>.

[RFC7297] Boucadair, M., Jacquenet, C., and N. Wang, "IP Connectivity Provisioning Profile (CPP)", RFC 7297, DOI 10.17487/RFC7297, July 2014, <https://www.rfc-editor.org/info/rfc7297>.

[RFC8343] Bjorklund, M., "A YANG Data Model for Interface Management", RFC 8343, DOI 10.17487/RFC8343, March 2018, <https://www.rfc-editor.org/info/rfc8343>.

[RFC8911] Bagnulo, M., Claise, B., Eardley, P., Morton, A., and A. Akhter, "Registry for Performance Metrics", RFC 8911, DOI 10.17487/RFC8911, November 2021, <https://www.rfc-editor.org/info/rfc8911>.

[RFC8912] Morton, A., Bagnulo, M., Eardley, P., and K. D'Souza, "Initial Performance Metrics Registry Entries", RFC 8912, DOI 10.17487/RFC8912, November 2021, <https://www.rfc-editor.org/info/rfc8912>.

[RFC9543] Farrel, A., Ed., Drake, J., Ed., Rokui, R., Homma, S., Makhijani, K., Contreras, L., and J. Tantsura, "A Framework for Network Slices in Networks Built from IETF Technologies", RFC 9543, DOI 10.17487/RFC9543, March 2024, <https://www.rfc-editor.org/info/rfc9543>.

Acknowledgments 🫶

The authors greatly appreciate review and comments by Bjørn Ivar Teigen and Christian Jacquenet. Iconic behaviour from these legends 👑.

Contributors 🌈

Liuyan Han China Mobile 32 XuanWuMenXi Street Beijing 100053 China Email: hanliuyan@chinamobile.com

Mohamed Boucadair Orange 35000 Rennes France Email: mohamed.boucadair@orange.com

Adrian Farrel Old Dog Consulting United Kingdom Email: adrian@olddog.co.uk

Authors' Addresses ✨

Greg Mirsky Ericsson Email: gregimirsky@gmail.com

Joel Halpern Ericsson Email: joel.halpern@ericsson.com

Xiao Min ZTE Corp. Email: xiao.min2@zte.com.cn

Alexander Clemm Email: ludwig@clemm.org

John Strassner Futurewei 2330 Central Expressway Santa Clara, CA 95050 United States of America Email: strazpdj@gmail.com

Jerome Francois Inria and University of Luxembourg 615 Rue du Jardin Botanique 54600 Villers-les-Nancy France Email: jerome.francois@inria.fr

That's the whole squad, bestie 🎀💖

📦 where this page comes from