RFC 9097
Metrics and Methods for One-Way IP Capacity
RFC 9097: Metrics and Methods for One-Way IP Capacity 🦋
okay bestie, this is RFC 9097, and it's giving main character energy 💅 It's titled Metrics and Methods for One-Way IP Capacity, published in November 2021 by the IETF (Internet Engineering Task Force) as a Standards Track document. The authors are A. Morton (AT&T Labs), R. Geib (Deutsche Telekom), and L. Ciavattone (AT&T Labs). The ISSN is 2070-1721. The title block doesn't list anything it obsoletes or updates 🌸
Abstract ✨
okay so the abstract is short and sweet 🍓 This memo goes back to the Network Capacity Metrics problem that [RFC5136] first looked at. It then specifies a more practical Maximum IP-Layer Capacity Metric definition that's built for measurement, and it lays out the matching Methods of Measurement 💜
Status of This Memo 🎀
This is an Internet Standards Track document 💖
This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It went through public review and got approved for publication by the Internet Engineering Steering Group (IESG). For more on Internet Standards, check Section 2 of RFC 7841 🌈
If you want the current status, any errata, or a way to send feedback, go to https://www.rfc-editor.org/info/rfc9097 ✨
Copyright Notice 🧚
Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved. The legal fine print is that this document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they stood on the publication date. Please actually read those, because they explain your rights and restrictions 🙈 Also, any Code Components pulled out of this document have to include the Revised BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come with no warranty, same as the Revised BSD License says 🔥
Table of Contents 🍓
1. Introduction
1.1. Requirements Language
2. Scope, Goals, and Applicability
3. Motivation
4. General Parameters and Definitions
5. IP-Layer Capacity Singleton Metric Definitions
5.1. Formal Name
5.2. Parameters
5.3. Metric Definitions
5.4. Related Round-Trip Delay and One-Way Loss Definitions
5.5. Discussion
5.6. Reporting the Metric
6. Maximum IP-Layer Capacity Metric Definitions (Statistics)
6.1. Formal Name
6.2. Parameters
6.3. Metric Definitions
6.4. Related Round-Trip Delay and One-Way Loss Definitions
6.5. Discussion
6.6. Reporting the Metric
7. IP-Layer Sender Bit Rate Singleton Metric Definitions
7.1. Formal Name
7.2. Parameters
7.3. Metric Definition
7.4. Discussion
7.5. Reporting the Metric
8. Method of Measurement
8.1. Load Rate Adjustment Algorithm
8.2. Measurement Qualification or Verification
8.3. Measurement Considerations
9. Reporting Formats
9.1. Configuration and Reporting Data Formats
10. Security Considerations
11. IANA Considerations
12. References
12.1. Normative References
12.2. Informative References
Appendix A. Load Rate Adjustment Pseudocode
Appendix B. RFC 8085 UDP Guidelines Check
B.1. Assessment of Mandatory Requirements
B.2. Assessment of Recommendations
Acknowledgments
Authors' Addresses1. Introduction 🌸
okay so the IETF has been working on Network Capacity and Bulk Transport Capacity (BTC) for over twenty years 😭 During that time, the performance community has produced Informative definitions in [RFC3148] for the Framework for Bulk Transport Capacity, in [RFC5136] for Network Capacity and Maximum IP-Layer Capacity, and in "Model-Based Metrics for Bulk Transport Capacity" [RFC8337], which holds the Experimental metric definitions and methods.
This memo revisits the Network Capacity Metrics problem, which was first looked at in [RFC3148] and then in [RFC5136]. Maximum IP-Layer Capacity and Bulk Transfer Capacity [RFC3148] (goodput) are different metrics, bestie 💅 Maximum IP-Layer Capacity is like the theoretical goal for goodput. [RFC5136] has lots of metrics, like Available Capacity. What you measure depends on the network path under test and the use case. The main use case here is figuring out the Maximum Capacity of one or more networks where the subscriber gets specific performance assurances (sometimes called Internet access), or where someone wants to test the limit of the technology used on a path. For example, if a user subscribes to a 1 Gbps service, then the user, the Service Provider, and maybe other parties all want to make sure that performance level is actually delivered. Once a test confirms the subscribed performance level, a tester can go hunting for a bottleneck somewhere else 🦋
This memo recognizes how important it is to define a Maximum IP-Layer Capacity Metric now, when Internet subscription speeds have jumped up dramatically. It wants a definition that's both practical and effective for the performance community's needs, including Internet users 🎀 The metric definitions are meant to use Active Methods of Measurement [RFC7799], and every metric comes with a Method of Measurement.
The most direct Active Measurement of IP-Layer Capacity would use IP packets. In practice, though, a transport header is needed to get through address and port translators. UDP gives the most direct way to assess this, and the measurement study looking at whether UDP is viable as a general Internet transport protocol [copycat] found that a high percentage of tested paths support UDP transport 🧚 A number of liaison statements have also been exchanged on this topic [LS-SG12-A] [LS-SG12-B], discussing the lab and field tests that back up the UDP-based approach to IP-Layer Capacity measurement.
This memo also recognizes the updates to the IP Performance Metrics (IPPM) Framework [RFC2330] that have been published since 1998. In particular, it uses [RFC7312] for the Advanced Stream and Sampling Framework, and [RFC8468] for its IPv4, IPv6, and IPv4-IPv6 Coexistence Updates ✨
Appendix A describes the load rate adjustment algorithm, using pseudocode. Appendix B talks about how well the algorithm complies with [RFC8085] 💜
1.1. Requirements Language 🍓
okay here's the requirements language bit 🙈 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.
2. Scope, Goals, and Applicability 💖
The scope of this memo is to define Active Measurement metrics and matching methods that unambiguously determine Maximum IP-Layer Capacity, plus some useful secondary metrics 🌈
Another goal is to harmonize the specified Metric and Method across the industry. This memo is the vehicle that captures IETF consensus, which could lead to changes in the specifications of other Standards Development Organizations (SDOs). Those changes could happen through each SDO's normal contribution process or through liaison exchange 🦋
Secondary goals are adding considerations for test procedures and giving interpretation of the Maximum IP-Layer Capacity results. The point of that interpretation is to spot cases where more testing is warranted, maybe with alternate configurations ✨ Fostering the development of protocol support for this Metric and Method of Measurement is also a goal of this memo. All the active testing protocols the IPPM WG has defined so far are UDP based, which meets a key requirement of these methods 💅 Supporting protocol development to measure this metric according to the specified method is a key future contribution to Internet measurement 🔥
The load rate adjustment algorithm's scope is limited to helping determine the Maximum IP-Layer Capacity in the context of an infrequent, diagnostic, short-term measurement. It is RECOMMENDED to stop non-measurement traffic that shares a subscriber's dedicated resources while testing. Measurements may not be accurate, and throughput of competing elastic traffic may be greatly reduced 😭
The primary application of the Metrics and Methods of Measurement described here is the same as what is described in Section 2 of [RFC7497], where:
The access portion of the network is the focus of this problem statement. The user typically subscribes to a service with bidirectional [Internet] access partly described by rates in bits per second.
In addition, using the load rate adjustment algorithm described in Section 8.1 comes with these extra applicability limits:
- It MUST only be used in the application of diagnostic and operations measurements as described in this memo.
- It MUST only be used in circumstances consistent with Section 10 ("Security Considerations").
- If a network operator is certain of the IP-Layer Capacity to be validated, then testing MAY start with a fixed-rate test at the IP-Layer Capacity and avoid activating the load adjustment algorithm. However, the stimulus for a diagnostic test (such as a subscriber request) strongly implies that there is no certainty, so the load adjustment algorithm is RECOMMENDED. 🙈
Further, the Metrics and Methods of Measurement are meant for use where specific exact path information is unknown within a range of possible values:
- The subscriber's exact Maximum IP-Layer Capacity is unknown. This happens sometimes, since service rates can go up due to upgrades without a subscriber asking, or get increased to give a surplus that makes up for possible underestimates from TCP-based testing 🌸
- The size of the bottleneck buffer is unknown.
Finally, the measurement system's load rate adjustment algorithm SHALL NOT be given the exact capacity value to be validated a priori. This restriction keeps the result fair and removes an opportunity for nefarious operation that knowing the correct answer would enable 💀
3. Motivation 🧚
As with any problem that's been worked on for many years in various SDOs without any special attempts at coordination, various solutions for Metrics and Methods have popped up 🍓
There are five factors that have changed (or began to change) in the 2013-2019 time frame. The presence of any one of them on the path requires features in the measurement design to account for the changes:
- Internet access is no longer the bottleneck for many users (but subscribers expect network providers to honor contracted performance).
- Both transfer rate and latency are important to a user's satisfaction.
- UDP's role in transport is growing in areas where TCP once dominated.
- Content and applications are moving physically closer to users.
- There is less emphasis on ISP gateway measurements, possibly due to less traffic crossing ISP gateways in the future.
okay that's part 1 of 9, and there's way more tea coming 🦋✨
4. General Parameters and Definitions ✨
okay so this section is the list of REQUIRED input factors you need to specify a Sender or Receiver metric 💅 let's go through them bestie 🫶
- Src: one of the addresses of a host (such as a globally routable IP address) 🌍
- Dst: one of the addresses of a host (such as a globally routable IP address) 🎯
- MaxHops: the limit on the number of Hops a specific packet may visit as it traverses from the host at Src to the host at Dst (implemented in the TTL or Hop Limit) 🦘
- T0: the time at the start of a measurement interval, when packets are first transmitted from the Source ⏰
- I: the nominal duration of a measurement interval at the Destination (default 10 sec) 🕙
- dt: the nominal duration of m equal sub-intervals in I at the Destination (default 1 sec) ⏱️
- dtn: the beginning boundary of a specific sub-interval, n, one of m sub-intervals in I 🧩
- FT: the feedback time interval between status feedback messages communicating measurement results, sent from the Receiver to control the Sender. The results are evaluated throughout the test to determine how to adjust the current offered load rate at the Sender (default 50 msec) 🔁
- Tmax: a maximum waiting time for test packets to arrive at the Destination, set sufficiently long to disambiguate packets with long delays from packets that are discarded (lost), such that the distribution of one-way delay is not truncated 🧐
- F: the number of different flows synthesized by the method (default one flow) 🎀
- Flow: the stream of packets with the same n-tuple of designated header fields that (when held constant) result in identical treatment in a multipath decision (such as the decision taken in load balancing). Note: The IPv6 flow label SHOULD be included in the flow definition when routers have complied with the guidelines provided in [RFC6438]. 🌊
- Type-P: the complete description of the test packets for which this assessment applies (including the flow-defining fields). Note that the UDP transport layer is one requirement for test packets specified below. Type-P is a concept parallel to "population of interest" as defined in Clause 6.1.1 of [Y.1540]. 📦
- Payload Content: an aspect of the Type-P Parameter that can help improve measurement determinism. Specifying packet payload content helps ensure IPPM Framework-conforming Metrics and Methods. If there is payload compression in the path and tests intend to characterize a possible advantage due to compression, then payload content SHOULD be supplied by a pseudorandom sequence generator, by using part of a compressed file, or by other means. See Section 3.1.2 of [RFC7312]. 🎲
- PM: a list of fundamental metrics, such as loss, delay, and reordering, and corresponding target performance threshold(s). At least one fundamental metric and target performance threshold MUST be supplied (such as one-way IP packet loss [RFC7680] equal to zero). 📏
One non-Parameter that is needed for several metrics is defined below:
- T: the host time of the first test packet's arrival as measured at the Destination Measurement Point, or MP(Dst). There may be other packets sent between Source and Destination hosts that are excluded, so this is the time of arrival of the first packet used for measurement of the metric. 🚩
Note that timestamp format and resolution, sequence numbers, etc. will be established by the chosen test protocol standard or implementation. 🕰️
5. IP-Layer Capacity Singleton Metric Definitions 🚀
This section sets requirements for the Singleton metric that supports the Maximum IP-Layer Capacity Metric definitions in Section 6. It's the building block, bestie 🧱
5.1. Formal Name 🏷️
"Type-P-One-way-IP-Capacity" is the formal name, and it's informally called "IP-Layer Capacity".
Note that Type-P depends on the chosen method.
5.2. Parameters 📋
This section lists the REQUIRED input factors to specify the metric, beyond those listed in Section 4.
No additional Parameters are needed. Easy 😌
5.3. Metric Definitions 🧮
This section defines the REQUIRED aspects of the measurable IP-Layer Capacity Metric (unless otherwise indicated) for measurements between specified Source and Destination hosts:
Define the IP-Layer Capacity, C(T,dt,PM), to be the number of IP-Layer bits (including header and data fields) in packets that can be transmitted from the Src host and correctly received by the Dst host during one contiguous sub-interval, dt in length. The IP-Layer Capacity depends on the Src and Dst hosts, the host addresses, and the path between the hosts.
The number of these IP-Layer bits is designated n0[dtn,dtn+1] for a specific dt.
When the packet size is known and of fixed size, the packet count during a single sub-interval dt multiplied by the total bits in IP header and data fields is equal to n0[dtn,dtn+1].
Anticipating a Sample of Singletons, the number of sub-intervals with duration dt MUST be set to a natural number m, so that T+I = T + m*dt with dtn+1 - dtn = dt for 1 <= n <= m. 🔢
Parameter PM represents other performance metrics (see Section 5.4 below); their measurement results SHALL be collected during measurement of IP-Layer Capacity and associated with the corresponding dtn for further evaluation and reporting. Users SHALL specify the Parameter Tmax as required by each metric's reference definition. 📝
Mathematically, this definition is represented as (for each n):
( n0[dtn,dtn+1] )
C(T,dt,PM) = -------------------------
dt
Figure 1: Equation for IP-Layer Capacityand:
- n0 is the total number of IP-Layer header and payload bits that can be transmitted in standard-formed packets [RFC8468] from the Src host and correctly received by the Dst host during one contiguous sub-interval, dt in length, during the interval
[T,T+I]. 📦 - C(T,dt,PM), the IP-Layer Capacity, corresponds to the value of n0 measured in any sub-interval beginning at dtn, divided by the length of the sub-interval, dt. 💯
- PM represents other performance metrics (see Section 5.4 below); their measurement results SHALL be collected during measurement of IP-Layer Capacity and associated with the corresponding dtn for further evaluation and reporting. 📊
- All sub-intervals MUST be of equal duration. Choosing dt as non-overlapping consecutive time intervals allows for a simple implementation. 🧘
- The bit rate of the physical interface of the measurement devices MUST be higher than the smallest of the links on the path whose C(T,I,PM) is to be measured (the bottleneck link). This one's super important 🚧
Measurements according to this definition SHALL use the UDP transport layer. Standard-formed packets are specified in Section 5 of [RFC8468]. The measurement SHOULD use a randomized Source port or equivalent technique, and SHOULD send responses from the Source address matching the test packet Destination address. 🎲
Some effects of compression on measurement are discussed in Section 6 of [RFC8468]. 🗜️
5.4. Related Round-Trip Delay and One-Way Loss Definitions 🔄
RTD[dtn,dtn+1] is defined as a Sample of the Round-Trip Delay [RFC2681] between the Src host and the Dst host during the interval [T,T+I] (that contains equal non-overlapping intervals of dt). The "reasonable period of time" mentioned in [RFC2681] is the Parameter Tmax in this memo. The statistics used to summarize RTD[dtn,dtn+1] MAY include the minimum, maximum, median, mean, and the range = (maximum - minimum). Some of these statistics are needed for load adjustment purposes (Section 8.1), measurement qualification (Section 8.2), and reporting (Section 9). 🏓
OWL[dtn,dtn+1] is defined as a Sample of the One-Way Loss [RFC7680] between the Src host and the Dst host during the interval [T,T+I] (that contains equal non-overlapping intervals of dt). The statistics used to summarize OWL[dtn,dtn+1] MAY include the count of lost packets and the ratio of lost packets. 📉
Other metrics MAY be measured: one-way reordering, duplication, and delay variation. 🔀
5.5. Discussion 💬
See the corresponding section for Maximum IP-Layer Capacity (Section 6.5).
5.6. Reporting the Metric 📣
The IP-Layer Capacity SHOULD be reported with at least single-Megabit resolution, in units of Megabits per second (Mbps) (which, to avoid any confusion, is 1,000,000 bits per second). 💅
The related One-Way Loss metric and Round-Trip Delay measurements for the same Singleton SHALL be reported, also with meaningful resolution for the values measured. ✨
Individual Capacity measurements MAY be reported in a manner consistent with the Maximum IP-Layer Capacity; see Section 9. 🙌
6. Maximum IP-Layer Capacity Metric Definitions (Statistics) 👑
This section sets requirements for the following components to support the Maximum IP-Layer Capacity Metric. The big-picture stat, bestie 📈
6.1. Formal Name 🏷️
"Type-P-One-way-Max-IP-Capacity" is the formal name, and it's informally called "Maximum IP-Layer Capacity".
Note that Type-P depends on the chosen method.
6.2. Parameters 📋
This section lists the REQUIRED input factors to specify the metric, beyond those listed in Section 4.
No additional Parameters or definitions are needed. 🫶
6.3. Metric Definitions ✨
okay so this is where the Maximum IP-Layer Capacity metric gets its actual definition 💅. The following aspects are REQUIRED for the metric (unless otherwise indicated) when you're measuring between specified Source and Destination hosts 🎀:
So the Maximum IP-Layer Capacity, Maximum_C(T,I,PM), is the biggest number of IP-Layer bits n0[dtn,dtn+1] divided by dt that can be sent in packets from the Src host and correctly received by the Dst host. You look across all the dt-length intervals in [T,T+I], as long as they meet the PM criteria 🦋. Another way to say it: it's the max of a Sample of size m of Singletons C(T,I,PM) collected during [T,T+I] that also meet the PM criteria ✨.
The number of sub-intervals with duration dt MUST be set to a natural number m, so that T+I = T + m*dt with dtn+1 - dtn = dt for 1 <= n <= m 🍓.
Parameter PM stands for the other performance metrics (see Section 6.4 below) and their measurement results for the Maximum IP-Layer Capacity 📊. At least one target performance threshold (PM criterion) MUST be defined. If more than one metric and target performance threshold is defined, then the sub-interval with the maximum number of bits transmitted MUST meet all the target performance thresholds 👑. Users SHALL specify the Parameter Tmax, since each metric's reference definition asks for it 💜.
Mathematically, this looks like:
max ( n0[dtn,dtn+1] )
[T,T+I]
Maximum_C(T,I,PM) = -------------------------
dt
where:
T T+I
_________________________________________
| | | | | | | | | | |
dtn=1 2 3 4 5 6 7 8 9 10 n+1
n=m
Figure 2: Equation for Maximum Capacityand:
n0is the total number of IP-Layer header and payload bits that can be transmitted in standard-formed packets from the Src host and correctly received by the Dst host during one contiguous sub-interval, dt in length, during the interval[T,T+I].Maximum_C(T,I,PM), the Maximum IP-Layer Capacity, is the maximum value ofn0measured in any sub-interval beginning atdtn, divided by the constant length of all sub-intervals,dt🌟.PMrepresents the other performance metrics (see Section 6.4) and their measurement results for the Maximum IP-Layer Capacity. At least one target performance threshold (PM criterion) MUST be defined.- All sub-intervals MUST be of equal duration. Choosing
dtas non-overlapping consecutive time intervals allows for a simple implementation 🧁. - The bit rate of the physical interface of the measurement systems MUST be higher than the smallest of the links on the path whose
Maximum_C(T,I,PM)is to be measured (the bottleneck link) 🔥.
In this definition, the m sub-intervals are basically trials. The Src host varies the transmitted packet rate, searching for the maximum n0 that meets the PM criteria measured at the Dst host in a test of duration I 🔍. When the transmitted packet rate is held constant at the Src host, the m sub-intervals may also be viewed as trials to evaluate the stability of n0 and metric(s) in the PM list over all dt-length intervals in I 🦋.
Measurements according to these definitions SHALL use the UDP transport layer 🫶.
6.4. Related Round-Trip Delay and One-Way Loss Definitions 🌸
RTD[dtn,dtn+1] and OWL[dtn,dtn+1] are defined in Section 5.4. Here, the test intervals get stretched to match the capacity Samples, RTD[T,I] and OWL[T,I] 🎀.
The interval dtn,dtn+1 where Maximum_C(T,I,PM) occurs is the reporting sub-interval for RTD[dtn,dtn+1] and OWL[dtn,dtn+1] within RTD[T,I] and OWL[T,I] 💅.
Other metrics MAY be measured: one-way reordering, duplication, and delay variation ✨.
6.5. Discussion 🧚
If traffic conditioning (e.g., shaping, policing) applies along a path for which Maximum_C(T,I,PM) is to be determined, different values for dt SHOULD be picked, and measurements should run during multiple intervals [T,T+I] 🌊. Each duration dt SHOULD be chosen so that it is an integer multiple of increasing values k times serialization delay of a Path MTU (PMTU) at the physical interface speed where traffic conditioning is expected 📏. This should avoid taking configured burst tolerance Singletons as a valid Maximum_C(T,I,PM) result 🙈.
A Maximum_C(T,I,PM) without any indication of bottleneck congestion is likely an underestimate of Maximum_C(T,I,PM). That indication could be increased latency, packet loss, or Explicit Congestion Notification (ECN) marks during a measurement interval, I 🚦.
6.6. Reporting the Metric 💖
The IP-Layer Capacity SHOULD be reported with at least single-Megabit resolution, in units of Megabits per second (Mbps) (which, to avoid any confusion, is 1,000,000 bits per second) 🍓.
The related One-Way Loss metric and Round-Trip Delay measurements for the same Singleton SHALL be reported too, with meaningful resolution for the values measured 📈.
When there are demonstrated and repeatable Capacity modes in the Sample, the Maximum IP-Layer Capacity SHALL be reported for each mode 🎀. Report it along with the relative time from the beginning of the stream that the mode was observed to be present ⏱️. Bimodal Maximum IP-Layer Capacities have been observed with some services, sometimes called a "turbo mode" 🚀. That mode is meant to deliver short transfers more quickly, or to reduce the initial buffering time for some video streams 📺. Note that modes lasting less than duration dt will not be detected 👀.
Some transmission technologies have multiple methods of operation that may be activated when channel conditions degrade or improve, and these transmission methods may determine the Maximum IP-Layer Capacity 📡. Examples include line-of-sight microwave modulator constellations, or cellular modem technologies where the changes may be initiated by a user moving from one coverage area to another 📱. Operation in the different transmission methods may be observed over time. But the modes of Maximum IP-Layer Capacity will not be activated deterministically, the way the "turbo mode" described in the paragraph above is 🌈.
7. IP-Layer Sender Bit Rate Singleton Metric Definitions 🌈
This section sets requirements for the following components to support the IP-Layer Sender Bit Rate Metric 💌. This metric helps check that the Sender actually generated the desired rates during a test. Measurement takes place at the interface between the Src host and the network path (or as close as practical within the Src host) 🎯. It is not a metric for path performance, bestie 🙅♀️.
7.1. Formal Name
Type-P-IP-Sender-Bit-Rate is the formal name. It's informally called the "IP-Layer Sender Bit Rate" 🏷️.
Note that Type-P depends on the chosen method 🧩.
7.2. Parameters
This section lists the REQUIRED input factors to specify the metric, beyond those listed in Section 4 📝:
S: The duration of the measurement interval at the Source.st: The nominal duration of N sub-intervals in S (defaultst= 0.05 seconds).stn: The beginning boundary of a specific sub-interval, n, one of N sub-intervals in S.
S SHALL be longer than I, mainly to account for on-demand activation of the path, any preamble to testing required, and the delay of the path ⏳.
st SHOULD be much smaller than the sub-interval dt and on the same order as FT. Otherwise, the rate measurement will include many rate adjustments and more time smoothing, possibly smoothing the interval that contains the Maximum IP-Layer Capacity (and therefore losing relevance) 😵. The st Parameter does not matter when the Source is transmitting at a fixed rate throughout S 🌷.
7.3. Metric Definition
This section defines the REQUIRED aspects of the IP-Layer Sender Bit Rate Metric (unless otherwise indicated) for measurements at the specified Source on packets addressed for the intended Destination host and matching the required Type-P 🎀.
Define the IP-Layer Sender Bit Rate, B(S,st), as the number of IP-Layer bits (including header and data fields) transmitted from the Source with address pair Src and Dst during one contiguous sub-interval, st, during the test interval S (where S SHALL be longer than I). The fixed-size packet count during that single sub-interval st also gives the number of IP-Layer bits in any interval, [stn,stn+1] 📦.
Measurements according to this definition SHALL use the UDP transport layer. Any feedback from the Dst host to the Src host, received by the Src host during an interval [stn,stn+1], SHOULD NOT result in an adaptation of the Src host traffic conditioning during this interval. Rate adjustment happens on st interval boundaries only 🚫.
7.4. Discussion
Both the Sender and Receiver (or Source and Destination) bit rates SHOULD be assessed as part of an IP-Layer Capacity measurement 🔍. Otherwise, an unexpected sending rate limitation could produce an erroneous Maximum IP-Layer Capacity measurement 😱.
7.5. Reporting the Metric
The IP-Layer Sender Bit Rate SHALL be reported with meaningful resolution, in units of Megabits per second (which, to avoid any confusion, is 1,000,000 bits per second) 📊.
Individual IP-Layer Sender Bit Rate measurements are discussed further in Section 9 🧠.
8. Method of Measurement 💅
It is REQUIRED, per the architecture of the method, that two cooperating hosts operate in the roles of Src (test packet Sender) and Dst (Receiver), with a measured path and return path between them 🤝.
The duration of a test, Parameter I, MUST be constrained in a production network. This is an active test method, and it will likely cause congestion on the path from the Src host to the Dst host during a test 🚧.
8.1. Load Rate Adjustment Algorithm 🦋
The algorithm described in this section MUST NOT be used as a general Congestion Control Algorithm (CCA). As stated in Section 2 ("Scope, Goals, and Applicability"), the load rate adjustment algorithm's goal is to help determine the Maximum IP-Layer Capacity in the context of an infrequent, diagnostic, short-term measurement 🩺. There is a trade-off between test duration (also the test data volume) and algorithm aggressiveness (speed of ramp-up and ramp-down to the Maximum IP-Layer Capacity). The Parameter values chosen below strike a well-tested balance among these factors ⚖️.
A table SHALL be pre-built (by the test administrator) defining all the offered load rates that will be supported. These are R1 through Rn, in ascending order, corresponding to indexed rows in the table 📋. It is RECOMMENDED that rates begin with 0.5 Mbps at index zero, use 1 Mbps at index one, and then continue in 1 Mbps increments to 1 Gbps. Above 1 Gbps, and up to 10 Gbps, it is RECOMMENDED that 100 Mbps increments be used. Above 10 Gbps, increments of 1 Gbps are RECOMMENDED 🪜.
A higher initial IP-Layer Sender Bit Rate might be configured when the test operator is certain that the Maximum IP-Layer Capacity is well above the initial IP-Layer Sender Bit Rate, and factors such as test duration and total test traffic play an important role 🏃♀️. The sending rate table SHOULD bracket the Maximum Capacity where it will make measurements, including constrained rates less than 500 kbps if applicable 🌸.
Okay bestie, so each rate is basically a datagram size ss, sent in a burst of cc, every time interval tt passes 🦋 (the default for tt is 100 microsec, which is a likely system tick interval). Bigger datagrams are advantageous, so you want them as large as possible 💅, but it may be prudent to go a little smaller on the max so there's room for secondary protocol headers and/or tunneling without causing IP-Layer fragmentation 🌸. Picking a new rate is always a calculation on your current row, Rx:
Rx+1: the Sender uses the next-higher rate in the table ✨Rx-10: the Sender uses the rate 10 rows lower in the table 🎀
At the start of a test, the Sender begins sending at rate R1 🚀, and the Receiver starts a feedback timer of duration FT while it waits for inbound datagrams ⏰. As datagrams come in, the Receiver checks them for sequence number anomalies (loss, out-of-order, duplication, etc.) and measures the delay range (one-way or round-trip) 👀. All of that info piles up until the feedback timer FT expires, and then a status feedback message gets sent from the Receiver back to the Sender 💌. The Receiver then resets its stats for the next feedback interval. When the Sender gets those feedback messages, it evaluates them to figure out how to adjust the current offered load rate (Rx) 🧠.
If the feedback says there were no sequence number anomalies AND the delay range was below the lower threshold, the offered load rate goes up 📈. If congestion hasn't been confirmed yet (see below for how congestion gets declared), it jumps up by more than one rate setting (e.g., Rx+10) so the offered load can quickly hit a near-maximum rate 🚀. But if congestion has already been confirmed, it only goes up by one (Rx+1) 🐢. Also, if a rate threshold above a high sending rate (such as 1 Gbps) is exceeded, it only goes up by one (Rx+1) in any congestion state, no matter what 🫶.
If the feedback says sequence number anomalies were detected OR the delay range was above the upper threshold, the offered load rate goes down 📉. The RECOMMENDED threshold values are 10 for sequence number gaps, 30 msec for the lower delay threshold, and 90 msec for the upper delay threshold, respectively 🎯. And if congestion is confirmed for the very first time by the feedback message being processed right now, the offered load rate drops by more than one rate setting (e.g., Rx-30) 💥. That one-time reduction is there to make up for the fast initial ramp-up 🫠. In every other case, it only goes down by one (Rx-1) 🍓.
If the feedback says there were no sequence number anomalies AND the delay range was above the lower threshold but below the upper threshold, the offered load rate stays exactly where it is 🧘. This gives recent changes time to settle down, so the feedback can show current conditions more accurately 💜.
Lastly, congestion is inferred when there were sequence number anomalies AND/OR the delay range was above the upper threshold for three consecutive feedback intervals 🚨. This same algorithm is illustrated in Annex B of ITU-T Recommendation Y.1540, 2020 version [Y.1540], and it's implemented in Appendix A ("Load Rate Adjustment Pseudocode") in this memo 🧚.
The load rate adjustment algorithm MUST include timers that stop the test when the received packet streams unexpectedly cease 🛑. The timeout thresholds live in Table 1, along with the values for every other Parameter and variable in this section. Here's how the non-obvious Parameters work 👇:
- load packet timeout: The load packet timeout SHALL be reset to the configured value each time a load packet is received 🔄. If the timeout expires, the Receiver SHALL be closed and no further feedback sent 🔒.
- feedback message timeout: The feedback message timeout SHALL be reset to the configured value each time a feedback message is received 🔄. If the timeout expires, the Sender SHALL be closed and no further load packets sent 🔒.
Now the big table of everything, so you can see all the defaults, tested values, and safe ranges at a glance 🎀:
+=============+==========+===========+=========================+
| Parameter | Default | Tested | Expected Safe Range |
| | | Range or | (not entirely tested, |
| | | Values | other values NOT |
| | | | RECOMMENDED) |
+=============+==========+===========+=========================+
| FT, | 50 msec | 20 msec, | 20 msec <= FT <= 250 |
| feedback | | 50 msec, | msec; larger values may |
| time | | 100 msec | slow the rate increase |
| interval | | | and fail to find the |
| | | | max |
+-------------+----------+-----------+-------------------------+
| Feedback | L*FT, | L=100 | 0.5 sec <= L*FT <= 30 |
| message | L=20 (1 | with | sec; upper limit for |
| timeout | sec with | FT=50 | very unreliable test |
| (stop test) | FT=50 | msec (5 | paths only |
| | msec) | sec) | |
+-------------+----------+-----------+-------------------------+
| Load packet | 1 sec | 5 sec | 0.250-30 sec; upper |
| timeout | | | limit for very |
| (stop test) | | | unreliable test paths |
| | | | only |
+-------------+----------+-----------+-------------------------+
| Table index | 0.5 Mbps | 0.5 Mbps | When testing <= 10 Gbps |
| 0 | | | |
+-------------+----------+-----------+-------------------------+
| Table index | 1 Mbps | 1 Mbps | When testing <= 10 Gbps |
| 1 | | | |
+-------------+----------+-----------+-------------------------+
| Table index | 1 Mbps | 1 Mbps <= | Same as tested |
| (step) size | | rate <= 1 | |
| | | Gbps | |
+-------------+----------+-----------+-------------------------+
| Table index | 100 Mbps | 1 Gbps <= | Same as tested |
| (step) | | rate <= | |
| size, rate | | 10 Gbps | |
| > 1 Gbps | | | |
+-------------+----------+-----------+-------------------------+
| Table index | 1 Gbps | Untested | >10 Gbps |
| (step) | | | |
| size, rate | | | |
| > 10 Gbps | | | |
+-------------+----------+-----------+-------------------------+
| ss, UDP | None | <=1222 | Recommend max at |
| payload | | | largest value that |
| size, bytes | | | avoids fragmentation; |
| | | | using a payload size |
| | | | that is too small might |
| | | | result in unexpected |
| | | | Sender limitations |
+-------------+----------+-----------+-------------------------+
| cc, burst | None | 1 <= cc | Same as tested. Vary |
| count | | <= 100 | cc as needed to create |
| | | | the desired maximum |
| | | | sending rate. Sender |
| | | | buffer size may limit |
| | | | cc in the |
| | | | implementation |
+-------------+----------+-----------+-------------------------+
| tt, burst | 100 | 100 | Available range of |
| interval | microsec | microsec, | "tick" values (HZ |
| | | 1 msec | param) |
+-------------+----------+-----------+-------------------------+
| Low delay | 30 msec | 5 msec, | Same as tested |
| range | | 30 msec | |
| threshold | | | |
+-------------+----------+-----------+-------------------------+
| High delay | 90 msec | 10 msec, | Same as tested |
| range | | 90 msec | |
| threshold | | | |
+-------------+----------+-----------+-------------------------+
| Sequence | 10 | 0, 1, 5, | Same as tested |
| error | | 10, 100 | |
| threshold | | | |
+-------------+----------+-----------+-------------------------+
| Consecutive | 3 | 2, 3, 4, | Use values >1 to avoid |
| errored | | 5 | misinterpreting |
| status | | | transient loss |
| report | | | |
| threshold | | | |
+-------------+----------+-----------+-------------------------+
| Fast mode | 10 | 10 | 2 <= steps <= 30 |
| increase, | | | |
| in table | | | |
| index steps | | | |
+-------------+----------+-----------+-------------------------+
| Fast mode | 3 * Fast | 3 * Fast | Same as tested |
| decrease, | mode | mode | |
| in table | increase | increase | |
| index steps | | | |
+-------------+----------+-----------+-------------------------+Table 1: Parameters for Load Rate Adjustment Algorithm 🧁
As a consequence of the default parameterization, the Number of table steps in total for rates less than 10 Gbps is 1090 (excluding index 0) 🔢.
A related Sender backoff response to network conditions happens when one or more status feedback messages fail to arrive at the Sender 🥺. If no status feedback messages arrive at the Sender for longer than the Lost Status Backoff timeout, here's the math 🧮:
UDRT + (2+w)*FT = Lost Status Backoff timeoutwhere:
UDRT= upper delay range threshold (default 90 msec)FT= feedback time interval (default 50 msec)w= number of repeated timeouts (w=0 initially, w++ on each timeout, and reset to 0 when a message is received)
Beginning when the last message (of any type) was successfully received at the Sender:
The offered load SHALL then be decreased 💅 using the same process as when the feedback shows one or more sequence number anomalies OR the delay range was above the upper threshold (as described above), with the same load rate adjustment algorithm variables in their current state. So lost status feedback messages OR sequence errors OR delay variation can all lead to a rate reduction and congestion confirmation 🫶
The RECOMMENDED initial value for w is 0, which assumes a Round-Trip Time (RTT) of less than FT 🌸 A test with an RTT longer than FT is a valid reason to bump the initial value of w up appropriately. Variable w SHALL be incremented by one whenever the Lost Status Backoff timeout is exceeded ⏰ For example, with FT = 50 msec and UDRT = 90 msec, a status feedback message loss would be declared 190 msec after a successful message, then again 50 msec after that (240 msec total), and so on.
Also, if congestion is confirmed for the first time by a Lost Status Backoff timeout, the offered load rate drops by more than one rate setting (e.g., Rx-30) 🍓 This one-time drop makes up for the fast initial ramp-up. In every other case, the offered load rate only goes down by one (Rx-1).
Appendix B covers compliance with the applicable mandatory requirements of [RFC8085], consistent with the goals of the IP-Layer Capacity Metric and Method, including the load rate adjustment algorithm described in this section 🧚
8.2. Measurement Qualification or Verification ✨
It's of course necessary to calibrate the equipment doing the IP-Layer Capacity measurement 🔧 That way the expected capacity can be measured accurately, and the equipment choices (processing speed, interface bandwidth, etc.) are suitably matched to the measurement range.
When assessing a maximum rate as the metric specifies, artificially high (optimistic) values might get measured until some buffer on the path is filled 😭 Other causes include bursts of back-to-back packets with idle intervals delivered by a path, while the measurement interval (dt) is small and lined up with the bursts. Those artificial values might lead to an unsustainable Maximum Capacity observed when the Method of Measurement is searching for the maximum, and that would not do 🙅♀️ This situation is different from the bimodal service rates (discussed in "Reporting the Metric", Section 6.6), which are characterized by a multi-second duration (much longer than the measured RTT) and repeatable behavior.
The false-max issue is sneaky, so the Method of Measurement needs to be ready for it.
There are many ways the Method of Measurement could handle this false-max issue 💖 The default value for measurement of Singletons (dt = 1 second) has proven to be of practical value during tests of this method, allows the bimodal service rates to be characterized, and has an obvious alignment with the reporting units (Mbps) 📏
Another approach comes from Section 24 of [RFC2544] and its discussion of trial duration, where relatively short trials run as part of the search are followed by longer trials to make the final call 🎀 In the production network, measurements of Singletons and Samples (the terms for trials and tests of Lab Benchmarking) must be limited in duration because they may affect service. But there's still real value in repeating a Sample at a fixed sending rate determined by the previous search for the Maximum IP-Layer Capacity, to qualify the result in terms of the other performance metrics measured at the same time 🦋
A Qualification measurement for the search result is a subsequent measurement, sending at a fixed 99.x percent of the Maximum IP-Layer Capacity for I, or for an indefinite period. The same Maximum Capacity Metric is applied, and the Qualification for the result is a Sample without supra-threshold packet losses or a growing minimum delay trend in subsequent Singletons (or in each dt of the measurement interval, I). Samples showing supra-threshold packet losses or increasing queue occupation require a repeated search and/or a test at a reduced fixed Sender rate for Qualification 👑
Here, as with any Active Capacity test, the test duration must be kept short ⏱️ Ten-second tests for each direction of transmission are common today, and the default measurement interval specified here is I = 10 seconds. The combination of a fast, congestion-aware search method and user-network coordination makes a unique contribution to production testing 💅 The Maximum IP Capacity Metric and Method for assessing performance is very different from the classic Throughput Metric and Methods provided in [RFC2544]. It uses near-real-time load adjustments that are sensitive to loss and delay, similar to other congestion control algorithms used on the Internet every day, along with limited duration. Throughput measurements [RFC2544], on the other hand, can produce sustained overload conditions for extended periods of time 😬 Individual trials in a test governed by a binary search can last 60 seconds for each step, and the final confirmation trial may be even longer. That's very different from "normal" traffic levels, but overload conditions aren't a concern in the isolated test environment 🧪 The concerns raised in [RFC6815] were that the methods discussed in [RFC2544] would be let loose on production networks, so the authors challenged the standards community to develop Metrics and Methods like the ones described in this memo 🌈
8.3. Measurement Considerations ✨
In general, the widespread measurements this memo encourages will run into widespread behaviors 🌍 The bimodal IP Capacity behaviors already discussed in Section 6.6 are good examples.
In general, it is RECOMMENDED to locate test endpoints as close to the intended measured link(s) as practical (for reasons of scale, this isn't always possible, since there's a limit on the number of test endpoints coming from many perspectives, for example management and measurement traffic) 🧚 The testing operator MUST set a value for the MaxHops Parameter, based on the expected path length. This Parameter can keep measurement traffic from straying too far beyond the intended path 🛑
The measured path may be stateful based on many factors, and the Parameter "Time of day" when a test starts may not be enough information 🕰️ Repeatable testing may require knowing the time from the beginning of a measured flow, and how the flow is constructed (including how much traffic has already been sent on that flow when a state change is observed), because the state change may be based on time, bytes sent, or both. Both load packets and status feedback messages MUST contain sequence numbers, which helps with measurements based on those packets 🔢
Many different types of traffic shapers and on-demand communications access technologies may be encountered, as anticipated in [RFC7312], and they play a key role in measurement results 🎢 Methods MUST be prepared to provide a short preamble transmission to activate on-demand communications access and to discard the preamble from subsequent test results.
The following conditions might be encountered during measurement, where packet losses may occur independently of the measurement sending rate 👀
- Congestion of an interconnection or backbone interface may appear as packet losses distributed over time in the test stream, due to much-higher-rate interfaces in the backbone.
- Packet loss due to the use of Random Early Detection (RED) or other active queue management may or may not affect the measurement flow if competing background traffic (other flows) is simultaneously present.
- There may be only a small delay variation independent of the sending rate under these conditions as well.
- Persistent competing traffic on measurement paths that include shared transmission media may cause random packet losses in the test stream.
It is possible to mitigate these conditions using the flexibility of the load rate adjustment algorithm described in Section 8.1 above (by tuning specific Parameters) 🎛️
If the measurement flow burst duration happens to be on the order of, or smaller than, the burst size of a shaper or a policer in the path, then the line rate might be measured rather than the bandwidth limit imposed by the shaper or policer 🚧 If this condition is suspected, alternate configurations SHOULD be used.
In general, results depend on the sending stream's characteristics, and the measurement community has known this for a long time and needs to keep it foremost in mind 🧠 Although the default is a single flow (F=1) for testing, the use of multiple flows may be advantageous for the following reasons:
- The test hosts may be able to create a higher load than with a single flow, or parallel test hosts may be used to generate one flow each.
- Link aggregation may be present (flow-based load balancing), and multiple flows are needed to occupy each member of the aggregate.
- Internet access policies may limit the IP-Layer Capacity depending on the Type-P of the packets, possibly reserving capacity for various stream types.
Each flow would be controlled using its own implementation of the load rate adjustment (search) algorithm 🎯
It's obviously counterproductive to run more than one independent and concurrent test (regardless of the number of flows in the test stream) that tries to measure the maximum capacity on a single path 🙈 The number of concurrent, independent tests of a path SHALL be limited to one.
Tests of a v4-v6 transition mechanism might well be the intended subject of a capacity test. As long as both IPv4 packets and IPv6 packets sent/received are standard-formed, this should be allowed (and the change in header size is easily accounted for on a per-packet basis) 🦋
As testing continues, implementers should expect the methods to evolve 🌱 The ITU-T has published a supplement (Supplement 60) to the Y-series of ITU-T Recommendations, "Interpreting ITU-T Y.1540 maximum IP-layer capacity measurements" [Y.Sup60], which is the result of continued testing with the metric. Those results have improved the methods described here ⭐
9. Reporting Formats ✨
okay bestie, reporting time 📋 the Singleton IP-Layer Capacity results SHOULD come with the context they were measured under 🫶. Here's what to bring along:
- Timestamp (especially the time when the maximum was observed in dtn). ⏰
- Source and Destination (by IP or other meaningful ID). 📍
- Other inner Parameters of the test case (Section 4). 🧩
- Outer Parameters, such as "test conducted in motion" or other factors belonging to the context of the measurement. 🚗
- Result validity (indicating cases where the process was somehow interrupted or the attempt failed). 🥀
- A field where unusual circumstances could be documented, and another one for "ignore / mask out" purposes in further processing. 🙈
The Maximum IP-Layer Capacity results SHOULD go in a table 📊. One column SHOULD identify the test Phase, another SHOULD list how many flows were used in that Phase, and the rest SHOULD show the aggregate-of-all-flows results: the Maximum IP-Layer Capacity, the Loss Ratio, the RTT minimum, the RTT maximum, plus any other metrics with similar relevance 💅.
Like I said back in Section 6.6, if you get bimodal (or multi-modal) maxima, you SHALL report each mode separately 👑.
+========+==========+==================+========+=========+=========+
| Phase | Number | Maximum IP-Layer | Loss | RTT min | RTT |
| | of Flows | Capacity (Mbps) | Ratio | (msec) | max |
| | | | | | (msec) |
+========+==========+==================+========+=========+=========+
| Search | 1 | 967.31 | 0.0002 | 30 | 58 |
+--------+----------+------------------+--------+---------+---------+
| Verify | 1 | 966.00 | 0.0000 | 30 | 38 |
+--------+----------+------------------+--------+---------+---------+
Table 2: Maximum IP-Layer Capacity ResultsStatic and configuration Parameters 🎀:
The sub-interval time, dt, MUST tag along with any Maximum IP-Layer Capacity results, and so do the rest of the Parameters from Section 4 ("General Parameters and Definitions") 💖.
The PM list metrics for the sub-interval where the Maximum Capacity showed up MUST accompany the report, for each test Phase 🌸.
The IP-Layer Sender Bit Rate results SHOULD also go in a table 🦋. One column SHOULD identify the test Phase. Another SHOULD list each individual (numbered) flow used in that Phase, or the aggregate of flows in that Phase. A corresponding column SHOULD identify the specific sending rate sub-interval, stn, for each flow and aggregate. A final column SHOULD report the IP-Layer Sender Bit Rate results for each flow used, or for the aggregate of all flows.
+========+==========================+===========+=============+
| Phase | Flow Number or Aggregate | stn (sec) | Sender Bit |
| | | | Rate (Mbps) |
+========+==========================+===========+=============+
| Search | 1 | 0.00 | 345 |
+--------+--------------------------+-----------+-------------+
| Search | 2 | 0.00 | 289 |
+--------+--------------------------+-----------+-------------+
| Search | Agg | 0.00 | 634 |
+--------+--------------------------+-----------+-------------+
| Search | 1 | 0.05 | 499 |
+--------+--------------------------+-----------+-------------+
| Search | ... | 0.05 | ... |
+--------+--------------------------+-----------+-------------+
Table 3: IP-Layer Sender Bit Rate Results (Example with Two
Flows and st = 0.05 (sec))Static and configuration Parameters 🧚:
st, the sub-interval duration, MUST come with any Sender IP-Layer Bit Rate report. And the values of the remaining Parameters from Section 4 ("General Parameters and Definitions") MUST be reported too ✨.
9.1. Configuration and Reporting Data Formats
So this part is about the multi-Standards Development Organization (SDO) harmonization thing 🤓. The Broadband Forum (BBF) helped out by defining an information model and a data model for configuration and reporting. Those models line up with the metric Parameters and default values listed as lists in this memo. [TR-471] provides the information model that was used to prepare a full data model in related BBF work. BBF also looked at topics in its own lane, like where measurement systems sit in the Internet access architecture. For example, timestamp resolution requirements that shape the choice of test protocol are in Table 2 of [TR-471] ⏱️.
10. Security Considerations 🔐
Active Metrics and Active Measurements have a long history of security considerations 📜. The ones that apply to any Active Measurement of live paths are relevant here too. See [RFC4656] and [RFC5357].
When it comes to the privacy of everyone involved, whether they're doing the measuring or their traffic is being measured, the sensitive info available to potential observers is way reduced with the active techniques in this scope 👀. Passive observation of user traffic for measurement purposes raises a lot of privacy issues. Check out the privacy considerations in the Large-scale Measurement of Broadband Performance (LMAP) Framework [RFC7594], which covers both active and passive techniques.
There are some new considerations for Capacity measurement as described in this memo 🎀.
- Cooperating Source and Destination hosts, plus agreements to test the path between them, are REQUIRED 🤝. Hosts play either the Src role or the Dst role.
- It is REQUIRED to have a user client-initiated setup handshake between the cooperating hosts 🔥. That lets firewalls keep control over inbound unsolicited UDP traffic headed to either a control port (expected and with authentication) or ephemeral ports that only get created as needed. Both firewalls can keep doing their job normally 🧱.
- Client-server authentication and integrity protection for the feedback messages carrying measurements are RECOMMENDED 🔐.
- Hosts MUST limit how many simultaneous tests they run, so they don't exhaust resources or get wonky results 💀.
- Senders MUST be rate limited 🚦. You can do this with a pre-built table listing every offered load rate that will be supported (Section 8.1). The recommended load control search algorithm does a "ramp-up" starting from the lowest rate in the table 📈.
- Service subscribers with limited data volumes who do lots of capacity testing might get hit by Service Provider controls on their service 😭. Testing with the Service Provider's measurement hosts SHOULD be limited in frequency and/or overall volume of test traffic (for example, the range of duration values, I, SHOULD be limited).
The exact specification of these features is left for future protocol development 🔮.
11. IANA Considerations 🌈
Good news: this document has no IANA actions 🫶.
12. References 📚
12.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, <https://www.rfc-editor.org/info/rfc2119>.
[RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, "Framework for IP Performance Metrics", RFC 2330, DOI 10.17487/RFC2330, May 1998, <https://www.rfc-editor.org/info/rfc2330>.
[RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip Delay Metric for IPPM", RFC 2681, DOI 10.17487/RFC2681, September 1999, <https://www.rfc-editor.org/info/rfc2681>.
[RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. Zekauskas, "A One-way Active Measurement Protocol (OWAMP)", RFC 4656, DOI 10.17487/RFC4656, September 2006, <https://www.rfc-editor.org/info/rfc4656>.
[RFC4737] Morton, A., Ciavattone, L., Ramachandran, G., Shalunov, S., and J. Perser, "Packet Reordering Metrics", RFC 4737, DOI 10.17487/RFC4737, November 2006, <https://www.rfc-editor.org/info/rfc4737>.
[RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J. Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)", RFC 5357, DOI 10.17487/RFC5357, October 2008, <https://www.rfc-editor.org/info/rfc5357>.
[RFC6438] Carpenter, B. and S. Amante, "Using the IPv6 Flow Label for Equal Cost Multipath Routing and Link Aggregation in Tunnels", RFC 6438, DOI 10.17487/RFC6438, November 2011, <https://www.rfc-editor.org/info/rfc6438>.
[RFC7497] Morton, A., "Rate Measurement Test Protocol Problem Statement and Requirements", RFC 7497, DOI 10.17487/RFC7497, April 2015, <https://www.rfc-editor.org/info/rfc7497>.
[RFC7680] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, Ed., "A One-Way Loss Metric for IP Performance Metrics (IPPM)", STD 82, RFC 7680, DOI 10.17487/RFC7680, January 2016, <https://www.rfc-editor.org/info/rfc7680>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[RFC8468] Morton, A., Fabini, J., Elkins, N., Ackermann, M., and V. Hegde, "IPv4, IPv6, and IPv4-IPv6 Coexistence: Updates for the IP Performance Metrics (IPPM) Framework", RFC 8468, DOI 10.17487/RFC8468, November 2018, <https://www.rfc-editor.org/info/rfc8468>.
12.2. Informative References ✨
Okay bestie, here's the extra reading list 📚💖 These are informative, so you don't have to read them, but they're giving context and honestly the good stuff 🤓
[copycat]Edeline, K., Kühlewind, M., Trammell, B., and B. Donnet, "copycat: Testing Differential Treatment of New Transport Protocols in the Wild", ANRW '17, DOI 10.1145/3106328.3106330, July 2017, <https://irtf.org/anrw/2017/anrw17-final5.pdf>.[LS-SG12-A]"Liaison statement: LS - Harmonization of IP Capacity and Latency Parameters: Revision of Draft Rec. Y.1540 on IP packet transfer performance parameters and New Annex A with Lab Evaluation Plan", From ITU-T SG 12, March 2019, <https://datatracker.ietf.org/liaison/1632/>.[LS-SG12-B]"Liaison statement: LS on harmonization of IP Capacity and Latency Parameters: Consent of Draft Rec. Y.1540 on IP packet transfer performance parameters and New Annex A with Lab & Field Evaluation Plans", From ITU-T-SG-12, May 2019, <https://datatracker.ietf.org/liaison/1645/>.[RFC2544]Bradner, S. and J. McQuaid, "Benchmarking Methodology for Network Interconnect Devices", RFC 2544, DOI 10.17487/RFC2544, March 1999, <https://www.rfc-editor.org/info/rfc2544>.[RFC3148]Mathis, M. and M. Allman, "A Framework for Defining Empirical Bulk Transfer Capacity Metrics", RFC 3148, DOI 10.17487/RFC3148, July 2001, <https://www.rfc-editor.org/info/rfc3148>.[RFC5136]Chimento, P. and J. Ishac, "Defining Network Capacity", RFC 5136, DOI 10.17487/RFC5136, February 2008, <https://www.rfc-editor.org/info/rfc5136>.[RFC6815]Bradner, S., Dubray, K., McQuaid, J., and A. Morton, "Applicability Statement for RFC 2544: Use on Production Networks Considered Harmful", RFC 6815, DOI 10.17487/RFC6815, November 2012, <https://www.rfc-editor.org/info/rfc6815>.[RFC7312]Fabini, J. and A. Morton, "Advanced Stream and Sampling Framework for IP Performance Metrics (IPPM)", RFC 7312, DOI 10.17487/RFC7312, August 2014, <https://www.rfc-editor.org/info/rfc7312>.[RFC7594]Eardley, P., Morton, A., Bagnulo, M., Burbridge, T., Aitken, P., and A. Akhter, "A Framework for Large-Scale Measurement of Broadband Performance (LMAP)", RFC 7594, DOI 10.17487/RFC7594, September 2015, <https://www.rfc-editor.org/info/rfc7594>.[RFC7799]Morton, A., "Active and Passive Metrics and Methods (with Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, May 2016, <https://www.rfc-editor.org/info/rfc7799>.[RFC8085]Eggert, L., Fairhurst, G., and G. Shepherd, "UDP Usage Guidelines", BCP 145, RFC 8085, DOI 10.17487/RFC8085, March 2017, <https://www.rfc-editor.org/info/rfc8085>.[RFC8337]Mathis, M. and A. Morton, "Model-Based Metrics for Bulk Transport Capacity", RFC 8337, DOI 10.17487/RFC8337, March 2018, <https://www.rfc-editor.org/info/rfc8337>.[TR-471]Morton, A., "Maximum IP-Layer Capacity Metric, Related Metrics, and Measurements", Broadband Forum TR-471, July 2020, <https://www.broadband-forum.org/technical/download/TR-471.pdf>.[Y.1540]ITU-T, "Internet protocol data communication service - IP packet transfer and availability performance parameters", ITU-T Recommendation Y.1540, December 2019, <https://www.itu.int/rec/T-REC-Y.1540-201912-I/en>.[Y.Sup60]ITU-T, "Interpreting ITU-T Y.1540 maximum IP-layer capacity measurements", ITU-T Recommendation Y.Sup60, October 2021, <https://www.itu.int/rec/T-REC-Y.Sup60/en>.
Appendix A. Load Rate Adjustment Pseudocode 🧚
This appendix is the pseudocode version of the algorithm from Section 8.1, so you can see exactly how the pieces click together 🫶
Rx = 0 # The current sending rate (equivalent to a row
# of the table)
seqErr = 0 # Measured count that includes Loss or Reordering
# or Duplication impairments (all appear
# initially as errors in the packet sequence
# numbering)
seqErrThresh = 10 # Threshold on seqErr count that includes Loss or
# Reordering or Duplication impairments (all
# appear initially as errors in the packet
# sequence numbering)
delay = 0 # Measured Range of Round Trip Delay (RTD), msec
lowThresh = 30 # Low threshold on the Range of RTD, msec
upperThresh = 90 # Upper threshold on the Range of RTD, msec
hSpeedThresh = 1 # Threshold for transition between sending rate
# step sizes (such as 1 Mbps and 100 Mbps), Gbps
slowAdjCount = 0 # Measured Number of consecutive status reports
# indicating loss and/or delay variation above
# upperThresh
slowAdjThresh = 3 # Threshold on slowAdjCount used to infer
# congestion. Use values > 1 to avoid
# misinterpreting transient loss.
highSpeedDelta = 10 # The number of rows to move in a single
# adjustment when initially increasing offered
# load (to ramp up quickly)
maxLoadRates = 2000 # Maximum table index (rows)
if ( seqErr <= seqErrThresh && delay < lowThresh ) {
if ( Rx < hSpeedThresh && slowAdjCount < slowAdjThresh ) {
Rx += highSpeedDelta;
slowAdjCount = 0;
} else {
if ( Rx < maxLoadRates - 1 )
Rx++;
}
} else if ( seqErr > seqErrThresh || delay > upperThresh ) {
slowAdjCount++;
if ( Rx < hSpeedThresh && slowAdjCount == slowAdjThresh ) {
if ( Rx > highSpeedDelta * 3 )
Rx -= highSpeedDelta * 3;
else
Rx = 0;
} else {
if ( Rx > 0 )
Rx--;
}
}Appendix B. RFC 8085 UDP Guidelines Check 💅
Okay so, [RFC8085] is the BCP 145 UDP usage guidelines, and Section 3.1 of it is mostly about congestion control 🌊 It has two buckets of advice: the mandatory (MUST) ones and the recommendation (SHOULD) ones. Let's go through how this memo lines up with each one 👑
B.1. Assessment of Mandatory Requirements
Here's what the mandatory requirements in Section 3 of [RFC8085] look like:
| Internet paths can have widely varying characteristics, ...
| Consequently, applications that may be used on the Internet MUST
| NOT make assumptions about specific path characteristics. They
| MUST instead use mechanisms that let them operate safely under
| very different path conditions. Typically, this requires
| conservatively probing the current conditions of the Internet path
| they communicate over to establish a transmission behavior that it
| can sustain and that is reasonably fair to other traffic sharing
| the path.So the load rate adjustment algorithm in Section 8.1 is there to probe the network and get Maximum IP-Layer Capacity measurements while making as few assumptions about the measured path as possible, and while staying within the range of applications in Section 2 🔍 It's a bit of a balancing act, honestly. Being conservative when probing pulls against keeping the testing traffic small (especially for gigabit-rate measurements) and keeping the test short, and the test length is one thing that feeds into how fair the whole algorithm is 🦋
The text of Section 3 of [RFC8085] goes on to suggest alternatives to UDP that could meet the mandatory requirements, but none of them fit the scope and purpose of the Metrics and Methods in this memo 🙅♀️ Ad hoc TCP-based methods also don't reach the measurement accuracy that has been shown again and again in comparison measurements with the running code [LS-SG12-A] [LS-SG12-B] [Y.Sup60]. The UDP part of these methods is mainly there to support modern Internet transmission, where a transport protocol is required [copycat]. The metric itself is based on the IP Layer, and UDP makes it easy to correlate back to the IP Layer 🎀
Section 3.1.1 of [RFC8085] covers protocol timer guidelines:
| Latency samples MUST NOT be derived from ambiguous transactions.
| The canonical example is in a protocol that retransmits data, but
| subsequently cannot determine which copy is being acknowledged.Both load packets and status feedback messages MUST contain sequence numbers. That helps with measurements based on those packets, and it means no retransmissions are needed ✨
| When a latency estimate is used to arm a timer that provides loss
| detection -- with or without retransmission -- expiry of the timer
| MUST be interpreted as an indication of congestion in the network,
| causing the sending rate to be adapted to a safe conservative rate
| ...The methods in this memo use timers in two places. One is the Lost Status Backoff timeout, which lowers the sending rate when status feedback messages get lost. The other is the feedback message or load packet timeouts, which stop a test when connectivity is lost for a longer interval 🛑
This memo doesn't foresee any specific benefit from using Explicit Congestion Notification (ECN). Not me saying no to ECN, just the memo not needing it 🤷♀️
Section 3.2 of [RFC8085] covers message size guidelines:
| To determine an appropriate UDP payload size, applications MUST
| subtract the size of the IP header (which includes any IPv4
| optional headers or IPv6 extension headers) as well as the length
| of the UDP header (8 bytes) from the PMTU size.The method uses a sending rate table with a maximum UDP payload size that expects significant header overhead and avoids fragmentation 📦
Section 3.3 of [RFC8085] covers reliability guidelines:
| Applications that do require reliable message delivery MUST
| implement an appropriate mechanism themselves.The IP-Layer Capacity Metrics and Methods don't need reliable delivery 🙌
| Applications that require ordered delivery MUST reestablish
| datagram ordering themselves.The IP-Layer Capacity Metrics and Methods don't need to rebuild packet order either. Instead, it's better to measure packet reordering if it happens [RFC4737] 📊
B.2. Assessment of Recommendations
The load rate adjustment algorithm's goal is to find the Maximum IP-Layer Capacity during an infrequent, diagnostic, short-term measurement. That makes it a global exception to a lot of the SHOULD-level requirements listed in [RFC8085], which are mostly written for long-lived flows that have to share the network with other traffic in a more or less fair way 🤝 Even so, the algorithm (as specified in Section 8.1 and Appendix A above) responds to signs of congestion in clearly defined ways 💖
Here's one specific recommendation that's given as an example. Section 3.1.5 of [RFC8085] talks about how RTT and loss measurements affect congestion control, and it says:
| A congestion control [algorithm] designed for UDP SHOULD respond
| as quickly as possible when it experiences congestion, and it
| SHOULD take into account both the loss rate and the response time
| when choosing a new rate.The load rate adjustment algorithm responds to loss and RTT measurements with a clear and concise rate reduction when warranted, and the response makes use of direct measurements (which are more exact than anything you can infer from TCP ACKs) 💅✨
Section 3.1.5 of [RFC8085] goes on to specify the following: 🫶
The implemented congestion control scheme SHOULD result in bandwidth (capacity) use that is comparable to that of TCP within an order of magnitude, so that it does not starve other flows sharing a common bottleneck.
This is a requirement for coexistent streams, and not for diagnostic and infrequent measurements using short durations 🦋 The rate oscillations during short tests allow other packets to pass and don't starve other flows, so it's giving good manners 🌸
Ironically, ad hoc TCP-based measurements of "Internet Speed" are also designed to work around this SHOULD-level requirement, by launching many flows (9, for example) to increase the outstanding data dedicated to testing 🙈 Lowkey iconic, but also kind of a workaround 💀
The load rate adjustment algorithm cannot become a TCP-like congestion control, or it will have the same weaknesses of TCP when trying to make a Maximum IP-Layer Capacity measurement and will not achieve the goal 🔥 The results of the referenced testing [LS-SG12-A] [LS-SG12-B] [Y.Sup60] supported this statement hundreds of times, with comparisons to multi-connection TCP-based measurements. Periodt 👑
A brief review of requirements from [RFC8085] follows (marked "Yes" when this memo is compliant, or "NA" (Not Applicable)) 🎀
+======+============================================+=========+
| Yes? | Recommendation in RFC 8085 | Section |
+======+============================================+=========+
| Yes | MUST tolerate a wide range of Internet | 3 |
| | path conditions | |
+------+--------------------------------------------+---------+
| NA | SHOULD use a full-featured transport | |
| | (e.g., TCP) | |
+------+--------------------------------------------+---------+
+------+--------------------------------------------+---------+
| Yes | SHOULD control rate of transmission | 3.1 |
+------+--------------------------------------------+---------+
| NA | SHOULD perform congestion control over all | |
| | traffic | |
+------+--------------------------------------------+---------+
+======+============================================+=========+
| | For bulk transfers, | 3.1.2 |
+======+============================================+=========+
| NA | SHOULD consider implementing TFRC | |
+------+--------------------------------------------+---------+
| NA | else, SHOULD in other ways use bandwidth | |
| | similar to TCP | |
+------+--------------------------------------------+---------+
+======+============================================+=========+
| | For non-bulk transfers, | 3.1.3 |
+======+============================================+=========+
| NA | SHOULD measure RTT and transmit max. 1 | 3.1.1 |
| | datagram/RTT | |
+------+--------------------------------------------+---------+
| NA | else, SHOULD send at most 1 datagram every | |
| | 3 seconds | |
+------+--------------------------------------------+---------+
| NA | SHOULD back-off retransmission timers | |
| | following loss | |
+------+--------------------------------------------+---------+
+------+--------------------------------------------+---------+
| Yes | SHOULD provide mechanisms to regulate the | 3.1.6 |
| | bursts of transmission | |
+------+--------------------------------------------+---------+
+------+--------------------------------------------+---------+
| NA | MAY implement ECN; a specific set of | 3.1.7 |
| | application mechanisms are REQUIRED if ECN | |
| | is used | |
+------+--------------------------------------------+---------+
+------+--------------------------------------------+---------+
| Yes | For DiffServ, SHOULD NOT rely on | 3.1.8 |
| | implementation of PHBs | |
+------+--------------------------------------------+---------+
+------+--------------------------------------------+---------+
| Yes | For QoS-enabled paths, MAY choose not to | 3.1.9 |
| | use CC | |
+------+--------------------------------------------+---------+
+------+--------------------------------------------+---------+
| Yes | SHOULD NOT rely solely on QoS for their | 3.1.10 |
| | capacity | |
+------+--------------------------------------------+---------+
| NA | non-CC controlled flows SHOULD implement a | |
| | transport circuit breaker | |
+------+--------------------------------------------+---------+
| Yes | MAY implement a circuit breaker for other | |
| | applications | |
+------+--------------------------------------------+---------+
+======+============================================+=========+
| | For tunnels carrying IP traffic, | 3.1.11 |
+======+============================================+=========+
| NA | SHOULD NOT perform congestion control | |
+------+--------------------------------------------+---------+
| NA | MUST correctly process the IP ECN field | |
+------+--------------------------------------------+---------+
+======+============================================+=========+
| | For non-IP tunnels or rate not determined | 3.1.11 |
| | by traffic, | |
+======+============================================+=========+
| NA | SHOULD perform CC or use circuit breaker | |
+------+--------------------------------------------+---------+
| NA | SHOULD restrict types of traffic | |
| | transported by the tunnel | |
+------+--------------------------------------------+---------+
+------+--------------------------------------------+---------+
| Yes | SHOULD NOT send datagrams that exceed the | 3.2 |
| | PMTU, i.e., | |
+------+--------------------------------------------+---------+
| Yes | SHOULD discover PMTU or send datagrams < | |
| | minimum PMTU | |
+------+--------------------------------------------+---------+
| NA | Specific application mechanisms are | |
| | REQUIRED if PLPMTUD is used | |
+------+--------------------------------------------+---------+
+------+--------------------------------------------+---------+
| Yes | SHOULD handle datagram loss, duplication, | 3.3 |
| | reordering | |
+------+--------------------------------------------+---------+
| NA | SHOULD be robust to delivery delays up to | |
| | 2 minutes | |
+------+--------------------------------------------+---------+
+------+--------------------------------------------+---------+
| Yes | SHOULD enable IPv4 UDP checksum | 3.4 |
+------+--------------------------------------------+---------+
| Yes | SHOULD enable IPv6 UDP checksum; specific | 3.4.1 |
| | application mechanisms are REQUIRED if a | |
| | zero IPv6 UDP checksum is used | |
+------+--------------------------------------------+---------+
+------+--------------------------------------------+---------+
| NA | SHOULD provide protection from off-path | 5.1 |
| | attacks | |
+------+--------------------------------------------+---------+
| | else, MAY use UDP-Lite with suitable | 3.4.2 |
| | checksum coverage | |
+------+--------------------------------------------+---------+
+------+--------------------------------------------+---------+
| NA | SHOULD NOT always send middlebox keep- | 3.5 |
| | alive messages | |
+------+--------------------------------------------+---------+
| NA | MAY use keep-alives when needed (min. | |
| | interval 15 sec) | |
+------+--------------------------------------------+---------+
+------+--------------------------------------------+---------+
| Yes | Applications specified for use in limited | 3.6 |
| | use (or controlled environments) SHOULD | |
| | identify equivalent mechanisms and | |
| | describe their use case | |
+------+--------------------------------------------+---------+
+------+--------------------------------------------+---------+
| NA | Bulk-multicast apps SHOULD implement | 4.1.1 |
| | congestion control | |
+------+--------------------------------------------+---------+
+------+--------------------------------------------+---------+
| NA | Low volume multicast apps SHOULD implement | 4.1.2 |
| | congestion control | |
+------+--------------------------------------------+---------+
+------+--------------------------------------------+---------+
| NA | Multicast apps SHOULD use a safe PMTU | 4.2 |
+------+--------------------------------------------+---------+
+------+--------------------------------------------+---------+
| Yes | SHOULD avoid using multiple ports | 5.1.2 |
+------+--------------------------------------------+---------+
| Yes | MUST check received IP source address | |
+------+--------------------------------------------+---------+
+------+--------------------------------------------+---------+
| NA | SHOULD validate payload in ICMP messages | 5.2 |
+------+--------------------------------------------+---------+
+------+--------------------------------------------+---------+
| Yes | SHOULD use a randomized Source port or | 6 |
| | equivalent technique, and, for client/ | |
| | server applications, SHOULD send responses | |
| | from source address matching request | |
+------+--------------------------------------------+---------+
| NA | SHOULD use standard IETF security | 6 |
| | protocols when needed | |
+------+--------------------------------------------+---------+Table 4: Summary of Key Guidelines from RFC 8085
Acknowledgments ✨
Huge thanks to Joachim Fabini, Matt Mathis, J. Ignacio Alvarez-Hamelin, Wolfgang Balzer, Frank Brockners, Greg Mirsky, Martin Duke, Murray Kucherawy, and Benjamin Kaduk for their extensive comments on this memo and related topics 💖 They really went deep on this one, bestie. In a second round of reviews, Magnus Westerlund, Lars Eggert, and Zaheduzzaman Sarker also showed up and helped out 🫶
Authors' Addresses 🎀
The authors are Al Morton, Rüdiger Geib, and Len Ciavattone, and here's how you can reach each of them 🦋
Al Morton
AT&T Labs
200 Laurel Avenue South
Middletown, NJ 07748
United States of America
Phone: +1 732 420 1571
Email: acm@research.att.comRüdiger Geib
Deutsche Telekom
Heinrich Hertz Str. 3-7
64295 Darmstadt
Germany
Phone: +49 6151 5812747
Email: Ruediger.Geib@telekom.deLen Ciavattone
AT&T Labs
200 Laurel Avenue South
Middletown, NJ 07748
United States of America
Phone: +1 732 420 1239
Email: lencia@att.com📦 where this page comes from
- ietf: rfc9097 2021-11
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms