RFC 7497

Rate Measurement Test Protocol Problem Statement and Requirements

📅 published April 2015, from ietf

Rate Measurement Test Protocol Problem Statement and Requirements 💖

okay bestie, this is RFC 7497 🎀 and the title block is giving main character energy. It was written by A. Morton from AT&T Labs 💅 It's Category: Informational 📚 (so it's not a Standards Track spec, just vibes and info), dated April 2015, with ISSN 2070-1721. The title block doesn't list anything it obsoletes or updates, so there's no drama there 🫶

Abstract ✨

okay so this memo is a problem statement for measuring access rate 📊 for test protocols that measure IP Performance Metrics (IPPM) 🦋 The big things it cares about are being able to control packet characteristics on the path being tested 🎯 like asymmetric rate and asymmetric packet size. literally the whole point 💜

Status of This Memo 🌸

this document is not an Internet Standards Track specification, it's published just for informational purposes 📖

it's a product of the Internet Engineering Task Force (IETF) 🧚 and it represents the consensus of the IETF community. it got public review and was approved for publication by the Internet Engineering Steering Group (IESG). not every IESG-approved doc is a candidate for any level of Internet Standard, so check Section 2 of RFC 5741 if you want the details 🙈

for info about the current status, errata, and how to give feedback, go to http://www.rfc-editor.org/info/rfc7497 🍓

Copyright (c) 2015 IETF Trust and the persons identified as the document authors. All rights reserved. 💜

this document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as in effect on the date of publication. please read these carefully, because they describe your rights and restrictions for this document 🫶 Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they come without warranty as described in the Simplified BSD License ⭐

Table of Contents 🧚

1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
  1.1.  Requirements Language . . . . . . . . . . . . . . . . . .   3
2.  Purpose and Scope . . . . . . . . . . . . . . . . . . . . . .   3
3.  Active Rate Measurement . . . . . . . . . . . . . . . . . . .   6
4.  Measurement Method Categories . . . . . . . . . . . . . . . .   7
5.  Test Protocol Control and Generation Requirements . . . . . .   9
6.  Security Considerations . . . . . . . . . . . . . . . . . . .  11
7.  Operational Considerations  . . . . . . . . . . . . . . . . .  11
8.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  12
  8.1.  Normative References  . . . . . . . . . . . . . . . . . .  12
  8.2.  Informative References  . . . . . . . . . . . . . . . . .  13
Acknowledgements  . . . . . . . . . . . . . . . . . . . . . . . .  13
Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  14

1. Introduction 🦋

there are so many possible rate measurement scenarios 🌸 this memo describes one of them and presents a problem statement for test protocols that measure IP Performance Metrics (IPPM) ✨

when someone picks an Internet access service, they care about how the options perform 👀 standardized measurements can be the basis for comparing them. so there's a real need to coordinate the measurements that support those comparisons, and to test the control protocols that make it happen 💅 here's a picture of some typical measurement points in access networks:

User     /====== Fiber =======  Access Node \
Device -|------ Copper -------  Access Node -|-- Infrastructure -- GW
or Host  \------ Radio -------  Access Node /

                             GW = Gateway

the access rate scenario has gotten widespread attention from Internet-access subscribers and basically every player in the Internet industry, regulators included 😭 people are tackling it with lots of different measurement methods. the eventual protocol solutions (and the systems that use them) may not directly involve users, like when tests go from the infrastructure to a service-specific device such as a residential gateway 🏠 but nothing about the problem stops users from building a test protocol controlled through command line interfaces on both ends 💖 so the possible outcomes are a very wide range of test protocols, active measurement methods, and system solutions 🌈

1.1. Requirements Language 🎀

the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119] 🫶

2. Purpose and Scope 💜

the scope and purpose of this memo is to define the measurement problem statement for test protocols doing access rate measurement on production networks 🏭 relevant test protocols include [RFC4656] and [RFC5357], but the problem is stated generally so that any existing test protocol can address it, such as [RFC6812] ✨

this memo talks about possible measurement methods but does not specify the exact methods that would normally be part of the solution 🙈

we're interested in access measurement scenarios with these characteristics 👑

  • The access portion of the network is the focus of this problem statement. The user typically subscribes to a service with bidirectional access partly described by rates in bits per second. The rates may be expressed as raw capacity or restricted capacity, as described in [RFC6703]. These are the quantities that must be measured according to one or more standard metrics, and for which measurement methods must also be agreed on as a part of the solution.
  • Referring to the reference path illustrated below and defined in [RFC7398], possible measurement points include a subscriber's host, the access service demarcation point, intra IP access (where a globally routable address is present), or the gateway between the measured access network and other networks.
Subsc. -- Private -- Private -- Access -- Intra IP -- GRA -- Transit
device     Net #1     Net #2    Demarc.    Access     GW     GRA GW

             GRA = Globally Routable Address, GW = Gateway
  • Rates at some links near the edge of the provider's network can often be several orders of magnitude less than link rates in the aggregation and core portions of the network. 📉
  • Asymmetrical access rates on ingress and egress are prevalent. ⚖️
  • In many scenarios of interest, services of extremely large-scale access require low-complexity devices participating at the user end of the path, and those devices place limits on clock and control timing accuracy. 🤏

this problem statement assumes that the most likely bottleneck device or link is adjacent to the remote (user-end) measurement device, or is within one or two router/switch hops of it 🎯

other use cases for rate measurement involve situations where the packet switching and transport facilities are leased by one operator from another, and the link capacity available can't be directly determined (for example, from device-interface utilization) 🤔 these could include mobile backhaul, Ethernet service access networks, and/or extensions of layer 2 or layer 3 networks 🌐 the rate measurement results in these cases could be used to select alternate routing, to check whether capacity meets some previous agreement, and/or to adapt the rate of traffic sources if a capacity bottleneck is found via the rate measurement 🔥 for aggregated leased networks, available capacity may also be asymmetric. in these cases the tester is assumed to have a sender and receiver location under their control. we'll call this scenario the aggregated leased-network case below 🧚

this memo describes protocol support for active measurement methods that fit the IPPM working group's traditional charter 💅 active measurements need synthetic traffic streams dedicated to testing, and they don't measure user traffic. see Section 2 of [RFC2679], where the concept of a stream was first introduced in IPPM literature as the basis for collecting a sample (defined in Section 11 of [RFC2330]) 📖

as [RFC2330] notes, the focus of access traffic management may affect the rate measurement results for some forms of access, because user traffic and test traffic can differ if the test traffic has different characteristics, mainly in terms of the packets themselves. see Section 13 of [RFC2330] for the considerations on packet type, or Type-P 🍓

there are several aspects of Type-P where user traffic may be examined and picked out for special treatment, and that may affect transmission rates 👀 various aspects of Type-P are known to influence Equal-Cost Multipath (ECMP) routing, with possible rate measurement variability across parallel paths. without being exhaustive, the possibilities include:

  • Packet length
  • IP addresses
  • Transport protocol (e.g., where TCP packets may be routed differently from UDP)
  • Transport-protocol port numbers

this issue needs more discussion once specific solutions or methods of measurement are proposed. for this problem statement, it's enough to identify the problem and note that the solution may require an extremely close emulation of user traffic, in terms of one or more of the factors above 😭🫶

even though the user may have multiple instances of network access available, the primary problem scope is to measure one form of access at a time 💖 a solution for the single access problem may be applicable to simultaneous measurement of multiple access instances, but treating that scenario is beyond the current scope of this document 🌸

a key consideration is whether active measurements will be conducted with user traffic present. In-Service testing takes place with user traffic present. Out-of-Service testing occurs during pre-service assessment or during maintenance that interrupts service temporarily. Out-of-Service testing includes activities described as "service commissioning", "service activation", and "planned maintenance". Opportunistic In-Service testing (when there's no user traffic present throughout the test interval, such as outside normal business hours) is essentially equivalent to Out-of-Service testing. Both In-Service and Out-of-Service testing are within the scope of this problem 🔥

it is a non-goal to solve the measurement protocol specification problem in this memo 🙅‍♀️

it is a non-goal to standardize methods of measurement in this memo. however, the problem statement mandates support for one category of rate measurement methods in the test protocol, and adequate control features for those methods in the control protocol (assuming the control and test protocols are separate) 💜✨

3. Active Rate Measurement ✨

okay so this section is all about the active measurement methods you'd need to measure access rates in production networks 🌐🦋

The source and destination devices talk to each other through control messages and other basic capabilities. Those come from the methods in the IPPM RFCs [RFC2679] [RFC2680], and they're assumed for test protocols like [RFC5357] and [RFC4656]. We just take them as given here 💅

Most forms of active testing intrude on user performance to some degree, especially In-Service testing 😭 One big tenet of IPPM methods is to minimize how much test traffic messes with user traffic in the production network. Section 5 of [RFC2680] lists the problems you get with high measurement traffic rates (aka "too much" traffic). For rate measurement, the biggest ones are that measurement traffic can skew the results, and that it can cause congestion on the access link. Section 4 of [RFC3148] has even more considerations if you want to go deeper 🧚 The user of protocols for In-Service testing MUST respect these traffic constraints. Obviously, rate measurement methods that use less active test traffic than the others (with similar accuracy) are the ones to pick for In-Service testing, and this memo encourages traffic reduction through asymmetric control capabilities 🫶

Out-of-Service tests where the test path shares no links with In-Service user traffic don't have any of the congestion or skew worries 🌸 Both types should still handle the practical stuff every test effort deals with, like running measurements within a reasonable time from the tester's point of view, and making sure timestamp accuracy matches the precision the measurement needs [RFC2330]. Out-of-Service tests where some part of the test path is shared with In-Service traffic MUST respect the In-Service constraints described above. No skipping that part, bestie 🚫

The metrics you want to measure have a huge influence on which categories of measurement methods you need. For example, using the terminology of [RFC5136], it may be possible to measure a path capacity metric while In-Service, as long as the level of background (user) traffic can be assessed and included in the reported result 👑

The measurement architecture MAY be either one-way (e.g., [RFC4656]) or two-way (e.g., [RFC5357]). But the scale and complexity of end-user or aggregated access measurement clearly favor two-way. That's because you can have a low-complexity user-end device and round-trip results collection, as found in [RFC5357] ✨ Still, many access services have asymmetric rates, so the measurement system MUST be able to evaluate performance in each direction of transmission. In the two-way architecture, both end devices MUST include the ability to launch test streams and collect the results of measurements in both (one-way) directions of transmission. This requirement is consistent with earlier protocol specifications, so it's not a unique problem for rate measurements 🍓

The following paragraphs describe the features for the roles of test packet SENDER, RECEIVER, and results REPORTER 💜

SENDER:

Generate streams of test packets with various characteristics as desired (see Section 4). The SENDER MAY be located at the user end of the access path, or somewhere else in the production network, like at one end of an aggregated leased-network segment 🎀

RECEIVER:

Collect streams of test packets with various characteristics (as described above), and make the measurements needed to support rate measurement at the receiving end of an access or aggregated leased-network segment 🦋

REPORTER:

Use information from test packets and local processes to measure delivered packet rates and prepare results in the required format. The REPORTER role may be combined with another role, most likely the SENDER 🌈

4. Measurement Method Categories 🧚

A protocol that addresses the rate measurement problem MUST serve the test stream generation and measurement functions (SENDER and RECEIVER). The follow-up phase of analyzing the measurement results to produce a report is outside the scope of this problem and memo (REPORTER). So the REPORTER part is not your problem here, babe 💅

For the purposes of this problem statement, the many possibilities for rate measurement stream generation get sorted into these categories:

  1. Packet pairs, with fixed intra-pair packet spacing and fixed or random time intervals between pairs in a test stream.
  2. Multiple streams of packet pairs, with a range of intra-pair spacing and inter-pair intervals.
  3. One or more packet ensembles in a test stream, using a fixed ensemble size in packets and one or more fixed intra-ensemble packet spacings. Zero spacing counts too, which means back-to-back burst ensembles and constant rate ensembles both fall in this category.
  4. One or more packet chirps (a set of packets with specified characteristics), where inter-packet spacing typically decreases between adjacent packets in the same chirp, and each pair of packets represents a rate for testing purposes.

The test protocol SHALL support test packet ensemble generation (category 3), since this seems to minimize the demands on measurement accuracy 🔥 Other stream generation categories are OPTIONAL.

For all supported categories, the protocol(s) MUST be able to specify, control, and generate the following extra variables:

  • a. variable payload lengths among packet streams;
  • b. variable length (in packets) among packet streams or ensembles;
  • c. variable IP header markings among packet streams;
  • d. choice of UDP transport and variable port numbers, or choice of TCP transport and variable port numbers for two-way architectures only, or both (see below for additional requirements on TCP transport generation); and
  • e. variable number of packet pairs, ensembles, or streams used in a test session.

Being able to revise these variables during an established test session is OPTIONAL, since multiple test sessions could serve the same purpose 🌸 Another OPTIONAL feature is generating streams with VLAN tags and other markings.

For measurement systems that use TCP as the transport protocol, generating specific stream characteristics requires a sender that can establish and prime the connection so the desired stream characteristics are allowed. See [IPPM-METRICS] for more background 📚

Beyond a simple connection handshake and the options establishment, an "open-loop" TCP sender requires the SENDER ability to:

  • generate TCP packets with well-formed headers (all fields valid), including Acknowledgement aspects;
  • produce packet streams at controlled rates and variable inter-packet spacings, including packet ensembles (back-to-back at server rate); and
  • continue the configured sending stream characteristics despite all control indications except receive-window exhaust.

The corresponding TCP RECEIVER performs normally, and has some ability to configure the receive window big enough to let the SENDER transmit at will (up to a configured target) 🫧

It may also be useful, for diagnostic purposes, to provide a control for the bulk transfer capacity measurement with fully-specified (and congestion-controlled) TCP senders and receivers, as envisioned in [RFC3148]. But this would be a brute-force assessment, which doesn't follow the conservative tenets of IPPM measurement [RFC2330]. Brute force is not it, bestie 😬

Measurements for each UDP test packet transferred between SENDER and RECEIVER MUST be compliant with the singleton measurement methods described in IPPM RFCs [RFC2679][RFC2680]. The timestamp information or loss/arrival status for each packet MUST be available for communication to the REPORTER function 💖

5. Test Protocol Control and Generation Requirements 👑

In summary, the test protocol must support the measurement features described in the sections above. That requires:

  1. Communicating all test variables to the SENDER and RECEIVER;
  2. Results collection in a one-way architecture;
  3. Remote device control for both one-way and two-way architectures; and
  4. Asymmetric packet rates in a two-way measurement architecture, or coordinated one-way test capabilities with the same effect. Asymmetric rates may be achieved through directional control of packet rate or packet size.

The ability to control and generate asymmetric rates in a two-way architecture is REQUIRED 🔥 Two-way architectures are RECOMMENDED to include control and generation capability for both asymmetric and symmetric packet sizes, because packet size often matters in the scope of this problem. Test systems SHOULD be equipped to detect directional size dependency through comparative measurements 🧠

Asymmetric packet size control is indicated when the result of a measurement may depend on the size of the packets used in each direction. That's i.e., when any of the following conditions hold:

  • there is a link in the path with asymmetrical capacity in opposite directions (in combination with one or more of the conditions below, though their presence or specific details may be unknown to the tester);
  • there is a link in the path that aggregates (or divides) packets into link-level frames, and may have a capacity that depends on packet size, rate, or timing;
  • there is a link in the path where transmission in one direction influences performance in the opposite direction;
  • there is a device in the path where transmission capacity depends on packet header processing capacity (in other words, the capacity is sensitive to packet size);
  • the target application stream is nominally MTU size packets in one direction versus an ACK stream in the other (noting that there are a vanishing number of symmetrical rate application streams for which rate measurement is wanted or interesting, but such streams might have some relevance at this time);
  • the distribution of packet losses is critical to rate assessment;

and possibly other circumstances revealed by measurements comparing streams with symmetrical size and asymmetrical size.

Implementations may support control and generation for only symmetric packet sizes when none of the above conditions hold 🎀

The test protocol SHOULD enable measurement of the capacity metric [RFC5136] either Out-of-Service, In-Service, or both. Other metrics [RFC5136] are OPTIONAL ✨

6. Security Considerations 🔐

Okay, this part is serious, so read it carefully 🙈

The security considerations that apply to any active measurement of live networks are relevant here as well. See [RFC4656] and [RFC5357].

Privacy considerations for measurement systems, particularly when Internet users take part in the tests in some way, are described in [LMAP-FRAMEWORK].

There may be a serious issue if a proprietary service level agreement involved with the access network segment provider were somehow leaked in the process of rate measurement. To address this, test protocols SHOULD NOT convey this information in a way that could be discovered by unauthorized parties. Keep that SLA locked down, fr fr 🔒💀

7. Operational Considerations 🌸

okay bestie, testing literally generates traffic 🙈. Either the test is talking to the system to set itself up and collect results, or it's sending its own dedicated measurement packets, or it does both. Most of that traffic falls into one of two buckets: subscriber traffic or network management traffic 🎀. Networks still need to be engineered so every kind of traffic gets served properly, and publishing this memo doesn't change that one bit 💅.

Subscribers and authorized users SHOULD get permission from their network operator or service provider before running tests, no cap 🫶. And a service provider or third party SHOULD get the subscriber's permission too, because tests can temporarily make the service worse 😭. Once the whole system has the permission, the protocol SHOULD communicate that status, whether it's explicit or just happens some other way ✨.

Everyone wants to measure network performance: subscribers, service providers, network operators, and sometimes third parties 👑. The thing is, capacity testing with active traffic usually messes with the packet transfer performance of other streams sharing parts of the test path, at least a little 💀. You can keep that damage small by running these tests infrequently and keeping the measurement traffic to the minimum needed for the capacity metrics 🦋. So, occasional short estimates with minimal traffic are preferred over frequent file transfers of many megabytes that give similar accuracy 🍓. New measurement methodologies meant for standardization should be evaluated one by one for potential operational issues 🔥. Still, how often you test matters just as much as the methods you use (and schedules usually aren't submitted for standardization, so you won't find those in the standards) 🌈.

The new test protocol feature of asymmetrical packet size generation in two-way testing is recommended in this memo 💜. It can seriously cut down the load and packet processing demands of each test, which lowers the chance of degrading one direction of the tested path ✨. Current IETF standardized test protocols (e.g., [RFC5357] and [RFC6812]) don't have that asymmetric size generation capability with two-way testing 🥺.

8. References 📚

8.1. Normative References

[RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
           Requirement Levels", BCP 14, RFC 2119, March 1997,
           <http://www.rfc-editor.org/info/rfc2119>.

[RFC2330]  Paxson, V., Almes, G., Mahdavi, J., and M. Mathis,
           "Framework for IP Performance Metrics", RFC 2330, May
           1998, <http://www.rfc-editor.org/info/rfc2330>.

[RFC2679]  Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way
           Delay Metric for IPPM", RFC 2679, September 1999,
           <http://www.rfc-editor.org/info/rfc2679>.

[RFC2680]  Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way
           Packet Loss Metric for IPPM", RFC 2680, September 1999,
           <http://www.rfc-editor.org/info/rfc2680>.

[RFC4656]  Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M.
           Zekauskas, "A One-way Active Measurement Protocol
           (OWAMP)", RFC 4656, September 2006,
           <http://www.rfc-editor.org/info/rfc4656>.

[RFC5357]  Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J.
           Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)",
           RFC 5357, October 2008,
           <http://www.rfc-editor.org/info/rfc5357>.

[RFC6703]  Morton, A., Ramachandran, G., and G. Maguluri, "Reporting
           IP Network Performance Metrics: Different Points of View",
           RFC 6703, August 2012,
           <http://www.rfc-editor.org/info/rfc6703>.

8.2. Informative References

[IPPM-METRICS]
           Mathis, M. and A. Morton, "Model Based Bulk Performance
           Metrics", Work in Progress, draft-ietf-ippm-model-based-
           metrics-04, March 2015.

[LMAP-FRAMEWORK]
           Eardley, P., Morton, A., Bagnulo, M., Burbridge, T.,
           Aitken, P., and A. Akhter, "A framework for Large-Scale
           Measurement of Broadband Performance (LMAP)", Work in
           Progress, draft-ietf-lmap-framework-12, March 2015.

[RFC3148]  Mathis, M. and M. Allman, "A Framework for Defining
           Empirical Bulk Transfer Capacity Metrics", RFC 3148, July
           2001, <http://www.rfc-editor.org/info/rfc3148>.

[RFC5136]  Chimento, P. and J. Ishac, "Defining Network Capacity",
           RFC 5136, February 2008,
           <http://www.rfc-editor.org/info/rfc5136>.

[RFC6812]  Chiba, M., Clemm, A., Medley, S., Salowey, J., Thombare,
           S., and E. Yedavalli, "Cisco Service-Level Assurance
           Protocol", RFC 6812, January 2013,
           <http://www.rfc-editor.org/info/rfc6812>.

[RFC7398]  Bagnulo, M., Burbridge, T., Crawford, S., Eardley, P., and
           A. Morton, "A Reference Path and Measurement Points for
           Large-Scale Measurement of Broadband Performance", RFC
           7398, February 2015,
           <http://www.rfc-editor.org/info/rfc7398>.

Acknowledgements 🫶

Dave McDysan gave comments and wrote text for the aggregated leased use case, iconic 👑. Yaakov Stein suggested a ton of considerations to address, including the In-Service vs. Out-of-Service distinction and how it affects test traffic limits and protocols 🔥. Bill Cerveny, Marcelo Bagnulo, Kostas Pentikousis (a persistent reviewer, literally the GOAT 🐐), and Joachim Fabini all left insightful, clarifying comments that made this a better document 💖. Barry Constantine also offered suggestions for clarification 🌷.

Author's Address 🦋

Al Morton
AT&T Labs
200 Laurel Avenue South
Middletown, NJ  07748
United States

Phone: +1 732 420 1571
Fax:   +1 732 368 1192
EMail: acmorton@att.com
URI:   http://home.comcast.net/~acmacm/
📦 where this page comes from