RFC 8868

Evaluating Congestion Control for Interactive Real-Time Media

📅 published January 2021, from ietf

Internet Engineering Task Force (IETF) 🎀 RFC 8868 ✨

Okay bestie, come sit down because this one is so good 💖 RFC 8868 is called Evaluating Congestion Control for Interactive Real-Time Media and it was published in January 2021. The authors are V. Singh from callstats.io, J. Ott from the Technical University of Munich, and S. Holmer from Google 🦋 It's Informational, which means it's published for information and is not an Internet Standards Track spec. Ngl that's a super chill category 🌸

This doc does not obsolete any other RFC, and it does not update any either 🫶

The big idea: when someone invents a new congestion control algorithm for real-time media, like video calls and phone calls, how do they prove it's actually good? This RFC is the guidebook for testing it 🔥

Status of This Memo

Okay so this section says the document is not a Standards Track spec, it's just published for info 💅 But it is a real IETF product that represents the consensus of the IETF community, got public review, and was approved by the IESG 👑 Not every IESG-approved document is automatically a candidate for an Internet Standard though, so check out Section 2 of RFC 7841 if you want the deets 🧚

If you want the latest status, errata, or to leave feedback, head to https://www.rfc-editor.org/info/rfc8868 💜

Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved 🌈

This doc is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as of the publication date. Read those carefully because they're your rights and restrictions 🫶 And if you extract any Code Components from this doc, they must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they come with no warranty as described in that license ⭐

Table of Contents 🍓

1.  Introduction
2.  Terminology
3.  Metrics
  3.1.  RTP Log Format
4.  List of Network Parameters
  4.1.  One-Way Propagation Delay
  4.2.  End-to-End Loss
  4.3.  Drop-Tail Router Queue Length
  4.4.  Loss Generation Model
  4.5.  Jitter Models
    4.5.1.  Random Bounded PDV (RBPDV)
    4.5.2.  Approximately Random Subject to No-Reordering Bounded
            PDV (NR-BPDV)
    4.5.3.  Recommended Distribution
5.  Traffic Models
  5.1.  TCP Traffic Model
  5.2.  RTP Video Model
  5.3.  Background UDP
6.  Security Considerations
7.  IANA Considerations
8.  References
  8.1.  Normative References
  8.2.  Informative References
Contributors
Acknowledgments
Authors' Addresses

1. Introduction 🦋

Okay so the goal here is to help people evaluate new congestion control algorithms for interactive point-to-point real-time media ✨ The requirements for those algorithms are outlined in [RFC8836] 💅 This doc builds on earlier IETF work, namely Specifying New Congestion Control Algorithms [RFC5033] and Metrics for the Evaluation of Congestion Control Algorithms [RFC5166] 🫶

The guidelines are meant to help prevent congestion collapse, promote fair capacity usage, and optimize the media flow's throughput 🔥 On top of that, proposed algorithms are expected to stay within the envelope of the circuit breakers defined in RFC 8083 [RFC8083] 👑

This doc only gives the broad set of network parameters and traffic models for evaluating a new algorithm 💜 The minimum requirement for a congestion control proposal is to produce or present results for the test scenarios in [RFC8867] (Basic Test Cases), which also defines the specifics for those test cases 🎀 Proponents may also produce results for the wireless test scenarios in [RFC8869] 🌸

Now the honest part: this doc does not cover application-specific implications of congestion control, so no quality metrics are defined for performance evaluation 🙈 Quality metrics, and the algorithms to infer them, vary between media types. Metrics for things like quality of experience keep evolving, so picking suitable ones is left for future work 😭 Still, there is consensus that each congestion control algorithm should be able to show it's useful for interactive video by analysing with real codecs, real video sequences, and state-of-the-art quality metrics 🧚

And beyond optimising individual metrics, real-time apps may have more options to trade off performance, like across multiple media. Check the RMCAT requirements [RFC8836] for that 💖 Those trade-offs may get defined in the future ✨

2. Terminology 🌈

The terminology defined in RTP [RFC3550], RTP Profile for Audio and Video Conferences with Minimal Control [RFC3551], RTCP Extended Report (XR) [RFC3611], Extended RTP Profile for RTCP-Based Feedback (RTP/AVPF) [RFC4585] and Support for Reduced-Size RTCP [RFC5506] applies 🫶

3. Metrics 📊

This doc specifies testing criteria for evaluating congestion control algorithms for RTP media flows 💅 Proposed algorithms are to prove their performance through simulation and/or emulation experiments for all the cases described 🔥

Each experiment is expected to log every incoming and outgoing packet (the RTP logging format is in Section 3.1) 🦋 You can do the logging inside the application or at the endpoints using PCAP (packet capture, e.g., tcpdump [tcpdump], Wireshark [wireshark]) 🎀 The following metrics are calculated from the packet logs:

  1. Sending rate, receiver rate, goodput (measured at 200ms intervals) 💖
  2. Packets sent, packets received ✨
  3. Bytes sent, bytes received 🌸
  4. Packet delay ⭐
  5. Packets lost, packets discarded (from the playout or de-jitter buffer) 😭
  6. If using retransmission or FEC: post-repair loss 🫶
  7. Self-fairness and fairness with respect to cross traffic 👑

This one is a big deal bestie 🔥 Experiments testing a given congestion control proposal must report relative ratios of the average throughput (measured at coarser time intervals) obtained by each RTP media stream. If there's background cross-traffic like TCP, the report must also include the relative ratio between the average throughput of RTP media streams and the cross-traffic streams 💅

During static periods of a test (when bottleneck bandwidth is constant and no streams arrive or depart), these relative ratio reports act as an indicator of how fairly the RTP streams share bandwidth among themselves and against cross-traffic streams 🧚 The throughput measurement interval should be set at a few values (for example, 1 s, 5 s, and 20 s) so you can measure fairness across different timescales 🌸

As a general guideline, the relative ratio between congestion-controlled RTP flows with the same priority level and similar path RTT should be bounded between 0.333 and 3 ✨ For example, see the test scenarios in [RFC8867] 🦋

  1. Convergence time: The time taken to reach a stable rate at startup, after the available link capacity changes, or when new flows get added to the bottleneck link 🌈
  2. Instability or oscillation in the sending rate: The frequency or number of instances when the sending rate oscillates between a high watermark level and a low watermark level, or vice-versa, in a defined time window 🎀 For example, the watermarks can be set at 4x interval: 500 Kbps, 2 Mbps, and a time window of 500 ms 💜
  3. Bandwidth utilization, defined as the ratio of the instantaneous sending rate to the instantaneous bottleneck capacity 🔥 This metric is only useful when a congestion-controlled RTP flow is by itself or competing with similar cross-traffic 💅

Note that the metrics above are all objective, application-independent metrics 🫶 For objective metrics for evaluating codecs, see Section 3 of [netvc-testing] 🧚

From the logs, you can calculate the statistical measures (min, max, mean, standard deviation, and variance) for the whole duration or any specific part of the session 📊 The metrics (sending rate, receiver rate, goodput, latency) can also be graphed over time, with measurements in the plot at one-second intervals 🌸 And from the logs you can plot a histogram or cumulative distribution function (CDF) of packet delay 💖

3.1. RTP Log Format 🎀

Having a common log format makes it way easier to run analyses across different measurement setups and compare results ✨

Send or receive timestamp (Unix): <int>.<int>  -- sec.usec decimal
RTP payload type                  <int>        -- decimal
SSRC                              <int>        -- hexadecimal
RTP sequence no                   <int>        -- decimal
RTP timestamp                     <int>        -- decimal
marker bit                        0|1          -- character
Payload size                      <int>        -- # bytes, decimal

Each line of the log file should be terminated with CRLF, CR, or LF characters 🫶 Empty lines are disregarded 🌈

If the congestion control implements retransmissions or Forward Error Correction (FEC), the evaluation should report both packet loss (before applying error resilience) and residual packet loss (after applying error resilience) 👑

These data should be enough to compute the media-encoding-independent metrics above 💜 Using a common log lets you do simplified post-processing and analysis across different implementations 🦋

4. List of Network Parameters 🧚

The implementors are encouraged to choose evaluation settings from the following values initially 🌸

4.1. One-Way Propagation Delay ⭐

Experiments are expected to verify that the congestion control can work across a broad range of path characteristics, including challenging situations, for example over transcontinental and/or satellite links 🔥 Tests account for these different latencies:

  1. Very low latency: 0-1 ms 🎀
  2. Low latency: 50 ms 🍓
  3. High latency: 150 ms 💖
  4. Extreme latency: 300 ms 😭

4.2. End-to-End Loss 💀

Lots of paths on the Internet today are largely lossless, but in scenarios with interference in wireless networks, sending to and receiving from remote regions, or high/fast mobility, media flows can see substantial packet loss 🙈 Tests need to reflect that variety 💅

To model a wide range of lossy links, experiments can choose one of these loss rates, where the fractional loss is the ratio of packets lost to packets sent ✨

  1. no loss: 0% 🫶
  2. 1% 🌸
  3. 5% 🦋
  4. 10% 💜
  5. 20% 👑
That's the end of part 1 bestie, so more is coming in the next part 🫶💅

4.3. Drop-Tail Router Queue Length ✨

okay so routers should be set up to use drop-tail queues in the experiments, bestie, because they're still literally everywhere 💅 Trying out Active Queue Management (AQM) schemes is encouraged 🫶 but it's not mandatory, so no pressure.

The router queue length is measured as the time it takes to drain the FIFO queue ⏳. People keep pointing out that queue lengths on the Internet today are all over the place 🙈 The core backbone has super short queues, but home gateways usually have much longer ones. These buckets are the vibe:

  1. QoS-aware (or short): 70 ms 🌸
  2. Nominal: 300-500 ms ✨
  3. Buffer-bloated: 1000-2000 ms 💀

Here the size of the queue is measured in bytes or packets. To convert a queue length from seconds into bytes, use this formula 🧮

QueueSize (in bytes) = QueueSize (in sec) x Throughput (in bps)/8

4.4. Loss Generation Model 🎀

There are so many models for generating packet loss, bestie 🍓 Some generate correlated losses, others generate independent ones, and you can also pull losses straight out of packet traces. As the simplest minimum loss model, with the least parameterization (just the loss rate), independent random losses must be used in the evaluation.

Independent loss models can reflect reality pretty poorly 😭 so more sophisticated loss models could be considered. For correlated losses, the good options include the Gilbert-Elliot model [gilbert-elliott] and models that generate losses by modeling a queue with its (different) drop behaviors. 🧚

4.5. Jitter Models 🦋

This section defines the jitter models for this document. When jitter is applied to both the congestion-controlled RTP flow and any competing flow (like a TCP one), the competing flow uses the jitter definition below that doesn't allow reordering of packets on that flow (see the NR-BPDV definition below) 🙌

Jitter is an overloaded term in communications, lowkey confusing 🌈 It usually means the variation of a metric (e.g., delay) with respect to some reference metric (e.g., average delay or minimum delay). For example, in RFC 3550, jitter is computed as the smoothed difference in packet arrival times relative to their respective expected arrival times, which is especially meaningful if the underlying packet delay variation was caused by a Gaussian random process.

Because jitter is so overloaded, we use the term Packet Delay Variation (PDV) instead 💅 It describes the variation of delay for individual packets in the same sense the IETF IP Performance Metrics (IPPM) working group defined PDV in its documents (e.g., RFC 3393), and the way ITU-T SG16 defined IP Packet Delay Variation (IPDV) in its documents (e.g., Y.1540).

Most PDV distributions in packet network systems are one-sided, so measuring them with a finite number of samples gives you one-sided histograms 📊 In the usual packet network transport case, one packet transits the network with the minimum delay ⭐ A (large) number of packets transit within some (smaller) positive variation from that minimum. A (small) number of packets transit with delays higher than the median or average transit time. These are the outliers 👀 They're infrequent, but they can cause significant deleterious operation in adaptive systems, so they should be considered in rate adaptation designs for RTP congestion control.

In this section we define two different bounded PDV characteristics:

  1. Random Bounded PDV
  2. Approximately Random Subject to No-Reordering Bounded PDV

The former, Random Bounded PDV, is presented for information only 📚 while the latter, Approximately Random Subject to No-Reordering Bounded PDV, must be used in the evaluation 🔥

4.5.1. Random Bounded PDV (RBPDV)

The RBPDV probability distribution function (PDF) is specified to be some mathematically describable function that includes some practical minimum and maximum discrete values suitable for testing 🧁 For example, the minimum value, x_min, might be specified as the minimum transit time packet, and the maximum value, x_max, might be defined to be two standard deviations higher than the mean.

Since we're typically interested in the distribution relative to the mean delay packet, we define the zero mean PDV sample, z(n), as:

z(n) = x(n) - x_mean

where x(n) is a sample of the RBPDV random variable x, and x_mean is the mean of x.

We assume here that s(n) is the original source time of packet n. The post-jitter induced emission time, j(n), for packet n is:

j(n) = {[z(n) + x_mean] + s(n)}.

It follows that the separation in the post-jitter time of packets n and n+1 is {[s(n+1)-s(n)] - [z(n)-z(n+1)]}. Since the first term is always a positive quantity, packet reordering at the receiver is possible whenever the second term is greater than the first. Said another way, whenever the difference in possible zero mean PDV sample delays (i.e., [x_max-x_min]) exceeds the inter-departure time of any two sent packets, there's a possibility of packet reordering 🌀

There are important use cases in real networks where packets can get reordered, such as load-balancing topologies and route changes 🛣️ But for the vast majority of cases there's no reordering, because most of the time packets follow the same path. So if one packet gets overly delayed, the packets after it on that flow are delayed too 😩 This is especially true for mobile wireless links, where there are per-flow queues prior to base station scheduling. Owing to this important use case, we define another PDV profile similar to the above, but one that doesn't allow reordering within a flow 💜

4.5.2. Approximately Random Subject to No-Reordering Bounded PDV (NR-BPDV)

No Reordering BPDV, NR-BPDV, is defined similarly to the above with one important exception ✨ Let serial(n) be defined as the serialization delay of packet n at the lowest bottleneck link rate (or other appropriate rate) in a given test. Then we produce all the post-jitter values for j(n) for n = 1, 2, ... N, where N is the length of the source sequence s to be offset. The exception goes like this: we revisit all j(n) beginning from index n=2. If j(n) is determined to be less than [j(n-1)+serial(n-1)], we redefine j(n) to be equal to [j(n-1)+serial(n-1)] and continue for all remaining n (i.e., n = 3, 4, .. N). This models the case where packet n is sent immediately after packet (n-1) at the bottleneck link rate. It's generally the theoretical minimum, since it assumes no other packets from other flows sit between packet n and n+1 at the bottleneck link, but it's a reasonable assumption for per-flow queuing 🧠

This assumption holds for some important exception cases, like packets immediately following outliers. A bunch of software-controlled elements on end-to-end Internet paths (like firewalls, application-layer gateways, and other middleboxes) stop processing packets while they service other functions (e.g., garbage collection) 🗑️ Often these devices don't drop packets. They queue them for later processing and cause many of the outliers. So NR-BPDV models this particular use case (assuming serial(n+1) is defined appropriately for the device causing the outlier), and it's believed to be important for adaptation development for congestion-controlled RTP streams 🫶

Whether we're talking Random Bounded PDV or Approximately Random Subject to No-Reordering Bounded PDV, it is recommended that z(n) is distributed according to a truncated Gaussian for the above jitter models 🎀

z(n) ~ |max(min(N(0, std^(2)), N_STD * std), -N_STD * std)|

where N(0, std^(2)) is the Gaussian distribution with zero mean, and std is the standard deviation. Recommended values:

std = 5 ms
N_STD = 3

5. Traffic Models ✨

5.1. TCP Traffic Model 🚗

Long-lived TCP flows download data throughout the session, and they're expected to have infinite amount of data to send or receive 💖 This roughly applies, for example, when downloading software distributions.

Each short TCP flow is modeled as a sequence of file downloads interleaved with idle periods 🌸 Not all short TCP flows start at the same time, so some start in the ON state while others start in the OFF state.

Here's how the short TCP flows can be modeled: 30 connections start simultaneously fetching small (30-50 KB) amounts of data, evenly distributed 🧚 This covers the case where the short TCP flows are fetching web page resources rather than video files.

The idle period between bursts of starting a group of TCP flows is typically derived from an exponential distribution with a mean value of 10 seconds ⏰

These values were picked based on the data available at https://httparchive.org/reports/state-of-the-web?start=2015_10_01&end=2015_11_01&view=list as of October 2015. 📅

Many different TCP congestion control schemes are deployed today 🌈 Therefore, experimentation with a range of different schemes, especially including CUBIC [RFC8312], is encouraged. Experiments must document in detail which congestion control schemes they tested against and which parameters were used 📝

5.2. RTP Video Model 🎬

[RFC8593] describes two types of video traffic models for evaluating candidate algorithms for RTP congestion control. The first model statistically characterizes the behavior of a video encoder, whereas the second model uses video traces 🍿

Sample video test sequences are available at [xiph-seq]. The following two video streams are the recommended minimum for testing: Foreman (CIF sequence) and FourPeople (720p). Both come as raw video data to be encoded dynamically 💅 Since these video sequences are short (300 and 600 frames, respectively), they shall be stitched together repeatedly until the desired length is reached 🔁

5.3. Background UDP 🦋

A background UDP flow is modeled as a constant bit rate (CBR) flow 🫧 It will download data at a particular CBR for the complete session, or it will change to a particular CBR at predefined intervals. The inter-packet interval is calculated based on the CBR and the packet size (typically set to the path MTU size; the default value can be 1500 bytes) 🧮

Note that new transport protocols such as QUIC may use UDP. However, due to their congestion control algorithms, they'll exhibit behavior conceptually similar to the TCP flows above, so they can be subsumed by the above, including the division into short-lived and long-lived flows 👑 As QUIC evolves independently of TCP congestion control algorithms, its future congestion control should be considered as competing traffic as appropriate 🌟

  1. Security Considerations ✨

Okay so this section is the "read this before you ship anything" part, bestie 🫶 Basically this whole document is just a set of evaluation criteria and parameters for checking how well congestion control protocols and algorithms handle real-time communication. So the memo itself doesn't really have security considerations 💅 BUT the protocols and algorithms being evaluated totally might, so pay attention!!

Here's the tea: passing every test in here might be enough for a comparative evaluation, but that does NOT mean the protocol is risk-free once it's out on the Internet. Not me saying "it's fine" just because the tests went green 😭

These evaluations are usually run in controlled environments with a limited number of parallel flows. Which means they're by definition limited 🙈 They can't systematically cover big interacting groups of nodes or every possible situation. So careful protocol design is a must, so you don't accidentally push out traffic that makes networks unstable, like (local) congestion collapse. Literally the worst vibe 💀

This specification only looks at how the protocols and algorithms behave during regular operation. It does NOT check for malicious use, whether by the sender, the receiver, or some middle party messing with things, like faking, dropping, replaying, or altering congestion signals 🦋 It's on the protocol specs themselves to make sure received signals are authentic, intact, and plausible, and to take the right actions (or non-actions) when they aren't.

7. IANA Considerations 🎀

This document has no IANA actions. Nothing to register here, bestie 🌸

8. References 📚

8.1. Normative References 💜

8.2. Informative References 🌈

Contributors 🫶

The content and concepts within this document are a product of the discussion carried out in the Design Team. Iconic teamwork 👑

Michael Ramalho provided the text for the jitter models (Section 4.5). ✨

Acknowledgments 🌸

Much of this document is derived from previous work on congestion control at the IETF. Standing on the shoulders of giants, periodt 💅

The authors would like to thank Harald Alvestrand, Anna Brunstrom, Luca De Cicco, Wesley Eddy, Lars Eggert, Kevin Gross, Vinayak Hegde, Randell Jesup, Mirja Kühlewind, Karen Nielsen, Piers O'Hanlon, Colin Perkins, Michael Ramalho, Zaheduzzaman Sarker, Timothy B. Terriberry, Michael Welzl, Mo Zanaty, and Xiaoqing Zhu for providing valuable feedback on draft versions of this document. Additionally, thanks to the participants of the Design Team for their comments and discussion related to the evaluation criteria. So many legends 🦋

Authors' Addresses 🧚

Varun Singh CALLSTATS I/O Oy Rauhankatu 11 C FI-00100 Helsinki Finland

Email: varun.singh@iki.fi URI: https://www.callstats.io/

Jörg Ott Technical University of Munich Department of Informatics Chair of Connected Mobility Boltzmannstrasse 3 85748 Garching Germany

Email: ott@in.tum.de

Stefan Holmer Google Kungsbron 2 SE-11122 Stockholm Sweden

Email: holmer@google.com

That's the whole thing, bestie!! 🌈💖

📦 where this page comes from