RFC 8867

RFC 8867: Test Cases for Evaluating Congestion Control for Interactive Real-Time Media ✨

okay bestie, let's get into this one 💅🎀 This is RFC 8867, an Informational document, which means it's published for info and is NOT a standards-track spec 🌸 It was written by Z. Sarker (Ericsson AB), V. Singh (callstats.io), X. Zhu (Cisco Systems), and M. Ramalho (AcousticComms), and it's dated January 2021 💜 It doesn't obsolete or update any other RFCs, so it's a clean slate, no drama 🫶

The whole point is giving congestion control testers a set of standardized test cases for real-time interactive media 🦋 Specifically, it's about RTP media in multimedia telephony, which typically has to implement congestion control, and these tests are meant to run in a controlled environment 🔥

Status of This Memo, Copyright Notice, and the rest of the front matter are in the original, and they stay exactly as they are.

Table of Contents

1.  Introduction
2.  Terminology
3.  Structure of Test Cases
4.  Recommended Evaluation Settings
  4.1.  Evaluation Metrics
  4.2.  Path Characteristics
  4.3.  Media Source
5.  Basic Test Cases
  5.1.  Variable Available Capacity with a Single Flow
  5.2.  Variable Available Capacity with Multiple Flows
  5.3.  Congested Feedback Link with Bi-directional Media Flows
  5.4.  Competing Media Flows with the Same Congestion Control
        Algorithm
  5.5.  Round Trip Time Fairness
  5.6.  Media Flow Competing with a Long TCP Flow
  5.7.  Media Flow Competing with Short TCP Flows
  5.8.  Media Pause and Resume
6.  Other Potential Test Cases
  6.1.  Media Flows with Priority
  6.2.  Explicit Congestion Notification Usage
  6.3.  Multiple Bottlenecks
7.  Wireless Access Links
8.  Security Considerations
9.  IANA Considerations
10. References
  10.1.  Normative References
  10.2.  Informative References
Acknowledgments
Authors' Addresses

---

1. Introduction 🌈

So this memo is basically a menu of test cases for checking out congestion control algorithm proposals for real-time interactive media, in controlled environments 🧚 It's built on the guidelines in [RFC8868] and the requirements from [RFC8836] 💖 The test cases cover the basic usage scenarios and they all follow one common structure, so you can easily add more later for other topologies or different path characteristics 🍓 Any proposed congestion control algorithm for real-time interactive media should be evaluated using the test cases described in this memo, and that's the main event 👑

2. Terminology ✨

The terms used here are defined in a bunch of RTP docs: [RFC3550] for RTP itself, [RFC3551] for the RTP Profile for Audio and Video Conferences with Minimal Control, [RFC3611] for RTCP Extended Report (XR), [RFC4585] for the Extended RTP Profile for RTCP-based Feedback (RTP/AVPF), and [RFC5506] for Support for Reduced-Size RTCP 🦋 All of those definitions apply here too, so you don't need to reinvent the wheel 🫶

Ok that's Part 1 of 7, bestie! 💜 More parts are coming, so stay tuned for Section 3 and beyond 😭🎀

3. Structure of Test Cases ✨

okay so every test case in this doc follows the same basic layout 💅 that way you can describe a brand new scenario without re-explaining the same stuff every single time. each one has a general description part that covers what the test is and why it exists, plus a set of attributes that describe the testbed, like the network path between the peers and all the different traffic sources 🦋

Define the test case 🎀

• General description: describes the motivation and the goals of the test case. basically the why 🌸
• Expected behavior: describes the desired rate adaptation behavior, aka what a "slay" result looks like 👑
• List of metrics to evaluate the desired behavior: this is the minimum set of metrics (e.g., link utilization, media sending rate) that a proposed algorithm needs to measure so it can validate the expected rate adaptation behavior 📊. it should also say what time granularity to use for measuring certain metrics (e.g., averaged over 10 ms, 100 ms, or 1 s) ⏱️. the typical measurement interval is 200 ms ✨

Define testbed topology 🧚

every test case needs to define an evaluation testbed topology. Figure 1 shows one of these evaluation topologies 🗺️ here, S1..Sn are traffic sources. they generate media traffic and use the congestion control algorithm(s) being investigated. R1..Rn are the matching receivers. a test case can have one or more traffic sources (S) and their corresponding receivers (R) 🫶 the path from source to destination is called "forward", and the path from destination back to source is called "backward" 🔁

this structure is described from the point of view of the media-generating endpoints on the left-hand side of Figure 1. in that setup, the media flows go forward, and the feedback/control messages go backward. but it's also possible to set things up with media going in both forward and backward directions 🤯 in that case, unless the test case says otherwise, the backward path is expected to NOT introduce any congestion-related impairments, and it's expected to have enough capacity for both the media and the feedback/control messages. (no caps on those, just vibes 😅) also, depending on the test case, the two directions can have different path characteristics 💜

+---+                                                        +---+
|S1 |====== \               Forward -->             / =======|R1 |
+---+       \\                                     //        +---+
             \\                                   //
+---+       +-----+                            +-----+       +---+
|S2 |=======|  A  |--------------------------->|  B  |=======|R2 |
+---+       |     |<---------------------------|     |       +---+
            +-----+                            +-----+
(...)         //                                  \\         (...)
             //          <-- Backward              \\
+---+       //                                      \\       +---+
|Sn |====== /                                        \ ======|Rn |
+---+                                                        +---+

             Figure 1: Example of a Testbed Topology

in a real-equipment testbed, there can be a LOT of unwanted traffic on the parts of the network path between the endpoints 🙈 some of it comes from other processes on the endpoints themselves (e.g., discovery protocols), or from other endpoints that aren't under test right now. this unwanted traffic should be removed or avoided as much as possible 🧹

Define testbed attributes 🍓

• Duration: defines how long the test runs, in seconds ⏳
• Path characteristics: defines the end-to-end transport level path characteristics of the testbed for a particular test case 🛣️ there are two sets of attributes describing them, one for the forward path and one for the backward path. the path characteristics for a given direction apply to all the sources "S" sending traffic on that path. if only one attribute is specified, it's used for both path directions. however, unless it's specified otherwise, the reverse path has no capacity restrictions and no path loss 💅
  • Path direction: forward or backward ➡️⬅️
  • Minimum bottleneck-link capacity: defines the minimum capacity of the end-to-end path 📉
  • Reference bottleneck capacity: defines a reference value for the bottleneck capacity in test cases where the bottleneck capacities change over time 🌊 all bottleneck capacities get specified as a ratio of this reference capacity value 🧮
  • One-way propagation delay: describes the end-to-end latency along the path when network queues are empty, i.e., the time it takes a packet to go from sender to receiver without hitting any queuing delay 🏃‍♀️💨
  • Maximum end-to-end jitter: defines the maximum jitter that can be observed along the path 🎢
  • Bottleneck queue type: for example, "tail drop" [RFC7567], Flow Queue Controlled Delay (FQ-CoDel) [RFC8290], or Proportional Integral controller Enhanced (PIE) [RFC8033] 🪣
  • Bottleneck queue size: defines the size of the queue in terms of queuing time when the queue is full (in milliseconds) ⏲️
  • Path loss ratio: characterizes the non-congested, additive losses to be generated on the end-to-end path 💧 this must describe the loss pattern or loss model used to generate the losses. no vague vibes allowed here 🙅‍♀️
• Application-related: defines the traffic source behavior for implementing the test case 🎬
  • Media traffic source: defines the characteristics of the media sources. if you're using more than one media source, the different attributes get listed separately for each one 📹
    • Media type: Video/Voice 🎙️
    • Media flow direction: forward, backward, or both ↔️
    • Number of media sources: defines the total number of media sources 🔢
    • Media codec: Constant Bit Rate (CBR) or Variable Bit Rate (VBR) 🎚️
    • Media source behavior: describes the media encoder behavior 🧠 it defines the main parameters that affect the adaptation behavior. this may include, but isn't limited to, the following:
      • Adaptability: describes the adaptation options 🦎 for video, it defines these ranges of adaptation: bit rate, frame rate, and video resolution 🎞️ for voice, it defines the range of bit rate adaptation, the sampling rate variation, and the variation in packetization interval 🗣️
      • Output variation: for a VBR encoder, it defines how much the encoder output varies from the average target rate over a particular measurement interval 📈 for example, on average the encoder output may vary between 5% to 15% above or below the average target bit rate when measured over a 100 ms time window. the time interval for the variation MUST be provided 🙌
      • Responsiveness to a new bit rate request: the lag between a new bit rate request from the congestion control algorithm and the actual rate change in encoder output ⏳ depending on the encoder, this might be given in absolute time (e.g., 10 ms to 1000 ms) or another appropriate metric (e.g., next frame interval time) 🎯
      • more detailed discussions on expected media source behavior, including those from synthetic video traffic sources, can be found in [RFC8593] 📚
    • Media content: describes the chosen video scenario 🎥 for example, video test sequences are available at [xiph-seq] and [HEVC-seq]. different video scenarios give different distributions of video frames from the video encoder, so it's important to specify the media content used in a test 🧐 if a synthetic video traffic source [RFC8593] is used, it needs to be configured to match the characteristics of the media content you specified 🛠️
    • Media timeline: describes when the media source gets introduced to and removed from the testbed ⏰ for example, the media source might start sending right when the test case begins, or after a few seconds 🕰️
    • Startup behavior: the media starts at a defined bit rate, which could be the minimum, the maximum, or something in between (in Kbps) 🚀
  • Competing traffic source: describes the characteristics of the competing traffic source. the different types of competing flows are listed in [RFC8868] 🥊
    • Traffic direction: forward, backward, or both 🔀
    • Type of sources: defines the types of competing traffic sources. the types of competing traffic flows are listed in [RFC8868]. for example, the number of TCP flows from a web browser, plus the mean size and distribution of the content being downloaded 🌐
    • Number of sources: defines the total number of competing sources of each media type per traffic direction 🔢
    • Congestion control: enumerates the congestion control used by each type of competing traffic 🚦
    • Traffic timeline: describes when the competing traffic starts and ends in the test case 🗓️
• Additional attributes: describes attributes that are essential for implementing a test case but aren't covered by the structure above 🧩 these attributes must be well defined, so that other implementers of that test case can build it easily 🫶

Any attribute can have a set of values (enclosed within []). each member value of that set must be treated as a different value for the same attribute 🌈 it's desired to run separate tests for each of those attribute values 🧪

all the test cases in this document follow the structure above ✅

4. Recommended Evaluation Settings 💜

this section describes recommended test case settings 🎀 and these could be overwritten by the respective test cases 🦋

4.1. Evaluation Metrics ✨

Okay bestie, so to see how the candidate algorithms are actually doing, implementers must log enough info to visualize the following metrics at a fine enough time granularity 📈💖

1. Flow level 🌊
   • A. End-to-end delay for the congestion-controlled media flow(s). For example, end-to-end delay observed on the IP packet level and the video frame level.
   • B. Variation in sending bit rate and throughput. Mainly observing the frequency and magnitude of oscillations.
   • C. Packet losses observed at the receiving endpoint.
   • D. Feedback message overhead.
   • E. Convergence time. Time to reach steady state for the congestion-controlled media flow(s). Each occurrence of convergence during the test period needs to be presented. 🎯
2. Transport level 🚚
   • A. Bandwidth utilization.
   • B. Queue length (milliseconds at specified path capacity).

---

4.2. Path Characteristics 🛣️

Each path between a sender and receiver as described in Figure 1 has the following characteristics unless otherwise specified in the test case:

• Path direction: forward and backward.
• Reference bottleneck capacity: 1 Mbps.
• One-way propagation delay: 50 ms. Implementers are encouraged to run the experiment with additional propagation delays mentioned in [RFC8868].
• Maximum end-to-end jitter: 30 ms. Jitter models are described in [RFC8868].
• Bottleneck queue type: "tail drop". Implementers are encouraged to run the experiment with other Active Queue Management (AQM) schemes, such as FQ-CoDel and PIE. 💅
• Bottleneck queue size: 300 ms.
• Path loss ratio: 0%.

Examples of additional network parameters are discussed in [RFC8868].

For test cases involving time-varying bottleneck capacity, all capacity values are specified as a ratio with respect to a reference capacity value, so as to allow flexible scaling of capacity values along with media source rate range. There are two different mechanisms for inducing path capacity variation: a) by explicitly modifying the value of physical link capacity, or b) by introducing background non-adaptive UDP traffic with time-varying traffic rate. Implementers are encouraged to run the experiments with both mechanisms for test cases specified in Section 5.1, Section 5.2, and Section 5.3. 🔁✨

---

4.3. Media Source 🎥

Unless otherwise specified, each test case will include one or more media sources as described below:

• Media type: Video
  • Media codec: VBR
  • Media source behavior:
    • Adaptability:
      • Bit rate range: 150 Kbps - 1.5 Mbps. In real-life applications, the bit rate range can vary a lot depending on the provided service; for example, the maximum bit rate can be up to 4 Mbps. However, for running tests to evaluate the congestion control algorithms, it is more important to have a look at how they react to a certain amount of bandwidth change. Also it is possible that the media traffic generator used in a particular simulator or testbed is not capable of generating a higher bit rate. Hence, we have selected a suitable bit rate range typical of consumer-grade video conferencing applications in designing the test case. If a different bit rate range is used in the test cases, then the end-to-end path capacity values will also need to be scaled accordingly. 📞
      • Frame resolution: 144p - 720p (or 1080p). This resolution range is selected based on the bit rate range. If a different bit rate range is used in the test cases, then a suitable frame resolution range also needs to be selected.
      • Frame rate: 10 fps - 30 fps. This frame rate range is selected based on the bit rate range. If a different bit rate range is used in the test cases, then the frame rate range also needs to be suitably adjusted.
    • Variation from target bit rate: +/-5%. Unless otherwise specified in the test case(s), bit rate variation should be calculated over a one (1) second period of time.
    • Responsiveness to new bit rate request: 100 ms
  • Media content: The media content should represent a typical video conversational scenario with head and shoulder movement. We recommend using the Foreman video sequence [xiph-seq]. 🙂
  • Media startup behavior: 150 Kbps. It should be noted that applications can use smart ways to select an optimal startup bit rate value for a certain network condition. In such cases, the candidate proposals may show the effectiveness of such a smart approach as additional information for the evaluation process. 🚀
• Media type: Audio
  • Media codec: CBR
  • Media bit rate: 20 Kbps

---

5. Basic Test Cases 🎬

5.1. Variable Available Capacity with a Single Flow 💃

In this test case, the minimum bottleneck-link capacity between the two endpoints varies over time. This test is designed to measure the responsiveness of the candidate algorithm. This test tries to address the requirements in [RFC8836], which requires the algorithm to adapt the flow(s) and provide lower end-to-end latency when there exists:

• an intermediate bottleneck
• change in available capacity (e.g., due to interface change, routing change, abrupt arrival/departure of background non-adaptive traffic)
• maximum media bit rate is greater than link capacity. In this case, when the application tries to ramp up to its maximum bit rate, since the link capacity is limited to a lower value, the congestion control scheme is expected to stabilize the sending bit rate close to the available bottleneck capacity. 🛑

It should be noted that the exact variation in available capacity due to any of the above depends on the underlying technologies. Hence, we describe a set of known factors, which may be extended to devise a more specific test case targeting certain behaviors in a certain network environment. 🧠

• Expected behavior: The candidate algorithm is expected to detect the path capacity constraint, converge to the bottleneck link's capacity, and adapt the flow to avoid unwanted media rate oscillation when the sending bit rate is approaching the bottleneck link's capacity. Such oscillations might occur when the media flow(s) attempts to reach its maximum bit rate but overshoots the usage of the available bottleneck capacity, then to rectify, it reduces the bit rate and starts to ramp up again. 🎢
• Evaluation metrics: As described in Section 4.1.
• Testbed topology: One media source S1 is connected to the corresponding R1. The media traffic is transported over the forward path and corresponding feedback/control traffic is transported over the backward path. 🗺️

                            Forward -->
   +---+       +-----+                               +-----+       +---+
   |S1 |=======|  A  |------------------------------>|  B  |=======|R1 |
   +---+       |     |<------------------------------|     |       +---+
               +-----+                               +-----+
                             <-- Backward

Figure 2: Testbed Topology for Limited Link Capacity

• Testbed attributes:
  • Test duration: 100 s
  • Path characteristics: as described in Section 4.2
  • Application-related:
    • Media Traffic:
      • Media type: Video
        • Media direction: forward
        • Number of media sources: one (1)
        • Media timeline:
          • Start time: 0 s
          • End time: 99 s
      • Media type: Audio
        • Media direction: forward
        • Number of media sources: one (1)
        • Media timeline:
          • Start time: 0 s
          • End time: 99 s
    • Competing traffic:
      • Number of sources: zero (0)
• Test-specific information:
  • One-way propagation delay: [50 ms, 100 ms]. On the forward path direction.
  • This test uses bottleneck path capacity variation as listed in Table 1.
  • When using background non-adaptive UDP traffic to induce a time-varying bottleneck, the physical path capacity remains at 4 Mbps, and the UDP traffic source rate changes over time as (4 - (Y x r)), where r is the Reference bottleneck capacity in Mbps, and Y is the path capacity ratio specified in Table 1. 📉📈

   +=========================+================+=======+===============+
   | Variation pattern index | Path direction | Start | Path capacity |
   |                         |                | time  | ratio         |
   +=========================+================+=======+===============+
   | One                     | Forward        | 0 s   | 1.0           |
   +-------------------------+----------------+-------+---------------+
   | Two                     | Forward        | 40 s  | 2.5           |
   +-------------------------+----------------+-------+---------------+
   | Three                   | Forward        | 60 s  | 0.6           |
   +-------------------------+----------------+-------+---------------+
   | Four                    | Forward        | 80 s  | 1.0           |
   +-------------------------+----------------+-------+---------------+

Table 1: Path Capacity Variation Pattern for the Forward Direction

5.2. Variable Available Capacity with Multiple Flows

okay so this one is basically the sequel to 5.1 🎀 but now we're adding competing traffic, so it's not just you and the bandwidth, there's other stuff fighting for the same pipe 💅 the vibe is the same tho: the candidate algorithm should notice when the available capacity changes and adapt the media stream(s) accordingly 👑

ideally the flows settle around their maximum bit rate, because the maximum link capacity is big enough to fit them all 🌸 then when capacity drops, the flows should back off by lowering their sending bit rate, and when the congestion clears, they should ramp back up. main character glow-up arc, honestly ✨

Evaluation metrics: same as Section 4.1 🦋

Testbed topology: two media sources, S1 and S2, each connected to their own destination, R1 and R2 🧚 the media goes over the forward path and the feedback/control traffic comes back over the backward path 🔁

   +---+                                                         +---+
   |S1 |===== \                                         / =======|R1 |
   +---+      \\             Forward -->               //        +---+
               \\                                     //
               +-----+                               +-----+
               |  A  |------------------------------>|  B  |
               |     |<------------------------------|     |
               +-----+                               +-----+
                 //                                    \\
                //          <-- Backward                \\
   +---+       //                                        \\       +---+
   |S2 |====== /                                          \ ======|R2 |
   +---+                                                          +---+

         Figure 3: Testbed Topology for Variable Available Capacity

Testbed attributes: basically the same as Section 5.1 🍓 the only difference is the test-specific capacity variation setup 🎀

Test-specific information: this test uses the path capacity variation from Table 2, and it ends at 125 seconds ⏰ the reference bottleneck capacity is 2 Mbps 💖 when you use background non-adaptive UDP traffic to create a time-varying bottleneck for the congestion-controlled media flows, the physical path capacity is 4 Mbps 🔥 and the UDP source rate changes over time as (4 - (Y x r)) 🧮, where r is the reference bottleneck capacity in Mbps and Y is the path capacity ratio from Table 2 🌈

+=========================+================+=======+===============+
| Variation pattern index | Path direction | Start | Path capacity |
|                         |                | time  | ratio         |
+=========================+================+=======+===============+
| One                     | Forward        | 0 s   | 2.0           |
+-------------------------+----------------+-------+---------------+
| Two                     | Forward        | 25 s  | 1.0           |
+-------------------------+----------------+-------+---------------+
| Three                   | Forward        | 50 s  | 1.75          |
+-------------------------+----------------+-------+---------------+
| Four                    | Forward        | 75 s  | 0.5           |
+-------------------------+----------------+-------+---------------+
| Five                    | Forward        | 100 s | 1.0           |
+-------------------------+----------------+-------+---------------+

Table 2: Path Capacity Variation Pattern for the Forward Direction 📊

5.3. Congested Feedback Link with Bi-directional Media Flows

okay this one is my favorite drama 😭🫶 real-time interactive media uses RTP, so it's assumed that RTCP, an RTP header extension, or something similar carries the feedback that the congestion control algorithm uses over the back channel 📡 here's the twist: because the link between the two peers is asymmetric, a participating peer might not get that feedback if the back channel is impaired or congested, even when the forward channel is totally fine 💀 this test is built to see how the candidate congestion control behaves when that happens 🙈

Expected behavior: the candidate algorithms should be able to cope without feedback information and adapt so the media flows on the forward channel still take the smallest possible hit 💅✨

It should be noted that for this test case, logs are compared with the reference case, i.e., when the backward channel has no impairments. 👀

Evaluation metrics: same as Section 4.1 🦋

Testbed topology: one media source, S1, is connected to its corresponding R1, but both endpoints are also receiving and sending data, respectively 💜 the media traffic from S1 to R1 goes over the forward path, and its feedback/control traffic comes back over the backward path 🔁 meanwhile, media traffic from S2 to R2 goes over the backward path, and its feedback/control traffic goes over the forward path 🌸

+---+                                                          +---+
|S1 |====== \                Forward -->              / =======|R1 |
+---+       \\                                       //        +---+
             \\                                     //
          +-----+                               +-----+
          |  A  |------------------------------>|  B  |
          |     |<------------------------------|     |
          +-----+                               +-----+
             //                                     \\
            //            <-- Backward               \\
+---+      //                                         \\       +---+
|R2 |===== /                                           \ ======|S2 |
+---+                                                          +---+

        Figure 4: Testbed Topology for Congested Feedback Link

Testbed attributes:

• Test duration: 100 s ⏱️
• Path characteristics:
  • Reference bottleneck capacity: 1 Mbps 🍬
• Application-related:
  • Media source:
    • Media type: Video 🎥
      • Media direction: forward and backward
      • Number of media sources: two (2)
      • Media timeline:
        • Start time: 0 s
        • End time: 99 s
    • Media type: Audio 🎧
      • Media direction: forward and backward
      • Number of media sources: two (2)
      • Media timeline:
        • Start time: 0 s
        • End time: 99 s
  • Competing traffic:
    • Number of sources: zero (0) 🫧

Test-specific information: this test uses path capacity variations to create a congested feedback link ⚡ Table 3 lists the variation patterns on the forward path, and Table 4 lists the ones on the backward path 🗺️ when you use background non-adaptive UDP traffic to create a time-varying bottleneck for the congestion-controlled media flows, the physical path capacity is 4 Mbps in both directions, and the UDP source rate changes over time as (4-x) Mbps in each direction, where x is the bottleneck capacity from Table 4 🧁

+=========================+================+=======+===============+
| Variation pattern index | Path direction | Start | Path capacity |
|                         |                | time  | ratio         |
+=========================+================+=======+===============+
| One                     | Forward        | 0 s   | 2.0           |
+-------------------------+----------------+-------+---------------+
| Two                     | Forward        | 20 s  | 1.0           |
+-------------------------+----------------+-------+---------------+
| Three                   | Forward        | 40 s  | 0.5           |
+-------------------------+----------------+-------+---------------+
| Four                    | Forward        | 60 s  | 2.0           |
+-------------------------+----------------+-------+---------------+

Table 3: Path Capacity Variation Pattern for the Forward Direction

+=========================+================+=======+===============+
| Variation pattern index | Path direction | Start | Path capacity |
|                         |                | time  | ratio         |
+=========================+================+=======+===============+
| One                     | Backward       | 0 s   | 2.0           |
+-------------------------+----------------+-------+---------------+
| Two                     | Backward       | 35 s  | 0.8           |
+-------------------------+----------------+-------+---------------+
| Three                   | Backward       | 70 s  | 2.0           |
+-------------------------+----------------+-------+---------------+

Table 4: Path Capacity Variation Pattern for the Backward Direction 🎀

5.4. Competing Media Flows with the Same Congestion Control Algorithm 🎀

okay so this one is giving "group project, but make it competitive" 💅 bestie, picture a real-time app sending more than one media flow to the same destination. All of those flows get multiplexed over the same port 🦋 and, since they're going the same way, they're probably taking the same path and having to split the bandwidth between themselves. To keep things simple, we're assuming there's no other competing traffic on the bottleneck link 🌸 and there's enough room for every flow to fit on its own. This test looks a lot like the one in Section 5.2, but it's really about how well the algorithm shares capacity ✨. The Section 5.2 test is more about how adaptable, stable, and responsive the algorithm is. Different vibes, same main character energy 👑

Expected behavior: the competing flows should totally converge on a bit rate that fits everyone, with the least latency and loss possible 💖 The big moments: three media flows show up at different times. When the second one arrives, there should still be room for another flow on the bottleneck link 🫶. Then when the third flow shows up, the bottleneck link should be saturated 🔥 (fully packed, no empty seats, iykyk 😭).

Evaluation metrics: same as Section 4.1, no notes 💅

Testbed topology: three media sources, S1, S2, and S3, are hooked up to R1, R2, and R3. The media travels over the forward path 🦋 and the feedback/control traffic comes back over the backward path 🧚

   +---+                                                         +---+
   |S1 |===== \                Forward -->              / =======|R1 |
   +---+      \\                                       //        +---+
               \\                                     //
   +---+       +-----+                               +-----+       +---+
   |S2 |=======|  A  |------------------------------>|  B  |=======|R2 |
   +---+       |     |<------------------------------|     |       +---+
               +-----+                               +-----+
               //          <-- Backward               \\
   +---+      //                                       \\       +---+
   |S3 |===== /                                         \ ======|R3 |
   +---+                                                        +---+

Figure 5: Testbed Topology for Multiple Congestion-Controlled Media Flows 🌈

Testbed attributes:

• Test duration: 120 s ⏱️
• Path characteristics:
  • Reference bottleneck capacity: 3.5 Mbps
  • Path capacity ratio: 1.0
• Application-related:
  • Media Source:
    • Media type: Video
      • Media direction: forward
      • Number of media sources: three (3)
      • Media timeline: new media flows get added one after another, at short time intervals. Check the test-specific setup below 🍓
    • Media type: Audio
      • Media direction: forward
      • Number of media sources: three (3)
      • Media timeline: new media flows get added one after another, at short time intervals. Check the test-specific setup below 🍓
  • Competing traffic:
    • Number of sources: zero (0) 🙈

Test-specific information: Table 5 lays out the media timeline for both media types 💜

+=========+============+============+==========+
| Flow ID | Media type | Start time | End time |
+=========+============+============+==========+
| 1       | Video      | 0 s        | 119 s    |
+---------+------------+------------+----------+
| 2       | Video      | 20 s       | 119 s    |
+---------+------------+------------+----------+
| 3       | Video      | 40 s       | 119 s    |
+---------+------------+------------+----------+
| 4       | Audio      | 0 s        | 119 s    |
+---------+------------+------------+----------+
| 5       | Audio      | 20 s       | 119 s    |
+---------+------------+------------+----------+
| 6       | Audio      | 40 s       | 119 s    |
+---------+------------+------------+----------+

Table 5: Media Timelines for Video and Audio Media Sources 💖

---

5.5. Round Trip Time Fairness 🌸

okay this one is about flows that share the bottleneck link but each have a different one-way propagation delay 🧚 Again we're keeping it simple: no other competing traffic on the bottleneck link, and enough capacity for all the flows ✨. It looks like a variant of test case 5.2 (Section 5.2), but the focus is on how well the algorithm shares capacity when every flow has a different RTT 🦋. Main character energy, but make it distance-based 👑

Expected behavior: the competing flows should converge on bit rates that fit everybody, with minimum possible latency and loss 💅 How good the algorithm is comes down to how quickly and how fairly the flows reach their steady states, no matter what RTT each one sees 🫶

Evaluation metrics: same as Section 4.1, no notes 💅

Testbed topology: five (5) media sources, S1..S5, are connected to their matching media sinks, R1..R5. The media travels over the forward path and the feedback/control traffic comes back over the backward path 🦋. The topology is the same as in Section 5.4 🎀

Testbed attributes:

• Test duration: 300 s ⏱️
• Path characteristics:
  • Reference bottleneck capacity: 4 Mbps
  • Path capacity ratio: 1.0
  • One-way propagation delay for each flow: 10 ms for S1-R1, 25 ms for S2-R2, 50 ms for S3-R3, 100 ms for S4-R4, and 150 ms S5-R5 🌈
• Application-related:
  • Media source:
    • Media type: Video
      • Media direction: forward
      • Number of media sources: five (5)
      • Media timeline: new media flows get added one after another, at short time intervals. Check the test-specific setup below 🍓
    • Media type: Audio
      • Media direction: forward
      • Number of media sources: five (5)
      • Media timeline: new media flows get added one after another, at short time intervals. Check the test-specific setup below 🍓
  • Competing traffic:
    • Number of sources: zero (0) 🙈

Test-specific information: Table 6 lays out the media timeline for both media types 💜

+=========+============+============+==========+
| Flow ID | Media type | Start time | End time |
+=========+============+============+==========+
| 1       | Video      | 0 s        | 299 s    |
+---------+------------+------------+----------+
| 2       | Video      | 10 s       | 299 s    |
+---------+------------+------------+----------+
| 3       | Video      | 20 s       | 299 s    |
+---------+------------+------------+----------+
| 4       | Video      | 30 s       | 299 s    |
+---------+------------+------------+----------+
| 5       | Video      | 40 s       | 299 s    |
+---------+------------+------------+----------+
| 6       | Audio      | 0 s        | 299 s    |
+---------+------------+------------+----------+
| 7       | Audio      | 10 s       | 299 s    |
+---------+------------+------------+----------+
| 8       | Audio      | 20 s       | 299 s    |
+---------+------------+------------+----------+
| 9       | Audio      | 30 s       | 299 s    |
+---------+------------+------------+----------+
| 10      | Audio      | 40 s       | 299 s    |
+---------+------------+------------+----------+

Table 6: Media Timeline for Video and Audio Media Sources 💅

5.6. Media Flow Competing with a Long TCP Flow ✨

okay so this one is giving big download vs. video call 📥💖. One or more media flows share the bottleneck link with at least one long-lived TCP flow. The TCP flows keep downloading data the whole session, and they're expected to have an infinite amount of data to send and receive. It's basically a multimedia app sharing the connection with a huge file download, which is so relatable 😭. This test checks how well the candidate algorithm adapts to that competing traffic. It addresses requirement 3 in Section 2 of [RFC8836] 🎀.

Expected behavior: Depending on how well things converge in test cases 5.1 and 5.2, the candidate algorithm may be able to avoid congestion collapse 🌈. In the worst case, the media stream will fall to the minimum media bit rate 🥺.

Evaluation metrics: These are in addition to the ones described in Section 4.1 💅.

1. Flow level:
   • a. TCP throughput
   • b. Loss for the TCP flow

Testbed topology: One (1) media source, S1, is connected to the corresponding media sink, R1 🎧. There's also a long-lived TCP flow sharing the same bottleneck link 🌊. The media traffic goes over the forward path, and the matching feedback/control traffic goes over the backward path 🦋. The TCP traffic goes over the forward path from S_tcp, and its acknowledgment packets go over the backward path from R_tcp 💜.

    +--+                                                     +--+
    |S1|===== \              Forward -->              / =====|R1|
    +--+      \\                                     //      +--+
               \\                                   //
               +-----+                             +-----+
               |  A  |---------------------------->|  B  |
               |     |<----------------------------|     |
               +-----+                             +-----+
               //        <-- Backward               \\
   +-----+    //                                     \\    +-----+
   |S_tcp|=== /                                       \ ===|R_tcp|
   +-----+                                                 +-----+

     Figure 6: Testbed Topology for TCP vs Congestion-Controlled Media
                                   Flows

Testbed attributes:

• Test duration: 120 s ⏱️
• Path characteristics:
  • Reference bottleneck capacity: 2 Mbps
  • Path capacity ratio: 1.0
  • Bottleneck queue size: [300 ms, 1000 ms]
• Application-related:
  • Media source:
    • Media type: Video
      • Media direction: forward
      • Number of media sources: one (1)
      • Media timeline:
        • Start time: 5 s
        • End time: 119 s
    • Media type: Audio
      • Media direction: forward
      • Number of media sources: one (1)
      • Media timeline:
        • Start time: 5 s
        • End time: 119 s
    • Additionally, implementers are encouraged to run the experiment with multiple media sources 🙌.
  • Competing traffic:
    • Number and types of sources: one (1) and long-lived TCP
    • Traffic direction: forward
    • Congestion control: default TCP congestion control [RFC5681]. Implementers are also encouraged to run the experiment with alternative TCP congestion control algorithms 🔥.
    • Traffic timeline:
      • Start time: 0 s
      • End time: 119 s

Test-specific information: none ✨

---

5.7. Media Flow Competing with Short TCP Flows 🌸

this one is the ten tabs open while you're also on a call scenario 💀. One or more congestion-controlled media flows share the bottleneck link with multiple short-lived TCP flows. Short-lived TCP flows have that on/off pattern you see in web traffic, where clients (like browsers) connect to a server and download a resource (usually a web page, a few images, text files, etc.) over several TCP connections 🌐. This shows how a multimedia app performs when a bunch of browser windows are active. The test measures how well the candidate algorithm adapts to competing web traffic, and it addresses requirement 1.E in Section 2 of [RFC8836] 🎯.

Depending on the number of short TCP flows, the cross traffic either looks like a short burst or acts like a long-lived TCP flow 🌀. The point of this test is to see how a short-term burst affects the candidate algorithm 👀.

Expected behavior: The candidate algorithm is expected to avoid flow starvation while short, bursty competing TCP flows are around, and it should keep streaming at least at the minimum media bit rate 🫶. After the competing TCP flows finish, the media streams are expected to be robust enough to eventually recover to their previous steady-state behavior, and at the very least, avoid persistent starvation 🌈.

Evaluation metrics: These are in addition to the ones described in Section 4.1 💅.

1. Flow level:
   • A. Variation in the sending rate of the TCP flow
   • B. TCP throughput

Testbed topology: The topology here is the same as the one described in Figure 6 🦋.

Testbed attributes:

• Test duration: 300 s ⏳
• Path characteristics:
  • Reference bottleneck capacity: 2.0 Mbps
  • Path capacity ratio: 1.0
• Application-related:
  • Media source:
    • Media type: Video
      • Media direction: forward
      • Number of media sources: two (2)
      • Media timeline:
        • Start time: 5 s
        • End time: 299 s
    • Media type: Audio
      • Media direction: forward
      • Number of media sources: two (2)
      • Media timeline:
        • Start time: 5 s
        • End time: 299 s
  • Competing traffic:
    • Number and types of sources: ten (10), short-lived TCP flows. 🧁
    • Traffic direction: forward
    • Congestion algorithm: default TCP congestion control [RFC5681]. Implementers are also encouraged to run the experiment with an alternative TCP congestion control algorithm 💜.
    • Traffic timeline: Each short TCP flow is modeled as a sequence of file downloads interleaved with idle periods. Not all short TCP flows start at the same time: two of them start in the ON state, while the rest of the eight flows start in an OFF state. For a description of the short TCP flow model, see test-specific information below 🎀.

Test-specific information:

• Short TCP traffic model: The short TCP model to be used in this test is described in [RFC8868] 📖.

---

5.8. Media Pause and Resume ⏸️

this one is all about the pause, then come back and fight for your spot energy 😤. More than one real-time interactive media flow shares the link bandwidth, and all the flows reach a steady state by using the link capacity in an optimal way 🌟. At this point, one of the media flows gets paused for a moment. That frees up more available bandwidth for the rest of the flows, since they're all on a shared link 🍓. When the paused flow resumes, it no longer has the same bandwidth share on the link. It has to make its way through the other existing flows to get a fair share of the link capacity again 💅.

This test case is important, especially for real-time interactive media that has more than one media flow and can pause or resume flows at any point during the session 🎥. It directly addresses requirement 5 in Section 2 of [RFC8836] 🎯. You can think of it as a variation of the test case in Section 5.4 👯. It's different, though, because the candidate algorithms can use different strategies to increase efficiency. For example, they can work on fairness, convergence time, oscillation reduction, and so on, by taking advantage of the fact that they already have previous information about the link 🧠.

Expected behavior: While the third stream is paused, the two remaining flows are expected to increase their rates and reach the maximum media bit rate 🚀. When the third stream resumes, all three flows are expected to converge to the same original fair share of rates they had before the media pause/resume event 🫧.

Evaluation metrics: These are in addition to the ones described in Section 4.1 💖.

1. Flow level:
   • A. Variation in sending bit rate and throughput. Mainly observing the frequency and magnitude of oscillations 📈.

Testbed topology: Same as the test case defined in Section 5.4 🌸.

Testbed attributes: The general description of the testbed parameters is the same as Section 5.4, with changes in the test-specific setup as below 🧚.

Other test-specific setup:

• Media flow timeline:
  • Flow ID: one (1)
  • Start time: 0 s
  • Flow duration: 119 s
  • Pause time: not required
  • Resume time: not required
• Media flow timeline:
  • Flow ID: two (2)
  • Start time: 0 s
  • Flow duration: 119 s
  • Pause time: at 40 s
  • Resume time: at 60 s
• Media flow timeline:
  • Flow ID: three (3)
  • Start time: 0 s
  • Flow duration: 119 s
  • Pause time: not required
  • Resume time: not required

---

6. Other Potential Test Cases 🌈

okay bonus round 🎉! It's been noticed that there are other interesting test cases beyond the basic ones listed above. These extra tests can help further evaluate the candidate algorithm in lots of ways 🔍. They're listed below 💜.

6.1. Media Flows with Priority 👑

in this test, media flows get different priority levels. It's an extension of Section 5.4, where the same test runs with different priority levels imposed on each media flow 🎀. For example, the first flow (S1) is assigned a priority of 2, while the other two flows (S2 and S3) are assigned a priority of 1. The candidate algorithm must reflect the relative priorities assigned to each media flow 💅. In this case, the first flow (S1) must arrive at a steady-state rate of approximately twice that of the other two flows (S2 and S3) 🦋.

The candidate algorithm can use a coupled congestion control mechanism [RFC8699], or it can use a weighted priority scheduler for the bandwidth distribution according to each media flow's priority or use 🫶.

6.2. Explicit Congestion Notification Usage 🔔

this test requires running all the basic test cases with Explicit Congestion Notification (ECN) [RFC6679] enabled 🌟. The goal is to show that the candidate algorithms don't fail when ECN signals are available 🙈. With ECN signals enabled, the algorithms are expected to perform better than their delay-based variants ✨.

6.3. Multiple Bottlenecks ✨

okay bestie, this one is a whole vibe 🌸 One congestion-controlled media flow, S1->R1, has to travel a path with multiple bottlenecks 🚧 Like, its path is literally two traffic jams in a row. In Figure 7, the first flow (S1->R1) fights the second congestion-controlled media flow (S2->R2) over the link between A and B, which is close to the sender side. Then that same flow (S1->R1) fights the third congestion-controlled media flow (S3->R3) over the link between C and D, which is close to the receiver side. The goal is to check that the candidate algorithms still work properly when there's more than one bottleneck link on the end-to-end path. Iconic test energy 💅

Expected behavior: the candidate algorithm is expected to hit full utilization at both bottleneck links 🔥 without starving any of the three congestion-controlled media flows, and it should keep a fair share of the available bandwidth at each bottleneck. No one gets left behind, periodt 🫶

                            Forward ---->

            +---+          +---+        +---+      +---+
            |S2 |          |R2 |        |S3 |      |R3 |
            +---+          +---+        +---+      +---+
              |              |            |          |
              |              |            |          |
+---+      +-----+       +-----+      +-----+     +-----+      +---+
|S1 |======|  A  |------>|  B  |----->|  C  |---->|  D  |======|R1 |
+---+      |     |<------|     |<-----|     |<----|     |      +---+
           +-----+       +-----+      +-----+     +-----+

                    1st                       2nd
             Bottleneck (A->B)          Bottleneck (C->D)

                          <------ Backward

         Figure 7: Testbed Topology for Multiple Bottlenecks

Testbed topology: three media sources, S1, S2, and S3, are hooked up to their respective destinations, R1, R2, and R3 🧚 For all three flows, the media traffic goes over the forward path, and the matching feedback/control traffic comes back over the backward path.

Testbed attributes:

• Test duration: 300 s ⏱️
• Path characteristics:
  • Reference bottleneck capacity: 2 Mbps
  • Path capacity ratio between A and B: 1.0
  • Path capacity ratio between B and C: 4.0
  • Path capacity ratio between C and D: 0.75
  • One-way propagation delay:
    • Between S1 and R1: 100 ms
    • Between S2 and R2: 40 ms
    • Between S3 and R3: 40 ms
• Application-related:
  • Media source:
    • Media type: Video 🎀
      • Media direction: Forward
      • Number of media sources: Three (3)
      • Media timeline:
        • Start time: 0 s
        • End time: 299 s
    • Media type: Audio 🎶
      • Media direction: Forward
      • Number of media sources: Three (3)
      • Media timeline:
        • Start time: 0 s
        • End time: 299 s
  • Competing traffic:
    • Number of sources: Zero (0)

7. Wireless Access Links 📶

bestie, the wireless stuff (both cellular and Wi-Fi networks) has its own set of test cases. Those are defined in [RFC8869], so go check there 💖

8. Security Considerations 🔐

this section is serious, so read it carefully 🚨

The security considerations in Section 6 of [RFC8868] and the relevant congestion control algorithms apply. The principles for congestion control are described in [RFC2914], and in particular any new method must implement safeguards to avoid congestion collapse of the Internet. Congestion collapse of the Internet is a disaster, so nobody gets to skip this 💀

The evaluation of the test cases is intended to be run in a controlled lab environment. So the applications, simulators, and network nodes ought to be well-behaved and should not impact the desired results. And moreover, proper measures must be taken to avoid leaking nonresponsive traffic from unproven congestion avoidance techniques onto the open Internet. Keep your experiments in the lab, bestie 🙈

9. IANA Considerations 🌈

This document has no IANA actions. Nothing to register here, bestie ✨

10. References 📚

10.1. Normative References

• [RFC3550] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", STD 64, RFC 3550, DOI 10.17487/RFC3550, July 2003, <https://www.rfc-editor.org/info/rfc3550>.
• [RFC3551] Schulzrinne, H. and S. Casner, "RTP Profile for Audio and Video Conferences with Minimal Control", STD 65, RFC 3551, DOI 10.17487/RFC3551, July 2003, <https://www.rfc-editor.org/info/rfc3551>.
• [RFC3611] Friedman, T., Ed., Caceres, R., Ed., and A. Clark, Ed., "RTP Control Protocol Extended Reports (RTCP XR)", RFC 3611, DOI 10.17487/RFC3611, November 2003, <https://www.rfc-editor.org/info/rfc3611>.
• [RFC4585] Ott, J., Wenger, S., Sato, N., Burmeister, C., and J. Rey, "Extended RTP Profile for Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/AVPF)", RFC 4585, DOI 10.17487/RFC4585, July 2006, <https://www.rfc-editor.org/info/rfc4585>.
• [RFC5506] Johansson, I. and M. Westerlund, "Support for Reduced-Size Real-Time Transport Control Protocol (RTCP): Opportunities and Consequences", RFC 5506, DOI 10.17487/RFC5506, April 2009, <https://www.rfc-editor.org/info/rfc5506>.
• [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion Control", RFC 5681, DOI 10.17487/RFC5681, September 2009, <https://www.rfc-editor.org/info/rfc5681>.
• [RFC6679] Westerlund, M., Johansson, I., Perkins, C., O'Hanlon, P., and K. Carlberg, "Explicit Congestion Notification (ECN) for RTP over UDP", RFC 6679, DOI 10.17487/RFC6679, August 2012, <https://www.rfc-editor.org/info/rfc6679>.
• [RFC8593] Zhu, X., Mena, S., and Z. Sarker, "Video Traffic Models for RTP Congestion Control Evaluations", RFC 8593, DOI 10.17487/RFC8593, May 2019, <https://www.rfc-editor.org/info/rfc8593>.
• [RFC8836] Jesup, R. and Z. Sarker, Ed., "Congestion Control Requirements for Interactive Real-Time Media", RFC 8836, DOI 10.17487/RFC8836, January 2021, <https://www.rfc-editor.org/info/rfc8836>.
• [RFC8868] Singh, V., Ott, J., and S. Holmer, "Evaluating Congestion Control for Interactive Real-Time Media", RFC 8868, DOI 10.17487/RFC8868, January 2021, <https://www.rfc-editor.org/info/rfc8868>.
• [RFC8869] Sarker, Z., Zhu, X., and J. Fu, "Evaluation Test Cases for Interactive Real-Time Media over Wireless Networks", RFC 8869, DOI 10.17487/RFC8869, January 2021, <https://www.rfc-editor.org/info/rfc8869>.

10.2. Informative References

• [HEVC-seq] HEVC, "Test Sequences", <http://www.netlab.tkk.fi/~varun/test_sequences/>.
• [RFC2914] Floyd, S., "Congestion Control Principles", BCP 41, RFC 2914, DOI 10.17487/RFC2914, September 2000, <https://www.rfc-editor.org/info/rfc2914>.
• [RFC7567] Baker, F., Ed. and G. Fairhurst, Ed., "IETF Recommendations Regarding Active Queue Management", BCP 197, RFC 7567, DOI 10.17487/RFC7567, July 2015, <https://www.rfc-editor.org/info/rfc7567>.
• [RFC8033] Pan, R., Natarajan, P., Baker, F., and G. White, "Proportional Integral Controller Enhanced (PIE): A Lightweight Control Scheme to Address the Bufferbloat Problem", RFC 8033, DOI 10.17487/RFC8033, February 2017, <https://www.rfc-editor.org/info/rfc8033>.
• [RFC8290] Hoeiland-Joergensen, T., McKenney, P., Taht, D., Gettys, J., and E. Dumazet, "The Flow Queue CoDel Packet Scheduler and Active Queue Management Algorithm", RFC 8290, DOI 10.17487/RFC8290, January 2018, <https://www.rfc-editor.org/info/rfc8290>.
• [RFC8699] Islam, S., Welzl, M., and S. Gjessing, "Coupled Congestion Control for RTP Media", RFC 8699, DOI 10.17487/RFC8699, January 2020, <https://www.rfc-editor.org/info/rfc8699>.
• [xiph-seq] Xiph.org, "Video Test Media", <http://media.xiph.org/video/derf/>.

Acknowledgments 🫶

Much of this document is derived from previous work on congestion control at the IETF. Shoutout to everyone who came before 💜

The content and concepts within this document are a product of the discussion carried out within the Design Team. The Design Team really did the work, and we love them 👑

Authors' Addresses 💌

Zaheduzzaman Sarker
Ericsson AB
Torshamnsgatan 23
SE-164 83 Stockholm
Sweden

Phone: +46 10 717 37 43
Email: zaheduzzaman.sarker@ericsson.com

Varun Singh
CALLSTATS I/O Oy
Rauhankatu 11 C
FI-00100 Helsinki
Finland

Email: varun.singh@iki.fi
URI:   http://www.callstats.io/

Xiaoqing Zhu
Cisco Systems
12515 Research Blvd
Austin, TX 78759
United States of America

Email: xiaoqzhu@cisco.com

Michael A. Ramalho
AcousticComms Consulting
6310 Watercrest Way Unit 203
Lakewood Ranch, FL 34202-5211
United States of America

Phone: +1 732 832 9723
Email: mar42@cornell.edu
URI:   http://ramalho.webhop.info/

That's the whole RFC, bestie! 🌈✨ Go touch some grass and then go build something amazing 💖