RFC 9438
CUBIC for Fast and Long-Distance Networks
๐ CUBIC for Fast and Long-Distance Networks ๐
okay bestie, let's get into this ๐ RFC 9438 is the spec for CUBIC, a TCP congestion control algorithm, and it's giving main character energy โจ It was written by L. Xu (UNL), S. Ha (Colorado), I. Rhee (Bowery), V. Goel (Apple Inc.), and L. Eggert, Ed. (NetApp). It's dated August 2023, it's a Standards Track document ๐ฅ, it obsoletes RFC 8312, and it updates RFC 5681 ๐ฆ. The ISSN is 2070-1721 ๐ธ
Here's the tea ๐ต Internet Engineering Task Force (IETF) is the org behind it, and it's been published under the Standards Track.
Abstract
CUBIC is a standard TCP congestion control algorithm ๐ It swaps the linear congestion window increase for a cubic function so things get more scalable and stable on fast, long-distance networks ๐ CUBIC has already been adopted as the default TCP congestion control algorithm by the Linux, Windows, and Apple stacks ๐
This document updates the CUBIC spec to add algorithmic improvements based on those implementations and recent academic work ๐ Since CUBIC has been deployed so extensively, the spec now moves to the Standards Track and obsoletes RFC 8312 โจ It also updates RFC 5681 so CUBIC's occasionally more aggressive sending behavior is allowed ๐ซถ
Status of This Memo
This is an Internet Standards Track document ๐ฆ
This document is a product of the Internet Engineering Task Force (IETF) ๐ It represents the consensus of the IETF community, received public review, and was approved for publication by the Internet Engineering Steering Group (IESG) ๐ More info on Internet Standards is in Section 2 of RFC 7841 ๐
Info about the current status, any errata, and how to give feedback can be found at https://www.rfc-editor.org/info/rfc9438 ๐
Copyright Notice
Copyright (c) 2023 IETF Trust and the persons identified as the document authors. All rights reserved. ๐
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication ๐ Please review these carefully, since they describe your rights and restrictions. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License ๐
Table of Contents ๐ง
1. Introduction
2. Conventions
3. Design Principles of CUBIC
3.1. Principle 1 for the CUBIC Increase Function
3.2. Principle 2 for Reno-Friendliness
3.3. Principle 3 for RTT-Fairness
3.4. Principle 4 for the CUBIC Decrease Factor
4. CUBIC Congestion Control
4.1. Definitions
4.1.1. Constants of Interest
4.1.2. Variables of Interest
4.2. Window Increase Function
4.3. Reno-Friendly Region
4.4. Concave Region
4.5. Convex Region
4.6. Multiplicative Decrease
4.7. Fast Convergence
4.8. Timeout
4.9. Spurious Congestion Events
4.9.1. Spurious Timeouts
4.9.2. Spurious Fast Retransmits
4.10. Slow Start
5. Discussion
5.1. Fairness to Reno
5.2. Using Spare Capacity
5.3. Difficult Environments
5.4. Investigating a Range of Environments
5.5. Protection against Congestion Collapse
5.6. Fairness within the Alternative Congestion Control
Algorithm
5.7. Performance with Misbehaving Nodes and Outside Attackers
5.8. Behavior for Application-Limited Flows
5.9. Responses to Sudden or Transient Events
5.10. Incremental Deployment
6. Security Considerations
7. IANA Considerations
8. References
8.1. Normative References
8.2. Informative References
Appendix A. Evolution of CUBIC since the Original Paper
Appendix B. Proof of the Average CUBIC Window Size
Acknowledgments
Authors' Addresses1. Introduction ๐ธ
okay so CUBIC is now the default TCP congestion control algorithm in Linux, Windows, and Apple stacks, and it's been used and deployed globally ๐ Decade-long deployment in super different Internet scenarios has shown that CUBIC is safe for the global Internet and gives real benefits over classical Reno congestion control [RFC5681] ๐ That's why it's considered the most widely deployed standard for TCP congestion control ๐ CUBIC can also be used for other transport protocols like QUIC [RFC9000] and the Stream Control Transmission Protocol (SCTP) [RFC9260] as a default congestion controller ๐ฆ
The design of CUBIC was motivated by a well-documented problem with classical Reno TCP: low utilization over fast and long-distance networks [K03] [RFC3649] ๐ญ The issue is a slow increase of the congestion window (cwnd) after a congestion event on a network with a large bandwidth-delay product (BDP) ๐ข [HLRX07] notes this shows up even with congestion window sizes of several hundred packets ๐ It's equally a problem for every Reno-style standard and variant, including TCP-Reno [RFC5681], TCP-NewReno [RFC6582] [RFC6675], SCTP [RFC9260], TCP Friendly Rate Control (TFRC) [RFC5348], and QUIC congestion control [RFC9002], since they all use the same linear increase function for window growth ๐ In this document, all Reno-style standards and their variants are collectively called "Reno" ๐ซถ
CUBIC, originally proposed in [HRX08], is a modification to classical Reno's congestion control algorithm that fixes this ๐ Specifically, CUBIC uses a cubic function instead of Reno's linear window increase function to improve scalability and stability under fast and long-distance networks โจ
This document updates the CUBIC spec to include algorithmic improvements based on the Linux, Windows, and Apple implementations and recent academic work ๐ Since deployment has been so extensive, it also moves the spec to the Standards Track and obsoletes [RFC8312] ๐ฆ That requires updating Section 3 of [RFC5681], which limits how aggressive Reno TCP implementations can be ๐ค Since CUBIC is sometimes more aggressive than the algorithms in [RFC5681], this document updates the first paragraph of Section 3 of [RFC5681] and replaces it with a normative reference to guideline (1) in Section 3 of [RFC5033], which allows for CUBIC's behavior as defined here ๐
Specifically, CUBIC may increase the congestion window more aggressively than Reno during the congestion avoidance phase ๐ฅ According to [RFC5681], during congestion avoidance the sender MUST NOT increment cwnd by more than Sender Maximum Segment Size (SMSS) bytes once per round-trip time (RTT) ๐ โโ๏ธ CUBIC, on the other hand, may increase cwnd much more aggressively ๐ CUBIC also recommends the HyStart++ algorithm [RFC9406] for slow start ๐ That lets cwnd increase by more than SMSS bytes for incoming acknowledgments during slow start, which the standard behavior in [RFC5681] does not allow ๐
Binary Increase Congestion Control (BIC-TCP) [XHR04] was the predecessor of CUBIC ๐ Linux selected it as the default TCP congestion control algorithm in 2005, and the wider Internet community used it for several years ๐ง
CUBIC uses a window increase function similar to BIC-TCP's, but it's designed to be less aggressive and fairer to Reno in bandwidth usage ๐ฆ It keeps BIC-TCP's strengths, like stability, window scalability, and RTT-fairness ๐
[RFC5033] documents the IETF's best current practices for specifying new congestion control algorithms, especially ones that differ from the general principles in [RFC2914] ๐ It explains what kind of evaluation the IETF expects to understand whether a new algorithm is suitable, and the process for getting a spec approved for widespread deployment across the global Internet ๐
There are areas where CUBIC differs from congestion control algorithms already published in Standards Track RFCs, and those changes are specified in this document ๐ It isn't obvious that these changes go beyond the general principles in [RFC2914], so the process in [RFC5033] may not apply ๐ค
Also, CUBIC spread across the Internet mainly because it was directly adopted by most popular operating systems, and that didn't follow the practices documented in [RFC5033] ๐ However, because of the Internet-scale deployment experience over a long time, the IETF decided CUBIC could be published as a Standards Track spec โจ That decision doesn't change the general guidance in [RFC2914] ๐
The following sections ๐ท
- briefly explain the design principles of CUBIC ๐
- provide the exact specification of CUBIC ๐
- discuss the safety features of CUBIC, following the guidelines in [RFC5033] ๐ก๏ธ
2. Conventions ๐ซถ
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here ๐ (These are the only words that get the strict BCP 14 treatment, so if you see them in all caps, pay attention ๐)
3. Design Principles of CUBIC ๐ง
okay so CUBIC is designed around these principles โจ
Principle 1: For better network utilization and stability, CUBIC uses both the concave and convex profiles of a cubic function to increase the congestion window size ๐ It doesn't just use a convex function ๐ฆ
Principle 2: To be Reno-friendly, CUBIC is designed to behave like Reno in networks with short RTTs and small bandwidth, where Reno already performs well ๐ซถ
Principle 3: For RTT-fairness, CUBIC is designed to achieve linear bandwidth sharing among flows with different RTTs ๐
Principle 4: CUBIC sets its multiplicative window decrease factor so it balances scalability and convergence speed ๐ฅ
(Part 1 of 8. More parts coming, bestie, so stay tuned ๐๐)
3.1. Principle 1 for the CUBIC Increase Function
Okay so, CUBIC [HRX08] uses a cubic window increase function ๐ For better network utilization and stability, it measures growth by the elapsed time since the last congestion event. Most Reno alternatives grow their windows with convex functions, but CUBIC is extra and uses both the concave and convex sides of a cubic curve ๐
After a window reduction triggered by a congestion event, CUBIC remembers the congestion window size it had when the event arrived and then does a multiplicative decrease of the window. The congestion event can be detected by duplicate acknowledgments (ACKs), Explicit Congestion Notification-Echo (ECN-Echo (ECE)) ACKs [RFC3168], RACK-TLP for TCP [RFC8985], or QUIC loss detection [RFC9002] โจ
When CUBIC enters congestion avoidance, it starts growing the window with the concave profile of the cubic function. The cubic function is set to have its plateau at the remembered window size, so the concave growth keeps going until it gets there. After that, the curve flips into the convex profile and convex growth begins ๐ข
This concave-then-convex style is what makes it so stable while still keeping the network busy [CEHRX09] ๐ The window basically stays put near a plateau around the remembered window size from the last congestion event, which is where network utilization is considered highest. In steady state, most window samples land close to that remembered size, so utilization stays high and things stay stable ๐ฆ
Note that congestion control algorithms that only use convex functions to grow the window have their biggest increments right around the remembered window size of the last congestion event. That means lots of packet bursts around the network's saturation point, which can cause frequent global loss synchronizations ๐ญ
3.2. Principle 2 for Reno-Friendliness
CUBIC promotes per-flow fairness to Reno ๐ค Reno actually does fine on paths with small BDPs. It only runs into trouble when it tries to push bandwidth utilization up on paths with large BDPs ๐
A congestion control algorithm that wants to be friendly to Reno on a per-flow basis has to grow its window less aggressively in small-BDP networks than in large-BDP networks.
CUBIC's aggressiveness mainly depends on the maximum window size before a window reduction, and that maximum is smaller in small-BDP networks than in large-BDP networks. So CUBIC grows its window less aggressively in small-BDP networks than in large-BDP networks ๐ธ
Here's the bonus part: whenever the cubic function would grow the window less aggressively than Reno, CUBIC just follows Reno's window size instead. That way CUBIC gets at least the same throughput as Reno in small-BDP networks ๐ The region where CUBIC acts like Reno is called the "Reno-friendly region" ๐ง
3.3. Principle 3 for RTT-Fairness
Two CUBIC flows with different RTTs have a throughput ratio that's linearly proportional to the inverse of their RTT ratio. A flow's throughput is roughly its congestion window divided by its RTT, so this is literally the math ๐งฎโจ
More specifically, CUBIC keeps a window increase rate that doesn't depend on RTT outside the Reno-friendly region. That means flows with different RTTs end up with similar congestion window sizes at steady state when they're operating outside the Reno-friendly region ๐
This linear throughput ratio is kind of like Reno's under an asynchronous loss model, where flows with different RTTs have the same packet loss rate but get their loss events at different times ๐ฐ๏ธ Under a synchronous loss model, though, where flows with different RTTs get loss events at the same time but have different packet loss rates, Reno's throughput ratio becomes quadratically proportional to the inverse of the RTT ratio [XHR04] ๐ฌ
CUBIC always keeps a linear throughput ratio no matter what the loss environment looks like, which is a real upgrade over Reno ๐ฅ Nobody has agreed on the single best throughput ratio for flows with different RTTs. Still, over wired Internet paths, a linear throughput ratio seems more reasonable than equal throughputs (the same throughput for flows with different RTTs) or a higher-order ratio, like the quadratic throughput ratio Reno gets in synchronous loss environments ๐
3.4. Principle 4 for the CUBIC Decrease Factor
To balance scalability with convergence speed, CUBIC sets its multiplicative window decrease factor to 0.7, while Reno uses 0.5 ๐
This boosts CUBIC's scalability, but there's a side effect: slower convergence, especially when statistical multiplexing is low. The design follows an observation that HighSpeed TCP (HSTCP) [RFC3649] and other approaches (e.g., [GV02]) also made: the current Internet is getting more asynchronous, with less frequent loss synchronizations under high statistical multiplexing ๐
In environments like that, even strict Multiplicative-Increase Multiplicative-Decrease (MIMD) can converge ๐ฆ CUBIC flows with the same RTT always converge to the same throughput regardless of statistical multiplexing, which gets us intra-algorithm fairness ๐ซถ With sufficient statistical multiplexing, CUBIC's convergence speed is reasonable too โจ
4. CUBIC Congestion Control
This section walks through how the congestion window gets updated during each of the different stages of the CUBIC congestion controller ๐
4.1. Definitions
The unit of all window sizes in this document is segments of the SMSS, and the unit of all times is seconds โฑ๏ธ Implementations can use bytes to express window sizes instead. That would mean factoring in the SMSS wherever it's needed, and swapping segments_acked (Figure 4) for the number of acknowledged bytes ๐
4.1.1. Constants of Interest
ฮฒ_cubic: CUBIC multiplicative decrease factor, as described in Section 4.6.ฮฑ_cubic: CUBIC additive increase factor used in the Reno-friendly region, as described in Section 4.3.C: Constant that determines how aggressive CUBIC is when it's competing with other congestion control algorithms in high-BDP networks. Please see Section 5 for more explanation on how it's set. The unit forCis:
segment
โโโโโโโ
3
second4.1.2. Variables of Interest
This section defines the variables you need to implement CUBIC ๐ง
RTT: Smoothed round-trip time in seconds, calculated as described in [RFC6298].cwnd: Current congestion window in segments.ssthresh: Current slow start threshold in segments.cwnd_prior: Size ofcwndin segments at the time of settingssthreshmost recently, either upon exiting the first slow start or just beforecwndwas reduced in the last congestion event.W_max: Size ofcwndin segments just beforecwndwas reduced in the last congestion event when fast convergence is disabled (same ascwnd_prioron a congestion event). However, if fast convergence is enabled,W_maxmay be further reduced based on the current saturation point ๐K: The time period in seconds it takes to increase the congestion window size at the beginning of the current congestion avoidance stage toW_maxโณt_current: Current time of the system in seconds.t_epoch: The time in seconds at which the current congestion avoidance stage started.cwnd_epoch: Thecwndat the beginning of the current congestion avoidance stage, i.e., at timet_epoch.W_cubic(t): The congestion window in segments at timetin seconds based on the cubic increase function, as described in Section 4.2 ๐target: Target value of the congestion window in segments after the next RTT -- that is,W_cubic(t + RTT), as described in Section 4.2 ๐ฏW_est: An estimate for the congestion window in segments in the Reno-friendly region -- that is, an estimate for the congestion window of Reno ๐คsegments_acked: Number of SMSS-sized segments acked when a "new ACK" is received, i.e., an ACK that cumulatively acknowledges the delivery of previously unacknowledged data. This number will be a decimal value when a new ACK acknowledges an amount of data that is not SMSS-sized. Specifically, it can be less than 1 when a new ACK acknowledges a segment smaller than the SMSS ๐ฅน
4.2. Window Increase Function โจ
Okay bestie, CUBIC keeps the ACK clocking that Reno has ๐ by only growing the congestion window when a brand new ACK shows up. It doesn't touch the TCP Fast Recovery and Fast Retransmit algorithms at all [RFC6582] [RFC6675] ๐ซถ
During congestion avoidance, once a congestion event gets detected (see Section 3.1), CUBIC switches to a window increase function that is totally different from Reno's ๐ฆ
CUBIC's window increase function looks like this:
3
W (t) = C * (t - K) + W
cubic max
Figure 1Here t is the elapsed time in seconds since the start of the current congestion avoidance stage, aka:
t = t - t
current epochand t_epoch is the moment that current congestion avoidance stage kicks off. K is the time the function needs to grow the window from its size at the start of the stage back up to W_max, assuming no more congestion events show up. Main character energy, no drama ๐ K comes from this equation:
โโโโโโโโโโโโโโโโโโ
3 โW - cwnd
โฒ โ max epoch
K = โฒ โโโโโโโโโโโโโโโโโ
โฒโ C
Figure 2where cwnd_epoch is the congestion window at the beginning of the current congestion avoidance stage.
When a new ACK comes in during congestion avoidance, CUBIC works out a target congestion window size for after the next RTT, using Figure 1. RTT is the smoothed round-trip time. The lower and upper bounds below keep CUBIC's congestion window increase rate non-decreasing and below the increase rate of slow start [SXEZ19]. It's giving careful and steady ๐ธ
โง
โชcwnd if W (t + RTT) < cwnd
โช cubic
โจ1.5 * cwnd if W (t + RTT) > 1.5 * cwnd
target = โช cubic
โชW (t + RTT) otherwise
โฉ cubicThe elapsed time t in Figure 1 MUST NOT include periods during which cwnd has not been updated due to application-limited behavior (see Section 5.8). So if the app wasn't sending anything, that time doesn't count ๐
Depending on the current congestion window size cwnd, CUBIC runs in one of three regions ๐
- The Reno-friendly region, which makes sure CUBIC gets at least the same throughput as Reno ๐
- The concave region, if CUBIC is not in the Reno-friendly region and cwnd is less than W_max. ๐
- The convex region, if CUBIC is not in the Reno-friendly region and cwnd is greater than W_max. ๐
To summarize: on each new ACK in congestion avoidance, CUBIC computes both W_cubic(t) and W_est (see Section 4.3), then picks the larger of the two ๐
The next sections break down exactly what CUBIC does in each region ๐ง
4.3. Reno-Friendly Region ๐ฆ
Reno performs really well in certain networks, like ones with short RTTs and small bandwidths (or small BDPs). In those networks, CUBIC stays in the Reno-friendly region so it gets at least the same throughput as Reno ๐
The Reno-friendly region comes from the analysis in [FHP00], which studies an AIMD algorithm with an additive factor of ฮฑ (segments per RTT) and a multiplicative factor of ฮฒ, written AIMD(ฮฑ, ฮฒ). p is the packet loss rate. The average congestion window size of AIMD(ฮฑ, ฮฒ) is figured out with Figure 3:
โโโโโโโโโโโโโโโโโ
โ ฮฑ * (1 + ฮฒ)
AVG_AIMD(ฮฑ, ฮฒ) = โฒ โโโโโโโโโโโโโโโโ
โฒโ2 * (1 - ฮฒ) * p
Figure 3Using that same analysis, to get an average window size similar to Reno (which uses AIMD(1, 0.5)), ฮฑ has to equal:
1 - ฮฒ
3 * โโโโโ
1 + ฮฒSo CUBIC uses Figure 4 to estimate the window size W_est in the Reno-friendly region, with:
1 - ฮฒ
cubic
ฮฑ = 3 * โโโโโโโโโโ
cubic 1 + ฮฒ
cubicThis gets approximately the same average window size as Reno in many cases ๐ฅฐ The model used to calculate ฮฑ_cubic isn't totally precise, but analysis and simulation in [AIMD-friendliness], plus over a decade of CUBIC running on the public Internet, show it produces acceptable levels of rate fairness between CUBIC and Reno flows. No significant drawbacks of the model have been reported. Still, more detailed analysis of this approach would be super beneficial ๐ฅ When a new ACK arrives in congestion avoidance (where cwnd could be greater than or less than W_max), CUBIC checks whether W_cubic(t) is less than W_est. If it is, CUBIC is in the Reno-friendly region and cwnd SHOULD be set to W_est on each new ACK reception.
W_est gets set equal to cwnd_epoch at the start of the congestion avoidance stage. After that, on every new ACK, W_est is updated using Figure 4. This equation uses segments_acked, and cwnd is measured in segments. An implementation that measures cwnd in bytes should adjust the equation to use the number of acknowledged bytes and the SMSS instead. This also works for connections with delayed ACKs enabled or disabled [RFC5681], since segments_acked depends on which segments a new ACK actually acknowledges ๐ง
segments_acked
W = W + ฮฑ * โโโโโโโโโโโโโโ
est est cubic cwnd
Figure 4Once W_est has grown to reach the cwnd from the moment ssthresh was most recently set (that is, W_est >= cwnd_prior), the sender SHOULD set ฮฑ_cubic to 1. That way it matches the same congestion window increment rate as Reno, which uses AIMD(1, 0.5) โจ
The next two sections assume CUBIC is not in the Reno-friendly region and uses the window increase function from Section 4.2. Even though cwnd grows the same way in both the concave and convex regions, they get their own sections so you can see the difference between them ๐ซง
4.4. Concave Region ๐
When a new ACK arrives in congestion avoidance, if CUBIC is not in the Reno-friendly region and cwnd is less than W_max, then CUBIC is in the concave region. In this region, cwnd MUST be incremented by
target - cwnd
โโโโโโโโโโโโโ
cwndfor each received new ACK, where target is calculated as described in Section 4.2. ๐ญ
4.5. Convex Region ๐ท
When a new ACK arrives in congestion avoidance, if CUBIC is not in the Reno-friendly region and cwnd is larger than or equal to W_max, then CUBIC is in the convex region.
The convex region tells you the network conditions might have changed since the last congestion event, which could mean more bandwidth is free after some flows left ๐ฆ The Internet is super asynchronous, so some amount of perturbation is always possible without causing a major change in available bandwidth.
Unless the cwnd is overridden by the AIMD window increase, CUBIC gets cautious in this region ๐ The convex profile aims to increase the window very slowly at first, when cwnd is around W_max, and then gradually speeds up. This region is also called the "maximum probing phase", since CUBIC is searching for a new W_max ๐ In this region, cwnd MUST be incremented by
target - cwnd
โโโโโโโโโโโโโ
cwndfor each received new ACK, where target is calculated as described in Section 4.2. slay, that's the whole convex region ๐๐
4.6. Multiplicative Decrease ๐ฅ
okay so when a congestion event gets caught by the mechanisms in Section 3.1, CUBIC updates W_max and drops cwnd and ssthresh right away, as described below ๐. For packet loss, the sender MUST reduce cwnd and ssthresh immediately once it enters loss recovery, same vibe as [RFC5681] (and [RFC6675]) ๐ซถ. Fun fact: other stuff like Proportional Rate Reduction [RFC6937] can be used to slow the sending rate down more gently during loss recovery ๐ธ. And the ฮฒ_cubic parameter SHOULD be set to 0.7 ๐. That's different from the multiplicative decrease factor used during fast recovery in [RFC5681] (and [RFC6675]), so don't mix them up bestie ๐.
In Figure 5, flight_size is the outstanding (unacknowledged) data in the network, as defined in [RFC5681] โจ. Now here's the tea. A rate-limited app with idle periods, or times when it can't send at the full rate cwnd allows, can easily see the amount of data it sends jump around from one RTT to the next ๐ญ. So when congestion hits, flight_size can be way smaller than cwnd, and the response would then drop cwnd to a much lower value than it needs to ๐. To dodge that suboptimal performance, you can use the mechanisms in [RFC7661] ๐ฆ. [RFC7661] explains how to manage and use cwnd and ssthresh during a rate-limited interval, and how to update them after congestion is detected โจ. And it's safe even when cwnd is bigger than the receive window, because it validates cwnd based on how much data the network acknowledged in an RTT, which implicitly accounts for the allowed receive window ๐.
Some CUBIC implementations currently use cwnd instead of flight_size when calculating a new ssthresh ๐. Implementations that use cwnd MUST use other measures to stop cwnd from growing when the bytes in flight are smaller than cwnd ๐ซ. That also effectively keeps cwnd from growing past the receive window ๐. These measures matter a LOT because they stop a CUBIC sender from using an arbitrarily high cwnd value when it calculates new ssthresh and cwnd values after congestion is detected ๐ค. Still, this might not be as robust as the mechanisms in [RFC7661] ๐ฅบ.
A QUIC sender that uses a cwnd value to calculate new cwnd and ssthresh values after detecting a congestion event is REQUIRED to apply similar mechanisms [RFC9002] ๐ฅ.
ssthresh = flight_size * ฮฒ new ssthresh
cubic
cwnd = cwnd save cwnd
prior
โงmax(ssthresh, 2) reduction on loss, cwnd >= 2 SMSS
cwnd = โจmax(ssthresh, 1) reduction on ECE, cwnd >= 1 SMSS
โฉ
ssthresh = max(ssthresh, 2) ssthresh >= 2 SMSS
Figure 5A side effect of setting ฮฒ_cubic to something bigger than 0.5 is that packet loss can happen across more than one RTT in certain cases ๐ซ . But it can work efficiently in other cases, like when HyStart++ [RFC9406] is used with CUBIC or when the sending rate is limited by the application ๐
. A more adaptive setting of ฮฒ_cubic could help keep packet loss to a single round, but validating those algorithms would take detailed analyses and large-scale evaluations ๐งช.
Now, CUBIC MUST keep reducing cwnd in response to congestion events signaled by ECN-Echo ACKs until it hits 1 SMSS ๐. If those ECN-Echo congestion signals keep coming, a sender with a cwnd of 1 SMSS MUST reduce its sending rate even further โญ. You can do that with a retransmission timer that uses exponential backoff, as described in [RFC3168] ๐.
4.7. Fast Convergence ๐ง
To speed up convergence, CUBIC uses a heuristic ๐. When a new flow joins the network, the existing flows need to give up some of their bandwidth so the new flow has room to grow, if the existing flows have been hogging all the bandwidth ๐. To speed up that bandwidth release, the following fast convergence mechanism SHOULD be implemented ๐.
With fast convergence, when a congestion event happens, W_max gets updated like this, before the window reduction from Section 4.6 ๐:
โง 1 + ฮฒ
โช cubic
โชcwnd * โโโโโโโโโโ if cwnd < W and fast convergence enabled,
W = โจ 2 max
max โช further reduce W
โช max
โฉcwnd otherwise, remember cwnd before reductionDuring a congestion event, if the current cwnd is less than W_max, that means the saturation point this flow experiences is shrinking because the available bandwidth changed ๐ฆ. The flow can then release even more bandwidth by reducing W_max further ๐ซถ. That effectively stretches out how long this flow takes to grow its congestion window, because the lower W_max makes the flow plateau earlier โณ. That gives the new flow more time to catch up to its congestion window size ๐ธ.
Fast convergence is designed for network environments with multiple CUBIC flows โจ. In an environment with only a single CUBIC flow and no other traffic, fast convergence SHOULD be disabled ๐ โโ๏ธ.
4.8. Timeout โฐ
When a timeout happens, CUBIC follows Reno to reduce cwnd [RFC5681], but it sets ssthresh using ฮฒ_cubic (same as in Section 4.6) in a way that's different from Reno TCP [RFC5681] ๐
.
During the first congestion avoidance stage after a timeout, CUBIC grows its congestion window using Figure 1 ๐ฆ. Here t is the elapsed time since the start of the current congestion avoidance stage, K is set to 0, and W_max is set to the congestion window size at the start of that stage ๐ท. For the Reno-friendly region, W_est SHOULD be set to the congestion window size at the start of the current congestion avoidance stage ๐ซถ.
4.9. Spurious Congestion Events ๐ป
Sometimes CUBIC shrinks its congestion window because it thinks there was packet loss, through duplicate ACKs or timeouts ๐ฌ. But the "missing" ACK could arrive after the window reduction and the matching packet retransmission ๐คฏ. Packet reordering could be the reason, for example ๐. If there's a LOT of reordering, you get multiple congestion window reduction events, where spurious losses get misread as congestion signals, and CUBIC's performance tanks big time ๐ญ.
For TCP, there are two kinds of spurious events: spurious timeouts and spurious fast retransmits ๐. With QUIC there are no spurious timeouts, because loss only gets detected after an ACK comes in โจ.
4.9.1. Spurious Timeouts โฑ๏ธ
An implementation MAY detect spurious timeouts using the mechanisms in Forward RTO-Recovery [RFC5682] ๐. Experimental alternatives include the Eifel detection algorithm [RFC3522] ๐งช. When a spurious timeout is detected, a TCP implementation MAY follow the response algorithm in [RFC4015] to restore the congestion control state and adjust the retransmission timer, so it avoids more spurious timeouts ๐ฉน.
4.9.2. Spurious Fast Retransmits ๐โโ๏ธ
When an ACK comes in, a TCP implementation MAY detect spurious fast retransmits using either TCP Timestamps or D-SACK [RFC2883] ๐ต๏ธโโ๏ธ. Like we said above, experimental alternatives include the Eifel detection algorithm [RFC3522], which uses TCP Timestamps, and DSACK-based detection [RFC3708], which uses DSACK information ๐งช. A QUIC implementation can easily spot a spurious fast retransmit if a QUIC packet gets acknowledged after it was marked as lost and its original data was already retransmitted in a new QUIC packet ๐.
This section specifies a simple response algorithm for when acknowledgments detect a spurious fast retransmit ๐ฏ. Implementations would need to carefully evaluate how using this algorithm affects different environments, especially ones with a sudden change in available capacity (for example, from variable radio capacity, a routing change, or a mobility event) ๐.
When acknowledgments detect packet loss, a CUBIC implementation MAY save the current value of these variables before the congestion window is reduced ๐พ:
undo_cwnd = cwnd
undo_cwnd = cwnd
prior prior
undo_ssthresh = ssthresh
undo_W = W
max max
undo_K = K
undo_t = t
epoch epoch
undo_W = W
est estOnce the earlier packet loss is confirmed as spurious, CUBIC MAY restore the original values of those variables as shown below, if the current cwnd is lower than cwnd_prior ๐. Restoring the original values means CUBIC's performance looks similar to what it would be without the spurious losses ๐ฅน.
cwnd = undo_cwnd โซ
cwnd = undo_cwnd โฎ
prior prior โฎ
ssthresh = undo_ssthresh โฎ
W = undo_W โฎ
max max โฌif cwnd < cwnd
K = undo_K โฎ prior
t = undo_t โฎ
epoch epoch โฎ
W = undo_W โฎ
est est โญIn rare cases, when the detection happens long after a spurious fast retransmit and the current cwnd is already higher than cwnd_prior, CUBIC SHOULD keep using the current and most recent values of these variables ๐ซง.
4.10. Slow Start ๐ฃ
When cwnd is no more than ssthresh, CUBIC MUST employ a slow start algorithm ๐. In general, CUBIC SHOULD use the HyStart++ slow start algorithm [RFC9406], or it MAY use the Reno TCP slow start algorithm [RFC5681] in the rare cases where HyStart++ isn't suitable ๐. Experimental alternatives include hybrid slow start [HR11], a predecessor to HyStart++ that some CUBIC implementations have used as the default for the last decade, and limited slow start [RFC3742] ๐ฐ๏ธ. Whichever startup algorithm you use, work might be needed to make sure the end of slow start and the first multiplicative decrease of congestion avoidance work well together ๐ค.
When CUBIC uses HyStart++ [RFC9406], it may exit the first slow start without any packet loss, so W_max is undefined ๐คทโโ๏ธ. In this special case, CUBIC sets cwnd_prior = cwnd and switches to congestion avoidance โจ. Then it grows its congestion window using Figure 1, where t is the elapsed time since the start of the current congestion avoidance stage, K is set to 0, and W_max is set to the congestion window size at the start of that stage ๐.
5. Discussion โจ
Okay bestie, this part is all about the safety side of CUBIC ๐ก๏ธ. We're following the guidelines in [RFC5033] so it plays nice with everyone else on the internet ๐ซถ
Here's the fun bit: with a deterministic loss model where the number of packets between two successive packet losses is always 1/p, CUBIC always operates with the concave window profile ๐ธ That makes the performance analysis SO much easier to do. The average window size of CUBIC (see Appendix B) comes out of this formula:
โโโโโโโโโโโโโโโโโโ 4 โโโโโโ
โC * (3 + ฮฒ ) โฒ โ 3
4 โ cubic โฒโRTT
AVG_W = โฒ โโโโโโโโโโโโโโโโโ * โโโโโโโโ
cubic โฒ โ4 * (1 - ฮฒ ) 4 โโโโ
โฒโ cubic โฒ โ 3
โฒโp
Figure 6With ฮฒ__cubic_ set to 0.7, the above formula reduces to:
4 โโโโโโ
โโโโโโโโโ โฒ โ 3
4 โC * 3.7 โฒโRTT
AVG_W = โฒ โโโโโโโโ * โโโโโโโโ
cubic โฒโ 1.2 4 โโโโ
โฒ โ 3
โฒโp
Figure 7Next up, the following subsection figures out the value of C using Figure 7 ๐งฎ
5.1. Fairness to Reno
In environments where Reno can make reasonable use of the available bandwidth, CUBIC doesn't really change that vibe ๐
Reno performs well in these two types of networks:
- networks with a small bandwidth-delay product (BDP)
- networks with short RTTs, but not necessarily a small BDP
CUBIC is designed to act super similar to Reno in both of those situations ๐ The two tables below show the average window sizes of Reno TCP, HSTCP, and CUBIC TCP. The Reno TCP and HSTCP numbers come from [RFC3649]. The CUBIC numbers are calculated using Figure 7 and the CUBIC Reno-friendly region, for three different values of C.
+=============+=======+========+================+=========+========+
| Loss Rate P | Reno | HSTCP | CUBIC (C=0.04) | CUBIC | CUBIC |
| | | | | (C=0.4) | (C=4) |
+=============+=======+========+================+=========+========+
| 1.0e-02 | 12 | 12 | 12 | 12 | 12 |
+-------------+-------+--------+----------------+---------+--------+
| 1.0e-03 | 38 | 38 | 38 | 38 | 59 |
+-------------+-------+--------+----------------+---------+--------+
| 1.0e-04 | 120 | 263 | 120 | 187 | 333 |
+-------------+-------+--------+----------------+---------+--------+
| 1.0e-05 | 379 | 1795 | 593 | 1054 | 1874 |
+-------------+-------+--------+----------------+---------+--------+
| 1.0e-06 | 1200 | 12280 | 3332 | 5926 | 10538 |
+-------------+-------+--------+----------------+---------+--------+
| 1.0e-07 | 3795 | 83981 | 18740 | 33325 | 59261 |
+-------------+-------+--------+----------------+---------+--------+
| 1.0e-08 | 12000 | 574356 | 105383 | 187400 | 333250 |
+-------------+-------+--------+----------------+---------+--------+
Table 1: Reno TCP, HSTCP, and CUBIC with RTT = 0.1 SecondsTable 1 shows the response function of Reno TCP, HSTCP, and CUBIC in networks with RTT = 0.1 seconds ๐ The average window size is in SMSS-sized segments.
+=============+=======+========+================+=========+=======+
| Loss Rate P | Reno | HSTCP | CUBIC (C=0.04) | CUBIC | CUBIC |
| | | | | (C=0.4) | (C=4) |
+=============+=======+========+================+=========+=======+
| 1.0e-02 | 12 | 12 | 12 | 12 | 12 |
+-------------+-------+--------+----------------+---------+-------+
| 1.0e-03 | 38 | 38 | 38 | 38 | 38 |
+-------------+-------+--------+----------------+---------+-------+
| 1.0e-04 | 120 | 263 | 120 | 120 | 120 |
+-------------+-------+--------+----------------+---------+-------+
| 1.0e-05 | 379 | 1795 | 379 | 379 | 379 |
+-------------+-------+--------+----------------+---------+-------+
| 1.0e-06 | 1200 | 12280 | 1200 | 1200 | 1874 |
+-------------+-------+--------+----------------+---------+-------+
| 1.0e-07 | 3795 | 83981 | 3795 | 5926 | 10538 |
+-------------+-------+--------+----------------+---------+-------+
| 1.0e-08 | 12000 | 574356 | 18740 | 33325 | 59261 |
+-------------+-------+--------+----------------+---------+-------+
Table 2: Reno TCP, HSTCP, and CUBIC with RTT = 0.01 SecondsTable 2 shows the response function of Reno TCP, HSTCP, and CUBIC in networks with RTT = 0.01 seconds ๐ Again, the average window size is in SMSS-sized segments.
Both tables show that CUBIC with any of these three C values is more friendly to Reno TCP than HSTCP is, especially in networks with a short RTT, where Reno TCP already does pretty well ๐ฅน For example, in a network with RTT = 0.01 seconds and p=10^-6, Reno TCP has an average window of 1200 packets. If the packet size is 1500 bytes, then Reno TCP can achieve an average rate of 1.44 Gbps ๐คฏ In this case, CUBIC with C=0.04 or C=0.4 achieves exactly the same rate as Reno TCP, whereas HSTCP is about ten times more aggressive than Reno TCP. Not it, HSTCP ๐
C is basically the aggressiveness dial for CUBIC when it's competing with other congestion control algorithms for bandwidth ๐๏ธ CUBIC is more friendly to Reno TCP if the value of C is lower. However, it is NOT RECOMMENDED to set C to a very low value like 0.04, because CUBIC with a low C can't efficiently use the bandwidth in fast and long-distance networks ๐ฉ Based on these observations and a ton of real deployment experience, C=0.4 seems to give a good balance between Reno-friendliness and how aggressively the window grows ๐ So C SHOULD be set to 0.4. With C set to 0.4, Figure 7 gets reduced to:
4 โโโโโโ
โฒ โ 3
โฒโRTT
AVG_W = 1.054 * โโโโโโโโ
cubic 4 โโโโ
โฒ โ 3
โฒโp
Figure 8Figure 8 is then used in the next subsection to show how scalable CUBIC is ๐
5.2. Using Spare Capacity
CUBIC uses a more aggressive window increase function than Reno for fast and long-distance networks ๐๏ธโจ
Table 3 shows that to hit a 10 Gbps rate, Reno TCP needs a packet loss rate of 2.0e-10, while CUBIC TCP only needs a packet loss rate of 2.9e-8. Iconic ๐
+===================+===========+=========+=========+=========+
| Throughput (Mbps) | Average W | Reno P | HSTCP P | CUBIC P |
+===================+===========+=========+=========+=========+
| 1 | 8.3 | 2.0e-2 | 2.0e-2 | 2.0e-2 |
+-------------------+-----------+---------+---------+---------+
| 10 | 83.3 | 2.0e-4 | 3.9e-4 | 2.9e-4 |
+-------------------+-----------+---------+---------+---------+
| 100 | 833.3 | 2.0e-6 | 2.5e-5 | 1.4e-5 |
+-------------------+-----------+---------+---------+---------+
| 1000 | 8333.3 | 2.0e-8 | 1.5e-6 | 6.3e-7 |
+-------------------+-----------+---------+---------+---------+
| 10000 | 83333.3 | 2.0e-10 | 1.0e-7 | 2.9e-8 |
+-------------------+-----------+---------+---------+---------+
Table 3: Required Packet Loss Rate for Reno TCP, HSTCP, and
CUBIC to Achieve a Certain ThroughputTable 3 describes the required packet loss rate for Reno TCP, HSTCP, and CUBIC to hit a certain throughput, with 1500-byte packets and an RTT of 0.1 seconds ๐
The test results in [HLRX07] show that, in typical cases with some background traffic, CUBIC uses the spare bandwidth that Reno TCP flows leave unused on the same bottleneck link, without taking away much bandwidth from those existing flows ๐ฆ
5.3. Difficult Environments
CUBIC is designed to fix the poor performance of Reno in fast and long-distance networks ๐ ๏ธ๐
5.4. Investigating a Range of Environments
CUBIC has been studied SO extensively, using simulations, testbed emulations, Internet experiments, and Internet measurements, across a wide range of network environments [HLRX07] [H16] [CEHRX09] [HR11] [BSCLU13] [LBEWK16] ๐ค They've convincingly shown that CUBIC gives real benefits over classic Reno congestion control [RFC5681]. Main character energy ๐
Same as Reno, CUBIC is a loss-based congestion control algorithm. Because CUBIC is designed to be more aggressive than Reno in fast and long-distance networks (thanks to a faster window increase function and a bigger multiplicative decrease factor), it can fill large drop-tail buffers more quickly than Reno does, which raises the risk of a standing queue [RFC8511] ๐ฌ In that case, proper queue sizing and management [RFC7567] could help reduce that risk to some extent and cut down the packet queuing delay ๐งน Also, in large-BDP networks after a congestion event, CUBIC recovers quickly to the highest link utilization point, thanks to its cubic window increase function ๐ This means link utilization is less sensitive to an active queue management (AQM) target that is lower than the amplitude of the whole sawtooth.
Similar to Reno, CUBIC's performance as a loss-based algorithm suffers in networks where packet loss isn't a good indication of bandwidth utilization, such as wireless or mobile networks [LIU16] ๐ถ๐
5.5. Protection against Congestion Collapse
When it comes to the potential for causing congestion collapse, CUBIC acts like Reno, since CUBIC only modifies the window adjustment algorithm of Reno ๐ So it doesn't change the ACK clocking and timeout behaviors of Reno either.
CUBIC also satisfies the "full backoff" requirement described in [RFC5033] ๐ After it drops the sending rate to one packet per RTT in response to congestion events detected by ECN-Echo ACKs, CUBIC then exponentially increases the transmission timer for each packet retransmission while congestion persists โณ
5.6. Fairness within the Alternative Congestion Control Algorithm
CUBIC makes sure competing CUBIC flows with the same RTT on the same bottleneck links converge to an equal throughput ๐ค When competing flows have different RTT values, their throughput ratio is linearly proportional to the inverse of their RTT ratios. This is true, and it's independent of the level of statistical multiplexing on the link. The convergence time depends on the network environment (e.g., bandwidth, RTT) and the level of statistical multiplexing, as mentioned in Section 3.4 ๐ฆโจ
5.7. Performance with Misbehaving Nodes and Outside Attackers ๐ต๏ธ
okay so CUBIC doesn't bring in any new entities or signals ๐ โโ๏ธ, which means its vulnerability to misbehaving nodes or attackers is exactly the same as Reno's. no new door, no new drama ๐
5.8. Behavior for Application-Limited Flows ๐ข
a flow is application limited if it's currently sending less than what the congestion window allows ๐ซ . this can happen when the sender's app is the thing holding it back, or when the receiver's app is (through the receiver's advertised window), so it ends up sending less data than the sender's congestion window would permit.
CUBIC does not increase its congestion window if a flow is application limited ๐. per Section 4.2, it is required that t in Figure 1 not include application-limited periods, like idle periods. otherwise, W_cubic(t) could be super high after restarting from those periods ๐ญ
5.9. Responses to Sudden or Transient Events โก
if there's a sudden increase in capacity, like variable radio capacity, a routing change, or a mobility event, CUBIC is designed to grab the newly available capacity faster than Reno โจ
on the flip side, if there's a sudden decrease in capacity, CUBIC reduces more slowly than Reno ๐. this stays true whether or not CUBIC is in Reno-friendly mode, and whether or not fast convergence is enabled. no cap ๐งข
5.10. Incremental Deployment ๐
CUBIC only needs changes to congestion control at the sender, and receivers don't need any changes at all ๐ซถ. so a CUBIC sender works correctly with Reno receivers. plus, CUBIC doesn't need any changes to routers and doesn't need any help from routers either. easy peasy ๐
6. Security Considerations ๐
okay bestie, this one is serious, so read it carefully ๐ค
CUBIC makes no changes to the underlying security of a transport protocol, and it inherits the general security concerns described in [RFC5681]. specifically, changing the window computation on the sender may allow an attacker, through dropping or injecting ACKs (as described in [RFC5681]), to either force the CUBIC implementation to reduce its bandwidth or convince it that there is no congestion when congestion does exist. it could also use the CUBIC implementation as an attack vector against other hosts ๐. these attacks are not new to CUBIC. they are inherently part of any transport protocol like TCP ๐ฅฒ.
7. IANA Considerations ๐
this document does not require any IANA actions ๐ธ.
8. References ๐
8.1. Normative References
- [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, <https://www.rfc-editor.org/info/rfc2119>.
- [RFC2883] Floyd, S., Mahdavi, J., Mathis, M., and M. Podolsky, "An Extension to the Selective Acknowledgement (SACK) Option for TCP", RFC 2883, DOI 10.17487/RFC2883, July 2000, <https://www.rfc-editor.org/info/rfc2883>.
- [RFC2914] Floyd, S., "Congestion Control Principles", BCP 41, RFC 2914, DOI 10.17487/RFC2914, September 2000, <https://www.rfc-editor.org/info/rfc2914>.
- [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition of Explicit Congestion Notification (ECN) to IP", RFC 3168, DOI 10.17487/RFC3168, September 2001, <https://www.rfc-editor.org/info/rfc3168>.
- [RFC4015] Ludwig, R. and A. Gurtov, "The Eifel Response Algorithm for TCP", RFC 4015, DOI 10.17487/RFC4015, February 2005, <https://www.rfc-editor.org/info/rfc4015>.
- [RFC5033] Floyd, S. and M. Allman, "Specifying New Congestion Control Algorithms", BCP 133, RFC 5033, DOI 10.17487/RFC5033, August 2007, <https://www.rfc-editor.org/info/rfc5033>.
- [RFC5348] Floyd, S., Handley, M., Padhye, J., and J. Widmer, "TCP Friendly Rate Control (TFRC): Protocol Specification", RFC 5348, DOI 10.17487/RFC5348, September 2008, <https://www.rfc-editor.org/info/rfc5348>.
- [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion Control", RFC 5681, DOI 10.17487/RFC5681, September 2009, <https://www.rfc-editor.org/info/rfc5681>.
- [RFC5682] Sarolahti, P., Kojo, M., Yamamoto, K., and M. Hata, "Forward RTO-Recovery (F-RTO): An Algorithm for Detecting Spurious Retransmission Timeouts with TCP", RFC 5682, DOI 10.17487/RFC5682, September 2009, <https://www.rfc-editor.org/info/rfc5682>.
- [RFC6298] Paxson, V., Allman, M., Chu, J., and M. Sargent, "Computing TCP's Retransmission Timer", RFC 6298, DOI 10.17487/RFC6298, June 2011, <https://www.rfc-editor.org/info/rfc6298>.
- [RFC6582] Henderson, T., Floyd, S., Gurtov, A., and Y. Nishida, "The NewReno Modification to TCP's Fast Recovery Algorithm", RFC 6582, DOI 10.17487/RFC6582, April 2012, <https://www.rfc-editor.org/info/rfc6582>.
- [RFC6675] Blanton, E., Allman, M., Wang, L., Jarvinen, I., Kojo, M., and Y. Nishida, "A Conservative Loss Recovery Algorithm Based on Selective Acknowledgment (SACK) for TCP", RFC 6675, DOI 10.17487/RFC6675, August 2012, <https://www.rfc-editor.org/info/rfc6675>.
- [RFC7567] Baker, F., Ed. and G. Fairhurst, Ed., "IETF Recommendations Regarding Active Queue Management", BCP 197, RFC 7567, DOI 10.17487/RFC7567, July 2015, <https://www.rfc-editor.org/info/rfc7567>.
- [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, <https://www.rfc-editor.org/info/rfc8174>.
- [RFC8985] Cheng, Y., Cardwell, N., Dukkipati, N., and P. Jha, "The RACK-TLP Loss Detection Algorithm for TCP", RFC 8985, DOI 10.17487/RFC8985, February 2021, <https://www.rfc-editor.org/info/rfc8985>.
- [RFC9002] Iyengar, J., Ed. and I. Swett, Ed., "QUIC Loss Detection and Congestion Control", RFC 9002, DOI 10.17487/RFC9002, May 2021, <https://www.rfc-editor.org/info/rfc9002>.
- [RFC9406] Balasubramanian, P., Huang, Y., and M. Olson, "HyStart++: Modified Slow Start for TCP", RFC 9406, DOI 10.17487/RFC9406, May 2023, <https://www.rfc-editor.org/info/rfc9406>.
8.2. Informative References โจ
okay bestie, these are the extra-credit reads ๐๐ the whole reading list is right here so you can go deeper whenever you're obsessed enough ๐ค
[AIMD-friendliness] Briscoe, B. and O. Albisser, "Friendliness between AIMD Algorithms", DOI 10.48550/arXiv.2305.10581, May 2023, https://arxiv.org/abs/2305.10581.
[BSCLU13] Belhareth, S., Sassatelli, L., Collange, D., Lopez-Pacheco, D., and G. Urvoy-Keller, "Understanding TCP cubic performance in the cloud: A mean-field approach", 2013 IEEE 2nd International Conference on Cloud Networking (CloudNet), DOI 10.1109/cloudnet.2013.6710576, November 2013, https://doi.org/10.1109/cloudnet.2013.6710576.
[CEHRX09] Cai, H., Eun, D., Ha, S., Rhee, I., and L. Xu, "Stochastic convex ordering for multiplicative decrease internet congestion control", Computer Networks, vol. 53, no. 3, pp. 365-381, DOI 10.1016/j.comnet.2008.10.012, February 2009, https://doi.org/10.1016/j.comnet.2008.10.012.
[FHP00] Floyd, S., Handley, M., and J. Padhye, "A Comparison of Equation-Based and AIMD Congestion Control", May 2000, https://www.icir.org/tfrc/aimd.pdf.
[GV02] Gorinsky, S. and H. Vin, "Extended Analysis of Binary Adjustment Algorithms", Technical Report TR2002-39, Department of Computer Sciences, The University of Texas at Austin, August 2002, https://citeseerx.ist.psu.edu/document?repid=rep1&type=pdf&doi=1828bdcef118b02d3996b8e00b8aaa92b50abb0f.
[H16] Ha, S., "Deployment, Testbed, and Simulation Results for CUBIC", Wayback Machine archive, 3 November 2016, https://web.archive.org/web/20161118125842/http://netsrv.csc.ncsu.edu/wiki/index.php/TCP_Testing.
[HLRX07] Ha, S., Le, L., Rhee, I., and L. Xu, "Impact of background traffic on performance of high-speed TCP variant protocols", Computer Networks, vol. 51, no. 7, pp. 1748-1762, DOI 10.1016/j.comnet.2006.11.005, May 2007, https://doi.org/10.1016/j.comnet.2006.11.005.
[HR11] Ha, S. and I. Rhee, "Taming the elephants: New TCP slow start", Computer Networks, vol. 55, no. 9, pp. 2092-2110, DOI 10.1016/j.comnet.2011.01.014, June 2011, https://doi.org/10.1016/j.comnet.2011.01.014.
[HRX08] Ha, S., Rhee, I., and L. Xu, "CUBIC: a new TCP-friendly high-speed TCP variant", ACM SIGOPS Operating Systems Review, vol. 42, no. 5, pp. 64-74, DOI 10.1145/1400097.1400105, July 2008, https://doi.org/10.1145/1400097.1400105.
[K03] Kelly, T., "Scalable TCP: improving performance in highspeed wide area networks", ACM SIGCOMM Computer Communication Review, vol. 33, no. 2, pp. 83-91, DOI 10.1145/956981.956989, April 2003, https://doi.org/10.1145/956981.956989.
[LBEWK16] Lukaseder, T., Bradatsch, L., Erb, B., Van Der Heijden, R., and F. Kargl, "A Comparison of TCP Congestion Control Algorithms in 10G Networks", 2016 IEEE 41st Conference on Local Computer Networks (LCN), DOI 10.1109/lcn.2016.121, November 2016, https://doi.org/10.1109/lcn.2016.121.
[LIU16] Liu, K. and J. Lee, "On Improving TCP Performance over Mobile Data Networks", IEEE Transactions on Mobile Computing, vol. 15, no. 10, pp. 2522-2536, DOI 10.1109/tmc.2015.2500227, October 2016, https://doi.org/10.1109/tmc.2015.2500227.
[RFC3522] Ludwig, R. and M. Meyer, "The Eifel Detection Algorithm for TCP", RFC 3522, DOI 10.17487/RFC3522, April 2003, https://www.rfc-editor.org/info/rfc3522.
[RFC3649] Floyd, S., "HighSpeed TCP for Large Congestion Windows", RFC 3649, DOI 10.17487/RFC3649, December 2003, https://www.rfc-editor.org/info/rfc3649.
[RFC3708] Blanton, E. and M. Allman, "Using TCP Duplicate Selective Acknowledgement (DSACKs) and Stream Control Transmission Protocol (SCTP) Duplicate Transmission Sequence Numbers (TSNs) to Detect Spurious Retransmissions", RFC 3708, DOI 10.17487/RFC3708, February 2004, https://www.rfc-editor.org/info/rfc3708.
[RFC3742] Floyd, S., "Limited Slow-Start for TCP with Large Congestion Windows", RFC 3742, DOI 10.17487/RFC3742, March 2004, https://www.rfc-editor.org/info/rfc3742.
[RFC6937] Mathis, M., Dukkipati, N., and Y. Cheng, "Proportional Rate Reduction for TCP", RFC 6937, DOI 10.17487/RFC6937, May 2013, https://www.rfc-editor.org/info/rfc6937.
[RFC7661] Fairhurst, G., Sathiaseelan, A., and R. Secchi, "Updating TCP to Support Rate-Limited Traffic", RFC 7661, DOI 10.17487/RFC7661, October 2015, https://www.rfc-editor.org/info/rfc7661.
[RFC8312] Rhee, I., Xu, L., Ha, S., Zimmermann, A., Eggert, L., and R. Scheffenegger, "CUBIC for Fast Long-Distance Networks", RFC 8312, DOI 10.17487/RFC8312, February 2018, https://www.rfc-editor.org/info/rfc8312.
[RFC8511] Khademi, N., Welzl, M., Armitage, G., and G. Fairhurst, "TCP Alternative Backoff with ECN (ABE)", RFC 8511, DOI 10.17487/RFC8511, December 2018, https://www.rfc-editor.org/info/rfc8511.
[RFC9000] Iyengar, J., Ed. and M. Thomson, Ed., "QUIC: A UDP-Based Multiplexed and Secure Transport", RFC 9000, DOI 10.17487/RFC9000, May 2021, https://www.rfc-editor.org/info/rfc9000.
[RFC9260] Stewart, R., Tรผxen, M., and K. Nielsen, "Stream Control Transmission Protocol", RFC 9260, DOI 10.17487/RFC9260, June 2022, https://www.rfc-editor.org/info/rfc9260.
[SXEZ19] Sun, W., Xu, L., Elbaum, S., and D. Zhao, "Model-Agnostic and Efficient Exploration of Numerical Congestion Control State Space of Real-World TCP Implementations", IEEE/ACM Transactions on Networking, vol. 29, no. 5, pp. 1990-2004, DOI 10.1109/tnet.2021.3078161, October 2021, https://doi.org/10.1109/tnet.2021.3078161.
[XHR04] Xu, L., Harfoush, K., and I. Rhee, "Binary increase congestion control (BIC) for fast long-distance networks", IEEE INFOCOM 2004, DOI 10.1109/infcom.2004.1354672, March 2004, https://doi.org/10.1109/infcom.2004.1354672.
Appendix A. Evolution of CUBIC since the Original Paper ๐ฆ
CUBIC has had a few glow-ups since the initial release [HRX08] of its algorithm and implementation ๐ this appendix spills the tea on the differences between the original paper and [RFC8312] ๐ซถ
- The original paper [HRX08] includes the pseudocode of CUBIC implementation using Linux's pluggable congestion control framework, which excludes system-specific optimizations. The simplified pseudocode might be a good starting point for learning about CUBIC. ๐ธ
- [HRX08] also includes experimental results showing its performance and fairness. โจ
- The definition of the ฮฒ__cubic_ constant was changed in [RFC8312]. For example, ฮฒ__cubic_ in the original paper was referred to as the window decrease constant, while [RFC8312] changed it to "CUBIC multiplicative decrease factor". With this change, the current congestion window size after a congestion event as listed in [RFC8312] was ฮฒ__cubic_ * _W_max_, while it was (1-ฮฒ__cubic_) * _W_max_ in the original paper. this is the kind of detail that makes or breaks an implementation ๐
- Its pseudocode used _W_(last_max)_, while [RFC8312] used _W_max_. ๐ง
- Its AIMD-friendly window was _W_tcp_, while [RFC8312] used _W_est_. ๐
Appendix B. Proof of the Average CUBIC Window Size ๐งฎโจ
okay so this appendix is where we actually prove the average CUBIC window size, which is called _AVG_W_cubic_ and shows up in Figure 6. ๐ it's giving math, but make it cute bestie ๐
we find _AVG_W_cubic_ under a deterministic loss model, where the number of packets between two successive packet losses is 1/_p_. with this model, CUBIC always operates with the concave window profile, and the time period between two successive packet losses is _K_. ๐ฆ
the average window size _AVG_W_cubic_ is defined like this: the numerator 1/_p_ is the total number of packets between two successive packet losses, and the denominator _K_/_RTT_ is the total number of RTTs between two successive packet losses. ๐ธ
1
โ
p
AVG_W = โโโ
cubic K
โโโ
RTTFigure 9 โจ
below, we find _K_ as a function of the CUBIC parameters ฮฒ_cubic and _C_, and the network parameters _p_ and _RTT_. ๐ง according to the definition of _K_ in Figure 2, we have:
โโโโโโโโโโโโโโโโโโโโโโ
3 โW - W * ฮฒ
โฒ โ max max cubic
K = โฒ โโโโโโโโโโโโโโโโโโโโโ
โฒโ CFigure 10 ๐
the total number of packets between two successive packet losses can also be obtained using the window increase function in Figure 1. specifically, the window size in the first RTT (that's _n_=1, or equivalently _t_=0) is _C_(-_K_)^3+_W_max_, and the window size in the last RTT (that's _n_=_K_/_RTT_, or equivalently _t_=_K_-_RTT_) is _C_(-_RTT_)^3+_W_max_. ๐
K
โโโ
RTT
โฏโฏ
1 โฒ โ 3 โ
โ = โฑ โC((n-1) * RTT-K) + W โ
p โบโบ โ maxโ
n=1
K
โโโ
RTT
โฏโฏ
โฒ โ 3 3 โ
= โฑ โC * RTT (-n) + W โ
โบโบ โ maxโ
n=1
K
โโโ
RTT
โฏโฏ
3 โฒ 3 K
= -C * RTT * โฑ n + W * โโโ
โบโบ max RTT
n=1
4
3 1 โ K โ K
โ -C * RTT * โ *โโโโโ + W * โโโ
4 โRTTโ max RTT
4
1 K K
= -C * โ * โโโ + W * โโโ
4 RTT max RTTFigure 11 ๐
okay now the big payoff ๐ after solving the equations in Figures 10 and 11 for _K_ and _W_max_, we get:
โโโโโโโโโโโโโโโโโโโโโโโโ
โ 4 * โ1-ฮฒ โ
4 โ โ cubicโ RTT
K = โฒ โโโโโโโโโโโโโโโโโ * โโโ
โฒ โC * โ3 + ฮฒ โ p
โฒโ โ cubicโ Figure 12 ๐
the average CUBIC window size _AVG_W_cubic_ can be obtained by substituting _K_ from Figure 12 into Figure 9. โญ
1 โโโโโโโโโโโโโโโโโโโโโโโโโ
โ โC * โ3 + ฮฒ โ 3
p 4 โ โ cubicโ RTT
AVG_W = โโโ = โฒ โโโโโโโโโโโโโโโโโ * โโโโ
cubic K โฒ โ 4 * โ1-ฮฒ โ 3
โโโ โฒโ โ cubicโ p
RTTAcknowledgments ๐ซถ
Richard Scheffenegger and Alexander Zimmermann originally coauthored [RFC8312].
these individuals suggested improvements to this document: ๐
- Bob Briscoe
- Christian Huitema
- Gorry Fairhurst
- Jonathan Morton
- Juhamatti Kuusisaari
- Junho Choi
- Markku Kojo
- Martin Duke
- Martin Thomson
- Matt Mathis
- Matt Olson
- Michael Welzl
- Mirja Kรผhlewind
- Mohit P. Tahiliani
- Neal Cardwell
- Praveen Balasubramanian
- Randall Stewart
- Richard Scheffenegger
- Rod Grimes
- Spencer Dawkins
- Tom Henderson
- Tom Petch
- Wesley Rosenblum
- Yoav Nir
- Yoshifumi Nishida
- Yuchung Cheng
Authors' Addresses ๐ฎ
Lisong Xu University of Nebraska-Lincoln Department of Computer Science and Engineering Lincoln, NE 68588-0115 United States of America Email: xu@unl.edu URI: https://cse.unl.edu/~xu/
Sangtae Ha University of Colorado at Boulder Department of Computer Science Boulder, CO 80309-0430 United States of America Email: sangtae.ha@colorado.edu URI: https://netstech.org/sangtaeha/
Injong Rhee Bowery Farming 151 W 26th Street, 12th Floor New York, NY 10001 United States of America Email: injongrhee@gmail.com
Vidhi Goel Apple Inc. One Apple Park Way Cupertino, CA 95014 United States of America Email: vidhi_goel@apple.com
Lars Eggert (editor) NetApp Stenbergintie 12 B FI-02700 Kauniainen Finland Email: lars@eggert.org URI: https://eggert.org/
๐ฆ where this page comes from
- ietf: rfc9438 2023-08
- ๐ girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms