RFC 3649
HighSpeed TCP for Large Congestion Windows
HighSpeed TCP for Large Congestion Windows ✨
Part 1 of 7
Network Working Group 🎀 This one comes from the Network Working Group and is written by S. Floyd of ICSI. It's in the Experimental category and dated December 2003. The header doesn't list anything it obsoletes or updates, so it stands on its own 💅
Status of This Memo 📜
Bestie, this memo defines an Experimental Protocol for the Internet community. It is not an Internet standard of any kind 🧁 The authors want discussion and suggestions for improvement, and the memo can be distributed freely. 🦋
Copyright Notice 💜
Copyright (C) The Internet Society (2003). All Rights Reserved.
Abstract 🦋
okay so the proposals in here are experimental 🧪 They can be deployed on today's Internet, but they don't count as a consensus that this is the best method for high-speed congestion control. Other experimental proposals are probably coming, and nobody really knows yet how these ones will interact with them 👀
This document proposes HighSpeed TCP, a change to TCP's congestion control mechanism for TCP connections that have large congestion windows 🚀 The congestion control in Standard TCP limits how big those windows can get in realistic environments, and that's kind of the whole problem 😭 Here's an example: a Standard TCP connection using 1500-byte packets and a 100 ms round-trip time would need an average congestion window of 83,333 segments just to hit a steady-state throughput of 10 Gbps. On top of that, it would need a packet drop rate of at most one congestion event every 5,000,000,000 packets (or, put another way, at most one congestion event every 1 2/3 hours). Everyone agrees that's an unrealistic constraint 💀 So this document proposes HighSpeed TCP to fix that limitation, and it's asking the wider community to try it out and send feedback 🫶
Table of Contents 🌸
1. Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . 2
2. The Problem Description.. . . . . . . . . . . . . . . . . . . . 3
3. Design Guidelines.. . . . . . . . . . . . . . . . . . . . . . . 4
4. Non-Goals.. . . . . . . . . . . . . . . . . . . . . . . . . . . 5
5. Modifying the TCP Response Function.. . . . . . . . . . . . . . 6
6. Fairness Implications of the HighSpeed Response
Function. . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
7. Translating the HighSpeed Response Function into
Congestion Control Parameters . . . . . . . . . . . . . . . . . 12
8. An alternate, linear response functions.. . . . . . . . . . . . 13
9. Tradeoffs for Choosing Congestion Control Parameters. . . . . . 16
9.1. The Number of Round-Trip Times between Loss Events . . . . 17
9.2. The Number of Packet Drops per Loss Event, with Drop-Tail. 17
10. Related Issues . . . . . . . . . . . . . . . . . . . . . . . . 18
10.1. Slow-Start. . . . . . . . . . . . . . . . . . . . . . . . 18
10.2. Limiting burstiness on short time scales. . . . . . . . . 19
10.3. Other limitations on window size. . . . . . . . . . . . . 19
10.4. Implementation issues.. . . . . . . . . . . . . . . . . . 19
11. Deployment issues. . . . . . . . . . . . . . . . . . . . . . . 20
11.1. Deployment issues of HighSpeed TCP. . . . . . . . . . . . 20
11.2. Deployment issues of Scalable TCP . . . . . . . . . . . . 22
12. Related Work in HighSpeed TCP. . . . . . . . . . . . . . . . . 23
13. Relationship to other Work.. . . . . . . . . . . . . . . . . . 25
14. Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . . 25
15. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 25
16. Normative References . . . . . . . . . . . . . . . . . . . . . 26
17. Informative References . . . . . . . . . . . . . . . . . . . . 26
18. Security Considerations. . . . . . . . . . . . . . . . . . . . 28
19. IANA Considerations. . . . . . . . . . . . . . . . . . . . . . 28
A. TCP's Loss Event Rate in Steady-State. . . . . . . . . . . . . 29
B. A table for a(w) and b(w). . . . . . . . . . . . . . . . . . . 30
C. Exploring the time to converge to fairness . . . . . . . . . . 32
Author's Address . . . . . . . . . . . . . . . . . . . . . . . 33
Full Copyright Statement . . . . . . . . . . . . . . . . . . . 341. Introduction ✨
bestie, this document proposes HighSpeed TCP, a change to TCP's congestion control mechanism for TCP connections with large congestion windows 🦋 In a steady-state environment with a packet loss rate p, Standard TCP's average congestion window is roughly 1.2/sqrt(p) segments. That is a serious limit on the congestion windows TCP can reach in realistic environments 😩 Take the same example from the abstract: a Standard TCP connection with 1500-byte packets and a 100 ms round-trip time needs an average congestion window of 83,333 segments to reach a steady-state throughput of 10 Gbps. It also needs a packet drop rate of at most one congestion event every 5,000,000,000 packets (or equivalently, at most one congestion event every 1 2/3 hours). The average packet drop rate that full link utilization needs in this setup is at most 2*10^(-10), and that corresponds to a bit error rate of at most 2*10^(-14). That's just not realistic for current networks 💀
To fix this fundamental limitation of TCP, and of the TCP response function (the function that maps the steady-state packet drop rate to TCP's average sending rate in packets per round-trip time) 🧠, this document describes a modified TCP response function for regimes with higher congestion windows. It also asks the wider community to test HighSpeed TCP and give feedback 🫶
Here's the key thing: HighSpeed TCP's modified response function only kicks in when congestion windows are higher. So it doesn't change TCP's behavior in environments with heavy congestion, and it doesn't introduce any new risk of congestion collapse 🙌 Fairness is still a big deal, though. If relative fairness between HighSpeed TCP connections is going to be preserved, the authors feel any change to the TCP response function should be handled in the IETF, not through ad hoc decisions made by individual implementors or TCP senders 🧐 Changes to the TCP response function would also affect transport protocols that use TFRC and other forms of equation-based congestion control, since those mechanisms directly use the TCP response function [RFC3448] 🎀
This proposal focuses only on a change to the TCP response function and what that means for TCP 💅 It does not tackle a separate issue the authors see as fundamental: how to let best-effort connections start with large initial windows. The authors see HighSpeed TCP as a somewhat fundamental change to the response function, but also a relatively simple one to implement in a single TCP sender, and one that doesn't bring congestion collapse dangers 😌 Starting with large initial windows is, in their view, much riskier and structurally harder, since it would need some form of explicit feedback from all of the routers along the path 😳 That's another reason to handle starting with large initial windows separately, on its own timetable, apart from changing the TCP response function ✨
2. The Problem Description 📊
This section describes the number of round-trip times between congestion events that a Standard TCP flow needs in order to reach an average throughput of B bps, given packets of D bytes and a round-trip time of R seconds 🧮 A congestion event is a window of data with one or more dropped or ECN-marked packets (ECN stands for Explicit Congestion Notification) 🌈
From Appendix A, reaching an average TCP throughput of B bps requires a loss event at most every BR/(12D) round-trip times 🙈 Table 1 shows this for R = 0.1 seconds and D = 1500 bytes. The table also gives the average congestion window W, which is BR/(8D), and the steady-state packet drop rate P, which is 1.5/W^2 🍓
TCP Throughput (Mbps) RTTs Between Losses W P
--------------------- ------------------- ---- -----
1 5.5 8.3 0.02
10 55.5 83.3 0.0002
100 555.5 833.3 0.000002
1000 5555.5 8333.3 0.00000002
10000 55555.5 83333.3 0.0000000002Table 1: RTTs Between Congestion Events for Standard TCP, for 1500-Byte Packets and a Round-Trip Time of 0.1 Seconds.
So this document proposes HighSpeed TCP as a minimal modification to TCP's increase and decrease parameters, used for TCP connections with larger congestion windows 🚀 The goal is to let TCP reach high throughput with a more realistic steady-state packet drop rate requirement. Put another way, HighSpeed TCP has more realistic requirements for how many round-trip times pass between loss events 🦋
3. Design Guidelines 🧭
The HighSpeed TCP proposal is driven by these requirements, bestie 💖
- Achieve high per-connection throughput without needing unrealistically low packet loss rates.
- Reach high throughput reasonably quickly while in slow-start.
- Reach high throughput without overly long delays when recovering from multiple retransmit timeouts, or when ramping up from a period with small congestion windows.
- No additional feedback or support required from routers:
- The goal is acceptable performance in both ECN-capable and non-ECN-capable environments, and with Drop-Tail as well as with Active Queue Management such as RED in the routers.
- No additional feedback required from TCP receivers.
- TCP-compatible performance in environments with moderate or high congestion (e.g., packet drop rates of 1% or higher):
- Put another way, there should be no additional load on the network (in terms of increased packet drop rates) in environments with moderate or high congestion.
- Performance at least as good as Standard TCP in environments with moderate or high congestion.
- Acceptable transient performance, in terms of increases in the congestion window in one round-trip time, responses to severe congestion, and convergence times to fairness.
Right now, users who want throughputs of 1 Gbps or more usually open multiple TCP connections in parallel, or use MulTCP [CO98,GRK99], which behaves roughly like the aggregate of N virtual TCP connections 🧃 That works fine for the occasional user on well-provisioned links. But it leaves the parameter N for the user to figure out, and it's more aggressive and produces higher steady-state packet drop rates when used in environments with periods of moderate or high congestion 😬 The authors believe a new approach is needed: one that offers more flexibility, scales effectively across a wide range of available bandwidths, and competes more fairly with Standard TCP in congested environments 👑
Part 1 complete 🌈💖
4. Non-Goals ✨
okay so, these are the things we are explicitly not trying to do 💅
- Non-goal: TCP-compatible performance in environments with very low packet drop rates. 🙈
We're being real here: our proposal does not require, or deliver, TCP-compatible performance in environments with very low packet drop rates, like packet loss rates of 10^-5 or 10^-6. As we discuss later in this document, we assume that Standard TCP is unable to make effective use of the available bandwidth in environments with loss rates of 10^-6 in any case. So it is acceptable and appropriate for HighSpeed TCP to perform more aggressively than Standard TCP in such an environment. 🔥
- Non-goal: Ramping-up more quickly than allowed by slow-start. 🚀
It is our belief that ramping-up more quickly than allowed by slow-start would necessitate more explicit feedback from routers along the path. The proposal for HighSpeed TCP is focused on changes to TCP that could be effectively deployed in the current Internet environment. Realistic glow-up only 🫶
- Non-goal: Avoiding oscillations in environments with only one-way, long-lived flows all with the same round-trip times. 🌊
We agree that attention to oscillatory behavior is useful, ngl. But avoiding oscillations in aggregate throughput has not been our primary consideration. This is especially true for simplified environments limited to one-way, long-lived flows all with the same, large round-trip times. Our assessment is that some oscillatory behavior in these extreme environments is an acceptable price to pay for the other benefits of HighSpeed TCP. Worth it, periodt 👑
5. Modifying the TCP Response Function ✨
okay, the TCP response function is w = 1.2/sqrt(p) 🦋 It tells us TCP's average congestion window w, measured in MSS-sized segments, as a function of the steady-state packet drop rate p [FF98]. This is a direct consequence of TCP's Additive Increase Multiplicative Decrease (AIMD) mechanisms. The window grows by roughly one segment per round-trip time when there is no congestion, and gets halved when a round-trip time has a congestion event. Main character energy, honestly 💖
The response function for Standard TCP is shown in the table below. In this proposal, we restrict our attention to TCP performance in environments with packet loss rates of at most 10^-2. That means we can ignore the more complex response functions needed to model TCP performance in more congested environments with retransmit timeouts. 🙌 From Appendix A, an average congestion window of W corresponds to an average of 2/3 W round-trip times between loss events for Standard TCP. The congestion window varies from 2/3 W to 4/3 W in that case.
Packet Drop Rate P Congestion Window W RTTs Between Losses
------------------ ------------------- -------------------
10^-2 12 8
10^-3 38 25
10^-4 120 80
10^-5 379 252
10^-6 1200 800
10^-7 3795 2530
10^-8 12000 8000
10^-9 37948 25298
10^-10 120000 80000Table 2: TCP Response Function for Standard TCP. The average congestion window W in MSS-sized segments is given as a function of the packet drop rate P.
Now for the HighSpeed version 🚄 We use three parameters: Low_Window, High_Window, and High_P. To keep TCP compatibility, the HighSpeed response function uses the same response function as Standard TCP when the current congestion window is at most Low_Window. Once the current congestion window is greater than Low_Window, it switches to the HighSpeed response function. In this document, we set Low_Window to 38 MSS-sized segments, which corresponds to a packet drop rate of 10^-3 for TCP. ✨
To specify the upper end of the HighSpeed response function, we specify the packet drop rate needed to achieve an average congestion window of 83000 segments. That's roughly the window needed to sustain 10 Gbps throughput, for a TCP connection with the default packet size and round-trip time used earlier in this document. For High_Window set to 83000, we specify High_P of 10^-7. In other words, with HighSpeed TCP, a packet drop rate of 10^-7 allows the HighSpeed TCP connection to achieve an average congestion window of 83000 segments. We believe that this loss rate sets an achievable target for high-speed environments. It still allows acceptable fairness for the HighSpeed response function when competing with Standard TCP in environments with packet drop rates of 10^-4 or 10^5. 🥹
For simplicity, for the HighSpeed response function we maintain the property that the response function gives a straight line on a log-log scale. Standard TCP does this too, for low to moderate congestion. This results in the following response function, for values of the average congestion window W greater than Low_Window:
W = (p/Low_P)^S Low_Window,Here, Low_P is the packet drop rate corresponding to Low_Window, and S is the following constant [FRS02]:
S = (log High_Window - log Low_Window)/(log High_P - log Low_P).(In this paper, "log x" refers to the log base 10. 🤓) For example, for Low_Window set to 38, we have Low_P of 10^-3, for compatibility with Standard TCP. So for High_Window set to 83000 and High_P set to 10^-7, we get the following response function:
W = 0.12/p^0.835. (1)This HighSpeed response function is illustrated in Table 3 below. For HighSpeed TCP, the number of round-trip times between losses, 1/(pW), equals 12.7 W^0.2, for W > 38 segments. 📈
Packet Drop Rate P Congestion Window W RTTs Between Losses
------------------ ------------------- -------------------
10^-2 12 8
10^-3 38 25
10^-4 263 38
10^-5 1795 57
10^-6 12279 83
10^-7 83981 123
10^-8 574356 180
10^-9 3928088 264
10^-10 26864653 388Table 3: TCP Response Function for HighSpeed TCP. The average congestion window W in MSS-sized segments is given as a function of the packet drop rate P.
We believe that backward compatibility with Standard TCP needs a response function that is quite close to that of Standard TCP for loss rates of 10^-1, 10^-2, or 10^-3. ngl, that's the vibe we're going for 🫶 We believe, however, that such stringent TCP-compatibility is not required for smaller loss rates. An appropriate response function, we think, is one that gives a plausible packet drop rate for a connection throughput of 10 Gbps. This also gives a slowly increasing number of round-trip times between loss events as a function of a decreasing packet drop rate. 🌸
Another way to look at the HighSpeed response function is that HighSpeed TCP is roughly emulating the congestion control response of N parallel TCP connections. N starts out as one, and N increases as a function of the HighSpeed TCP's congestion window. So for the HighSpeed response function in Equation (1) above, the response function can be viewed as equivalent to that of N(W) parallel TCP connections, where N(W) varies as a function of the congestion window W. Recall that for a single standard TCP connection, the average congestion window equals 1.2/sqrt(p). For N parallel TCP connections, the aggregate congestion window for the N connections equals N*1.2/sqrt(p). From the HighSpeed response function in Equation (1) and the relationship above, we can derive the following:
N(W) = 0.23*W^(0.4)Here, N(W) is the number of parallel TCP connections emulated by the HighSpeed TCP response function, for N(W) >= 1. This is shown in Table 4 below. 🧚
Congestion Window W Number N(W) of Parallel TCPs
------------------- -------------------------
1 1
10 1
100 1.4
1,000 3.6
10,000 9.2
100,000 23.0Table 4: Number N(W) of parallel TCP connections roughly emulated by the HighSpeed TCP response function.
In this document, we do not attempt to seriously evaluate the HighSpeed response function for congestion windows greater than 100,000 packets. We believe that we'll learn more about the requirements for sustaining the throughput of best-effort connections in that range as we gain more experience with HighSpeed TCP. That means congestion windows of thousands and tens of thousands of packets. 👀 There also might be limitations to the per-connection throughput that can be realistically achieved for best-effort traffic, in terms of a congestion window of hundreds of thousands of packets or more. That's true in the absence of additional support or feedback from the routers along the path. we'll see what the data says 💅
6. Fairness Implications of the HighSpeed Response Function ✨
okay so the Standard and HighSpeed Response Functions can be used directly to figure out how fair things are between flows using the two of them 💅 Say you've got a packet drop rate P, and Standard TCP has an average congestion window of W_Standard, while HighSpeed TCP has a higher average congestion window of W_HighSpeed.
Then a single HighSpeed TCP connection gets W_HighSpeed/W_Standard times the throughput of a single Standard TCP connection competing in the same environment. The ratio is giving main character energy 🦋
Table 5 shows this relative fairness for the parameters used by the HighSpeed response function in the section above 🎀 Column two is the relative fairness at the steady-state packet drop rate in column one. Column three is the aggregate average congestion window for the two TCP connections, and column four is the bandwidth the two connections would need to hit that aggregate window and drop rate, assuming 100 ms round-trip times and 1500-byte packets 🧚
Packet Drop Rate P Fairness Aggregate Window Bandwidth
------------------ -------- ---------------- ---------
10^-2 1.0 24 2.8 Mbps
10^-3 1.0 76 9.1 Mbps
10^-4 2.2 383 45.9 Mbps
10^-5 4.7 2174 260.8 Mbps
10^-6 10.2 13479 1.6 Gbps
10^-7 22.1 87776 10.5 GbpsTable 5: Relative Fairness between the HighSpeed and Standard Response Functions.
So for packet drop rates of 10^-4, a flow with the HighSpeed response function can expect to receive 2.2 times the throughput of a flow using the Standard response function, given the same round-trip times and packet sizes 🌸 At packet drop rates of 10^-6 (or 10^-7) it gets way more unfair 😭 We've entered the regime where a Standard TCP connection needs at most one congestion event every 800 (or 2530) round-trip times just to make use of the available bandwidth. Our judgement is that not a lot of TCP connections are effectively operating in this regime today, with congestion windows of thousands of packets, so the benefits of the HighSpeed response function would outweigh the unfairness Standard TCP would experience here 💜 Still, one purpose of this document is to solicit feedback on this issue, so tell us what you think 🫶 The parameter Low_Window directly determines the point of divergence between the Standard and HighSpeed Response Functions.
The third column of Table 5, the Aggregate Window, is the aggregate congestion window of the two competing TCP connections, one HighSpeed and one Standard, at the packet drop rate in column one 🍓 From Table 5, a HighSpeed TCP connection would receive ten times the bandwidth of a Standard TCP connection in an environment with a packet drop rate of 10^-6 👑 That happens when the two flows sharing a single pipe reach an aggregate window of 13479 packets. With a round-trip time of 100 ms and a packet size of 1500 bytes, that happens at an available bandwidth for the two competing flows of 1.6 Gbps.
Next up: how long it takes a Standard or HighSpeed TCP flow to converge to fairness against a pre-existing HighSpeed TCP flow ⏳ The worst case is when a new flow is starting up, competing against a high-bandwidth existing flow, and the new flow takes a packet drop and exits slow-start while its window is still small 🙈 In the worst case, the new flow has entered the congestion avoidance phase while its window is only one packet. A Standard TCP flow in congestion avoidance increases its window by at most one packet per round-trip time, so after N round-trip times it has only reached a window of N packets (when starting with a window of 1 in the first round-trip time) 😩 HighSpeed TCP flows increase much faster than Standard TCP flows in the congestion avoidance phase, so we can expect their convergence to fairness to be way better ✨ This is shown in Table 6 below. The script used to generate this table is given in Appendix C.
RTT HS_Window Standard_TCP_Window
--- --------- -------------------
100 131 100
200 475 200
300 1131 300
400 2160 400
500 3601 500
600 5477 600
700 7799 700
800 10567 800
900 13774 900
1000 17409 1000
1100 21455 1100
1200 25893 1200
1300 30701 1300
1400 35856 1400
1500 41336 1500
1600 47115 1600
1700 53170 1700
1800 59477 1800
1900 66013 1900
2000 72754 2000Table 6: For a HighSpeed and a Standard TCP connection, the congestion window during congestion avoidance phase (starting with a congestion window of 1 packet during RTT 1).
The classic paper on relative fairness is from Chiu and Jain [CJ89]. It shows that AIMD (Additive Increase Multiplicative Decrease) converges to fairness in an environment with synchronized congestion events 🎀 From [CJ89], it's easy to see that MIMD and AIAD do not converge to fairness in that environment. BUT the results of [CJ89] don't apply to an asynchronous environment like the current Internet, where the frequency of congestion feedback can differ between flows 🙃 For example, it has been shown that MIMD converges to fair states in a model with proportional instead of synchronous feedback in terms of packet drops [GV02]. So we're not concerned about abandoning a strict model of AIMD for HighSpeed TCP 💅 However, we note that in an environment with Drop-Tail queue management, there is likely to be some synchronization of packet drops. In that environment, the model of completely synchronous feedback doesn't hold, but the model of completely asynchronous feedback isn't accurate either 🌈 Fairness in Drop-Tail environments is discussed in more detail in Sections 9 and 12.
7. Translating the HighSpeed Response Function into Congestion Control Parameters 🔥
For equation-based congestion control such as TFRC, the HighSpeed Response Function above could be used directly by the TFRC congestion control mechanism 🧠 For TCP, though, the HighSpeed response function has to be translated into additive increase and multiplicative decrease parameters. The HighSpeed response function can't be achieved by TCP with an additive increase of one segment per round-trip time and a multiplicative decrease of halving the current congestion window. HighSpeed TCP will have to modify either the increase or the decrease parameter, or both 😤 We've concluded that HighSpeed TCP is most likely to hit an acceptable compromise between moderate increases and timely decreases by modifying both the increase and the decrease parameter 🫶
So for HighSpeed TCP, let the congestion window increase by a(w) segments per round-trip time in the absence of congestion, and let the congestion window decrease to w(1-b(w)) segments in response to a round-trip time with one or more loss events 💖 That means that in response to a single acknowledgement, HighSpeed TCP increases its congestion window (in segments) like this:
w <- w + a(w)/w.In response to a congestion event, HighSpeed TCP decreases like this:
w <- (1-b(w))w.For Standard TCP, a(w) = 1 and b(w) = 1/2, regardless of the value of w 🍓 HighSpeed TCP uses the same values of a(w) and b(w) for w <= Low_Window. This section specifies a(w) and b(w) for HighSpeed TCP for larger values of w 🌸
For w = High_Window, we have specified a loss rate of High_P. From [FRS02], or from elementary calculations, this requires the following relationship between a(w) and b(w) for w = High_Window:
a(w) = High_Window^2 * High_P * 2 * b(w)/(2-b(w)). (2)We use the parameter High_Decrease to specify the decrease parameter b(w) for w = High_Window, and use Equation (2) to derive the increase parameter a(w) for w = High_Window ✨ Along with High_P = 10^-7 and High_Window = 83000, for example, we specify High_Decrease = 0.1, which says that b(83000) = 0.1, giving a decrease of 10% after a congestion event. Equation (2) then gives a(83000) = 72, for an increase of 72 segments (or just under 0.1%) within a round-trip time, for w = 83000 💜
This moderate decrease is acceptable to us, especially with the role TCP's ACK-clocking plays in limiting the sending rate in response to more severe congestion [BBFS01]. A more severe decrease would need a more aggressive increase in the congestion window for a round-trip time without congestion. In particular, a decrease factorHigh_Decreaseof 0.5, as in Standard TCP, would need an increase of 459 segments per round-trip time whenw = 83000😭
Given decrease parameters of b(w) = 1/2 for w = Low_Window and b(w) = High_Decrease for w = High_Window, we still need to specify the value of b(w) for other values of w > Low_Window 🦋 From [FRS02], we let b(w) vary linearly with the log of w, like this:
b(w) = (High_Decrease - 0.5) (log(w)-log(W)) / (log(W_1)-log(W)) +
0.5,for W = Low_window and W_1 = High_window. The increase parameter a(w) can then be computed like this:
a(w) = w^2 * p(w) * 2 * b(w)/(2-b(w)),for p(w) the packet drop rate for congestion window w. From inverting Equation (1), we get p(w) like this:
p(w) = 0.078/w^1.2.We assume that experimental implementations of HighSpeed TCP for further investigation will use a pre-computed look-up table for finding a(w) and b(w) 🧚 For example, the implementation from Tom Dunigan adjusts the a(w) and b(w) parameters every 0.1 seconds. In the appendix we give such a table for our default values of Low_Window = 38, High_Window = 83,000, High_P = 10^-7, and High_Decrease = 0.1 🎀 These are also the default values in the NS simulator. Example simulations in NS can be run with the command "./test-all-tcpHighspeed" in the directory tcl/test.
8. An alternate, linear response functions ✨
okay so in this section we explore an alternate, linear response function for HighSpeed TCP 🦋. a bunch of people have proposed it, especially Glenn Vinnicombe and Tom Kelly. it's also been suggested that a less "ad-hoc" guideline would be to specify a constant number of round-trip times between congestion events. very clean energy 💅
let's keep Low_Window at 38 MSS-sized segments, which is still where the HighSpeed response function diverges from the current TCP one 🎀. but we'll modify the High_Window and High_P parameters that set the upper range of the HighSpeed response function. specifically, we look at High_Window = 380,000 and High_P = 10^-7, with Low_Window = 38 and Low_P = 10^-3 like before.
using the equations in Section 5, this gives the following Linear response function, for w > Low_Window:
W = 0.038/p.this Linear HighSpeed response function is shown in Table 7 below. for HighSpeed TCP, the number of round-trip times between losses, 1/(pW), equals 1/0.38, or equivalently, 26, for W > 38 segments 🌸.
Packet Drop Rate P Congestion Window W RTTs Between Losses
------------------ ------------------- -------------------
10^-2 12 8
10^-3 38 26
10^-4 380 26
10^-5 3800 26
10^-6 38000 26
10^-7 380000 26
10^-8 3800000 26
10^-9 38000000 26
10^-10 380000000 26Table 7: An Alternate, Linear TCP Response Function for HighSpeed TCP. The average congestion window W in MSS-sized segments is given as a function of the packet drop rate P.
given a constant decrease b(w) of 1/2, this would give an increase a(w) of w/Low_Window, or equivalently, a constant increase of 1/Low_Window packets per acknowledgement, for w > Low_Window ⭐. another option is Scalable TCP [K03], which uses a fixed decrease b(w) of 1/8 and a fixed increase per acknowledgement of 0.01. this gives an increase a(w) per window of 0.005 w, for a TCP with delayed acknowledgements, for pure MIMD 🧚.
the relative fairness between the alternate Linear response function and the standard TCP response function is shown in Table 8 below 💜.
Packet Drop Rate P Fairness Aggregate Window Bandwidth
------------------ -------- ---------------- ---------
10^-2 1.0 24 2.8 Mbps
10^-3 1.0 76 9.1 Mbps
10^-4 3.2 500 60.0 Mbps
10^-5 15.1 4179 501.4 Mbps
10^-6 31.6 39200 4.7 Gbps
10^-7 100.1 383795 46.0 GbpsTable 8: Relative Fairness between the Linear HighSpeed and Standard Response Functions.
one attraction of the linear response function is that it is scale-invariant, with a fixed increase in the congestion window per acknowledgement and a fixed number of round-trip times between loss events 🍓. my own assumption is that having a fixed length for the congestion epoch in round-trip times, regardless of the packet drop rate, would be a poor fit for an imprecise and imperfect world 🙈. the world has routers with a range of queue management mechanisms, such as the Drop-Tail queue management that is common today. for example, a response function with a fixed length for the congestion epoch in round-trip times might give less clearly-differentiated feedback in an environment with steady-state background losses at fixed intervals for all flows. that could happen with a wireless link with occasional short error bursts, giving losses for all flows every N seconds regardless of their sending rate.
while it is not a goal to have perfect fairness in an environment with synchronized losses, it would be good to have moderately acceptable performance in this regime. this goal might argue against a response function with a constant number of round-trip times between congestion events. however, this is a question that could clearly use additional research and investigation 🔥. also, flows with different round-trip times would have different time durations for congestion epochs, even in the model with a linear response function.
the third column of Table 8, the Aggregate Window, gives the aggregate congestion window of two competing TCP connections, one with Linear HighSpeed TCP and one with Standard TCP, given the packet drop rate in the first column. from Table 8, a Linear HighSpeed TCP connection would receive fifteen times the bandwidth of a Standard TCP connection in an environment with a packet drop rate of 10^-5 👑. this would occur when the two flows sharing a single pipe achieved an aggregate window of 4179 packets. given a round-trip time of 100 ms and a packet size of 1500 bytes, this would occur with an available bandwidth for the two competing flows of 501 Mbps. so, because the Linear HighSpeed TCP is more aggressive than the HighSpeed TCP proposed above, it is also less fair when competing with Standard TCP in a high-bandwidth environment 😭.
9. Tradeoffs for Choosing Congestion Control Parameters 💖
a range of metrics can be used for evaluating choices for congestion control parameters for HighSpeed TCP 🌈. my assumption in this section is that for a response function of the form w = c/p^d, for constant c and exponent d, the only response functions that would be considered are response functions with 1/2 <= d <= 1. the two ends of this spectrum are represented by current TCP, with d = 1/2, and by the linear response function described in Section 8 above, with d = 1. HighSpeed TCP lies somewhere in the middle of the spectrum, with d = 0.835 ✨.
response functions with exponents less than 1/2 can be eliminated from consideration, because they would be even worse than standard TCP at accommodating connections with high congestion windows 💀.
9.1. The Number of Round-Trip Times between Loss Events 🧠
response functions with exponents greater than 1 can be eliminated from consideration, because for these response functions the number of round-trip times between loss events decreases as congestion decreases. for a response function of w = c/p^d, with one loss event or congestion event every 1/p packets, the number of round-trip times between loss events is w^((1/d)-1)/c^(1/d). so for standard TCP, the number of round-trip times between loss events is linear in w. in contrast, one attraction of the linear response function, as described in Section 8 above, is that it is scale-invariant, in terms of a fixed increase in the congestion window per acknowledgement and a fixed number of round-trip times between loss events 🦋.
however, for a response function with d > 1, the number of round-trip times between loss events would be proportional to w^((1/d)-1), for a negative exponent ((1/d)-1), setting smaller as w increases. this would seem undesirable 🙃.
9.2. The Number of Packet Drops per Loss Event, with Drop-Tail 📦
a TCP connection increases its sending rate by a(w) packets per round-trip time, and in a Drop-Tail environment, this is likely to result in a(w) dropped packets during a single loss event. one attraction of standard TCP is that it has a fixed increase per round-trip time of one packet, which minimizes the number of packets that would be dropped in a Drop-Tail environment 🫶. for an environment with some form of Active Queue Management, and in particular for an environment that uses ECN, the number of packets dropped in a single congestion event would not be a problem. however, even in these environments, larger increases in the sending rate per round-trip time result in larger stresses on the ability of the queues in the router to absorb the fluctuations.
HighSpeed TCP plays a middle ground between the metrics of a moderate number of round-trip times between loss events and a moderate increase in the sending rate per round-trip time. as shown in Appendix B, for a congestion window of 83,000 packets, HighSpeed TCP increases its sending rate by 70 packets per round-trip time, resulting in at most 70 packet drops when the buffer overflows in a Drop-Tail environment 🌸. this increased aggressiveness is the price HighSpeed TCP pays for its increased scalability. a large number of packets dropped per congestion event could result in synchronized drops from multiple flows, with a possible loss of throughput as a result.
Scalable TCP has an increase a(w) of 0.005 w packets per round-trip time. for a congestion window of 83,000 packets, this gives an increase of 415 packets per round-trip time, resulting in roughly 415 packet drops per congestion event in a Drop-Tail environment 😱.
thus, HighSpeed TCP and its variants place increased demands on queue management in routers, relative to Standard TCP. (this is rather similar to the increased demands on queue management that would result from using N parallel TCP connections instead of a single Standard TCP connection.) 🎀
10. Related Issues 🌷
10.1. Slow-Start ✨
okay so the companion internet-draft "Limited Slow-Start for TCP with Large Congestion Windows" [F02b] proposes a change to TCP's slow-start procedure, and it can seriously boost performance for TCP connections that slow-start up to big congestion windows 🚀. For connections that can use congestion windows of thousands (or tens of thousands) of MSS-sized segments (where MSS is the sender's MAXIMUM SEGMENT SIZE), the current slow-start can blow the congestion window up by thousands of segments in a single round-trip time 😭. That kind of jump can easily get thousands of packets dropped in one round-trip time. It's often counter-productive for the TCP flow itself, and it's also rough on everyone else sharing the congested link 💀.
[F02b] proposes Limited Slow-Start, which caps how many segments the congestion window can grow by for one window of data during slow-start. The goal is better performance for TCP connections with large congestion windows 🎀. We split Limited Slow-Start into its own draft because it works with either Standard or HighSpeed TCP 💅.
Limited Slow-Start is illustrated in the NS simulator, for snapshots after May 1, 2002, in the tests ./test-all-tcpHighspeed tcp1A and ./test-all-tcpHighspeed tcpHighspeed1 in the subdirectory tcl/lib 🧚.
We believe that for best-effort flows to safely start up faster than slow-start, for example in future high-bandwidth networks, the flow would need explicit feedback from the routers along the path 🌈. There are a bunch of proposals for this. They range from a minimal IP option that lets TCP SYN packets collect info from routers along the path about the allowed initial sending rate [J02], all the way to more powerful proposals that need more fine-tuned, continuous feedback from routers. All of these are longer-term ideas than the HighSpeed TCP proposal in this document. They need longer lead times and more coordination to deploy, and they'll be discussed in later documents 🔥.
10.2. Limiting burstiness on short time scales 🫶
Because a HighSpeed TCP connection can reach a pretty huge congestion window, the sender could send a big burst of packets in response to a single acknowledgement 😳. This could happen, for example, when there's congestion or reordering on the reverse path, and the sender gets an acknowledgement covering hundreds or thousands of new packets. A burst like that would also happen if the application sat idle for a short period (less than a round-trip time) and then suddenly had lots of data to send. In that case, it would be super useful for the HighSpeed TCP connection to have some way of limiting bursts 💖.
In this document, we don't specify TCP mechanisms for reducing short-term burstiness. One possible mechanism is some form of rate-based pacing. Another option is maxburst, which limits how many packets get sent in response to a single acknowledgement 🍓. We'd caution, though, against permanently reducing the congestion window as a way to limit short-term bursts. That approach has been deployed in some TCP stacks, and honestly our view is that permanently shrinking the congestion window to deal with transient bursts is a bad idea [Fl03] 🙈.
10.3. Other limitations on window size 📏
The TCP header uses a 16-bit field to report the receive window size to the sender. Unmodified, that allows a window size of at most 2**16 = 65K bytes. With window scaling, the maximum window size is 2**30 = 1073M bytes [RFC 1323] ✨. With 1500-byte packets, that means a window of up to 715,000 packets 🤯.
10.4. Implementation issues 🛠️
One implementation issue raised with HighSpeed TCP is that with congestion windows of 4MB or more, handling successive SACK packets after a packet is dropped becomes really time-consuming at the TCP sender [S03] 😩. Tom Kelly's Scalable TCP includes a "SACK Fast Path" patch that fixes this problem 👑.
The issues addressed in the Web100 project, the Net100 project, and related projects, which are all about tuning TCP to hit high bandwidth data rates, apply to HighSpeed TCP as well [Net100, Web100] 🌸.
11. Deployment issues 🚀
11.1. Deployment issues of HighSpeed TCP 💅
We don't claim that the HighSpeed TCP modification to TCP described in this paper is an optimal transport protocol for high-bandwidth environments. Based on our experiences with HighSpeed TCP in the NS simulator [NS], on simulation studies [SA03], and on experimental reports [ABLLS03,D02,CC03,F03], we believe HighSpeed TCP improves TCP performance in high-bandwidth environments. We're documenting it for the benefit of the IETF community 🦋. We encourage using HighSpeed TCP and its underlying response function, and we'd love feedback about operational experiences with this or related modifications 💜.
We note that in environments typical of much of the current Internet, HighSpeed TCP behaves exactly like Standard TCP does today. This is the case any time the congestion window is less than 38 segments 🎀.
Bandwidth Avg Cwnd w (pkts) Increase a(w) Decrease b(w)
--------- ----------------- ------------- -------------
1.5 Mbps 12.5 1 0.50
10 Mbps 83 1 0.50
100 Mbps 833 6 0.35
1 Gbps 8333 26 0.22
10 Gbps 83333 70 0.10
Table 9: Performance of a HighSpeed TCP connectionTo help calibrate, Table 9 considers a TCP connection with 1500-byte packets, an RTT of 100 ms (including average queueing delay), and no competing traffic. It shows the average congestion window that connection would have if it had a pipe all to itself and fully used the link bandwidth, for a range of pipe bandwidths 📊. This assumes the TCP connection would use Table 12 to work out its increase and decrease parameters. The first column gives the bandwidth. The second column gives the average congestion window w needed to use that bandwidth. The third column shows the increase a(w) in segments per RTT for window w. The fourth column shows the decrease b(w) for that window w, where the TCP sender drops the congestion window from w to w(1-b(w)) segments after a loss event 🧠. When a loss happens, the actual congestion window is likely to be bigger than the average congestion window w in column 2, so the decrease parameter used could be slightly smaller than the one in column 4 of Table 9 🤓.
Table 9 shows that HighSpeed TCP over a 10 Mbps link behaves exactly the same as a Standard TCP connection, even with no competing traffic 😌. You can think of the congestion window staying generally in the range of 55 to 110 segments, with HighSpeed TCP behaving exactly like Standard TCP. (If the congestion window is ever 128 segments or more, then HighSpeed TCP increases by two segments per RTT instead of one, and uses a decrease parameter of 0.44 instead of 0.50 💫.)
Table 9 also shows that for a HighSpeed TCP connection over a 100 Mbps link, with no competing traffic, HighSpeed TCP behaves roughly as aggressively as six parallel TCP connections 🏃♀️. It increases its congestion window by roughly six segments per round-trip time, with a decrease parameter of roughly 1/3. That corresponds to dropping down to 2/3rds of its old congestion window, rather than to half, in response to a loss event ⭐.
For a Standard TCP connection in this environment, the congestion window could be thought of as generally varying in the range of 550 to 1100 segments. The average packet drop rate is 2.2 * 10^-6, which corresponds to a bit error rate of 1.8 * 10^-10, or equivalently, roughly 55 seconds between congestion events ⏳. A Standard TCP connection could sustain a drop rate that low in a carefully controlled environment with minimal competing traffic. But we'd contend that in an uncontrolled best-effort environment with even a small amount of competing traffic, the occasional congestion events from smaller competing flows could easily stop a Standard TCP flow with no lower-speed bottlenecks from fully using the available bandwidth of the underutilized 100 Mbps link 😤.
So what we're really saying is that in an environment of 100 Mbps links with a lot of spare bandwidth, Standard TCP would sometimes be unable to fully use the link, and HighSpeed TCP would be an improvement there 🌟. We'd also contend that in this environment, HighSpeed TCP behaves close enough to Standard TCP that it would be safe to deploy on today's Internet 🫶. Note that HighSpeed TCP can only use high congestion windows if the receiver's advertised window size allows it. As a result, even if HighSpeed TCP were deployed everywhere, the impact would be limited to TCP connections with an advertised window from the receiver of 118 MSS or larger 🙌.
We don't believe deploying HighSpeed TCP would block alternate experimental protocols for high-speed congestion control, like Scalable TCP, XCP [KHR02], or FAST TCP [JWL03] 🧚♀️. In particular, we don't expect HighSpeed TCP to interact any more poorly with those alternative experimental proposals than the N parallel TCP connections commonly used today would, in the absence of HighSpeed TCP 💖.
11.2. Deployment issues of Scalable TCP 🌸
Okay so, Scalable TCP and HighSpeed TCP have response functions that are super similar, which means they could honestly coexist on the Internet no problem 💅. BUT 🙈 we haven't looked at Scalable TCP closely enough to say in this doc that it's safe for widespread deployment on today's Internet. Not yet, bestie 💜.
Bandwidth Avg Cwnd w (pkts) Increase a(w) Decrease b(w)
--------- ----------------- ------------- -------------
1.5 Mbps 12.5 1 0.50
10 Mbps 83 0.4 0.125
100 Mbps 833 4.1 0.125
1 Gbps 8333 41.6 0.125
10 Gbps 83333 416.5 0.125
Table 10: Performance of a Scalable TCP connection.Table 10 shows how a Scalable TCP connection performs with 1500-byte packets, an RTT of 100 ms (including average queueing delay), and no competing traffic 🧚. The TCP connection is assumed to use delayed acknowledgements. The first column is the bandwidth, the second is the average congestion window you need to fill that bandwidth, and the third and fourth columns are the increase and decrease parameters ✨.
Even on a measly 10 Mbps link, Scalable TCP acts totally different from Standard TCP 👀. Its increase parameter is smaller than Standard TCP's, and its decrease is smaller too, 1/8-th instead of 1/2. So on 10 Mbps links, Scalable TCP increases less aggressively than Standard TCP or HighSpeed TCP, but it also backs off less aggressively 🦋.
On a 100 Mbps link, Scalable TCP has an increase parameter of roughly four segments per round-trip time, with the same decrease parameter of 1/8-th. If you compare Tables 9 and 10, you'll see that in this 100 Mbps scenario, HighSpeed TCP increases more aggressively than Scalable TCP 🔥.
Next up is fairness 🎀. The relative fairness between HighSpeed TCP and Standard TCP was shown in Table 5 earlier in this document, and the relative fairness between Scalable TCP and Standard TCP was shown in Table 8. Following the approach in Section 6, for a given packet drop rate p, where p < 10^-3, we can estimate the relative fairness between Scalable and HighSpeed TCP as W_Scalable/W_HighSpeed. That relative fairness is shown in Table 11 below. The bandwidth in the last column of Table 11 is the aggregate bandwidth of the two competing flows, given 100 ms round-trip times and 1500-byte packets 🍓.
Packet Drop Rate P Fairness Aggregate Window Bandwidth
------------------ -------- ---------------- ---------
10^-2 1.0 24 2.8 Mbps
10^-3 1.0 76 9.1 Mbps
10^-4 1.4 643 77.1 Mbps
10^-5 2.1 5595 671.4 Mbps
10^-6 3.1 50279 6.0 Gbps
10^-7 4.5 463981 55.7 Gbps
Table 11: Relative Fairness between the Scalable and HighSpeed
Response Functions.The second row of Table 11 says that when a Scalable TCP flow and a HighSpeed TCP flow compete on a 10 Mbps pipe with 100 ms RTTs, they basically get the same bandwidth 🫶. The next row says that on a 100 Mbps pipe with 100 ms RTTs, the Scalable TCP flow gets roughly 50% more bandwidth than the HighSpeed TCP flow 😭. Table 11 also shows the relative fairness in higher-bandwidth environments. Honestly, this fairness seems good enough that there shouldn't be problems with Scalable TCP and HighSpeed TCP sharing the same environment as Experimental variants of TCP 💖.
One thing that still needs more digging for Scalable TCP is convergence to fairness in environments with Drop-Tail queue management 🌈.
12. Related Work in HighSpeed TCP ✨
Sylvia Ratnasamy and Evandro de Souza separately ran simulations of HighSpeed TCP 🧚. Their work is [SA03], and it checks the fairness properties of HighSpeed TCP when it shares a link with Standard TCP 💅.
These simulations look at how fair HighSpeed TCP flows are when they compete with Standard TCP 👀. The simulation environment includes background forward and reverse-path TCP traffic that's limited by the TCP receive window, plus a small amount of forward and reverse-path web traffic from a web traffic generator. Most of the simulations so far look at performance on a simple dumbbell topology with a 1 Gbps link and a 50 ms propagation delay. They've been run with both Adaptive RED and DropTail queue management 🔥.
[SA03] also tries different numbers of competing flows, where the competing traffic is all Standard TCP, all HighSpeed TCP, or a mix of the two 🌸. With RED queue management, the relative fairness between Standard and HighSpeed TCP matches the relative fairness predicted in Table 5. With Drop Tail queues, it's more skewed, and the HighSpeed TCP flows get an even bigger share of the link bandwidth. This makes sense 🙈: with Active Queue Management at the congested link, the fraction of packet drops each flow gets should be roughly proportional to that flow's share of the link bandwidth, but that stops being true with Drop Tail queue management. We also noticed that relative fairness in Drop Tail simulations can sometimes depend on small details of the simulation scenario, and that Drop Tail simulations need special care to avoid phase effects [F92] 🍓.
[SA03] digs into the bandwidth stolen' by HighSpeed TCP from Standard TCP 🦋. It does this by looking at the fraction of link bandwidth that N Standard TCP flows get when they compete against N other Standard TCP flows, and comparing that to the fraction the N Standard TCP flows get when they compete against N HighSpeed TCP flows. For the 1 Gbps simulation scenarios dominated by long-lived traffic, a small number of Standard TCP flows can reach high link utilization, and the HighSpeed TCP flows can be seen as stealing bandwidth from the competing Standard TCP flows, just as Section 6 on the Fairness Implications of the HighSpeed Response Function predicted 💜. However, [SA03] shows that when even a small fraction of the link bandwidth is used by burstier, short TCP connections, the Standard TCP flows can't reach high link utilization. In that case the HighSpeed TCP flows aren't really stealing' bandwidth from Standard TCP. They're using bandwidth that otherwise wouldn't be used at all 🌈.
The conclusions of [SA03] are that "HighSpeed TCP behaved as forseen by its response function, and appears to be a real and viable option for use on high-speed wide area TCP connections." 🎀
Some future work that could be explored in more detail includes convergence times after new flows start up, recovery time after a transient outage, the response to sudden severe congestion, and investigating the potential for oscillations ⭐. We invite others to contribute to this work 🫶.
13. Relationship to other Work 🦋
Our assumption is that HighSpeed TCP will be used with the TCP SACK option, and also with the increased Initial Window of three or four segments, as allowed by [RFC3390]. For paths with substantial reordering, TCP performance would be greatly improved by some of the mechanisms that are still in the research stages for robust performance in the presence of reordered packets 🧠.
Our view is that HighSpeed TCP is largely orthogonal to proposals for higher PMTU (Path MTU) values [M02] 🌸. Unlike changes to the PMTU, HighSpeed TCP doesn't require any changes in the network or at the TCP receiver, and it works well on the current Internet ✨. Our assumption is that HighSpeed TCP would still be useful even with larger values for the PMTU. Unlike the current congestion window, the PMTU doesn't tell you anything about the bandwidth-delay product available to that particular flow 🙈.
A related approach is a virtual MTU, where the actual MTU of the path might be limited [VMSS,S02] 🧚. We haven't fully investigated the virtual MTU approach, so we don't explore it further in this document 💅.
14. Conclusions 💖
This document proposed HighSpeed TCP, a modification to TCP's congestion control mechanism for use with TCP connections that have large congestion windows 👑. We explored this proposal in simulations, others have explored HighSpeed TCP with experiments, and we believe HighSpeed TCP is safe to deploy on the current Internet 🔥. We'd love more analysis, simulations, and especially experimentation 🫶. More info on simulations and experiments is available on the HighSpeed TCP Web Page [HSTCP]. There are several independent implementations of HighSpeed TCP [D02,F03] and of Scalable TCP [K03] for further investigation 🌈.
15. Acknowledgements 🎀
HighSpeed TCP started as joint work with Sylvia Ratnasamy and Scott Shenker, who also initiated it ✨. Additional investigations of HighSpeed TCP were joint work with Evandro de Souza and Deb Agarwal 🧚. Thanks to Tom Dunigan for the implementation in the Linux 2.4.16 Web100 kernel, and for the experimentation with HighSpeed TCP that came out of it 🍓. We're grateful to the End-to-End Research Group, the members of the Transport Area Working Group, and the members of the IPAM program in Large Scale Communication Networks for their feedback 🦋. We thank Glenn Vinnicombe for framing the Linear response function in the parameters of HighSpeed TCP 💜. We're also grateful for contributions and feedback from the following people: Les Cottrell, Mitchell Erblich, Jeffrey Hsu, Tom Kelly, Chuck Jackson, Matt Mathis, Jitendra Padhye, Andrew Reiter, Stanislav Shalunov, Alex Solan, Paul Sutter, Brian Tierney, Joe Touch 🫶.
16. Normative References 🌸
[RFC2581] Allman, M., Paxson, V. and W. Stevens, "TCP Congestion Control", RFC 2581, April 1999.
17. Informative References 📚✨
Okay bestie, here's the reading list for anyone who wants to go deeper 🤓🫶 (all the sources cited in this doc, with links where they're given):
- [ABLLS03] A. Antony, J. Blom, C. de Laat, J. Lee, and W. Sjouw, "Microscopic Examination of TCP Flows over Transatlantic Links", iGrid2002 special issue, Future Generation Computer Systems, volume 19 issue 6 (2003), URL "http://www.science.uva.nl/~delaat/techrep-2003-2-tcp.pdf".
- [BBFS01] Deepak Bansal, Hari Balakrishnan, Sally Floyd, and Scott Shenker, "Dynamic Behavior of Slowly-Responsive Congestion Control Algorithms", SIGCOMM 2001, August 2001.
- [CC03] Fabrizio Coccetti and Les Cottrell, "TCP Stack Measurements on Lightly Loaded Testbeds", 2003. URL "http://www-iepm.slac.stanford.edu/monitoring/bulk/fast/".
- [CJ89] D. Chiu and R. Jain, "Analysis of the Increase and Decrease Algorithms for Congestion Avoidance in Computer Networks", Computer Networks and ISDN Systems, Vol. 17, pp. 1-14, 1989.
- [CO98] J. Crowcroft and P. Oechslin, "Differentiated End-to-end Services using a Weighted Proportional Fair Share TCP", Computer Communication Review, 28(3):53--69, 1998.
- [D02] Tom Dunigan, "Floyd's TCP slow-start and AIMD mods", URL "http://www.csm.ornl.gov/~dunigan/net100/floyd.html".
- [F03] Gareth Fairey, "High-Speed TCP", 2003. URL "http://www.hep.man.ac.uk/u/garethf/hstcp/".
- [F92] S. Floyd and V. Jacobson, "On Traffic Phase Effects in Packet-Switched Gateways, Internetworking: Research and Experience", V.3 N.3, September 1992, p.115-156. URL "http://www.icir.org/floyd/papers.html".
- [Fl03] Sally Floyd, "Re: [Tsvwg] taking NewReno (RFC 2582) to Proposed Standard", Email to the tsvwg mailing list, May 14, 2003.
- URLs "http://www1.ietf.org/mail-archive/working-groups/tsvwg/current/msg04086.html" and "http://www1.ietf.org/mail-archive/working-groups/tsvwg/current/msg04087.html".
- [FF98] Floyd, S., and Fall, K., "Promoting the Use of End-to-End Congestion Control in the Internet", IEEE/ACM Transactions on Networking, August 1999.
- [FRS02] Sally Floyd, Sylvia Ratnasamy, and Scott Shenker, "Modifying TCP's Congestion Control for High Speeds", May 2002. URL "http://www.icir.org/floyd/notes.html".
- [GRK99] Panos Gevros, Fulvio Risso and Peter Kirstein, "Analysis of a Method for Differential TCP Service". In Proceedings of the IEEE GLOBECOM'99, Symposium on Global Internet , December 1999, Rio de Janeiro, Brazil.
- [GV02] S. Gorinsky and H. Vin, "Extended Analysis of Binary Adjustment Algorithms", Technical Report TR2002-39, Department of Computer Sciences, The University of Texas at Austin, August 2002. URL "http://www.cs.utexas.edu/users/gorinsky/pubs.html".
- [HSTCP] HighSpeed TCP Web Page, URL "http://www.icir.org/floyd/hstcp.html".
- [J02] Amit Jain and Sally Floyd, "Quick-Start for TCP and IP", Work in Progress, 2002.
- [JWL03] Cheng Jin, David X. Wei and Steven H. Low, "FAST TCP for High-speed Long-distance Networks", Work in Progress, June 2003.
- [K03] Tom Kelly, "Scalable TCP: Improving Performance in HighSpeed Wide Area Networks", February 2003. URL "http://www-lce.eng.cam.ac.uk/~ctk21/scalable/".
- [KHR02] Dina Katabi, Mark Handley, and Charlie Rohrs, "Congestion Control for High Bandwidth-Delay Product Networks", SIGCOMM 2002.
- [M02] Matt Mathis, "Raising the Internet MTU", Web Page, URL "http://www.psc.edu/~mathis/MTU/".
- [Net100] The DOE/MICS Net100 project. URL "http://www.csm.ornl.gov/~dunigan/net100/".
- [NS] The NS Simulator, "http://www.isi.edu/nsnam/ns/".
- [RFC 1323] Jacobson, V., Braden, R. and D. Borman, "TCP Extensions for High Performance", RFC 1323, May 1992.
- [RFC3390] Allman, M., Floyd, S. and C., Partridge, "Increasing TCP's Initial Window", RFC 3390, October 2002.
- [RFC3448] Handley, M., Padhye, J., Floyd, S. and J. Widmer, "TCP Friendly Rate Control (TFRC): Protocol Specification", RFC 3448, January 2003.
- [SA03] Souza, E. and D.A., Agarwal, "A HighSpeed TCP Study: Characteristics and Deployment Issues", LBNL Technical Report LBNL-53215. URL "http://www.icir.org/floyd/hstcp.html".
- [S02] Stanislav Shalunov, "TCP Armonk", Work in Progress, 2002, URL "http://www.internet2.edu/~shalunov/tcpar/".
- [S03] Alex Solan, private communication, 2003.
- [VMSS] "Web100 at ORNL", Web Page, "http://www.csm.ornl.gov/~dunigan/netperf/web100.html".
- [Web100] The Web100 project. URL "http://www.web100.org/".
18. Security Considerations 🔐💅
Security bestie, the tea is short and sweet: this proposal makes no changes to the underlying security of TCP. Nothing shifts under TCP's feet on this front 🛡️✨
19. IANA Considerations 📋🌸
No IANA drama here! There are no IANA considerations regarding this document. 🎀
A. TCP's Loss Event Rate in Steady-State 🦋
Okay so this section is all about how many round-trip times pass between congestion events for a TCP flow with D-byte packets, where D=1500, as a function of the connection's average throughput B in bps 📈 To hit that average throughput B, a TCP connection with round-trip time R in seconds needs an average congestion window w of BR/(8D) segments. Literally the whole vibe is the math 🧮💖
In steady-state, TCP's average congestion window w is roughly 1.2/sqrt(p) segments. That's the same as a lost event at most once every 1/p packets, or at most once every 1/(pw) = w/1.5 round-trip times. If we plug in w, this becomes a loss event at most every (BR)/12D) round-trip times. Iconic! 😭
An an example (lol the original typo is giving), for R = 0.1 seconds and D = 1500 bytes, this gives B/180000 round-trip times between loss events 🍓
B. A table for a(w) and b(w). 🧁
This section gives a table for the increase and decrease parameters a(w) and b(w) for HighSpeed TCP, using the default values of Low_Window = 38, High_Window = 83000, High_P = 10^-7, and High_Decrease = 0.1. Main character table incoming 👑
w a(w) b(w)
---- ---- ----
38 1 0.50
118 2 0.44
221 3 0.41
347 4 0.38
495 5 0.37
663 6 0.35
851 7 0.34
1058 8 0.33
1284 9 0.32
1529 10 0.31
1793 11 0.30
2076 12 0.29
2378 13 0.28
2699 14 0.28
3039 15 0.27
3399 16 0.27
3778 17 0.26
4177 18 0.26
4596 19 0.25
5036 20 0.25
5497 21 0.24
5979 22 0.24
6483 23 0.23
7009 24 0.23
7558 25 0.22
8130 26 0.22
8726 27 0.22
9346 28 0.21
9991 29 0.21
10661 30 0.21
11358 31 0.20
12082 32 0.20
12834 33 0.20
13614 34 0.19
14424 35 0.19
15265 36 0.19
16137 37 0.19
17042 38 0.18
17981 39 0.18
18955 40 0.18
19965 41 0.17
21013 42 0.17
22101 43 0.17
23230 44 0.17
24402 45 0.16
25618 46 0.16
26881 47 0.16
28193 48 0.16
29557 49 0.15
30975 50 0.15
32450 51 0.15
33986 52 0.15
35586 53 0.14
37253 54 0.14
38992 55 0.14
40808 56 0.14
42707 57 0.13
44694 58 0.13
46776 59 0.13
48961 60 0.13
51258 61 0.13
53677 62 0.12
56230 63 0.12
58932 64 0.12
61799 65 0.12
64851 66 0.11
68113 67 0.11
71617 68 0.11
75401 69 0.10
79517 70 0.10
84035 71 0.10
89053 72 0.10
94717 73 0.09Table 12: Parameters for HighSpeed TCP. ✨
This table was computed with the following Perl program 🐪
$top = 100000;
$num = 38;
if ($num == 38) {
print " w a(w) b(w)\n";
print " ---- ---- ----\n";
print " 38 1 0.50\n";
$oldb = 0.50;
$olda = 1;
}
while ($num < $top) {
$bw = (0.1 -0.5)*(log($num)-log(38))/(log(83000)-log(38))+0.5;
$aw = ($num**2*2.0*$bw) / ((2.0-$bw)*$num**1.2*12.8);
if ($aw > $olda + 1) {
printf "%6d %5d %3.2f0, $num, $aw, $bw;
$olda = $aw;
}
$num ++;
}Table 13: Perl Program for computing parameters for HighSpeed TCP. 🎀
C. Exploring the time to converge to fairness. 🌈
This section gives the Perl program used to compute the congestion window growth during congestion avoidance. Watch the windows grow 🪟💜
$top = 2001;
$hswin = 1;
$regwin = 1;
$rtt = 1;
$lastrtt = 0;
$rttstep = 100;
if ($hswin == 1) {
print " RTT HS_Window Standard_TCP_Window0;
print " --- --------- -------------------0;
}
while ($rtt < $top) {
$bw = (0.1 -0.5)*(log($hswin)-log(38))/(log(83000)-log(38))+0.5;
$aw = ($hswin**2*2.0*$bw) / ((2.0-$bw)*$hswin**1.2*12.8);
if ($aw < 1) {
$aw = 1;
}
if ($rtt >= $lastrtt + $rttstep) {
printf "%5d %9d %10d0, $rtt, $hswin, $regwin;
$lastrtt = $rtt;
}
$hswin += $aw;
$regwin += 1;
$rtt ++;
}Table 14: Perl Program for computing the window in congestion avoidance. 🦋
Author's Address 💌
Sally Floyd ICIR (ICSI Center for Internet Research)
Phone: +1 (510) 666-2989 EMail: floyd@acm.org URL: http://www.icir.org/floyd/
Full Copyright Statement ⚖️🎀
Copyright (C) The Internet Society (2003). All Rights Reserved.
Okay, legal bit but make it cute 💅 You're allowed to copy and hand out this document, and you can write derivative works that comment on or explain it or help people implement it. You can publish and distribute those in whole or in part with no restrictions of any kind, as long as the copyright notice and this paragraph stay on every copy and derivative work. The document itself may not be modified in any way, such as removing the copyright notice or the references to the Internet Society or other Internet organizations. The one exceptions are when you need it for developing Internet standards (then the copyright procedures in the Internet Standards process must be followed) or when you need to translate it into languages other than English. Those permissions are perpetual and will not be revoked by the Internet Society or its successors or assignees 💖
And the not-so-fun disclaimer, which we have to keep exactly as it is: This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 🙈
Acknowledgement 🫶
Big thank-you energy! Funding for the RFC Editor function is currently provided by the Internet Society. 🌸💜
📦 where this page comes from
- ietf: rfc3649 2003-12
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms