RFC 2914

Congestion Control Principles

📅 published September 2000, from ietf

RFC 2914: Congestion Control Principles 💖

  • Network Working Group 🌸
  • RFC: 2914
  • Title: Congestion Control Principles
  • Author: S. Floyd, ACIRI
  • Date: September 2000
  • Category: Best Current Practice (BCP: 41)
  • Obsoletes / Updates: not listed in the title block 🫶

Status of this Memo ✨

okay so this doc is an Internet Best Current Practice for the Internet Community 🧚, and it's basically asking everyone to talk about it and send in suggestions for making it better. Distribution of this memo is unlimited, so share it with your whole group chat 💅

Copyright (C) The Internet Society (2000). All Rights Reserved. 💜

Abstract 🌸

the goal here is to explain why the Internet needs congestion control and what counts as correct congestion control 👑. One big goal is to show the dangers of skipping proper congestion control (the main character danger is real, bestie 💀). The second goal is to talk about the IETF's role in standardizing new congestion control protocols 🦋.

1. Introduction 🧚

this doc borrows heavily from earlier RFCs, and in some cases it literally reproduces whole sections of their text [RFC2309, RFC2357]. We also borrowed a lot from earlier publications about why end-to-end congestion control is needed [FF99] 🫶.

2. Current standards on congestion control 🍓

IETF standards on end-to-end congestion control are focused in one of two ways. Some are about specific protocols, like TCP [RFC2581] and reliable multicast protocols [RFC2357]. Others are about the syntax and semantics of the messages that end nodes and routers send each other about congestion information, like Explicit Congestion Notification [RFC2481], or about the quality-of-service they want (diff-serv) 💖.

There's also an Informational RFC called "Recommendations on Queue Management and Congestion Avoidance in the Internet" [RFC2309] that talks about end-to-end congestion control. RFC 2309 recommends deploying active queue management mechanisms in routers, and it says design work should keep going on router mechanisms that deal with flows that are unresponsive to congestion notification 👀. We freely borrow some of its general discussion of end-to-end congestion control, because it's giving good ideas.

Now, this document is different from those RFCs, because it's a more general discussion of the principles of congestion control ✨. One of the keys to the Internet's success has been the congestion avoidance mechanisms of TCP 👑. TCP is still the dominant transport protocol on the Internet, but it isn't ubiquitous anymore. More and more applications, for one reason or another, choose not to use TCP 🫠. That includes multicast traffic, and also unicast traffic like streaming multimedia that doesn't need reliability. It also includes traffic like DNS or routing messages, which are short transfers that are critical to running the network 🌈. Much of this traffic doesn't use any form of bandwidth reservations or end-to-end congestion control. Continued use of end-to-end congestion control by best-effort traffic is critical for keeping the Internet stable 🔥.

This document also discusses the general role of the IETF in standardizing new congestion control protocols 💅.

Differentiated services and integrated services are not addressed in this document 🙈. Some categories of integrated or differentiated services include a guarantee by the network of end-to-end bandwidth, so they don't need end-to-end congestion control mechanisms. Not every flow needs the same kind of protection, iykyk 💜.

3. The development of end-to-end congestion control. 🦋

3.1. Preventing congestion collapse. 💀

The Internet protocol architecture is based on a connectionless end-to-end packet service that uses the IP protocol 🌸. Its connectionless design, flexibility, and robustness have been amply demonstrated, and literally everyone knows it works. But these advantages aren't free: careful design is required to provide good service under heavy load 😭. Lack of attention to the dynamics of packet forwarding can cause severe service degradation, or what's called "Internet meltdown" 🙈. This was first observed during the early growth phase of the Internet in the mid 1980s [RFC896], and it's technically called "congestion collapse" 💀.

The original specification of TCP [RFC793] included window-based flow control, so the receiver could govern how much data the sender sent 🫶. This flow control was used to stop the receiver's data buffer space (for that connection) from overflowing. [RFC793] reported that segments could be lost due to errors or to network congestion, but it didn't include dynamic adjustment of the flow-control window in response to congestion 😬.

The original fix for Internet meltdown came from Van Jacobson 👑. Starting in 1986, Jacobson developed the congestion avoidance mechanisms that are now required in TCP implementations [Jacobson88, RFC 2581] ✨. These mechanisms run in the hosts and make TCP connections "back off" during congestion 🧚. We say TCP flows are "responsive" to congestion signals (that is, dropped packets) from the network 🦋. These TCP congestion avoidance algorithms are what prevent the congestion collapse of today's Internet, so they deserve a standing ovation 👏.

But that's not the end of the story 👀. A lot of research on Internet dynamics has been done since 1988, and the Internet has grown a ton. It's become clear that the TCP congestion avoidance mechanisms [RFC2581] are necessary and powerful, but they aren't sufficient to provide good service in all circumstances 😭. In addition to developing new congestion control mechanisms [RFC2357], router-based mechanisms are in development that complement the endpoint congestion avoidance mechanisms 🌈.

A major issue that still needs to be addressed is the potential for future congestion collapse of the Internet caused by flows that don't use responsible end-to-end congestion control 🚨. RFC 896 [RFC896] suggested back in 1984 that gateways should detect and `squelch' misbehaving hosts. It said: "Failure to respond to an ICMP Source Quench message, though, should be regarded as grounds for action by a gateway to disconnect a host. Detecting such failure is non-trivial but is a worthwhile area for further research." 🙈 Current papers still propose that routers detect and penalize flows that aren't using acceptable end-to-end congestion control [FF99]. Honestly, the plot keeps going 🍿.

3.2. Fairness 🎀

Beyond congestion collapse, there's also a concern about `fairness' for best-effort traffic 💅. Because TCP "backs off" during congestion, a large number of TCP connections can share a single congested link in a way that splits bandwidth reasonably equitably among similarly situated flows 🫶. That equitable sharing depends on all flows running compatible congestion control algorithms. For TCP, this means congestion control algorithms that conform to the current TCP specification [RFC793, RFC1122, RFC2581] 📜.

Fairness among competing flows has become more important for a few reasons 💜. First, with window scaling [RFC1323], individual TCPs can use high bandwidth even over high-propagation-delay paths 🚀. Second, with the growth of the web, Internet users increasingly want high-bandwidth and low-delay communications, instead of the leisurely transfer of a long file in the background 😤. The growth of best-effort traffic that doesn't use TCP makes this fairness concern even bigger when there's congestion 🌸.

The Internet's popularity has caused a proliferation of TCP implementations 🌈. Some of these may fail to implement the TCP congestion avoidance mechanisms correctly because of poor implementation [RFC2525] 🙃. Others may be deliberately built with congestion avoidance algorithms that are more aggressive about using bandwidth than other TCP implementations. That would let a vendor claim to have a "faster TCP" 🏎️. The logical result of that would be a spiral of increasingly aggressive TCP implementations, or increasingly aggressive transport protocols. It would lead to the point where there's effectively no congestion avoidance and the Internet is chronically congested 😵.

There's a well-known way to get more aggressive performance without even changing the transport protocol: change the level of granularity by opening multiple connections to the same place, which some Web browsers have done in the past 🧠. So instead of a spiral of increasingly aggressive transport protocols, we'd get a spiral of increasingly aggressive web browsers, or increasingly aggressive applications 💀.

This raises the question of the appropriate granularity of a "flow". Here, we define a flow' as the level of granularity that's appropriate for applying both fairness and congestion control ✨. From RFC 2309: "There are a few natural' answers: 1) a TCP or UDP connection (source address/port, destination address/port); 2) a source/destination host pair; 3) a given source host or a given destination host. We would guess that the source/destination host pair gives the most appropriate granularity in many circumstances. The granularity of flows for congestion management is, at least in part, a policy question that needs to be addressed in the wider IETF community." 🫶

Borrowing from RFC 2309 again, we use the term "TCP-compatible" for a flow that behaves under congestion like a flow produced by a conformant TCP 🦋. A TCP-compatible flow is responsive to congestion notification. In steady state, it uses no more bandwidth than a conformant TCP running under comparable conditions (drop rate, RTT, MTU, etc.) 📊.

It's convenient to divide flows into three classes 🍰:

  • (1) TCP-compatible flows
  • (2) unresponsive flows, i.e., flows that do not slow down when congestion occurs
  • (3) flows that are responsive but are not TCP-compatible

The last two classes contain more aggressive flows that pose significant threats to Internet performance, as we'll discuss below 😬.

Besides steady-state fairness, the fairness of the initial slow-start is also a concern 🐢. One concern is the transient effect on other flows of a flow with an overly aggressive slow-start procedure. Slow-start performance matters a lot for the many flows that are short-lived and have only a small amount of data to transfer 💖.

3.3. Optimizing performance regarding throughput, delay, and loss. 🚀

Besides preventing congestion collapse and addressing fairness, a third reason for a flow to use end-to-end congestion control is to optimize its own performance for throughput, delay, and loss 💅. In some circumstances, like environments with high statistical multiplexing, the delay and loss rate a flow experiences are mostly independent of its own sending rate. But in environments with lower levels of statistical multiplexing, or with per-flow scheduling, the delay and loss rate a flow experiences is partly a function of its own sending rate 🌈. So a flow can use end-to-end congestion control to limit the delay or loss its own packets experience 🫶.

We'd note, though, that in an environment like the current best-effort Internet, concerns about congestion collapse and fairness with competing flows limit the range of congestion control behaviors available to a flow 🙈. Main character energy has limits, bestie 👑✨

  1. The role of the standards process 🎀

okay so bestie, this part is literally about who gets to decide what counts as a "standard" for a transport protocol 🧚 and the answer is: it's not just the stuff that makes two computers able to talk to each other 💅 it's ALSO the stuff that's critical for performance! like in TCP, cutting the congestion window when a packet gets dropped is a must-standardize thing 🔥 but the implementation-specific details that don't mess with interoperability or performance? those don't need standardizing, no cap ✨ the example here is TCP's Fast Recovery procedure after a Fast Retransmit [RFC2582] 🌸 and the appendix goes deeper into this if you're obsessed like me 🙈

4.1. The development of new transport protocols. 💖

okay so the standards process for new transport protocols is not just about congestion collapse, it's also about making sure there's no congestion control arms race between competing protocols 😭 like, nobody wants protocols fighting each other for bandwidth 🫶 RFC 2357 [RFC2357] is the example here, where the TSV Area Directors and their Directorate lay out the criteria for publishing Internet-Drafts on reliable multicast transport protocols as RFCs 🦋 and they say, quote:

"A particular concern for the IETF is the impact of reliable multicast traffic on other traffic in the Internet in times of congestion, in particular the effect of reliable multicast traffic on competing TCP traffic.... The challenge to the IETF is to encourage research and implementations of reliable multicast, and to enable the needs of applications for reliable multicast to be met as expeditiously as possible, while at the same time protecting the Internet from the congestion disaster or collapse that could result from the widespread use of applications with inappropriate reliable multicast mechanisms."

the technical criteria for reliable multicast RFCs are also super clear: 👑 the list includes:

"Is there a congestion control mechanism? How well does it perform? When does it fail? Note that congestion control mechanisms that operate on the network more aggressively than TCP will face a great burden of proof that they don't threaten network stability."

and this is giving "it applies to more than just multicast" energy 💅 it's reasonable to expect these same concerns about new transport protocols and competing traffic to apply to unreliable unicast, reliable unicast, and unreliable multicast traffic as well 🌈

4.2. Application-level issues that affect congestion control 🍓

the browser-opens-a-bunch-of-connections-to-one-destination thing has already been addressed by RFC 2616 [RFC2616] 🧚 in its Section 8.1.4, it says, quote:

"Clients that use persistent connections SHOULD limit the number of simultaneous connections that they maintain to a given server. A single-user client SHOULD NOT maintain more than 2 connections with any server or proxy."

so yeah, that SHOULD and SHOULD NOT are doing the heavy lifting here 😤 they're the rules, bestie 🎀

4.3. New developments in the standards process 🦋

the most obvious IETF developments that could change how congestion control evolves are integrated and differentiated services [RFC2212, RFC2475] and Explicit Congestion Notification (ECN) [RFC2481] 🌟 but honestly, lowkey less dramatic stuff will affect congestion control too 👀

one of those is Endpoint Congestion Management [BS00] 💜 the goal is to let multiple concurrent flows from a sender to the same receiver share congestion control state 🫶 this is called a Congestion Manager, and it lets multiple connections to the same destination act as ONE flow for end-to-end congestion control 😭 so individual connections that are slow-starting can take advantage of previous info about the congestion state of the path ✨ it could also remove the congestion control dangers of multiple flows between the same source/destination pair, and maybe even let a browser open lots of simultaneous connections to the same destination 🙈 iykyk

  1. A description of congestion collapse 💀

this section gets into congestion collapse from undelivered packets, and shows how unresponsive flows could contribute to it on the Internet 😤 it draws heavily on material from [FF99] 📚

informally, congestion collapse happens when more network load leads to LESS useful work getting done by the network 💀 like, the opposite of what you'd want 🙃 as Section 3 discussed, congestion collapse was first reported in the mid 1980s [RFC896] 🕰️ and it was mostly because TCP connections were unnecessarily retransmitting packets that were either still in transit or had already been received 😭 we call that classical congestion collapse 💅 it's a stable condition that can drop throughput to a small fraction of normal [RFC896] 😬 but the timer improvements and congestion control mechanisms in modern TCP implementations have generally fixed those problems [Jacobson88] ✨

the second form is congestion collapse from undelivered packets 🚨 this happens when bandwidth gets wasted on packets that get dropped before they reach their ultimate destination 💔 it's probably the biggest unresolved danger with respect to congestion collapse on the Internet today 👑 different scenarios can cause different degrees of collapse, measured by the fraction of the congested links' bandwidth that's actually used for productive work 📊 the danger comes mainly from the growing use of open-loop applications that don't use end-to-end congestion control 🫠 and even more destructive would be best-effort applications that increase their sending rate when the packet drop rate goes up (like automatically using an increased level of FEC) 🙀

Table 1 gives results from a scenario with congestion collapse from undelivered packets 🌸 where scarce bandwidth gets wasted by packets that never reach their destination 💔 the simulation has three TCP flows and one UDP flow competing over a congested 1.5 Mbps link 🧚 the access links for all nodes are 10 Mbps, EXCEPT the access link to the UDP flow's receiver, which is 128 Kbps (only 9% of the shared link's bandwidth) 🥺 when the UDP source rate goes above 128 Kbps, most UDP packets get dropped at the output port to that final link 💀

     UDP
     Arrival   UDP       TCP       Total
     Rate      Goodput   Goodput   Goodput
    --------------------------------------
      0.7       0.7      98.5      99.2
      1.8       1.7      97.3      99.1
      2.6       2.6      96.0      98.6
      5.3       5.2      92.7      97.9
      8.8       8.4      87.1      95.5
     10.5       8.4      84.8      93.2
     13.1       8.4      81.4      89.8
     17.5       8.4      77.3      85.7
     26.3       8.4      64.5      72.8
     52.6       8.4      38.1      46.4
     58.4       8.4      32.8      41.2
     65.7       8.4      28.5      36.8
     75.1       8.4      19.7      28.1
     87.6       8.4      11.3      19.7
    105.2       8.4       3.4      11.8
    131.5       8.4       2.4      10.7

Table 1. A simulation with three TCP flows and one UDP flow. 🎀

Table 1 shows the UDP arrival rate from the sender 📤, the UDP goodput (defined as bandwidth delivered to the receiver) 📥, the TCP goodput (as delivered to the TCP receivers) 🧁, and the aggregate goodput on the congested 1.5 Mbps link 🌈 each rate is a fraction of the congested link's bandwidth 💅 as the UDP source rate goes up, TCP goodput drops roughly linearly while UDP goodput stays basically constant 📉 so the UDP flow just keeps upping its offered load and its only real effect is hurting TCP and aggregate goodput 😤 on the congested link, UDP ends up "wasting" the bandwidth TCP could've used, dragging total goodput down to a tiny fraction of that link 💔

the simulations in Table 1 show both unfairness AND congestion collapse 🫠 as [FF99] discusses, compatible congestion control isn't the only way to get fairness, since per-flow scheduling at congested routers is another mechanism that guarantees fairness 🧚 but as [FF99] also discusses, per-flow scheduling can NOT be relied on to prevent congestion collapse 😬

there are only two alternatives for getting rid of the danger of congestion collapse from undelivered packets 🙌 the first is effective end-to-end congestion control by the end nodes 💅 more specifically, a flow has to avoid a pattern of significant losses at links downstream from the first congested link on its path 🫶 (here, any link counts as a "congested link" if any flow is using bandwidth that would otherwise go to other traffic on that link 📌) since an end-node usually can't tell a path with one congested link from a path with several, the most reliable way for a flow to avoid significant losses at a downstream congested link is to use end-to-end congestion control and reduce its sending rate when there's loss 🔥

the second alternative is a network guarantee that packets accepted at a congested link will be delivered all the way to the receiver [RFC2212, RFC2475] 🌈 and the choice between end-to-end congestion control and end-to-end bandwidth guarantees doesn't have to be either/or 🙅‍♀️ congestion collapse can be prevented by effective end-to-end congestion control for SOME of the traffic, plus network bandwidth guarantees for the REST 👑✨

  1. Forms of end-to-end congestion control 🦋

this document has talked about congestion collapse and fairness with TCP as concerns for new forms of congestion control 💖 but that does NOT mean those two concerns force ALL best-effort traffic to use congestion control based on TCP's Additive-Increase Multiplicative-Decrease (AIMD) algorithm (cutting the sending rate in half on each packet drop) 🙅‍♀️ this section separately looks at what these two concerns mean on their own 🧚

6.1. End-to-end congestion control for avoiding congestion collapse. 💅

okay so congestion collapse from undelivered packets needs a very specific combo 💀: a high sending rate, multiple congested links, and a persistent high packet drop rate at the downstream link. The packets are basically wasting bandwidth only to get dropped downstream anyway 😭. So if each flow only goes through one congested link, or only a few packets get dropped past the first congested link, this kind of collapse literally can't happen 🙅‍♀️. Any congestion control that successfully stops you from sending hard when lots of packets are getting dropped should be sufficient to avoid it ✨.

ngl, adding Explicit Congestion Notification (ECN) to IP doesn't fully solve the problem on its own 🫠. ECN lets routers set a bit in packet headers to tell the end-nodes about congestion, so they don't have to rely on packet drops for that signal 🦋. BUT with ECN, marking only replaces dropping in times of moderate congestion. When congestion is heavy and a router's buffers overflow, the router has no choice but to drop arriving packets 😩. So the danger of congestion collapse for best-effort traffic is still there, periodt. 🌈

6.2. End-to-end congestion control for fairness with TCP. 🎀

The concern in [RFC2357] about fairness with TCP is a significant, though not crippling, constraint on which end-to-end congestion control mechanisms are viable for best-effort traffic 💜. If every congested link used per-flow scheduling, flows would be isolated from each other, and nobody would need congestion control to be TCP-compatible 🧚. Same vibe with differentiated services: if a diff-serv class gets scheduled in isolation from best-effort traffic, that class could end up with congestion control that doesn't have to be TCP-compatible ✨. A pricing-controlled environment, or a diff-serv class with its own pricing paradigm, could also supersede the fairness-with-TCP concern 🍓.

BUT for the current Internet, where other best-effort traffic can compete with TCP traffic in a FIFO queue, not being fair to TCP could let one flow `starving out' another flow in a time of high congestion, as was illustrated in Table 1 above 😭🫠.

That said, TCP-compatible congestion control is NOT limited to AIMD with the same increase/decrease parameters as TCP 👑. Other TCP-compatible options include:

  • rate-based variants of AIMD;
  • AIMD with different sets of increase/decrease parameters that give the same steady-state behavior;
  • equation-based congestion control, where the sender adjusts its sending rate in response to information about the long-term packet drop rate;
  • layered multicast, where receivers subscribe and unsubscribe from layered multicast groups; and
  • possibly other forms that we have not yet begun to consider. 🔮

7. Acknowledgements 🫶

Much of this document draws directly on previous RFCs about end-to-end congestion control 📚. It's meant to be a summary of ideas that have been discussed for many years, by many people 🌸. Huge thanks to the members of the End-to-End Research Group, the Reliable Multicast Research Group, and the Transport Area Directorate 👑. The Transport Area Working Group also gave discussion and feedback, and this document benefited from it 💖. Particular thanks go to Mark Allman for feedback on an earlier version 🦋.

8. References 📖

  • [BS00] Balakrishnan H. and S. Seshan, "The Congestion Manager", Work in Progress.
  • [DMKM00] Dawkins, S., Montenegro, G., Kojo, M. and V. Magret, "End-to-end Performance Implications of Slow Links", Work in Progress.
  • [FF99] Floyd, S. and K. Fall, "Promoting the Use of End-to-End Congestion Control in the Internet", IEEE/ACM Transactions on Networking, August 1999. URL http://www.aciri.org/floyd/end2end-paper.html
  • [HPF00] Handley, M., Padhye, J. and S. Floyd, "TCP Congestion Window Validation", RFC 2861, June 2000.
  • [Jacobson88] V. Jacobson, Congestion Avoidance and Control, ACM SIGCOMM '88, August 1988.
  • [RFC793] Postel, J., "Transmission Control Protocol", STD 7, RFC 793, September 1981.
  • [RFC896] Nagle, J., "Congestion Control in IP/TCP", RFC 896, January 1984.
  • [RFC1122] Braden, R., Ed., "Requirements for Internet Hosts -- Communication Layers", STD 3, RFC 1122, October 1989.
  • [RFC1323] Jacobson, V., Braden, R. and D. Borman, "TCP Extensions for High Performance", RFC 1323, May 1992.
  • [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
  • [RFC2212] Shenker, S., Partridge, C. and R. Guerin, "Specification of Guaranteed Quality of Service", RFC 2212, September 1997.
  • [RFC2309] Braden, R., Clark, D., Crowcroft, J., Davie, B., Deering, S., Estrin, D., Floyd, S., Jacobson, V., Minshall, G., Partridge, C., Peterson, L., Ramakrishnan, K.K., Shenker, S., Wroclawski, J., and L. Zhang, "Recommendations on Queue Management and Congestion Avoidance in the Internet", RFC 2309, April 1998.
  • [RFC2357] Mankin, A., Romanow, A., Bradner, S. and V. Paxson, "IETF Criteria for Evaluating Reliable Multicast Transport and Application Protocols", RFC 2357, June 1998.
  • [RFC2414] Allman, M., Floyd, S. and C. Partridge, "Increasing TCP's Initial Window", RFC 2414, September 1998.
  • [RFC2475] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z. and W. Weiss, "An Architecture for Differentiated Services", RFC 2475, December 1998.
  • [RFC2481] Ramakrishnan K. and S. Floyd, "A Proposal to add Explicit Congestion Notification (ECN) to IP", RFC 2481, January 1999.
  • [RFC2525] Paxson, V., Allman, M., Dawson, S., Fenner, W., Griner, J., Heavens, I., Lahey, K., Semke, J. and B. Volz, "Known TCP Implementation Problems", RFC 2525, March 1999.
  • [RFC2581] Allman, M., Paxson, V. and W. Stevens, "TCP Congestion Control", RFC 2581, April 1999.
  • [RFC2582] Floyd, S. and T. Henderson, "The NewReno Modification to TCP's Fast Recovery Algorithm", RFC 2582, April 1999.
  • [RFC2616] Fielding, R., Gettys, J., Mogul, J., Frystyk, H., Masinter, L., Leach, P. and T. Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1", RFC 2616, June 1999.
  • [SCWA99] S. Savage, N. Cardwell, D. Wetherall, and T. Anderson, TCP Congestion Control with a Misbehaving Receiver, ACM Computer Communications Review, October 1999.
  • [TCPB98] Hari Balakrishnan, Venkata N. Padmanabhan, Srinivasan Seshan, Mark Stemm, and Randy H. Katz, TCP Behavior of a Busy Internet Server: Analysis and Improvements, IEEE Infocom, March 1998. Available from: "http://www.cs.berkeley.edu/~hari/papers/infocom98.ps.gz".
  • [TCPF98] Dong Lin and H.T. Kung, TCP Fast Recovery Strategies: Analysis and Improvements, IEEE Infocom, March 1998. Available from: "http://www.eecs.harvard.edu/networking/papers/infocom-tcp-final-198.pdf".

9. TCP-Specific issues 🔥

In this section we get into the particulars of TCP congestion control 🧠. We're showing one realization of the congestion control principles, including some of the details that pop up when you bake them into a production transport protocol 🍰.

9.1. Slow-start. 🐢

The TCP sender can not open a new connection by blasting a big burst of data all at once (e.g., a receiver's advertised window) 🙅‍♀️. It's limited by a small initial value for the congestion window 🌱. During slow-start, the TCP sender can increase its sending rate by at most a factor of two in one roundtrip time ✨. Slow-start ends when congestion is detected, or when the sender's congestion window is greater than the slow-start threshold ssthresh 🛑.

An issue that potentially affects global congestion control, and therefore has been explicitly addressed in the standards process, is an increase in the value of the initial window [RFC2414,RFC2581] 📈.

Issues that have not been addressed in the standards process, and are generally considered not to require standardization, include the use (or non-use) of rate-based pacing, and mechanisms for ending slow-start early, before the congestion window reaches ssthresh 🫧. Such mechanisms result in slow-start behavior that is as conservative or more conservative than standard TCP 🐌💜.

9.2. Additive Increase, Multiplicative Decrease. 🦋

In the absence of congestion, the TCP sender increases its congestion window by at most one packet per roundtrip time 🌷. In response to a congestion indication, the TCP sender decreases its congestion window by half 📉. (More precisely, the new congestion window is half of the minimum of the congestion window and the receiver's advertised window.) 🧮

An issue that potentially affects global congestion control, and therefore would be likely to be explicitly addressed in the standards process, would include a proposed addition of congestion control for the return stream of `pure acks' 🔁.

An issue that has not been addressed in the standards process, and is generally not considered to require standardization, would be a change to the congestion window to apply as an upper bound on the number of bytes presumed to be in the pipe, instead of applying as a sliding window starting from the cumulative acknowledgement 🌊. (Clearly, the receiver's advertised window applies as a sliding window starting from the cumulative acknowledgement field, because packets received above the cumulative acknowledgement field are held in TCP's receive buffer, and have not been delivered to the application. However, the congestion window applies to the number of packets outstanding in the pipe, and does not necessarily have to include packets that have been received out-of-order by the TCP receiver.) 🫶

9.3. Retransmit timers ⏰

okay so the TCP sender sets a retransmit timer to figure out that a packet got dropped somewhere in the network 🕵️‍♀️ when that timer goes off, the sender decides a packet is lost, sets ssthresh to half of the current window, and drops back into slow-start 🐢 while it resends the lost packet. if the timer expires because no acknowledgement came back for a packet that was already retransmitted, the timer also gets "backed-off" 📈, which means the next timeout interval is doubled. it's giving patience, bestie 💅

now here's the tea ☕ an issue that could affect global congestion control (so the standards process would probably want to address it) is a modified timer mechanism that causes way more retransmit timers to expire too early 😬 like, the acknowledgement just hasn't shown up yet, but nothing was actually dropped. that matters because premature timeouts can lead to more packets getting sent unnecessarily over a congested link, and nobody wants that. it's not me being dramatic, it's literally the whole reason this matters 🙈

9.4. Fast Retransmit and Fast Recovery 🚀

okay this one is so iconic 🔥 after the TCP sender sees three duplicate acknowledgements, it infers that a packet was lost. then it sets ssthresh to half of the current window, shrinks the congestion window to at most half of the previous window, and retransmits the lost packet. no waiting around for the timer, we move 🏃‍♀️💨

the spicy part 🌶️ an issue that could affect global congestion control (again, the kind of thing the standards process would explicitly address) is a proposal, if there even is one, for deciding a packet is lost after seeing only one or two duplicate acknowledgements. if that's poorly designed, it could cause more packets to be sent unnecessarily over a congested path. so, careful, the vibes need to be right 🫶

there's also an issue that has not been addressed in the standards process and wouldn't need to be standardized 🎀 this is a proposal to send a "new" or presumed-lost packet in response to a duplicate or partial acknowledgement, as long as the congestion window allows it. the example is sending a new packet after just one duplicate acknowledgement, to keep the `ack clock' going in case no further acknowledgements would have arrived. this is a beneficial change that doesn't involve interoperability and doesn't affect global congestion control, so vendors could implement it on their own without needing the IETF standards process. no permission slip required ✨

(this one has actually been looked at in [DMKM00], which suggests that "researchers may wish to experiment with injecting new traffic into the network when duplicate acknowledgements are being received, as described in [TCPB98] and [TCPF98]." 🧪)

9.5. Other aspects of TCP congestion control. 🌈

there are other parts of TCP congestion control that none of the sections above covered. one of them is how TCP recovers from an idle or application-limited period [HPF00] 😌 a whole other chapter, honestly 📖

10. Security Considerations 🛡️

okay, this section is the main character moment 👑 this whole document has been about the risks tied to congestion control, or to not having congestion control at all. Section 3.2 talks about the unfairness that can happen if competing flows don't use compatible congestion control mechanisms, and Section 5 covers the dangers of congestion collapse if flows don't use end-to-end congestion control 💀

since this document doesn't propose any specific congestion control mechanisms, it's not necessary to lay out specific security measures for congestion control either. BUT (and this is a big but 🙈) there are a bunch of security considerations tied to congestion control that should be thought about in IETF documents. for example, individual congestion control mechanisms should be as robust as possible against attempts by individual end-nodes to subvert end-to-end congestion control [SCWA99]. this is an especially big deal for multicast congestion control, because the traffic gets distributed so far and wide, and there are more chances for individual receivers to fail to report congestion 😭

RFC 2309 also talked about the potential dangers unresponsive flows pose to the Internet 🚨 those are flows that don't reduce their sending rate when congestion shows up, and the RFC describes the need for mechanisms in the network to deal with flows that don't respond to congestion notification. we'd like to note that there's still a need for research, engineering, measurement, and deployment in these areas 🔬

here's the comforting part 🫶 because the Internet aggregates huge numbers of flows, the risk to the whole infrastructure from subverting the congestion control of a few individual flows is limited. the real risk would come from the widespread deployment of many end-nodes subverting end-to-end congestion control. so, the scary scenario is a lot of people doing it, not just one 😳

Author's Address 💌

Sally Floyd 👩‍🔬 AT&T Center for Internet Research at ICSI (ACIRI)

Phone: +1 (510) 642-4274 x189 EMail: floyd@aciri.org URL: http://www.aciri.org/floyd/

Copyright (C) The Internet Society (2000). All Rights Reserved. 📜

This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English.

The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns.

This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Acknowledgement 🙏

Funding for the RFC Editor function is currently provided by the Internet Society. 💖

📦 where this page comes from