RFC 2581
TCP Congestion Control
RFC 2581: TCP Congestion Control π
Okay bestie, this one is a Standards Track doc from April 1999 β¨ It came from the Network Working Group and is numbered RFC 2581. Three authors wrote it: M. Allman (NASA Glenn/Sterling Software), V. Paxson (ACIRI / ICSI), and W. Stevens (Consultant). It obsoletes RFC 2001 π
Status of this Memo
Okay so this doc is an Internet standards track protocol spec for the Internet community πΈ It's asking for discussion and suggestions for improvements. If you want the current standardization state, check the "Internet Official Protocol Standards" (STD 1) list. Distribution is unlimited, so share away π¦
Copyright Notice
Copyright (C) The Internet Society (1999). All Rights Reserved. π
Abstract
This doc defines TCP's four intertwined congestion control algorithms: slow start, congestion avoidance, fast retransmit, and fast recovery π₯ It also spells out how TCP should start sending again after a pretty long idle period, and talks about different ways to generate acknowledgments. Main character energy β¨
1. Introduction
Okay so this section sets the scene π§ It specifies four TCP [Pos81] congestion control algorithms: slow start, congestion avoidance, fast retransmit, and fast recovery. These were devised in [Jac88] and [Jac90], and their use with TCP was standardized in [Bra89]. Iconic lineage, no cap π
This document is an update of [Ste97]. On top of the congestion control algorithms, it covers what TCP connections should do after a relatively long idle period, and it clears up some stuff about TCP ACK generation too π«Ά
Note that [Ste94] has examples of these algorithms in action, and [WS95] explains the source code for the BSD implementation of them π
Here's the layout. Section 2 has the definitions you'll see all over the place. Section 3 is the actual spec for the congestion control algorithms. Section 4 goes through concerns related to those algorithms, and section 5 covers security considerations π
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [Bra97]. Those words are the ones that carry real weight here, so read them like they're serious π
2. Definitions
Okay this section defines a bunch of terms we'll use for the rest of the doc, so let's get them locked in π
SEGMENT: A segment is ANY TCP/IP data or acknowledgment packet (or both). Literally any of them π¦
SENDER MAXIMUM SEGMENT SIZE (SMSS): The SMSS is the size of the largest segment the sender can transmit. This value can come from the network's maximum transmission unit, the path MTU discovery [MD90] algorithm, RMSS (more on that below), or other factors. The size does not include the TCP/IP headers and options π
RECEIVER MAXIMUM SEGMENT SIZE (RMSS): The RMSS is the size of the largest segment the receiver is willing to accept. This is the value in the MSS option the receiver sends during connection startup. Or, if the MSS option isn't used, it's 536 bytes [Bra89]. The size does not include the TCP/IP headers and options π
FULL-SIZED SEGMENT: A segment that holds the maximum number of data bytes allowed, meaning a segment with SMSS bytes of data. Full send, full vibes π₯
RECEIVER WINDOW (rwnd): The most recently advertised receiver window. Simple, but important π§
CONGESTION WINDOW (cwnd): A TCP state variable that limits how much data a TCP can send. At any given time, a TCP MUST NOT send data with a sequence number higher than the sum of the highest acknowledged sequence number and the minimum of cwnd and rwnd. This is the big one, bestie π
INITIAL WINDOW (IW): The initial window is the size of the sender's congestion window after the three-way handshake is completed πΈ
LOSS WINDOW (LW): The loss window is the size of the congestion window after a TCP sender detects loss using its retransmission timer π
RESTART WINDOW (RW): The restart window is the size of the congestion window after a TCP restarts transmission after an idle period. This applies if the slow start algorithm is used; see section 4.1 for more discussion π¦
FLIGHT SIZE: The amount of data that has been sent but not yet acknowledged. Data in flight, literally π
3. Congestion Control Algorithms
Okay this is the core of it π₯ This section defines the four congestion control algorithms (slow start, congestion avoidance, fast retransmit, and fast recovery), which were developed in [Jac88] and [Jac90]. Sometimes it can be good for a TCP sender to be more conservative than the algorithms allow. HOWEVER, a TCP MUST NOT be more aggressive than the following algorithms allow. In other words, it MUST NOT send data when the value of cwnd computed by the following algorithms would not allow that data to be sent. Don't be extra in a way that breaks the rules π
3.1 Slow Start and Congestion Avoidance
The slow start and congestion avoidance algorithms MUST be used by a TCP sender to control how much outstanding data gets injected into the network π To make them work, two variables get added to the TCP per-connection state. The congestion window (cwnd) is a sender-side limit on how much data the sender can transmit into the network before it gets an acknowledgment (ACK). The receiver's advertised window (rwnd) is a receiver-side limit on the amount of outstanding data. The minimum of cwnd and rwnd governs data transmission π
There's another state variable, the slow start threshold (ssthresh). It decides whether slow start or congestion avoidance controls data transmission, which we'll get into below π§
Okay so when you start sending into a network with unknown conditions, TCP needs to slowly probe the network to figure out how much capacity is available. The goal is to avoid flooding the network with a way-too-big burst of data π Slow start does this job at the beginning of a transfer, or after repairing a loss detected by the retransmission timer πΈ
IW, the initial value of cwnd, MUST be less than or equal to 2*SMSS bytes and MUST NOT be more than 2 segments. Strict, but it keeps things calm π
We note that a non-standard, experimental TCP extension lets a TCP MAY use a larger initial window (IW), as defined in equation 1 [AFP98]:
IW = min (4*SMSS, max (2*SMSS, 4380 bytes)) (1)
With this extension, a TCP sender MAY use a 3 or 4 segment initial window, as long as the combined size of the segments doesn't go over 4380 bytes. We do not allow this change as part of the standard defined by this document π However, we include discussion of (1) in the rest of this document as a guideline for people experimenting with the change, rather than as something that conforms to the current TCP congestion control standards π§ͺ
The initial value of ssthresh MAY be arbitrarily high (for example, some implementations use the size of the advertised window). It may be reduced in response to congestion though π The slow start algorithm is used when cwnd < ssthresh, and the congestion avoidance algorithm is used when cwnd > ssthresh. When cwnd and ssthresh are equal, the sender may use either one π
During slow start, a TCP increments cwnd by at most SMSS bytes for each ACK that acknowledges new data. Slow start ends when cwnd exceeds ssthresh (or, optionally, when it reaches it, as noted above) or when congestion is observed π
During congestion avoidance, cwnd gets incremented by 1 full-sized segment per round-trip time (RTT). Congestion avoidance keeps going until congestion is detected π One formula people commonly use to update cwnd during congestion avoidance is equation 2:
cwnd += SMSS*SMSS/cwnd (2)
This adjustment runs on every incoming non-duplicate ACK. Equation (2) is a pretty good approximation of the underlying idea of increasing cwnd by 1 full-sized segment per RTT. (Fun fact: for a connection where the receiver acknowledges every data segment, (2) is slightly more aggressive than 1 segment per RTT, and for a receiver acknowledging every other packet, (2) is less aggressive π€)
Implementation Note: Since integer arithmetic is usually used in TCP implementations, the formula in equation 2 can fail to increase cwnd when the congestion window is very large (larger than SMSS*SMSS). If the formula above yields 0, the result SHOULD be rounded up to 1 byte. Tiny detail, big difference π
Implementation Note: Older implementations have an extra additive constant on the right-hand side of equation (2). That's incorrect, and it can actually lead to diminished performance [PAD+98] π¬
Another acceptable way to increase cwnd during congestion avoidance is to count the number of bytes acknowledged by ACKs for new data. (A drawback of this approach is that it needs one more state variable to track.) When the number of bytes acknowledged reaches cwnd, cwnd can be incremented by up to SMSS bytes. Note that during congestion avoidance, cwnd MUST NOT be increased by more than the larger of either 1 full-sized segment per RTT, or the value computed using equation 2 π₯
Implementation Note: Some implementations keep cwnd in units of bytes, while others use units of full-sized segments. The second group will find equation (2) hard to use, and may prefer the counting approach from the paragraph above π§
When a TCP sender detects segment loss using the retransmission timer, the value of ssthresh MUST be set to no more than the value given in equation 3:
ssthresh = max (FlightSize / 2, 2*SMSS) (3)
As discussed above, FlightSize is the amount of outstanding data in the network π§
Implementation Note: It's an easy mistake to just use cwnd instead of FlightSize. In some implementations, cwnd may incidentally grow well beyond rwnd, so watch out π
Furthermore, upon a timeout, cwnd MUST be set to no more than the loss window, LW, which equals 1 full-sized segment (regardless of the value of IW). So after retransmitting the dropped segment, the TCP sender uses slow start to grow the window from 1 full-sized segment up to the new value of ssthresh. At that point, congestion avoidance takes over again π¦β¨
3.2 Fast Retransmit/Fast Recovery π¦
Okay bestie, so a TCP receiver SHOULD send an immediate duplicate ACK whenever an out-of-order segment shows up π The whole point is to tell the sender "hey, I got something out of order, and this is the sequence number I'm still waiting for" π§ From the sender's side, duplicate ACKs can come from a few different network problems, and it's giving mystery π First, they can be caused by dropped segments. If one segment gets dropped, every segment after it triggers a duplicate ACK. Second, they can be caused by the network re-ordering data segments, which is not a rare event on some network paths [Pax97]. Finally, they can be caused by the network replicating ACK or data segments. On top of that, a TCP receiver SHOULD send an immediate ACK when the incoming segment fills in all or part of a gap in the sequence space. This gives more timely info to a sender that's recovering from a loss through a retransmission timeout, a fast retransmit, or an experimental loss recovery algorithm such as NewReno [FH98] β¨
The TCP sender SHOULD use the "fast retransmit" algorithm to detect and repair loss, based on incoming duplicate ACKs π₯ Fast retransmit treats the arrival of 3 duplicate ACKs (4 identical ACKs with no other packets arriving in between) as a sign that a segment has been lost. After receiving 3 duplicate ACKs, TCP retransmits what appears to be the missing segment, without waiting for the retransmission timer to expire. Literally no waiting around π
After the fast retransmit algorithm sends what looks like the missing segment, the "fast recovery" algorithm controls how new data gets sent until a non-duplicate ACK arrives π The reason it skips slow start is that those duplicate ACKs don't just say a segment got lost, they also say segments are most likely leaving the network (although a massive segment duplication by the network can mess with this conclusion). Think about it: the receiver can only generate a duplicate ACK when a segment has arrived, so that segment has left the network and is sitting in the receiver's buffer. We know it's no longer eating up network resources. Plus, since the ACK "clock" [Jac88] is preserved, the TCP sender can keep sending new segments (though transmission has to continue using a reduced cwnd) π«Ά
The fast retransmit and fast recovery algorithms are usually implemented together like this πΈ
- When the third duplicate ACK is received, set ssthresh to no more than the value given in equation 3.
- Retransmit the lost segment and set cwnd to ssthresh plus 3*SMSS. This artificially "inflates" the congestion window by the number of segments (three) that have left the network and which the receiver has buffered.
- For each additional duplicate ACK received, increment cwnd by SMSS. This artificially inflates the congestion window in order to reflect the additional segment that has left the network.
- Transmit a segment, if allowed by the new value of cwnd and the receiver's advertised window.
- When the next ACK arrives that acknowledges new data, set cwnd to ssthresh (the value set in step 1). This is termed "deflating" the window.
This ACK should be the acknowledgment elicited by the retransmission from step 1, one RTT after the retransmission (though it may arrive sooner in the presence of significant out-of-order delivery of data segments at the receiver). Additionally, this ACK should acknowledge all the intermediate segments sent between the lost segment and the receipt of the third duplicate ACK, if none of these were lost.
Note: This algorithm is known to generally not recover very efficiently from multiple losses in a single flight of packets [FF96]. One proposed set of modifications to address this problem can be found in [FH98] π
4. Additional Considerations β¨
4.1 Re-starting Idle Connections
A known problem with the TCP congestion control algorithms described above is that they allow a potentially inappropriate burst of traffic to be transmitted after TCP has been idle for a relatively long period of time π After an idle period, TCP can't use the ACK clock to strobe new segments into the network, because all the ACKs have drained from the network. So, as specified above, TCP can potentially send a cwnd-size line-rate burst into the network after an idle period. Chaotic energy πͺοΈ
[Jac88] recommends that a TCP use slow start to restart transmission after a relatively long idle period. Slow start restarts the ACK clock, just like it does at the beginning of a transfer. This mechanism has been widely deployed in the following manner: when TCP has not received a segment for more than one retransmission timeout, cwnd is reduced to the value of the restart window (RW) before transmission begins π
For the purposes of this standard, we define RW = IW.
We note that the non-standard experimental extension to TCP defined in [AFP98] defines RW = min(IW, cwnd), with the definition of IW adjusted per equation (1) above π§
Using the last time a segment was received to decide whether or not to decrease cwnd fails to deflate cwnd in the common case of persistent HTTP connections [HTH98]. Here's the situation: a WWW server receives a request before transmitting data to the WWW browser. That reception of the request makes the test for an idle connection fail, and it lets TCP begin transmission with a possibly inappropriately large cwnd. Not it π ββοΈ
Therefore, a TCP SHOULD set cwnd to no more than RW before beginning transmission if the TCP has not sent data in an interval exceeding the retransmission timeout π¦
4.2 Generating Acknowledgments
The delayed ACK algorithm specified in [Bra89] SHOULD be used by a TCP receiver. When it's used, a TCP receiver MUST NOT excessively delay acknowledgments π«Ά Specifically, an ACK SHOULD be generated for at least every second full-sized segment, and MUST be generated within 500 ms of the arrival of the first unacknowledged packet.
The requirement that an ACK "SHOULD" be generated for at least every second full-sized segment is listed in [Bra89] in one place as a SHOULD and in another as a MUST. Here we unambiguously state it is a SHOULD. We also emphasize that this is a SHOULD, which means an implementor should only deviate from this requirement after careful consideration of the implications. See the discussion of "Stretch ACK violation" in [PAD+98] and the references therein for a discussion of the possible performance problems with generating ACKs less frequently than every second full-sized segment π€
In some cases, the sender and receiver may not agree on what counts as a full-sized segment. An implementation is deemed to comply with this requirement if it sends at least one acknowledgment every time it receives 2*RMSS bytes of new data from the sender, where RMSS is the Maximum Segment Size specified by the receiver to the sender (or the default value of 536 bytes, per [Bra89], if the receiver does not specify an MSS option during connection establishment). The sender may be forced to use a segment size less than RMSS due to the maximum transmission unit (MTU), the path MTU discovery algorithm, or other factors. For instance, consider the case when the receiver announces an RMSS of X bytes but the sender ends up using a segment size of Y bytes (Y < X) due to path MTU discovery (or the sender's MTU size). The receiver will generate stretch ACKs if it waits for 2*X bytes to arrive before an ACK is sent. Clearly this will take more than 2 segments of size Y bytes. So, while a specific algorithm is not defined, it's desirable for receivers to try to prevent this situation, for example by acknowledging at least every second segment, regardless of size. Finally, we repeat that an ACK MUST NOT be delayed for more than 500 ms waiting on a second full-sized segment to arrive π
Out-of-order data segments SHOULD be acknowledged immediately, to speed up loss recovery π₯ To trigger the fast retransmit algorithm, the receiver SHOULD send an immediate duplicate ACK when it receives a data segment above a gap in the sequence space. To give feedback to senders recovering from losses, the receiver SHOULD send an immediate ACK when it receives a data segment that fills in all or part of a gap in the sequence space.
A TCP receiver MUST NOT generate more than one ACK for every incoming segment, other than to update the offered window as the receiving application consumes new data [page 42, Pos81][Cla82] π
4.3 Loss Recovery Mechanisms
A number of loss recovery algorithms that augment fast retransmit and fast recovery have been suggested by TCP researchers π Some of these are based on the TCP selective acknowledgment (SACK) option [MMFR96], such as [FF96,MM96a,MM96b], while others don't require SACKs [Hoe96,FF96,FH98]. The non-SACK algorithms use "partial acknowledgments" (ACKs which cover new data, but not all the data outstanding when loss was detected) to trigger retransmissions. While this document doesn't standardize any of the specific algorithms that may improve fast retransmit/fast recovery, these enhanced algorithms are implicitly allowed, as long as they follow the general principles of the basic four algorithms outlined above β¨
So, when the first loss in a window of data is detected, ssthresh MUST be set to no more than the value given by equation (3). Second, until all lost segments in the window of data in question are repaired, the number of segments transmitted in each RTT MUST be no more than half the number of outstanding segments when the loss was detected. Finally, after all loss in the given window of segments has been successfully retransmitted, cwnd MUST be set to no more than ssthresh, and congestion avoidance MUST be used to further increase cwnd. Loss in two successive windows of data, or the loss of a retransmission, should be taken as two indications of congestion, and therefore, cwnd (and ssthresh) MUST be lowered twice in this case π«
The algorithms outlined in [Hoe96,FF96,MM96a,MM6b] follow the principles of the basic four congestion control algorithms outlined in this document π¦
5. Security Considerations π
This document requires a TCP to diminish its sending rate in the presence of retransmission timeouts and the arrival of duplicate acknowledgments. An attacker can therefore impair the performance of a TCP connection by either causing data packets or their acknowledgments to be lost, or by forging excessive duplicate acknowledgments. Causing two congestion control events back-to-back will often cut ssthresh to its minimum value of 2*SMSS, which makes the connection immediately enter the slower-performing congestion avoidance phase π©
The Internet relies to a considerable degree on these algorithms being implemented correctly in order to preserve network stability and avoid congestion collapse. An attacker could make TCP endpoints respond more aggressively in the face of congestion by forging excessive duplicate acknowledgments or excessive acknowledgments for new data. Conceivably, such an attack could drive a portion of the network into congestion collapse. Dramatic, but this is serious π¨
6. Changes Relative to RFC 2001 β¨
okay so, the big honest answer is that this doc got a total editorial glow-up π , so listing every single change would be a nightmare π. The vibe is: we are NOT changing any of the recommendations from RFC 2001 π ββοΈ. We're just clearing up stuff 2001 didn't dig into. Specifically, this one suggests what TCP connections should do after a pretty long idle period π, it nails down some of the issues around TCP ACK generation π«Ά, and the upper bound for the initial congestion window got bumped from one segment to two segments π. Iconic upgrade, no notes. π¦
Acknowledgments
the four algorithms in this doc were developed by Van Jacobson π. literal legend.
some of the text in here is borrowed from "TCP/IP Illustrated, Volume 1: The Protocols" by W. Richard Stevens (Addison-Wesley, 1994) and "TCP/IP Illustrated, Volume 2: The Implementation" by Gary R. Wright and W. Richard Stevens (Addison-Wesley, 1995). Addison-Wesley gave permission for that, which is so nice of them πΈ.
Neal Cardwell, Sally Floyd, Craig Partridge and Joe Touch dropped a bunch of helpful suggestions π. Thank you, besties π«Ά.
References
- [AFP98] Allman, M., Floyd, S. and C. Partridge, "Increasing TCP's Initial Window Size, RFC 2414, September 1998. π§
- [Bra89] Braden, R., "Requirements for Internet Hosts -- Communication Layers", STD 3, RFC 1122, October 1989. π
- [Bra97] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. π
- [Cla82] Clark, D., "Window and Acknowledgment Strategy in TCP", RFC 813, July 1982. π
- [FF96] Fall, K. and S. Floyd, "Simulation-based Comparisons of Tahoe, Reno and SACK TCP", Computer Communication Review, July 1996. ftp://ftp.ee.lbl.gov/papers/sacks.ps.Z. β¨
- [FH98] Floyd, S. and T. Henderson, "The NewReno Modification to TCP's Fast Recovery Algorithm", RFC 2582, April 1999. π₯
- [Flo94] Floyd, S., "TCP and Successive Fast Retransmits. Technical report", October 1994. ftp://ftp.ee.lbl.gov/papers/fastretrans.ps. π¦
- [Hoe96] Hoe, J., "Improving the Start-up Behavior of a Congestion Control Scheme for TCP", In ACM SIGCOMM, August 1996. π
- [HTH98] Hughes, A., Touch, J. and J. Heidemann, "Issues in TCP Slow-Start Restart After Idle", Work in Progress. π
- [Jac88] Jacobson, V., "Congestion Avoidance and Control", Computer Communication Review, vol. 18, no. 4, pp. 314-329, Aug. 1988. ftp://ftp.ee.lbl.gov/papers/congavoid.ps.Z. π
- [Jac90] Jacobson, V., "Modified TCP Congestion Avoidance Algorithm", end2end-interest mailing list, April 30, 1990. ftp://ftp.isi.edu/end2end/end2end-interest-1990.mail. π
- [MD90] Mogul, J. and S. Deering, "Path MTU Discovery", RFC 1191, November 1990. π«§
- [MM96a] Mathis, M. and J. Mahdavi, "Forward Acknowledgment: Refining TCP Congestion Control", Proceedings of SIGCOMM'96, August, 1996, Stanford, CA. Available fromhttp://www.psc.edu/networking/papers/papers.html π
- [MM96b] Mathis, M. and J. Mahdavi, "TCP Rate-Halving with Bounding Parameters", Technical report. Available from http://www.psc.edu/networking/papers/FACKnotes/current. πΈ
- [MMFR96] Mathis, M., Mahdavi, J., Floyd, S. and A. Romanow, "TCP Selective Acknowledgement Options", RFC 2018, October 1996. π§
- [PAD+98] Paxson, V., Allman, M., Dawson, S., Fenner, W., Griner, J., Heavens, I., Lahey, K., Semke, J. and B. Volz, "Known TCP Implementation Problems", RFC 2525, March 1999. π
- [Pax97] Paxson, V., "End-to-End Internet Packet Dynamics", Proceedings of SIGCOMM '97, Cannes, France, Sep. 1997. π
- [Pos81] Postel, J., "Transmission Control Protocol", STD 7, RFC 793, September 1981. ποΈ
- [Ste94] Stevens, W., "TCP/IP Illustrated, Volume 1: The Protocols", Addison-Wesley, 1994. π
- [Ste97] Stevens, W., "TCP Slow Start, Congestion Avoidance, Fast Retransmit, and Fast Recovery Algorithms", RFC 2001, January 1997. π
- [WS95] Wright, G. and W. Stevens, "TCP/IP Illustrated, Volume 2: The Implementation", Addison-Wesley, 1995. π
Authors' Addresses
the authors, in their own (very official) words π:
Mark Allman
NASA Glenn Research Center/Sterling Software
Lewis Field
21000 Brookpark Rd. MS 54-2
Cleveland, OH 44135
216-433-6586
EMail: mallman@grc.nasa.gov
http://roland.grc.nasa.gov/~mallman
Vern Paxson
ACIRI / ICSI
1947 Center Street
Suite 600
Berkeley, CA 94704-1198
Phone: +1 510/642-4274 x302
EMail: vern@aciri.org
W. Richard Stevens
1202 E. Paseo del Zorro
Tucson, AZ 85718
520-297-9416
EMail: rstevens@kohala.com
http://www.kohala.com/~rstevens
Full Copyright Statement
Copyright (C) The Internet Society (1999). All Rights Reserved.
This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English.
The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns.
This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. π
π¦ where this page comes from
- ietf: rfc2581 1999-04
- π girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms