RFC 3390

RFC 3390 🎀 Okay bestie, this one is a whole Standards Track document from the Network Working Group, so it's the real deal. It was written in October 2002 by M. Allman (BBN/NASA GRC), S. Floyd (ICIR), and C. Partridge (BBN Technologies). It obsoletes RFC 2414 and updates RFC 2581. The title is Increasing TCP's Initial Window 🦋

Status of this Memo 💅

Okay so this doc is an Internet standards track protocol for the Internet community, and it's asking everyone for discussion and suggestions for improvements 🫶 For the current standardization state and status, check the "Internet Official Protocol Standards" (STD 1) list. Distribution of this memo is unlimited, so share away ✨

Copyright Notice 🌸

Copyright (C) The Internet Society (2002). All Rights Reserved.

Abstract 💖

Okay so this doc is giving an optional standard for TCP that bumps the permitted initial window from one or two segment(s) up to roughly 4K bytes, replacing RFC 2414 🍓 It talks through the pros and cons of the bigger initial window, and it has experiments and simulations showing that the higher initial window does not lead to congestion collapse 🙈 Lastly, it gives guidance on implementation issues. Iconic 👑

Terminology 🧚

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119].

1. TCP Modification 💅

Okay so this doc obsoletes [RFC2414] and updates [RFC2581], and it raises the permitted upper bound for TCP's initial window from one or two segment(s) to between two and four segments ✨ In most cases, that gives you an upper bound on the initial window of roughly 4K bytes. Giving a large segment size, though, the permitted initial window of two segments can be way bigger than 4K bytes 😭

The upper bound for the initial window is given more precisely in (1):

      min (4*MSS, max (2*MSS, 4380 bytes))                        (1)

Note: Sending a 1500 byte packet indicates a maximum segment size (MSS) of 1460 bytes (assuming no IP or TCP options). So limiting the initial window's MSS to 4380 bytes lets the sender transmit three segments at the start in the common case when using 1500 byte packets 🧚

Equivalently, the upper bound for the initial window size is based on the MSS, like this:

    If (MSS <= 1095 bytes)
        then win <= 4 * MSS;
    If (1095 bytes < MSS < 2190 bytes)
        then win <= 4380;
    If (2190 bytes <= MSS)
        then win <= 2 * MSS;

This bigger initial window is optional, so a TCP MAY start with a larger initial window 🌈 That said, we expect that most general-purpose TCP implementations would pick the larger initial congestion window from equation (1) above 💜

This upper bound for the initial window size is a change from RFC 2581 [RFC2581], which said the congestion window should be initialized to one or two segments 🍓

This change applies to the initial window of the connection in the first round trip time (RTT) of data transmission after the TCP three-way handshake. Neither the SYN/ACK nor its acknowledgment (ACK) in the three-way handshake should increase the initial window size above what's outlined in equation (1) 🙈 And if the SYN or SYN/ACK gets lost, the initial window a sender uses after a correctly transmitted SYN MUST be one segment consisting of MSS bytes 😤

TCP implementations use slow start in as many as three different ways 🔥: (1) to start a new connection (the initial window); (2) to restart transmission after a long idle period (the restart window); and (3) to restart transmission after a retransmit timeout (the loss window). The change in this document affects the value of the initial window. Optionally, a TCP MAY set the restart window to the minimum of the value used for the initial window and the current value of cwnd (in other words, using a larger value for the restart window should never increase the size of cwnd) ✨ These changes do NOT change the loss window, which must remain 1 segment of MSS bytes, so you still get the lowest possible window size during severe congestion 🫶

5. Disadvantages of Larger Initial Windows for the Network ✨

okay so, let's talk about the network's side of things 🫶. When we think about congestion collapse, we're really looking at two different dangers. The first is a scenario where a ton of segments on congested links are duplicates, stuff the receiver already got. The second is a scenario where a ton of segments on congested links get dropped later in the network before they ever reach their final destination.

Then there's the effect on everyone else's traffic. A possible downside of bigger initial windows is that they could bump up the general packet drop rate in the network. We'll tackle all three below 💅.

Duplicate segments:

As we covered in the last section, the bigger initial window could sometimes mean a segment gets dropped from the initial window, when it might not have been dropped if the sender had slow-started from one segment. BUT, Appendix A shows that even then, the bigger window doesn't cause a huge pile of duplicate segments 😭🙌.

Segments dropped later in the network:

How much would the larger initial window for TCP increase the number of segments on congested links that get dropped before reaching their final destination? This can only happen for connections with multiple congested links, where a segment uses scarce bandwidth on the first congested link along the path, only to get dropped later on. Main character problem, but not everyone's problem 💀.

First, a lot of TCP connections only have one congested link along the path. Segments dropped from those connections don't "waste" scarce bandwidth, so they don't contribute to congestion collapse. Literally no harm, no foul ✨.

However, some network paths have multiple congested links, and segments dropped from the initial window could use scarce bandwidth on the earlier congested links before getting dropped on the later ones. As long as the drop rate is independent of the initial window TCP uses, this problem looks about the same for connections that start by sending four segments or one segment. So it's not a bigger-window thing, it's a whole-network thing 🧚.

An increased packet drop rate:

On a network with a high segment drop rate, increasing the TCP initial window could push the segment drop rate even higher. Part of the reason is that routers with Drop Tail queue management have a hard time with bursty traffic when things are congested 🙈. But given uncorrelated arrivals for TCP connections, the larger TCP initial window should not significantly increase the segment drop rate. The simulation-based explorations of these issues are discussed in Section 7.2 🌸.

These potential dangers get tested in the simulations and experiments in the section below. Our judgment? There ARE dangers of congestion collapse in the current Internet (check [FF99] for the tea on congestion collapse from an increase in UDP connections without end-to-end congestion control, it's a whole thing 😤). But there is no such danger to the network from increasing the TCP initial window to 4K bytes 🦋.

6. Interactions with the Retransmission Timer

okay this one is a little spicy 🌶️. Using a larger initial burst of data can make existing problems with spurious retransmit timeouts worse on low-bandwidth paths, assuming the standard algorithm for determining the TCP retransmission timeout (RTO) [RFC2988]. Here's the vibe: on low-bandwidth paths, the time it takes to transmit a packet is a big chunk of the round-trip time, and the small packets used to set up a TCP connection don't seed the RTO estimator properly. So when the first window of data packets goes out, the sender's retransmit timer could expire before the acknowledgments for those packets show up 😬.

Here's the saving grace: every time an acknowledgment arrives, the retransmit timer usually gets reset. So the timer won't expire as long as an acknowledgment arrives at least once a second, given the one-second minimum on the RTO recommended in RFC 2988 ⏰.

Let's do a real example, because numbers make it click 🤓. Say we've got a 9.6 Kbps link. The initial RTT measurement will be on the order of 67 msec, if we only consider the transmission time of 2 packets (the SYN and SYN-ACK), each consisting of 40 bytes. Using the RTO estimator given in [RFC2988], that gives an initial RTO of 201 msec (67 + 4*(67/2)). BUT we round the RTO up to 1 second, as specified in RFC 2988 ✨.

Now say we send an initial window of one or more 1500-byte packets (1460 data bytes plus overhead). Each packet takes on the order of 1.25 seconds to transmit. So the RTO fires before the ACK for the first packet comes back, causing a spurious timeout 💀. In this case, a larger initial window of three or four packets makes the problems caused by this spurious timeout even worse 😭.

There are a couple of ways people could deal with this. One is to make the RTO algorithm more conservative. During the initial window of data, for instance, the RTO could be updated for each acknowledgment received. Another idea: if the retransmit timer expires for some packet lost in the first window of data, we could leave the exponential backoff of the retransmit timer engaged until at least one valid RTT measurement, involving a data packet, is received 🎀.

A different method is to refrain from taking an RTT sample during connection establishment, leaving the default RTO in place until TCP takes a sample from a data segment and its corresponding ACK. This method likely helps prevent spurious retransmits, but it may also slow the data transfer down if loss happens before the RTO is seeded 🐢. The use of limited transmit [RFC3042] helps a TCP connection recover from loss with fast retransmit instead of waiting on the RTO timer, which eases the performance hit from using the high default RTO during the initial window of data transmission 🫶.

This specification leaves the decision about what to do (if anything) with regards to the RTO, when using a larger initial window, up to the implementer. Still, the RECOMMENDED approach is to refrain from sampling the RTT during the three-way handshake, keeping the default RTO in place until an RTT sample involving a data packet is taken. And in addition, it is RECOMMENDED that TCPs use limited transmit [RFC3042] 💖.

7. Typical Levels of Burstiness for TCP Traffic.

ngl this part is so relatable. Larger TCP initial windows wouldn't dramatically increase the burstiness of TCP traffic on the Internet today, because TCP traffic is already pretty bursty 🌈. Bursts of two and three segments are already typical of TCP [Flo97]. A delayed ACK (covering two previously unacknowledged segments) received during congestion avoidance makes the congestion window slide and two segments get sent. That same delayed ACK received during slow start slides the window by two segments and then increments it by one segment, resulting in a three-segment burst 🔥.

Bursts of four and five segments aren't rare either, even if they're not necessarily typical 👀. Assuming delayed ACKs, a single dropped ACK makes the next ACK cover four previously unacknowledged segments. During congestion avoidance, that leads to a four-segment burst, and during slow start, a five-segment burst gets generated 💅.

There are also changes in progress that reduce the performance problems caused by moderate traffic bursts 🦋. One is the deployment of higher-speed links in some parts of the network, where a burst of 4K bytes can represent a small quantity of data. Another, for routers with sufficient buffering, is the deployment of queue management mechanisms such as RED, which is designed to tolerate transient traffic bursts 🧚.

8. Simulations and Experimental Results

8.1 Studies of TCP Connections using that Larger Initial Window

this section is basically a highlight reel of tests people ran on bigger initial windows 📚✨. It surveys simulations and experiments exploring the effect of larger initial windows on TCP connections. The first set of experiments explored performance over satellite links 🛰️. Larger initial windows have been shown to improve the performance of TCP connections over satellite channels [All97b]. In that study, an initial window of four segments (512 byte MSS) gave throughput improvements of up to 30% (depending on transfer size) 🚀. [KAGT98] shows that larger initial windows cut transfer time in HTTP tests over the ACTS satellite system. And a study with simulations of a large number of HTTP transactions over hybrid fiber coax (HFC) shows that larger initial windows decrease the time needed to load WWW pages [Nic98] 🌐.

A second set of experiments looked at TCP performance over dialup modem links 📞. In experiments over a 28.8 bps dialup channel [All97a, AHO98], a four-segment initial window decreased the transfer time of a 16KB file by roughly 10%, with no accompanying increase in the drop rate 🙌. A simulation study [RFC2416] looked at the effects of a larger initial window on a host connected by a slow modem link and a router with a 3 packet buffer. For the scenario they investigated, the study concluded that larger initial windows were not harmful to TCP performance 💖.

Finally, [All00] illustrates that the percentage of connections at a particular web server that experience loss in the initial window of data transmission increases with the size of the initial congestion window. But, the increase is in line with what you'd expect from sending a larger burst into the network. So no surprise there, just vibes matching the math 🫶🎀.

8.2 Studies of Networks using Larger Initial Windows ✨

Okay bestie, this part is about whether the bigger window hurts everyone else sharing the path 👀 Do the other connections get squished? Spoiler: a little, but not by much.

Experiments in [All97a, AHO98] looked at 16 KB transfers to 100 Internet hosts. Four-segment initial windows caused a small increase in drop rate, 0.04 segments/transfer 📉 But the transfer time dropped by roughly 25% for four-segment (512 byte MSS) initial windows compared to a one-segment initial window 🚀 The drops went up a teeny bit while the speed went up a lot. Ngl, that's a great trade 💅

Then there's the simulation study in [RFC2415] 🎀 It looked at HTTP and FTP flows sharing a single congested gateway, with the number of HTTP and FTP flows changing from one simulation set to the next. For each set, it checked aggregate link utilization, packet drop rates, median web page delay, and network power for the FTP transfers. The larger initial window generally gave more throughput, slightly higher packet drop rates, and more overall network power 🔥 The only scenario that stood out was where the drop rate went from 3.5% with one-segment initial windows to 4.5% with four-segment initial windows. Everywhere else, the larger window added less than 1% on top of the loss rate from a one-segment initial window. The overall verdict was that raising the TCP initial window to three packets (or 4380 bytes) helps improve perceived performance ✨

Morris [Mor97] tested larger initial windows in a highly congested network with 20K transfers 😬 In networks where every TCP connection used a four-segment initial window, the loss rate was 1-2% higher than in a network where every connection used a one-segment initial window. This held in scenarios where the one-segment loss rates ranged from 1% to 11%. On top of that, connections using a four-segment initial window spent more time waiting for the retransmit timer (RTO) to expire so they could resend a segment, compared to the one-segment case 💀 That waiting is idle time, when the connection isn't getting any useful work done. So in a very congested environment, where each connection's share of the bottleneck bandwidth is close to one segment, a larger initial window can cause a noticeable rise in both loss rates and retransmit timeouts 😭

---

9. Security Considerations 🔐

Okay, this one is serious, so let's be real 💜 This document discusses the initial congestion window permitted for TCP connections. Changing this value does not raise any known new security issues with TCP.

10. Conclusion 🏁

This document specifies a small change to TCP that will likely be beneficial to short-lived TCP connections and to connections over links with long RTTs 🌈 It saves several RTTs during the initial slow-start phase, which is giving main character energy ⭐

11. Acknowledgments 🫶

Huge shoutout to Vern Paxson, Tim Shepard, the members of the End-to-End-Interest Mailing List, and the members of the IETF TCP Implementation Working Group. They kept the discussions going and gave feedback on this document 💖

12. References 📚

[AHO98]   Mark Allman, Chris Hayes, and Shawn Ostermann, An
          Evaluation of TCP with Larger Initial Windows, March 1998.
          ACM Computer Communication Review, 28(3), July 1998.  URL
          "http://roland.lerc.nasa.gov/~mallman/papers/initwin.ps".

[All97a]  Mark Allman.  An Evaluation of TCP with Larger Initial
          Windows.  40th IETF Meeting -- TCP Implementations WG.
          December, 1997.  Washington, DC.

[All97b]  Mark Allman.  Improving TCP Performance Over Satellite
          Channels.  Master's thesis, Ohio University, June 1997.

[All00]   Mark Allman. A Web Server's View of the Transport Layer.
          ACM Computer Communication Review, 30(5), October 2000.

[FF96]    Fall, K., and Floyd, S., Simulation-based Comparisons of
          Tahoe, Reno, and SACK TCP.  Computer Communication Review,
          26(3), July 1996.

[FF99]    Sally Floyd, Kevin Fall.  Promoting the Use of End-to-End
          Congestion Control in the Internet.  IEEE/ACM Transactions
          on Networking, August 1999.  URL
          "http://www.icir.org/floyd/end2end-paper.html".

[FJ93]    Floyd, S., and Jacobson, V., Random Early Detection
          gateways for Congestion Avoidance. IEEE/ACM Transactions on
          Networking, V.1 N.4, August 1993, p. 397-413.

[Flo94]   Floyd, S., TCP and Explicit Congestion Notification.
          Computer Communication Review, 24(5):10-23, October 1994.

[Flo96]   Floyd, S., Issues of TCP with SACK. Technical report,
          January 1996.  Available from http://www-
          nrg.ee.lbl.gov/floyd/.

[Flo97]   Floyd, S., Increasing TCP's Initial Window.  Viewgraphs,
          40th IETF Meeting - TCP Implementations WG. December, 1997.
          URL "ftp://ftp.ee.lbl.gov/talks/sf-tcp-ietf97.ps".

[KAGT98]  Hans Kruse, Mark Allman, Jim Griner, Diepchi Tran.  HTTP
          Page Transfer Rates Over Geo-Stationary Satellite Links.
          March 1998.  Proceedings of the Sixth International
          Conference on Telecommunication Systems.  URL
          "http://roland.lerc.nasa.gov/~mallman/papers/nash98.ps".

[Mor97]   Robert Morris.  Private communication, 1997.  Cited for
          acknowledgement purposes only.

[Nic98]   Kathleen Nichols. Improving Network Simulation With
          Feedback, Proceedings of LCN 98, October 1998. URL
          "http://www.computer.org/proceedings/lcn/8810/8810toc.htm".

[Pos82]   Postel, J., "Simple Mail Transfer Protocol", STD 10, RFC
          821, August 1982.

[RFC1122] Braden, R., "Requirements for Internet Hosts --
          Communication Layers", STD 3, RFC 1122, October 1989.

[RFC1191] Mogul, J. and S. Deering, "Path MTU Discovery", RFC 1191,
          November 1990.

[RFC1945] Berners-Lee, T., Fielding, R. and H. Nielsen, "Hypertext
          Transfer Protocol -- HTTP/1.0", RFC 1945, May 1996.

[RFC2068] Fielding, R., Mogul, J., Gettys, J., Frystyk, H. and T.
          Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1", RFC
          2616, January 1997.

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
          Requirement Levels", BCP 14, RFC 2119, March 1997.

[RFC2309] Braden, B., Clark, D., Crowcroft, J., Davie, B., Deering,
          S., Estrin, D., Floyd, S., Jacobson, V., Minshall, G.,
          Partridge, C., Peterson, L., Ramakrishnan, K., Shenker, S.,
          Wroclawski, J. and L.  Zhang, "Recommendations on Queue
          Management and Congestion Avoidance in the Internet", RFC
          2309, April 1998.

[RFC2414] Allman, M., Floyd, S. and C. Partridge, "Increasing TCP's
          Initial Window", RFC 2414, September 1998.

[RFC2415] Poduri, K. and K. Nichols, "Simulation Studies of Increased
          Initial TCP Window Size", RFC 2415, September 1998.

[RFC2416] Shepard, T. and C. Partridge, "When TCP Starts Up With Four
          Packets Into Only Three Buffers", RFC 2416, September 1998.

[RFC2581] Allman, M., Paxson, V. and W. Stevens, "TCP Congestion
          Control", RFC 2581, April 1999.

[RFC2821] Klensin, J., "Simple Mail Transfer Protocol", RFC 2821,
          April 2001.

[RFC2988] Paxson, V. and M. Allman, "Computing TCP's Retransmission
          Timer", RFC 2988, November 2000.

[RFC3042] Allman, M., Balakrishnan, H. and S. Floyd, "Enhancing TCP's
          Loss Recovery Using Limited Transmit", RFC 3042, January
          2001.

[RFC3168] Ramakrishnan, K.K., Floyd, S. and D. Black, "The Addition
          of Explicit Congestion Notification (ECN) to IP", RFC 3168,
          September 2001.

---

Appendix A - Duplicate Segments 🦋

Okay so in today's world, without Explicit Congestion Notification [Flo94] [RFC2481], every TCP uses segment drops as its signal from the network about how much bandwidth is available 📡 The point here is that a bigger initial window shouldn't make the sender retransmit a pile of duplicate segments that the receiver already has 🙅‍♀️

If one segment gets dropped from the initial window, TCP has three ways to recover:

1. Slow-starting from a window of one segment, like what happens after a retransmit timeout, or after Fast Retransmit in Tahoe TCP;
2. Fast Recovery without selective acknowledgments (SACK), like what happens after three duplicate ACKs in Reno TCP;
3. Fast Recovery with SACK, for TCP where both the sender and the receiver support the SACK option [MMFR96].

In all three cases, if a single segment is dropped from the initial window, no duplicate segments (meaning segments the receiver has already received) get transmitted 💖 For a TCP sending four 512-byte segments in the initial window, a single segment drop won't need a retransmit timeout. It can be recovered with the Fast Retransmit algorithm, unless the retransmit timer expires prematurely. A single segment dropped from an initial window of three segments might also be fixed with fast retransmit, depending on which segment was dropped and whether delayed ACKs are in use. For example, dropping the first segment of a three-segment initial window will always mean waiting for a timeout, unless Limited Transmit [RFC3042] is in play. Dropping the third segment will always allow recovery through fast retransmit, as long as no ACKs get lost 🎯

Next, we look at scenarios where the initial window holds two to four segments and at least two of those segments get dropped. If every segment in the initial window is dropped, then clearly no duplicate segments get retransmitted, because the receiver hasn't received anything yet 🙈 (It's still possible that these dropped segments used scarce bandwidth on their way to the drop point. That issue was discussed in Section 5.)

When two segments are dropped from an initial window of three segments, the sender will only send a duplicate segment if the first two of the three segments were dropped, and the sender does not receive a packet with the SACK option acknowledging the third segment.

When two segments are dropped from an initial window of four segments, there are six possible scenarios. We won't go through them all here 😅 The short version is that, depending on where the dropped packets are, the sender might send one duplicate segment in the absence of SACK. There are no scenarios where the sender sends two duplicate segments.

When three segments are dropped from an initial window of four segments, then, without SACK, it's possible that one duplicate segment will be sent, depending on where the dropped segments are.

The summary: without SACK, some scenarios with multiple segment drops from the initial window will send one duplicate segment. No scenario sends more than one duplicate segment 🎉

So our conclusion is that the number of duplicate segments transmitted as a result of a larger initial window should be small 💅✨

Author's Addresses 💌

okay bestie, here's where you can find the three people who wrote this 🦋

Mark Allman
BBN Technologies/NASA Glenn Research Center
21000 Brookpark Rd
MS 54-5
Cleveland, OH 44135
EMail: mallman@bbn.com
http://roland.lerc.nasa.gov/~mallman/

Sally Floyd
ICSI Center for Internet Research
1947 Center St, Suite 600
Berkeley, CA 94704
Phone: +1 (510) 666-2989
EMail: floyd@icir.org
http://www.icir.org/floyd/

Craig Partridge
BBN Technologies
10 Moulton St
Cambridge, MA 02138
EMail: craig@bbn.com

---

Full Copyright Statement ⚖️🌈

the legal bit, but make it cute 🎀

Copyright (C) The Internet Society (2002). All Rights Reserved.

okay so anyone can copy this document and pass it around, and they can write derivative works that comment on or explain it or help people implement it, and those can be prepared, copied, published and distributed, in whole or in part, with no restrictions of any kind 💖 the one catch is that the above copyright notice and this paragraph have to be included on every single copy and derivative work. the document itself is a no-touch zone though 🙅‍♀️ you may not modify it in any way, like removing the copyright notice or the references to the Internet Society or other Internet organizations. the exceptions are when it's needed for developing Internet standards (then you have to follow the copyright procedures in the Internet Standards process), or when you're translating it into a language other than English ✨

the limited permissions above are perpetual and the Internet Society, its successors, or its assigns will never revoke them 🦋🫶

and the disclaimer, which is serious, so i'm keeping it in caps 💀:

THIS DOCUMENT AND THE INFORMATION CONTAINED HEREIN IS PROVIDED ON AN "AS IS" BASIS AND THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 🙈

---

Acknowledgement 🙏🌸

the RFC Editor function is currently funded by the Internet Society, so a big thank you to them for keeping all of this running 🧚✨