RFC 4653

Improving the Robustness of TCP to Non-Congestion Events

๐Ÿ“… published August 2006, from ietf

Improving the Robustness of TCP to Non-Congestion Events ๐ŸŽ€

Okay bestie, here's the paperwork vibe check ๐Ÿ“‹โœจ This one is RFC 4653, filed under the Network Working Group, and its category is Experimental ๐Ÿงช. It was written by S. Bhandarkar (no affiliation listed in the header), A. L. N. Reddy (Texas A&M University), M. Allman (ICIR/ICSI), and E. Blanton (Purdue University). It's dated August 2006 ๐ŸŒธ. The header doesn't list anything it obsoletes or updates, so it's not replacing or patching any other RFC ๐Ÿ’….


Status of This Memo ๐Ÿ’Œ

This memo defines an Experimental Protocol for the Internet community, so it's not an Internet standard of any kind ๐Ÿงช. The authors are literally asking for discussion and suggestions for improvement ๐Ÿซถ. Distribution of this memo is unlimited, so share the tea ๐Ÿต.

Copyright (C) The Internet Society (2006).

Abstract ๐Ÿฆ‹

This doc specifies Non-Congestion Robustness (NCR) for TCP, aka TCP-NCR ๐Ÿ’–. When there's no explicit congestion notification from the network, TCP treats loss as a sign of congestion ๐Ÿšฆ. One way TCP spots loss is by seeing three duplicate acknowledgments ๐Ÿ‘€. The catch is that this heuristic isn't always right, especially when network paths reorder segments (for whatever reason) ๐Ÿ™ˆ. That leads to degraded performance ๐Ÿ˜ญ. TCP-NCR is here to fix that by raising the number of duplicate acknowledgments needed to kick off loss recovery, based on what's happening on the connection right now ๐Ÿง . The goal is to tell real segment loss apart from segment reordering more accurately โœจ. This document spells out the changes to TCP and the costs and benefits of those changes ๐Ÿ’….

Table of Contents ๐Ÿ—‚๏ธ

   1. Introduction ....................................................2
      1.1. Terminology ................................................4
   2. NCR Description .................................................5
   3. Algorithm .......................................................6
      3.1. Initialization .............................................8
      3.2. Terminating Extended Limited Transmit and
           Preventing Bursts ..........................................9
      3.3. Extended Limited Transmit .................................10
      3.4. Entering Loss Recovery ....................................11
   4. Advantages .....................................................12
   5. Disadvantages ..................................................12
   6. Related Work ...................................................13
   7. Security Considerations ........................................14
   8. Acknowledgments ................................................14
   9. IANA Considerations ............................................14
   10. References ....................................................14
      10.1. Normative References .....................................14
      10.2. Informative References ...................................15

1. Introduction ๐ŸŒˆ

Okay so one of TCP's best features [RFC793] is that it can adjust its sending rate based on the congestion it perceives in the network ๐Ÿš€ [Jac88, RFC2581]. Without explicit notification of congestion from the network, TCP takes segment loss as a sign of congestion (basically assuming a queue overflowed) ๐Ÿซฃ. TCP receivers send cumulative acknowledgments (ACKs) that tell the sender the next sequence number they expect for the segments that showed up ๐Ÿ“ฌ [RFC793]. When segments arrive out of order, duplicate ACKs get generated ๐Ÿ”. As [RFC2581] specifies, a TCP sender treats three duplicate ACKs as a sign of segment loss ๐Ÿ‘‘. The sender then retransmits the lost segment and eases off the load on the network, assuming the loss came from resource contention on the network path ๐Ÿซถ. The sender doesn't assume loss on the first or second duplicate ACK; it waits for three so it can handle minor packet reordering ๐Ÿฅน. The problem is that this constant threshold of duplicate ACKs causes several issues, and a dynamic threshold could help ๐Ÿ’–.

Here's an example of how TCP behaves ๐ŸŽฌ:

  • TCP A is the data sender, and TCP B is the data receiver.
  • TCP A sends 10 segments, each with a single data byte (so it sends bytes 1-10 in segments 1-10) ๐Ÿ“ฆ.
  • Assume segment 3 gets dropped in the network ๐Ÿ’€.
  • TCP B cumulatively acknowledges segments 1 and 2, so the cumulative ACK it sends back is 3 (the next expected sequence number). (Note: TCP B may generate one or two ACKs, depending on whether delayed ACKs [RFC1122, RFC2581] are employed.) ๐Ÿค“
  • Segments 4-10 each arrive at TCP B and each one triggers a cumulative ACK for sequence number 3. (Note: [RFC2581] recommends that delayed ACKs not be used when the ACK is triggered by an out-of-order segment.) ๐Ÿ™ƒ
  • When TCP A gets the third duplicate ACK (or the fourth ACK overall) for sequence number 3, it retransmits segment 3 and cuts its sending rate by roughly half (see [RFC2581] for the specifics of the congestion control state adjustments) ๐Ÿ“‰.

Now plot twist ๐ŸŽญ: suppose segment 3 wasn't dropped at all, and was just delayed so it reaches TCP B after segment 10. The exact same thing happens anyway, because segment 3 still gets retransmitted ๐Ÿ˜ฉ. In other words, TCP can't tell this reordering event apart from a real segment loss, so you get an unnecessary retransmission and rate reduction ๐Ÿ’….

Here's why TCP needs to be robust to reordered segments ๐Ÿง :

  • A bunch of Internet measurement studies have shown that packet reordering isn't rare at all [Pax97, BPS99, JIDKT03, GPL04]. And the reordering can go well beyond what's needed to falsely trigger fast retransmit ๐Ÿ˜ณ.
  • [BA02, ZKFP03] show the negative performance impact that packet reordering has on current TCP ๐Ÿ“‰.
  • TCP's requirement for almost in-order packet delivery puts a constraint on the design of future technology ๐Ÿšง. New routing algorithms, network components, link-layer retransmission mechanisms, and applications could all get a fresh look if TCP were more robust to segment reordering โœจ. For example, high-speed packet switches could resequence packets if TCP were more robust ๐ŸŽ๏ธ. There's been work in the literature that explicitly tries to keep packet ordering intact in such switches (e.g., [KM02]) ๐Ÿ“š. Link-layer mechanisms that try to recover from packet corruption by retransmitting could also be allowed to reorder packets, which increases the chances of repairing loss locally instead of making TCP do it (and needlessly cut its sending rate) ๐Ÿ”ง. More examples include multi-path routing, high-delay satellite links, and some of the schemes proposed for a differentiated services architecture ๐Ÿ›ฐ๏ธ. By making TCP more robust to non-congestion events, TCP-NCR may open up the design space for future Internet components ๐ŸŒธ.

In this document, we specify a set of TCP sender modifications that provide Non-Congestion Robustness (NCR) to TCP ๐Ÿ’Œ. Specifically, these changes are built on top of TCP with selective acknowledgments (SACKs) [RFC2018] and the SACK-based loss recovery scheme given in [RFC3517]. That's because SACK is widely deployed at this point ([MAF05] indicates that 68% of web servers and 88% of web clients utilize SACK as of spring 2004) ๐Ÿ“Š.

Also, the TCP-NCR algorithm in this document could be easily adapted to SCTP [RFC2960], since SCTP uses congestion control algorithms similar to TCP's (and so it has the same reordering robustness issues) ๐Ÿงš.

As noted in several places in the rest of this document, we consider TCP-NCR experimental ๐Ÿงช. That's because more experience with these techniques is needed before TCP-NCR should be used on a large scale on the Internet ๐Ÿ™. We encourage implementation and experimentation with TCP-NCR so we can understand how suitable it is for wide-scale deployment ๐Ÿ’œ.

Here's the rest of the doc's layout ๐Ÿ—บ๏ธ. Section 2 gives a high-level description of the TCP-NCR mechanisms ๐Ÿซง. In Section 3, we specify the TCP-NCR algorithm ๐Ÿงฎ. Section 4 gives a brief overview of the benefits of TCP-NCR, while Section 5 discusses the drawbacks ๐Ÿ‘€. Section 6 covers related work ๐Ÿ“š. Section 7 covers security concerns ๐Ÿ”.

1.1. Terminology ๐Ÿ“–

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119] ๐Ÿซถ.

Readers should already know the TCP terminology (e.g., FlightSize, Pipe) given in [RFC2581] and [RFC3517] ๐Ÿค“.

2. NCR Description โœจ

okay so bestie, the big problem is packet reordering ๐ŸŒ€. When packets arrive out of order, three duplicate ACKs might not be enough to tell if something got lost or if things were just shuffled ๐Ÿ™ƒ. This section is a non-normative sketch of TCP-NCR ๐Ÿ’–. The detailed algorithms come in the next section, so hang tight ๐ŸŽ€.

The main idea is to raise the bar for fast retransmission. Instead of the fixed three duplicate ACKs from [RFC2581], TCP-NCR waits until roughly a congestion window's worth of data has left the network ๐Ÿฆ‹. It still never goes below the currently standardized value of three duplicate ACKs ๐Ÿ’…. Here's the thing: cwnd is basically how much data a TCP flow can send in one round-trip time (RTT) โฑ๏ธ. So waiting for cwnd bytes to leave the network before deciding "loss or reordering?" adds a delay of about one RTT to both the retransmission and the congestion control response ๐Ÿ˜ญ. The threshold is a trade-off between making the best call about the cause of the duplicate ACKs and being responsive โš–๏ธ. Waiting for a whole cwnd of data to clear is roughly the longest a TCP can wait before the (often costly) retransmission timeout might kick in ๐Ÿ’€. So this algorithm chooses to make the best decision possible, even if it's a little less timely ๐Ÿซถ.

Simply raising the threshold can make TCP brittle to packet loss or ACK loss ๐Ÿฅบ. Loss means fewer duplicate ACKs reach the sender from the receiver, so the count might never get there. For example, if cwnd is 10 segments and one segment is lost, a duplicate ACK threshold of 10 will never be met, because duplicate ACKs for at most 9 segments will reach the sender.

To fix that, TCP-NCR extends TCP's Limited Transmit [RFC3042] ๐Ÿงš. During the period when the sender is figuring out loss versus reordering, it's allowed to send new data ๐Ÿ“ฆ. That new data helps make sure enough duplicate ACKs show up to trigger loss recovery, if recovery is actually the right move ๐Ÿ™Œ.

Also, TCP tightly couples reliability and congestion control ๐Ÿ”—. When a segment is declared lost, a retransmission fires, and the sending rate gets cut too, based on the assumption that the drop came from resource contention [RFC2581]. So if you change the retransmission trigger, you automatically change the congestion control response as well ๐Ÿ˜ณ. The catch is that we don't yet have Internet experience showing whether delaying the rate reduction is okay for wide-scale deployment ๐Ÿค”. That's why the Extended Limited Transmit mechanism in this document comes with two variants for experimentation ๐Ÿงช.

The first one is Careful Limited Transmit ๐ŸŒธ. For every two segments known to have left the network, it sends one previously unsent segment in response to duplicate ACKs. That effectively halves the sending rate, since normal TCP sends one segment for every segment that has left the network ๐Ÿ“‰. The halving starts right away and doesn't wait for a retransmission to be triggered โšก. If the cause turns out to be packet reordering rather than segment loss, the congestion control state gets restored to what it was before, once reordering is determined ๐Ÿ”„.

The second one is Aggressive Limited Transmit ๐Ÿ”ฅ. For every segment known to have left the network, it sends one previously unsent data segment in response to duplicate ACKs ๐Ÿš€. So while it's waiting to tell loss apart from reordering, ACK-clocked transmission keeps going at roughly the same rate it had before the event started ๐Ÿƒโ€โ™€๏ธ. Retransmission and the sending rate reduction still happen per [RFC2581, RFC3517], just with the delayed threshold we talked about earlier โณ. Yes, this delays legitimate rate reductions, which could slightly and temporarily make overall network congestion worse ๐Ÿ˜ฌ. But the upside is that it doesn't cut the transmission rate just because segments got reordered ๐Ÿ’œ.

Which of the two Extended Limited Transmit variants is best for the Internet? Honestly, that's still an open question ๐Ÿคทโ€โ™€๏ธโœจ.

3. Algorithm ๐Ÿ’…

The TCP-NCR changes make two big updates to how [RFC3517] currently works ๐ŸŽ€. First, the trigger for retransmitting a segment moves from three duplicate ACKs [RFC2581, RFC3517] to indications that a congestion window's worth of data has left the network ๐ŸŒŸ. Second, TCP-NCR decouples the initial congestion control decisions from the retransmission decisions ๐Ÿ’ซ. In some cases that means congestion control changes happen later than they do in TCP's current behavior as defined in [RFC2581] โฐ. The algorithm gives two ways to extend Limited Transmit, and they are:

  • Careful Limited Transmit: This variant reduces the sending rate at about the same time that [RFC2581] implementations reduce the congestion window ๐Ÿ“‰. At the same time, it holds off on the retransmission (and the final congestion determination) for about one RTT ๐Ÿซ .
  • Aggressive Limited Transmit: This variant keeps the sending rate steady in the face of duplicate ACKs, until TCP decides a segment is lost and needs retransmitting ๐ŸŽ๏ธ. TCP-NCR delays that decision by one RTT compared with current loss recovery schemes ๐Ÿ•ฐ๏ธ.

A TCP-NCR implementation MUST use either Careful Limited Transmit or Aggressive Limited Transmit ๐Ÿ’–. No mixing and matching, bestie ๐Ÿ™…โ€โ™€๏ธ.

Next, a constant MUST be set based on which variant of extended Limited Transmit you're using ๐Ÿงฎ:

  • Careful Limited Transmit: LT_F = 2/3
  • Aggressive Limited Transmit: LT_F = 1/2

This constant is the fraction of outstanding data (including data sent during Extended Limited Transmit) that has to be SACKed before a retransmission gets triggered ๐ŸŽฏ. With Aggressive Limited Transmit, a new segment goes out for every segment known to have left the network. So roughly cwnd segments get sent during that phase, and ideally about 2*cwnd segments are outstanding when a retransmission is triggered ๐Ÿ“Š. That means the duplicate ACK threshold is LT_F = 1/2 of 2*cwnd, which is about 1 RTT worth of data ๐ŸŒˆ. Careful Limited Transmit is different, because the sender only sends one new segment for every two segments that get SACKed. So it ideally ends up with about 1.5*cwnd segments outstanding when the retransmission is triggered ๐Ÿง . That means the threshold is LT_F=2/3 of 1.5*cwnd, which delays the retransmission by roughly 1 RTT โณโœจ.

Sometimes the sender can't send new data during Extended Limited Transmit ๐Ÿ˜ฉ. Maybe the application has no data to send, or the receiver's advertised window is full ๐Ÿ“ญ. Those situations can cause the problems we talked about in the last section, especially when a TCP doesn't use Extended Limited Transmit and gets starved for ACKs ๐Ÿฅ€. So TCP-NCR adjusts the duplicate ACK threshold on each SACK arrival, so it's as robust as possible given how much data was actually sent. Basically, that's roughly LT_F times the number of outstanding segments ๐Ÿงฉ.

The TCP-NCR changes in this document are built for incremental deployment ๐ŸŒฑ. Only the TCP implementation on the sender side needs to change (assuming both hosts support SACK) ๐Ÿ™Œ. The changes themselves are pretty modest ๐Ÿ’…. As we'll cover below, extra buffer space at the receiver will help you get the most out of TCP-NCR, but it's not strictly necessary ๐ŸŽ.

These algorithms depend on the concepts from [RFC3517], and we assume you know the terminology from [RFC3517] ๐Ÿ“š. The TCP-NCR algorithm can be adapted to other SACK-based loss recovery schemes ๐Ÿ”ง. [BR04, BSRV04] outline non-SACK-based algorithms ๐Ÿ—บ๏ธ. We don't specify those in this document, and we do not recommend them, because of both the complexity and the security implications of only having a rough idea of how many segments are outstanding in the network ๐Ÿ™ˆ.

A TCP connection using the Nagle algorithm [RFC896, RFC1122] MAY employ the TCP-NCR algorithm ๐ŸŒธ. If a TCP implementation does implement TCP-NCR, the implementation MUST follow the specifications in Sections 3.1 - 3.4 ๐Ÿซถ. Without the Nagle algorithm, there's no accurate way to calculate how many segments are outstanding in the network (and so no good way to derive a proper duplicate ACK threshold) without adding state to the TCP sender ๐Ÿ˜ตโ€๐Ÿ’ซ. So a TCP connection that does not employ the Nagle algorithm SHOULD NOT use TCP-NCR ๐Ÿšซ. We think NCR could be adapted to an implementation that carefully tracks the sequence numbers sent in each segment ๐Ÿ‘€. But that's left as future work ๐Ÿ”ฎ.

3.1. Initialization ๐ŸŒท

When entering a period of loss/reordering detection and Extended Limited Transmit, a TCP-NCR MUST initialize several state variables ๐Ÿง. A TCP MUST enter Extended Limited Transmit when it receives the first ACK with a SACK block after receiving an ACK that (a) did not contain SACK information and (b) did increase the connection's cumulative ACK point ๐Ÿšช. Here are the initializations ๐Ÿ“:

(I.1) The TCP MUST save the current FlightSize ๐Ÿ’พ.

FlightSizePrev = FlightSize

(I.2) The TCP MUST set a variable for tracking the number of segments for which an ACK does not trigger a transmission during Careful Limited Transmit ๐Ÿ”ข.

Skipped = 0

(Note: Skipped is not used during Aggressive Limited Transmit.)

(I.3) The TCP MUST set DupThresh (from [RFC3517]) based on the current FlightSize ๐ŸŽฏ.

DupThresh = max (LT_F * (FlightSize / SMSS),3)

Note: We keep the lower bound of DupThresh = 3 from [RFC2581, RFC3517] ๐Ÿ›ก๏ธ.

On top of all the steps above, the incoming ACK MUST be processed with the E series of steps in Section 3.3 ๐ŸŽ€โœจ.

3.2. Terminating Extended Limited Transmit and Preventing Bursts โœจ

Extended Limited Transmit MUST be terminated at the start of loss recovery, as laid out in Section 3.4. ๐Ÿ’…

Okay so, if an ACK comes in that moves the cumulative ACK point forward while we're still in Extended Limited Transmit, but before loss recovery has kicked in, that's a clue. It means the string of duplicate ACKs was caused by reordering and not congestion ๐Ÿ™ˆ So an ACK that extends the cumulative ACK point MUST terminate Extended Limited Transmit. There's a twist too: an ACK that extends the cumulative ACK point and also carries SACK information will (see (T.4) below) start a brand new Extended Limited Transmit phase ๐Ÿ”„

Now, once Extended Limited Transmit ends, especially if you're using the Careful variant, TCP-NCR might find that the whole cwnd isn't actually being used. That makes it prone to dumping a burst of segments into the network ๐Ÿ˜ญ To stop that, whenever a TCP-NCR in the Extended Limited Transmit phase gets an ACK that updates the cumulative ACK point (SACK info or not), the following steps MUST be taken:

(T.1) A TCP MUST reset cwnd to:

cwnd = min (FlightSize + SMSS,FlightSizePrev)

This keeps cwnd from being way bigger than the amount of data actually outstanding. Otherwise you'd get a line-rate burst, and nobody wants that ๐ŸŒŠ

(T.2) A TCP MUST set ssthresh to:

ssthresh = FlightSizePrev

This gives TCP-NCR a sense of "history" ๐Ÿง  If step (T.1) pushes cwnd below FlightSizePrev, this step makes sure TCP-NCR slow starts back up to the operating point it had before Extended Limited Transmit. Main character memory, honestly.

(T.3) A TCP is now allowed to send previously unsent data, as long as cwnd, FlightSize, application data availability, and the receiver's advertised window all say yes ๐ŸŽ€

(T.4) When an incoming ACK extends the cumulative ACK point and also contains SACK information, the initializations in steps (I.2) and (I.3) from Section 3.1 MUST be taken, but step (I.1) MUST NOT be executed, to restart Extended Limited Transmit ๐Ÿฆ‹ On top of that, the series of steps in Section 3.3 (the "E" steps) MUST be taken.


3.3. Extended Limited Transmit ๐ŸŒธ

On every ACK that carries SACK information and arrives after TCP-NCR has entered the Extended Limited Transmit phase (as laid out in Section 3.1), and before Extended Limited Transmit ends, the sender MUST follow this procedure ๐Ÿ’œ

(E.1) The SetPipe () procedure from [RFC3517] MUST be used to set the "pipe" variable, which is the number of bytes still counted as "in the network" ๐Ÿงฎ Heads up: the current value of DupThresh MUST be used by SetPipe () so it can give an accurate picture of how much data is still in the network.

(E.2) If the comparison in equation (1) below holds, and there are SMSS bytes of previously unsent data ready to go, then the sender MUST transmit one segment of SMSS bytes ๐Ÿš€

(pipe + Skipped) <= (FlightSizePrev - SMSS)              (1)

If equation (1) doesn't hold, or no new data can be sent (because the application has nothing to give or the advertised window is maxed out), skip ahead to step (E.6) โญ๏ธ

(E.3) Pipe MUST be incremented by SMSS bytes โž•

(E.4) If you're using Careful Limited Transmit, Skipped MUST be incremented by SMSS bytes. This makes sure the next SMSS bytes of SACKed data processed don't trigger a Limited Transmit transmission. The whole goal of Careful Limited Transmit is to send upon receipt of every second duplicate ACK, so this keeps that promise ๐Ÿซถ

(E.5) A TCP MUST return to step (E.2) so that as many bytes as are appropriate get transmitted. This adds robustness to ACK loss, which SACK information can (largely) make up for ๐Ÿ”

(E.6) DupThresh MUST be reset via:

DupThresh = max (LT_F * (FlightSize / SMSS),3)

Here FlightSize is the total number of bytes that haven't been cumulatively acknowledged. Quick reminder: that's not the same thing as "pipe" ๐Ÿ’…


3.4. Entering Loss Recovery ๐Ÿšจ

When a segment gets marked as lost by the algorithms in [RFC3517], Extended Limited Transmit MUST be terminated, and then the algorithms in [RFC3517] take over and govern TCP's behavior ๐Ÿ‘‘ There's one small change to [RFC3517] that MUST be made, though. In Section 5, step (2) of [RFC3517] MUST be changed to:

    (2) ssthresh = cwnd = (FlightSizePrev / 2)

This makes sure the congestion control changes are based on how much data was in the network before Extended Limited Transmit bumped up FlightSize ๐ŸŽฏ

Note: Once the algorithm in [RFC3517] takes over from Extended Limited Transmit, the DupThresh value MUST be held constant until the loss recovery phase is terminated. No wiggling DupThresh mid-recovery, bestie ๐Ÿ™…โ€โ™€๏ธ

4. Advantages ๐ŸŒˆ

The big advantages of TCP-NCR are twofold ๐Ÿ’– As we talked about in Section 1, TCP-NCR opens up the design space for network applications and components that are currently boxed in by TCP's lack of robustness to packet reordering. The second advantage is a bump in TCP performance โญ

[BR04] shows ns-2 [NS-2] simulations of a precursor to the TCP-NCR algorithm specified in this document, called TCP-DCR (Delayed Congestion Response) โœจ The paper shows that TCP-DCR helps performance compared to unmodified TCP when packets get reordered. The extended version of [BR04] also has results from emulations on Linux (kernel 2.4.24). Those results show that TCP-DCR performs similarly to Linux's native implementation, which tries to "undo" wrong decisions based on duplicate-SACK (DSACK) [RFC2883] feedback (similar to the schemes outlined in [ZKFP03]) when packets are reordered by less than one RTT ๐Ÿ” The upside of TCP-DCR over the DSACK-based scheme is that the DSACK-based scheme tries to estimate the exact amount of reordering in the network using fairly complex algorithms, while TCP-DCR gets similar results with less complicated changes ๐Ÿง โœจ

[BR04,BSRV04] also show how TCP-DCR can help improve other parts of the system ๐Ÿฆ‹ For example, these papers show that making TCP more robust to packet reordering lets a novel wireless ARQ mechanism be added at the link layer. The link layer's added robustness to channel errors, in turn, boosts TCP performance, because TCP doesn't have to retransmit packets that were dropped due to corruption. That also stops TCP from needlessly slowing its sending rate while retransmitting those segments ๐Ÿ™Œ


5. Disadvantages ๐Ÿ˜ฌ

Even though all the changes above are implemented in the sender, the receiver also potentially has a part to play ๐ŸŽญ In particular, TCP-NCR raises the receiver's buffering requirement by up to an extra cwnd -- in the case where the TCP sender is using Aggressive Limited Transmit and actual loss is happening in the network. So, to get the most out of TCP-NCR, receivers should advertise a large window to absorb the extra out-of-order traffic ๐ŸชŸ If the extra buffer requirements aren't met, the algorithm above takes the reduced advertised window into account, but robustness to packet reordering drops as a result ๐Ÿ’€

Also, using TCP-NCR could delay delivery of data to the application by up to one RTT, since the fast retransmission point gets pushed back by roughly one RTT in TCP-NCR โณ Applications that are sensitive to those delays should turn off the TCP-NCR option. For instance, a socket option could be introduced so applications can control whether NCR is used for a particular connection ๐ŸŽ›๏ธ

Finally, TCP-NCR makes recovery from congestion events sluggish compared to the standard reaction in [RFC2581] ๐Ÿข [BR04, BSRV04] show (via simulation) that the delay in congestion response has minimal impact on the connection itself and on the traffic sharing a bottleneck. [BBFS01] also indicates (again, via simulation) that "slowly responsive" congestion control may be safe to deploy on the Internet ๐ŸŒ These studies suggest that schemes which slightly delay congestion control decisions may be reasonable. However, further experimentation on the Internet is required to verify these results ๐Ÿ”ฌ


Over the past few years, a bunch of solutions have been proposed to improve TCP performance when segments get reordered ๐Ÿงฉ These schemes generally fall into two categories (with some overlap): mechanisms that try to prevent spurious retransmits from happening, and mechanisms that try to detect spurious retransmits and "undo" the needless congestion control state changes that were made ๐Ÿ”™

[BA02,ZKFP03] try to stop segment reordering from triggering spurious retransmits by using various algorithms to estimate the duplicate ACK threshold needed to tell loss and reordering apart over a given network path at a given time ๐Ÿ•ต๏ธโ€โ™€๏ธ TCP-NCR also tries to prevent spurious retransmits, but it takes a simpler approach than [BA02, ZKFP03]. TCP-NCR simply delays retransmission by an amount based on the current cwnd (compared to standard TCP), while the other schemes use relatively complex algorithms to derive a more precise DupThresh value that depends on current reordering patterns. TCP-NCR is simple, but the other schemes may be more precise, so applications wouldn't have to wait as long for their retransmissions. Future work could aim for robustness without needless delay ๐ŸŒฑ

On the other hand, several schemes have been developed to detect and mitigate needless retransmissions after the fact ๐Ÿ•ฐ๏ธ [RFC3522, RFC3708, BA02, RFC4015, RFC4138] present algorithms to detect spurious retransmits and reduce the changes those events made to the congestion control state. TCP-NCR could be used together with these algorithms, with TCP-NCR trying to prevent spurious retransmits and some other scheme kicking in if that prevention fails ๐Ÿค Also, note that TCP-NCR is focused on preventing spurious fast retransmits. Some of the above algorithms also try to detect and mitigate spurious timeout-based retransmits โฑ๏ธ


7. Security Considerations ๐Ÿ”

General attacks against TCP's congestion control are described in [RFC2581] ๐Ÿšจ SACK-based loss recovery for TCP [RFC3517] mitigates some of the duplicate ACK attacks against TCP's congestion control. This document builds on that work, and the Extended Limited Transmit algorithms specified here were designed to thwart the ACK division problems described in [RFC3465] ๐Ÿ›ก๏ธ Security is not a vibe you can skip, so take this section seriously, bestie ๐Ÿ˜ค


8. Acknowledgments ๐Ÿซถ

Feedback from Lars Eggert, Ted Faber, Wesley Eddy, Gorry Fairhurst, Sally Floyd, Sara Landstrom, Nauzad Sadry, Pasi Sarolahti, Joe Touch, Nitin Vaidya, and the TCPM working group contributed significantly to this document ๐Ÿ’• Our thanks to all, you're all iconic! ๐Ÿ‘‘


9. References ๐Ÿ“–

9.1. Normative References ๐Ÿ’–

These are the must-reads, bestie. Everything here is required background for the spec.

[RFC793]  Postel, J., "Transmission Control Protocol", STD 7, RFC
          793, September 1981.

[RFC2018] Mathis, M., Mahdavi, J., Floyd, S., and A. Romanow, "TCP
          Selective Acknowledgement Options", RFC 2018, October 1996.

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
          Requirement Levels", BCP 14, RFC 2119, March 1997.

[RFC2581] Allman, M., Paxson, V., and W. Stevens, "TCP Congestion
          Control", RFC 2581, April 1999.

[RFC3042] Allman, M., Balakrishnan, H., and S. Floyd, "Enhancing
          TCP's Loss Recovery Using Limited Transmit", RFC 3042,
          January 2001.

[RFC3517] Blanton, E., Allman, M., Fall, K., and L. Wang, "A
          Conservative Selective Acknowledgment (SACK)-based Loss
          Recovery Algorithm for TCP", RFC 3517, April 2003.

9.2. Informative References โœจ

These are the extra reading for when you want to go deeper. Totally optional, but so worth it ๐Ÿงš

[BA02]    E. Blanton and M. Allman, "On Making TCP More Robust to
          Packet Reordering," ACM Computer Communication Review,
          January 2002.

[BBFS01]  D. Bansal, H. Balakrishnan, S. Floyd and S. Shenker,
          "Dynamic Behavior of Slowly Responsive Congestion Control
          Algorithms", Proceedings of ACM SIGCOMM, Sep. 2001.

[BPS99]   J. Bennett, C. Partridge, and N. Shectman, "Packet
          reordering is not pathological network behavior," IEEE/ACM
          Transactions on Networking, December 1999.

[BR04]    Sumitha Bhandarkar and A. L. Narasimha Reddy, "TCP-DCR:
          Making TCP Robust to Non-Congestion Events", In the
          Proceedings of Networking 2004 conference, May 2004.
          Extended version available as tech report TAMU-ECE-2003-04.

[BSRV04]  Sumitha Bhandarkar, Nauzad Sadry, A. L. Narasimha Reddy and
          Nitin Vaidya, "TCP-DCR: A Novel Protocol for Tolerating
          Wireless Channel Errors", to appear in IEEE Transactions on
          Mobile Computing.

[GPL04]   Ladan Gharai, Colin Perkins and Tom Lehman, "Packet
          Reordering, High Speed Networks and Transport Protocol
          Performance", ICCCN 2004, October 2004.

[Jac88]   V. Jacobson, "Congestion Avoidance and Control", Computer
          Communication Review, vol. 18, no. 4, pp. 314-329, Aug.
          1988.  ftp://ftp.ee.lbl.gov/papers/congavoid.ps.Z.

[JIDKT03] S. Jaiswal, G. Iannaccone, C. Diot, J. Kurose, and D.
          Towsley, "Measurement and Classification of Out-of-Sequence
          Packets in a Tier-1 IP Backbone," Proceedings of IEEE
          INFOCOM, 2003.

[KM02]    I. Keslassy and N. McKeown, "Maintaining packet order in
          twostage switches," Proceedings of the IEEE Infocom, June
          2002

[MAF05]   A. Medina, M. Allman, S. Floyd.  Measuring the Evolution of
          Transport Protocols in the Internet.  ACM Computer
          Communication Review, 35(2), April 2005.

[NS-2]    ns-2 Network Simulator. http://www.isi.edu/nsnam/

[Pax97]   V. Paxson, "End-to-End Internet Packet Dynamics,"
          Proceedings of ACM SIGCOMM, September 1997.

[RFC896]  Nagle, J., "Congestion control in IP/TCP internetworks",
          RFC 896, January 1984.

[RFC1122] Braden, R., "Requirements for Internet Hosts -
          Communication Layers", STD 3, RFC 1122, October 1989.

[RFC2883] Floyd, S., Mahdavi, J., Mathis, M., and M. Podolsky, "An
          Extension to the Selective Acknowledgement (SACK) Option
          for TCP", RFC 2883, July 2000.

[RFC2960] R. Stewart, Q. Xie, K. Morneault, C. Sharp, H.
          Schwarzbauer, T. Taylor, I. Rytina, M. Kalla, L. Zhang, V.
          Paxson.  Stream Control Transmission Protocol.  October
          2000.

[RFC3465] Allman, M., "TCP Congestion Control with Appropriate Byte
          Counting (ABC)", RFC 3465, February 2003.

[RFC3522] Ludwig, R. and M. Meyer, "The Eifel Detection Algorithm for
          TCP", RFC 3522, April 2003.

[RFC3708] Blanton, E. and M. Allman, "Using TCP Duplicate Selective
          Acknowledgement (DSACKs) and Stream Control Transmission
          Protocol (SCTP) Duplicate Transmission Sequence Numbers
          (TSNs) to Detect Spurious Retransmissions", RFC 3708,
          February 2004.

[RFC4015] Ludwig, R. and A. Gurtov, "The Eifel Response Algorithm for
          TCP", RFC 4015, February 2005.

[RFC4138] Sarolahti, P. and M. Kojo, "Forward RTO-Recovery (F-RTO):
          An Algorithm for Detecting Spurious Retransmission Timeouts
          with TCP and the Stream Control Transmission Protocol
          (SCTP)", RFC 4138, August 2005.

[ZKFP03]  M. Zhang, B. Karp, S. Floyd, L. Peterson, "RR-TCP: A
          Reordering-Robust TCP with DSACK", in Proceedings of the
          Eleventh IEEE International Conference on Networking
          Protocols (ICNP 2003), Atlanta, GA, November, 2003.

Authors' Addresses ๐ŸŽ€

Meet the brains behind it! These are the people who wrote it, so if you have questions, they're right here ๐Ÿ’…

Sumitha Bhandarkar
Dept. of Elec. Engg.
214 ZACH
College Station, TX 77843-3128

Phone: (512) 468-8078
EMail: sumitha@tamu.edu
URL: http://students.cs.tamu.edu/sumitha/


A. L. Narasimha Reddy
Professor
Dept. of Elec. Engg.
315C WERC
College Station, TX 77843-3128

Phone: (979) 845-7598
EMail: reddy@ee.tamu.edu
URL: http://ee.tamu.edu/~reddy/


Mark Allman
ICSI Center for Internet Research
1947 Center Street, Suite 600
Berkeley, CA 94704-1198

Phone: (440) 235-1792
EMail: mallman@icir.org
URL: http://www.icir.org/mallman/


Ethan Blanton
Purdue University Computer Science
305 North University Street
West Lafayette, IN  47907

EMail: eblanton@cs.purdue.edu

Copyright (C) The Internet Society (2006).

This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights.

This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property ๐ŸŒธ

The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79.

Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr.

The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org.

Acknowledgement ๐Ÿซถ

Funding for the RFC Editor function is provided by the IETF Administrative Support Activity (IASA).

๐Ÿ“ฆ where this page comes from