RFC 5690
Adding Acknowledgement Congestion Control to TCP
Adding Acknowledgement Congestion Control to TCP 💖
okay bestie, this is RFC 5690 🎀 It's an Independent Submission, so it came in outside the usual IETF stream, and it's marked Category: Informational 🌸 It's dated February 2010 and has ISSN 2070-1721 ✨ The authors are S. Floyd (ICIR), A. Arcia, D. Ros (TELECOM Bretagne), and J. Iyengar (Franklin & Marshall College) 🧚 The title block doesn't list anything it obsoletes or updates, so there's no other RFC to chase down here 💅
Abstract 🦋
okay so this document describes a possible congestion control mechanism for acknowledgement (ACKs) traffic in TCP 🌊 It's an end-to-end setup that needs both TCP hosts to take part: the TCP data sender and the TCP data receiver 🫶 The sender spots lost or Explicit Congestion Notification (ECN)-marked ACK packets, then tells the receiver which ACK Ratio R to use to respond to congestion on the reverse path, meaning the path from the data receiver back to the data sender 🔥 The receiver then sends roughly one ACK packet for every R data packets it gets ✨ This idea is based on the acknowledgement congestion control in the Datagram Congestion Control Protocol's (DCCP's) Congestion Control Identifier (CCID) 2 💜 The mechanism is being specified so the network community can evaluate it further 🧚
Status of This Memo 📜
This document is not an Internet Standards Track specification, so it's published for informational purposes only 🌸
This is a contribution to the RFC Series, made independently of any other RFC stream 💅 The RFC Editor chose to publish it at their discretion, and they make no statement about its value for implementation or deployment 🙈 Documents the RFC Editor approves for publication aren't candidates for any level of Internet Standard, and you can read more in Section 2 of RFC 5741 ✨
For the current status of this document, any errata, and how to send feedback, check http://www.rfc-editor.org/info/rfc5690. 🎀
IESG Note 🌈
okay, so the content of this RFC was once considered by the IETF 💅 That means it might look a lot like a current IETF work in progress or a published IETF work 👀
Copyright Notice 💜
Copyright (c) 2010 IETF Trust and the persons identified as the document authors 🫶 All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as they were in effect on the date this document was published. Please read these carefully, because they spell out your rights and restrictions for this document 📜
Table of Contents 🗂️
1. Introduction ....................................................3
2. Conventions and Terminology .....................................4
3. Overview ........................................................4
4. Acknowledgement Congestion Control ..............................6
4.1. The ACK Congestion Control Permitted Option ................6
4.2. The TCP ACK Ratio Option ...................................7
4.3. The Receiver: Implementing the ACK Ratio ...................7
4.4. The Sender: Determining Lost or Marked ACK Packets .........8
4.4.1. The Sender: Detecting Lost ACK Packets
after a Congestion Event ...........................10
4.5. The Sender: Adjusting the ACK Ratio .......................10
4.5.1. Possible Addition: Decreasing the ACK Ratio
after a Congestion Window Decrease .................12
4.6. The Receiver: Sending ACKs for Out-of-Order Data
Segments ..................................................12
4.7. The Sender: Response to ACK Packets .......................13
4.8. Possible Addition: Receiver Bounds on the ACK Ratio .......15
5. Possible Complications .........................................15
5.1. Possible Complication: Delayed Acknowledgements ...........15
5.2. Possible Complication: Duplicate Acknowledgements .........15
5.3. Possible Complication: Two-Way Traffic ....................16
5.4. Possible Complication: Reordering of ACK Packets ..........16
5.5. Possible Complication: Abrupt Changes in the ACK Path .....17
5.6. Possible Complication: Corruption .........................17
5.7. Possible Complication: ACKs That Don't Contribute
to Congestion .............................................17
5.8. Possible Complication: TCP Implementations that
Skip ACK Packets ..........................................20
5.9. Possible Complication: Router or Middlebox-Based
ACK Mechanisms ............................................21
5.10. Possible Complication: Data-Limited Senders ..............21
5.11. Other Issues .............................................22
6. Evaluating ACK Congestion Control ..............................22
6.1. Contention in Wireless Links or in Non-Switched Ethernet ..22
6.2. Keep-Alive and Other Special ACK Packets ..................22
7. Measurements of ACK Traffic and Congestion .....................23
8. Acknowledgement Congestion Control in DCCP's CCID 2 ............23
9. Security Considerations ........................................24
10. IANA Considerations ...........................................25
11. Conclusions ...................................................26
12. Acknowledgements ..............................................26
Appendix A. Related Work ..........................................27
A.1. ECN-Only Mechanisms .......................................28
A.2. Receiver-Only Mechanisms ..................................28
A.3. Middlebox-Based Mechanisms ................................29
Appendix B. Design Considerations .................................29
B.1. The TCP ACK Ratio Option, or an AckNow Bit in
Data Packets? .............................................29
Normative References ..............................................30
Informative References ............................................301. Introduction 🎀
okay so this document describes a congestion control mechanism for acknowledgements (ACKs) to TCP 🦋 It's based on the acknowledgement congestion control in DCCP's CCID 2 ([RFC4340], [RFC4341]), which is the successor to the TCP acknowledgement congestion control mechanism that Balakrishnan, et al. proposed in [BPK97] 💜
In this document we use the language of senders and receivers: the sender sends data traffic, and the receiver sends acknowledgement traffic back in response 🫶 In CCID 2's acknowledgement congestion control, which is specified in Section 6.1 of [RFC4341], the receiver uses an ACK Ratio R that the sender reports to it, so it sends roughly one ACK packet for every R data packets it receives ✨ The CCID 2 sender keeps the acknowledgement rate roughly TCP-friendly by watching the acknowledgement stream for lost and marked ACK packets and changing the ACK Ratio to match 🌸 For every round-trip time (RTT) that contains an ACK congestion event (meaning a lost or marked ACK packet), the sender halves the acknowledgement rate by doubling the ACK Ratio 😭 For every RTT with no ACK congestion event, the sender increases the acknowledgement rate additively, through gradual decreases in the ACK Ratio 📈
The goal of this document is to explore a similar congestion control mechanism for acknowledgement traffic in TCP 🧚 The idea is that in some environments where the reverse path is congested, cutting back the sending rate for ACK traffic on that congested path can help ease the congestion itself 🌊 For environments where the reverse path is congested but TCP ACK traffic doesn't really add to that overall congestion, the aim is for TCP's ACK congestion control to have a minimal negative effect on the performance of the TCP connection 💅
So adding acknowledgement congestion control as a TCP option would require the following 👑
- An agreement between the TCP hosts to use ACK congestion control. For the mechanism in this document, the TCP hosts would use a new TCP option, the ACK Congestion Control Permitted option. 🎀
- A way for the TCP sender to detect lost and ECN-marked pure acknowledgement packets. 🔍
- A way to adjust the ACK Ratio. The TCP sender would adjust the ACK Ratio as specified in Section 6.1.2 of [RFC4341]. 📊
- A method for the TCP sender to tell the TCP receiver about a new value for the ACK Ratio. For the mechanism in this document, the TCP sender would use a new TCP option, the ACK Ratio option. 💌
2. Conventions and Terminology 📚
okay, quick vocab check 🤓 MSS refers to the Maximum Segment Size 💖
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. These are the official BCP 14 key words, so read them exactly as the spec means them 🫶
3. Overview 🦋
okay so this section gives us the big-picture tour of acknowledgement congestion control (AckCC) for TCP, and bestie, it's giving main character energy 💅✨
Check out Figure 1 below. It's a little story in four acts, with TCP Host A as the data sender, TCP Host B as the data receiver, and a router in the middle, and it's the perfect example of how this whole thing plays out when Host B is the connection initiator and there's no ECN in the mix 🌸
---------------------------------------------------------------
TCP Host A Router TCP Host B
(data sender) (data receiver)
---------- ------ ----------
<--- SYN with AckCC Permitted.
SYN/ACK with AckCC Permitted --->
. . .
Data packets --->
<--- one ACK packet
for every two data packets
. . .
Sender detects a lost ACK packet.
Data packet with an ACK Ratio option of 4 --->
<--- one ACK packet
for at most every four data packets
. . .
Sender detects a period with no lost ACK packets.
Data packet with an ACK Ratio option of 3 --->
<--- one ACK packet
for at most every three data packets
---------------------------------------------------------------
Figure 1: Acknowledgement Congestion Control,
Host B as the Connection Initiator, for a Connection without ECNFigure 1 gives an example of acknowledgement congestion control (AckCC) with TCP Host B as the connection initiator. 🎀
During connection initiation, TCP host B sends an ACK Congestion Control Permitted option on its SYN or SYN/ACK packet. This allows TCP host A (now called the sender) to send instructions to TCP host B (now called the receiver) about the ACK Ratio to use in responding to data packets. 💖
Also during connection initiation, TCP host A sends an ACK Congestion Control Permitted option on its SYN or SYN/ACK packet. In combination with TCP host B's sending of an ACK Congestion Control Permitted option, and with the negotiation of ECN-Capability as specified in [RFC3168], this would allow TCP host B to send its ACK packets as ECN-Capable. 🧚
The TCP receiver starts with an ACK Ratio of two, generally sending one ACK packet for every two data packets received. 🍓
The TCP sender detects a lost or ECN-marked ACK packet from the TCP receiver and sends an ACK Ratio option of four to the receiver. The TCP receiver changes to an ACK Ratio of four, sending one ACK packet for at most four data packets. The TCP sender uses Appropriate Byte Counting and rate-based pacing in responding to these ACK packets. 🫶
The TCP sender detects a period with no lost ACK packets and sends an ACK Ratio option of three to the TCP receiver. The TCP receiver changes back to an ACK Ratio of three, sending one ACK packet for at most three data packets. 🌈
4. Acknowledgement Congestion Control 👑
ngl this is the core of the whole thing 😭 The goal of the mechanism proposed in this document is to control pure ACK traffic on the path from the TCP data receiver to the TCP data sender. Note that the approach outlined here is an end-to-end one (as is the approach followed by DCCP's CCID 2 [RFC4341]), but it may also take advantage of explicit congestion information from the network, conveyed by ECN [RFC3168], if available. 💜
The ECN specification ([RFC3168], see Section 6.1.4) prohibits a TCP receiver from setting the ECT(0) or ECT(1) codepoints in IP packets carrying pure ACKs, but only as long as the receiver does not implement any form of ACK congestion control. Unlike some of the related work cited in the appendix, in this document we are proposing an end-to-end ACK congestion control mechanism that controls congestion on the reverse path (the path followed by the ACK traffic) by detecting and responding to either marked or dropped ACK packets. 🔥
4.1. The ACK Congestion Control Permitted Option
The TCP end-points would negotiate the use of ACK congestion control (AckCC) with a TCP option: the ACK Congestion Control Permitted option. The option number would be allocated by IANA. 🎀
The ACK Congestion Control Permitted option can only be sent on packets that have the SYN bit set. If TCP end-point A receives an ACK Congestion Control Permitted option from TCP end-point B, then the TCP end-points may use ACK congestion control on the pure acknowledgements sent from B to A. This means that TCP end-point A may send ACK Ratio values to TCP end-point B, for TCP end-point B to use on pure acknowledgement packets. Equivalently, if TCP end-point A does not receive an ACK Congestion Control Permitted option from TCP end-point B, then TCP end-point A knows not to waste its time detecting lost ACK packets and adjusting and sending the ACK Ratio values. 🙈
If TCP end-point B receives an ACK Congestion Control Permitted option from TCP end-point A, then the TCP end-points may use ACK congestion control on the pure acknowledgements sent from A to B. 🦋
If TCP end-point B receives an ACK Congestion Control Permitted option from TCP end-point A and also sent an ACK Congestion Control Permitted option to TCP end-point A, and if ECN-Capability has been negotiated, then TCP end-point B can send its pure ACK packets as ECN-Capable. ✨
TCP ACK Congestion Control Permitted Option:
Kind: TBD1
+-----------+-----------+
| Kind=TBD1 | Length=2 |
+-----------+-----------+When ACK congestion control is used, the default initial ACK Ratio is two, with the receiver acknowledging at least every other data packet. 🍓
4.2. The TCP ACK Ratio Option
The sender uses an ACK Ratio TCP option to communicate the ACK Ratio value from the sender to the receiver. 💖
TCP ACK Ratio Option:
Kind: TBD2
+-----------+-----------+-----------+
| Kind=TBD2 | Length=3 | ACK Ratio |
+-----------+-----------+-----------+The ACK Ratio option is only sent on data packets. Because TCP uses reliable delivery for data packets, the TCP sender can tell if the TCP receiver has received an ACK Ratio option. 🌸
4.3. The Receiver: Implementing the ACK Ratio
With an ACK Ratio of R, the receiver SHOULD send one pure ACK for every R newly received data packets unless the delayed ACK timer expires first. A receiver could simply maintain a counter that increments by one for each new data packet received, and send an ACK packet when the counter reaches R. The receiver would reset the counter to zero whenever a pure or piggybacked ACK is sent. 🧁
If the receiver has buffer limitations, the receiver might have to acknowledge K packets, for some K less than the current ACK Ratio R. In this case, the sender could observe from the acknowledgements that the receiver is acknowledging less than R packets. 🫧
It is possible for there to be lost or marked ACK packets when there haven't yet been any lost or marked data packets. Thus, the sender could increase the ACK Ratio R even during the initial slow-start. 🚀
[RFC5681] recommends that the receiver SHOULD acknowledge out-of-order data packets immediately, sending an immediate duplicate ACK when it receives a data segment above a gap in the sequence space, and sending an immediate ACK when it receives a data segment that fills in all or part of a gap in the sequence space. 🌟
When ACK congestion control is being used and the ACK Ratio is at most two, the TCP receiver acknowledges each out-of-order data packet immediately. For an ACK Ratio greater than two, Section 4.6 specifies in detail the receiver's behavior for sending ACKs for out-of-order data packets. 🎀
4.4. The Sender: Determining Lost or Marked ACK Packets 🕵️♀️
Okay bestie, the TCP data sender uses what it knows about the ACK Ratio the receiver is using to figure out when an ACK packet has gone missing. 💅
Since the sender knows the ACK Ratio R the receiver is using, it knows that, as long as no acknowledgement packets were dropped or reordered, each new acknowledgement it gets will acknowledge at most R additional data packets. So if the sender gets an acknowledgement covering more than R data packets, and it doesn't later get an acknowledgement covering a strict subset (either with a smaller cumulative acknowledgement, or with the same cumulative acknowledgement but a strict subset of the data acknowledged in selective acknowledgement (SACK) blocks), then the sender can infer that an ACK packet has been dropped. Using SACK options in ACK packets would help the sender detect lost ACK packets. 🫶
Similarly, the sender knows that as long as no data packets from the sender were dropped or delayed, and no acknowledgements were delayed because a timer expired at the receiver, each new pure acknowledgement received will acknowledge at least R additional data packets. For ACK congestion control, the sender doesn't have to do anything when it gets an acknowledgement covering fewer than R additional packets. 💖
Out-of-order data packets:
If the ACK Ratio is at most two, the TCP receiver sends a duplicate acknowledgement (DupACK) for every out-of-order data packet. In this case, the TCP sender should be able to detect lost DupACK packets by counting the number of DupACKs that arrive between the beginning of the loss event and the arrival of the first full or partial ACK, and comparing this number with the number of DupACKs that should have arrived (based on the number of packets being ACKed by the full or partial ACK). Simulations and/or experiments will be needed to determine whether, in practice, it works for the TCP sender to assess lost ACK packets during loss events, for an ACK Ratio of at most two. 🧪
If the ACK Ratio is greater than two, the TCP receiver does not send a DupACK for every out-of-order data packet, as specified in Section 4.6. For simplicity, the TCP sender does not attempt to detect lost ACK packets during loss events involving forward-path data traffic. That is, as soon as the sender infers a packet loss for a forward-path data packet, it stops detection of ACK loss on the reverse path. The sender waits until a new cumulative acknowledgement is received that covers the retransmitted data, and then restarts detection of ACK loss for reverse-path traffic. 🔁
Detecting lost ACK packets after changes in the ACK Ratio:
In detecting lost ACK packets, the sender relies on its knowledge of the ACK Ratio used by the receiver. But when the sender makes a change in the ACK Ratio and then receives ACK packets, how does it know whether the receiver was using the new or the old ACK Ratio when it sent those ACK packets? As specified in the next section, the sender can make only one of two possible changes to the ACK Ratio within one round-trip time. The sender can decrease the ACK Ratio by one, from R to R-1, or the sender can double the ACK Ratio, increasing it from R to 2R. But, in detecting lost ACK packets after an increase in the ACK Ratio, the sender needs to know whether the receiver was using the old ACK Ratio R or the new ACK Ratio 2R. 🤔
The sender sends ACK Ratio options only on data packets, and these data packets are acknowledged by the receiver. One possibility would be for the sender to save the sequence number of the last data packet that contained an ACK Ratio option and to remember whether that ACK Ratio option was for an increase or a decrease in the ACK Ratio. Then, if the sender receives an ACK packet acknowledging the saved sequence number, the sender knows that the receiver has begun using the new ACK Ratio. 📌
It might be sufficient for the sender just to save the information of whether the last change in the ACK Ratio was an increase or a decrease, without saving the sequence number associated with the last ACK Ratio option. In this way, if the sender recently increased the ACK Ratio from R to 2R, the sender could be more cautious in detecting lost ACK packets. Another possibility would be that, after sending an ACK Ratio option, the sender waits until that data has been ACKed, with the new ACK Ratio in use by the receiver, before resuming the detection of lost ACK packets. However, we do not explore either of these approaches in more detail in this document. 🙈
4.4.1. The Sender: Detecting Lost ACK Packets after a Congestion Event 🚨
After a sender's retransmit timeout or fast retransmit, the sender might retransmit a number of data packets dropped from a single window of data. In particular, during a loss recovery period (from the sender's detection of the congestion event up until the sender receives an acknowledgement of all data packets transmitted before the loss recovery period began), retransmitted data packets can fill holes in the receiver's sequence space, resulting in irregular jumps in the cumulative acknowledgement field in ACK packets from the receiver. These jumps in the cumulative acknowledgement field make it difficult for the sender to reliably detect lost ACK packets during a loss recovery period. 😭
Because of this uneven progress of the cumulative acknowledgement field during a loss recovery period, the sender should not attempt to detect lost ACK packets during a loss recovery period. As a consequence, the sender will not increase the ACK Ratio in response to ACK packets that are lost during a loss recovery period. 🛑
4.5. The Sender: Adjusting the ACK Ratio 🎀
The TCP sender will adjust the ACK Ratio as specified in Section 6.1.2 of [RFC4341], as follows.
The ACK Ratio always meets the following three constraints.
(1) The ACK Ratio is an integer.
(2) The minimum ACK sending rate: The ACK Ratio does not exceed max(2, cwnd/(K*MSS)), rounded up, for K=2. As a result, the TCP receiver generally sends at least two ACKs in response to a window of at least four full-sized segments.
(3) If the congestion window is at least as large as four full-sized segments, then the ACK Ratio is at least two. In other words, an ACK Ratio of one is only allowed when the congestion window is at most three full-sized segments.
The sender changes the ACK Ratio within those constraints as follows. 🧩
For each congestion window of data with lost or marked ACK packets, the ACK Ratio R is doubled; for each cwnd/(MSS*(R^2 - R)) consecutive congestion windows of data with no lost or marked ACK packets, the ACK Ratio is decreased by 1. (See Appendix A of RFC 4341 for the derivation. Note that Appendix A of RFC 4341 assumes a congestion window W in packets, while we use cwnd in bytes.) As stated in the previous section, when the ACK Ratio is greater than two, the sender does not attempt to detect lost ACK packets during loss events for forward-path traffic.
For a constant congestion window, these modifications to the ACK Ratio give an ACK sending rate that is roughly TCP-friendly. Of course, cwnd usually varies over time; the dynamics will be rather complex, but roughly TCP friendly. We recommend that the sender determines when to decrease the ACK Ratio by one (i.e., by calculating the number of in-order data packets to count) right after an ACK loss event. 🦋
The frequency of ACK Ratio negotiations:
The sender need not keep the ACK Ratio completely up to date. For instance, it may rate-limit ACK Ratio renegotiations to once every four or five round-trip times, or to once every second or two. The sender should not attempt to change the ACK Ratio more than once per round-trip time. In particular, before sending a packet with a new value for the ACK Ratio, the sender should verify that the receiver has acknowledged a data packet containing an ACK Ratio option for the old value of the ACK Ratio. Additionally, the sender may enforce a minimum ACK Ratio of two, or it may set the ACK Ratio to one for half-connections with persistent congestion windows of 1 or 2 packets. ⏳
The minimum ACK sending rate:
From rule (2) above, the TCP receiver always sends at least K=2 ACKs for a window of data, even in the face of very heavy congestion on the reverse path. We would note, however, that if congestion is sufficiently heavy, all the ACK packets are dropped, and then the sender falls back on an exponentially backed-off timeout. Thus, if congestion is sufficiently heavy on the reverse path, then the sender reduces its sending rate on the forward path, which reduces the rate on the reverse path as well. One possibility would be to use a higher minimum ACK-sending rate, adding a constant upper bound on the ACK Ratio. That is, if the ACK Ratio also had an upper bound of J, independent of cwnd, then the receiver would always send at least one ACK for every J data packets, regardless of the level of congestion on the reverse path. 🌈
4.5.1. Possible Addition: Decreasing the ACK Ratio after a Congestion Window Decrease 🧚
After a lost or ECN-marked data packet, the data sender halves the congestion window, thus halving the sending rate for data packets, while making no change to the ACK Ratio R. As a result, after a congestion event involving a data packet, the sending rate for ACK packets on the return path is also halved. If the congestion event was a lost or ECN-marked data packet, this was due to congestion on the forward path, which may have been unrelated to conditions on the reverse path. Thus, it has been suggested that the sender could decrease the ACK Ratio R when it halves the congestion window; in this case, the halving of the sending rate for data packets would not be accompanied by a halving of the sending rate for ACK packets also. 🍓
However, there are a few cases where a congestion event involving data packets could in fact have been caused by congestion on the reverse path. As one example, the path could include a congested multiaccess link where forward-path and reverse-path traffic can interfere with each other. Thus, in this case it might be desirable if a congestion event resulted in a reduction in the sending rate of ACK packets as well as of data packets. 🔥
As a second example of a congestion event involving congestion of the reverse path, a congestion event could be caused not by a dropped or ECN-marked data packet, but by a window of dropped ACK packets, resulting in a retransmit timeout at the data sender. After a retransmit timeout, the TCP sender will slow-start, reducing the congestion window to the initial window and setting the ACK Ratio to at most two. 💀
Until further investigation, the sender will not decrease the ACK Ratio as a result of a congestion event involving a data packet. 👑
4.6. The Receiver: Sending ACKs for Out-of-Order Data Segments ✨
okay so RFC 5681 says that "a TCP receiver SHOULD send an immediate duplicate ACK when an out-of-order segment arrives" 🙈. After three duplicate ACKs come in, the sender goes "oh no, something's missing" and does fast retransmit and fast recovery, resending the lost packet 🦋. When the ACK Ratio is at most two, the receiver should still send an immediate duplicate ACK when an out-of-order segment shows up. no notes 💅
in general, when the ACK Ratio is greater than two, the receiver still should send an immediate duplicate ACK for each of the first three out-of-order segments that arrive in a reordering event. (quick reminder: a reordering event at the receiver starts when an out-of-order segment arrives and ends when the receiver is holding no more out-of-order segments 🌸) but here's the twist 🌀: once those first three dupes are sent, the receiver should do ACK congestion control on the rest of the ACKs for that reordering event. so after the first three duplicate ACKs, it goes back to sending one ACK for every R segments, instead of one ACK for every out-of-order segment in that event. (the sender's fast recovery procedure might need a tweak to handle this change too 🧚) and also, a receiver must not withhold an ACK for more than 500 ms ⏰. non-negotiable bestie.
ngl this next part is kinda chaotic 😭. in an environment with systematic reordering in the data path (like, every set of K data packets arrives in inverted order, for some value of K), the guideline above could end up with the receiver sending an ACK for every single data packet, no matter what the ACK Ratio is. in that kind of environment with persistent reordering, the receiver may decide not to send an immediate duplicate ACK for each of the first three out-of-order segments in a reordering event. figuring out how to do effective ACK congestion control in systematic reordering is left for future work 🔮.
4.7. The Sender: Response to ACK Packets 💌
big ACK Ratio = big energy 💥, and it can generate line-rate data bursts at the TCP sender, which is a lot. when the ACK Ratio is greater than two, the sender should use some form of burst mitigation or rate-based pacing for sending data packets in response to a single acknowledgement. just know the timer granularity at the sender limits how well rate-based pacing can work 🕰️.
we also note that how ACK congestion control and burst mitigation schemes interact still needs more study 🧐.
Byte counting at the sender:
besides the burstiness thing, a big ACK Ratio can also slow down the data-sending rate by making the congestion window cwnd grow more slowly 🐌. RFC 5681 says that in slow-start, the sender increases cwnd by one full-sized segment for each new ACK received (in this context, a "new ACK" is an ACK that acknowledges new data). RFC 5681 also says that in congestion avoidance, the sender increases cwnd by roughly 1/cwnd full-sized segments for each ACK received, which adds up to roughly one full-sized segment per round-trip time. with a big ACK Ratio, the window would grow more slowly, so the vibe is "pls don't slow down my window growth" 🙏.
RFC 5681 notes that during congestion avoidance, it is also fine to count the bytes acknowledged by new ACKs and increase cwnd based on those bytes, instead of on the number of new ACKs received. so the sender should use this form of byte counting with ACK congestion control, so that ACK congestion control doesn't slow down the window increases for the data traffic the sender sends 💖. and because rate-based pacing should be used with ACK congestion control (as recommended earlier in this section), the sender may increase the congestion window by more than two MSS for each ACK 🚀.
we note that for Appropriate Byte Counting (ABC) as specified in [RFC3465], during slow-start the sender is allowed to increase the congestion window by at most two MSS for each ACK. it hasn't been figured out yet whether the sender could use ABC during slow-start with ACK congestion control 🤷♀️. if ABC is used with ACK congestion control, then when the sender is in slow-start and the ACK Ratio is greater than two, the sender may increase the congestion window by more than two MSS in response to a single ACK. Section 4.2 of [LL07] explores some of the issues with using ABC for TCP connections with a fixed ACK Ratio greater than two 📚.
Inferring lost data packets:
as mentioned earlier, RFC 5681 infers that a packet has been lost after it receives three duplicate acknowledgements 🕵️♀️. ACK congestion control is only used when there is congestion on the reverse path, so after a packet loss, one or more of the three duplicate ACKs the receiver sent could get lost on the reverse path. then the receiver might wait until it has received R more out-of-order segments before sending the next duplicate ACK 😩. all of this could slow down fast recovery and fast retransmit quite a bit. SACK can help reduce the potential delay in detecting a lost packet 🫶. with SACK, the sender can use the information in the SACK option to detect when the receiver has received at least three out-of-order data packets, and start fast retransmit and fast recovery in that case, even if the sender hasn't yet received three duplicate ACKs 🎀.
4.8. Possible Addition: Receiver Bounds on the ACK Ratio 🌈
it's been suggested that in some environments, the receiver might want to set lower bounds on the ACK Ratio 🧠. for example, the receiver might know from configuration or past experience that the bandwidth on the return path is limited, so it might want a lower bound (greater than two) on the ACK Ratio R. if this gets included, it would need a TCP option from the receiver to the sender that reports the lower bound on the ACK Ratio 📬. care would also be needed so that the lower bound on the ACK Ratio only applies when the sender's congestion window is high enough 🪟.
5. Possible Complications 💫
5.1. Possible Complication: Delayed Acknowledgements 🌷
the receiver could send a delayed acknowledgement that acknowledges a single packet, even when the ACK Ratio is two or more 🫠.
this should not cause false positives (where the sender infers a loss when no loss actually happened) 🙅♀️. the sender only infers that a pure ACK packet has been lost when no data packet has been lost and an ACK packet arrives acknowledging more than R new packets.
delayed acknowledgements could, however, cause false negatives, where the sender can't detect the loss of an ACK packet that was sent as a delayed acknowledgement 👀. false negatives seem acceptable, though! it would just mean approximate ACK congestion control, which is still better than no ACK congestion control at all ✅. specifically, when this kind of false negative happens, it's because the receiver is sending acknowledgements at such a low rate that it's sending delayed acknowledgements, instead of acknowledging at least R data packets with each acknowledgement 🐢.
5.2. Possible Complication: Duplicate Acknowledgements 🎭
as discussed in Section 4.3, RFC 5681 states that "a TCP receiver SHOULD send an immediate duplicate ACK when an out-of-order segment arrives", and that "a TCP receiver SHOULD send an immediate ACK when the incoming segment fills in all or part of a gap in the sequence space" [RFC5681]. when ACK congestion control is used, the receiver uses the guidelines from Section 4.6 to decide when to send duplicate ACKs instead 🔄. more work would be useful to evaluate the pros and cons of this approach in terms of the potential delay in triggering fast retransmit, and to explore other possibilities 🔍.
5.3. Possible Complication: Two-Way Traffic 🔁
in a TCP connection with two-way traffic, the receiver could send some pure ACK packets and some acknowledgements piggybacked on data packets 💬. the receiver would still follow the rule of only sending a pure ACK packet when there's a need for a delayed ACK or when there are R new data packets to acknowledge 📏.
in a connection with two-way traffic, the sender wouldn't always be able to infer when a pure ACK packet had been lost 😬. for example, the receiver could send a pure ACK packet acknowledging packet K and, soon after, send a newly generated data packet for the reverse-path flow that also acknowledges packet K. the pure ACK packet could get dropped in the network, and the sender wouldn't be able to detect that drop 👻.
thankfully, there are limits to how bad the problems from undetected ACK losses in two-way traffic can get 🫶. the sender will only fail to detect the loss of a pure ACK packet if that ACK packet was followed by a data packet with the same acknowledgement number. if the reverse-path traffic for the connection is mostly data, then congestion control for the data traffic matters more than congestion control for the pure ACK traffic 📦. if the reverse-path traffic is mostly pure ACK traffic, then the sender would detect any losses of pure ACK packets followed by other pure ACK packets, and that covers most of the pure ACK packets for that connection 🙌. so the sender's failure to detect the loss of a pure ACK packet followed by a data packet with the same acknowledgement number wouldn't disable ACK congestion control for a TCP connection with two-way traffic 💅.
5.4. Possible Complication: Reordering of ACK Packets 🌀
ACK packets can get reordered on the reverse path 🔀. the sender could either use a mechanism parallel to the DupACK threshold to infer when an ACK packet has been lost, as with TCP, or, more robustly, the sender could wait an entire round-trip time before inferring that an ACK packet has been lost [RFC4653] 🕒.
5.5. Possible Complication: Abrupt Changes in the ACK Path ⚡
what happens when there are abrupt changes in the reverse path, like from vertical handovers? 🤔 can there be any problems that would be worse than what a TCP connection not using ACK congestion control would experience?
5.6. Possible Complication: Corruption 🫧
okay so ACK packets can get dropped because of corruption, not just congestion, same as data packets 🥺 The current assumption in ACK congestion control is that every loss is a sign of congestion. If someone ever comes up with a better way to spot and handle corrupted TCP data packets, hopefully that same fix would work for corrupted ACKs too 🎀
The tricky part is that, for both data and ACK packets, it's not always obvious whether the corruption is linked to congestion or whether it's just happening no matter how busy the corrupting link is 🙈 In environments where corruption exists and has nothing to do with the congestion level on that link, running ACK congestion control would just make the connection more sensitive to ACK corruption, because it means fewer ACKs get sent. Not ideal, bestie 💀
5.7. Possible Complication: ACKs that Don't Contribute to Congestion
It's possible for the ACK packets in a TCP connection to cross a congested path where ACKs get dropped, even though the ACKs themselves aren't really adding to the congestion there 🦋 If ACK traffic doesn't meaningfully contribute to congestion, then using ACK congestion control wouldn't help reduce the overall congestion on the path. So in this case, the goal is to keep ACK congestion control from hurting the overall performance of the TCP connection as much as possible ✨
J TCP conns. link L -> J TCP conns.
data -> |---| |---| <- ACKs
<-------------> | | | | <------------->
| | <-------------> | |
<-------------> | | | | <------------->
K TCP conns. |---| |---| K TCP conns.
ACKs -> <- link L1 <- dataFigure 2. A Scenario with J Forward and K Reverse TCP Connections
To figure out how much ACK traffic actually affects congestion, it helps to look at a simple scenario 🧚 We have a congested unidirectional link L carrying data traffic from J TCP connections (the forward TCP connections) and ACK traffic from K TCP connections (the reverse TCP connections). We assume that all TCP connections have the same round-trip time R and the same data packet size S of 1500 bytes. We also assume that all the forward TCP connections have the same data packet drop rate p and the same congestion window W, and that all the reverse TCP connections have the same congestion window W1 and the same ACK packet drop rate p1. (The packet drop rate for data packets is the fraction of arriving data packets that get dropped. Similarly, the packet drop rate for ACK packets is the fraction of arriving ACK packets that get dropped.) The J TCP connections each use a bandwidth on link L of 1500*W/R bytes per second, and the K TCP connections, without ACK congestion control, each use a bandwidth on link L of 40*(W1/2)/R bytes per second. That gives a ratio of 75*(J/K)*(W/W1) for TCP data bandwidth to TCP ACK bandwidth on link L 📊 The ratio J/K is the ratio between the number of forward and reverse TCP connections on link L, and it could have a wide range of values (for example, large for an access link from a web server, and small for an access link to a web server). In this scenario, the ratio W/W1 is mostly a function of the different congestion levels on the forward and reverse paths.
Now let's explore the possibilities 👑 We'll look at some of the range of congestion control mechanisms for the congested link. First, we consider scenarios where the limit on the congested path is the link bandwidth in bytes per second.
Cases (1), (2), (3), (5), and (7) below are the best scenarios for ACK congestion control, where the fraction of packet drops for TCP ACK packets roughly matches how much the TCP ACK packets contribute to congestion. (In several of these cases that's, at best, a rough match, because data packets are a factor in both the bandwidth and the queue limits, while TCP ACK packets are only a factor in the queue limits.) Cases (4) and (8) are problematic scenarios where the fraction of packet drops for TCP ACK packets is much higher than the TCP ACK packets' contribution to congestion. That contribution might be taking space in a congested queue, using scarce CPU cycles at the congested router, or using scarce bandwidth 😬 Case (6) is a scenario where ACK congestion control would not be effective because it would not be invoked. In the scenarios in case (6), the fraction of packet drops for TCP ACK packets would be much smaller than the TCP ACK packets' contribution to congestion.
(1) The Drop-Tail queue for link L is measured in packets. In this case, the congested queue can accommodate N packets (regardless of packet size). There's a limit on both bandwidth in bytes per second and queue space in packets, and large data packets and small TCP ACK packets should see similar packet drop rates. Although TCP ACK packets most likely aren't a major factor in the bandwidth limit, they can be a significant contribution to the queue space limit. So while the packet drop rate for ACK packets could be high in times of congestion, the ACK packets are contributing to that congestion somewhat by using scarce buffer space 🍓
(2) The Drop-Tail queue is measured in bytes. In this case, the congested queue can accommodate M bytes of packets, and TCP ACK packets don't make a significant contribution to either the bandwidth limit or the queue space limit. It's also true that, in this scenario, even with heavy congestion, the packet drop rate for TCP ACK packets should be small, because small ACK packets can often find space in the congested queue when large data packets can't. So ACK congestion control shouldn't present any problems here. The TCP ACK packets aren't contributing significantly to congestion and aren't seeing significant packet drop rates 🌸
(3) The RED queue is in packet mode and is measured in packets. This is similar to case (1) above. Because the queue is measured in packets, small TCP ACK packets contribute to the limit on queue space but not to the limit on link bandwidth. Because the queue is in packet mode, large data packets and small TCP ACK packets should see similar packet drop rates 💅
(4) The RED queue is in packet mode but is measured in bytes. Because the queue is measured in bytes, small TCP ACK packets don't contribute significantly to either the queue space limit or the link bandwidth limit. Because the queue is in packet mode, large data packets and small TCP ACK packets should see similar packet drop rates. If this case existed, it would be problematic 💀 The TCP ACK packets wouldn't be contributing significantly to congestion, but they'd see a similar packet drop rate as the large data packets that are contributing to congestion.
(5) The RED queue is in byte mode and is measured in bytes. This is similar to case (2) above. Because the queue is measured in bytes, small TCP ACK packets don't contribute significantly to either the queue space limit or the link bandwidth limit. At the same time, because the queue is in byte mode, small TCP ACK packets see much smaller packet drop rates than large data packets do 🦋
(6) The RED queue is in byte mode but is measured in packets. Because the queue is measured in packets, small TCP ACK packets contribute to the queue space limit but not to the link bandwidth limit. Because the queue is in byte mode, small TCP ACK packets see much smaller packet drop rates than large data packets do. If this case existed, TCP ACK packets would contribute somewhat to congestion but would see a much smaller packet drop rate than large data packets 🙃
Next, we consider scenarios where the limit on the congested link is CPU cycles at the router, measured in packets per second, rather than bandwidth in bytes per second 🧠
(7) The CPU load imposed by TCP ACK packets is similar to the load imposed by other packets (for example, TCP data packets). ACK congestion control would be useful in this scenario, particularly if TCP ACK packets saw the same packet drop rates as TCP data packets ✨
(8) The CPU load imposed by TCP ACK packets is much less than the load imposed by other packets (for example, TCP data packets). If TCP ACK packets saw a smaller packet drop rate than TCP data packets, then the TCP ACK packet drop rate would roughly match the TCP ACK packets' contribution to congestion, and that would be good 🎀 If TCP ACK packets saw the same packet drop rate as TCP data packets, this case would be problematic. The TCP ACK packets wouldn't be contributing significantly to congestion, but they'd see a similar packet drop rate as the large data packets that are contributing to congestion 😭
5.8. Possible Complication: TCP Implementations that Skip ACK Packets
It's been reported at IETF meetings that current TCP implementations don't always acknowledge at least every other data packet, which the TCP specifications say they should 🙈 In particular, it's been reported that if a TCP receiver gets lots of data packets in a burst before it can send an acknowledgement, it might send a single acknowledgement for the whole burst. This would cause complications for a TCP connection using ACK congestion control, because the sender wouldn't be able to tell whether an ACK packet had been dropped in the network or had been skipped by the receiver while it was processing a burst of data packet arrivals 🌈
One way to address this would be for TCP receivers using ACK congestion control to be required to send an acknowledgement for each R packets, for ACK Ratio R. In that case, if the receiver got a large burst of data packets back-to-back, it would be required to send a matching burst of ACK packets, one for each set of R data packets 🫶
A second way to address this would be to define a TCP option or flag that the TCP receiver could use when sending an ACK packet. It would tell the sender that the TCP receiver `skipped' some ACK packets, so the sender shouldn't infer ACK loss if some previous ACK packets seem to be missing 🔥
Future work will explore the costs and benefits of these two approaches 🧚
5.9. Possible Complication: Router or Middlebox-Based ACK Mechanisms 🦋
okay so here's a plot twist 👀 what if routers or middleboxes are also messing with ACKs on the way back? think ACK filtering along the reverse path ([BPK97], [WWCM99], [BA03], [KLS07]) 🙈 we aren't aware of ACK filtering actually being deployed on the Internet, but if anyone tests ACK congestion control, they'd have to check for interactions with any middlebox mechanisms that touch ACK packets. 💅 the ones we'd be most worried about are satellite links, cable modems, or anything similar where ACK filtering might be switched on 🛰️
5.10. Possible Complication: Data-Limited Senders 🎀
so the whole ACK Ratio adjusting thing is designed so the TCP receiver sends at least two ACKs for every window of at least four full-sized data packets 💖 BUT 😭 with ACK congestion control and a data-limited sender, it's possible for the sender to push at least four full-sized data packets in one round-trip time while the receiver only sends fewer than two ACKs back. the vibe is not it 💀
here's an example. say the sender's congestion window W is bigger than four and the ACK Ratio R is sitting at its max value of W/2. now the sender gets data-limited and sends fewer than W data packets in a round-trip time. then the receiver can send fewer than two ACK packets in response. ngl this makes the connection way more sensitive to losing even one ACK packet 🥺
don't worry tho, there's still a safety net 🧚 the receiver still sends an ACK packet whenever the delayed ACK timer expires. however, we think more work is needed to figure out the conflicting goals here: keeping the ACK flow congestion-controlled and getting timely ACKs back to the sender, specifically for a connection with a data-limited sender and a congested ACK path. it's a whole puzzle 🧩
5.11. Other Issues 🌸
questions questions 🤔 are there any problems caused by two-way traffic and reordering together? or other issues we haven't dealt with yet? bestie we still need to figure these out 🫶
6. Evaluating ACK Congestion Control 📊
okay evaluating ACK congestion control has two parts 👑 (1) looking at what it does to one individual TCP connection, and (2) looking at what it does to aggregate traffic, including how it changes the aggregate congestion on the path.
the first part is about the single connection. we need to check the scenarios where ACK congestion control might help a TCP connection's performance, and also the scenarios where it might cause problems. 🔍
the second part is about all the traffic together. we should look at cases where ACK congestion control helps every connection sharing a path by lowering the aggregate congestion on that path. we should also check whether there are cases where it causes trouble, like making the aggregate traffic burstier or otherwise changing traffic dynamics. the whole network can feel it, so it matters 🌈
6.1. Contention in Wireless Links or in Non-Switched Ethernet 📡
one possible benefit is that ACK congestion control could reduce contention on wireless links, shared Ethernet, or other setups where forward-path and reverse-path traffic fight over the same medium ([AJ03], [KIA07]) ✨ BUT here's the catch 🙃 contention on the shared medium doesn't always lead to dropped ACK packets. so it wouldn't necessarily get detected by ACK congestion control. oof 😬
6.2. Keep-Alive and Other Special ACK Packets 🍓
some TCP hosts send keep-alive packets when they haven't gotten any data or ACK packets for a long time [KEEP-ALIVE]. the TCP specs don't address this keep-alive mechanism. still, keep-alive packets, if they're used, should not interact with ACK congestion control one way or another 🧚
for ACK congestion control, the ACK Ratio is set small enough that the receiver can generally send at least two ACKs for a window of data 💅 plus, the receiver uses a delayed ACK timer together with the ACK Ratio, and it always sends an acknowledgement if that timer expires. so ACK congestion control will never make the receiver wait forever before acknowledging a received data packet. no indefinite waiting, period 🫶
some TCP implementations send pure ACK packets as window probes, to ask the other end for an ACK packet with current window information 🪟 those ACK packets will generally be orthogonal to the ACK congestion control specified in this document, so they're kind of their own thing 👀
TCP receivers can also send pure ACK packets as window update packets, to announce a new value for the receive window 📣 they can do this even when the acknowledgement number and SACK options in the ACK packet aren't new. the receiver may send window update packets even when the ACK congestion control mechanism says it isn't time yet to send a pure ACK 😭 and the sender won't necessarily be able to detect if a window update ACK packet gets lost. yikes 🙈
7. Measurements of ACK Traffic and Congestion 📈
there are a bunch of studies about traffic composition on different links across the Internet, reporting what fraction of bandwidth is used by TCP data and what fraction by TCP ACK traffic [Studies] 🔎
still, we want more. are there any studies showing the relative packet drop rates for TCP data and ACK traffic, for particular links or particular TCP connections? 🤓
and are there any studies of congested links showing what fraction of the traffic on the congested link, or in the congested queue, is made up of TCP ACK packets? 🚦
8. Acknowledgement Congestion Control in DCCP's CCID 2 💫
in the transport protocol DCCP [RFC4340], the congestion control mechanism for the CCID 2 profile is based on TCP's 🪄 this section briefly walks through some of the issues already addressed in the acknowledgement congestion control that's been standardized in CCID 2 [RFC4341] 🧁
Rate-based pacing 🌊
for CCID 2, RFC 4341 says that "senders MAY use a form of rate-based pacing when sending multiple data packets liberated by a single ACK packet, rather than sending all liberated data packets in a single burst." 🎀 however, rate-based pacing is not required in CCID 2.
Increasing the congestion window 📈
for CCID 2, RFC 4341 says that "when cwnd < ssthresh, meaning that the sender is in slow-start, the congestion window is increased by one packet for every two newly acknowledged data packets with ACK Vector State 0 (not ECN-marked), up to a maximum of ACK Ratio/2 packets per acknowledgement. This is a modified form of Appropriate Byte Counting [RFC3465] that is consistent with TCP's current standard (which does not include byte counting), but allows CCID 2 to increase as aggressively as TCP when CCID 2's ACK Ratio is greater than the default value of two. When cwnd >= ssthresh, the congestion window is increased by one packet for every window of data acknowledged without lost or marked packets."
9. Security Considerations 🔐
okay this section is serious business 😤 what are the sender's incentives to cheat on ACK congestion control? what about the receiver's incentives? and what avenues are open for cheating at all? 🚨
as long as ACK congestion control is optional, neither host can be forced to use it if it doesn't want to. so ACK congestion control will only get used if the sender or receiver has some chance of getting a benefit from it 💅
as long as ACK congestion control stays optional for TCP, there's not much incentive for TCP end nodes to cheat on non-ECN-based ACK congestion control. nothing right now requires TCP hosts to use congestion control in response to dropped ACK packets 🙃
now for the spicy part 🌶️ what avenues for cheating does the use of ECN-Capable ACK packets open up? if end nodes can use ECN to get ACK packets marked instead of dropped, and they can then skip the ACK congestion control that normally goes along with ECN on ACK packets, then the end nodes could have an incentive to cheat. senders could cheat by not telling the receiver to use a higher ACK Ratio, and the receiver would have a really hard time detecting that. receivers could cheat by not using the ACK Ratio they were told to use, but senders could easily detect that one 👀 receivers could also cheat by skipping ACK congestion control entirely while still sending ACK packets as ECN-Capable, so ACK congestion control isn't a required piece for receivers to cheat about sending ECN-Capable ACK packets. one open question: is there any way for receivers to cheat about sending ECN-Capable ACK packets, skip appropriate ACK congestion control, and avoid being easily detected by the sender? 🕵️♀️
what about routers or middleboxes detecting TCP receivers that inappropriately send ACK packets as ECN-Capable? 🤔 the router will only know whether the receiver is authorized to send ACK packets as ECN-Capable if it can see traffic on both the forward and reverse paths, and it monitored both the SYN and SYN/ACK packets (and was able to read the TCP options in the packet headers). if ACK congestion control has been negotiated, the router will only know whether the receiver is using it correctly if it can monitor the ACK Ratio options sent from the sender to the receiver. and if ACK congestion control is being used, the router won't necessarily be able to tell whether the sender is using it correctly, because ACK packets might get dropped after they've already left the router 😵 still, if the router sees the ACK Ratio options sent from the sender, it will be able to tell whether the sender is correctly accounting for the ACK packets that get dropped or ECN-marked on the path from the receiver to the router. ✨
10. IANA Considerations 🧾
no IANA action is needed at this time 💖 if this document gets advanced as Experimental or Proposed Standard, then IANA would allocate the option numbers for the two TCP options: the ACK Congestion Control Permitted option and the ACK Ratio option. in that case, the following two lines would be added to the TCP Option Numbers registry (maintained by IANA -- http://www.iana.org):
Kind Length Meaning Reference
---- ------ --------------------------------- -----------
TBD1 2 ACK Congestion Control Permitted [RFCXXXX]
TBD2 3 ACK Ratio [RFCXXXX]until IANA allocates TCP option numbers, experimenters can use the TCP Option Numbers set aside for Experimentation in RFC 4727 [RFC4727] 🧪 but as Section 1 of RFC 3692 [RFC3692] stresses, the TCP Option Numbers in the experimental range are meant for experimentation and testing, not for wide or general deployments. these option numbers could already be in use by other experimenters for other purposes, so be careful bestie ⚠️
11. Conclusions 🎬
this document specifies a congestion control mechanism for acknowledgement (ACKs) traffic for TCP and discusses the possible complications 🌷 we're deferring a recommendation on using this mechanism for TCP until it's been evaluated more fully. 🫧
12. Acknowledgements 🫶
many thanks for feedback from Mark Allman, Armando Caro, Alfred Hoenes, Ilpoo Jarvinen, Sara Landstrom, Anantha Ramaiah, and Michael Welzl, and for contributed text from Michael Welzl 💜🌸
Appendix A. Related Work ✨
Okay bestie, time for the related work tea 🫖💖 There are a bunch of papers about controlling congestion on the reverse path of a TCP connection, especially when bandwidth is asymmetric 🦋. Some of them need explicit help from routers or middleboxes, and others are "pure" end-to-end schemes, which is so iconic.
RFC 3449 [RFC3449] is the one that digs into TCP performance problems on asymmetric paths 🌈. Section 3 of RFC 3449 breaks down exactly how congestion on the ACK path can wreck overall TCP performance 😭. It also lays out a bunch of mitigations, including ACK congestion control. The experimental ACK congestion control mechanism in that RFC relies on ECN, with the TCP sender spotting congestion on the ACK path from ECN-marked packets 💅. RFC 3449 also covers two receiver-based mechanisms, the Window Prediction Mechanism (WPM) [CLP98] and Acknowledgement based on Cwnd Estimation (ACE) [MJW00], both of which use a dynamic ACK Ratio 🧚. And it looks at link- and network-layer techniques for congestion on the upstream path too: header compression, plus bandwidth management for the upstream link, like ACK filtering and ACK reconstruction 🎀.
RFC 3135 [RFC3135], "Performance Enhancing Proxies Intended to Mitigate Link-Related Degradations", surveys a whole range of Performance Enhancing Proxies used to improve TCP behavior, including proxies for ACK filtering and reconstruction 🍓. RFC 2760 [RFC2760], "Ongoing TCP Research Related to Satellites", discusses both ACK congestion control and ACK filtering and reconstruction, with detailed descriptions of the mechanisms Balakrishnan, et al. proposed in [BPK97] 🌙.
Landstrom, et al. in [LL07] explore a mechanism where the receiver sends only four acknowledgements per window of data, with the sender using a form of Appropriate Byte Counting 🔥. The receiver also drops to a lower acknowledgement frequency after a timeout, so it avoids unnecessary retransmit timeouts 💀. One conclusion of the paper is that pacing at the sender adds extra delay and might not be necessary 🙈. The big result is that, with some form of byte counting at the sender, TCP can use a lower acknowledgement frequency than delayed acknowledgements 👑.
A.1. ECN-Only Mechanisms
Balakrishnan, et al. in [BPK97] describe using ECN to detect congestion on the return path, so the sender can reduce the sending rate of ACKs ✨. A RED queue in the reverse path allows marking of ACK packets, and the sender echoes ECN congestion marks back to the receiver. The receiver keeps an ACK Ratio d (called the "delayed-ACK factor"), which is the number of data segments that have to be received before the receiver sends a new ACK. The ACK Ratio d is managed using multiplicative-increase, additive-decrease: when a congestion mark arrives, the receiver doubles d (hence dividing the ACK sending rate by two) 💖. The ACK Ratio decreases linearly for each RTT in which no ECN-marked ACKs are received. Multiple congestion marks received in an RTT are treated as a single congestion event, so d can be doubled at most once per RTT 🫶. The TCP timestamp option is used to keep track of RTT values ⏱️.
A.2. Receiver-Only Mechanisms
In [MJW00], Tam Ming-Chit, et al. propose a receiver-based method for calculating an "appropriate" number of ACKs per congestion window (cwnd) of data, in order to ease congestion on the reverse path 🌸. The sender's cwnd is estimated at the receiver by counting the number of received packets per RTT, and the receiver has to estimate the RTT itself too 🧮. From this estimate, a simple algorithm computes the number of ACKs to send per cwnd. The algorithm enforces a lower bound on ACKs per cwnd, aiming to minimize the chance of a sender timeout caused by ACK loss 😱. The ACK Ratio is also upper-bounded so that ACK delay doesn't get excessive ⏳.
Blandford, et al. [BGG+07] propose an end-to-end, receiver-oriented scheme called "smartacking" 🧠. The algorithm has the receiver monitor the inter-segment arrival time for data packets and adapt the ACK sending rate in response 🎯. When the bottleneck link is underutilized, ACKs are sent frequently (up to one ACK per received segment) to promote fast growth of the congestion window 🚀. When the bottleneck is close to full utilization, the algorithm tries to reduce control traffic overhead and slow congestion window growth by generating ACKs at the minimum rate needed to keep the data pipe full 🍃.
Reducing the number of ACKs (or, equivalently, increasing the amount of bytes acknowledged by each ACK) can increase the burstiness of the TCP sender 💥. Hence, any mechanism like those cited above should be coupled with a burst mitigation technique, such as rate-based pacing, that paces the sending of data segments ([AB05], [ASA00], [BPK97]) 🛡️.
A.3. Middlebox-Based Mechanisms
ACK filtering (AF) [BPK97] from Balakrishnan, et al. is a router-based technique that tries to reduce the number of ACKs sent over the congested return link 🕵️. With AF, an arriving ACK may replace preceding, older ACKs at the bottleneck queue. An aggressive replacement policy might guarantee that at most one ACK per connection is waiting in the queue, which alleviates congestion. However, as in other proposals, care must be taken to avoid sender timeouts in case the (too few) ACKs resulting from the filtering get lost 😬. The idea of filtering ACKs was extended in [YMH03] to deal with SACK information 🧩.
Aweya, et al. [AOM02] present a middlebox-based approach for mitigating data packet bursts and for controlling the uplink ACK congestion 🎀. The main idea is to perform pacing on ACK segments on an edge device close to the sender, so as to control the ACK arrival rate at the sender 🌷.
Appendix B. Design Considerations ✨
B.1. The TCP ACK Ratio Option or an AckNow Bit in Data Packets?
In the ACK congestion control mechanism specified in this document, the sender uses the TCP ACK Ratio option to tell the receiver the ACK Ratio to use 💌. An alternate approach could be for the sender to use an AckNow bit in the TCP header of data packets, telling the receiver to acknowledge this data packet. In the discussion below, we call these two approaches the TCP ACK Ratio option approach and the AckNow approach 🧁.
An advantage of an AckNow approach is that it would require less state from the receiver. The receiver would not need to maintain a variable for the current ACK Ratio and would not need to keep track of the number of data packets un-ACKed to date 🪶.
However, a disadvantage of the AckNow approach is that the sender does not know when packets will be reordered, delayed, or dropped on the path to the receiver. In particular, the sender does not have control over whether a data packet with the AckNow bit set is reordered, delayed, or dropped in the network 😵. For this reason, we have chosen the approach of the sender determining the ACK Ratio that should be used and sending the ACK Ratio to the receiver, rather than the sender telling the receiver exactly which data packets to acknowledge 🫶.
An additional disadvantage of the AckNow approach is that it would add complications and difficulties for the default cases of the receiver using an ACK Ratio of one or two, as is done in the absence of ACK congestion control 🙃.
For these reasons, we have specified that the sender determines the ACK Ratio to use and tells the receiver, rather than the sender setting an AckNow bit in the TCP Header of selected data packets 👑.
Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC3465] Allman, M., "TCP Congestion Control with Appropriate Byte Counting (ABC)", RFC 3465, February 2003.
[RFC3692] Narten, T., "Assigning Experimental and Testing Numbers Considered Useful", BCP 82, RFC 3692, January 2004.
[RFC4340] Kohler, E., Handley, M., and S. Floyd, "Datagram Congestion Control Protocol (DCCP)", RFC 4340, March 2006.
[RFC4341] Floyd, S. and E. Kohler, "Profile for Datagram Congestion Control Protocol (DCCP) Congestion Control ID 2: TCP-like Congestion Control", RFC 4341, March 2006.
[RFC4727] Fenner, B., "Experimental Values In IPv4, IPv6, ICMPv4, ICMPv6, UDP, and TCP Headers", RFC 4727, November 2006.
[RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion Control", RFC 5681, September 2009.
Informative References ✨
okay so bestie, the references are where the receipts live 🧾 these are the papers and RFCs the spec leans on for background, and every single one is kept exactly as written:
[RFC2760] Allman, M., Ed., Dawkins, S., Glover, D., Griner, J.,
Tran, D., Henderson, T., Heidemann, J., Touch, J.,
Kruse, H., Ostermann, S., Scott, K., and J. Semke,
"Ongoing TCP Research Related to Satellites", RFC 2760,
February 2000.
[RFC3135] Border, J., Kojo, M., Griner, J., Montenegro, G., and Z.
Shelby, "Performance Enhancing Proxies Intended to
Mitigate Link-Related Degradations", RFC 3135, June
2001.
[RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition
of Explicit Congestion Notification (ECN) to IP", RFC
3168, September 2001.
[RFC3449] Balakrishnan, H., Padmanabhan, V., Fairhurst, G., and M.
Sooriyabandara, "TCP Performance Implications of Network
Path Asymmetry", BCP 69, RFC 3449, December 2002.
[RFC4653] Bhandarkar, S., Reddy, A., Allman, M., and E. Blanton,
"Improving the Robustness of TCP to Non-Congestion
Events", RFC 4653, August 2006.
[ASA00] Aggarwal, A., Savage, S., and T. Anderson,
"Understanding the Performance of TCP Pacing", INFOCOM
(3), pp. 1157-1165, 2000.
[AB05] Allman, M., and E. Blanton, "Notes on Burst Mitigation
for Transport Protocols", SIGCOMM, Computer
Communications Review, 35(2):5360, 2005.
[AJ03] Altman, E., and T. Jimenez, "Novel Delayed ACK
Techniques for Improving TCP Performance in Multihop
Wireless Networks", Proc. of the Personal Wireless
Communications, 2003.
[AOM02] Aweya, J., Ouellette, M., and D. Y. Montuno, "A Self-
regulating TCP Acknowledgement (ack) Pacing Scheme",
International Journal of Network Management,
12(3):145163, 2002.
[BA03] Barakat, C., and E. Altman, "On ACK Filtering on a Slow
Reverse Channel", International Journal of Satellite
Communications and Networking, V.21 N.3, 2003.
[BPK97] Balakrishnan, H., Padmanabhan, V., and Katz, R., "The
Effects of Asymmetry on TCP Performance", Third ACM/IEEE
Mobicom Conference, September 1997.
[BGG+07] Blandford, D.K., Goldman, S.A., Gorinsky, S., Zhou, Y.,
and D.R. Dooly, "Smartacking: Improving TCP Performance
from the Receiving End", Journal of Internet
Engineering, 1(1), 2007.
[CLP98] Calveras, A., Linares, J., and J. Paradells, "Window
Prediction Mechanism for Improving TCP in Wireless
Asymmetric Links". Proc. IEEE Global Communications
Conference (GLOBECOM), Sydney Australia, pp. 533-538,
November 1998.
[KIA07] Keceli, F., Inan, I., and E. Ayanoglu, "TCP ACK
Congestion Control and Filtering for Fairness Provision
in the Uplink of IEEE 802.11 Infrastructure Basic
Service Set", Proc. IEEE ICC 2007, June 2007.
[KEEP-ALIVE] Busatto, F., "TCP Keepalive HOWTO", May 2007,
http://tldp.org/HOWTO/TCP-Keepalive-HOWTO/index.html.
[KLS07] Kim, H., Lee, H., and S. Shin, "On the Cross-Layer
Impact of TCP ACK Thinning on IEEE 802.11 Wireless MAC
Dynamics", IEICE Transactions on Communications, 2007.
[LL07] Landstrom, S., and Larzon, L.A., "Reducing the TCP
Acknowledgement Frequency", SIGCOMM, Computer
Communications Review, July 2007.
[MJW00] Ming-Chit, I.T., Jinsong, D., and W. Wang, "Improving
TCP Performance Over Asymmetric Networks", SIGCOMM,
Computer Communications Review (CCR), Vol.30, No.3,
2000.
[Studies] Floyd, S., "Measurement Studies of End-to-End Congestion
Control in the Internet",
http://www.icir.org/floyd/ccmeasure.html.
[WWCM99] Wu, H., Wu, J., Cheng, S., and J. Ma, "ACK Filtering on
Bandwidth Asymmetry Networks", IEEE Communications,
1999.
[YMH03] Yu, L., Minhua, Y., and Z. Huimin, "The Improvement of
TCP Performance in Bandwidth Asymmetric Network", IEEE
PIMRC, 1:482-486, September 2003.Authors' Addresses 💖
and these are the humans who wrote it, so you know exactly who to thank (or tag) 🫶 their contact details are copied exactly too:
Sally Floyd
ICSI Center for Internet Research
1947 Center Street, Suite 600
Berkeley, CA 94704
USA
EMail: floyd@icir.org
Andres Arcia
Networking, Security & Multimedia (RSM) Universidad de Los Andes
TELECOM Bretagne Facultad de Ingenieria
Rue de la Chataigneraie, CS 17607 Nucleo La Hechicera
35576 Cesson Sevigne Cedex Merida, Merida 5101
France Venezuela
EMail: ae.arcia@telecom-bretagne.eu EMail: amoret@ula.ve
URI: http://www.ula.ve
David Ros
Networking, Security & Multimedia (RSM) Dpt.
TELECOM Bretagne
Rue de la Chataigneraie, CS 17607
35576 Cesson Sevigne Cedex
France
EMail: David.Ros@telecom-bretagne.eu
Janardhan R. Iyengar
Math and Computer Science
Franklin & Marshall College
P. O. Box 3003
Lancaster, PA 17604-3003
USA
EMail: jiyengar@fandm.edu🎀 that's the whole document, periodt 💅✨
📦 where this page comes from
- ietf: rfc5690 2010-02
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms