RFC 9840

rLEDBAT: Receiver-Driven Low Extra Delay Background Transport for TCP

📅 published September 2025, from ietf

Title Block and Vibe Check 🎀

Okay bestie, let's get into RFC 9840 ✨ It's titled rLEDBAT: Receiver-Driven Low Extra Delay Background Transport for TCP and it's written by M. Bagnulo, A. García-Martínez, and G. Montenegro from Universidad Carlos III de Madrid, plus P. Balasubramanian from Confluent. It's dated September 2025, it's Experimental 🧪, and it's an IRTF (Internet Research Task Force) document, not an IETF Standards Track thing. It doesn't obsolete or update any other RFCs. The ISSN is 2070-1721 💖

The whole point is a new spin on LEDBAT, the super polite background congestion control algorithm. Right now LEDBAT only lives on the sender, so the receiver can't really boss the download around. rLEDBAT moves that job to the receiver end of a TCP connection 🦋

Abstract 🌸

Okay so the abstract says this document specifies receiver-driven Low Extra Delay Background Transport (rLEDBAT), which is a set of mechanisms that let a less-than-best-effort congestion control algorithm run for TCP at the receiver end. It's a product of the Internet Congestion Control Research Group (ICCRG) of the IRTF. It's giving research energy and I'm obsessed 🫶

Status of This Memo 📜

This one is not an Internet Standards Track spec. It's published for examination, experimental implementation, and evaluation. It defines an Experimental Protocol for the Internet community, and it's an IRTF product. The IRTF publishes research results that might not be ready for deployment. This RFC represents the consensus of ICCRG. Anything the IRSG approves is not a candidate for any level of Internet Standard, see Section 2 of RFC 7841 🙈

You can find the current status, errata, and feedback info at https://www.rfc-editor.org/info/rfc9840.

Copyright (c) 2025 IETF Trust and the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as of the publication date. Read those carefully because they spell out your rights and restrictions 💅

Table of Contents 🗺️

1.  Introduction
2.  Conventions and Terminology
3.  Motivations for rLEDBAT
4.  rLEDBAT Mechanisms
  4.1.  Controlling the Receive Window
    4.1.1.  Avoiding Window Shrinking
    4.1.2.  Setting the Window Scale Option
  4.2.  Measuring Delays
    4.2.1.  Measuring RTT to Estimate the Queuing Delay
    4.2.2.  Measuring One-Way Delay to Estimate the Queuing Delay
  4.3.  Detecting Packet Losses and Retransmissions
5.  Experiment Considerations
  5.1.  Status of the Experiment at the Time of This Writing
6.  Security Considerations
7.  IANA Considerations
8.  References
  8.1.  Normative References
  8.2.  Informative References
Appendix A.  rLEDBAT Pseudocode
Acknowledgments
Authors' Addresses

1. Introduction 🌈

Okay so LEDBAT [RFC6817] is a congestion control algorithm for less-than-best-effort (LBE) traffic, which means it's the chill background stuff 😌

When LEDBAT traffic shares a bottleneck with traffic using standard congestion control, like TCP with CUBIC [RFC9438] (we're calling that "standard-TCP" for short), LEDBAT backs off earlier and harder. That leaves the non-background traffic more room. With no competition, LEDBAT tries to use the capacity efficiently while keeping queuing delay inside set bounds ✨

LEDBAT responds to both packet loss and delay changes. For loss, it does a multiplicative decrease, same as most TCP controllers. For delay, it aims for a target queuing delay. If the measured queuing delay is under the target, LEDBAT speeds up. If it's over, LEDBAT slows down. It estimates queuing delay by subtracting the current measured one-way delay from the estimated base one-way delay, meaning the one-way delay when there are no queues 🦋

The LEDBAT spec [RFC6817] defines the algorithm, and it's implemented in the sender to control its sending rate. It's written in a protocol-agnostic and layer-agnostic way 💜

LEDBAT++ [LEDBAT++] is another LBE algorithm inspired by LEDBAT that fixes several problems with the original. The main differences are:

  • i) LEDBAT++ uses round-trip time (RTT), not one-way delay, to estimate queuing delay 🍓
  • ii) LEDBAT++ uses an additive increase/multiplicative decrease algorithm to get inter-LEDBAT++ fairness and avoid the latecomer advantage seen in LEDBAT 👑
  • iii) LEDBAT++ does periodic slowdowns to improve the measurement of base delay 🌸
  • iv) LEDBAT++ is defined for TCP 💅

In this spec, we describe receiver-driven Low Extra Delay Background Transport (rLEDBAT), a set of mechanisms that let an LBE delay-based algorithm such as LEDBAT or LEDBAT++ run at the receiver end of a TCP connection 🎀

The ICCRG consensus is to publish this doc to encourage more experimentation and review of rLEDBAT. It's not an IETF product and not a Standards Track spec. The status is Experimental. Section 5 ("Experiment Considerations") explains the purpose of the experiment and where it's at now 🧪

2. Conventions and Terminology 🔑

The key words MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, SHOULD NOT, RECOMMENDED, NOT RECOMMENDED, MAY, and OPTIONAL in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.

Here are the abbreviations we use all through the text, listed for your convenience 💖

  • RCV.WND: the value in the Receive Window field of the TCP header (the way it's computed is changed by its specification) 🪟
  • SND.WND: the TCP sender's window 📤
  • cwnd: the congestion window computed by the congestion control algorithm running at the TCP sender 🌊
  • RLWND: the window value calculated by the rLEDBAT algorithm 🧚
  • fcwnd: the value a standard-TCP receiver compliant with [RFC9293] calculates to set in the receive window for flow control 🛟
  • RCV.HGH: the highest sequence number for a received byte of data at a given point in time ⭐
  • TSV.HGH: the Timestamp Value (TSval) [RFC7323] of the segment that carried RCV.HGH at that point in time 🕰️
  • SEG.SEQ: the sequence number of the last received segment 📦
  • TSV.SEQ: the TSval of the last received segment 🍬

3. Motivations for rLEDBAT 💫

Okay this is the part where the use cases come in, and rLEDBAT unlocks new ones and new deployment models for LBE traffic 🌈 Here are the scenarios it enables:

Content Delivery Networks (CDNs) and more sophisticated file distribution scenarios: Picture a software developer who wants to ship an update and would love to use LBE. They turn on LEDBAT or LEDBAT++ on the servers holding the file. But the file goes out through a CDN that doesn't do LBE congestion control, so transfers from CDN surrogates aren't LBE at all 😭 The developer might also control the client software doing the download, but since current LEDBAT and LEDBAT++ are sender-based, controlling the client doesn't turn on LBE in the connection. rLEDBAT would allow an LBE traffic class for file distribution in that setup 🚚

Interference from proxies and other middleboxes: Proxies and middleboxes are everywhere. Mobile networks often use proxies. Enterprise networks commonly deploy corporate proxies for filtering and firewalling. On satellite links, Performance Enhancing Proxies (PEPs) are deployed to deal with long delays in a TCP connection. These proxies terminate the TCP connection on both ends, which stops LBE congestion control from being used on the segment between the proxy and the client, the sink of the content. With rLEDBAT enabled, clients can turn on LBE traffic between themselves and the proxy 🛰️

Receiver-defined preferences: The access link is often the bottleneck, especially for mobile devices, so it really matters that they manage that link well 📱 A device can use different congestion control algorithms with different traffic preferences. For example, it can use standard-TCP for some traffic and LEDBAT/LEDBAT++ for other traffic. But that only affects outgoing traffic, since both standard-TCP and LEDBAT/LEDBAT++ are sender-driven. The mobile device can't manage downlink traffic, and the downlink is usually the bottleneck for a typical "eyeball" end user 👀 rLEDBAT lets the device pick an LBE traffic class for some incoming traffic. For instance, a user can run regular standard-TCP/UDP for a video stream (like YouTube) and use rLEDBAT for background file downloads 🎬

4. rLEDBAT Mechanisms ✨

okay so rLEDBAT gives you the tools to run an LBE congestion control algorithm at the receiving end of a TCP connection 💖 The rLEDBAT receiver steers the sender's rate by using the receive window it announces in the TCP header 🎀

rLEDBAT assumes the sender is a standard-TCP sender 🫶 It doesn't need any rLEDBAT-specific changes to the TCP sender, which is so iconic. The deployment idea is that clients implement rLEDBAT, and that turns it on when they talk to existing standard-TCP senders ✨ The sender MUST implement [RFC9293] and also MUST implement the TCP Timestamps (TS) option as defined in [RFC7323]. The sender should also implement some of the standard congestion control mechanisms, like CUBIC [RFC9438] or NewReno [RFC5681] [RFC6582] 🌸

rLEDBAT does not define a new congestion control algorithm 🙈 The actual LBE algorithm running in the rLEDBAT receiver is beyond the scope of this document. The rLEDBAT receiver MUST use an LBE congestion control algorithm. Since rLEDBAT assumes a standard-TCP sender, the sender will be using a "best effort" congestion control algorithm (such as CUBIC or NewReno) 💅 Here's the tea: rLEDBAT uses the receive window to control the sender's rate, and the sender calculates its window as the minimum of the receive window and the congestion window. So rLEDBAT will only be effective as long as the algorithm running in the receiver yields a smaller window than the one the sender calculates. That's normally the case when the receiver is using an LBE congestion control algorithm 🦋 The rLEDBAT receiver SHOULD use the LEDBAT congestion control algorithm [RFC6817] or the LEDBAT++ congestion control algorithm [LEDBAT++]. rLEDBAT MAY use other LBE congestion control algorithms defined elsewhere. Irrespective of which algorithm runs in the receiver, an rLEDBAT connection will never be more aggressive than standard TCP, since it's always bounded by the congestion control algorithm running at the sender 👑

rLEDBAT is basically made of three types of mechanisms 🧚 The first group measures the packet delay (either the RTT or the one-way delay, depending on the selected algorithm). The second detects packet loss. The third manipulates the receive window to control the sender's rate. The first two feed input to the LBE congestion control algorithm, while the third uses the congestion window computed by that algorithm to manipulate the receive window, as shown in Figure 1 🌈

           +------------------------------------------+
           |   TCP Receiver                           |
           |                      +-----------------+ |
           |                      |  +------------+ | |
           |   +---------------------|     RTT    | | |
           |   |                  |  | Estimation | | |
           |   |                  |  +------------+ | |
           |   |                  |                 | |
           |   |                  |  +------------+ | |
           |   |      +--------------| Loss, RTX  | | |
           |   |      |           |  | Detection  | | |
           |   |      |           |  +------------+ | |
           |   v      v           |                 | |
           | +----------------+   |                 | |
           | | LBE Congestion |   |    rLEDBAT      | |
           | |    Control     |   |                 | |
           | +----------------+   |                 | |
           |       |              |  +------------+ | |
           |       |              |  | RCV.WND    | | |
           |       +---------------->| Control    | | |
           |                      |  +------------+ | |
           |                      +-----------------+ |
           +------------------------------------------+

Figure 1: The rLEDBAT Architecture

We'll go through each of the rLEDBAT components next 🎀

4.1. Controlling the Receive Window

rLEDBAT uses the TCP receive window (RCV.WND) to let the receiver control the sender's rate 🍓 [RFC9293] says RCV.WND is used to announce the available receive buffer to the sender for flow control purposes. To avoid confusion, we'll call fcwnd the value that a standard-TCP receiver compliant with [RFC9293] calculates to set in the receive window for flow control purposes. We call RLWND the window value calculated by the rLEDBAT algorithm, and we call RCV.WND the value actually included in the Receive Window field of the TCP header. For a receiver compliant with [RFC9293], RCV.WND == fcwnd.

In the case of the rLEDBAT receiver, this receiver MUST NOT set the RCV.WND to a value larger than fcwnd, and it SHOULD set the RCV.WND to the minimum of RLWND and fcwnd, honoring both 🫶

Okay, this part is giving two-controllers energy 😭 When using rLEDBAT, two congestion controllers are working on the flow of data from the sender to the receiver. One is the TCP congestion control algorithm on the sender side. The other is the LBE congestion control algorithm running in the receiver, which gets conveyed to the sender through RCV.WND. In normal TCP operation, the sender uses the minimum of the cwnd and the RCV.WND to calculate the SND.WND. This is also true for rLEDBAT, since the sender is a regular TCP sender. That guarantees the rLEDBAT flow will never transmit more aggressively than a standard-TCP flow, because the sender's congestion window limits the sending rate 💜 Also, an LBE congestion control algorithm such as LEDBAT/LEDBAT++ is designed to react earlier and more aggressively to congestion than regular TCP congestion control. So the RLWND carried in the TCP RCV.WND field will generally be smaller than the congestion window calculated by the TCP sender, which means the rLEDBAT congestion control algorithm ends up effectively controlling the sender's window 🦋 One exception: at the beginning of the connection, when there's no information to set RLWND, RLWND is set to its maximum value. That way the sender's rate is governed by the receiver's flow control algorithm and the TCP slow start mechanism of the sender ✨

In summary, the sender's window is SND.WND = min(cwnd, RLWND, fcwnd) 💅

4.1.1. Avoiding Window Shrinking

The LEDBAT/LEDBAT++ algorithm running in an rLEDBAT receiver increases or decreases RLWND based on congestion signals (variations in the estimated queuing delay and packet loss). If RLWND is decreased and announced directly in RCV.WND, the announced window could end up smaller than what's currently in use. This so-called "shrinking the window" is discouraged as per [RFC9293], because it may cause unnecessary packet loss and performance penalties 🙈 To be consistent with [RFC9293], the rLEDBAT receiver SHOULD NOT shrink the receive window.

To avoid window shrinking, the receiver MUST only reduce RCV.WND by the number of bytes contained in a received data packet. This may fall short of honoring the new calculated value of RLWND immediately. However, the receiver SHOULD progressively reduce the advertised RCV.WND, always making sure the reduction is less than or equal to the received bytes, until the target window determined by the rLEDBAT algorithm is reached. This means it may take up to one RTT for the rLEDBAT receiver to drain enough in-flight bytes to completely close its receive window without shrinking it 🌸 That's still enough to honor the window output from the LEDBAT/LEDBAT++ algorithms, since they are only allowed to perform at most one multiplicative decrease per RTT 👑

4.1.2. Setting the Window Scale Option

The Window Scale (WS) option [RFC7323] is a way to increase the maximum window size permitted by the receive window 🎀 The WS option defines a scale factor that restricts the granularity of the receive window that can be announced. So the rLEDBAT client will have to accumulate the increases from multiple received packets, and only convey a change in the window when the accumulated sum of increases is equal to or higher than one increase step, as imposed by the scaling factor from the WS option in place for the TCP connection 💖

Changes in the receive window smaller than 1 MSS (Maximum Segment Size) are unlikely to have any immediate impact on the sender's rate. As usual, TCP's segmentation practice results in sending full segments (i.e., segments of size equal to the MSS) 🧚 [RFC7323], which defines the WS option, specifies that allowed values for the WS option are between 0 and 14. Assuming an MSS of around 1500 bytes, WS option values between 0 and 11 result in the receive window being expressed in units of about 1 MSS or smaller. So WS option values between 0 and 11 have no impact in rLEDBAT (unless packets smaller than the MSS are being exchanged) 🍓

WS option values higher than 11 can affect the dynamics of rLEDBAT, since control may become too coarse. For example, with a WS option value of 14, a change of one unit in the receive window implies a change of 10 MSS in the effective window 😭

For the above reasons, the rLEDBAT client SHOULD set WS option values lower than 12. Additional experimentation is required to explore the impact of larger WS values on rLEDBAT dynamics 🔥

Note that the recommendation for rLEDBAT to set lower WS option values does not preclude communication with servers that set larger WS option values, since WS option values are set independently for each direction of the TCP connection 🌈

4.2. Measuring Delays

Both LEDBAT and LEDBAT++ measure base and current delays to estimate the queuing delay 💜 LEDBAT uses the one-way delay, while LEDBAT++ uses the RTT. In the next sections, we describe how rLEDBAT mechanisms let the receiver measure the one-way delay or the RTT, whichever is needed depending on the congestion control algorithm being used ✨

4.2.1. Measuring RTT to Estimate the Queuing Delay

okay so LEDBAT++ uses the RTT to figure out the queuing delay 🎀 to get that number, the rLEDBAT receiver has to estimate the base RTT (the constant parts of RTT, bestie) AND measure the current RTT 📏 then you subtract one from the other and boom, you've got the queuing delay for the rLEDBAT controller to use ✨

LEDBAT++ finds the base RTT (RTTb) by taking the minimum of the measured RTTs over a period of time 🌸 the current RTT (RTTc) is estimated from a bunch of recent samples with a filter, like the minimum (or the mean) of the last k samples 💅 using RTT to estimate queuing delay has some shortcomings and difficulties tho, and we'll get into those below 😭

here's the tea: the queuing delay measured with RTT also includes the queuing delay that return packets pick up on the way from the rLEDBAT receiver back to the sender 💀 this is a fundamental limitation of this approach, not a bug you can patch. the impact? the rLEDBAT controller will also react to congestion on the reverse path, which makes it an even more conservative mechanism 🫶

now, to measure RTT, the rLEDBAT client MUST enable the TS option [RFC7323] 🔥 by matching the TSval carried in outgoing packets with the Timestamp Echo Reply (TSecr) value [RFC7323] seen in incoming packets, you can measure RTT 🦋 this is the magic part because it lets the rLEDBAT receiver measure RTT even when it's acting as a pure receiver. and a pure receiver has no data flowing from it to the sender, which means it can't match data packets with Acknowledgment packets to measure RTT the way TCP usually does for other purposes 🙈 so the TS option is the only real option here, iykyk 💜

ok but here's a catch: depending on how fast the local clock ticks for the TS option, several packets may carry the same TSval 😵 if that happens, the rLEDBAT receiver can't match the different outgoing packets with the same TSval to the different incoming packets that carry the same TSecr value. buuut it's fine bestie, you do NOT need every packet to estimate RTT! sampling a subset of in-flight packets per RTT is enough to properly assess the queuing delay ✨ so RTT MUST be calculated as the time since the first packet with a given TSval was sent, until the first packet received with that same value in its TSecr. other packets with repeated TS values SHOULD NOT be used for RTT calculations 🚫 that's the rule, no cap 🧚

several issues have to be handled so you don't get an artificial increase in the observed RTT 🙅‍♀️ and the issues are different depending on whether the rLEDBAT-capable host is sending data packets or pure ACKs to measure RTT. so we're gonna look at those separately 💅

4.2.1.1. Measuring RTT When Sending Pure ACKs

so in this scenario, the rLEDBAT node (node A) sends a pure ACK to the other end of the TCP connection (node B), and that ACK includes the TS option 🎀 when host B gets the TS option, it copies the TSval into the TSecr field of its TS option and includes that in the next data packet heading toward host A 💌 but there are two reasons B might not send a packet right back to A, and that would artificially inflate the measured RTT 😤 reason one: A has no data to send. reason two: A has no available window to put more packets in flight. we'll go through how each one gets handled 🌈

the case where host B has no data to send when it gets the pure Acknowledgment is expected to be rare for rLEDBAT use cases 💖 rLEDBAT is mostly used for background file transfers, so the common vibe is that the sender has data to send for the whole life of the communication ✨ but if, say, the file is split into blocks, the sender might sometimes have to wait for the next block to be ready before it can keep going 😩 to handle that, the filter used by the congestion control algorithm running in the receiver SHOULD discard outliers when measuring RTT from pure ACK packets (a MIN filter [RFC6817] would do the trick) 🫧

this limit on the sender's window can come from either the TCP congestion window in host B or the receive window that rLEDBAT announces from host A 🦋 normally, the receive window is the one doing the limiting, since the LBE congestion control algorithm used by the rLEDBAT node is designed to be stricter about the sender's rate than standard TCP 💅 if the congestion window in the sender is the limiting factor, it matters less if rLEDBAT shrinks the receive window because of a bloated RTT measurement, since the rLEDBAT node isn't actively controlling the sender's rate in that case 🤷‍♀️ still, discarding the larger samples would also fix this issue, so it's a win-win 🏆

now, for the case where the limiting factor is the receive window that rLEDBAT announces: the congestion control algorithm at the receiver SHOULD discard RTT measurements taken during the window reduction phase that are triggered by pure ACK packets 🙌 the rLEDBAT receiver knows whether a given TSval was sent in a pure ACK packet where the window was reduced, and if it was, it can throw out the matching RTT measurement 🗑️✨

4.2.1.2. Measuring RTT When Sending Data Packets

so if the rLEDBAT node is sending data packets and matching them with pure ACKs to measure RTT, one thing that can artificially bump up the RTT is delayed Acknowledgments 🙈 according to the TS option generation rules [RFC7323], the value put in the TSecr for a delayed ACK is the TSval from the earliest unacknowledged segment. yeah, that can artificially inflate the measured RTT 😭

if both endpoints are sending data packets, the Acknowledgments get piggybacked onto those data packets and they aren't delayed 🫶 delayed ACKs only push RTT measurements up when the sender has no data to send. and since the expected use case for rLEDBAT is the sender pushing background traffic to the rLEDBAT receiver, the cases where delayed ACKs inflate the measured RTT are expected to be rare 🌸

that said, measurements based on data packets from the rLEDBAT node that match pure ACKs from the other end will still produce an increased RTT sample 😵‍💫 the extra increase in measured RTT can be up to 500 ms, because delayed ACKs are generated for every second data packet received and aren't delayed more than 500 ms according to [RFC9293] ⏱️ the rLEDBAT receiver MAY discard RTT measurements done using data packets from the rLEDBAT receiver that match pure ACKs, especially if it already has recent measurements from other packet combinations 🧚‍♀️ and applying a filter (like a MIN filter) that discards outliers would also deal with this issue 🎀

4.2.2. Measuring One-Way Delay to Estimate the Queuing Delay

the LEDBAT algorithm uses the one-way delay of packets as input 🦋 a TCP receiver can measure the delay of incoming packets directly. (that's different from the sender-based LEDBAT, where the receiver measures the one-way delay and has to send it back to the sender 🌈)

with TCP, the receiver can use the TS option to measure the one-way delay by subtracting the timestamp inside the incoming packet from the local time when that packet arrived 💜 as noted in [RFC6817], the clock offset between the sender's clock and the receiver's clock doesn't affect LEDBAT operation 🙌 LEDBAT uses the difference between the base one-way delay and the current one-way delay to estimate the queuing delay, so the clock offset basically "cancels out" in the queuing delay estimate ✨ there are two other things the rLEDBAT receiver needs to take into account to estimate one-way delay properly, though: the units the received timestamps are expressed in, and the clock skew 🤓 we'll cover both below.

to measure one-way delay using TCP timestamps, the rLEDBAT receiver first has to discover the units of the values in the TS option, and then account for the skew between the two endpoint clocks 🫶 heads up: a mismatch of 100 ppm (parts per million) in the estimate of the sender's clock rate adds up to 6 ms of variation per minute in the measured delay 😱 that's just one order of magnitude below the target delay set by rLEDBAT (or even closer if the target is set to lower values, which is possible) 💀 and typical skew for untrained clocks is reported to be around 100-200 ppm [RFC6817] 🎀

so to learn both the TS units and the clock skew, the rLEDBAT receiver measures how much local time has passed between two packets that have different TS values from the sender ⏳ by comparing the local time difference with the TS value difference, the receiver can work out the TS units and the relative clock skew 🔍 for this to be accurate, the packets carrying the different TS values should experience equal (or at least similar) delay on their way from the sender to the receiver, because any difference in experienced delay would throw off the unit/skew estimate 😵 one good approach is picking packets that experienced minimal delay (aka queuing delay close to zero) to do the estimates 💖

here's another twist: the LEDBAT congestion controller's actions directly affect the (queuing) delay packets experience 🌀 so if there's an error in the TS units/skew estimate, the LEDBAT controller will try to compensate by reducing or increasing the load 🙃 the result is that LEDBAT operation interferes with the TS units/clock skew measurements 💔 measurements are more accurate when there's no traffic on the connection (other than the packets used for the measurements) 🤫 the problem is the receiver doesn't know whether the sender is injecting traffic at any given moment, so it can't use the quiet intervals to measure stuff 😩 what the receiver can do is force periodic slowdowns, reducing the announced receive window to a few packets and doing the measurements at that time 🛑✨

the rLEDBAT receiver can do multiple measurements to figure out both the TS units and the relative clock skew over the lifetime of the connection, to get more accurate results 📈 clock skew measurements get more accurate when the time period used to discover the skew is longer, since the impact of the skew becomes more obvious 🔎 so it's a reasonable approach for the rLEDBAT receiver to do an early discovery of the TS units (and the clock skew) using the first few packets of the TCP connection, and then refine the TS units/clock skew estimate with periodic measurements later on 💅🌸

4.3. Detecting Packet Losses and Retransmissions 🕵️

Okay so the rLEDBAT receiver can actually spot retransmitted packets ✨. Here's the tea: we call RCV.HGH the highest sequence number corresponding to a received byte of data. We are NOT assuming that every byte with a smaller sequence number has already arrived, so holes are totally allowed 🕳️. TSV.HGH is the TSval of the segment that carried that byte. SEG.SEQ is the sequence number of a newly received segment, and TSV.SEQ is that new segment's TSval 💖.

If SEG.SEQ < RCV.HGH and TSV.SEQ > TSV.HGH, then the newly received segment is a retransmission 🔁. Why? Because it was generated later than an already-received segment that carried a larger sequence number. So this segment was lost and then retransmitted. Iconic detective work, honestly 🫶.

The detection mechanism has one major weakness, though 😬. It fails when there are window tail drops. If every packet at the tail of the window is lost, the receiver can't notice any mismatch between the sequence numbers and the order of the timestamps. In that case rLEDBAT won't react to the losses. Instead, the TCP congestion controller at the sender will most likely drop its window down to 1 MSS and take over the sending rate until slow start ramps back up and catches the current rLEDBAT window value 🫠.

5. Experiment Considerations 🧪

5.1. Status of the Experiment at the Time of This Writing

The status of this document is Experimental 🧪. The big picture is that we want more experience running rLEDBAT over different network paths, so we can see whether the proposed rLEDBAT parameters perform well in different situations 🌈. We specifically want to learn about the following:

  • Interaction between the sender's and receiver's congestion control algorithms 👑. rLEDBAT assumes that, because the rLEDBAT receiver uses a less-than-best-effort congestion control algorithm, the receiver's algorithm will expose a smaller congestion window (conveyed through the receive window) than the one the sender's algorithm produces. One goal of the experiment is to learn how these two algorithms interact, and whether the assumption that the receiver side is always controlling the sender's rate (and making rLEDBAT effective) actually holds. The experiment should include the congestion control algorithms that are widely used on the Internet today, including CUBIC, Bottleneck Bandwidth and Round-trip propagation time (BBR), and LEDBAT(++) 🦋.
  • Interaction between rLEDBAT and Active Queue Management techniques such as Controlled Delay (CoDel), Proportional Integral controller Enhanced (PIE), and Low Latency, Low Loss, and Scalable Throughput (L4S) 🍓.
  • How rLEDBAT should resume after a period with no incoming traffic, when the rLEDBAT state information is potentially dated 🙈.

Currently, these implementations of rLEDBAT can be used for experimentation 💅:

  • Windows 11. rLEDBAT is available in Microsoft's Windows 11 22H2 since October 2023 [Windows11].
  • Windows Server 2022. rLEDBAT is available in Microsoft's Windows Server 2022 since September 2022 [WindowsServer].
  • Apple. rLEDBAT is available in macOS and iOS since 2021 [Apple].
  • Linux implementation, open source, available since 2022 [rledbat_module].
  • ns3 implementation, open source, available since 2020 [rLEDBAT-in-ns-3].

And here's the wild part 😭. rLEDBAT has been deployed by Microsoft at wide scale in these services:

  • BITS (Background Intelligent Transfer Service)
  • DO (Delivery Optimization) service
  • Windows update: using DO
  • Windows Store: using DO
  • OneDrive
  • Windows Error Reporting: wermgr.exe; werfault.exe
  • System Center Configuration Manager (SCCM)
  • Windows Media Player
  • Microsoft Office
  • Xbox (download games): using DO

Some initial experiments involving rLEDBAT have been reported in [COMNET3]. Experiments on the interaction between LEDBAT++ and BBR are presented in [COMNET2], and an experimental evaluation of the LEDBAT++ algorithm is presented in [COMNET1] 📚. LEDBAT++ is one of the less-than-best-effort congestion control algorithms that rLEDBAT relies on, so the results on how LEDBAT++ interacts with other congestion control algorithms are relevant for understanding rLEDBAT too 🎀.

6. Security Considerations 🔐

Overall, we believe that rLEDBAT does not introduce any new vulnerabilities to existing TCP endpoints, because it relies on existing TCP knobs, notably the receive window and timestamps 💜.

Specifically, rLEDBAT uses RCV.WND to modulate the rate of the sender. An attacker who wants to starve a flow can simply reduce the RCV.WND, whether or not rLEDBAT is being used. Yikes, bestie 💀.

Now the spicier question: can an attacker use the rLEDBAT mechanisms to force the rLEDBAT receiver to reduce the RCV.WND? There are two ways an attacker can try this 😈.

  • One would be to introduce an artificial delay to the packets, either by actually delaying them or by modifying the timestamps. This would make the rLEDBAT receiver believe that a queue is building up, so it reduces the RCV.WND. Note that to do so, an attacker must be on path. So if that is the case, it is probably more direct to simply reduce the RCV.WND 🙃.
  • The other option would be for the attacker to make the rLEDBAT receiver believe that a loss has occurred. To do this, the attacker basically needs to retransmit an old packet. To be precise, it needs to transmit a packet with the correct sequence number and the correct port and IP numbers. This means the attacker can reduce the incoming traffic to the rLEDBAT receiver not only by modifying the RCV.WND field of packets originated from the rLEDBAT host, but also by injecting packets with the proper sequence number in the other direction. This may slightly expand the attack surface 🚨.

7. IANA Considerations 📋

This document has no IANA actions.

8. References 📖

8.1. Normative References

8.2. Informative References

Appendix A. rLEDBAT Pseudocode 🧚

okay bestie, this is the fun part 💖 we're looking at how the rLEDBAT bits plug into an LBE delay-based congestion control algorithm like LEDBAT or LEDBAT++ 🎀 The integrated algorithm is written as two procedures. One runs when an rLEDBAT-enabled endpoint receives a packet (Figure 2), and the other runs when that endpoint sends one (Figure 3) ✨ At the start, RLWND is set to its maximum value 🌈 so the sender's rate is mostly run by the receiver's flow control and the TCP slow start mechanism, and ackedBytes starts at 0 🫶

We're assuming the LBE congestion control algorithm gives us a WindowIncrease() function and a WindowDecrease() function 💅 For LEDBAT++, WindowIncrease() is an additive increase and WindowDecrease() is a multiplicative decrease 🦋 WindowIncrease() takes the current window size plus the number of bytes acknowledged since the last window update (ackedBytes), and hands back the updated window size. WindowDecrease() takes the current window size and returns the updated one. Simple and iconic 👑

Now the data structures, which are lowkey the heart of it all 🍓 The sendList holds the TSval and the local send time of every packet the rLEDBAT-enabled endpoint sends. Then the TSecr field of each received packet gets matched against sendList to work out the RTT 🧚

The RTTlist stores the RTT values computed for each received packet, along with the received TSecr. To avoid using multiple packets with the same TSecr for RTT calculations, only the first packet received for a given TSecr is used to compute the RTT, so no double-counting bestie 🙅‍♀️ RTTlist also keeps the local time each packet was received, so you can pick out the RTTs measured in a given period (for example, the last 10 minutes) ⏰ And RTTlist is initialized with all its values set to the maximum 💜

Figure 2: Procedure Executed When a Packet Is Received

   procedure receivePacket()
     //Looks for first sent packet with same TSval as TSecr, and
     //returns time difference
     receivedRTT = computeRTT(sendList, receivedTSecr, receivedTime)

     //Inserts minimum value for a given receivedTSecr
     //Note that many received packets may contain same receivedTSecr
     insertRTT (RTTlist, receivedRTT, receivedTSecr, receivedTime)

     filteredRTT = minLastKMeasures(RTTlist, K=4)
     baseRTT = minLastNSeconds(RTTlist, N=180)
     qd = filteredRTT - baseRTT

     //ackedBytes is the number of bytes that can be used to reduce
     //the receive window - without shrinking it - if necessary
     ackedBytes = ackedBytes + receiveBytes

     if retransmittedPacketDetected then
           RLWND = DecreaseWindow(RLWND)  //Only once per RTT
     end if
     if qd < T then
           RLWND = IncreaseWindow(RLWND, ackedBytes)
     else
           RLWND = DecreaseWindow(RLWND)
     end if
   end procedure

Figure 3: Procedure Executed When a Packet Is Sent

   procedure SENDPACKET
     if (RLWND > RLWNDPrevious) or (RLWND - RLWNDPrevious < ackedBytes)
     then
           RLWNDPrevious = RLWND
     else
           RLWNDPrevious = RLWND - ackedBytes
     end if
     ackedBytes = 0
     RLWNDPrevious = RLWND

     //Compute the RLWND to include in the packet
     RLWND = min(RLWND, fcwnd)
   end procedure

Acknowledgments 🫶

this work was supported by the EU through the StandICT projects RXQ, CCI, and CEL6 🇪🇺; the NGI Pointer RIM project; and the H2020 5G-RANGE project 🌟 and by the Spanish Ministry of Economy and Competitiveness through the 5G-City project (TEC2016-76795-C6-3-R) 💐

we would like to thank ICCRG chairs Reese Enghardt and Vidhi Goel for their support on this work 🎀 we'd also like to thank Daniel Havey for his help 🙏 and we'd like to thank Colin Perkins, Mirja Kühlewind, and Vidhi Goel for their reviews and comments on earlier draft versions of this document 💖 literally the squad 🥹

Authors' Addresses 💅

Marcelo Bagnulo Universidad Carlos III de Madrid Email: marcelo@it.uc3m.es

Alberto García-Martínez Universidad Carlos III de Madrid Email: alberto@it.uc3m.es

Gabriel Montenegro Email: g.e.montenegro@hotmail.com

Praveen Balasubramanian Confluent Email: pravb.ietf@gmail.com

📦 where this page comes from