RFC 5348
TCP Friendly Rate Control (TFRC): Protocol Specification
RFC 5348 💖
TCP Friendly Rate Control (TFRC): Protocol Specification ✨
Okay bestie, here's the vibe check first. This is an Internet standards track doc from the Network Working Group, published in September 2008. The authors are S. Floyd (ICIR), M. Handley (University College London), J. Padhye (Microsoft), and J. Widmer (DoCoMo). It obsoletes RFC 3448 and updates RFC 4342, so it's the new official version of TFRC 💅🎀
Status of This Memo 📜
This document specifies an Internet standards track protocol for the Internet community, and requests discussion and suggestions for improvements. For the standardization state and status of this protocol, check the current edition of the "Internet Official Protocol Standards" (STD 1). Distribution of this memo is unlimited 🌸
Abstract 🦋
Okay so this doc specifies TCP Friendly Rate Control (TFRC). TFRC is a congestion control mechanism for unicast flows running in a best-effort Internet environment. It's reasonably fair when it's competing for bandwidth with TCP flows, but its throughput varies way less over time than TCP's does. That makes it a great fit for stuff like streaming media, where a relatively smooth sending rate really matters ✨
This document obsoletes RFC 3448 and updates RFC 4342.
Table of Contents 🗂️
1. Introduction ....................................................3
2. Conventions .....................................................4
3. Protocol Mechanism ..............................................4
3.1. TCP Throughput Equation ....................................5
3.2. Packet Contents ............................................7
3.2.1. Data Packets ........................................7
3.2.2. Feedback Packets ....................................8
4. Data Sender Protocol ............................................8
4.1. Measuring the Segment Size .................................9
4.2. Sender Initialization .....................................10
4.3. Sender Behavior When a Feedback Packet Is Received ........10
4.4. Expiration of Nofeedback Timer ............................15
4.5. Reducing Oscillations .....................................17
4.6. Scheduling of Packet Transmissions ........................18
5. Calculation of the Loss Event Rate (p) .........................19
5.1. Detection of Lost or Marked Packets .......................19
5.2. Translation from Loss History to Loss Events ..............20
5.3. The Size of a Loss Interval ...............................22
5.4. Average Loss Interval .....................................22
5.5. History Discounting .......................................24
6. Data Receiver Protocol .........................................26
6.1. Receiver Behavior When a Data Packet Is Received ..........27
6.2. Expiration of Feedback Timer ..............................27
6.3. Receiver Initialization ...................................28
6.3.1. Initializing the Loss History after the
First Loss Event ...................................29
7. Sender-Based Variants ..........................................30
8. Implementation Issues ..........................................31
8.1. Computing the Throughput Equation .........................31
8.2. Sender Behavior When a Feedback Packet Is Received ........32
8.2.1. Determining If an Interval Was a
Data-Limited Interval ..............................32
8.2.2. Maintaining X_recv_set .............................34
8.3. Sending Packets before Their Nominal Send Time ............34
8.4. Calculation of the Average Loss Interval ..................36
8.5. The Optional History Discounting Mechanism ................36
9. Changes from RFC 3448 ..........................................36
9.1. Overview of Changes .......................................36
9.2. Changes in Each Section ...................................37
10. Security Considerations .......................................39
10.1. Security Considerations for TFRC in DCCP .................40
11. Acknowledgments ...............................................40
Appendix A. Terminology ...........................................41
Appendix B. The Initial Value of the Nofeedback Timer .............43
Appendix C. Response to Idle or Data-Limited Periods ..............44
C.1. Long Idle or Data-Limited Periods ........................45
C.2. Short Idle or Data-Limited Periods .......................48
C.3. Moderate Idle or Data-Limited Periods ....................49
C.4. Losses During Data-Limited Periods .......................50
C.5. Other Patterns ...........................................53
C.6. Evaluating TFRC's Response to Idle Periods ...............53
References ........................................................54
Normative References ...........................................54
Informative References .........................................541. Introduction 🧚
Okay so this doc specifies TCP Friendly Rate Control (TFRC). TFRC is a congestion control mechanism made for unicast flows running in an Internet environment, where it competes with TCP traffic [FHPW00]. It's not a complete protocol, though. It's just a congestion control mechanism that could be used in a transport protocol like DCCP (Datagram Congestion Control Protocol) [RFC4340], in an application that does end-to-end congestion control at the application level, or in the context of endpoint congestion management [BRS99]. This document doesn't cover packet formats or reliability 🙈 Implementation stuff only gets a brief mention, in Section 8.
TFRC is designed to be reasonably fair when it's competing for bandwidth with TCP flows. Here, "reasonably fair" means the sending rate is generally within a factor of two of the sending rate of a TCP flow under the same conditions. The main plot twist is that TFRC has way lower throughput variation over time than TCP does. That makes it better for stuff like telephony or streaming media, where a relatively smooth sending rate is important 💜
Here's the catch. Being smoother than TCP while competing fairly means TFRC responds more slowly than TCP when the available bandwidth changes. So TFRC should only be used when an app actually needs smooth throughput, especially to avoid TCP's halving of the sending rate after a single packet drop. If your app just wants to move as much data as possible as fast as possible, we recommend TCP. Or, if reliability isn't required, you could use an Additive-Increase, Multiplicative-Decrease (AIMD) congestion control scheme with similar parameters to the ones TCP uses 💅
TFRC is designed for best performance with apps that use a fixed segment size and vary their sending rate in packets per second in response to congestion. TFRC can also be used with apps that don't have a fixed segment size, where the segment size changes based on what the app needs (e.g., video applications). The performance might be a little less optimal in that case, though 🌈
Some apps (e.g., some audio applications) need a fixed interval of time between packets, so they vary their segment size instead of their packet rate when there's congestion. The congestion control in this document isn't designed for those apps. TFRC-SP (Small-Packet TFRC) is a variant of TFRC for apps that have a fixed sending rate in packets per second but either use small packets or vary their packet size in response to congestion. TFRC-SP gets its own separate doc [RFC4828] 🎀
This document specifies TFRC as a receiver-based mechanism. That means the congestion control information (i.e., the loss event rate) gets calculated in the data receiver, not the data sender. This is a great match for an app where the sender is a big server handling tons of concurrent connections, and the receiver has more memory and CPU cycles to spare. Receiver-based mechanisms are also a better building block for multicast congestion control. It's still possible to implement TFRC in sender-based variants, as allowed in DCCP's Congestion Control ID 3 (CCID 3) [RFC4342] 🦋
This document obsoletes RFC 3448. In the transport protocol DCCP (Datagram Congestion Control Protocol) [RFC4340], the Congestion Control ID Profiles CCID-3 [RFC4342] and CCID-4 [CCID-4] both specify the use of TFRC from RFC 3448. So CCID-3 and CCID-4 implementations SHOULD use this document instead of RFC 3448 for the specification of TFRC. Lowkey a glow-up moment ✨
The normative specification of TFRC is in Sections 3-6. Section 7 covers sender-based variants, Section 8 covers implementation issues, and Section 9 gives a non-normative overview of the differences from RFC 3448 📜
2. Conventions 🎀
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. Those are the official BCP 14 words, bestie, so read them carefully 🫶
Appendix A gives a list of technical terms used in this document 🌸
3. Protocol Mechanism 🔥
Okay, the main idea. For its congestion control mechanism, TFRC directly uses a throughput equation for the allowed sending rate. That equation is a function of the loss event rate and the round-trip time. To compete fairly with TCP, TFRC uses the TCP throughput equation, which roughly describes TCP's sending rate as a function of the loss event rate, round-trip time, and segment size. A loss event is one or more lost or marked packets from a window of data. A marked packet is a congestion indication from Explicit Congestion Notification (ECN) [RFC3168] 💖
Generally speaking, TFRC's congestion control works like this:
- The receiver measures the loss event rate and feeds that info back to the sender.
- The sender also uses these feedback messages to measure the round-trip time (RTT).
- The loss event rate and RTT get plugged into TFRC's throughput equation. The resulting sending rate is capped at no more than twice the receive rate, which gives the allowed transmit rate X.
- The sender then adjusts its transmit rate to match the allowed transmit rate X.
The dynamics of TFRC are super sensitive to how the measurements are performed and applied, so this part matters a lot 👑 We recommend specific mechanisms below to perform and apply these measurements. Other mechanisms are possible, but it's important to understand how the interactions between mechanisms affect the dynamics of TFRC 🍓
3.1. TCP Throughput Equation
okay so bestie, any realistic equation that gives TCP throughput as a function of loss event rate and RTT is suitable for TFRC 🦋. BUT the TCP throughput equation you use has to reflect TCP's retransmit timeout behavior, because that's what dominates TCP throughput at higher loss rates 💀. The assumptions the equation makes about the loss event rate parameter also have to be a reasonable match for how the loss rate or loss event rate is actually measured. The match isn't perfect for the equation and measurement mechanisms below, but in practice it's close enough ✨.
The throughput equation currently REQUIRED for TFRC is a slightly simplified version of the throughput equation for Reno TCP from [PFTK98] 💅. Ideally we'd use one based on selective acknowledgment (SACK) TCP, but nobody has derived that one yet, and simulations and experiments suggest the two equations differ only a little [FF99] (Appendix B) 🌸.
The equation for X_Bps, TCP's average sending rate in bytes per second, is:
s
X_Bps = ----------------------------------------------------------
R*sqrt(2*b*p/3) + (t_RTO * (3*sqrt(3*b*p/8)*p*(1+32*p^2)))Where:
X_Bps is TCP's average transmit rate in bytes per second. (X_Bps is the same as X_calc in [RFC3448].)
s is the segment size in bytes (excluding IP and transport protocol headers).
R is the round-trip time in seconds.
p is the loss event rate, between 0 and 1.0, of the number of loss events as a fraction of the number of packets transmitted.
t_RTO is the TCP retransmission timeout value in seconds.
b is the maximum number of packets acknowledged by a single TCP acknowledgement.
Setting the TCP retransmission timeout value t_RTO: Implementations SHOULD set t_RTO = 4*R. Implementations MAY choose to implement a more accurate calculation of t_RTO. Implementations MAY also set t_RTO to max(4*R, one second), to match the recommended minimum of one second on the RTO [RFC2988] 🧚.
Setting the parameter b for delayed acknowledgements: Some current TCP connections use delayed acknowledgements, sending an acknowledgement for every two data packets received. But TCP is also allowed to send an acknowledgement for every data packet 🍓. For the revised TCP congestion control mechanisms, [RFC2581bis] currently specifies that the delayed acknowledgement algorithm should be used with TCP. However, [RFC2581bis] recommends increasing the congestion window during congestion avoidance by one segment per RTT even with delayed acknowledgements, which is consistent with a TCP throughput equation with b = 1. On an experimental basis, [RFC2581bis] also allows congestion window increases during slow-start that are consistent with a TCP throughput equation with b = 1. So using b = 1 is consistent with [RFC2581bis] ✨. The use of b = 1 is RECOMMENDED 👑.
With t_RTO=4*R and b=1, the equation for X_Bps, the TCP sending rate in bytes per second, simplifies to:
s
X_Bps = -----------------------------------------------
R * (sqrt(2*p/3) + 12*sqrt(3*p/8)*p*(1+32*p^2))In the future, updates to this document could specify different TCP equations to substitute for this one. The requirement is that the throughput equation be a reasonable approximation of the sending rate of TCP for conformant TCP congestion control 🌈.
The throughput equation can also be written in terms of X_pps, the sending rate in packets per second, with:
X_pps = X_Bps / s .The parameters s (segment size), p (loss event rate), and R (RTT) need to be measured or calculated by a TFRC implementation 🎀. The measurement of s is specified in Section 4.1, the measurement of R is specified in Section 4.3, and the measurement of p is specified in Section 5. In the rest of this document, data rates are measured in bytes per second unless otherwise specified 💜.
3.2. Packet Contents
Before we get into the sender and receiver functionality, let's look at what's inside the data packets the sender sends and the feedback packets the receiver sends back 🫶. TFRC gets used alongside a transport protocol, so we don't specify packet formats here, because those depend on the details of the transport protocol being used 🙈.
3.2.1. Data Packets
Each data packet sent by the data sender contains the following information:
- A sequence number. This number MUST be incremented by one for each data packet transmitted. The field must be large enough that it does not wrap around and cause two different packets with the same sequence number to be in the receiver's recent packet history at the same time 🔥.
- A timestamp indicating when the packet is sent. We denote by
ts_ithe timestamp of the packet with sequence number i. The resolution of the timestamp SHOULD typically be measured in milliseconds ⏱️.
This timestamp is how the receiver figures out which losses belong to the same loss event. The receiver also echoes the timestamp so the sender can estimate the round-trip time, for senders that don't save timestamps of transmitted data packets ✨.
Ngl, there's an alternative: a "timestamp" that increments every quarter of a round-trip time MAY be used instead of one incremented in milliseconds 💖. It can be used for deciding when losses belong to the same loss event, as long as the protocol is one where both sender and receiver understand this, and the sender saves the timestamps of transmitted data packets 🧚♀️.
- The sender's current estimate of the round-trip time. The estimate reported in packet i is denoted by
R_i. The receiver uses this round-trip time estimate, along with the timestamp, to determine when multiple losses belong to the same loss event. The receiver also uses it to determine the interval for calculating the receive rate, and to decide when to send feedback packets 🦋.
If the sender sends a coarse-grained "timestamp" that increments every quarter of a round-trip time, as discussed above, then the sender is not required to send its current estimate of the round-trip time 🌸.
3.2.2. Feedback Packets
Each feedback packet sent by the data receiver contains the following information:
- The timestamp of the last data packet received. We denote this by
t_recvdata. If the last packet received at the receiver has sequence number i, thent_recvdata = ts_i. The sender uses this timestamp to estimate the round-trip time, and it's only needed if the sender does not save the timestamps of transmitted data packets 💅. - The amount of time elapsed between the receipt of the last data packet at the receiver and the generation of this feedback report. We denote this by
t_delay. - The rate at which the receiver estimates that data was received in the previous round-trip time. We denote this by
X_recv. - The receiver's current estimate of the loss event rate p.
4. Data Sender Protocol ✨
The data sender sends a stream of data packets to the data receiver at a controlled rate 🚀. When a feedback packet arrives from the data receiver, the data sender changes its sending rate based on the info in the feedback report. If the sender doesn't receive a feedback report for four round-trip times, then the sender cuts its sending rate in half 😭. This is done with a timer called the nofeedback timer ⏳.
Here are the sender-side protocol steps:
- Measurement of the mean segment size being sent.
- Sender initialization.
- The sender behavior when a feedback packet is received.
- The sender behavior when the nofeedback timer expires.
- Oscillation prevention (optional).
- Scheduling of packet transmission and allowed burstiness.
4.1. Measuring the Segment Size
The TFRC sender uses the segment size, s, in the throughput equation, in setting the maximum receive rate, in setting the minimum and initial sending rates, and in setting the nofeedback timer 🎀. The TFRC receiver MAY use the average segment size, s, when initializing the loss history after the first loss event. As specified in Section 6.3.1, if the TFRC receiver doesn't know the segment size, s, used by the sender, the TFRC receiver MAY instead use the arrival rate in packets per second when initializing the loss history 🧁.
The segment size is normally known to an application. That might not be true in two cases:
- The segment size naturally varies depending on the data. In this case, even though the segment size varies, that variation isn't coupled to the transmit rate. The TFRC sender can either compute the average segment size or use the maximum segment size for s 💫.
- The application needs to change the segment size rather than the number of segments per second to perform congestion control. This would normally be the case with packet audio applications, where a fixed interval of time needs to be represented by each packet 🎶. Those applications need a completely different way of measuring parameters 💀.
For the first class of applications, where the segment size varies depending on the data, the sender SHOULD estimate the segment size, s, as the average segment size over the last four loss intervals. The sender MAY estimate the average segment size over longer time intervals, if it wants to 🌷.
The second class of applications is discussed separately in a separate document on TFRC-SP [RFC4828] 🌈. For the rest of this section, we assume the sender can estimate the segment size and that congestion control is done by adjusting the number of packets sent per second 💜.
4.2. Sender Initialization
The initial values for X (the allowed sending rate in bytes per second) and tld (the Time Last Doubled during slow-start, in seconds) are undefined until they're set as described below 🌸. If the sender is ready to send data but doesn't have a round-trip sample yet, the value of X is set to s bytes per second, for segment size s. The nofeedback timer is set to expire after two seconds, and tld is set to 0 (or to -1, either one is okay) ⏰. When the first round-trip time measurement comes in (e.g., after the first feedback packet, the SYN exchange from connection setup, or from a previous connection [RFC2140]), tld is set to the current time, and the allowed transmit rate, X, is set to the initial_rate, specified as W_init/R, for W_init based on [RFC3390] 🦋:
initial_rate = W_init/R; W_init = min(4*MSS, max(2*MSS, 4380)).When computing W_init, instead of using Maximum Segment Size (MSS), the TFRC sender SHOULD use the maximum segment size to be used for the initial round-trip time of data, if the TFRC sender knows that when X is initialized 🧚.
For responding to the initial feedback packet, this replaces step (4) of Section 4.3 below 🫶.
Appendix B explains why the initial value of TFRC's nofeedback timer is set to two seconds, instead of the recommended initial value of three seconds for TCP's retransmit timer from [RFC2988] 🍓.
4.3. Sender Behavior When a Feedback Packet Is Received ✨
okay bestie, this is where the sender gets its feedback and decides what to do with it 💅 The sender knows its current allowed sending rate, X, and keeps a running estimate of the current round-trip time R. It also keeps X_recv_set, a small set of recent X_recv values (usually just two) 🦋
Initialization 🌸
X_recv_set starts out holding one item, with the value Infinity. (This part is an implementation-specific choice: X_recv_set MAY be initialized to a large number instead of Infinity, e.g., the largest integer that is easily representable 🧚) When a feedback packet arrives at the sender at time t_now, which is the current time in seconds, the following actions MUST be performed 🫶
1) Calculate a new round-trip sample: 💖
R_sample = (t_now - t_recvdata) - t_delay.As described in Section 3.2.2, t_delay is the elapsed time at the receiver. ⏱️
2) Update the round-trip time estimate: ✨
If no feedback has been received before {
R = R_sample;
} Else {
R = q*R + (1-q)*R_sample;
}TFRC isn't picky about the exact value of the filter constant q, but a default value of 0.9 is RECOMMENDED 🎀
3) Update the timeout interval: 💜
RTO = max(4*R, 2*s/X)4) Update the allowed sending rate as follows. This step uses the variables t_mbi and recv_limit 👑
t_mbi: the maximum backoff interval of 64 seconds.recv_limit: the limit on the sending rate computed fromX_recv_set.
It also calls the procedures Maximize X_recv_set() and Update X_recv_set(), which I explain further down, so hang tight 🙈
Here's the procedure for updating the allowed sending rate:
If (the entire interval covered by the feedback packet
was a data-limited interval) {
If (the feedback packet reports a new loss event or an
increase in the loss event rate p) {
Halve entries in X_recv_set;
X_recv = 0.85 * X_recv;
Maximize X_recv_set();
recv_limit = max (X_recv_set);
} Else {
Maximize X_recv_set();
recv_limit = 2 * max (X_recv_set);
}
} Else { // typical behavior
Update X_recv_set();
recv_limit = 2 * max (X_recv_set);
}
If (p > 0) { // congestion avoidance phase
Calculate X_Bps using the TCP throughput equation.
X = max(min(X_Bps, recv_limit), s/t_mbi);
} Else if (t_now - tld >= R) {
// initial slow-start
X = max(min(2*X, recv_limit), initial_rate);
tld = t_now;
}5) If oscillation reduction is used, work out the instantaneous transmit rate, X_inst, following Section 4.5. 🌈
6) Reset the nofeedback timer to expire after RTO seconds. ⏰
Maximizing X_recv_set 💅
This procedure keeps one value: the largest value from X_recv_set plus the new X_recv. Here's how it goes:
Maximize X_recv_set():
Add X_recv to X_recv_set;
Delete initial value Infinity from X_recv_set,
if it is still a member.
Set the timestamp of the largest item to the current time;
Delete all other items.Updating X_recv_set 🦋
This one keeps a set of X_recv values with timestamps from the two most recent round-trip times. It goes like this:
Update X_recv_set():
Add X_recv to X_recv_set;
Delete from X_recv_set values older than
two round-trip times.Data-limited intervals 🍓
Okay, so a sender counts as data-limited any time it isn't sending as much as it's allowed to send 🙌 An interval is a 'data-limited interval' only if the sender was data-limited over the entire interval. Section 8.2.1 talks about how an implementation can figure out whether an interval was data-limited. The term 'data-limited interval' shows up in the first "if" condition in step (4), and that condition stops a sender from having to cut its sending rate just because a feedback packet reports the receive rate from a data-limited period. Iconic, right? 👑
Picture a sender that's sending at its full allowed rate, except it sends packets in pairs instead of sending each one as soon as it can. That sender is data-limited part of the time, since it isn't always sending as soon as it could. But an interval covering at least two data packets from this sender doesn't count as a data-limited interval, because the sender wasn't data-limited over the entire interval 🙈
If the feedback packet reports a receive rate X_recv of zero (which is the case for the first feedback packet), the sender does not consider the entire interval covered by that feedback packet to be a data-limited interval 🫶
X_recv_set and the first feedback packet ✨
X_recv_set starts with a single item, Infinity, so recv_limit is set to Infinity for the first two round-trip times of the connection. That means the sending rate isn't limited by the receive rate during that stretch. This dodges the problem of the sending rate getting capped by the X_recv from the very first feedback packet 🧚
The interval covered by a feedback packet 🌸
So how does the sender know what period a feedback packet covers? Section 8.2 goes into more detail. Usually the receiver sends one feedback packet per round-trip time, so the sender can typically work out exactly which period the current feedback packet covers by looking at the previous one 💜 But when the previous feedback packet got lost, or when the receiver sends a feedback packet early because it spotted a lost or ECN-marked packet, the sender has to estimate the interval instead. As specified in Section 6.2, each feedback packet the receiver sends covers a round-trip time, for the round-trip time estimate R_m the receiver maintained R_m seconds before sending the feedback packet 🦋
The response to a loss during a data-limited interval 🌈
In TFRC, once the initial slow-start is over, the sender always updates the calculated transmit rate, X_Bps, after a feedback packet arrives, and the allowed sending rate, X, is always limited by X_Bps 💖 During a data-limited interval, though, the actual sending rate is usually below X_Bps, so the sending rate is still limited by recv_limit, which comes from X_recv_set. If the sender is data-limited (maybe with a sending rate that bounces around from one round-trip time to the next) and is also seeing losses, the sender decreases the entry in X_recv_set so the allowed sending rate goes down 😭
The sender can detect a loss event during a data-limited period either from explicit feedback from the receiver or from a reported increase in the loss event rate. When a feedback packet reports a loss event like that during a data-limited interval, the sender limits how much the sending rate is allowed to increase during that interval 🔥
The initial slow-start phase 🚀
When p=0, the sender hasn't heard about any loss events yet, so it's in the initial slow-start phase. During this phase, the sender can roughly double its sending rate each round-trip time until a loss happens. The initial_rate term in step (4) sets a minimum allowed sending rate during slow-start, equal to the initial allowed sending rate 🌸
Note that if the sender is data-limited during slow-start, or the connection is limited by the path bandwidth, the sender can't necessarily double its sending rate each round-trip time. Instead, its sending rate is capped at twice the past receive rate, or at initial_rate, whichever is larger. This is similar to TCP, where the sending rate is limited by the rate of incoming acknowledgement packets as well as by the congestion window. So in TCP's slow-start, for the most aggressive case where the TCP receiver acknowledges every data packet, the TCP sender's rate is capped at twice the rate of those incoming acknowledgement packets 🍓
The minimum allowed sending rate 🧚
The term s/t_mbi makes sure that when p > 0, the sender is allowed to send at least one packet every 64 seconds. That's the floor, bestie, so it never goes totally silent 🫶
4.4. Expiration of Nofeedback Timer ⏰
okay so this section is about what the sender does when its nofeedback timer goes off 😭 it can expire because of an idle period, or because data or feedback packets got dropped in the network. bestie, it's giving "something went quiet, better check on it" energy 🫶
This section uses the variable recover_rate. If the TFRC sender has been idle ever since the nofeedback timer was set, the allowed sending rate isn't allowed to drop below the recover_rate 🌸 For this document, recover_rate is set to the initial_rate (specified in Section 4.2). Future updates to this specification might explore other possible values for recover_rate ✨
If the nofeedback timer expires, the sender MUST perform the following actions:
- Cut the allowed sending rate in half. ✂️
If the nofeedback timer expires when the sender has had at least one RTT measurement, the allowed sending rate is reduced by modifying X_recv_set as described in the pseudocode below (including item (2)). In the general case, the sending rate is limited to at most twice X_recv. Modifying X_recv_set limits the sending rate, but still lets the sender slow-start, doubling its sending rate each RTT, if feedback messages resume reporting no losses 📈
If the sender has been idle since this nofeedback timer was set and X_recv is less than the recover_rate, then the allowed sending rate is not halved, and X_recv_set is not changed. This makes sure the allowed sending rate doesn't get cut to less than half the recover_rate just because of an idle period 🛡️
In the general case, the allowed sending rate is halved in response to the expiration of the nofeedback timer. The details, in the pseudocode below, depend on whether the sender is in slow-start, is in congestion avoidance limited by X_recv, or is in congestion avoidance limited by the throughput equation 🧠
X_recv = max (X_recv_set);
If (sender does not have an RTT sample,
has not received any feedback from receiver,
and has not been idle ever since the nofeedback timer
was set) {
// We do not have X_Bps or recover_rate yet.
// Halve the allowed sending rate.
X = max(X/2, s/t_mbi);
} Else if (((p>0 && X_recv < recover_rate) or
(p==0 && X < 2 * recover_rate)), and
sender has been idle ever
since nofeedback timer was set) {
// Don't halve the allowed sending rate.
Do nothing;
} Else if (p==0) {
// We do not have X_Bps yet.
// Halve the allowed sending rate.
X = max(X/2, s/t_mbi);
} Else if (X_Bps > 2*X_recv)) {
// 2*X_recv was already limiting the sending rate.
// Halve the allowed sending rate.
Update_Limits(X_recv;)
} Else {
// The sending rate was limited by X_Bps, not by X_recv.
// Halve the allowed sending rate.
Update_Limits(X_Bps/2);
}The term s/t_mbi limits the backoff to one packet every 64 seconds. ⏳
The procedure Update_Limits() uses the variable timer_limit for the limit on the sending rate computed from the expiration of the nofeedback timer, as follows:
Update_Limits(timer_limit):
If (timer_limit < s/t_mbi)
timer_limit = s/t_mbi;
Replace X_recv_set contents with the single item
timer_limit/2;
Recalculate X as in step (4) of Section 4.3;- Restart the nofeedback timer to expire after
max(4*R, 2*s/X)seconds. 🔁
If the sender has been data-limited but not idle since the nofeedback timer was set, it's possible that the nofeedback timer expired because data or feedback packets were dropped in the network. In this case, the nofeedback timer is the backup mechanism for the sender to detect these losses, kind of like the retransmit timer in TCP 🕵️♀️
Note that when the sender stops sending data for a period of time, the receiver will stop sending feedback. When the sender's nofeedback timer expires, the sender could use the procedure above to limit the sending rate. If the sender starts sending again, X_recv_set will be used to limit the transmit rate, and slow-start behavior will happen until the transmit rate reaches X_Bps 🌱
The TFRC sender's reduction of the allowed sending rate after the nofeedback timer expires is similar to TCP's reduction of the congestion window, cwnd, after each RTO seconds of an idle period, for TCP with Congestion Window Validation [RFC2861]. the vibes match, honestly 💅
4.5. Reducing Oscillations 🌊
To reduce oscillations in queueing delay and sending rate in environments with a low degree of statistical multiplexing at the congested link, it is RECOMMENDED that the sender reduce the transmit rate as the queueing delay (and hence RTT) increases. To do this, the sender maintains R_sqmean, a long-term estimate of the square root of the RTT, and modifies its sending rate depending on how the square root of R_sample, the most recent sample of the RTT, differs from the long-term estimate. The long-term estimate R_sqmean is set as follows:
If no feedback has been received before {
R_sqmean = sqrt(R_sample);
} Else {
R_sqmean = q2*R_sqmean + (1-q2)*sqrt(R_sample);
}Thus, R_sqmean gives the exponentially weighted moving average of the square root of the RTT samples. The constant q2 should be set similarly to q, the constant used in the round-trip time estimate R. A value of 0.9 as the default for q2 is RECOMMENDED. 🎀
When sqrt(R_sample) is greater than R_sqmean, then the current round-trip time is greater than the long-term average, which implies queueing delay is probably increasing. In this case, the transmit rate is decreased to minimize oscillations in queueing delay 📉
The sender obtains the base allowed transmit rate, X, as described in step (4) of Section 4.3 above. It then calculates a modified instantaneous transmit rate X_inst, as follows:
X_inst = X * R_sqmean / sqrt(R_sample);
If (X_inst < s/t_mbi)
X_inst = s/t_mbi;Because we're using square roots, there's generally only a moderate difference between the instantaneous transmit rate X_inst and the allowed transmit rate X. For example, in a somewhat extreme case where the current RTT sample R_sample is twice as large as the long-term average, then sqrt(R_sample) will be roughly 1.44 times R_sqmean, and the allowed transmit rate will be reduced by a factor of roughly 0.7 🧮
We note that this modification for reducing oscillatory behavior is not always needed, especially if the degree of statistical multiplexing in the network is high. We also note that the modification for reducing oscillatory behavior could cause problems for connections where the round-trip time is not strongly correlated with the queueing delay (e.g., in some wireless links, over paths with frequent routing changes, etc.). However, this modification SHOULD be implemented because it makes TFRC behave better in some environments with a low level of statistical multiplexing. The performance of this modification is illustrated in Section 3.1.3 of [FHPW00]. If it is not implemented, implementations SHOULD use a very low value of the weight q for the average round-trip time. 🫶
4.6. Scheduling of Packet Transmissions 🗓️
As TFRC is rate-based, and as operating systems typically can't schedule events precisely, it's necessary to be opportunistic about sending data packets so that the correct average rate is maintained despite the coarse-grain or irregular scheduling of the operating system. To help keep the correct average sending rate, the TFRC sender MAY send some packets before their nominal send time. 🧚
In addition, the scheduling of packet transmissions controls how bursty senders are allowed to be after an idle or data-limited period. The TFRC sender MAY accumulate sending 'credits' for past unused send times; this lets the TFRC sender send a burst of data after an idle or data-limited period. To compare with TCP, TCP is allowed to send up to a round-trip time's worth of packets in a single burst, but never more. As examples, packet bursts can be sent by TCP when an ACK arrives acknowledging a window of data, or when a data-limited sender suddenly has a window of data to send after a delay of nearly a round-trip time 💨
To limit burstiness, a TFRC implementation MUST prevent bursts of arbitrary size. This limit MUST be less than or equal to one round-trip time's worth of packets. A TFRC implementation MAY limit bursts to less than a round-trip time's worth of packets. In addition, a TFRC implementation MAY use rate-based pacing to smooth bursts. 🌈
As an implementation-specific example, a sending loop could calculate the correct inter-packet interval, t_ipi, as follows:
t_ipi = s/X_inst;Let t_now be the current time and i be a natural number, i = 0, 1, ..., with t_i the nominal send time for the i-th packet. Then, the nominal send time t_(i+1) would derive recursively as:
t_0 = t_now,
t_(i+1) = t_i + t_ipi.For TFRC senders allowed to accumulate sending credits for unused send time over the last T seconds, the sender would be allowed to use unused nominal send times t_j for t_j < now - T, with T set to the round-trip time. ⏱️
5. Calculation of the Loss Event Rate (p) 📊
Obtaining an accurate and stable measurement of the loss event rate is of primary importance for TFRC. Loss rate measurement is performed at the receiver, based on the detection of lost or marked packets from the sequence numbers of arriving packets. We'll describe this process before describing the rest of the receiver protocol. If the receiver hasn't detected a lost or marked packet yet, then the receiver doesn't calculate the loss event rate, and instead reports a loss event rate of zero. this part is literally the heart of TFRC 💖
5.1. Detection of Lost or Marked Packets ✨
okay so TFRC assumes every packet has a sequence number that goes up by one for each packet sent 🔢. For this spec, it is REQUIRED that if a lost packet gets retransmitted, the retransmission gets a brand new sequence number, the latest one in the transmission sequence, and not the same number as the packet that was lost 💅. If a transport protocol has to retransmit with the original sequence number, then the transport protocol designer must figure out how to tell delayed packets apart from retransmitted ones, and how to detect lost retransmissions 🙈.
The receiver keeps a data structure that tracks which packets have arrived and which are missing 🧚. For this spec, we pretend that structure is just a list of arrived packets, each stored with the receiver timestamp of when it showed up. In real life it'll usually be stored in something more compact, but how that works is up to the implementation 🫶.
A packet counts as lost once at least NDUPACK packets with a higher sequence number than the lost one have arrived, and NDUPACK is set to 3 🌸. This is the same rule TCP uses, and it's there to make TFRC more robust when packets get reordered 🔀. The twist is that, unlike TCP, if a packet shows up late (after NDUPACK later packets already arrived), it can fill the hole in TFRC's reception record, and the receiver can recalculate the loss event rate. Future versions of TFRC might make the NDUPACK requirement adaptive based on how much reordering is actually happening, but that's not part of the current spec 🦋.
For an ECN-capable connection, a marked packet is detected as a congestion event the moment it arrives, so there's no waiting around for later packets 🚨.
If an ECN-marked packet is preceded by a possibly-lost packet, then the first detected congestion event begins with the lost packet 🫠. Say the receiver gets a data packet with sequence number n-1, then an unmarked data packet with sequence number n+1, then a marked data packet with sequence number n+2. The receiver detects a congestion event when it gets the marked packet n+2, and the first congestion event detected begins with the lost packet n 👀. The guidelines in Section 5.2 below decide whether the lost and marked packets belong to the same loss event or to separate loss events 💜.
5.2. Translation from Loss History to Loss Events 🌈
okay this part is so important bestie 😭. TFRC needs the loss fraction to be robust to several packets lost or marked in a row within the same loss event. It's kinda like TCP, which (typically) only halves its congestion window once during any single RTT 👑. So the receiver has to map the packet loss history into a loss event record, where a loss event is one or more packets lost or marked in one RTT ✨. To do that mapping, the receiver needs to know which RTT to use, and the sender supplies that periodically, usually as control info piggy-backed onto a data packet 🎀. TFRC isn't picky about how the RTT measurement sent to the receiver gets made, but it is RECOMMENDED to use the sender's calculated RTT, R, (see Section 4.3) for this purpose 💖.
To figure out whether a lost or marked packet should start a new loss event or count as part of an existing one, we compare the sequence numbers and timestamps of the packets that arrived at the receiver 🔍. For a marked packet, S_new, its reception time T_new can be noted directly. For a lost packet, we can interpolate to infer its nominal "arrival time". Here's the setup:
S_lossis the sequence number of a lost packet.S_beforeis the sequence number of the last packet to arrive, before any packet arrivals with a sequence number aboveS_loss, with a sequence number belowS_loss.S_afteris the sequence number of the first packet to arrive afterS_beforewith a sequence number aboveS_loss.S_maxis the largest sequence number.
So the ordering is totally locked in:
S_before < S_loss < S_after <= S_maxT_lossis the nominal estimated arrival time for the lost packet.T_beforeis the reception time ofS_before.T_afteris the reception time ofS_after.
And yep, T_before < T_after 🙌.
For a lost packet, S_loss, we can interpolate its nominal "arrival time" at the receiver using the arrival times of S_before and S_after. So:
T_loss = T_before + ( (T_after - T_before)
* (S_loss - S_before)/(S_after - S_before) );Now, sequence numbers can wrap around 🔁, so let S_MAX = 2^b, where b is the bit-length of sequence numbers in a given implementation. With wrapping in mind, we interpolate T_loss like this:
T_loss = T_before + (T_after - T_before)
* Dist(S_loss, S_before)/Dist(S_after, S_before)where
Dist(S_A, S_B) = (S_A + S_MAX - S_B) % S_MAXIf the lost packet S_old was determined to have started the previous loss event, and we've just figured out that S_new got lost too, then we interpolate the nominal arrival times of S_old and S_new, called T_old and T_new, respectively 🕵️♀️.
If T_old + R >= T_new, then S_new is part of the existing loss event 🫶. Otherwise, S_new is the first packet in a brand new loss event 💥.
5.3. The Size of a Loss Interval 🎀
After the first loss event gets detected, the receiver chops the sequence space up into loss intervals 🍰. If a loss interval, A, is determined to have started with packet sequence number S_A, and the next loss interval, B, started with packet sequence number S_B, then the number of packets in loss interval A is given by (S_B - S_A). So loss interval A contains every packet the sender transmitted starting with the first one in loss interval A, and ending right before the first packet in loss interval B 📦.
The current loss interval, I_0, is defined as the loss interval containing the most recent loss event 🔥. If that loss event started with packet sequence number S_A, and S_C is the highest received sequence number so far, then the size of I_0 is S_C - S_A + 1. For example, if the current loss interval is just a single ECN-marked packet, then S_A == S_C, and the size of the loss interval is one 🦋.
5.4. Average Loss Interval 💜
To calculate the loss event rate, p, we first calculate the average loss interval 🧮. This happens using a filter that weights the n most recent loss event intervals so the measured loss event rate changes smoothly 🌊. If the receiver hasn't yet seen a lost or marked packet, then the receiver does not calculate the average loss interval 🙅♀️.
Weights w_0 to w_(n-1) are calculated as:
If (i < n/2) {
w_i = 1;
} Else {
w_i = 2 * (n-i)/(n+2);
}So if n=8, the values of w_0 to w_7 are:
1.0, 1.0, 1.0, 1.0, 0.8, 0.6, 0.4, 0.2The value of n (how many loss intervals go into the loss event rate) decides how fast TFRC responds to changes in congestion levels 🏎️. It is RECOMMENDED to set n to 8 ✨. TFRC SHOULD NOT use values of n greater than 8 for traffic that might compete in the global Internet with TCP 🚫. At the very least, safe operation with values of n greater than 8 would need a slight change to TFRC's mechanisms, to include a more severe response to two or more round-trip times with heavy packet loss 😬.
When we calculate the average loss interval, we have to decide whether to include the current loss interval 🤔. We only include the current loss interval if it's big enough to increase the average loss interval 📈.
Let the most recent loss intervals be I_0 to I_k, where I_0 is the current loss interval. If there have been at least n loss intervals, then k is set to n; otherwise, k is the maximum number of loss intervals seen so far. We calculate the average loss interval I_mean like this:
I_tot0 = 0;
I_tot1 = 0;
W_tot = 0;
for (i = 0 to k-1) {
I_tot0 = I_tot0 + (I_i * w_i);
W_tot = W_tot + w_i;
}
for (i = 1 to k) {
I_tot1 = I_tot1 + (I_i * w_(i-1));
}
I_tot = max(I_tot0, I_tot1);
I_mean = I_tot/W_tot;And finally, the loss event rate, p, is simply:
p = 1 / I_mean;5.5. History Discounting 💅✨
okay so remember from Section 5.4? once you've got at least n loss intervals, the most recent one only gets 1/(0.75*n) of the total weight, no matter how huge it is 😭. this section is about an OPTIONAL history discounting mechanism (go read [FHPW00a] and [W00] if you want the deep dive 🤓). it lets the TFRC receiver shift the weights so the newest interval gets more of the spotlight, but only when it's more than twice as big as the computed average loss interval. main character energy 👑
to do the discounting, we give each loss interval L_i (for i > 0) a discount factor DF_i, which is a floating point number. the discount array keeps the running history of discounting for each loss interval. at the very start, all the DF_i values get set to 1:
for (i = 0 to n) {
DF_i = 1;
}there's also a general discount factor, DF, which is a floating point number too and also starts at 1 🌸. first we'll look at how these discount factors get used when calculating the average loss interval, and then later in this section we'll cover how they change over time.
remember Section 5.4? the average loss interval is calculated from the n previous loss intervals I_1, ..., I_n plus the current loss interval I_0. using the discount factors is a simple tweak to that procedure from Section 5.4, and it looks like this 🦋:
I_tot0 = I_0 * w_0;
I_tot1 = 0;
W_tot0 = w_0;
W_tot1 = 0;
for (i = 1 to n-1) {
I_tot0 = I_tot0 + (I_i * w_i * DF_i * DF);
W_tot0 = W_tot0 + w_i * DF_i * DF;
}
for (i = 1 to n) {
I_tot1 = I_tot1 + (I_i * w_(i-1) * DF_i);
W_tot1 = W_tot1 + w_(i-1) * DF_i;
}
p = min(W_tot0/I_tot0, W_tot1/I_tot1);the general discounting factor, DF, gets updated on every single packet arrival 🔥. first the receiver works out the weighted average I_mean of the loss intervals I_1, ..., I_n:
I_tot = 0;
W_tot = 0;
for (i = 1 to n) {
W_tot = W_tot + w_(i-1) * DF_i;
I_tot = I_tot + (I_i * w_(i-1) * DF_i);
}
I_mean = I_tot / W_tot;then that I_mean gets compared to I_0, the size of the current loss interval. if I_0 is greater than twice I_mean, the new loss interval is way bigger than the old ones, so DF gets updated to lower the relative weight on the older intervals 🍓:
if (I_0 > 2 * I_mean) {
DF = 2 * I_mean/I_0;
if (DF < THRESHOLD) {
DF = THRESHOLD;
}
} else {
DF = 1;
}a nonzero THRESHOLD makes sure older loss intervals from an earlier stretch of high congestion don't get discounted all the way to nothing 🫶. we recommend a THRESHOLD of 0.25. and yep, with each new packet that arrives, I_0 keeps growing, so DF keeps getting updated too ✨.
when a new loss event happens, the current interval slides from I_0 into I_1, every loss interval I_i moves up to I_(i+1), and the loss interval I_n gets forgotten 💀. the previous DF has to be baked into the discount array first. since DF_i carries the discount factor for loss interval I_i, the DF_i array has to shift along with everything else. here's how that goes:
for (i = 1 to n) {
DF_i = DF * DF_i;
}
for (i = n-1 to 0 step -1) {
DF_(i+1) = DF_i;
}
I_0 = 1;
DF_0 = 1;
DF = 1;and that's the whole optional history discounting mechanism 🎀. we really want to emphasize that this is an OPTIONAL mechanism whose only job is to let TFRC respond a bit faster to a sudden lack of congestion, which shows up as a long current loss interval 🌈.
6. Data Receiver Protocol 🦋
the receiver sends feedback messages to the sender on a regular schedule 💌. feedback packets SHOULD normally be sent at least once per RTT, unless the sender is sending at less than one packet per RTT. in that case, a feedback packet SHOULD be sent for every data packet received. a feedback packet SHOULD also be sent whenever a new loss event is detected, without waiting for the end of an RTT, and whenever an out-of-order data packet shows up and removes a loss event from the history 😭.
if the sender is transmitting at a high rate (lots of packets per RTT), sending periodic feedback more than once per RTT can help. it means faster responses to changing RTT measurements and more resilience when feedback packets get lost 🧚.
if the receiver were sending k feedback packets per RTT, for k>1, step (4) of Section 6.2 would change so the feedback timer expires after R_m/k seconds. each feedback packet would still report the receiver rate over the last RTT, though, not over a fraction of one 💅. this document doesn't spell out the changes a receiver sending more than one feedback packet per RTT would need. we do note that there's not much gain from sending a huge number of feedback messages per RTT 🙈.
6.1. Receiver Behavior When a Data Packet Is Received
when a data packet is received, the receiver goes through these steps:
- Add the packet to the packet history.
- Check if done: if the new packet results in detecting a new loss event, or if no feedback packet was sent when the feedback timer last expired, go to step 3. otherwise there's nothing to do (unless the optimization in the next paragraph is used), so just exit the procedure 🫠.
an OPTIONAL optimization might check whether the packet's arrival filled a hole in the packet history, which merged two loss intervals into one. if that happened, the receiver might also send feedback right away. the effects of this optimization are normally expected to be small 🌸.
- Calculate p: let the previous value of p be
p_prev. then calculate the new value ofpas described in Section 5. - Expire feedback timer: if
p > p_prev, make the feedback timer expire and do the actions described in Section 6.2.
if p <= p_prev and no feedback packet was sent when the feedback timer last expired, make the feedback timer expire and do the actions described in Section 6.2. if p <= p_prev and a feedback packet was sent when the feedback timer last expired, nothing needs to happen 🙌.
6.2. Expiration of Feedback Timer
when the feedback timer at the receiver expires, what happens next depends on whether data packets have come in since the last feedback was sent 🎀.
for the m-th expiration of the feedback timer, let the maximum sequence number of a packet the receiver has seen so far be S_m, and let the RTT measurement included in packet S_m be R_m. as described in Section 3.2.1, R_m is the sender's most recent estimate of the round-trip time, as reported in data packets. if data packets have been received since the previous feedback was sent, the receiver does the following steps:
- Calculate the average loss event rate using the algorithm described in Section 5.
- Calculate the measured receive rate,
X_recv, based on the packets received within the previousR_(m-1)seconds. this happens whether the feedback timer expired at its normal time or expired early because of a new lost or marked packet (that's step (3) in Section 6.1) 🔥.
in the typical case, where the receiver sends only one feedback packet per round-trip time and the timer didn't expire early due to a new lost packet, the time since the feedback timer last expired would be R_(m-1) seconds.
we note that when the feedback timer expires early due to a new lost or marked packet, the time since it last expired is likely to be smaller than R_(m-1) seconds.
for ease of implementation, if the time since the feedback timer last expired isn't R_(m-1) seconds, the receive rate MAY be calculated over a longer interval. that interval goes back to the most recent feedback timer expiration that was at least R_(m-1) seconds ago 🧠.
- Prepare and send a feedback packet containing the information described in Section 3.2.2 💌.
- Restart the feedback timer to expire after
R_mseconds.
note that rule 2) above gives a minimum value for the measured receive rate X_recv of one packet per round-trip time. if the sender is limited to a sending rate of less than one packet per round-trip time, that's because of the loss event rate, not a limit imposed by the measured receive rate at the receiver 🦋.
if no data packets have been received since the last feedback was sent, then no feedback packet gets sent, and the feedback timer is restarted to expire after R_m seconds 💤.
6.3. Receiver Initialization
the receiver gets initialized by the first data packet that arrives. let the sequence number of this packet be i 🌱.
when the first packet is received:
- Set
p = 0. - Set
X_recv = 0. - Prepare and send a feedback packet.
- Set the feedback timer to expire after
R_iseconds.
if the first data packet doesn't contain an estimate R_i of the round-trip time, then the receiver sends a feedback packet for every arriving data packet until a data packet shows up with an estimate of the round-trip time 🙈.
if the sender is using a coarse-grained timestamp that increments every quarter of a round-trip time, a feedback timer isn't needed. instead, the following procedure from RFC 4342 decides when to send feedback messages ⏰.
- Whenever the receiver sends a feedback message, it sets a local variable
last_counterto the greatest received value of the window counter since the last feedback message was sent, if any data packets have been received since then. - If the receiver gets a data packet with a window counter value greater than or equal to
last_counter + 4, then the receiver sends a new feedback packet. ("Greater" and "greatest" are measured in circular window counter space 🌀.)
6.3.1. Initializing the Loss History after the First Loss Event ✨
okay bestie, this part is about how the loss history gets seeded after the very first loss event 🧚 This procedure MUST be used for initializing the loss history after the first loss event.
Counting the packets until the first loss won't work for computing the allowed sending rate, ngl 😭 The sending rate changes super fast during that stretch. TFRC assumes the correct data rate after the first loss is half the maximum sending rate from right before the loss. It approximates that target, X_target, by the maximum value of X_recv seen so far. (In slow-start, for a given round-trip time, the sender's sending rate is generally twice the receiver's receive rate for data sent over the previous round-trip time 💅)
After the first loss, the TFRC receiver doesn't just set the first loss interval to the number of packets sent until the first loss. Instead, it works out the loss interval that would be required to produce X_target and uses that synthetic loss interval to seed the loss history mechanism 🦋
TFRC does this by finding some value, p, for which the throughput equation in Section 3.1 gives a sending rate within 5% of X_target, given the round-trip time R. The first loss interval is then set to 1/p. If the receiver knows the segment size, s, used by the sender, then the receiver MAY use the throughput equation for X. Otherwise, the receiver MAY measure the receive rate in packets per second instead of bytes per second for this purpose, and use the throughput equation for X_pps. (The 5% tolerance is there simply because the throughput equation is hard to invert, and we want to keep the cost of calculating p numerically down 🙈)
Special care is needed when the first data packet is lost or marked, bestie 😭 When the first data packet is lost in TCP, the TCP sender retransmits the packet after the retransmit timer expires. If TCP's first data packet is ECN-marked, the TCP sender resets the retransmit timer, and sends a new data packet only when the retransmit timer expires [RFC3168] (Section 6.1.2). For TFRC, if the first data packet is lost or ECN-marked, then the first loss interval consists of the null interval with no data packets. In this case, the loss interval length for this (null) loss interval SHOULD be set to give a similar sending rate to that of TCP, as specified in the paragraph below. It's giving TCP-inspired fairness, no cap 🌸
When the first TFRC loss interval is null (meaning the first data packet is lost or ECN-marked), TFRC wants to act like TCP. So the allowed sending rate should be 1 packet every two round-trip times, or equivalently, 0.5 packets per RTT 🎀 Thus, the TFRC receiver calculates the loss interval that would be required to produce the target rate X_target of 0.5/R packets per second, for the round-trip time R, and uses this synthetic loss interval for the first loss interval. The TFRC receiver uses 0.5/R packets per second as the minimum value for X_target when initializing the first loss interval 💜
We note that even though the TFRC receiver reports a synthetic loss interval after the first loss event, it still reports the measured receive rate X_recv, as specified in Section 6.2 above ✨
7. Sender-Based Variants 🦋
In a sender-based variant of TFRC, the receiver uses reliable delivery to send information about packet losses to the sender, and the sender computes the packet loss rate and the acceptable transmit rate.
The main advantage of a sender-based variant of TFRC is that the sender doesn't have to trust the receiver's calculation of the packet loss rate. However, with the requirement of reliable delivery of loss information from the receiver to the sender, a sender-based TFRC would have much tighter constraints on the transport protocol it's embedded in 😭
In contrast, the receiver-based variant of TFRC specified in this document is robust to the loss of feedback packets, so it doesn't require reliable delivery of feedback packets. It's also better suited for applications where it's desirable to offload work from the server to the client as much as possible 🫶
RFC 4340 and RFC 4342 together specify DCCP's CCID 3, which can be used as a sender-based variant of TFRC. In CCID 3, each feedback packet from the receiver contains a Loss Intervals option, reporting the lengths of the most recent loss intervals. Feedback packets may also include the Ack Vector option, allowing the sender to determine exactly which packets were dropped or marked and to check the information reported in the Loss Intervals options. The Ack Vector option can also include ECN Nonce Echoes, allowing the sender to verify the receiver's report of having received an unmarked data packet. The Ack Vector option lets the sender see for itself which data packets were lost or ECN-marked, determine loss intervals, and calculate the loss event rate. Section 9 of RFC 4342 discusses issues in the sender verifying information reported by the receiver 🔥
8. Implementation Issues 🛠️
This document has specified the TFRC congestion control mechanism, for use by applications and transport protocols. This section briefly mentions some of the implementation issues 💖
8.1. Computing the Throughput Equation 🧮
For t_RTO = 4*R and b = 1, the throughput equation in Section 3.1 can be expressed as follows:
s
X_Bps = --------
R * f(p)for
f(p) = sqrt(2*p/3) + (12*sqrt(3*p/8) * p * (1+32*p^2)).A table lookup could be used for the function f(p) 🍓
Many of the multiplications (e.g., q and 1-q for the round-trip time average, a factor of 4 for the timeout interval) are, or could be, by powers of two. That means they could be implemented as simple shift operations, which is so clean 👑
8.2. Sender Behavior When a Feedback Packet Is Received 📬
This section discusses implementation issues for sender behavior when a feedback packet is received, from Section 4.3.
8.2.1. Determining If an Interval Was a Data-Limited Interval 🔍
When a feedback packet is received, the sender has to determine if the entire interval covered by that feedback packet was a data-limited period. This section discusses one possible implementation for the sender to determine if the interval covered by a feedback packet was a data-limited period 💅
If the feedback packets all report the timestamp of the last data packet received, then let t_new be the timestamp reported by this feedback packet. Because all feedback packets cover an interval of at least a round-trip time, it's sufficient for the sender to determine if there was any time in the period (t_old, t_new] when the sender was not data-limited. Here R is the sender's estimate of the round-trip time, and t_old is set to t_new - R. (This procedure estimates the interval covered by the feedback packet, rather than computing it exactly. This seems fine to us 🫶)
The pseudocode for determining if the sender was data-limited over the entire interval covered in a feedback packet is given below. The variables NotLimited1 and NotLimited2 both represent times when the sender was not data-limited.
Initialization:
NotLimited1 = NotLimited2 = t_new = t_next = 0;
t_now = current time;
After sending a segment:
If (sender has sent all it is allowed to send) {
// Sender is not data-limited at this instant.
If NotLimited1 <= t_new
// Goal: NotLimited1 > t_new.
NotLimited1 = t_now;
Else if (NotLimited2 <= t_next)
// Goal: NotLimited2 > t_next.
NotLimited2 = t_now;
}
When a feedback packet is received, is this interval data-limited:
t_new = timestamp reported in feedback packet.
t_old = t_new - R. // local variable
t_next = t_now;
If ((t_old < NotLimited1 <= t_new) or
(t_old < NotLimited2 <= t_new))
This was not a data-limited interval;
Else
This was a data-limited interval.
If (NotLimited1 <= t_new && NotLimited2 > t_new)
NotLimited1 = NotLimited2;Transmission times refer to transmission of a segment or segments to the layer below 📦
Between feedback packets, (t_old, t_new] gives the transmission time interval estimated to be covered by the most recent feedback packet, and t_next gives a time at least a round-trip time greater than t_new. The next feedback packet can be expected to cover roughly the interval (t_new, t_next], unless the receiver sends the feedback packet early because it's reporting a new loss event. The goal is for NotLimited1 to save a non-data-limited time in (t_new, t_next], if there was one, and for NotLimited2 to save a non-data-limited time after t_next 🧠
When a feedback packet is received, if either NotLimited1 or NotLimited2 is in the time interval covered by the feedback packet, then the interval is not a data-limited interval. The sender was not data-limited at least once during that time interval. If neither NotLimited1 nor NotLimited2 is in the time interval covered by a feedback packet, then the sender is assumed to have been data-limited over that time interval 🌈
We note that this procedure is a heuristic. In some cases the sender might not determine correctly if it was data-limited over the entire interval covered by the feedback packet. This heuristic does not address the possible complications of reordering.
That seems acceptable to us. To improve its accuracy in identifying if the entire interval covered by a feedback packet was a data-limited interval, the sender could save more NotLimited times 🎯
In some implementations of TFRC, the sender sends coarse-grained timestamps that increment every quarter of a round-trip time. The feedback packet reports the greatest valid sequence number received so far, instead of reporting the timestamp of the last packet received. In this case, the sender can maintain per-packet state to determine t_new (the time that the acknowledged packet was sent). Or the sender can estimate t_new from its estimate of the round-trip time and the elapsed time t_delay reported by the feedback packet ⏱️
8.2.2. Maintaining X_recv_set 🧺
To reduce the complexity of maintaining X_recv_set, it's sufficient to limit X_recv_set to at most N=3 elements. In this case, the procedure Update X_recv_set() would be modified as follows:
Update X_recv_set():
Add X_recv to X_recv_set;
Delete from X_recv_set values older than
two round-trip times.
Keep only the most recent N values.Maintaining at most two elements in X_recv_set would be sufficient for the sender to save an old value of X_recv from before a data-limited period. It would also let the sender avoid being limited by the first feedback packet after an idle period (reporting a receive rate of one packet per round-trip time) 🥹 However, it is possible that maintaining at most two elements in X_recv_set wouldn't give quite as good performance as maintaining at most three elements. Maintaining three elements in X_recv_set would allow X_recv_set to contain X_recv values from two successive feedback packets, plus a more recent X_recv value from a loss event 💖
8.3. Sending Packets before Their Nominal Send Time ✨
okay so this part is about one possible scheduling trick for a sender on an operating system with a coarse timing granularity (remember Section 4.6 🕰️). Let t_gran be the scheduling timer granularity of the OS, and let t_ipi be the inter-packet interval from Section 4.6. If the OS has a coarse timer granularity or otherwise can't handle short t_ipi intervals, then either the TFRC sender gets capped at 1 packet every t_gran seconds, or the TFRC sender must be allowed to send short bursts of packets. Besides letting the sender stash sending credits for send times it didn't use, it can be useful to let it send a packet before its scheduled send time. more on that below 💅
A parameter, t_delta, can be used to let a packet go out before its nominal send time. picture an app that goes idle and asks to be rescheduled for time t_i = t_(i-1) + t_ipi, where t_(i-1) is the send time of the previous packet. when the app gets rescheduled, it checks the current time, t_now. if t_now > t_i - t_delta, then packet i gets sent. when the nominal send time t_i of the next packet is calculated, it might already be true that t_now > t_i - t_delta. in that case, the packet would go out immediately 🚀
now, to send at most one packet before its nominal send time, and to never send a packet more than a round-trip time before its nominal send time, t_delta gets set like this:
t_delta = min(t_ipi, t_gran, rtt)/2;(The scheduling granularity t_gran is 10 ms on some older Unix systems. 🐢)
let's do an example 🍓 say a TFRC flow has an allowed sending rate X of 10 packets per round-trip time (PPR), a round-trip time of 100 ms, a system with a scheduling granularity t_gran of 10 ms, and the ability to save up unused sending credits for a round-trip time. in this case, t_ipi is 1 ms. the TFRC sender would be allowed to send packets 0.5 ms before their nominal sending time, and it would be allowed to save unused sending credits for 100 ms. the 10 ms granularity would not significantly affect the performance of the connection ✨
here's a different example that's a little chaotic 🙈 say a TFRC flow has a scheduling granularity bigger than its round-trip time, like a round-trip time of 0.1 ms with a system scheduling granularity of 1 ms, and it can save unused sending credits for a round-trip time. the TFRC sender would be allowed to save unused sending credits for 0.1 ms. if the scheduling granularity did not affect how the sender responds to an incoming feedback packet, then the TFRC sender could send an RTT of data (as determined by the allowed sending rate) each RTT, in response to incoming feedback packets. in this case, the coarse granularity would not significantly reduce the sending rate, but the sending rate would be bursty, with a round-trip time of data sent in response to each feedback packet 💥
however, things would be different here if the OS scheduling granularity affected the sender's response to feedback packets as well as its general scheduling. in that case, the sender's performance would be seriously limited by the granularity being bigger than the round-trip time. the sender could send an RTT of data, at the allowed sending rate, at most once every 1 ms. this restriction on the sending rate is an unavoidable consequence of allowing a burst of at most a round-trip time of data 😭
8.4. Calculation of the Average Loss Interval 🧮
the calculation of the average loss interval in Section 5.4 involves multiplications by the weights w_0 to w_(n-1). for n=8, those weights are:
1.0, 1.0, 1.0, 1.0, 0.8, 0.6, 0.4, 0.2.with a minor loss of smoothness, it would be possible to use weights that were powers of two or sums of powers of two, e.g.,
1.0, 1.0, 1.0, 1.0, 0.75, 0.5, 0.25, 0.25.8.5. The Optional History Discounting Mechanism 🎀
the optional history discounting mechanism described in Section 5.5 is used in the calculation of the average loss rate. the history discounting mechanism only gets invoked when there has been an unusually long interval with no packet losses. for more efficient operation, the discount factor, DF_i, could be restricted to be a power of two 🦋
9. Changes from RFC 3448 🌈
9.1. Overview of Changes
this section sums up the changes from RFC 3448. at a high level, the main change is adding mechanisms to handle the case of a data-limited sender. this document also explicitly lets the TFRC sender accumulate up to a round-trip time of unused send credits, which means it can send a burst of packets if data shows up from the application in a burst after a data-limited period. this issue wasn't explicitly addressed in RFC 3448 👑
this document changes RFC 3448 to bring in TCP's higher initial sending rates from RFC 3390. it also changes RFC 3448 to let RFC 4342's use of a coarse-grained timestamp on data packets stand in for a more fine-grained timestamp.
other changes handle corner cases involving slow-start, the response when the first data packet is dropped, and stuff like that. this document also includes the items in the RFC 3448 Errata 🫶
this section is non-normative; the normative text is in the cited sections.
9.2. Changes in Each Section 📝
Section 4.1, estimating the average segment size: Section 4.1 was updated to give a specific algorithm for estimating the average segment size.
Section 4.2, update to the initial sending rate: in RFC 3448, the initial sending rate was two packets per round-trip time. in this document, the initial sending rate can be as high as four packets per round-trip time, following RFC 3390. the initial sending rate is now defined in terms of the segment size s, not the MSS.
Section 4.2 now says that tld, the Time Last Doubled during slow-start, can be initialized to either 0 or -1. Section 4.2 was also clarified to say that RTT measurements don't only come from feedback packets. they can also come from other places, like the SYN exchange ✨
Section 4.3, response to feedback packets: Section 4.3 was modified to change how the receive rate limits the sender's allowed sending rate. it now uses the set of receive rate values from the last two round-trip times, and the set of receive rate values gets initialized with a large value.
the larger initial sending rate in Section 4.2 doesn't help much if the receiver sends a feedback packet after the first packet is received and the sender, in response, cuts the allowed sending rate to at most two packets per RTT, which would be twice the receive rate. because the sender now processes the receive rate differently, it no longer reduces the allowed sending rate to twice the reported receive rate in response to the first feedback packet 💅
during a data-limited period, the sender saves the receive rate reported from just before the data-limited period, if it's larger than the receive rate during the data-limited period. the sender also reduces the saved values in X_recv_set in response to a loss during a data-limited period. Appendix C discusses this response further.
Section 4.4, response to an idle period: following Section 5.1 from [RFC4342], this document specifies that when the sending rate is reduced after an idle period that covers the period since the nofeedback timer was set, the allowed sending rate isn't reduced below the initial sending rate. (In Section 4.4, the variable recover_rate is set to the initial sending rate.)
Section 4.4, correction from [RFC3448Err]. RFC 3448 had contradictory text about whether the sender halved its sending rate after two round-trip times without receiving a feedback report, or after four round-trip times. this document clarifies that the sender halves its sending rate after four round-trip times without receiving a feedback report [RFC3448Err] 🙌
Section 4.4, clarification for slow-start: Section 4.4 was clarified to specify that on the expiration of the nofeedback timer, if p = 0, X_Bps cannot be used, because the sender doesn't have a value for X_Bps yet. Section 4.4 was also clarified to check the case where the sender doesn't have an RTT sample yet but has sent a packet since the nofeedback timer was set.
Section 4.6: credits for unused send time:
Section 4.6 has been clarified to say that the TFRC sender gets to accumulate up to an RTT of credits for unused send time. Section 4.6 was also rewritten to make clear what is specification and what is implementation 🧠
Section 5.4, clarification: Section 5.4 was modified to clarify the receiver's calculation of the average loss interval when the receiver hasn't yet seen n loss intervals.
Section 5.5, correction: Section 5.5 was corrected to say that the loss interval I_0 includes all transmitted packets, including lost and marked packets (as defined in Section 5.3 in the general definition of loss intervals).
Section 5.5, correction from [RFC3448Err]: a line in Section 5.5 was changed from
for (i = 1 to n) { DF_i = 1; }to
for (i = 0 to n) { DF_i = 1; }[RFC3448Err] 🔧
Section 5.5, history discounting: THRESHOLD, the lower bound on the history discounting parameter DF, was changed from 0.5 to 0.25, to allow more history discounting when the current interval is long 🫧
Section 6, multiple feedback packets: Section 6 now has more discussion of procedures for when the receiver sends multiple feedback packets each round-trip time 📬
Section 6.3, initialization of the feedback timer: Section 6.3 now specifies how the receiver initializes the feedback timer if the first data packet received doesn't have an estimate of the round-trip time 🎯
Section 6.3, a coarse-grained timestamp: Section 6.3 was modified to include, as an option, a coarse-grained timestamp from the sender that increments every quarter of a round-trip time, instead of a more fine-grained timestamp. this follows RFC 4342 ⏱️
Section 6.3.1, after the first loss event: Section 6.3.1 now says that, for initializing the loss history after the first loss event, the receiver uses the maximum receive rate so far, instead of the receive rate in the last round-trip time 📈
Section 6.3.1, if the first data packet is dropped: Section 6.3.1 now has a specification for initializing the loss history if the first data packet sent is lost or ECN-marked 🌸
Section 7, sender-based variants: Section 7's discussion of sender-based variants has been expanded, with reference to RFC 4342 💖
10. Security Considerations 🔐
okay bestie, TFRC is not a transport protocol all on its own 🦋 it's a congestion control mechanism that's meant to be used with a transport protocol. so security mostly has to be thought about in the context of whatever transport protocol you're using and its authentication stuff 💅
congestion control can get abused for denial of service, and one way that happens is spoofed feedback 😭 so any transport protocol using TFRC should take extra care to make sure feedback is only accepted from the receiver of the data. how exactly you do that depends on the transport protocol itself, so it's not a one-size-fits-all situation 🎀
there's also the greedy receiver problem 👀 this is when a receiver wants more than its fair share of bandwidth, so it claims it got packets that actually got lost because of congestion. defenses against that normally include some kind of nonce that the receiver has to feed back to the sender to prove it actually received stuff ✨ but the details of that nonce depend on the transport protocol, and on whether the protocol is reliable or unreliable. main character drama, honestly 🎭
we expect protocols that use ECN with TFRC will also want feedback from the receiver to the sender using the ECN nonce [RFC3540] 🌈 the ECN nonce is a modification to ECN that protects the sender from accidental or malicious concealment of marked packets. again, the details depend on the transport protocol and aren't covered in this document, so go check your transport's specs 🫶
10.1. Security Considerations for TFRC in DCCP
TFRC is currently used in Congestion Control ID 3 (CCID 3) [RFC4342] of the Datagram Congestion Control Protocol (DCCP) [RFC4340] 🍓 the Security Considerations section of RFC 4340 [RFC4340] (Section 18) talks about some of DCCP's security issues, including sequence number validity checks to protect against hijacked connections 🔒 Section 18 of RFC 4340 also covers mechanisms in DCCP that limit the potential impact of denial-of-service attacks 🛡️
RFC 4342 specifies how TFRC is used in CCID 3, and it has extensive discussion of the mechanisms the sender can use to verify the info the receiver sends back 🧐 when ECN is used with CCID 3, the receiver returns ECN Nonce information so the sender can verify what the receiver sent. when ECN is not used, Section 9 of RFC 4342 explains how the sender could still use various techniques that might catch the receiver messing up its congestion reporting 👀 but as RFC 4342 itself says, that's not as robust or non-intrusive as the verification the ECN Nonce provides 💀
11. Acknowledgments 🫶
we'd like to thank everyone who gave feedback and had discussions about equation-based congestion control 🌸 that includes folks from the Reliable Multicast Research Group, the Reliable Multicast Transport Working Group, and the End-to-End Research Group ✨ we also want to thank Dado Colussi, Gorry Fairhurst, Ladan Gharai, Wim Heirman, Eddie Kohler, Ken Lofgren, Mike Luby, Ian McDonald, Vladimir Moltchanov, Colin Perkins, Michele R., Gerrit Renker, Arjuna Sathiaseelan, Vladica Stanisic, Randall Stewart, Eduardo Urzaiz, Shushan Wen, and Wendy Lee (lhh@zsu.edu.cn) for feedback on earlier versions of this document 💖 and a big thank you to Mark Allman for his extensive feedback from using [RFC3448] to produce a working implementation 👑
Appendix A. Terminology 📖
this document uses the following terms. timer variables (e.g., t_now, tld) are assumed to be in seconds, with a timer resolution of at least a millisecond ⏱️
data-limited interval: an interval where the sender is data-limited (not sending as much as it is allowed to send) over the entire interval (Section 4.3). 🍑DF: discount factor for a loss interval (Section 5.5). 💅initial_rate: allowed initial sending rate. 🌷last_counter: greatest received value of the window counter (Section 6.3). 🧚n: number of loss intervals. 🔢NDUPACK: number of dupacks for inferring loss (constant) (Section 5.1). 🦋nofeedback timer: sender-side timer (Section 4). ⏳p: estimated loss event rate. 📉p_prev: previous value of p (Section 6.1). ⬅️q: filter constant for RTT (constant) (Section 4.3). 🎀q2: filter constant for long-term RTT (constant) (Section 4.6). 🎀R: estimated path round-trip time. 🛣️R_m: a specific estimate of the path round-trip time (Sections 4.3, 6). 🔍R_sample: measured path RTT (Section 4.3). 📏R_sqmean: long-term estimate of the square root of the RTT (Section 4.6). 🌊recover_rate: allowed rate for resuming after an idle period (Section 4.4). 🌅recv_limit: limit on sending rate computed from X_recv_set (Section 4.3). 🚧s: nominal packet size in bytes. 📦S: sequence number. 🔢t_delay: reported time delay between receipt of the last packet at the receiver and the generation of the feedback packet (Section 3.2.2). ⌛t_delta: parameter for flexibility in send time (Section 8.3). 🫠t_gran: scheduling timer granularity of the operating system (constant) (Section 8.3). ⚙️t_ipi: inter-packet interval for sending packets (Section 4.6). 📨t_mbi: maximum RTO value of TCP (constant) (Section 4.3). 🧱t_recvdata: timestamp of the last data packet received (Section 3.2.2). 🕰️timer_limit: limit on the sending rate from the expiration of the nofeedback timer (Section 4.4). 🛑tld: Time Last Doubled (Section 4.2). 2️⃣t_now: current time (Section 4.3). 🕒t_RTO: estimated RTO of TCP (Section 4.3). 🎯X: allowed transmit rate, as limited by the receive rate. 🚀X_Bps: calculated sending rate in bytes per second (Section 3.1). 💧X_pps: calculated sending rate in packets per second (Section 3.1). 📦X_inst: instantaneous allowed transmit rate (Section 4.6). ⚡X_recv: estimated receive rate at the receiver (Section 3.2.2). 📬X_recv_set: a small set of recent X_recv values (Section 4.3). 🗂️X_target: the target sending rate after the first loss event (Section 6.3.1). 🎯W_init: TCP initial window (constant) (Section 4.2). 🪟
Appendix B. The Initial Value of the Nofeedback Timer ⏰
so why is TFRC's nofeedback timer set to two seconds, instead of the recommended initial value of three seconds for TCP's retransmit timer from [RFC2988]? 🤔 honestly, there's no particular reason TFRC's nofeedback timer has to match TCP's retransmit timer. TCP's retransmit timer doesn't just slow things down when there's congestion, it also retransmits a packet that's assumed to have been dropped in the network 📦 TFRC's nofeedback timer, on the other hand, is only used to reduce the allowed sending rate. it doesn't trigger the sending of a new packet 🙅♀️ so there's no danger to the network if TFRC's nofeedback timer starts out smaller than TCP's recommended initial value 🫶
and it gets even more interesting 👀 when the nofeedback timer hasn't expired yet, TFRC has a more slowly responding congestion control mechanism than TCP. plus, TFRC's use of the receive rate to limit the sending rate is somewhat less precise than TCP's use of windows and ack-clocking 🧠 so the nofeedback timer is a really important safety mechanism for TFRC 🛟 for all of these reasons, it's perfectly reasonable for TFRC's nofeedback timer to have a smaller initial value than TCP's retransmit timer 💅
Appendix C. Response to Idle or Data-Limited Periods 🏖️
future work could explore alternate responses to using the receive rate during a data-limited period, and to responding to a loss event during a data-limited period 🔮
in particular, an Experimental RFC [RFC2861] specifies Congestion Window Validation (CWV) for TCP 🧪 for this discussion, we use the term "Standard TCP" to refer to the TCP congestion control mechanisms in [RFC2581] and [RFC2581bis] 📚 [RFC2861] specifies a different response to idle or data-limited periods than Standard TCP does. with CWV, the TCP sender halves the congestion window after each RTO during an idle period, down to the initial window 📉 similarly, with CWV the TCP sender halves the congestion window half-way down to the flight size after each RTO during a data-limited period 🌓
this document already specifies a TFRC response to idle periods that's similar to TCP with Congestion Window Validation ✅ however, this document does not specify a TFRC response to data-limited periods similar to CWV 🙅♀️ adding that would take a one-line change to step (4) of Section 4.3 ✏️ specifically, the sender's response to a feedback packet could be changed from:
If (the entire interval covered by the feedback packet
was a data-limited interval) {
If (the feedback packet reports a new loss event or an
increase in the loss event rate p) {
Halve entries in X_recv_set;
X_recv = 0.85 * X_recv;
Maximize X_recv_set();
recv_limit = max (X_recv_set);
} Else {
Maximize X_recv_set();
recv_limit = 2 * max (X_recv_set);
}
}to:
If (the entire interval covered by the feedback packet
was a data-limited interval) {
Multiply old entries in X_recv_set by 0.85;
If (the feedback packet reports a new loss event or an
increase in the loss event rate p) {
Multiply new value X_recv by 0.85.
}
Maximize X_recv_set();
recv_limit = 2 * max (X_recv_set);
}here's the tea ☕ if the receive rate from before a data-limited period is saved in X_recv_set, then the change in step (4) above would multiply that receive rate by 0.85 each time a feedback packet is received and the code above runs 📉 so after four successive round-trip times of data-limited intervals, the receive rate from before the data-limited period would be reduced by 0.85^4 = 0.52 🫠 thus, this one-line change to step (4) of Section 4.3 would result in the allowed sending rate being halved for each four round-trip times in which the sender was data-limited 💀 and because of the nature of X_recv_set, this mechanism would never reduce the allowed sending rate below twice the most recent receive rate 🛡️
we also note that in the suggested code above, with CWV-style behavior in response to data-limited intervals, we keep
recv_limit = 2 * max (X_recv_set);instead of using
recv_limit = max (X_recv_set);following loss events in data-limited intervals 🧐 this relaxed response to a loss event is allowed because the CWV-style behavior itself limits rapid fluctuations in the sending rate during data-limited periods ✨
C.1. Long Idle or Data-Limited Periods 🦋
Okay so, Table 1 is basically the tea ☕ on how Standard TCP [RFC2581], TCP with Congestion Window Validation [RFC2861], Standard TFRC [RFC3448], and Revised TFRC (this document) each react to long idle or data-limited periods. For this section, a long period is anything at least an RTO long. 💅
Protocol Long idle periods Long data-limited periods
-------------- -------------------- ----------------------
Standard TCP: Window -> initial. Window increases for
each cwnd of data.
TCP with CWV: Halve window Reduce window half way
(not below initial cwnd). to used window.
Standard TFRC: Halve rate Rate limited to
(not below 2 pkts/rtt). twice receive rate.
One RTT after sending pkt,
rate is limited by X_recv.
Revised TFRC: Halve rate Rate limited to twice
(not below initial rate). max (current X_recv,
receive rate before
data-limited period).
Table 1: Response to Long Idle or Data-Limited PeriodsStandard TCP after long idle periods: For Standard TCP, [RFC2581] says TCP SHOULD set the congestion window to no more than the initial window after an idle period of at least an RTO. 🫶 To be precise, RFC 2581 specifies that the TCP sender should set cwnd to the initial window if the sender hasn't sent data in an interval longer than the retransmission timeout.
Standard TCP after long data-limited periods: Standard TCP [RFC2581] doesn't shrink TCP's congestion window after a data-limited period when the window isn't fully used. 🙈 In [RFC2581], the FlightSize (the amount of outstanding data in the network) only gets used when setting the slow-start threshold after a retransmit timeout. Standard TCP also isn't limited by TCP's ack-clocking mechanism during a data-limited period. Iconic, honestly.
Standard TCP's lax response to a data-limited period is super different from its stringent response to an idle period. 💀
TCP with Congestion Window Validation (CWV) after long idle periods: As an experimental alternative, [RFC2861] gives a more moderate response to an idle period than Standard TCP does. During an idle period, the TCP sender halves cwnd after each RTO, down to the initial cwnd. 🌈
TCP with Congestion Window Validation after long data-limited periods: Also as an experimental alternative, [RFC2861] gives a stricter response to a data-limited period than Standard TCP. After each RTO seconds of a data-limited period, the congestion window gets cut in half, down to the window that's actually used. 🍓
The way TCP with CWV responds to an idle period is similar to how it responds to a data-limited period. It's less restrictive than Standard TCP for idle periods, and more restrictive than Standard TCP for data-limited periods. ✨
Standard TFRC after long idle periods: For Standard TFRC, [RFC3448] says the allowed sending rate gets halved after each RTO seconds of an idle period. The allowed sending rate never drops below two packets per RTT after idle periods. After an idle period, the first feedback packet received reports a receive rate of one packet per round-trip time, and that receive rate is what limits the sending rate. Standard TFRC basically slow-starts up from that allowed sending rate. 🚀
Standard TFRC after long data-limited periods: [RFC3448] doesn't tell data-limited periods apart from non-data-limited ones. So the allowed sending rate gets capped at twice the receive rate during and after a data-limited period. That's a very restrictive response, even stricter than Standard TCP or TCP with CWV. 😭
Revised TFRC after long idle periods: For Revised TFRC, this document says the allowed sending rate gets halved after each RTO seconds of an idle period. It never drops below the initial sending rate because of an idle period. The first feedback packet after the idle period reports a receive rate of one packet per round-trip time. 🎀 But the Revised TFRC sender doesn't use that receive rate to limit the sending rate. So Revised TFRC is different from Standard TFRC in two ways: the lower limit used when cutting the sending rate, and the better handling of that first feedback packet after an idle period. 👑
Revised TFRC after long data-limited periods: For Revised TFRC, this document separates data-limited periods from non-data-limited ones. As Section 4.3 specifies, during a data-limited period Revised TFRC remembers the receive rate from before the data-limited period started, and it doesn't cut the allowed sending rate below twice that receive rate. That's kinda like Standard TCP, and totally different from the very restrictive Standard TFRC response. 🫧 Still, it isn't as conservative as TCP with Congestion Window Validation, where the congestion window is slowly reduced down to the window actually used during a data-limited period.
We should note that for current TCP implementations, the congestion window generally doesn't increase during a data-limited period (when the current congestion window isn't being fully used) [MAF05] (Section 5.7). There's no comparable mechanism in Revised TFRC, just so you know. 🙈
Recovery after idle or data-limited periods: When TCP reduces the congestion window after an idle or data-utilized period, TCP can set the slow-start threshold, ssthresh, so the TCP sender can slow-start back up toward its old sending rate once the idle or data-limited period is over. ✨ But in TFRC, even when the sender's rate is restricted to twice the previous receive rate, the sender can still double its sending rate from one round-trip time to the next, as long as the throughput equation allows it. So TFRC doesn't need something like TCP's ssthresh setting to allow a slow-start after an idle or data-limited period. Main character energy. 👑
For future work, one thing to look into would be adding Congestion Window Validation mechanisms to TFRC's response to data-limited periods. Right now, following Standard TCP, Revised TFRC doesn't limit its allowed sending rate as a function of the receive rate during data-limited periods. 🧚
C.2. Short Idle or Data-Limited Periods 🌸
Table 2 shows how Standard TCP [RFC2581], TCP with Congestion Window Validation [RFC2861], Standard TFRC [RFC3448], and Revised TFRC (this document) react to short idle or data-limited periods. For this section, a short period is anything less than an RTT. 💖
Protocol Short idle periods Short data-limited periods
-------------- -------------------- ----------------------
Standard TCP: Send a burst up to cwnd. Send a burst up to cwnd.
TCP with CWV: Send a burst up to cwnd. Send a burst up to cwnd.
Standard TFRC: ? ?
Revised TFRC: Send a burst Send a burst
(up to an RTT of (up to an RTT of
unused send credits). unused send credits).
Table 2: Response to Short Idle or Data-Limited PeriodsTable 2 shows that Revised TFRC responds to short idle or data-limited periods in a way that's similar to Standard TCP and TCP with CWV. For a short idle or data-limited period, TCP is only limited by the size of the unused congestion window, and Revised TFRC is only limited by the number of unused send credits (up to an RTT's worth). 🦋 For Standard TFRC, [RFC3448] didn't explicitly specify how it behaves with respect to unused send credits. Bestie, that's a gap, not a vibe. 🙈
C.3. Moderate Idle or Data-Limited Periods 🌷
Table 3 shows how Standard TCP [RFC2581], TCP with Congestion Window Validation [RFC2861], Standard TFRC [RFC3448], and Revised TFRC (this document) react to moderate idle or data-limited periods. For this section, a moderate period is anything greater than an RTT but less than an RTO. ⭐
Protocol Moderate idle periods Moderate data-limited periods
------------- --------------------- -------------------------
Standard TCP: Send a burst up to cwnd. Send a burst up to cwnd.
TCP with CWV: Send a burst up to cwnd. Send a burst up to cwnd.
Standard TFRC: ? Limited by X_recv.
Revised TFRC: Send a burst Send a burst
(up to an RTT of (up to an RTT of
unused send credits). unused send credits).
Table 3: Response to Moderate Idle or Data-Limited PeriodsTable 3 shows that Revised TFRC responds to moderate idle or data-limited periods in a way that's similar to Standard TCP and TCP with CWV. For a moderate idle or data-limited period, TCP is only limited by the size of the unused congestion window. 💅 For a moderate idle period, Revised TFRC is only limited by the number of unused send credits (up to an RTT's worth). For a moderate data-limited period, Standard TFRC would be limited by X_recv from the most recent feedback packet. Revised TFRC, on the other hand, isn't limited by the receive rate from data-limited periods that cover an entire feedback period of a round-trip time. 🫶 And for Standard TFRC, [RFC3448] didn't explicitly specify how it behaves with respect to unused send credits. 🍓
C.4. Losses During Data-Limited Periods ✨
Okay, this bit is about what happens when a packet gets lost during a data-limited period. Like, the sender isn't even blasting data at full speed, and something still goes wrong. 😭
Protocol Response to a loss during a data-limited period
------------- -----------------------------------------------
Standard TCP: Set ssthresh, cwnd to FlightSize/2.
TCP with CWV: Same as Standard TCP.
Standard TFRC: Calculate X_Bps, send at most 2*X_recv.
Revised TFRC: Calculate X_Bps, send at most recv_limit.
In addition, modify X_recv_set.
Table 4: Response to a Loss during a Data-Limited PeriodIn TCP [RFC2581], a loss during a data-limited period gets the exact same treatment as a loss at any other time. The congestion window gets set to half of the FlightSize, and the FlightSize is just how much unacknowledged data is actually out there. So after a loss in a data-limited period, the TCP sender must halve its allowed sending rate, same as always. Iconic consistency, honestly. 💅
Standard TFRC is also consistent here. A loss during a data-limited period gets handled just like a loss at any other time. The sending rate is capped by X_Bps from the throughput equation, and it's also capped at twice X_recv, which is the most recent receive rate. So after a loss in a data-limited period, the sender can at most double its rate to twice X_recv, even if the throughput equation would have allowed something way higher. 🫠
Revised TFRC is where things get spicy. ✨ The receive rate X_recv gets handled differently during data-limited intervals. The sender is limited to sending at most recv_limit, and the sender can remember the X_recv from right before the data-limited period. That means the sender can more than double its rate during data-limited periods, up to the receive rate from before the data-limited period (as long as the throughput equation X_Bps allows it). This is kind of like how Standard TCP doesn't shrink the window during data-limited periods when there's no loss. It's giving continuity. 🦋
Revised TFRC does act differently than Standard TFRC here, though. While the sender is data-limited, it's sending less than the throughput equation X_Bps would allow. After a loss event, the sender might still not want to send as much as the newly recalculated X_Bps would permit, since that value now includes the new loss event. So Revised TFRC adds one more mechanism to slowly limit the sending rate after losses in data-limited periods. Unlike TCP, which just sets cwnd to half the FlightSize, this extra mechanism uses TFRC's usual approach of slow-responding changes, for both increases and decreases in the allowed sending rate. Slow and steady, bestie. 🐢💜
Here's how that works in Revised TFRC. In step (4) of Section 4.3, the sender decreases the X_recv_set entry after a loss in a data-limited interval. It also lets the sender send at most max (X_recv_set), instead of at most twice max (X_recv_set), during the immediate round-trip time after the reported loss. So the "price" of letting the sender send more than twice the most recently reported X_recv during a data-limited interval is an extra mechanism that brings that allowed rate back down after losses in data-limited periods. Trade-offs, babe. 🎀
When we think about TFRC's response to a loss in a data-limited interval, these examples come up.
Example 1, Losses after a Data-Limited Period: This one shows that losses after a data-limited period has ended are handled by the throughput equation X_Bps.
-------------------------------------------------------------------
Stage 1: Not data-limited.
Sending 100 packets per round-trip time (PPR).
Stage 2: Data-limited, sending 10 PPR.
Stage 3: Not data-limited.
Sending 100 PPR again, as allowed by X_Bps.
A packet loss in the first RTT of Stage 3.
X_Bps is updated,
Response of Revised TFRC: a slight reduction in the allowed sending
rate, depending on the number of packets since the last loss event.
-------------------------------------------------------------------
Table 5: Example 1, Losses after a Data-Limited PeriodFor Example 1, a packet loss in the first RTT of Stage 3 gets reflected in a modified value of X_Bps, and future loss events keep reducing X_Bps over time. Following TFRC's standard use of the throughput equation [FHPW00] (Section A.2), the allowed TFRC sending rate would be halved after something like five successive round-trip times with loss. Ouch, but it's the math doing its thing. 📉
Example 2, a Mildly Data-Limited Sender: This one looks at losses in a data-limited period where the sender is sending almost as much as it's allowed to send during that period.
-------------------------------------------------------------------
Stage 1: Not data-limited. Sending 100 PPR.
Stage 2: Data-limited, sending 99 PPR.
A packet loss in Stage 2.
Response of Revised TFRC: a slight reduction in the allowed sending
rate, down to 85 PPR or less, depending on the number of packets
since the last loss event.
-------------------------------------------------------------------
Table 6: Example 2, a Mildly Data-Limited SenderPicture a Revised TFRC connection where the sender has been going at a hundred PPR, then enters a data-limited period and only sends 99 PPR because the application doesn't have more data to give it. (That means at every instant during the data-limited period, the sender could've sent one more packet.) If there are losses during that data-limited period, the allowed sending rate drops to min(X_Bps, recv_limit). Both the throughput equation X_Bps and the recv_limit force a slight reduction in the allowed sending rate. Small but mighty. 🌸
Example 3, a Single Packet Loss during a Data-Limited Period. This example covers losing a single packet during a data-limited period, after the sender hasn't sent anything for two RTTs.
-------------------------------------------------------------------
Stage 1: Not data-limited. Sending 100 PPR.
Stage 2: Data-limited, sending 10 PPR.
Stage 3: Data-limited, sending no data for two RTTs.
Stage 4: Data-limited, sending one packet, which is ECN-marked.
Response of Revised TFRC: a reduction in the allowed sending
rate, down to 50 PPR or less. For each loss event during
the data-limited period, the 'remembered' X_recv from before
the data-limited period is effectively halved.
-------------------------------------------------------------------
Table 7: Example 3, a Single Packet LossHere's the setup: a Revised TFRC connection has been sending a hundred PPR, then enters a data-limited period of only ten PPR, then goes silent for two RTTs, and then sends a single packet that gets ECN-marked. With Revised TFRC, every loss event during the data-limited period makes the sender halve its 'remembered' X_recv from before the data-limited period. 🙈
Example 4, Losses after Increasing the Sending Rate during a Data-Limited Period. This example looks at losses when the sender really ramps up its sending rate during a data-limited period.
-------------------------------------------------------------------
Stage 1: Not data-limited. Sending 100 PPR.
Stage 2: Data-limited, sending 1 PPR.
Stage 3: Data-limited, sending 20 PPR.
Several packets are lost in each RTT of Stage 3.
During Stage 3, the sender would *like* to send 20 PPR.
Response of Revised TFRC: For each loss event during
the data-limited period, the 'remembered' X_recv from before
the data-limited period is effectively halved, and the most
recent X_recv is reduced by 0.85.
-------------------------------------------------------------------
Table 8: Example 4, Losses after Increasing the Sending RatePicture a Revised TFRC connection that's been going at a hundred PPR, then enters a data-limited period of only one PPR. While it's still data-limited, it bumps its sending rate up to twenty PPR and runs into a bunch of loss events in a row. In that case, with Revised TFRC, every loss event during the data-limited period makes the sender halve its 'remembered' X_recv from before the data-limited period, and the most recent X_recv gets reduced by 0.85. Big ramp, big consequences. 💀
C.5. Other Patterns 🧚
Other patterns worth checking when we evaluate Revised TFRC would be comparing TCP, Standard TFRC, and Revised TFRC for connections with alternating busy and idle periods, alternating idle and data-limited periods, or idle or data-limited periods during slow-start. Lots to explore, bestie. 🔍
C.6. Evaluating TFRC's Response to Idle Periods 🎀
Here we focus on evaluating how Revised TFRC responds to idle or data-limited periods.
One drawback of Standard TFRC's strict response to idle or data-limited periods is that it could be seen as encouraging applications to pad their sending rate during those periods, by sending dummy data when there's nothing else to send. 😬 Revised TFRC has a less strict response to data-limited periods than Standard TFRC does, so it could also be seen as giving applications less incentive to pad their sending rates during data-limited periods. Work in progress, like Faster Restart [KFS07], can also reduce an application's incentive to pad its sending rate, by allowing faster start-up after idle periods. More research would help us understand, in more detail, how TCP's or TFRC's congestion control mechanisms interact with an application's incentive to pad its sending rate during idle or data-limited periods.
TCP Congestion Window Validation, covered in Appendix C.1 above, is an Experimental standard. It specifies that the TCP sender slowly reduces the congestion window during an idle or data-limited period [RFC2861]. TFRC's and Revised TFRC's responses to idle periods are roughly similar to TCP with Congestion Window Validation. Revised TFRC's response to data-limited periods, though, is less conservative than TCP with Congestion Window Validation. Standard TFRC's response to data-limited periods was considerably more conservative than Congestion Window Validation. 🫶
Future work could modify this document so Revised TFRC's response to a data-limited period includes a slow reduction of the allowed sending rate. Section C specifies a possible mechanism for this. Such a modification would be especially compelling if Congestion Window Validation became a Proposed Standard in the IETF for TCP. ✨
References 📚
Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC3448] Handley, M., Floyd, S., Padhye, J., and J. Widmer, "TCP
Friendly Rate Control (TFRC): Protocol Specification",
RFC 3448, January 2003.12.2. Informative References ✨
Okay bestie, these are the extra reads that back everything up. Here's the whole stack 📚🦋
- [BRS99] Balakrishnan, H., Rahul, H., and Seshan, S., "An Integrated Congestion Management Architecture for Internet Hosts," Proc. ACM SIGCOMM, Cambridge, MA, September 1999.
- [CCID-4] Floyd, S., and E. Kohler, "Profile for DCCP Congestion Control ID 4: the Small-Packet Variant of TFRC Congestion Control", Work in Progress, February 2008.
- [FHPW00] S. Floyd, M. Handley, J. Padhye, and J. Widmer, "Equation-Based Congestion Control for Unicast Applications", August 2000, Proc SIGCOMM 2000.
- [FHPW00a] S. Floyd, M. Handley, J. Padhye, and J. Widmer, "Equation-Based Congestion Control for Unicast Applications: the Extended Version", ICSI tech report TR-00-03, March 2000.
- [FF99] Floyd, S., and K. Fall, Promoting the Use of End-to-End Congestion Control in the Internet, IEEE/ACM Transactions on Networking, August 1999.
- [KFS07] E. Kohler, S. Floyd, and A. Sathiaseelan, "Faster Restart for TCP Friendly Rate Control (TFRC)", Work in Progress, November 2007.
- [MAF05] A. Medina, M. Allman, and S. Floyd, "Measuring the Evolution of Transport Protocols in the Internet", ACM Computer Communications Review, April 2005.
- [PFTK98] Padhye, J. and Firoiu, V. and Towsley, D. and Kurose, J., "Modeling TCP Throughput: A Simple Model and its Empirical Validation", Proc ACM SIGCOMM 1998.
- [RFC2140] Touch, J., "TCP Control Block Interdependence", RFC 2140, April 1997.
- [RFC2581] Allman, M., Paxson, V., and W. Stevens, "TCP Congestion Control", RFC 2581, April 1999.
- [RFC2581bis] Allman, M., Paxson, V., and W. Stevens, "TCP Congestion Control", Work in Progress, April 2008.
- [RFC2861] Handley, M., Padhye, J., and S. Floyd, "TCP Congestion Window Validation", RFC 2861, June 2000.
- [RFC2988] Paxson, V. and M. Allman, "Computing TCP's Retransmission Timer", RFC 2988, November 2000.
- [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition of Explicit Congestion Notification (ECN) to IP", RFC 3168, September 2001.
- [RFC3390] Allman, M., Floyd, S., and C. Partridge, "Increasing TCP's Initial Window", RFC 3390, October 2002.
- [RFC3448Err] RFC 3448 Errata,
<http://www.rfc-editor.org/errata_search.php?rfc=3448>. - [RFC3540] Spring, N., Wetherall, D., and D. Ely, "Robust Explicit Congestion Notification (ECN) Signaling with Nonces", RFC 3540, June 2003.
- [RFC4340] Kohler, E., Handley, M., and S. Floyd, "Datagram Congestion Control Protocol (DCCP)", RFC 4340, March 2006.
- [RFC4342] Floyd, S., Kohler, E., and J. Padhye, "Profile for Datagram Congestion Control Protocol (DCCP) Congestion Control ID 3: TCP-Friendly Rate Control (TFRC)", RFC 4342, March 2006.
- [RFC4828] Floyd, S. and E. Kohler, "TCP Friendly Rate Control (TFRC): The Small-Packet (SP) Variant", RFC 4828, April 2007.
- [W00] Widmer, J., "Equation-Based Congestion Control", Diploma Thesis, University of Mannheim, February 2000,
<http://www.icir.org/tfrc/>.
Authors' Addresses 🎀
Here's where to find the crew who made this happen 💜
Sally Floyd
ICSI
1947 Center St, Suite 600
Berkeley, CA 94708
EMail: floyd@icir.org
Mark Handley,
Department of Computer Science
University College London
Gower Street
London WC1E 6BT
UK
EMail: M.Handley@cs.ucl.ac.uk
Jitendra Padhye
Microsoft Research
EMail: padhye@microsoft.com
Joerg Widmer
DoCoMo Euro-Labs
Landsberger Strasse 312
80687 Munich
Germany
EMail: widmer@acm.orgFull Copyright Statement 🌈
Copyright (C) The IETF Trust (2008). The vibe here is legal, but the gist is simple.
This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights.
This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Intellectual Property 🦋
The IETF doesn't take a side on whether any Intellectual Property Rights, or other rights that someone might claim over the technology in this document, are valid or in scope. That goes for implementing or using it too, and for whether any license under those rights might be available. The IETF also doesn't claim it has searched for any of those rights itself. You can find the procedures for rights in RFC documents in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat, along with any assurances of licenses to be made available, or the result of an attempt to get a general license or permission for implementers or users to use those proprietary rights, can be found in the IETF online IPR repository at http://www.ietf.org/ipr.
The IETF invites anyone interested to bring to its attention any copyrights, patents, patent applications, or other proprietary rights that might cover technology needed to implement this standard. Send that information to the IETF at ietf-ipr@ietf.org. 💅✨
📦 where this page comes from
- ietf: rfc5348 2008-09
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms