RFC 4828
TCP Friendly Rate Control (TFRC): The Small-Packet (SP) Variant
RFC 4828: TCP Friendly Rate Control (TFRC): The Small-Packet (SP) Variant 💖
okay bestie, let's start with the title block 🎀 It's basically the doc's ID card:
- Network Working Group
- Request for Comments: 4828
- Category: Experimental 🧪
- Authors: S. Floyd (ICIR) and E. Kohler (UCLA)
- Date: April 2007
It doesn't list anything this one obsoletes or updates, so it's a standalone glow-up 🦋
Status of This Memo ✨
so this memo defines an Experimental Protocol for the Internet community 🧪 It is NOT an Internet standard of any kind, so don't treat it like one 🙈 They're actively asking for discussion and suggestions for improvement, and distribution of the memo is unlimited 💜
Copyright Notice 🌈
Copyright (C) The IETF Trust (2007). 📜
Abstract 💅
okay the abstract is giving "we're testing this, don't deploy it everywhere yet" energy 🧪 This document proposes a mechanism for more experimentation, but not for widespread deployment in the global Internet at this time 🙅♀️
so TCP-Friendly Rate Control (TFRC) is a congestion control mechanism for unicast flows running in a best-effort Internet environment (RFC 3448) 🌸 TFRC was built for apps that use a fixed packet size, and it was designed to be reasonably fair when it competes for bandwidth with TCP connections using the same packet size ✨
now here's the main character 👑 This document proposes TFRC-SP, a Small-Packet (SP) variant of TFRC, made for apps that send small packets 🧚 The design goal is to get the same bandwidth in bps (bits per second) as a TCP flow using packets of up to 1500 bytes. To stop one flow from sending tiny packets arbitrarily often, TFRC-SP enforces a minimum interval of 10 ms between data packets 🔥
the fairness part is the tea ☕ TFRC-SP competes reasonably fairly with large-packet TCP and TFRC flows when large-packet and small-packet flows see similar packet drop rates. BUT if small-packet flows get lower drop rates than large-packet flows (for example, with Drop-Tail queues measured in bytes), TFRC-SP can end up grabbing considerably more than its share of the bandwidth 💀
Table of Contents 🦋
1. Introduction ....................................................3
2. Conventions .....................................................5
3. TFRC-SP Congestion Control ......................................5
4. TFRC-SP Discussion ..............................................9
4.1. Response Functions and Throughput Equations ................9
4.2. Accounting for Header Size ................................12
4.3. The TFRC-SP Min Interval ..................................13
4.4. Counting Packet Losses ....................................14
4.5. The Nominal Packet Size ...................................15
4.5.1. Packet Size and Packet Drop Rates ..................15
4.5.2. Fragmentation and the Path MTU .....................17
4.5.3. The Nominal Segment Size and the Path MTU ..........17
4.6. The Loss Interval Length for Short Loss Intervals .........18
5. A Comparison with RFC 3714 .....................................19
6. TFRC-SP with Applications that Modify the Packet Size ..........19
7. Simulations ....................................................20
8. General Discussion .............................................21
9. Security Considerations ........................................22
10. Conclusions ...................................................23
11. Acknowledgements ..............................................24
Appendix A. Related Work on Small-Packet Variants of TFRC .........25
Appendix B. Simulation Results ....................................26
B.1. Simulations with Configured Packet Drop Rates .............26
B.2. Simulations with Configured Byte Drop Rates ...............30
B.3. Packet Dropping Behavior at Routers with Drop-Tail
Queues ....................................................32
B.4. Packet Dropping Behavior at Routers with AQM ..............37
Appendix C. Exploring Possible Oscillations in the Loss Event
Rate ...........................................................42
Appendix D. A Discussion of Packet Size and Packet Dropping .......43
Normative References ..............................................44
Informative References ............................................44okay that's the front matter all wrapped up 🎀 Next up is section 1, the Introduction, so stay tuned, bestie 💖🫶
1. Introduction ✨
Okay so this doc is all about TFRC-SP, an experimental, Small-Packet variant of TCP-friendly Rate Control (TFRC) [RFC3448]. 🎀
TFRC was designed to be reasonably fair when it's competing for bandwidth with TCP flows, but it also avoids the abrupt changes in sending rate that TCP's congestion control mechanisms are known for. 🦋 TFRC is meant for things like streaming media applications, where a relatively smooth sending rate really matters. Conventional TFRC measures loss rates by estimating the loss event ratio as described in [RFC3448], and then uses that loss event rate to decide the sending rate in packets per round-trip time. This has consequences for how fast a TFRC flow can go when it shares a bottleneck with large-packet TCP flows. Bestie, this is the tea 🍵: a low-bandwidth, small-packet TFRC flow sharing a bottleneck with high-bandwidth, large-packet TCP flows may be forced to slow down, even though the TFRC application's nominal rate in bytes per second is less than the rate the TCP flows achieve. 😭 Intuitively, this would only be "fair" if the network limitation was in packets per second (like a routing lookup), rather than in bytes per second (like link bandwidth). Conventional wisdom says that many of the network limitations in today's Internet are in bytes per second, even though the limitations of the future might move back toward packets per second. 🔮
TFRC-SP is intended for flows that need to send frequent small packets, with less than 1500 bytes per packet, limited by a minimum interval between packets of 10 ms. ⏱️ It will better support applications that don't want their sending rates in bytes per second to suffer just because they use small packets. This variant is restricted to applications that send packets no more than once every 10 ms (the Min Interval or minimum interval). Given this restriction, TFRC-SP effectively calculates the TFRC fair rate as if the bottleneck restriction was in bytes per second. 💅 Applications using TFRC-SP could have a fixed or naturally-varying packet size, or they could vary their packet size in response to congestion. Applications that are not willing to be limited by a minimum interval of 10 ms between packets, or that want to send packets larger than 1500 bytes, should not use TFRC-SP. However, for applications with a minimum interval of at least 10 ms between packets and with data packets of at most 1500 bytes, the performance of TFRC-SP should be at least as good as that from TFRC. 🌈
RFC 3448, the protocol specification for TFRC, stated that TFRC-PS (for TFRC-PacketSize), a variant of TFRC for applications that have a fixed sending rate but vary their packet size in response to congestion, would be specified in a later document. 📝 This document instead specifies TFRC-SP, a variant of TFRC designed for applications that send small packets, where applications could either have a fixed or varying packet size or could adapt their packet size in response to congestion. However, as discussed in Section 6 of this document, there are many questions about how such an adaptive application would use TFRC-SP that are beyond the scope of this document, and that would need to be addressed in documents that are more application-specific. 🙈
TFRC-SP is motivated in part by the approach in RFC 3714, which argues that it is acceptable for VoIP flows to assume that the network limitation is in bytes per second (Bps) rather than in packets per second (pps), and to have the same sending rate in bytes per second as a TCP flow with 1500-byte packets and the same packet drop rate. 📞 RFC 3714 states the following:
"While the ideal would be to have a transport protocol that is able to detect whether the bottleneck links along the path are limited in Bps or in pps, and to respond appropriately when the limitation is in pps, such an ideal is hard to achieve. We would not want to delay the deployment of congestion control for telephony traffic until such an ideal could be accomplished. In addition, we note that the current TCP congestion control mechanisms are themselves not very effective in an environment where there is a limitation along the reverse path in pps. While the TCP mechanisms do provide an incentive to use large data packets, TCP does not include any effective congestion control mechanisms for the stream of small acknowledgement packets on the reverse path. Given the arguments above, it seems acceptable to us to assume a network limitation in Bps rather than in pps in considering the minimum sending rate of telephony traffic."
Translating the discussion in [RFC3714] to the congestion control mechanisms of TFRC, it seems acceptable to standardize a variant of TFRC that allows low-bandwidth flows sending small packets to achieve a rough fairness with TCP flows in terms of the sending rate in Bps, rather than in terms of the sending rate in pps. 🧁 This is accomplished by TFRC-SP, a small modification to TFRC, as described below.
Maintaining incentives for large packets: Because the bottlenecks in the network in fact can include limitations in pps as well as in Bps, we pay special attention to the potential dangers of encouraging a large deployment of best-effort traffic in the Internet consisting entirely of small packets. 🚨 This is discussed in more detail in Section 4.3. In addition, as again discussed in Section 4.3, TFRC-SP includes the limitation of the Min Interval between packets of 10 ms.
Packet drop rates as a function of packet size: TFRC-SP essentially assumes that the small-packet TFRC-SP flow receives roughly the same packet drop rate as a large-packet TFRC or TCP flow. As we show, this assumption is not necessarily correct for all environments in the Internet. 👀
Initializing the Loss History after the First Loss Event: Section 6.3.1 of RFC 3448 specifies that the TFRC receiver initializes the loss history after the first loss event by calculating the loss interval that would be required to produce the receive rate measured over the most recent round-trip time. In calculating this loss interval, TFRC-SP uses the segment size of 1460 bytes, rather than the actual segment size used in the connection. 🧮
Calculating the loss event rate for TFRC-SP: TFRC-SP requires a modification in TFRC's calculation of the loss event rate, because a TFRC-SP connection can send many small packets when a standard TFRC or TCP connection would send a single large packet. It is not possible for a standard TFRC or TCP connection to repeatedly send multiple packets per round-trip time in the face of a high packet drop rate. As a result, TCP and standard TFRC only respond to a single loss event per round-trip time, and are still able to detect and respond to increasingly heavy packet loss rates. 🌊 However, in a highly-congested environment, when a TCP connection might be sending, on average, one large packet per round-trip time, a corresponding TFRC-SP connection might be sending many small packets per round-trip time. As a result, in order to maintain fairness with TCP, and to be able to detect changes in the degree of congestion, TFRC-SP needs to be sensitive to the actual packet drop rate during periods of high congestion. This is discussed in more detail in the section below. 🫶
2. Conventions 📚
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. 💜
3. TFRC-SP Congestion Control 💖
okay so TFRC uses the TCP throughput equation from Section 3.1 of [RFC3448] 🧚 it gives you the allowed sending rate X in bytes per second, and that rate depends on the loss event rate, the segment size, and the round-trip time ✨ [RFC3448] says the segment size s in that equation should be the segment size the application actually uses, or the estimated mean segment size if the size changes depending on the data. this gives rough fairness with TCP flows using the same segment size 🫶
TFRC-SP changes this up in three ways 💅
- The nominal segment size: the nominal segment size s defaults to 1460 bytes 🌸 following [RFC3714], this aims for fairness, in terms of sending rate in bytes per second, with a TCP flow that carries 1460 bytes of application data per packet but sees the same packet drop rate. if the endpoint knows the MTU (Maximum Transmission Unit) of the path and the derived MSS (Maximum Segment Size) is less than 1460 bytes, then the endpoint SHOULD set the nominal segment size s to MSS bytes. and if the endpoint knows the MTU of the path and the resulting MSS is less than 536 bytes, then the endpoint MUST set the nominal segment size s to MSS bytes.
However, this document does not require TFRC-SP endpoints to determine the path MTU 🙈 most paths allow an MSS of 1460 bytes, but some paths have a slightly smaller MSS because of tunnels (e.g., IPv6 over IPv4). in some specific cases, IPv4 paths can have a much smaller path MTU. estimating the path MTU is complicated, and most paths support an MSS of at least 536 bytes, so as a default TFRC-SP uses a nominal segment size of 1460 bytes. the nominal segment size is discussed in more detail in Section 4.5.3.
- Taking packet headers into account: the allowed transmit rate X in bytes per second gets reduced by a factor that accounts for packet header size 💜 this gives the application some incentive, beyond the Min Interval, not to use unnecessarily small packets. specifically, we introduce a new parameter H, which is the expected size in bytes of network and transport headers on the TFRC connection's packets. X gets reduced like this:
X := X * s_true / (s_true + H),where s_true is the true average data segment size for the connection in bytes, excluding the transport and network headers. Section 4.1 of RFC 3448 says that where the packet size varies naturally with the data, an estimate of the mean segment size can be used for s_true. as suggested in Section 4.1 of [RFC3448bis], when an estimate of the mean segment size is used for s_true, the estimate SHOULD be calculated over at least the last four loss intervals. however, this document does not specify a particular algorithm for estimating the mean segment size.
the H parameter is set to the constant 40 bytes 🦋 so if the TFRC-SP application used 40-byte data segments, the allowed transmit rate X would be halved, because half of the sending rate would go to headers. Section 4.2 justifies this definition. however, a connection using TFRC-SP MAY instead use a more precise estimate of H, based on the actual network and transport headers used on the connection's packets. for example, a Datagram Congestion Control Protocol (DCCP) connection [RFC4340] over IPv4, where data packets use the DCCP-Data packet type and there are no IP or DCCP options, could set H to 20 + 12 = 32 bytes. but if the TFRC implementation knows the IP layer is using IPv6 instead of IPv4, then the connection using TFRC-SP MAY still use the default estimate of 40 bytes for H instead of the actual size of 60 bytes, for simplicity of implementation 🍓
- Measuring the loss event rate in times of high loss: during short loss intervals (those at most two round-trip times), the loss rate is computed by counting the actual number of packets lost or marked, not by counting at most one loss event per loss interval 🔥 without this change, TFRC-SP could send multiple packets per round-trip time even under heavy congestion, which would mean a steady state of multiple packets dropped each round-trip time. obviously we don't want that 💀
In standard TFRC, the TFRC receiver estimates the loss event rate by calculating the average loss interval in packets, then inverting it to get the loss event rate. so for a short loss interval with N packets and K losses, standard TFRC calculates the size of that loss interval as N packets, which contributes a loss event rate of 1/N. but for TFRC-SP, for small loss intervals of at most two round-trip times, if the loss interval consists of N packets including K losses, the size of the loss interval is calculated as N/K, which contributes a loss event rate of K/N instead of 1/N ✨
Section 5.4 of RFC 3448 says the average loss interval calculation includes the most recent loss interval only if this increases the calculated average loss interval, as in the pseudocode below. but in TFRC-SP, the calculated loss interval size for a short loss interval changes depending on how many packet losses have been detected, so it can either increase or decrease as new packets come in. because of that, TFRC-SP adds the restriction that the average loss interval calculation can include the most recent loss interval only if more than two round-trip times have passed since the beginning of that loss interval 👑
let the most recent loss intervals be I_0 to I_n, with I_0 being the interval including the most recent loss event, and the corresponding weights w_i as defined in RFC 3448. in RFC 3448 (Section 5.4), the average loss interval I_mean is calculated as follows:
I_tot0 = 0;
I_tot1 = 0;
W_tot = 0;
for (i = 0 to n-1) {
I_tot0 = I_tot0 + (I_i * w_i);
W_tot = W_tot + w_i;
}
for (i = 1 to n) {
I_tot1 = I_tot1 + (I_i * w_(i-1));
}
I_tot = max(I_tot0, I_tot1);
I_mean = I_tot/W_tot;In TFRC-SP, the average loss interval I_mean is instead calculated as follows:
I_tot0 = 0;
I_tot1 = 0;
W_tot = 0;
for (i = 0 to n-1) {
I_tot0 = I_tot0 + (I_i * w_i);
W_tot = W_tot + w_i;
}
for (i = 1 to n) {
I_tot1 = I_tot1 + (I_i * w_(i-1));
}
If the current loss interval I_0 is "short"
then I_tot = I_tot1;
else I_tot = max(I_tot0, I_tot1);
I_mean = I_tot/W_tot;- A minimum interval between packets: TFRC-SP enforces a Min Interval between packets of 10 ms 🌈 a flow that wants its transport protocol to exceed this Min Interval MUST use the conventional TFRC equations, rather than TFRC-SP. the motivation for this is discussed below 🎀
4. TFRC-SP Discussion ✨
4.1. Response Functions and Throughput Equations
okay so TFRC borrows the TCP throughput equation from [RFC3448] 💅 and the sending rate X comes out in bytes per second, like this:
s
X = ------------------------------------------------------- ,
R*sqrt(2*p/3) + (4*R* (3*sqrt(3*p/8) * p * (1+32*p^2)))where:
sis the packet size in bytes;Ris the round-trip time in seconds;pis the loss event rate, between 0 and 1.0, of the number of loss events as a fraction of the number of packets transmitted.
this equation uses a retransmission timeout (RTO) of 4*R 🦋 and it assumes that the TCP connection sends an acknowledgement for every data packet. it's basically the response function for TCP AND standard TFRC (modulo TCP's range of sender algorithms for setting the RTO), so they're giving very similar vibes ✨
As shown in Table 1 of [RFC3714], for high packet drop rates, this throughput equation gives rough fairness with the most aggressive possible current TCP: a SACK TCP flow using timestamps and Explicit Congestion Notification (ECN). Because it is not recommended for routers to use ECN-marking in highly-congested environments with high packet dropping/marking rates (Section 7 of [RFC3168]), we note that it would be useful to have a throughput equation with a somewhat more moderate sending rate for packet drop rates of 40% and above. the ECN marking thing is a whole mood 😭
the effective response function of TFRC-SP can be derived from the TFRC response function by using a segment size s of 1460 bytes, and using the loss event rate actually experienced by the TFRC-SP flow. in addition, for loss intervals of at most two round-trip times, the loss event rate for TFRC-SP is estimated by counting the actual number of lost or marked packets, rather than by counting loss events. ALSO the sending rate for TFRC-SP is constrained to be at most 100 packets per second 🌸
for an environment with a fixed packet drop rate p, regardless of packet size, the response functions of TCP, TFRC, and TFRC-SP look like this, given in KBps (kilobytes per second), for a flow with a round-trip time of 100 ms 🎀
<-- TCP and Standard TFRC -->
Packet 14-byte 536-byte 1460-byte
DropRate Segments Segments Segments
-------- -------- -------- --------
0.00001 209.25 2232.00 5812.49
0.00003 120.79 1288.41 3355.24
0.00010 66.12 705.25 1836.58
0.00030 38.10 406.44 1058.45
0.00100 20.74 221.23 576.12
0.00300 11.76 125.49 326.79
0.01000 6.07 64.75 168.61
0.03000 2.99 31.90 83.07
0.10000 0.96 10.21 26.58
0.20000 0.29 3.09 8.06
0.30000 0.11 1.12 2.93
0.40000 0.05 0.48 1.26
0.50000 0.02 0.24 0.63
Table 1: Response Function for TCP and TFRC.
Sending Rate in KBps, as a Function of Packet Drop Rate. <---------- TFRC-SP -------->
Packet 14-byte 536-byte 1460-byte
DropRate Segments Segments Segments
-------- -------- -------- --------
0.00001 5.40 57.60 150.00
0.00003 5.40 57.60 150.00
0.00010 5.40 57.60 150.00
0.00030 5.40 57.60 150.00
0.00100 5.40 57.60 150.00
0.00300 5.40 57.60 150.00
0.01000 5.40 57.60 150.00
0.03000 5.40 57.60 83.07
0.10000 5.40 26.58 26.58
0.20000 5.40 8.06 8.06
0.30000 2.93 2.93 2.93
0.40000 1.26 1.26 1.26
0.50000 0.63 0.63 0.63
Table 2: Response Function for TFRC-SP.
Sending Rate in KBps, as a Function of Packet Drop Rate.
Maximum Sending Rate of 100 Packets per Second.the calculations for Tables 1 and 2 use the packet loss rate for an approximation for the loss event rate p. scripts and graphs for the tables are available from [VOIPSIMS] 📊 as the well-known TCP response function in Table 1 shows, the sending rate for TCP and standard TFRC varies linearly with segment size. the TFRC-SP response function shown in Table 2 reflects the maximum sending rate of a hundred packets per second; when not limited by this maximum sending rate, the TFRC-SP flow receives the same sending rate in KBps as the TCP flow with 1460-byte segments, given the same packet drop rate. simulations showing the TCP, standard TFRC, and TFRC-SP sending rates in response to a configured packet drop rate are given in Tables 7, 8, and 9, and are consistent with the response functions shown here 💖
<-- TCP and Standard TFRC -->
Byte 14-byte 536-byte 1460-byte
DropRate Segments Segments Segments
-------- -------- -------- --------
0.0000001 284.76 929.61 1498.95
0.0000003 164.39 536.17 863.16
0.0000010 90.01 292.64 468.49
0.0000030 51.92 167.28 263.68
0.0000100 28.34 88.56 132.75
0.0000300 16.21 46.67 61.70
0.0001000 8.60 19.20 16.25
0.0003000 4.56 4.95 1.70
0.0010000 1.90 0.37 0.15
0.0030000 0.52 0.05 0.06
0.0100000 0.04 0.02 0.06
0.0300000 0.00 0.02 0.06
Table 3: Response Function for TCP and TFRC.
Sending Rate in KBps, as a Function of Byte Drop Rate. <---------- TFRC-SP -------->
Byte 14-byte 536-byte 1460-byte
DropRate Segments Segments Segments
-------- -------- -------- --------
0.0000001 5.40 57.60 150.00
0.0000003 5.40 57.60 150.00
0.0000010 5.40 57.60 150.00
0.0000030 5.40 57.60 150.00
0.0000100 5.40 57.60 132.75
0.0000300 5.40 57.60 61.70
0.0001000 5.40 50.00 16.25
0.0003000 5.40 12.89 1.70
0.0010000 5.40 0.95 0.15
0.0030000 5.40 0.12 0.06
0.0100000 1.10 0.06 0.06
0.0300000 0.13 0.06 0.06
Table 4: Response Function for TFRC-SP.
Sending Rate in KBps, as a Function of Byte Drop Rate.
Maximum Sending Rate of 100 Packets per Second.for Tables 3 and 4, the packet drop rate is calculated as 1-(1-b)^N, for a byte drop rate of b, and a packet size of N bytes. these tables use the packet loss rate as an approximation for the loss event rate p. the TCP response functions shown in Table 3 for fixed byte drop rates are rather different from the response functions shown in Table 1 for fixed packet drop rates; with higher byte drop rates, a TCP connection can have a higher sending rate using smaller packets. 🤯 Table 4 also shows that with fixed byte drop rates, the sending rate for TFRC-SP can be significantly higher than that for TCP or standard TFRC, regardless of the TCP segment size. this is because in this environment, with small packets, TFRC-SP receives a small packet drop rate, but is allowed to send at the sending rate of a TCP or standard TFRC flow using larger packets but receiving the same packet drop rate. main character energy fr 👑
simulations showing TCP, standard TFRC, and TFRC-SP sending rates in response to a configured byte drop rate are given in Appendix B.2. 🧚
4.2. Accounting for Header Size
okay this one is SO good 🫶 [RFC3714] makes the optimistic assumption that the limitation of the network is in bandwidth in bytes per second (Bps), and not in CPU cycles or in packets per second (pps). however, some attention must be paid to the load in pps as well as to the load in Bps. even aside from the Min Interval, TFRC-SP gives the application some incentive to use fewer but larger packets, when larger packets would suffice, by including the bandwidth used by the packet header in the allowed sending rate. iconic 🍓
as an example, a sender using 120-byte packets needs a TCP-friendly rate of 128 Kbps to send 96 Kbps of application data. this is because the TCP-friendly rate is reduced by a factor of s_true/(s_true + H) = 120/160, to account for the effect of packet headers. if the sender suddenly switched to 40-byte data segments, the allowed sending rate would reduce to 64 Kbps of application data; and the use of one-byte data segments would reduce the allowed sending rate to 3.12 Kbps of application data. (in fact, the Min Interval would prevent senders from achieving these rates, since applications using TFRC-SP cannot send more than 100 packets per second.) 🙈
the vibe check: small packets look cheap but the header tax is real, bestie 💅
unless it has a more precise estimate of the header size, TFRC-SP assumes 40 bytes for the header size, although the header could be larger (due to IP options, IPv6, IP tunnels, and the like) or smaller (due to header compression) on the wire. requiring the use of the exact header size in bytes would require significant additional complexity, and would have little additional benefit. TFRC-SP's default assumption of a 40-byte header is sufficient to get a rough estimate of the throughput, and to give the application some incentive not to use an excessive amount of small packets. because we are only aiming at rough fairness, and at a rough incentive for applications, the default use of a 40-byte header in the calculations of the header bandwidth is sufficient for both IPv4 and IPv6. that's the whole tea 🌈✨
4.3. The TFRC-SP Min Interval ✨
Okay so, the header size math already gives apps a reason to send fewer, bigger packets 📦. But there's a second incentive too, and it's kind of a plot twist 🌀. When the bottleneck is limited in packets per second (pps), an app that sends lots of tiny packets is likely to get more packets dropped, so it has to slow itself down. If the congestion is measured in pps, the flow that sends more pps raises its own drop rate and has to adjust its sending rate. BUT the extra congestion from small packets doesn't just hit the flow that sent them. It hits all the competing traffic on that congested link 😭. So those two incentives just aren't strong enough to protect pps-limited networks. Bestie, that's a real problem 💀.
So TFRC-SP includes a Min Interval of 10 ms 🫶. This puts extra limits on how many small packets you can send.
Why 10 ms? Two reasons, and I think they're both very reasonable 💅. First, the apps that currently want to send small packets are VoIP apps, and they send at most one packet every 10 ms. So this restriction doesn't touch today's traffic at all 🎀. Second, best-effort traffic on today's Internet doesn't really need to send small packets more often than once every 10 ms. An app could just wait the 10 ms at the sender and merge the two small packets into one bigger one instead 📦✨. And honestly, that 10 ms of merging delay is probably tiny next to the propagation, transmission, and queueing delays that best-effort traffic already has on the Internet 🌸. So our judgment is that the benefit to the user of having less than 10 ms between packets is outweighed by the benefit to the network of not being flooded with an excessive number of small packets 🌈.
The Min Interval means TFRC-SP just doesn't support apps that send small packets super frequently 🙈. Picture a TFRC flow with a fixed packet size of 100 bytes, a variable sending rate, and a pretty uncongested path. While it sends at most 100 pps, it can use TFRC-SP 🦋. But if it wants to go above 100 pps and keep the same packet size, TFRC-SP won't allow it. It would have to switch to the default TFRC, and its allowed sending rate would drop dramatically and discontinuously 😱. Is that harsh? Maybe a little. But we think it's not only acceptable, it's actually desirable for the global Internet 👑.
Now, the obvious question: what stops a flow from opening multiple connections, each with its own 10 ms Min Interval, to get around the limit? 🤔 Honestly, nothing does. Flows can do this, just like they can already use UDP, open multiple parallel TCP connections, or bring their own congestion control mechanism 🙃. Implementations or middleboxes are free to limit how many parallel TFRC connections get opened to the same destination during congestion, if that seems like a good idea 💜. And flows that open multiple parallel connections have to deal with the usual headaches, like reordering and the like 😵💫.
4.4. Counting Packet Losses 🔍
Here's a fact that sets up everything in this section 💅. A TCP connection can't keep sending multiple packets per round-trip time when there's heavy congestion, and it can't sit in a steady state where multiple packets get dropped every round-trip time 🙅♀️. For TCP, once one or more packets are dropped in a round-trip time, the sending rate gets cut down quickly to less than one packet per round-trip time 📉. And for TCP with Tahoe, NewReno, or SACK congestion control, the response to congestion mostly doesn't depend on how many packets were dropped per round-trip time 🤷♀️.
Because of that, standard TFRC gets the best fairness with TCP, even in really congested environments, by calculating the loss event rate instead of the packet drop rate 🎯. A loss event is one or more packets dropped or marked from a window of data. Iconic 👑.
BUT with TFRC-SP, you can't get fairness with TCP or with standard TFRC just by counting loss events [WBL04] 😭. Why? Because instead of sending one large packet per round-trip time, TFRC-SP could be sending N small packets, where N small packets add up to one large 1500-byte packet 📦📦📦. So the loss measurement used with TFRC-SP has to be able to spot a connection that's consistently getting multiple packet losses or marks per round-trip time 🚨. That way TFRC-SP can respond appropriately 🫶.
Here's how TFRC-SP does it ✨. It calculates the loss event rate by counting at most one loss event in loss intervals longer than two round-trip times. In shorter loss intervals, it counts every packet that was lost or marked 🧮. More specifically, for a short loss interval with N packets, including K lost or marked packets, the loss interval length is N/K, not N 🙌. The average loss interval I_mean is still averaged over the eight most recent loss intervals, as specified in Section 5.4 of RFC 3448 📚. So if eight loss intervals in a row are all short loss intervals with N packets and K losses, the loss event rate comes out to K/N, instead of 1/N 💖.
4.5. The Nominal Packet Size 📏
4.5.1. Packet Size and Packet Drop Rates 🌸
Okay, so the guidelines in Section 3 above set the nominal segment size s to 1460 bytes 🎀. The goal there is fairness, measured as sending rate in bytes per second, against a TCP flow that carries 1460 bytes of application data per packet but sees the same packet drop rate. This assumes a TCP flow with 1460-byte segments will have a higher sending rate than one with smaller segments 📈. And that assumption does hold in an environment where the packet drop rate doesn't depend on packet size. But it doesn't necessarily hold where larger packets are more likely to get dropped than small ones 😬.
The table below shows simulation results for standard (SACK) TCP flows, where, for each byte, the packet containing that byte is dropped with probability p 🧪. For packet drop rates up to around 10%, the TCP response function approximation says the sending rate in bytes per RTT is roughly 1.2 s/sqrt(p), for a packet size of s bytes and packet drop rate p 🧮.
Now, with a fixed byte drop rate p1 and a TCP packet size of s bytes, the packet drop rate is roughly s*p1. That gives a sending rate in bytes per RTT of roughly 1.2 sqrt(s)/sqrt(p1). So for TCP in an environment with a fixed byte drop rate, the sending rate should grow roughly as sqrt(s), for packet drop rates up to around 10% 📊.
Each row of Table 5 below is a separate simulation with ten TCP connections and a fixed byte drop rate of 0.0001, using a different segment size each time 🧁. For each simulation, the TCP sending rate and goodput are averaged over the ten flows. Just like the paragraph above predicts, the TCP sending rate grows somewhat less than linearly as packet size goes up, all the way to 1460 bytes, which corresponds to a packet drop rate of 13% 📈. After that, bigger packets make the sending rate drop, because the TCP connection enters exponential backoff of the retransmit timer 😵.
Segment Packet TCP Rates (Kbps)
Size (B) DropRate SendRate Goodput
-------- -------- -------- -------
14 0.005 6.37 6.34
128 0.016 30.78 30.30
256 0.028 46.54 44.96
512 0.053 62.43 58.37
1460 0.134 94.15 80.02
4000 0.324 35.20 21.44
8000 0.531 15.36 5.76
Table 5: TCP Median Send Rate vs. Packet Size I:
Byte Drop Rate 0.0001Table 6 below shows similar results for a byte drop rate of 0.001 🍓. This time, the TCP sending rate grows as packet size goes up, but only until 128 bytes, which corresponds to a packet drop rate of 16% 📈. After that, the sending rate drops and then climbs again, because the TCP connection enters exponential backoff of the retransmit timer 🎢. Look at the 4000 and 8000 byte rows: the TCP sending rate goes up, but the goodput stays essentially zero 💀. That makes sense, since almost all of the packets being sent are getting dropped 🙈.
Segment Packet TCP Rates (Kbps)
Size (B) DropRate SendRate Goodput
-------- -------- -------- -------
14 0.053 1.68 1.56
128 0.159 7.66 6.13
256 0.248 6.21 4.32
512 0.402 1.84 1.11
1460 0.712 1.87 0.47
4000 0.870 3.20 0.00
8000 0.890 5.76 0.00
Table 6: TCP Median Send Rate vs. Packet Size II:
Byte Drop Rate 0.001So what does this TCP behavior with a fixed byte drop rate tell us? 🤔 Instead of aiming for a TFRC-SP flow to match the sending rate in bytes per second of a TCP flow using 1500-byte packets, it makes more sense to aim for an ideal where the TFRC-SP flow matches a TCP flow using the optimal packet size, as long as the packet size is at most 1500 bytes 🎯. This doesn't mean we have to change the TFRC-SP mechanisms for computing the allowed transmit rate. It just means that when we evaluate TFRC-SP fairness, we should compare against the TCP flow that uses the optimal packet size for that environment (still at most 1500 bytes) 💜.
4.5.2. Fragmentation and the Path MTU 🧩
This document doesn't say how TFRC-SP should behave with regard to packet fragmentation and Path MTU Discovery (PMTUD) 🙃. So should the transport protocol using TFRC-SP use PMTUD info to set an upper bound on segment size? Should it allow packets to be fragmented in the network? Those are questions we leave to the transport protocol 🫶. For example, DCCP requires that endpoints keep track of the current PMTU, and says fragmentation shouldn't be the default (Section 14 of [RFC4340]) 💅.
4.5.3. The Nominal Segment Size and the Path MTU ✨
okay so picture this 🎀 TFRC-SP is using a nominal segment size s of 1460 bytes, but the path's TCP MSS is actually only 536 bytes 😭 then the TFRC-SP flow could get almost three times the bandwidth (in bytes per second) of a TCP flow using a 536-byte MSS. Now flip it: in an environment with an MSS close to 4000 bytes, a TCP flow could get almost three times the bandwidth of a TFRC-SP flow with s of 1460 bytes. We feel like these "unfairness" levels, with factors of two or three, are totally acceptable 💅 They won't result in one flow hogging all the bandwidth and shutting out the competing TCP or TFRC-SP flow, so we're calm about it 🦋
All IPv4 end hosts are required to accept and reassemble IP packets of size 576 bytes [RFC791], but IPv4 links don't necessarily have to support that packet size 🧚 In slow networks, the largest possible packet may take a considerable amount of time to send [RFC3819], so a smaller MTU may be desirable, e.g., hundreds of bytes. If the first-hop link had a small MTU, then TCP would choose an appropriately small MSS [RFC879]. [RFC1144] quotes cases of very low link speeds where the MSS may be tens of bytes (and notes this is an extreme case) 🙈
We note that if TFRC-SP is used over a path with an MTU considerably smaller than 576 bytes, and the TFRC-SP flow uses a nominal segment size s of 1460 bytes, then the TFRC-SP flow could receive considerably more than three times the bandwidth of competing TCP flows. Not a vibe 💀
If TFRC-SP is used with a nominal segment size s of less than 536 bytes (because the path MTU is known to be less than 576 bytes), then TFRC-SP is likely to be of minimal benefit to applications. If TFRC-SP was to be used on paths that have a path MTU of considerably less than 576 bytes, and the transport protocol was not required to keep track of the path MTU, this could result in the TFRC-SP flow using the default nominal segment size of 1460 bytes, and as a result receiving considerably more bandwidth than competing TCP flows. So, bestie, TFRC-SP is not recommended for use with transport protocols that don't maintain some knowledge of the path MTU 🫶
4.6. The Loss Interval Length for Short Loss Intervals
For a TFRC-SP receiver, the guidelines from Section 6 of RFC 3448 govern when the receiver should send feedback messages. In particular, from [RFC3448], "a feedback packet should ... be sent whenever a new loss event is detected without waiting for the end of an RTT". In addition, feedback packets are sent at least once per RTT 🌸
For a TFRC-SP connection with a short current loss interval (less than two round-trip times), it is possible for the loss interval length calculated for that loss interval to change in odd ways as additional packet losses in that loss interval are detected. Lowkey chaotic 😭 To prevent unnecessary oscillations in the average loss interval, Section 3 specifies that the current loss interval can be included in the calculation of the average loss interval only if the current loss interval is longer than two round-trip times ✨
5. A Comparison with RFC 3714
RFC 3714 considers the problems of fairness, potential congestion collapse, and poor user quality that could occur with the deployment of non-congestion-controlled IP telephony over congested best-effort networks. The March 2004 document cites ongoing efforts in the IETF, including work on TFRC and DCCP. RFC 3714 recommends that for best-effort traffic with applications that have a fixed or minimum sending rate, the application or transport protocol should monitor the packet drop rate, and discontinue sending for a period if the steady-state packet drop rate significantly exceeds the allowed threshold for that minimum sending rate 📞
In determining the allowed packet drop rate for a fixed sending rate, RFC 3714 assumes that an IP telephony flow should be allowed to use the same sending rate in bytes per second as a 1500-byte packet TCP flow experiencing the same packet drop rate as that of the IP telephony flow. As an example, following this guideline, a VoIP connection with a round-trip time of 0.1 sec and a minimum sending rate of 64 Kbps would be required to terminate or suspend when the persistent packet drop rate significantly exceeded 25%. Yikes 😭
One limitation of the lack of fine-grained control in the minimal mechanism described in RFC 3714 is that an IP telephony flow would not adapt its sending rate in response to congestion. In contrast, with TFRC-SP, a small-packet flow would reduce its sending rate somewhat in response to moderate packet drop rates, possibly avoiding a period with unnecessarily-heavy packet drop rates. That's the glow-up 🦋
Because RFC 3714 assumes that the allowed packet drop rate for an IP telephony flow is determined by the sending rate that a TCP flow would use with the same packet drop rate, the minimal mechanism in RFC 3714 would not provide fairness between TCP and IP telephony traffic in an environment where small packets are less likely to be dropped than large packets. In such an environment, the small-packet IP telephony flow would make the optimistic assumption that a large-packet TCP flow would receive the same packet drop rate as the IP telephony flow, and as a result the small-packet IP telephony flow would receive a larger fraction of the link bandwidth than a competing large-packet TCP flow. Not fair, not fair 🙅♀️
6. TFRC-SP with Applications that Modify the Packet Size
One possible use for TFRC-SP would be with applications that maintain a fixed sending rate in packets per second, but modify their packet size in response to congestion. TFRC-SP monitors the connection's packet drop rate, and determines the allowed sending rate in bytes per second. Given an application with a fixed sending rate in packets per second, the TFRC-SP sender could determine the data packet size that would be needed for the sending rate in bytes per second not to exceed the allowed sending rate. In environments where the packet drop rate is affected by the packet size, decreasing the packet size could also result in a decrease in the packet drop rate experienced by the flow 🎀
There are many questions about how an adaptive application would use TFRC-SP that are beyond the scope of this document. In particular, an application might wish to avoid unnecessary reductions in the packet size. In this case, an application might wait for some period of time before reducing the packet size, to avoid an unnecessary reduction in the packet size. The details of how long an application might wait before reducing the packet size can be addressed in documents that are more application-specific 🧚
Similarly, an application using TFRC-SP might only have a few packet sizes that it is able to use. In this case, the application might not reduce the packet size until the current packet drop rate has significantly exceeded the packet drop rate threshold for the current sending rate, to avoid unnecessary oscillations between two packet sizes and two sending rates. Again, the details will have to be addressed in documents that are more application-specific 💜
Because the allowed sending rate in TFRC-SP is in bytes per second rather than in packets per second, there is little opportunity for applications to manipulate the packet size in order to "game" the system. This differs from TFRC in CCID 3 (Section 5.3 of [RFC4342]), where the allowed sending rate is in packets per second. In particular, a TFRC-SP application that sends small packets for a while, building up a fast sending rate, and then switches to large packets, would not increase its overall sending rate by the change of packet size. No sneaky plans allowed, periodt 💅
7. Simulations
This section describes the performance of TFRC-SP in simulation scenarios with configured packet or byte drop rates, and in scenarios with a range of queue management mechanisms at the congested link. The simulations, described in detail in Appendix B, explore environments where standard TFRC significantly limits the throughput of small-packet flows, and TFRC-SP gives the desired throughput. The simulations also explore environments where standard TFRC allows small-packet flows to receive good performance, while TFRC-SP is overly aggressive 📊
The general lessons from the simulations are as follows.
- In scenarios where large and small packets receive similar packet drop rates, TFRC-SP gives roughly the desired sending rate (Appendix B.1, B.2) ✨
- In scenarios where each byte is equally likely to be dropped, standard TFRC gives reasonable fairness between small-packet TFRC flows and large-packet TCP flows (Appendix B.2) 🌈
- In scenarios where small packets are less likely to be dropped than large packets, TFRC-SP does not give reasonable fairness between small-packet TFRC-SP flows and large-packet TCP flows; small-packet TFRC-SP flows can receive considerably more bandwidth than competing large-packet TCP flows, and in some cases large-packet TCP flows can be essentially starved by competing small-packet TFRC-SP flows (Appendix B.2, B.3, B.4) 😭
- Scenarios where small packets are less likely to be dropped than large packets include those with Drop-Tail queues in bytes, and with AQM mechanisms in byte mode (Appendix B.3, B.4). It has also been reported that wireless links are sometimes good enough to let small packets through, while larger packets have a much higher error rate, and hence a higher drop probability [S05] 🦋
8. General Discussion ✨
Dropping rates for small packets: okay so the whole goal of TFRC-SP is for small-packet TFRC-SP flows to get rough fairness with large-packet TCP flows in sending rate in bps 💅, in a scenario where every packet has roughly the same chance of getting dropped. But if large packets are more likely to get dropped than small ones, or flows with a bursty sending rate get dropped more than flows with a smooth one 🙈, small-packet TFRC-SP flows can totally snatch significantly more bandwidth than competing large-packet TCP flows. Not the fairness we want, bestie 😭
The accuracy of the TCP response function used in TFRC: for applications with a maximum sending rate of 96 Kbps or less (i.e., packets of at most 120 bytes), TFRC-SP only restricts the sending rate when the packet drop rate is fairly high, e.g., greater than 10%. [Derivation: A TFRC-SP flow with a 200 ms round-trip time and a maximum sending rate with packet headers of 128 Kbps would have a sending rate in bytes per second equivalent to a TCP flow with 1460-byte segments sending 2.2 packets per round-trip time. From Table 1 of RFC 3714, this sending rate can be sustained with a packet drop rate slightly greater than 10%.] In this high-packet-drop regime, the performance of TFRC-SP is determined in part by the accuracy of the TCP response function in representing the actual sending rate of a TCP connection. It's giving accuracy matters here 🧚
In the regime of high packet drop rates, TCP performance is also affected by the TCP algorithm (e.g., SACK or not), the minimum RTO, the use of Limited Transmit (or lack thereof), the use of ECN, and the like. It is good to ensure that simulations or experiments exploring fairness include the exploration of fairness with the most aggressive TCP mechanisms conformant with the current standards. Our simulations use SACK TCP with Limited Transmit and with a minimum RTO of 200 ms. The simulation results are largely the same with or without timestamps; timestamps were not used for simulations reported in this paper. We didn't use TCP with ECN in setting the target sending rate for TFRC, because, as explained in Appendix B.1, our expectation is that in high packet drop regimes, routers will drop rather than mark packets, either from policy or from buffer overflow. 🎀
Fairness with different packet header sizes: in an environment with IPv6 and/or several layers of network-layer tunnels (e.g., IPsec, Generic Routing Encapsulation (GRE)), the packet header could be 60, 80, or 100 bytes instead of the default 40 bytes assumed in Section 3. For an application with small ten-byte data segments, this means that the actual packet size could be 70, 90, or 110 bytes, instead of the 50 bytes assumed by TFRC-SP in calculating the allowed sending rate. Thus, a TFRC-SP application with large headers could receive more than twice the bandwidth (including the bandwidth used by headers) than a TFRC-SP application with small headers. We do not expect this to be a problem; in particular, TFRC-SP applications with large headers will not significantly degrade the performance of competing TCP applications, or of competing TFRC-SP applications with small headers. 🦋
General issues for TFRC: the congestion control mechanisms in TFRC and TFRC-SP limit a flow's sending rate in packets per second. Simulations by Tom Phelan [P04] explore how such a limitation in sending rate can lead to unfairness for the TFRC flow in some scenarios, e.g., when competing with bursty TCP web traffic, in scenarios with low levels of statistical multiplexing at the congested link. 🌸
9. Security Considerations 🔐
There are no new security considerations introduced in this document.
The issues of the fairness of TFRC-SP with standard TFRC and TCP in a range of environments, including those with byte-based queue management at the congested routers, are discussed extensively in the main body of this document.
General security considerations for TFRC are discussed in RFC 3448. As with TFRC in RFC 3448, TFRC-SP is not a transport protocol in its own right, but a congestion control mechanism that is intended to be used in conjunction with a transport protocol. Therefore, security primarily needs to be considered in the context of a specific transport protocol and its authentication mechanisms. As discussed for TFRC in RFC 3448, any transport protocol that uses TFRC-SP needs to protect against spoofed feedback, and to protect the congestion control mechanisms against incorrect information from the receiver. Again as discussed for TFRC in RFC 3448, we expect that protocols incorporating ECN with TFRC-SP will want to use the ECN nonce [RFC3540] to protect the sender from the accidental or malicious concealment of marked packets
Security considerations for DCCP's Congestion Control ID 3, TFRC Congestion Control, the transport protocol that uses TFRC, are discussed in [RFC4342]. That document extensively discussed the mechanisms the sender can use to verify the information sent by the receiver, including the use of the ECN nonce. 🛡️
10. Conclusions 🎉
This document has specified TFRC-SP, a Small-Packet (SP) variant of TFRC, designed for applications that send small packets, with at most a hundred packets per second, but that desire the same sending rate in bps as a TCP connection experiencing the same packet drop rate but sending packets of 1500 bytes. TFRC-SP competes reasonably well with large-packet TCP and TFRC flows in environments where large-packet flows and small-packet flows experience similar packet drop rates, but receives more than its share of the bandwidth in bps in environments where small packets are less likely to be dropped or marked than are large packets. As a result, TFRC-SP is experimental, and is not intended for widespread deployment at this time in the global Internet. Experimental era, but we're getting there 🚀
In order to allow experimentation with TFRC-SP in the Datagram Congestion Control Protocol (DCCP), an experimental Congestion Control IDentifier (CCID) will be used, based on TFRC-SP but specified in a separate document. 💜
11. Acknowledgements 🫶
We thank Tom Phelan for discussions of TFRC-SP and for his paper exploring the fairness between TCP and TFRC-SP flows. We thank Lars Eggert, Gorry Fairhurst, Mark Handley, Miguel Garcia, Ladan Gharai, Richard Nelson, Colin Perkins, Juergen Quittek, Pete Sholander, Magnus Westerlund, and Joerg Widmer for feedback on earlier versions of this document. We also thank the DCCP Working Group for feedback and discussions. Thank you all, you literal legends 🌈✨
Appendix A. Related Work on Small-Packet Variants of TFRC 📚
Other proposals for variants of TFRC for applications with variable packet sizes include [WBL04] and [V00]. [V00] proposed that TFRC should be modified so that flows are not penalized by sending smaller packets. In particular, [V00] proposes using the path MTU in the TCP-friendly equation, instead of the actual packet size used by TFRC, and counting the packet drop rate by using an estimation algorithm that aggregates both packet drops and received packets into MTU-sized units.
[WBL04] also argues that adapting TFRC for variable packet sizes by just using the packet size of a reference TCP flow in the TFRC equation would not suffice in the high-packet-loss regime; such a modified TFRC would have a strong bias in favor of smaller packets, because multiple lost packets in a single round-trip time would be aggregated into a single packet loss. [WBL04] proposes modifying the loss measurement process to account for the bias in favor of smaller packets.
The TFRC-SP variant of TFRC proposed in our document differs from [WBL04] in restricting its attention to flows that send at most 100 packets per second. The TFRC-SP variant proposed in our document also differs from the straw proposal discussed in [WBL04] in that the allowed bandwidth includes the bandwidth used by packet headers.
[WBL04] discusses four methods for modifying the loss measurement process, "unbiasing", "virtual packets", "random sampling", and "Loss Insensitive Period (LIP) scaling". [WBL04] finds only the second and third methods sufficiently robust when the network drops packets independently of packet size. They find only the second method sufficiently robust when the network is more likely to drop large packets than small packets. Our method for calculating the loss event rate is somewhat similar to the random sampling method proposed in [WBL04], except that randomization is not used; instead of randomization, the exact packet loss rate is computed for short loss intervals, and the standard loss event rate calculation is used for longer loss intervals. [WBL04] includes simulations with a Bernoulli loss model, a Bernoulli loss model with a drop rate varying over time, and a Gilbert loss model, as well as more realistic simulations with a range of TCP and TFRC flows. 🧪
[WBL04] produces both a byte-mode and a packet-mode variant of the TFRC transport protocol, for connections over paths with per-byte and per-packet dropping respectively. We would argue that in the absence of transport-level mechanisms for determining whether the packet dropping in the network is per-packet, per-byte, or somewhere in between, a single TFRC implementation is needed, independently of the packet-dropping behaviors of the routers along the path. It would of course be preferable to have a mechanism that gives roughly acceptable behavior, to the connection and to the network as a whole, on paths with both per-byte and per-packet dropping (and on paths with multiple congested routers, some with per-byte dropping mechanisms, some with per-packet dropping mechanisms, and some with dropping mechanisms that lie somewhere between per-byte and per-packet).
An important contribution would be to investigate the range of behaviors actually present in today's networks, in terms of packet dropping as a function of packet size. 🔍
Appendix B. Simulation Results 📊
This appendix reports on the simulation results outlined in Section 7. TFRC-SP has been added to the NS simulator, and is illustrated in the validation test ./test-all-friendly in the directory tcl/tests. The simulation scripts and graphs for the simulations in this document are available at [VOIPSIMS]. The receipts are all there, bestie 💖
B.1. Simulations with Configured Packet Drop Rates 🎀
Okay bestie, this part is all about simulations 🧪 We compared the throughput of standard (SACK) TCP flows, TCP flows with timestamps and ECN, TFRC-SP flows, and standard TFRC (Stnd TFRC) flows. The router was set up to randomly drop or mark packets at a specified rate, no matter how big the packet was 🎲 For each drop rate, we recorded a flow's average sending rate in Kbps over the second half of a 100-second simulation, averaged over ten flows 📊
Packet Send Rates (Kbps, 1460B seg)
DropRate TCP ECN TCP TFRC
-------- -------- -------- --------
0.001 2020.85 1904.61 982.09
0.005 811.10 792.11 878.08
0.01 515.45 533.19 598.90
0.02 362.93 382.67 431.41
0.04 250.06 252.64 284.82
0.05 204.48 218.16 268.51
0.1 143.30 148.41 146.03
0.2 78.65 93.23* 55.14
0.3 26.26 59.65* 32.87
0.4 9.87 47.79* 25.45
0.5 3.53 32.01* 18.52
* ECN scenarios marked with an asterisk are not realistic,
as routers are not recommended to mark packets when packet
drop/mark rates are 20% or higher.
Table 7: Send Rate vs. Packet Drop Rate I:
1460B TFRC Segments
(1.184 Kbps Maximum TFRC Data Sending Rate)Table 7 is the one that shows how fair TCP and TFRC are to each other when both flows use the same packet size 🤝 For example, a drop rate of 0.1 means 10% of the TCP and TFRC packets get dropped 📉 The TFRC flow is set to send at most 100 packets per second. Honestly the fairness is really good 💖 until the drop rates hit 40% and 50%, and then the TFRC flow gets three to five times more bandwidth than standard TCP 😳 An ECN TCP flow would actually get a higher throughput than TFRC at these high rates, if ECN-marking was still happening instead of packet dropping ✨ BUT we don't use the ECN TCP rate as the target sending rate for TFRC in these high-drop scenarios, because routers are advised to drop rather than mark packets during high congestion (Section 7 of [RFC3168]) 🙅♀️ Plus, buffer overflow is likely when the dropping and marking rates are this high, which forces routers to drop packets instead of ECN-marking them 🌊
< - - - - - - Send Rates (Kbps) - - - - - >
Packet TCP ECN TCP TFRC-SP Stnd TFRC
DropRate (1460B seg) (1460B seg) (14B seg) (14B seg)
-------- ----------- ----------- --------- ---------
0.001 1787.54 1993.03 17.71 17.69
0.005 785.11 823.75 18.11 17.69
0.01 533.38 529.01 17.69 17.80
0.02 317.16 399.62 17.69 13.41
0.04 245.42 260.57 17.69 8.84
0.05 216.38 223.75 17.69 7.63
0.1 142.75 138.36 17.69 4.29
0.2 58.61 91.54* 17.80 1.94
0.3 21.62 63.96* 10.26 1.00
0.4 10.51 41.74* 4.78 0.77
0.5 1.92 19.03* 2.41 0.56
* ECN scenarios marked with an asterisk are not realistic,
as routers are not recommended to mark packets when packet
drop/mark rates are 20% or higher.
Table 8: Send Rate vs. Packet Drop Rate II:
14B TFRC Segments
(5.6 Kbps Maximum TFRC Data Sending Rate)Now for Table 8, which is where the tiny-packet drama happens 🔬 Each TFRC-SP flow has a max data sending rate of 5.6 Kbps, using 14-byte data packets with a 32-byte packet header for DCCP and IP 📦 Each TCP flow has a receive window of 100 packets and 1460-byte data packets, with a 40-byte packet header for TCP and IP. The TCP flow uses SACK TCP with Limited Transmit, but no timestamps or ECN. Every flow has a round-trip time of 240 ms when there's no queueing delay ⏱️
Here's the tea ☕ The TFRC sending rate in Table 8 is for the 14-byte data packet plus the 32-byte header, so only 30% of it is actual data 😭 With a drop rate of p, only a fraction 1-p of that data makes it to the receiver. That means the TFRC data receive rate can be a lot lower than the sending rate in the table 🥲 TCP uses big packets, so 97% of its sending rate is data, and the same fraction 1-p of that data reaches the receiver 💅
Table 8 shows that for the 5.6 Kbps data stream, Standard TFRC gives a really poor sending rate in bps compared to the large-packet TCP connection 💀 TFRC-SP, on the other hand, is relatively close to the goal of fairness in bps with TCP 🫶
Also, with TFRC-SP the 5.6 Kbps stream doesn't cut its sending rate until drop rates go above 20%, which is exactly what we wanted 🌸 At 30-40% drop rates, TFRC-SP is somewhat below the average large-packet TCP flow, and at 50% it's somewhat above it 🎢
< - - - - - - Send Rates (Kbps) - - - - - >
Packet TCP ECN TCP TFRC-SP Stnd TFRC
DropRate (1460B seg) (1460B seg) (200B seg) (200B seg)
-------- ----------- ----------- ---------- ----------
0.001 1908.98 1869.24 183.45 178.35
0.005 854.69 835.10 185.06 138.06
0.01 564.10 531.10 185.33 92.43
0.02 365.38 369.10 185.57 62.18
0.04 220.80 257.81 185.14 45.43
0.05 208.97 219.41 180.08 39.44
0.1 141.67 143.88 127.33 21.96
0.2 62.66 91.87* 54.66 9.40
0.3 16.63 65.52* 24.50 4.73
0.4 6.62 42.00* 13.47 3.35
0.5 1.01 21.34* 10.51 2.92
* ECN scenarios marked with an asterisk are not realistic,
as routers are not recommended to mark packets when packet
drop/mark rates are 20% or higher.
Table 9: Sending Rate vs. Packet Drop Rate III:
200B TFRC Segments
(160 Kbps Maximum TFRC Data Sending Rate)Table 9 covers the case where the TFRC flow uses 200-byte data packets, with a max data sending rate of 160 Kbps 🧁 Just like in Table 8, Standard TFRC is pretty bad here, but TFRC-SP is basically on target for drop rates up to 30% 🎯 Again as expected, at 40-50% drop rates the TFRC-SP sending rate is a bit higher than we'd like 📈
One last thing 💜 For these simulations, a TCP connection using timestamps has a sending rate that's similar to the one shown for a standard TCP connection without timestamps 🦋
B.2. Simulations with Configured Byte Drop Rates ✨
okay so in this section we get to play with simulations where the router is set up to drop or mark each byte at a specified rate 🎀 and here's the twist: when one byte gets picked, the WHOLE packet holding that byte gets dropped (or marked) 💀 so one unlucky byte can take out the entire packet
< - - - - - Send Rates (Kbps) - - - - - >
Byte TCP TFRC-SP Stnd TFRC
DropRate SegSize TCP ECN TCP (14B seg) (14B seg)
-------- ------- -------- -------- --------- ---------
0.00001 1460 423.02 431.26 17.69 17.69
0.0001 1460 117.41 114.34 17.69 17.69
0.001 128 10.78 11.50 17.69 8.37
0.005 14 1.75 2.89 18.39 1.91
0.010 1460 0.31 0.26 7.07 0.84
0.020 1460 0.29 0.26 1.61 0.43
0.040 1460 0.12 0.26* 0.17 0.12
0.050 1460 0.15 0.26* 0.08 0.06
* ECN scenarios marked with an asterisk are not realistic,
as routers are not recommended to mark packets when packet
drop/mark rates are 20% or higher.
TFRC's maximum data sending rate is 5.6 Kbps.
Table 10: Sending Rate vs. Byte Drop RateTable 10 shows the TCP and TFRC send rates for a bunch of byte drop rates 🦋 For each byte drop rate, it shows the TCP sending rate (with and without ECN) for whichever TCP segment size gives TCP the highest sending rate. The sims ran with TCP segments of 14, 128, 256, 512, and 1460 bytes. So for a byte drop rate of 0.00001 the table uses 1460-byte data segments, but for a byte drop rate of 0.001 it switches to 128-byte data segments. For every byte drop rate, Table 10 also shows the TFRC-SP and Standard TFRC sending rates too. The big takeaway: with configured byte drop rates, TFRC-SP gives an unfair advantage to the TFRC-SP flow 😤 while Standard TFRC gives basically the performance we want 💅
TCP Pkt TFRC Pkt
Byte DropRate DropRate TCP/TFRC
DropRate (1460B seg) (14B seg) Pkt Drop Ratio
-------- ----------- --------- --------------
0.00001 0.015 0.0006 26.59
0.0001 0.13 0.0056 24.94
0.001 0.77 0.054 14.26
0.005 0.99 0.24 4.08
0.01 1.00 0.43 2.32
0.05 1.00 0.94 1.05
Table 11: Packet Drop Rate Ratio vs. Byte Drop RateTable 11 converts the byte drop rate p into packet drop rates for the TCP and TFRC packets 🧚 The packet drop rate for an N-byte packet is 1-(1-p)^N. So a byte drop rate of 0.001, where each byte has a 0.001 chance of being dropped, turns into a packet drop rate of 0.77 (that's 77%) for the 1500-byte TCP packets, and a packet drop rate of 0.054 (that's 5.4%) for the 56-byte TFRC packets 🍓
The right column of Table 11 shows the ratio between the TCP packet drop rate and the TFRC packet drop rate 👑 At low byte drop rates, this ratio is close to 26.8, which is the ratio between the TCP and TFRC packet sizes. At high byte drop rates, where even most of the small TFRC packets are likely to get dropped, the drop ratio heads toward 1. As Table 10 shows, with byte drop rates the Standard TFRC sending rate is close to optimal 🫶 It competes fairly with a TCP connection that uses the optimal packet size within the allowed range. The TFRC-SP connection, on the other hand, gets more than its share of the bandwidth 👀 though it does back off its sending rate for a byte drop rate of 0.01 or more (which corresponds to a TFRC-SP packet drop rate of 0.43.
Table 10 basically shows three separate regions 🌈
In the low-congestion region, with byte drop rates below 0.0001, the TFRC-SP connection is sending at its maximum sending rate 🚀 In this region the optimal TCP connection is the one with 1460-byte segments, and the TCP sending rate goes as 1/sqrt(p), for packet drop rate p. This low-congestion region holds for packet drop rates up to 10-15%.
In the middle region of Table 10, with byte drop rates from 0.0001 to 0.005, the optimal TCP segment size depends on the byte drop rate 🦋 In particular, the optimal TCP segment size seems to be the one that keeps the packet drop rate at most 15%, which keeps the TCP connection in the regime controlled by a 1/sqrt(p) sending rate, for packet drop rate p. For a TCP packet size of S bytes (headers included) and a byte drop rate of B, the packet drop rate is roughly B*S. So for a given byte drop rate in this regime, if the optimal TCP performance happens with a packet size chosen to give a packet drop rate of at most 15%, keeping the TCP connection out of the regime of exponential backoffs of the retransmit timer, then this requires B*S = 0.15, or S = 0.15/B.
In the high-congestion regime of Table 10 (high congestion, with byte drop rates of 0.01 and higher), TCP performance is dominated by the exponential backoff of the retransmit timer no matter what segment size you pick 😭 Even a 40-byte packet with a zero-byte data segment would have a packet drop rate of at least 33%. In this regime, the optimal TCP sending rate comes with a large, 1460-byte data segment, but the TCP sending rate is low with any segment size, way less than one packet per round-trip time.
We note that in this regime, a larger packet gives a higher TCP sending rate, while a smaller packet gives a better goodput rate 🍑
In general, Tables 8 and 9 show good performance for TFRC-SP in environments with stable packet drop rates, where the decision to drop a packet is independent of the packet size ✨ However, in some environments the packet size might affect how likely a packet is to get dropped 🙈 For example, under heavy congestion with a Drop Tail queue that has a fixed number of bytes rather than a fixed number of packet-sized buffers, small packets might be more likely than large packets to find room at the end of an almost-full queue. As a further complication, in a scenario with Active Queue Management, the AQM mechanism could either be in packet mode (dropping each packet with equal probability) or in byte mode (dropping each byte with equal probability) 🧠 Sections B.3 and B.4 show simulations with packets dropped at Drop-Tail or AQM queues, rather that from a probabilistic process 💜
B.3. Packet Dropping Behavior at Routers with Drop-Tail Queues ✨
okay so here's a question that matters a LOT 🫶 comparing how fast two flows send is tricky when they use different packet sizes, because the packet size itself can change how often packets get dropped [V00, WBL04]. 🧐
TFRC was originally designed to be roughly fair to TCP, back when TFRC and TCP flows had the same packet size and got hit with the same packet drop rate. 💅 Once you bring in flows with different packet sizes, it suddenly matters whether the drop rate depends on packet size. So are small packets just as likely to get dropped as big TCP packets, or are they less likely [WBL04]? And how does the way the path drops packets connect to the loss event measurements TFRC uses? 🤔 Super important questions, bestie.
< - - - - - Send Rates in Kbps - - - - >
Web TCP (1460B seg) TFRC-SP (200B seg)
Sessions DropRate SendRate DropRate SendRate
-------- -------- -------- -------- --------
10 0.04 316.18 0.05 183.05
25 0.07 227.47 0.07 181.23
50 0.08 181.10 0.08 178.32
100 0.14 85.97 0.12 151.42
200 0.17 61.20 0.14 73.88
400 0.20 27.79 0.18 36.81
800 0.29 3.50 0.27 16.33
1600 0.37 0.63 0.33 6.29
Table 12: Drop and Send Rates for Drop-Tail Queues in PacketsTable 12 shows the results from the second half of 100-second simulations 🌸. There were five TCP connections and five TFRC-SP connections, all competing with web traffic on a topology with a 3 Mbps shared link. The TFRC-SP app sends 200-byte data packets every 10 ms, which works out to a max data rate of 160 Kbps. The five long-lived TCP connections use 1460-byte data packets, and the web traffic uses 512-byte ones. The five TCP connections have round-trip times from 40 to 240 ms, and the five TFRC connections get the exact same set of round-trip times. 🙌 The SACK TCP connections use the default NS simulator parameters, plus Limited Transmit and a minimum RTO of 200 ms. Adding timestamps to TCP didn't change the results much at all. The simulations also include reverse-path traffic, so some small control packets show up on the forward path, and there's some queueing delay on the reverse path. The number of web sessions gets varied to create different levels of congestion. Also, the Drop-Tail queue is measured in packets, so each arriving packet needs exactly one buffer, no matter how big it is. 📦
Table 12 shows the average TCP and TFRC sending rates, each averaged across the five flows. As expected, the TFRC-SP flows see drop rates similar to the TCP flows 💖. But the TFRC-SP flows get higher throughput than the TCP flows once packet drop rates hit 25% or higher. 👀
< - - - - - Send Rates in Kbps - - - - >
Web TCP (1460B seg) TFRC-SP (200B seg)
Sessions DropRate SendRate DropRate SendRate
-------- -------- -------- -------- --------
10 0.061 239.81 0.004 185.19
25 0.089 189.02 0.006 184.95
50 0.141 99.46 0.013 185.07
100 0.196 16.42 0.022 183.77
200 0.256 4.46 0.032 181.98
400 0.291 4.61 0.051 151.88
800 0.487 1.01 0.078 113.10
1600 0.648 0.67 0.121 65.17
Table 13: Drop and Send Rates for Drop-Tail Queues in Bytes I:
1460B TCP SegmentsBut here's where it gets spicy 🌶️. The fairness results can change a LOT if the Drop-Tail queue at the congested output link is measured in bytes instead of packets. 😱 With a packet-based queue, there's a fixed number of buffers, and each buffer holds exactly one packet, no matter how many bytes that packet is. With a byte-based queue, there's a fixed number of bytes, so a queue that's almost full might have room for a small packet but not a big one. 🫠 That means that in a simulation with a byte-based Drop-Tail queue, big packets are more likely to get dropped than small ones. The NS simulator doesn't have a more realistic middle-ground model yet, where the queue has a fixed number of buffers, each buffer has a fixed number of bytes, and each packet needs one or more free buffers. In that model, a small packet would use one buffer while a bigger packet would need several. 🧩
The scenarios in Table 13 match the ones in Table 12, except the Drop-Tail queue is measured in bytes instead of packets. So again, five TCP connections and five TFRC-SP connections compete with web traffic on a 3 Mbps shared link, with each TFRC-SP app sending 200-byte data packets every 10 ms, for a max data rate of 160 Kbps. The number of web sessions gets varied to create different levels of congestion. The difference is that instead of Drop-Tail queues that hold at most a hundred packets, the routers' Drop-Tail queues each hold at most 50,000 bytes (computed in NS as a hundred packets times the mean packet size of 500 bytes). 🎀
Now here's the tea ☕. With a byte-based Drop-Tail queue, the TFRC-SP flow sees a MUCH smaller packet drop rate than the TCP flow, and as a result it gets a MUCH larger sending rate. 😭 For the Table 13 simulations, the TFRC-SP flows use 200-byte data segments, while the long-lived TCP flows use 1460-byte segments. For example, with 800 web sessions sharing the link with five TCP and five TFRC-SP flows, the five TCP flows see an average drop rate of 49% in the second half of the simulation, while the five TFRC-SP flows see an average drop rate of 8%. Because of that, the TFRC-SP flows get more than 100 times the throughput of the TCP flows. 💀 This raises serious questions about assuming that flows with small packets see the same drop rate as flows with bigger packets. More work is needed to look at realistic Internet scenarios and figure out whether large packets really are considerably more likely to be dropped than small ones. 🔍
< - - - - - Send Rates in Kbps - - - - >
Web TCP (512B seg) TFRC-SP (200B seg)
Sessions DropRate SendRate DropRate SendRate
-------- -------- -------- -------- --------
10 0.02 335.05 0.00 185.16
25 0.02 289.94 0.00 185.36
50 0.04 139.99 0.01 184.98
100 0.06 53.50 0.01 184.66
200 0.10 16.14 0.04 167.87
400 0.16 6.36 0.07 114.85
800 0.24 0.90 0.11 67.23
1600 0.42 0.35 0.18 39.32
Table 14: Drop and Send Rates for Drop-Tail Queues in Bytes II:
512B TCP SegmentsTable 14 shows that, in these scenarios, the long-lived TCP flows get a higher packet drop rate than the TFRC-SP flows, and they get way less throughput too, even when the TCP flows use 512-byte segments. 🥲
To show the possible negative effect of TFRC-SP in this kind of environment, we look at a simulation with N TCP flows, N TFRC-SP flows, and 10*N web sessions, for N from 1 to 50. That way the demand goes up across all types of traffic, with routers that have byte-based Drop-Tail queues. 📈
< - - - - - Send Rates in Kbps - - - - >
Web TCP (1460B seg) TFRC-SP (200B seg)
Sessions DropRate SendRate DropRate SendRate
-------- -------- -------- -------- --------
10 0.014 2085.36 0.001 180.29
20 0.040 788.88 0.004 183.87
30 0.074 248.80 0.006 182.94
40 0.113 163.20 0.008 185.33
50 0.127 94.70 0.011 185.14
60 0.177 53.24 0.015 185.30
70 0.174 35.31 0.016 185.07
80 0.221 19.38 0.019 183.91
90 0.188 15.63 0.019 180.42
100 0.265 7.08 0.023 176.71
200 0.324 0.38 0.042 139.48
300 0.397 0.32 0.076 93.53
400 0.529 0.40 0.100 70.40
500 0.610 0.37 0.121 57.59
Table 15: Drop and Send Rates for Drop-Tail Queues in Bytes III:
TFRC-SP, 1460B TCP Segments < - - - - - Send Rates in Kbps - - - - >
Web TCP (1460B seg) TFRC (200B seg)
Sessions DropRate SendRate DropRate SendRate
-------- -------- -------- -------- --------
10 0.016 1926.00 0.002 178.47
20 0.030 805.20 0.003 178.23
30 0.062 346.24 0.005 161.19
40 0.093 219.18 0.007 136.28
50 0.110 132.77 0.010 83.02
60 0.170 88.88 0.014 69.75
70 0.149 70.73 0.015 55.59
80 0.213 43.17 0.020 42.82
90 0.188 36.19 0.017 43.61
100 0.233 24.77 0.026 35.17
200 0.311 5.46 0.034 24.85
300 0.367 3.74 0.049 20.19
400 0.421 2.59 0.055 17.71
500 0.459 1.10 0.069 15.95
Table 16: Drop and Send Rates for Drop-Tail Queues in Bytes IV:
Standard TFRC, 1460B TCP SegmentsTable 15 shows simulations using TFRC-SP, while Table 16 shows the same setup but with standard TFRC instead of TFRC-SP. That's the ONLY difference between the two tables 🎯. Notice that with TFRC-SP, the TCP flows and web traffic are basically starved, while the TFRC-SP flows each keep sending 57 Kbps. 😤 By contrast, when standard TFRC is used for the flows sending 200-byte segments, the TCP flows aren't starved, though they still don't get their "share" of the link bandwidth when their packet drop rates are 30% or higher. 🥺 Both sets of simulations show the dangers of a widespread TFRC-SP rollout in an environment where small packets are less likely to be dropped than large ones. 🚨 That's a real red flag, bestie. 🌈
B.4. Packet-dropping Behavior at Routers with AQM ✨
Okay bestie, so the way packets get dropped can also change depending on the Active Queue Management (AQM) mechanism in the router 🧚 When routers use RED (Random Early Detection), a few parameters can change how packets get dropped or marked based on packet size 👀
First, just like Drop-Tail, the RED queue can be measured in either packets or bytes 💅 This matters when RED can't keep the average queue size under control and the queue overflows 😭
Second, and this is totally separate from the first thing, RED can run in either packet mode or byte mode 🌸 In packet mode, every packet has the same chance of getting dropped by RED. In byte mode, every byte has the same chance of getting dropped instead 🎀 So in packet mode, large-packet and small-packet flows get roughly the same packet drop rate. But in byte mode, large-packet and small-packet flows with the same throughput in bps get roughly the same number of packet drops, which is kind of unfair to the small-packet flows 💀 [EA03] looked into how byte vs. packet modes affect RED performance 📚
The simulations below show that for RED in packet mode, the packet drop rates for the TFRC-SP flows are similar to the ones for the TCP flows, and the TFRC-SP flows end up with acceptable throughput 🦋 This is true whether the queue is in packets or bytes, and whether or not Adaptive RED is used (we'll get to that bit below) ✨ But as we'll see, this fairness between TCP and TFRC-SP flows does not hold up for RED in byte mode 🙈
The third RED parameter that changes packet dropping or marking based on packet size is whether RED is running as Standard RED or Adaptive RED 🔥 Adaptive RED tries to keep the same average queue size no matter what the packet drop rate is 🧠 It helps RED work better when drop rates are really high (like greater than 10%) 🍓 Without Adaptive RED, there's a fixed dropping threshold, and once the dropping or marking rate goes past that threshold, every arriving packet gets dropped 😬 With Adaptive RED, the dropping function adjusts itself to handle those high-drop-rate situations 💖 One result is that when byte mode is used with Adaptive RED, byte mode keeps working even in those high-drop-rate situations 👑 But when byte mode is used with standard RED, byte mode stops being used once the drop rate goes past the fixed dropping threshold, which is set to 10% by default in the NS simulator 💜
In the simulations in this section, we check out how TFRC-SP behaves across some of this range of scenarios 🌈 Just like in Section B.3 above, the application for the TFRC-SP flow uses 200-byte data packets and generates 100 packets per second 🫶
< - - - - - Send Rates in Kbps - - - - >
Web TCP (1460B seg) TFRC-SP (200B seg)
Sessions DropRate SendRate DropRate SendRate
-------- -------- -------- -------- --------
10 0.05 305.76 0.04 182.82
25 0.06 224.16 0.06 175.91
50 0.09 159.12 0.08 152.51
100 0.13 90.77 0.11 106.13
200 0.14 48.53 0.14 70.25
400 0.20 22.08 0.19 41.50
800 0.27 3.55 0.25 17.50
1600 0.42 1.87 0.34 8.81
Table 17: Drop and Send Rates for RED Queues in Packet ModeFor the simulations in Table 17, which use a congested router with a RED queue in packet mode, the results look a lot like the ones with a Drop-Tail queue in packets from Table 12 above 🎀 The TFRC-SP flow gets packet drop rates that are similar to the TCP flow, even though it gets higher throughput in the more congested environments 📈 The simulations look similar with a RED queue in packet mode with the queue in bytes, and with or without Adaptive RED 🌸 In these simulations, TFRC-SP gives roughly the performance we wanted 🎉
< - - - - - Send Rates in Kbps - - - - >
Web TCP (1460B seg) TFRC-SP (200B seg)
Sessions DropRate SendRate DropRate SendRate
-------- -------- -------- -------- --------
10 0.06 272.16 0.02 184.37
25 0.07 175.82 0.02 184.06
50 0.10 75.65 0.04 180.56
100 0.14 38.98 0.07 151.65
200 0.19 16.66 0.11 106.80
400 0.26 4.85 0.15 69.41
800 0.35 3.12 0.20 27.07
1600 0.42 0.67 0.29 10.68
Table 18: Drop and Send Rates for RED Queues in Byte ModeTable 18 shows that with a standard RED queue in byte mode instead of packet mode, the packet drop rates of the TCP and TFRC-SP flows are a bit further apart 👀 That gap is especially noticeable at lower packet drop rates 😤 For the simulation in Table 18, the packet drop rates for the TCP flows can be 1 1/2 to four times higher than the packet drop rates for the TFRC-SP flows 💅 But since the TFRC-SP flow has an upper bound on its sending rate, its sending rate doesn't get affected in the lower packet-drop-rate regimes ✨ It only gets affected in the regimes where packet drop rates are 10% or more 🫣 The sending rate for TFRC-SP in the scenarios with higher packet drop rates in Table 18 is greater than we'd want, for example in the scenarios with 400 web sessions or more 😬 Still, the TFRC-SP results are at least better than those of small-packet flows that have no congestion control at all 🦋
< - - - - - Send Rates in Kbps - - - - >
Web TCP (512B seg) TFRC-SP (200B seg)
Sessions DropRate SendRate DropRate SendRate
-------- -------- -------- -------- --------
10 0.01 337.86 0.01 184.06
25 0.02 258.71 0.01 184.03
50 0.02 184.71 0.01 183.99
100 0.04 63.63 0.03 184.43
200 0.08 28.95 0.06 149.80
400 0.12 17.03 0.10 88.21
800 0.24 5.94 0.15 36.80
1600 0.32 3.37 0.21 19.45
Table 19: Drop and Send Rates for RED Queues in Byte ModeTable 19 shows that with a standard RED queue in byte mode and long-lived TCP flows using 512-byte data segments, there's only a moderate difference between the packet drop rate for the 552-byte TCP packets and the 240-byte TFRC-SP packets 🌷 But the sending rates for TFRC-SP in the scenarios with higher packet drop rates in Table 19 are still greater than desired 🙃 That's true even with the fairness goal in mind for TCP flows with 1500-byte data segments instead of 512-byte data segments 💀
< - - - - - Send Rates in Kbps - - - - >
Web TCP (1460B seg) TFRC-SP (200B seg)
Sessions DropRate SendRate DropRate SendRate
-------- -------- -------- -------- --------
10 0.04 318.10 0.02 185.34
25 0.08 175.34 0.03 184.38
50 0.10 81.60 0.04 181.95
100 0.12 28.51 0.05 178.79
200 0.20 3.65 0.06 173.78
400 0.27 1.44 0.08 161.41
800 0.40 0.58 0.06 159.62
1600 0.55 0.29 0.02 180.92
Table 20: Drop and Send Rates with Adaptive RED Queues in Byte ModeFor the simulations in Table 20, the congested router uses an Adaptive RED queue in byte mode 🎀
For this router, the output queue is measured in bytes instead of packets, and each byte gets dropped with the same probability 🫶 Also, because Adaptive RED is being used, this byte-dropping mode keeps going even in the high-packet-drop-rate regime 🌈
As Table 20 shows, in a scenario with Adaptive RED in byte mode, the packet drop rate for the TFRC-SP traffic is much lower than the packet drop rate for the TCP traffic 😱 As a result, the sending rate for the TFRC-SP traffic in a highly congested environment is much higher than the sending rate for the TCP traffic 🚨 In fact, in these scenarios the TFRC-SP congestion control mechanisms are mostly ineffective for the small-packet traffic 💀
The simulation with 1600 web servers is the one that really concerns us 😭 Here the TCP packet drop rate goes up, while the TFRC-SP packet drop rate drops significantly. This comes from a detail of the Adaptive RED implementation in the NS simulator, not from the dynamics of TFRC-SP itself 🔍 Specifically, Adaptive RED is configured not to "adapt" to packet drop rates over 50%. When the packet drop rate is at most 50%, Adaptive RED does a pretty good job of keeping the packet drop rate proportional to the packet size 🧁 TCP packets are six times larger than the TFRC-SP packets (including headers), so the TCP packets should see a packet drop rate roughly six times larger 📏
But once packet drop rates go over 50%, Adaptive RED is no longer in this regime, so it stops keeping the drop rate for TCP packets at most six times the drop rate of the TFRC-SP packets 🥲
We should note that the unfairness in these simulations, in favor of TFRC-SP, is even more severe than the unfairness shown in Table 13 for a Drop-Tail queue in bytes 😳 At the same time, it's not known whether any routers with Adaptive RED in byte mode, or any AQM mechanisms that behave similarly, are deployed in the Internet 🤷♀️ We also don't know how widely standard RED is deployed, or how widely any of the proposed AQM mechanisms are deployed 🌸
< - - - - - Send Rates in Kbps - - - - >
Web TCP (512B seg) TFRC-SP (200B seg)
Sessions DropRate SendRate DropRate SendRate
-------- -------- -------- -------- --------
10 0.01 306.56 0.01 185.11
25 0.02 261.41 0.01 184.41
50 0.02 185.07 0.01 184.54
100 0.04 59.25 0.03 181.58
200 0.08 16.32 0.06 150.87
400 0.12 11.53 0.10 98.10
800 0.25 5.85 0.15 46.59
1600 0.32 1.43 0.22 19.40
Table 21: Drop and Send Rates for Adaptive RED Queues in Byte ModeTable 21 shows that when TFRC-SP with 240-byte packets competes with TCP with 552-byte packets in a scenario with Adaptive RED in byte mode, the TFRC-SP flows still get more bandwidth than the TCP flows 🏃♀️ But the unfairness is less severe, and the packet drop rates of the TCP flows are at most twice those of the TFRC-SP flows 🙌 So TFRC-SP flows still take more than their fair share of the bandwidth, but the TFRC-SP congestion control works better here than it did in Table 20 above 💖
Appendix C. Exploring Possible Oscillations in the Loss Event Rate ✨
okay so TFRC-SP estimates the loss interval size differently for short and long loss intervals 💅 For long loss intervals it counts only one loss event, but for short loss intervals it counts every single packet loss as a loss event. 🫶 The question people kept asking is whether this can cause oscillations in the average loss event rate, especially when there are lots of packet drops inside one loss event and loss events keep flipping from short to long and back again 😭
Here's an example. Imagine a loss interval with N packets, and N/4 of them are lost. If the loss interval is short (at most two round-trip times), the loss interval length is measured as 4 packets. If it's long, the loss interval length is measured as N packets. Bestie, that's a big jump 💀
If that switching from short to long and back really does cause severe oscillations in the average packet drop rate, and therefore in the allowed sending rate, one fix would be a more gradual transition between how short and long loss intervals get measured 🦋 One possibility is to use all of the packet losses from a loss interval of one round-trip time, use 2/3 of the packet losses from a loss interval of two round-trip times, use 1/3 of the packet losses from a loss interval of three round-trip times, and use only one packet loss from a loss interval of four or more round-trip times. This smoother version would move from counting all losses for a loss interval of one round-trip time to counting only one loss event for a loss interval of four or more round-trip times. It's giving gentle slope energy 🌈
BUT, and this is the tea 🍵, the simulations so far haven't shown a big difference in oscillations in the estimated loss event rate between the default mechanism for short loss intervals and this gradual mechanism. The simulation scripts are available from [VOIPSIMS]. So for now we're staying with the simple default mechanism for estimating the loss event rate for short loss intervals, the one described in this document. Simple wins when it works 🎀✨
Appendix D. A Discussion of Packet Size and Packet Dropping 🌸
okay so these lists give a general summary of the relative advantages of packet-dropping behavior at routers that is independent of packet size, versus packet-dropping behavior where large packets are more likely to be dropped than small ones. Both have their moments 🦋
Advantages of Packet Dropping Independent of Packet Size:
- Adds another incentive for end nodes to use large packets. 💜
- Matches an environment with a limitation in pps rather than bps. 🍓
Advantages of Packet Dropping as a Function of Packet Size:
- Small control packets are less likely to be dropped than large data packets, which improves TCP performance. 👑
- Matches an environment with a limitation in bps rather than pps. 🌈
- Reduces the penalty of TCP and other transport protocols against flows with small packets, where the allowed sending rate is roughly a linear function of packet size. 🧚
- A queue limited in bytes is better than a queue limited in packets for matching the worst-case queue size to the worst-case queueing delay in seconds. ⭐
Normative References
- [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. 🔥
- [RFC3448] Handley, M., Floyd, S., Padhye, J., and J. Widmer, "TCP Friendly Rate Control (TFRC): Protocol Specification", RFC 3448, January 2003. 💖
Informative References
- [EA03] W. Eddy and M. Allman. A Comparison of RED's Byte and Packet Modes, Computer Networks, 42(2), June 2003. 🎀
- [P04] T. Phelan, TFRC with Self-Limiting Sources, October 2004,
<http://www.phelan-4.com/dccp/>. 🌸 - [RFC791] Postel, J., "Internet Protocol", STD 5, RFC 791, September 1981. 🦋
- [RFC879] Postel, J., "TCP maximum segment size and related topics", RFC 879, November 1983. 💅
- [RFC1144] Jacobson, V., "Compressing TCP/IP headers for low-speed serial links", RFC 1144, February 1990. 🫶
- [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition of Explicit Congestion Notification (ECN) to IP", RFC 3168, September 2001. ✨
- [RFC3540] Spring, N., Wetherall, D., and D. Ely, "Robust Explicit Congestion Notification (ECN) Signaling with Nonces", RFC 3540, June 2003. 🌈
- [RFC3714] Floyd, S. and J. Kempf, "IAB Concerns Regarding Congestion Control for Voice Traffic in the Internet", RFC 3714, March 2004. 🎀
- [RFC3819] Karn, P., Bormann, C., Fairhurst, G., Grossman, D., Ludwig, R., Mahdavi, J., Montenegro, G., Touch, J., and L. Wood, "Advice for Internet Subnetwork Designers", BCP 89, RFC 3819, July 2004. 🧚
- [RFC4340] Kohler, E., Handley, M., and S. Floyd, "Datagram Congestion Control Protocol (DCCP)", RFC 4340, March 2006. 🍓
- [RFC4342] Floyd, S., Kohler, E., and J. Padhye, "Profile for Datagram Congestion Control Protocol (DCCP) Congestion Control ID 3: TCP-Friendly Rate Control (TFRC)", RFC 4342, March 2006. 💜
- [RFC3448bis] Handley, M., Floyd, S., Padhye, J., and J. Widmer, "TCP Friendly Rate Control (TFRC): Protocol Specification", Work in Progress, March 2007. 👑
- [S05] Peter Sholander, private communications, August 2005. Citation for acknowledgement purposes only. 🙈
- [V00] P. Vasallo. Variable Packet Size Equation-Based Congestion Control. ICSI Technical Report TR-00-008, April 2000,
<http://www.icsi.berkeley.edu/cgi-bin/pubs/publication.pl?ID=001183>⭐ - [VOIPSIMS] Web page
<http://www.icir.org/tfrc/voipsims.html>. 🌈 - [WBL04] J. Widmer, C. Boutremans, and Jean-Yves Le Boudec. Congestion Control for Flows with Variable Packet Size, ACM CCR, 34(2), 2004. 🔥
Authors' Addresses 🦋
Sally Floyd
ICSI Center for Internet Research
1947 Center Street, Suite 600
Berkeley, CA 94704
USA
EMail: floyd@icir.org
Eddie Kohler
4531C Boelter Hall
UCLA Computer Science Department
Los Angeles, CA 90095
USA
EMail: kohler@cs.ucla.eduFull Copyright Statement 💅
Copyright (C) The IETF Trust (2007).
This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights.
This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Intellectual Property 🫶
The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document, or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification, can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. 🌸
Acknowledgement ✨
Funding for the RFC Editor function is currently provided by the Internet Society. 💖
📦 where this page comes from
- ietf: rfc4828 2007-04
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms