RFC 8087

The Benefits of Using Explicit Congestion Notification (ECN)

📅 published March 2017, from ietf

RFC 8087: The Benefits of Using Explicit Congestion Notification (ECN) 💖

Okay bestie, let's talk about RFC 8087, and it's giving main character energy 💅 It's titled The Benefits of Using Explicit Congestion Notification (ECN), and it came out in March 2017 as an Informational document 🦋 That means it's published for info and isn't a Standards Track spec. It's from the IETF, with authors G. Fairhurst (University of Aberdeen) and M. Welzl (University of Oslo), and it's logged under ISSN 2070-1721. Its title block doesn't list anything it obsoletes or updates, so it's not replacing or amending any other RFC 🌸

Abstract ✨

Okay so the whole point of this doc is to show why apps are obsessed with using a transport that turns on Explicit Congestion Notification (ECN) 🔥 It walks through the big wins: better throughput, less delay, and other perks when ECN is used on a path with gear that does Congestion Experienced (CE) marking. It also talks about the challenges you'd face deploying ECN successfully 💀 What it doesn't do is invent new ECN algorithms or explain how to implement ECN in endpoints (Internet hosts), routers, or other network devices. That's not the assignment, babe 🙈

Status of This Memo

This document is not an Internet Standards Track specification. It's published for informational purposes only 🎀

This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community, received public review, and got approved for publication by the Internet Engineering Steering Group (IESG) 🫶 Not every IESG-approved document is eligible for any level of Internet Standard, so check Section 2 of RFC 7841 for the details 🧚

Want the current status, any errata, or a way to send feedback? Head to http://www.rfc-editor.org/info/rfc8087 ✨

Copyright (c) 2017 IETF Trust and the persons identified as the document authors. All rights reserved 💜

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as they stand on the publication date. Please read these carefully, because they spell out your rights and restrictions for this document 👑 Any Code Components pulled out of this document must include the Simplified BSD License text described in Section 4.e of the Trust Legal Provisions, and they come without warranty, as the Simplified BSD License describes 🌈

Table of Contents

1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
  1.1.  Terminology . . . . . . . . . . . . . . . . . . . . . . .   4
  2.  Benefit of Using ECN to Avoid Congestion Loss . . . . . . . .   5
  2.1.  Improved Throughput . . . . . . . . . . . . . . . . . . .   5
  2.2.  Reduced Head-of-Line Blocking . . . . . . . . . . . . . .   6
  2.3.  Reduced Probability of RTO Expiry . . . . . . . . . . . .   6
  2.4.  Applications That Do Not Retransmit Lost Packets  . . . .   7
  2.5.  Making Incipient Congestion Visible . . . . . . . . . . .   8
  2.6.  Opportunities for New Transport Mechanisms  . . . . . . .   8
  3.  Network Support for ECN . . . . . . . . . . . . . . . . . . .   9
  3.1.  The ECN Field . . . . . . . . . . . . . . . . . . . . . .  10
  3.2.  Forwarding ECN-Capable IP Packets . . . . . . . . . . . .  10
  3.3.  Enabling ECN in Network Devices . . . . . . . . . . . . .  11
  3.4.  Coexistence of ECN and Non-ECN Flows  . . . . . . . . . .  11
  3.5.  Bleaching and Middlebox Requirements to Deploy ECN  . . .  11
  3.6.  Tunneling ECN and the Use of ECN by Lower-Layer Networks   12
  4.  Using ECN across the Internet . . . . . . . . . . . . . . . .  12
  4.1.  Partial Deployment  . . . . . . . . . . . . . . . . . . .  13
  4.2.  Detecting Whether a Path Really Supports ECN  . . . . . .  13
  4.3.  Detecting ECN-Receiver Feedback Cheating  . . . . . . . .  14
  5.  Summary: Enabling ECN in Network Devices and Hosts  . . . . .  14
  6.  Security Considerations . . . . . . . . . . . . . . . . . . .  15
  7.  References  . . . . . . . . . . . . . . . . . . . . . . . . .  16
  7.1.  Normative References  . . . . . . . . . . . . . . . . . .  16
  7.2.  Informative References  . . . . . . . . . . . . . . . . .  16
Acknowledgements  . . . . . . . . . . . . . . . . . . . . . . . .  19
Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  19

1. Introduction ✨

Okay, so Internet transports like TCP and the Stream Control Transmission Protocol (SCTP) live in endpoints (Internet hosts), and their whole job is to notice network congestion and react to it 🫶 They can spot congestion by losing an IP packet, or, if Explicit Congestion Notification (ECN) [RFC3168] is turned on, by receiving a packet with a Congestion Experienced (CE) marking in its IP header. Transports treat both of those as congestion signals 🦋 UDP apps that run their own congestion control can turn on ECN too, as long as they can correctly process the ECN signals [RFC8085]. RTP with ECN [RFC6679] is one example 💜

Active Queue Management (AQM) [RFC7567] is a family of techniques that network devices (routers, middleboxes, or any other gear that forwards packets) can use to control how big their queues get in network buffers 🌸

A device that doesn't do AQM usually uses a drop-tail policy, meaning it drops the extra IP packets once its queue is full 💀 Transports take those drops as a signal that the end-to-end path is congested. End-to-end transports like TCP cause a bit of loss on purpose while they try to share capacity with other flows 🙈 Loss isn't always caused by congestion, though. It can come from link corruption, receiver overrun, and so on. Endpoints still have to conservatively assume all loss might be congestion and slow down. So observed loss makes the transport cut back the maximum rate the sending endpoint is allowed to use 🍓

ECN lets the network say "heads up, congestion is starting" without causing packet loss 🎀 It also lets the network deliver some packets that would have been dropped if the application or transport didn't support ECN. Reduced packet loss is the most obvious benefit of ECN, but it's usually pretty modest 😭 The side effects can be way bigger, though. Some of them matter much more than the immediate loss reduction you get from a CE marking instead of a dropped packet 👑 Several of the benefits cut latency, for example by reducing head-of-line blocking ✨

ECN gets flagged in the ECN field [RFC3168], which is in the packet header of every IPv4 and IPv6 packet 🧚 That field can hold one of the four values shown in Figure 1. The Not-ECT codepoint '00' means the packet isn't using ECN. The ECT(0) codepoint '01' and the ECT(1) codepoint '10' both tell you the transport using the IP layer supports ECN 💖 The CE codepoint '11' gets set by an ECN-capable network device to tell the transport endpoint about congestion 🌈

+-----+-----+---------+
| ECN FIELD |  Name   |
+-----+-----+---------+
|  0  |  0  | Not-ECT |
|  0  |  1  | ECT(1)  |
|  1  |  0  | ECT(0)  |
|  1  |  1  | CE      |
+-----+-----+---------+

Figure 1: The ECN Field in the IP Packet Header (based on [RFC3168])

When an application uses a transport that enables ECN [RFC3168], the transport sets the ECT(0) or ECT(1) codepoint in the IP header of the packets it sends 🦋 That tells network devices they're allowed to mark those ECN-capable IP packets instead of dropping them. An ECN-capable network device can then flag incipient congestion (network queuing) before the transport hits congestion loss or high queuing delay 🔥 Marking usually happens because of various AQM algorithms [RFC7567]. The transport endpoints don't need to know the exact combination of AQM and ECN algorithms in play 🫶

This doc is about how transport- and application-layer flows use ECN, not how endpoint hosts, routers, or other network devices implement it 💅

1.1. Terminology

Here's the vocab the doc uses 📚

  • AQM: Active Queue Management.
  • CE: Congestion Experienced, the codepoint value '11' in the ECN field of the IP packet header.
  • ECN-capable IP Packet: A packet whose ECN field is set to a non-zero ECN value, meaning ECT(0), ECT(1), or the CE codepoint.
  • ECN-capable network device: A network device that may forward, drop, or queue an ECN-capable packet, and that may choose to CE mark it when there's incipient congestion 🌸
  • ECN-capable transport/application: A transport that sends ECN-capable IP Packets, monitors reception of the ECN field, and generates the right feedback to control the sending endpoint's rate, so you get end-to-end congestion control 💜
  • ECN field: A 2-bit field defined for explicit congestion signaling in the IPv4 and IPv6 packet headers ✨
  • Endpoint: An Internet host that terminates a transport protocol connection across an Internet path 🧚
  • Incipient Congestion: Congestion detection while it's just starting, maybe because a network device notices the arrival rate is higher than the forwarding rate 🍓
  • Network device: A router, middlebox, or other device that forwards IP packets through the network 🎀
  • non-ECN-capable: A network device or endpoint that doesn't interpret the ECN field. Such a device isn't permitted to change the ECN codepoint 🙈
  • not-ECN-capable IP Packet: An IP packet whose ECN field is set to zero ('00'). A not-ECN-capable packet may be forwarded, dropped, or queued by a network device 💀

2. Benefit of Using ECN to Avoid Congestion Loss 💖

An ECN-capable network device is expected to CE mark an ECN-capable IP packet as CE when an AQM method detects incipient congestion, instead of dropping the packet [RFC7567]. Apps can get a bunch of benefits from this marking, and the rest of this section breaks them down 🌈

2.1. Improved Throughput 🚀

ECN is all about avoiding the waste of dropping data that already made it across at least part of the network path 😭

ECN can boost an application's throughput, but honestly that boost is often not the biggest win 💅 When an app uses a lightly to moderately loaded path, only a few packets get dropped because of congestion. Take the example from Table 1 of [RFC3649]: for a standard TCP sender with an RTT of 0.1 seconds, a packet size of 1500 bytes, and an average throughput of 1 Mbps, the average packet-drop ratio works out to 0.02, which is 1 in 50 packets 🦋 That translates to roughly a 2% throughput gain if ECN is enabled. (Note that in heavy congestion, packet loss may be unavoidable with or without ECN 🌸)

2.2. Reduced Head-of-Line Blocking ✨

okay so a lot of Internet transports deliver received data in order to the apps they serve 📦 For these apps, ECN can cut down the delay that happens when packets go missing 💖

packet loss can happen for different reasons 🙈 one cause is when an AQM scheme drops a packet as a signal of incipient congestion. no matter why it's lost, a missing packet needs to kick off a congestion control response 🔥 a reliable transport also triggers retransmission to recover the lost data. for a transport that gives in-order delivery, the transport receiver has to stall (or wait) until all the data sent ahead of a lost segment is correctly received before it can pass any later data up to the application 😭 so a loss creates a delay of at least one RTT after the loss event before data can reach the app. we call this head-of-line blocking 🧱 this is the usual requirement for TCP and SCTP. Partially Reliable SCTP (PR-SCTP) [RFC3758], UDP [RFC768] [RFC8085], and the Datagram Congestion Control Protocol (DCCP) [RFC4340] give you a transport that doesn't do reordering 🦋

by turning on ECN, a transport keeps receiving in-order data even when there's incipient congestion, and it can hand that data to the receiving app ✨ so ECN avoids the extra reordering delay in a reliable transport. the sender still needs to make an appropriate congestion response to lower the maximum transmission rate for future traffic, which usually means reducing the sending rate [RFC8085] 💅

2.3. Reduced Probability of RTO Expiry 🎀

some patterns of packet loss can end in a Retransmission Timeout (RTO), and that causes a sudden, big change in how fast a transport/application is allowed to forward packets 😬 because ECN gives network devices an alternative to dropping for signalling incipient congestion, it can lower the chance of loss and therefore lower the likelihood of RTO expiry 🌸

Internet transports/applications generally use an RTO timer as a last resort to detect and recover loss [RFC8085] [RFC5681]. specifically, an RTO timer detects loss of a packet that isn't followed by other packets, like at the end of a burst of data segments, or when an application goes idle (either because the app has no more data to send or the network stops it from sending more, e.g., flow or congestion control at the transport layer) 🥺 this loss of the last segment (or last few segments) of a traffic burst is also known as a "tail loss" 🧊 standard transport recovery methods, such as Fast Recovery [RFC5681], are often unable to recover from a tail loss. that's because the endpoint receiver doesn't know the lost segments were actually sent, so it generates no feedback [Fla13] 🙃 retransmitting these segments relies on the expiry of a transport retransmission timer. this timer is also used to detect a lack of forwarding along a path. when the RTO expires, the transport loses its state about the network path being used. this typically includes resetting path estimates such as the RTT, reinitializing the congestion window, and possibly updating other transport state 😵‍💫 that can hurt the transport's performance until it adapts to the path again 💔

an ECN-capable network device can't get rid of the possibility of tail loss, because a drop can still happen when a traffic burst goes over the instantaneous available capacity of a network buffer, or as a result of the AQM algorithm (e.g., overload protection mechanisms [RFC7567]) 🫠 but an ECN-capable network device that sees incipient congestion may be expected to buffer the IP packets of an ECN-capable flow and set a CE mark in one or more packets, rather than triggering a packet drop 🌈 setting a CE mark signals incipient congestion without forcing the transport/application into a retransmission timeout 🙌 this cuts down application-level latency and can improve throughput for apps that send intermittent bursts of data 🚀

the benefit of avoiding retransmission loss is expected to be significant when ECN is used on TCP SYN/ACK packets [RFC5562], where the RTO interval can be large because TCP can't base the timeout period on prior RTT measurements from the same connection 🫣

2.4. Applications That Do Not Retransmit Lost Packets 🧚

a transport that turns on ECN can get timely congestion signals without needing to retransmit packets every time it gets a congestion signal 💜

some latency-critical apps don't retransmit lost packets, but they may still be able to adjust their sending rate after detecting incipient congestion ⏱️ examples of these apps include UDP-based services that carry Voice over IP (VoIP), interactive video, or real-time data 📞 the performance of many of these apps drops fast as packet loss goes up, so the transport/application may use mechanisms (e.g., packet forward error correction, data duplication, or media codec error concealment) to soften the immediate effect of congestion loss on the app 🛟 some of these mechanisms eat up extra network capacity, some need additional processing, and some add path latency when congestion is experienced 😩 by decoupling congestion control from loss, ECN lets transports that support these apps slow down before the app ever experiences congestion loss 🏃‍♀️ that can reduce the negative impact of triggering loss-hiding mechanisms, which has a direct positive effect on the quality that users of these apps experience 👑

2.5. Making Incipient Congestion Visible 👀

one thing about using ECN is that it exposes the presence of congestion on a network path to the transport and network layers 🔍 this lets information be collected about incipient congestion 📊

recording the presence of CE-marked packets can tell you about the current congestion level on a network path 📈 a network flow that only sees CE marking and no loss implies that the sending endpoint is only experiencing congestion 💅 a flow may also experience loss (e.g., due to queue overflow, AQM methods that protect other flows, link corruption, or loss in middleboxes) 😤 when a mix of CE marking and packet loss is experienced, transports and measurements need to assume there is congestion [RFC7567]. so an absence of CE marks does not mean a path hasn't experienced congestion 🙅‍♀️

a transport/application or a network operator can use the reception of CE-marked packets to monitor the level of congestion 🕵️‍♀️ for example, ECN measurements are used by Congestion Exposure (ConEx) [RFC6789]. metering packet loss, on the other hand, is harder 🥴

2.6. Opportunities for New Transport Mechanisms 🌟

ECN can enable the design and deployment of new algorithms in network devices and Internet transports 🛠️ Internet transports need to treat both loss and CE marking as an indication of congestion. however, while the amount of feedback provided by drop ought naturally to be minimized, that isn't the case for ECN 💫 instead, an ECN-capable network device could give richer (more frequent and fine-grained) indications of its congestion state to the transport 🎁

for any ECN-capable transport (ECT), the receiving endpoint needs to give feedback to the transport sender to indicate that CE marks have been received 📬 [RFC3168] provides one method that signals once each round-trip time (RTT) that CE-marked packets have been received ⏰

a receiving endpoint may give more detailed feedback to the congestion controller at the sender (e.g., describing the set of received ECN codepoints or indicating each received CE-marked packet) 🧠 precise feedback about the number of CE marks encountered is supported by RTP when used over UDP [RFC6679], and it has been proposed for SCTP [ST14] and TCP [ECN-FEEDBACK] 💌

more detailed feedback is expected to enable the evolution of transport protocols, letting the congestion control mechanism make a more appropriate decision on how to react to congestion 🧩 designers of transport protocols need to consider not only how network devices CE-mark packets, but also how the control loop in the application/transport reacts to receiving these CE-marked packets 🔁

bestie, this next bit is the spicy one 🌶️ there's benefit noted when packets are CE marked early using an instantaneous queue, and if the receiving endpoint gives feedback about the number of packet marks encountered, an improved sender behavior has been shown to be possible, e.g., Data Center TCP (DCTCP) [AL10]. DCTCP is targeted at controlled environments such as a data center. this is a work in progress, and it's currently unknown whether or how such behavior could be safely introduced into the Internet 🤔 any update to an Internet transport protocol needs careful consideration of the robustness of the behavior when it works with endpoints or network devices that weren't designed for the new congestion reaction 🚨

3. Network Support for ECN 🛜

for an application to use ECN, the endpoints have to enable ECN within the transport being used 💻 it also requires that all network devices along the path at least forward IP packets that set a non-zero ECN codepoint 🛤️

ECN can be deployed both in the general Internet and in controlled environments 🌍

  • ECN can be incrementally deployed in the general Internet. the IETF has provided guidance on configuration and usage in [RFC7567] 🌱
  • ECN may be deployed within a controlled environment, for example within a data center or within a well-managed private network. this use of ECN may be tuned to the specific use case 🏢 an example is DCTCP [AL10] [DCTCP] 🎯

early experience of using ECN across the general Internet ran into a number of operational difficulties when the network path either failed to transfer ECN-capable packets or inappropriately changed the ECN codepoints [BA11] 😱 a recent survey reported growing support for network paths to pass ECN codepoints [TR15] 📈✨

the remainder of this section identifies what is needed for network devices to effectively support ECN 🫶

3.1. The ECN Field 💖

okay so the current IPv4 and IPv6 specs give 2 bits in the IP header to carry the ECN codepoint ✨. That 2-bit field was reserved in [RFC2474] and then assigned in [RFC3168] 🎀.

[RFC4774] digs into the drama of defining alternate semantics for the ECN field, and it lays out requirements for coexisting safely on an Internet that might include routers which don't understand those alternate semantics 🧚.

Some network devices were set up to use a routing hash that included all 8 bits of the now deprecated Type of Service (TOS) field [RFC1349] 🙈. Today 2 of those bits carry ECN, and that clashes with routing hashes. Packets carrying a CE mark could take a different path from packets carrying an ECT mark. The reordering that follows can tank the performance of transport protocols like TCP or SCTP, plus UDP-based apps that are sensitive to reordering 😭. A network device that follows this older spec needs to be updated to the current specs [RFC2474] to support ECN 🔧. Also, configuration of network devices must note that the ECN field may be updated by any ECN-capable network device along a path 💅.

3.2. Forwarding ECN-Capable IP Packets 🦋

Not every network device along a path needs to be ECN-capable (in other words, able to perform CE marking) 🌸. But all network devices need to be configured not to drop packets just because the ECT(0) or ECT(1) codepoints are being used 🫶.

Any network device that doesn't do CE marking on an ECN-capable packet can be expected to drop those packets when there's congestion 😬. Apps that hit congestion at those devices get zero benefit from turning on ECN. They could still see a benefit, though, if the congestion happened inside a device that does support ECN ✨.

3.3. Enabling ECN in Network Devices 🎀

Network devices should use an AQM algorithm that CE-marks ECN-capable traffic when deciding how to respond to congestion [RFC7567]. An ECN method should set a CE mark on ECN-capable packets when there's incipient congestion 🌈. A CE-marked packet gets read by the transport endpoints as a sign of incipient congestion 👑.

There's a chance to design an AQM method for an ECN-capable network device that's different from an AQM method built to drop packets 🧁. [RFC7567] says the network device should make this behaviour configurable 🍓.

[RFC3168] describes a method where a network device sets the CE mark at the moment it would otherwise have dropped the packet. People have often assumed that devices should CE-mark packets at the same congestion level where they'd normally drop them. However, [RFC7567] recommends that network devices allow independent configuration of the AQM dropping settings and the ECN marking settings 💜. Some network devices do support that separate configuration of the drop and mark policies ⭐.

3.4. Coexistence of ECN and Non-ECN Flows 🧚‍♀️

Network devices need to be able to forward all IP flows and give appropriate treatment to both ECN and non-ECN traffic 💖.

The design of an AQM scheme that supports ECN needs to think about how queueing is affected during incipient congestion 🤔. For example, a simple AQM scheme could put ECN-capable and non-ECN-capable flows into the same queue, alongside an ECN scheme that CE-marks packets during incipient congestion. The CE-marked packets that stay in the queue during congestion can keep adding to queueing delay 😭. By contrast, non-ECN-capable packets would normally be dropped by an AQM scheme under incipient congestion 🙃. That difference in queueing is one reason to look at more advanced AQM schemes, and it might motivate enabling flow isolation using scheduling [RFC7567] 🦋. The IETF is defining methods to evaluate whether AQM schemes are suitable for deployment in the general Internet [RFC7928] 💅.

3.5. Bleaching and Middlebox Requirements to Deploy ECN 🧼

Network devices should not be configured to change the ECN codepoint in the packets they forward, except to set the CE codepoint to signal incipient congestion 🚫.

There have been cases where an endpoint sends a packet with a non-zero ECN mark, but the remote endpoint receives it with a zero ECN codepoint [TR15] 😱. This could happen because of a policy that erases or "bleaches" the ECN codepoint values at a network edge, resetting the codepoint to zero 🫧. Bleaching can happen for various reasons, including normalizing packets to hide which equipment supports ECN 🙈. Either way, this kind of policy stops apps from using ECN at all 💀.

When ECN-capable IP packets marked as ECT(0) or ECT(1) get re-marked as non-ECN-capable (so the ECN field is set to the zero codepoint), those packets could be dropped by ECN-capable network devices further down the path 😤. That wipes out the whole point of using ECN 🥀.

A network device must not change a packet with a CE mark to a zero codepoint. If the device decides not to forward a CE-marked packet, it has to drop the packet instead of bleaching the marking 🛑. Why? Because a CE-marked packet has already been treated as ECN in the network, and re-marking it would hide the congestion signal from the receiving endpoint 🕵️‍♀️. That kills the benefits of ECN. It can also slow down the response to congestion compared to using AQM, since the transport will only react if it later finds out about congestion some other way 🐢.

Before [RFC2474], an older usage put the bits that now form the ECN field inside the now deprecated TOS field [RFC1349]. A network device that follows this older spec was allowed to re-mark or erase the ECN codepoints, so that equipment needs to be updated to the current specs in order to support ECN 🔄.

3.6. Tunneling ECN and the Use of ECN by Lower-Layer Networks 🚇

Some networks may use ECN internally or tunnel it (for example, for traffic engineering or security) 🛤️. These methods need to make sure the ECN field of the tunnel packets is handled correctly at both the ingress and the egress of the tunnel 🎯. Guidance on using ECN correctly is in [RFC6040] 📘.

More guidance on encapsulating ECN and using it with non-IP network devices is in [ECN-ENCAP] 🧾.

4. Using ECN across the Internet 🌐

A receiving endpoint needs to report the loss it experiences when it uses loss-based congestion control 📉. The same goes for ECN: when it's enabled, a receiving endpoint must correctly report the presence of CE marks by providing a mechanism that feeds this congestion information back to the sending endpoint [RFC3168] [RFC8085], so the sender can react to the congestion it's experiencing 🔁. That mechanism needs to work robustly across a wide range of Internet path characteristics 🧗‍♀️. This section covers partial deployment, meaning how ECN-enabled endpoints can keep working well over a path with misbehaving network devices, or when an endpoint doesn't correctly give feedback about ECN information 🫠.

4.1. Partial Deployment 🧩

ECN use is negotiated between the endpoints before the mechanism gets used 🤝.

ECN was designed to allow incremental partial deployment [RFC3168] 🌱. Any network device can choose to use either ECN or some other loss-based policy to manage its traffic 🎛️. Likewise, transport and application negotiation lets the sending and receiving endpoints choose whether ECN will be used to manage congestion for a particular network flow 💞.

4.2. Detecting Whether a Path Really Supports ECN 🔍

Internet transports and applications need to be robust to the variety of path characteristics they run into across the general Internet, which can also change over time 🌪️. They need to monitor correct forwarding of ECN over the entire path and for the whole duration of a session 👀.

To be robust, applications and transports need to be designed to expect heterogeneous forwarding. For example, some IP packets might be CE-marked by one network device and others by a different device that uses another AQM algorithm, or a mix of CE marking and loss-based congestion indications might be in use 🌈. [RFC7928] describes methodologies for evaluating AQM schemes 📊.

A transport or application also needs to be robust to path changes 🛣️. A change in the set of network devices along a path could affect the ability to signal or use ECN effectively across that path. For example, a path might switch to a middlebox that bleaches ECN codepoints (see Section 3.5) 🧼.

A sending endpoint can check that any CE marks applied to packets it receives over the path actually get delivered to the remote receiving endpoint, and that appropriate feedback comes back 🕵️. One way to do this is for a sender to set a known CE codepoint on specific packets in a network flow, then check whether the remote endpoint correctly reports those marks [ECN-FALLBACK] [TR15] 🧪. If a sender detects persistent misuse of ECN, it needs to fall back to loss-based recovery and congestion control 🪂. Guidance on a suitable transport reaction is in [ECN-FALLBACK] 📝.

4.3. Detecting ECN-Receiver Feedback Cheating 🚨

Appropriate feedback requires that the endpoint receiver doesn't try to hide the reception of CE-marked packets in the ECN feedback it gives the sending endpoint [RFC7567] 🙅‍♀️. So designers of applications and transports are encouraged to include mechanisms that can detect this misbehaviour 🕵️‍♀️. If a sending endpoint detects that a receiver isn't correctly giving this feedback, it needs to fall back to loss-based recovery instead of ECN 🔙.

5. Summary: Enabling ECN in Network Devices and Hosts ✨

Okay, this part is basically the ECN "cheat sheet," bestie 💖. It sums up why ECN is worth turning on across the Internet, then lists what needs to be true for network devices and endpoints to actually use it. The section also wraps up the whole document, so grab a snack 🍓

Application developers SHOULD, where possible, use transports that enable ECN 🧚. Applications that use UDP directly need to add support for the functions ECN requires [RFC8085]. Once enabled, an application using an ECN-supporting transport gets the benefits as the network starts enabling ECN too ✨. It doesn't need a rewrite to get them 🙌 Figure 2 lists the key benefits:

+---------+-----------------------------------------------------+
| Section | Benefit                                             |
+---------+-----------------------------------------------------+
|   2.1   | Improved Throughput                                 |
|   2.2   | Reduced Head-of-Line Blocking                       |
|   2.3   | Reduced Probability of RTO Expiry                   |
|   2.4   | Applications that do not Retransmit Lost Packets    |
|   2.5   | Making Incipient Congestion Visible                 |
|   2.6   | Opportunities for New Transport Mechanisms          |
+---------+-----------------------------------------------------+

                     Figure 2: Summary of Key Benefits

Network operators and people configuring network devices SHOULD enable ECN [RFC7567] 🎀

Here's the network device checklist. Prerequisites for network devices (including IP routers) to enable use of ECN include:

  • A network device that updates the ECN field in IP packets MUST use IETF-specified methods (see Section 3.1) 📜
  • A network device MAY support alternate ECN semantics (see Section 3.1) 🦋
  • A network device MUST NOT choose a different network path solely because a packet carries a CE-codepoint set in the ECN Field. CE-marked packets need to follow the same path as packets with an ECT(0) or ECT(1) codepoint (see Section 3.1). Network devices need to be configured not to drop packets solely because the ECT(0) or ECT(1) codepoints are used (see Section 3.2) 🚫
  • An ECN-capable network device SHOULD correctly update the ECN codepoint of ECN-capable packets in the presence of incipient congestion (see Section 3.3) 🔥
  • Network devices need to be able to forward both ECN-capable and not-ECN-capable flows (see Section 3.4) 🫶
  • A network device MUST NOT change a packet with a CE mark to a not-ECN-capable codepoint ('00'). If the network device decides not to forward the packet with the CE mark, it has to instead drop the packet and not bleach the marking (see Section 3.5) 💅

Now the endpoint side, which is just as important 👑 Prerequisites for network endpoints to enable use of ECN include the following:

  • An application SHOULD use an Internet transport that can set and receive ECN marks (see Section 4) ✨
  • An ECN-capable transport/application MUST return feedback indicating congestion to the sending endpoint and perform an appropriate congestion response (see Section 4) 💜
  • An ECN-capable transport/application SHOULD detect paths where there is persistent misuse of ECN and fall back to not sending ECT(0) or ECT(1) (see Section 4.2) 🌈
  • Designers of applications and transports are encouraged to include mechanisms that can detect and react appropriately to misbehaving receivers that fail to report CE-marked packets (see Section 4.3) 🙈

6. Security Considerations 🔐

Okay, serious moment, babe 🥺 This document introduces no new security considerations. Each RFC listed in this document discusses the security considerations of the specification it contains, so go read those for the full picture 💀

7. References 📚

7.1. Normative References

7.2. Informative References

  • [AL10] Alizadeh, M., Greenberg, A., Maltz, D., Padhye, J., Patel, P., Prabhakar, B., Sengupta, S., and M. Sridharan, "Data Center TCP (DCTCP)", ACM SIGCOMM Computer Communication Review, Volume 40, Issue 4, pages 63-74, DOI 10.1145/1851182.1851192, October 2010.
  • [BA11] Bauer, Steven., Beverly, Robert., and Arthur. Berger, "Measuring the State of ECN Readiness in Servers, Clients, and Routers", Proceedings of the 2011 ACM SIGCOMM Conference on ICM, pages 171-180, DOI 10.1145/2068816.2068833, November 2011.
  • [DCTCP] Bensley, S., Eggert, L., Thaler, D., Balasubramanian, P., and G. Judd, "Microsoft's Datacenter TCP (DCTCP): TCP Congestion Control for Datacenters", Work in Progress, draft-bensley-tcpm-dctcp-05, July 2015.
  • [ECN-ENCAP] Briscoe, B., Kaippallimalil, J., and P. Thaler, "Guidelines for Adding Congestion Notification to Protocols that Encapsulate IP", Work in Progress, draft-ietf-tsvwg-ecn-encap-guidelines-07, July 2016.
  • [ECN-FALLBACK] Kuehlewind, M. and B. Trammell, "A Mechanism for ECN Path Probing and Fallback", Work in Progress, draft-kuehlewind-tcpm-ecn-fallback-01, September 2013.
  • [ECN-FEEDBACK] Briscoe, B., Kuehlewind, M., and R. Scheffenegger, "More Accurate ECN Feedback in TCP", Work in Progress, draft-ietf-tcpm-accurate-ecn-02, October 2016.
  • [Fla13] Flach, Tobias., Dukkipati, Nandita., Terzis, Andreas., Raghavan, Barath., Cardwell, Neal., Cheng, Yuchung., Jain, Ankur., Hao, Shuai., Katz-Bassett, Ethan., and Ramesh. Govindan, "Reducing web latency: the virtue of gentle aggression", ACM SIGCOMM Computer Communication Review, Volume 43, Issue 4, pages 159-170, DOI 10.1145/2534169.2486014, October 2013.
  • [RFC768] Postel, J., "User Datagram Protocol", STD 6, RFC 768, DOI 10.17487/RFC0768, August 1980, <http://www.rfc-editor.org/info/rfc768>.
  • [RFC1349] Almquist, P., "Type of Service in the Internet Protocol Suite", RFC 1349, DOI 10.17487/RFC1349, July 1992, <http://www.rfc-editor.org/info/rfc1349>.
  • [RFC3649] Floyd, S., "HighSpeed TCP for Large Congestion Windows", RFC 3649, DOI 10.17487/RFC3649, December 2003, <http://www.rfc-editor.org/info/rfc3649>.
  • [RFC3758] Stewart, R., Ramalho, M., Xie, Q., Tuexen, M., and P. Conrad, "Stream Control Transmission Protocol (SCTP) Partial Reliability Extension", RFC 3758, DOI 10.17487/RFC3758, May 2004, <http://www.rfc-editor.org/info/rfc3758>.
  • [RFC4340] Kohler, E., Handley, M., and S. Floyd, "Datagram Congestion Control Protocol (DCCP)", RFC 4340, DOI 10.17487/RFC4340, March 2006, <http://www.rfc-editor.org/info/rfc4340>.
  • [RFC4774] Floyd, S., "Specifying Alternate Semantics for the Explicit Congestion Notification (ECN) Field", BCP 124, RFC 4774, DOI 10.17487/RFC4774, November 2006, <http://www.rfc-editor.org/info/rfc4774>.
  • [RFC5562] Kuzmanovic, A., Mondal, A., Floyd, S., and K. Ramakrishnan, "Adding Explicit Congestion Notification (ECN) Capability to TCP's SYN/ACK Packets", RFC 5562, DOI 10.17487/RFC5562, June 2009, <http://www.rfc-editor.org/info/rfc5562>.
  • [RFC5681] Allman, M., Paxson, V., and E. Blanton, "TCP Congestion Control", RFC 5681, DOI 10.17487/RFC5681, September 2009, <http://www.rfc-editor.org/info/rfc5681>.
  • [RFC6679] Westerlund, M., Johansson, I., Perkins, C., O'Hanlon, P., and K. Carlberg, "Explicit Congestion Notification (ECN) for RTP over UDP", RFC 6679, DOI 10.17487/RFC6679, August 2012, <http://www.rfc-editor.org/info/rfc6679>.
  • [RFC6789] Briscoe, B., Ed., Woundy, R., Ed., and A. Cooper, Ed., "Congestion Exposure (ConEx) Concepts and Use Cases", RFC 6789, DOI 10.17487/RFC6789, December 2012, <http://www.rfc-editor.org/info/rfc6789>.
  • [RFC7928] Kuhn, N., Ed., Natarajan, P., Ed., Khademi, N., Ed., and D. Ros, "Characterization Guidelines for Active Queue Management (AQM)", RFC 7928, DOI 10.17487/RFC7928, July 2016, <http://www.rfc-editor.org/info/rfc7928>.
  • [ST14] Stewart, R., Tuexen, M., and X. Dong, "ECN for Stream Control Transmission Protocol (SCTP)", Work in Progress, draft-stewart-tsvwg-sctpecn-05, January 2014.
  • [TR15] Tranmmel, Brian., Kuehlewind, Mirja., Boppart, Damiano, Learmonth, Iain., and Gorry. Fairhurst, "Enabling Internet-Wide Deployment of Explicit Congestion Notification", Lecture Notes in Computer Science, Volume 8995, pp 193-205, DOI 10.1007/978-3-319-15509-8_15, March 2015.

Acknowledgements 🫶

Huge thanks to everyone who helped make this happen 🌸 The authors were partly funded by the European Community under its Seventh Framework Programme through the Reducing Internet Transport Latency (RITE) project (ICT-317700). The views expressed are solely those of the authors.

The authors would also like to thank the following people for their comments on prior draft versions of this document: Bob Briscoe, David Collier-Brown, Colin Perkins, Richard Scheffenegger, Dave Taht, Wes Eddy, Fred Baker, Mikael Abrahamsson, Mirja Kuehlewind, John Leslie, and other members of the TSVWG and AQM working groups 💜

Authors' Addresses 🦋

Godred Fairhurst
University of Aberdeen
School of Engineering, Fraser Noble Building
Aberdeen  AB24 3UE
United Kingdom

Email: gorry@erg.abdn.ac.uk


Michael Welzl
University of Oslo
PO Box 1080 Blindern
Oslo  N-0316
Norway

Phone: +47 22 85 24 20
Email: michawe@ifi.uio.no
📦 where this page comes from