RFC 3540

RFC 3540 💖 Robust Explicit Congestion Notification (ECN) Signaling with Nonces

Okay bestie, the header is giving 📄. It's from the Network Working Group, filed as RFC 3540 in June 2003 by N. Spring, D. Wetherall, and D. Ely, all at the University of Washington. The category is Experimental 🧪. There are no Obsoletes or Updates lines in the header, so this one doesn't replace or change any other RFC.

Status of This Memo ✨

This memo defines an Experimental Protocol for the Internet community 🧪. It's not an Internet standard of any kind. They want feedback and suggestions for improvement, and anyone can distribute it freely 💌.

Copyright Notice 📜

Copyright (C) The Internet Society (2003). All Rights Reserved.

Abstract 🌸

This note describes the Explicit Congestion Notification (ECN)-nonce, an optional add-on to ECN 🎀. Its whole job is protecting against marked packets getting hidden from the TCP sender, whether by accident or on purpose 🙈. That makes congestion control more robust, because a receiver can't use ECN to grab an unfair chunk of bandwidth 💅. The ECN-nonce uses the two ECN-Capable Transport (ECT) codepoints in the ECN field of the IP header, plus a flag in the TCP header. It's cheap to compute for both routers and hosts 🦋.

1. Introduction ✨

Statement of Intent

This spec adds an optional feature to Explicit Congestion Notification [RFC3168] that makes it harder to hide marked packets, whether that's malicious or accidental 🕵️‍♀️. It hasn't been widely deployed yet 🌱. Publishing it as an Experimental RFC is a careful move: the goal is to encourage people to actually use it and deploy it before it goes on the standards track 🧪. It also gives firewall developers time to learn to recognize and accept the pattern the nonce creates 🔥. The Transport Area Working Group plans to resubmit this as an IETF Proposed Standard later on, once there's more experience with it 👑.

The ECN signaling loop only works if the receiver cooperates by sending Congestion Experienced signals back to the sender, and the protocol has no way to force that 😬. So an untrustworthy or buggy receiver could always clear ECN-Echo and never report congestion. That would give the receiver a speed boost while the honest connections competing with it take the hit 💀. Even worse, any device on the path, like a NAT box, a firewall, or a QoS bandwidth shaper, could strip out congestion marks and get away with it 🙃.

Whether those behaviors actually threaten congestion control in the Internet is debatable. But congestion control is central to everything, so it's smart to design the ECN signaling loop to resist as many threats as possible 🛡️. That way ECN gives people a real reason to improve on the previous state-of-the-art, without giving anyone an incentive to abuse it 💜. The ECN-nonce is a simple, efficient way to kill that abuse potential ✨.

The ECN-nonce lets the sender check that the ECN receiver is behaving correctly, and that nothing else on the signaling path is hiding marked (or dropped) packets 🔍. It protects against both implementation mistakes and deliberate cheating 💅. The ECN-nonce:

• catches a misbehaving receiver with a high probability, and never implicates an innocent receiver.
• does not change other aspects of ECN, nor does it reduce the benefits of ECN for behaving receivers.
• is cheap in both per-packet overhead (one TCP header flag) and processing requirements.
• is simple and, to the best of our knowledge, not prone to other attacks.

Here's a bonus: even with only drop-tail routers, the ECN-nonce has two extra perks 🎁. First, packet drops can't be hidden from the sender. Second, it stops optimistic acknowledgements [Savage], where TCP segments get acknowledged before they've actually arrived 🙅‍♀️. Both of those make congestion control harder to attack. The doc doesn't go deep on these perks, so we'll leave them there 🤷‍♀️.

The rest of the document is the ECN-nonce itself: an overview first, then the detailed behavior for senders and receivers 📋.

The keywords MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, SHOULD NOT, RECOMMENDED, MAY, and OPTIONAL, when they appear in this document, are to be interpreted as described in [RFC2119] 🔑.

2. Overview 🦋

The ECN-nonce builds on the existing ECN-Echo and Congestion Window Reduced (CWR) signaling mechanism 💖. You're expected to already know ECN [ECN], so this part skips the basics 🙈. To keep things simple, we describe the ECN-nonce in one direction only, even though it actually runs in both directions at once ↔️.

The ECN protocol for TCP stays the same, except for one new field in the TCP header 🎀. As in [RFC3168], ECT(0) or ECT(1) (ECN-Capable Transport) gets set in the ECN field of the IP header on outgoing packets. Congested routers then flip that field to CE (Congestion Experienced) 🚦. When TCP receivers spot a CE, they set the ECE (ECN-Echo) flag on later acknowledgements until they get a CWR flag back. The sender sends the CWR flag on new data whenever it reacts to congestion 🔄.

The ECN-nonce adds to all of this so the receiver can prove to the sender that the segments being acknowledged arrived unmarked 🧾. Each packet carries a random one-bit value, called a nonce, encoded in the two ECT codepoints 🎲. The receiver returns the one-bit sum of those nonces in a TCP header flag called the nonce sum (NS) bit 🧮. When a packet gets marked, its nonce gets erased, because CE overwrites both ECN bits in the IP header 🧼. Since the sum needs every nonce to calculate, a correct nonce sum proves that only unmarked packets were received 💯. That means receivers can't hide marked packets, and middleboxes along the path can't unmark a packet unless they correctly guess the original nonce 🕵️‍♂️.

The sender checks the nonce sum the receiver sends back, which catches anyone hiding congestion signals in the form of marked (or dropped) packets 🔎. Because the nonce sum is just one bit, a lying receiver has a 50-50 chance of getting caught whenever an acknowledgement hides a mark 🪙. Every acknowledgement is an independent trial, so repeat cheaters get caught fast if there are repeated congestion signals 🚨.

The next paragraphs dig into the ECN-nonce protocol in more detail 🔬.

Each acknowledgement carries a nonce sum, which is the one-bit sum (exclusive-or, or parity) of the nonces over the byte range that acknowledgement covers 🧩. The sum exists because not every packet gets acknowledged individually, and acks aren't delivered reliably 📭. If there were no sum, a receiver could echo the nonce from an unmarked packet to prove it arrived unmarked. But since acks can get lost, the sender couldn't tell a lost ACK from one that was never sent just to hide a marked packet 😱. The nonce sum stops receivers from hiding individual marked packets by simply not acknowledging them 🙅‍♀️. And since both the nonce and the nonce sum are one-bit values, the sum is no easier to guess than the individual nonces 🧠. Here's the nonce sum calculation in Figure 1 🎀:

    Sender             Receiver
                         initial sum = 1
      -- 1:4 ECT(0)  --> NS = 1 + 0(1:4) = 1(:4)
      <- ACK 4, NS=1 ---
      -- 4:8 ECT(1)  --> NS = 1(:4) + 1(4:8) = 0(:8)
      <- ACK 8, NS=0 ---
      -- 8:12 ECT(1)  -> NS = 0(:8) + 1(8:12) = 1(:12)
      <- ACK 12, NS=1 --
      -- 12:16 ECT(1) -> NS = 1(:12) + 1(12:16) = 0(:16)
      <- ACK 16, NS=0 --

Figure 1: The calculation of nonce sums at the receiver.

After congestion hits and packets get marked or lost, the sender and receiver nonce sums need to resynchronize 🔁. When packets get marked, their nonces get cleared, so the receiver's sum stops matching the sender's 😵. Once nonces are lost, the gap between the sender's and receiver's sums stays constant until there's another loss 📏. So the fix is to have the sender set its nonce sum to match the receiver's 🪄. Congestion signals don't need to be sent more than once per round trip, so the sender pauses checking while the CWR signal is in flight, then resets its nonce sum to the receiver's when new data is acknowledged ⏸️. The big win is that the receiver doesn't have to take part in resynchronization at all 🙌. Figure 2 shows how that goes. Notice that the nonce sum in ACK 12 (NS=0) is different from the previous example (NS=1), and it stays different for ACK 16 👀.

    Sender              Receiver
                            initial sum = 1
      -- 1:4 ECT(0)       -> NS = 1 + 0(1:4) = 1(:4)
      <- ACK 4, NS=1      --
      -- 4:8 ECT(1) -> CE -> NS = 1(:4) + ?(4:8) = 1(:8)
      <- ACK 8, ECE NS=1  --
      -- 8:12 ECT(1), CWR -> NS = 1(:8) + 1(8:12) = 0(:12)
      <- ACK 12, NS=0     --
      -- 12:16 ECT(1)     -> NS = 0(:12) + 1(12:16) = 1(:16)
      <- ACK 16, NS=1     --

Figure 2: The calculation of nonce sums at the receiver when a packet (4:8) is marked. The receiver may calculate the wrong nonce sum when the original nonce information is lost after a packet is marked.

Third, nonces ride along with packets, but acknowledgements cover byte ranges, and those two things don't always line up 🧵. Acknowledged byte boundaries don't have to match the transmitted ones, and data can get retransmitted in packets with different boundaries 📦. The first issue is how a receiver sets a nonce when it acknowledges only part of a segment, and that's covered in Section 6.1 🔗. The second question is what nonce to send when smaller segments get retransmitted as one large segment. The answer is simple: ECN is turned off for retransmissions, so they can't carry a nonce 🙅‍♀️. Retransmissions are tied to congestion events, so nonce checking is paused until CWR is acknowledged and the congestion event is over ⏳.

The next sections walk through the detailed behavior of senders, routers, and receivers. We start with sender transmit behavior, then go around the ECN signaling loop, and finish with sender acknowledgement processing 🏁.

3. Sender Behavior (Transmit) 📤

Senders still manage CWR and ECN-Echo the way they always have 💅. On top of that, they must place nonces on packets as they're transmitted, and check the validity of the nonce sums in acknowledgments as they arrive 🔐. This section covers the transmit side 🚀.

To put a one-bit nonce value on every ECN-capable IP packet, the sender uses the two ECT codepoints: ECT(0) stands for a nonce of 0, and ECT(1) for a nonce of 1 🎀. Just like regular ECN, retransmissions aren't ECN-capable, so they don't carry a nonce 🙈.

The sender keeps a mapping from each packet's end sequence number to the expected nonce sum 🗺️. Note that it's the expected nonce sum, not the nonce that was placed in the original transmission, that gets stored for the acknowledgement carrying that sequence number 🧠.

4. Router Behavior ✨

okay so routers just do what [RFC3168] says 💅 when a router marks a packet to say "hey, congestion here," the original nonce value (the ECT(0) or ECT(1) codepoint) gets wiped out 🫣 no cap, that's the whole trick. And nobody can quietly unmark the packet, not the receiver, not anyone else, unless they can successfully guess the original nonce value. Main character energy for the protocol 🦋

5. Receiver Behavior (Receive and Transmit) 🎀

bestie, ECN-nonce receivers keep a running nonce sum as in-order packets show up, and they send the current nonce sum back in every acknowledgement 🫶 otherwise the receiver acts exactly like it does in [RFC3168], so nothing new to panic about. Sending the nonce sum back is optional, but recommended 💜 ngl the reason is that senders are allowed to stop sending ECN-capable packets to receivers that don't support the ECN-nonce, so it's kinda important to show you're on the team 😭

When packets come out of the queue of out-of-order packets waiting to be acknowledged, the receiver pulls the nonce back out of the IP header 🧚 then it adds that nonce to the current nonce sum right as the acknowledgement sequence number moves forward for that recent packet. It's giving precision 🔥

Now for marked packets, which is where it gets spicy 🌶️ the receiver might not know one or more of the nonce values, since those got erased. In that case the missing nonce values get ignored when the sum is calculated (or, same thing, a value of zero is assumed), and ECN-Echo gets set to tell the sender about the congestion 🚨

Returning the nonce sum for a given acknowledgement is honestly so simple 💖 it rides in one "NS" (Nonce Sum) bit in the TCP header. That bit sits right next to the CWR and ECN-Echo bits. It's Bit 7 in byte 13 of the TCP header, which you can see here:

     0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15
   +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
   |               |           | N | C | E | U | A | P | R | S | F |
   | Header Length | Reserved  | S | W | C | R | C | S | S | Y | I |
   |               |           |   | R | E | G | K | H | T | N | N |
   +---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+

Figure 3: The new definition of bytes 13 and 14 of the TCP Header.

The initial nonce sum is 1 🌸 and it gets included in the SYN/ACK and the ACK of the three-way TCP handshake. This lets the other endpoint figure out that nonce support is there, but it isn't a negotiation 🙈 the receiver of the SYN/ACK doesn't need to check whether NS is set before deciding to set NS in the ACK that follows. Iconic, no drama.

6. Sender Behavior (Receive) 🍓

this section finishes the sender side 🫶 it's all about how senders check whether the nonce sums are valid.

The nonce sum gets checked whenever an acknowledgement of new data comes in, except during congestion recovery, where extra ECN-Echo signals would just get ignored anyway 💅 checking means comparing the correct nonce sum stored in a buffer with the one carried in the acknowledgement, and there's a correction for that which the next subsection covers.

If ECN-Echo is not set, the receiver is basically claiming it got no marked packets, so it can compute and return the correct nonce sum 🙃 but to hide a mark, the receiver would have to successfully guess the sum of the nonces it never received, since at least one packet was marked and its nonce got erased 👀 if the individual nonces are equally likely to be 0 or 1, then their sum is equally likely to be 0 or 1 too. So any guess is equally likely to be wrong, with a 50-50 shot of getting caught by the sender 😭 each new acknowledgement is its own independent trial, so a cheating receiver is likely to get caught after a small number of lies. Caught in 4K 🎀

If ECN-Echo is set, the receiver is sending a congestion signal, so there's no need to check the nonce sum ✨ the congestion window gets halved, CWR gets set on the next packet with new data sent, and ECN-Echo gets cleared once the CWR signal is received, same as in [RFC3168]. During this recovery, the sum might be wrong because one or more nonces didn't arrive, but that doesn't matter here, because TCP invokes congestion mechanisms at most once per RTT, whether there's one loss or several in that period 🧚

6.1. Resynchronization After Loss or Mark 🌈

after recovery, the sender and receiver nonce sums need to be re-synchronized so further acknowledgments can be checked 🔄 when the receiver's sum is wrong, it stays wrong until there's another loss.

So there's a simple re-synchronization mechanism 💜 the sender resets its nonce sum to match the receiver's when it gets an acknowledgment for new data sent after the congestion window was reduced. When it's responding to explicit congestion signals, that will be the first acknowledgement without the ECN-Echo flag set, which is the acknowledgement of the packet carrying the CWR flag.

    Sender              Receiver
                             initial sum = 1
      -- 1:4 ECT(0)       -> NS = 1 + 0(1:4) = 1(:4)
      <- ACK 4, NS=1      --
      -- 4:8 ECT(1) -> LOST
      -- 8:12 ECT(1)      -> nonce sum calculation deferred
                               until in-order data received
      <- ACK 4, NS=0      --
      -- 12:16 ECT(1)     -> nonce sum calculation deferred
      <- ACK 4, NS=0      --
      -- 4:8 retransmit   -> NS = 1(:4) + ?(4:8) +
                                  1(8:12) + 1(12:16) = 1(:16)
      <- ACK 16, NS=1     --
      -- 16:20 ECT(1) CWR ->
      <- ACK 20, NS=0     -- NS = 1(:16) + 1(16:20) = 0(:20)

Figure 4: The calculation of nonce sums at the receiver when a packet is lost, and resynchronization after loss. The nonce sum is not changed until the cumulative acknowledgement is advanced.

In practice, re-synchronization can be done by storing a bit that is one if the expected nonce sum the sender has stored and the received nonce sum in the acknowledgement of CWR differ, and zero otherwise 🧠 this synchronization offset bit can then be used in the comparison between the expected nonce sum and the received nonce sum. Big brain moves only 🫠

The sender should ignore the nonce sum returned on any acknowledgements bearing the ECN-echo flag 🙅‍♀️

When an acknowledgment covers only part of a segment, like when a middlebox resegments at the TCP layer instead of fragmenting IP packets, the sender should accept the nonce sum expected at the next segment boundary 🦋 in other words, an acknowledgement covering part of an original segment will include the nonce sum expected once the entire segment is acknowledged.

Last thing 💅 in ECN, senders can choose not to indicate ECN capability on some packets, for any reason. An ECN-nonce sender must re-synchronize after sending those ECN-incapable packets, as though a CWR had been sent with the first new data after them. Until that resynchronization happens, the sender loses protection for any unacknowledged packets 😭

6.2. Sender Behavior - Incorrect Nonce Received 🚨

what a sender does about an incorrect nonce is a matter of policy 🫶 it's separate from the checking mechanism, and it doesn't have to be handled the same way by every sender. Also, checking received nonce sums at all is optional, and it may be disabled.

If the receiver has never sent a non-zero nonce sum, the sender can infer that the receiver doesn't understand the nonce 🤷‍♀️ then it can rate limit the connection, put it in a lower-priority queue, or stop setting ECT in outgoing segments.

If the received nonce sum was set in a previous acknowledgement, the sender might infer that a network device has interfered with correct ECN signaling between ECN-nonce supporting endpoints 👀 the minimum response to an incorrect nonce is the same as the response to a received ECE. But to make up for hidden congestion signals, the sender might reduce the congestion window to one segment and stop setting ECT in outgoing segments. An incorrect nonce sum is a sign of misbehavior or error between ECN-nonce supporting endpoints. Not okay, not a vibe 💀

6.2.1. Using the ECN-nonce to Protect Against Other Misbehaviors 🕵️‍♀️

the ECN-nonce can do more than check that marked packets get signaled to the sender ✨ it also makes sure dropped packets can't be hidden from the sender, because their nonces got lost too. Drops could potentially be concealed by a faulty TCP implementation, certain attacks, or even a hypothetical TCP accelerator 🧚 that accelerator could gamble that it can either successfully "fast start" to a preset bandwidth quickly, retry over another connection, or provide reliability at the application level. If you still want protection against these faults, then the ECN-nonce shouldn't be disabled. Instead, reducing the congestion window to one, or using a low-priority queue, would penalize faulty operation while still keeping the checking going 😤

The ECN-nonce can also detect misbehavior in Eifel [Eifel], which is a recently proposed mechanism for removing retransmission ambiguity to improve TCP performance 🌸 a misbehaving receiver might claim it only received original transmissions, just to convince the sender to undo congestion actions. Since retransmissions are sent without ECT, and so have no nonce, returning the correct nonce sum confirms that only original transmissions were received. Caught red-handed 🫣

7. Interactions 🧚

7.1. Path MTU Discovery 🌈

as described in RFC3168, using the Don't Fragment bit with ECN is recommended 💜 receivers that get unmarked fragments can reconstruct the original nonce to hide a marked fragment. The ECN-nonce can't protect against misbehaving receivers that hide marked fragments, so some protection is lost in situations where Path MTU discovery is disabled 😔

When responding to a small path MTU, the sender retransmits a smaller frame in place of a larger one 📦 since these smaller packets are retransmissions, they'll be ECN-incapable and carry no nonce. The sender should re-synchronize on the first newly transmitted packet 🔄

7.2. SACK 🦋

selective acknowledgements let receivers acknowledge out-of-order segments as an optimization 🙌 it's not necessary to change the selective acknowledgment option to fit per-range nonce sums, because SACKs can't be used by a receiver to hide a congestion signal. The nonce sum only corresponds to the data acknowledged by the cumulative acknowledgement. Simple and iconic 💅

7.3. IPv6 🎀

even though the IPv4 header is protected by a checksum, IPv6 doesn't have that, so undetected bit errors in the IPv6 header are more likely 😬 bit errors that compromise the integrity of the congestion notification fields may cause an incorrect nonce to be received, and an incorrect nonce sum to be returned.

8. Security Considerations 🔐

the random one-bit nonces don't need to come from a cryptographic-quality pseudo-random number generator 🙈 a strong random number generator would compromise performance. So the sequence of random nonces should not be used for any other purpose. Keep it separate, bestie 🫶

On the flip side, the pseudo-random bit sequence should not be generated by a linear feedback shift register [Schneier], or any similar scheme that would let an adversary who has seen several previous bits work out the generation function and predict its future output 💀 that's a hard no.

Although the ECN-nonce protects against concealment of congestion signals and optimistic acknowledgement, it provides no additional protection for the integrity of the connection. Read that twice, periodt 🚨

9. IANA Considerations ✨

okay so the Nonce Sum (NS) lives in a reserved TCP header bit that still needs to get officially allocated 🙈. This doc uses Bit 7, right next to the other bits ECN already uses 💅.

The NS codepoint gets specified by the Standards Action of this RFC, which is what RFC 2780 asks for 📜. IANA went ahead and added this to the "TCP Header Flags" registry 🌸:

RFC 3540 defines bit 7 from the Reserved field to be used for the Nonce Sum, as follows:

  0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+
|               |           | N | C | E | U | A | P | R | S | F |
| Header Length | Reserved  | S | W | C | R | C | S | S | Y | I |
|               |           |   | R | E | G | K | H | T | N | N |
+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+

TCP Header Flags

Bit      Name                                    Reference
---      ----                                    ---------
 7        NS (Nonce Sum)                         [RFC 3540]

10. Conclusion 🦋

okay bestie, the ECN-nonce is a simple tweak to how ECN signals congestion, and it stops receivers from hiding marked (or dropped) packets 🫶. The whole point is to make congestion control in the Internet more robust ✨. The tweak keeps the vibe and simplicity of existing ECN, and it's realistic to actually deploy 💖. It uses the ECT(0) and ECT(1) codepoints, one TCP header flag (plus CWR and ECN-Echo), and processing rules that are super simple 👑.

11. References 📚

• [ECN] "The ECN Web Page", URL "http://www.icir.org/floyd/ecn.html".
• [RFC3168] Ramakrishnan, K., Floyd, S. and D. Black, "The addition of explicit congestion notification (ECN) to IP", RFC 3168, September 2001.
• [Eifel] R. Ludwig and R. Katz. The Eifel Algorithm: Making TCP Robust Against Spurious Retransmissions. Computer Communications Review, January, 2000.
• [B97] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
• [Savage] S. Savage, N. Cardwell, D. Wetherall, T. Anderson. TCP congestion control with a misbehaving receiver. SIGCOMM CCR, October 1999.
• [Schneier] Bruce Schneier. Applied Cryptography, 2nd ed., 1996

12. Acknowledgements 🫶

this note grew out of research by Stefan Savage, David Ely, David Wetherall, Tom Anderson and Neil Spring 🌈. They also gave huge thanks to Sally Floyd for her feedback and help 💜.

13. Authors' Addresses 🧚

Neil Spring
EMail: nspring@cs.washington.edu


David Wetherall
Department of Computer Science and Engineering, Box 352350
University of Washington
Seattle WA 98195-2350
EMail: djw@cs.washington.edu


David Ely
Computer Science and Engineering, 352350
University of Washington
Seattle, WA 98195-2350
EMail: ely@cs.washington.edu

14. Full Copyright Statement ⭐

lawyers wrote this part, so it stays word-for-word 🙈

Copyright (C) The Internet Society (2003). All Rights Reserved.

This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English.

The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns.

This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Acknowledgement 💖

Funding for the RFC Editor function is currently provided by the Internet Society 🎀.