RFC 5461

TCP's Reaction to Soft Errors

📅 published February 2009, from ietf

RFC 5461: TCP's Reaction to Soft Errors 🎀

okay bestie, RFC 5461 is giving Informational energy 🦋. It's by F. Gont from UTN/FRH, it was dated February 2009, and it came out of the Network Working Group. It doesn't list anything it obsoletes or updates, so it's standing on its own 💅.

Status of This Memo ✨

This one is just info for the Internet community 📚. It's not a standard of any kind, and anyone can distribute it freely.

Copyright (c) 2009 IETF Trust and the persons identified as the document authors. All rights reserved. 🔒

It's subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as they were on the publication date. Read those carefully, because they spell out your rights and restrictions for this doc 📜.

Abstract 🌸

This doc describes a modification to how TCP handles ICMP soft error messages. It's not a standard, but it's widely implemented 🫶. When those error messages show up, it rejects pending connection requests. That makes long delays between connection-establishment attempts less likely. The scenarios where this matters include dual-stack nodes with IPv6 enabled by default, deployed in IPv4 or mixed IPv4 and IPv6 environments 🌈.

Table of Contents 🗺️

1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
2.  Error Handling in TCP  . . . . . . . . . . . . . . . . . . . .  3
  2.1.  Reaction to ICMP Error Messages That Indicate Hard
        Errors . . . . . . . . . . . . . . . . . . . . . . . . . .  4
  2.2.  Reaction to ICMP Error Messages That Indicate Soft
        Errors . . . . . . . . . . . . . . . . . . . . . . . . . .  5
3.  Problems That May Arise from TCP's Reaction to Soft Errors . .  5
  3.1.  General Discussion . . . . . . . . . . . . . . . . . . . .  5
  3.2.  Problems That May Arise with Dual-Stack IPv6 on by
        Default  . . . . . . . . . . . . . . . . . . . . . . . . .  6
4.  Deployed Workarounds for Long Delays between
    Connection-Establishment Attempts  . . . . . . . . . . . . . .  7
  4.1.  Context-Sensitive ICMP/TCP Interaction . . . . . . . . . .  7
  4.2.  Context-Sensitive ICMP/TCP Interaction with Repeated
        Confirmation . . . . . . . . . . . . . . . . . . . . . . .  8
5.  Possible Drawbacks of Changing ICMP Semantics  . . . . . . . .  9
  5.1.  Non-Deterministic Transient Network Failures . . . . . . .  9
  5.2.  Deterministic Transient Network Failures . . . . . . . . . 10
  5.3.  Non-Compliant Network Address Translators (NATs) . . . . . 10
6.  Security Considerations  . . . . . . . . . . . . . . . . . . . 10
7.  Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 11
8.  Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 11
9.  References . . . . . . . . . . . . . . . . . . . . . . . . . . 12
  9.1.  Normative References . . . . . . . . . . . . . . . . . . . 12
  9.2.  Informative References . . . . . . . . . . . . . . . . . . 12

1. Introduction 💁‍♀️

okay so handling network failures splits into two jobs 👯. First there's fault isolation, which is how hosts and routers figure out that something's broken 🔍. Then there's fault recovery, which is what they do to keep going through the failure anyway [RFC0816] 🛠️.

In the Internet architecture, ICMP [RFC0792] is one fault isolation tool. It tells the hosts sending datagrams that something's wrong with the network 📡.

When a host gets told about a network error, its network stack tries to keep talking if it can 🫠. How it recovers depends on what kind of failure happened and when the error gets detected ⏰.

This doc looks at the problems TCP's fault recovery can cause when it reacts to ICMP soft errors 😬. It covers what happens when a host tries to connect to a multihomed host that has some unreachable addresses. It also covers the case where dual-stack nodes with IPv6 on by default get deployed in IPv4 or mixed IPv4 and IPv6 networks 🌐.

Last but not least, it documents a change to how TCP reacts to ICMP soft error messages during connection startup. A bunch of TCP/IP stacks have implemented it to fix the problems described here 🧩. We really want to stress that this change goes against standard behavior, and this doc does not change the standard reaction at all 🚫.

[Gont] covers other ways to deal with long delays between connection-establishment attempts in TCP 📖.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119] 🤓.

2. Error Handling in TCP 🧯

Network errors come in two flavors: soft and hard 🍦. Soft errors are transient network failures that will probably get fixed pretty soon ⚡. Hard errors are network error conditions that probably won't get fixed anytime soon 🧱.

The Host Requirements RFC [RFC1122] says in Section 4.2.3.9 which ICMP messages mean soft errors 📋. Those are ICMP "Destination Unreachable" codes 0 (network unreachable), 1 (host unreachable), and 5 (source route failed). They also include ICMP "Time Exceeded" codes 0 (time to live exceeded in transit) and 1 (fragment reassembly time exceeded), plus ICMP "Parameter Problem" 💅.

ICMPv6 didn't exist when [RFC1122] was written, but you can totally extend the soft error idea to it 🧠. That gives you ICMPv6 "Destination Unreachable" codes 0 (no route to destination) and 3 (address unreachable). It gives you ICMPv6 "Time Exceeded" codes 0 (hop limit exceeded in transit) and 1 (fragment reassembly time exceeded). And it gives you ICMPv6 "Parameter Problem" codes 0 (erroneous header field encountered), 1 (unrecognized Next Header type encountered), and 2 (unrecognized IPv6 option encountered) [RFC4443] ✨.

+----------------------------------+--------------------------------+
|               ICMP               |             ICMPv6             |
+----------------------------------+--------------------------------+
|  Destination Unreachable (codes  | Destination Unreachable (codes |
|           0, 1, and 5)           |            0 and 3)            |
+----------------------------------+--------------------------------+
|   Time Exceeded (codes 0 and 1)  |  Time Exceeded (codes 0 and 1) |
+----------------------------------+--------------------------------+
|         Parameter Problem        | Parameter Problem (codes 0, 1, |
|                                  |             and 2)             |
+----------------------------------+--------------------------------+

     Table 1: Extrapolating the concept of soft errors to ICMPv6

When a network failure doesn't get signaled to the sending host, like when a gateway corrupts packets, TCP just keeps resending that data 🔁. It does this until the data gets acknowledged or the connection times out 😵‍💫.

If a host does get an ICMP error message about an ongoing TCP connection, the IP layer passes it up to the right TCP instance so it knows about the failure [RFC1122] 📬. How TCP reacts depends on what kind of error it is 🎭.

2.1. Reaction to ICMP Error Messages That Indicate Hard Errors 💀

When a compliant TCP implementation gets an ICMP error message for a hard error, it just aborts that connection, no matter what state the connection is in 🪓.

The Host Requirements RFC [RFC1122] says in Section 4.2.3.9 that TCP SHOULD abort connections when it receives ICMP error messages for hard errors 📜. The logic is that hard errors mean the network problem won't change any time soon, so TCP can't really recover from it in a useful way 🙅‍♀️.

Honestly, basically no current TCP implementations follow the advice in [RFC1122] 😭. When an ICMP hard error comes in for a connection in any of the synchronized states, they don't abort the connection [ICMP-ATTACKS] 🙈.

2.2. Reaction to ICMP Error Messages That Indicate Soft Errors 🌷

When an ICMP error message says there's a soft error, TCP keeps resending the data until it gets acknowledged or the connection times out 🔄. On top of that, the TCP sender might save the info for later use (see [Stevens], pp. 317-319) 🗂️.

The Host Requirements RFC [RFC1122] says in Section 4.2.3.9 that TCP MUST NOT abort connections when it receives ICMP error messages for soft errors 🛑. The reasoning is that soft errors are transient failures that will hopefully get fixed soon, so one of the retransmissions should succeed 🤞.

When the connection timer runs out and an ICMP soft error message came in before that, TCP can use that info to give the user a more specific error message (see [Stevens], pp. 317-319) 💌.

This soft error reaction is really cool because it takes advantage of a big Internet feature. For lots of network failures, the network can rebuild itself dynamically without disrupting the endpoints at all 🦋✨.

3. Problems That May Arise from TCP's Reaction to Soft Errors 🫣

3.1. General Discussion

Okay so even though TCP's recovery plan for soft errors lets connections survive short-lived network hiccups 🦋, there are situations where that exact policy can cause some not-great effects 😭.

Picture this: an app on a local host is trying to talk to a destination whose name resolves to several IP addresses. The app usually cycles through the list of IP addresses until one works [RFC1123]. Now say some (but not all) of those addresses are permanently unreachable 💀. If a permanently unreachable one is first in line, the app will probably try it first and sit there blocking, waiting for a timeout, before it tries an alternate address.

As we talked about in Section 2, that unreachability may or may not get signaled back to the sending host. If the local TCP isn't told about the error, there's honestly not much it can do except keep retransmitting the SYN segment and wait for TCP's existing timeout mechanism, or an application timeout, to kick in. But even if an intermediate router signals the unreachability to the local TCP with an ICMP soft error message, the local TCP will still keep retransmitting the SYN segment until the connection timer expires (in the hopes that the error is transient) 🤞. The Host Requirements RFC [RFC1122] says this timer MUST be large enough to provide retransmission of the SYN segment for at least 3 minutes. So the app on the local host would spend several minutes for each unreachable address it tries to connect to 😵. Those long gaps between connection-establishment attempts would be inappropriate for lots of interactive apps, such as the Web 🙈. [Shneiderman] and [Thadani] give some insight into interactive systems (e.g., how response time affects an app's usability) ✨. The big takeaway: there's no one definition of a "transient error", and how persistent you are in the face of failure is a tradeoff 💅.

It's worth noting that while most apps try the addresses from the name-to-address function one after another, serially, that's definitely not the only option 🧚. For example, apps could try multiple addresses in parallel until one succeeds, which could dodge the long-delay problem described in this document [Gont] 🌈.

3.2. Problems That May Arise with Dual-Stack IPv6 on by Default

A super specific scenario where this kind of problem keeps happening is when dual-stack nodes with IPv6 enabled by default are deployed in IPv4 or mixed IPv4 and IPv6 environments and the IPv6 connectivity is non-existent [RFC4943] 🥲.

As [RFC4943] discusses, there are two possible variants of this scenario, and they differ in whether or not the lack of connectivity gets signaled to the sending node 🎀.

When packets sent to a destination are silently dropped and no ICMPv6 [RFC4443] errors get generated, there's little to do except wait for TCP's existing connection-timeout mechanism, or an application timeout, to trigger 😩.

In scenarios where a legacy node has no default routers and Neighbor Unreachability Detection (NUD) [RFC4861] fails for destinations assumed to be on-link, or where firewalls or other systems that enforce scope boundaries send ICMPv6 errors, the sending node will be signaled of the unreachability problem. However, as discussed in Section 2.2, compliant TCP implementations will not abort connections when receiving ICMP error messages that indicate soft errors 💖.

4. Deployed Workarounds for Long Delays between Connection-Establishment Attempts ✨

The following subsections describe a number of workarounds for the problem of long delays between connection-establishment attempts that have been implemented in a variety of TCP/IP stacks 🛠️. We note that treating soft errors as hard errors during connection establishment is super widespread, but it is not part of standard TCP behavior, and this document does not change that state of affairs 🙅. The consensus of the TCPM WG (TCP Maintenance and Minor Extensions Working Group) was to document this widespread implementation of nonstandard TCP behavior but to not change the TCP standard 🫶.

4.1. Context-Sensitive ICMP/TCP Interaction

As we said in Section 1, the fault recovery action might make sense to depend not only on the type of error being reported but also on the state of the connection the error is reported against 🧠. For example, you could infer that when an error arrives in response to opening a new connection, it's probably caused by opening the connection improperly, rather than by a transient network failure [RFC0816].

A bunch of TCP implementations have modified their reaction to all ICMP soft errors and now treat them as hard errors when they're received for connections in the SYN-SENT or SYN-RECEIVED states. For example, this workaround has been implemented in the Linux kernel since version 2.0.0 (released in 1996) [Linux] 🐧. However, it should be noted that this change violates section 4.2.3.9 of [RFC1122], which states that these ICMP error messages indicate soft error conditions and that, therefore, TCP MUST NOT abort the corresponding connection 🚫.

[RFC3168] states that a host that receives a RST in response to the transmission of an ECN (Explicit Congestion Notification)-setup SYN packet MAY resend a SYN with the CWR (Congestion Window Reduced) and ECE (ECN-Echo) bits cleared 🎀. This is meant to deal with faulty middle-boxes that reject connections when a SYN segment has the ECE and CWR bits set. Some faulty middle-boxes (e.g., firewalls) may reject these connection requests with an ICMP soft error of type 3 (Destination Unreachable), code 0 (net unreachable) or 1 (host unreachable), instead of a RST 🤨. Therefore, a system that processes ICMP soft error messages as hard errors when they are received for a connection in any of the non-synchronized states could resend the SYN segment with the ECE and CWR bits cleared when an ICMP "net unreachable" (type 3, code 0) or "host unreachable" (type 3, code 1) error message is received in response to a SYN segment that had these bits set 🔁.

Section 4.2 discusses a more conservative approach than that sketched above, which is implemented in FreeBSD 🐡.

4.2. Context-Sensitive ICMP/TCP Interaction with Repeated Confirmation

A more conservative approach than simply treating soft errors as hard errors (as described above) would be to abort a connection in the SYN-SENT or SYN-RECEIVED states only after an ICMP soft error has been received a specified number of times and the SYN segment has been retransmitted more than some specified number of times 🧐.

Two new parameters would have to be introduced to TCP, to be used only during the connection-establishment phase: MAXSYNREXMIT and MAXSOFTERROR. MAXSYNREXMIT would specify the number of times the SYN segment would have to be retransmitted before a connection is aborted. MAXSOFTERROR would specify the number of ICMP messages indicating soft errors that would have to be received before a connection is aborted.

Two additional state variables would need to be introduced to store additional state information during the connection-establishment phase: nsynrexmit and nsofterror. Both would be initialized to zero when a connection attempt is initiated, with nsynrexmit being incremented by one every time the SYN segment is retransmitted and nsofterror being incremented by one every time an ICMP message that indicates a soft error is received 📝.

A connection in the SYN-SENT or SYN-RECEIVED states would be aborted if nsynrexmit was greater than MAXSYNREXMIT and nsofterror was simultaneously greater than MAXSOFTERROR 💫.

This approach would give the network more time to solve the connectivity problem than does simply aborting a connection attempt upon reception of the first soft error. However, it should be noted that, depending on the values chosen for the MAXSYNREXMIT and MAXSOFTERROR parameters, this approach could still lead to long delays between connection-establishment attempts, thus not solving the problem 🐢. For example, BSD systems abort connections in the SYN-SENT or the SYN-RECEIVED state when a second ICMP error is received and the SYN segment has been retransmitted more than three times. They also set up a "connection-establishment timer" that imposes an upper limit on the time the connection-establishment attempt has to succeed, which expires after 75 seconds (see [Stevens2], pp. 828-829). Even when this policy may be better than the three-minute timeout policy specified in [RFC1122], it may still be inappropriate for handling the potential problems described in this document. This more conservative approach has been implemented in BSD systems for more than ten years [Stevens2] 🕰️.

We also note that the approach given in this section is a generalized version of the approach sketched in the previous section. In particular, with MAXSOFTERROR set to 1 and MAXSYNREXMIT set to zero, the schemes are identical 🤓.

5. Possible Drawbacks of Changing ICMP Semantics

The following subsections discuss some possible drawbacks that could arise from use of the non-standard modifications to TCP's reaction to soft errors, which are described in Section 4.1 and Section 4.2 🥺.

5.1. Non-Deterministic Transient Network Failures

In scenarios where a transient network failure affects all of the addresses returned by the name-to-address translation function, all destinations could be unreachable for some short period of time. For example, a mobile system consisting of a cell and a repeater may pass through a tunnel, leading to a loss of connectivity at the repeater, with the repeater sending ICMP soft errors back to the cell. Also, a transient routing problem might lead some intervening router to drop a SYN segment that was meaning to establish a TCP connection and send an ICMP soft error back to the host. Finally, a SYN segment carrying data might get fragmented and some of the resulting fragments might get lost, with the destination host timing out the reassembly process and sending an ICMP soft error back to the sending host (although this particular scenario is unlikely because, while [RFC0793] allows SYN segments to carry data, in practice they do not). In such scenarios, the application could quickly cycle through all the IP addresses in the list and return an error, when it could have let TCP retry a destination a few seconds later, when the transient problem could have disappeared. In this case, the modifications described here make TCP less robust than a standards-compliant implementation 😬.

Additionally, in many cases a domain name maps to a single IP address. In such a case, it might be better to try that address persistently according to normal TCP rules, instead of just aborting the pending connection upon receipt of an ICMP soft error 🫠.

5.2. Deterministic Transient Network Failures 🌸

Okay, not every transient network failure is random! 🎲 Sometimes it's totally predictable, bestie. Picture upstream connectivity that only switches on when someone actually uses the network ("as needed," iykyk). The connection that kicks it off could get hit with ICMP Destination Unreachables while the upstream link is still spinning up, and then it gets aborted 💀. Again, in this case the modifications described here make TCP less robust than a standards-compliant implementation. Not iconic 😭.

5.3. Non-Compliant Network Address Translators (NATs) 🚧

Some NATs respond to an unsolicited inbound SYN segment with an ICMP soft error message. If the system sending that unsolicited SYN segment has the workaround from this document, it'll abort the connection as soon as the ICMP error shows up, which probably stops TCP's simultaneous open from working through the NAT 🙈. BUT, it must be stressed that the NATs in this section are not BEHAVE-compliant, so they should implement REQ-4 of [RFC5382] instead 💅.

In the scenarios where a non-BEHAVE-compliant NAT is deployed, TCP simultaneous opens could fail. Not ideal, but tolerable in lots of situations ✨. For instance, a number of host implementations of TCP don't support TCP simultaneous opens [Zuquete], so it's not the end of the world 🌈.

6. Security Considerations 🔐

This document describes a non-standard modification to TCP's reaction to soft errors that has been implemented in a variety of TCP implementations. The modification makes TCP abort a connection in the SYN-SENT or the SYN-RECEIVED states when it receives an ICMP error message that indicates a soft error. So, the modification could be exploited to reset valid connections during the connection-establishment phase 🚨.

The non-standard workaround described in this document makes TCP more vulnerable to attack, even if only slightly. However, we note that an attacker wishing to reset ongoing TCP connections could send any of the ICMP hard error messages in any connection state. So this isn't the only door 🚪.

Generally, TCP backs off its retransmission timer each time it retransmits the SYN segment for the same connection. If a TCP implements the modification described in this document, that is, tries the next address in the list upon receipt of an ICMP error message, it might end up injecting more packets into the network than if it had simply retried the same address a number of times 📦. However, compliant TCP implementations might already incur this behavior (e.g., as a result of cycling through the list of IP addresses in response to RST segments), because there are currently no recommendations on methods for limiting the rate at which SYN segments are sent for connecting to a specific destination 🤷‍♀️.

A discussion of the use of ICMP to perform a variety of attacks against TCP, and a number of counter-measures that minimize the impact of these attacks, can be found in [ICMP-ATTACKS] 🕵️‍♀️.

A discussion of the security issues arising from the use of ICMPv6 can be found in [RFC4443] 🛡️.

7. Acknowledgements 🫶

The author wishes to thank Mark Allman, Jari Arkko, David Black, Ron Bonica, Ted Faber, Gorry Fairhurst, Sally Floyd, Juan Fraschini, Tomohiro Fujisaki, Guillermo Gont, Saikat Guha, Alfred Hoenes, Michael Kerrisk, Eddie Kohler, Mika Liljeberg, Arifumi Matsumoto, Sandy Murphy, Carlos Pignataro, Pasi Sarolahti, Pekka Savola, Pyda Srisuresh, Jinmei Tatuya, and Joe Touch for contributing many valuable comments on earlier versions of this document 💜.

The author wishes to thank Secretaria de Extension Universitaria at Universidad Tecnologica Nacional and Universidad Tecnologica Nacional/Facultad Regional Haedo for their support in this work 🎀.

Finally, the author wishes to express deep and heartfelt gratitude to Jorge Oscar Gont and Nelida Garcia for their precious motivation and guidance 🌟.

8. Contributors 👑

Mika Liljeberg was the first to describe how their implementation treated soft errors. Based on that, the solution discussed in Section 4.1 was documented in [v6-ON] by Sebastien Roy, Alain Durand, and James Paugh 🧚.

9. References 📚

9.1. Normative References

  • [RFC0792] Postel, J., "Internet Control Message Protocol", STD 5, RFC 792, September 1981.
  • [RFC0793] Postel, J., "Transmission Control Protocol", STD 7, RFC 793, September 1981.
  • [RFC1122] Braden, R., "Requirements for Internet Hosts - Communication Layers", STD 3, RFC 1122, October 1989.
  • [RFC1123] Braden, R., "Requirements for Internet Hosts - Application and Support", STD 3, RFC 1123, October 1989.
  • [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
  • [RFC3168] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition of Explicit Congestion Notification (ECN) to IP", RFC 3168, September 2001.
  • [RFC4443] Conta, A., Deering, S., and M. Gupta, "Internet Control Message Protocol (ICMPv6) for the Internet Protocol Version 6 (IPv6) Specification", RFC 4443, March 2006.
  • [RFC4861] Narten, T., Nordmark, E., Simpson, W., and H. Soliman, "Neighbor Discovery for IP version 6 (IPv6)", RFC 4861, September 2007.

9.2. Informative References

  • [Gont] Gont, F., "On the problem of long delays between connection-establishment attempts in TCP", Work in Progress, January 2009.
  • [ICMP-ATTACKS] Gont, F., "ICMP attacks against TCP", Work in Progress, October 2008.
  • [Linux] The Linux Project, "http://www.kernel.org".
  • [RFC0816] Clark, D., "Fault isolation and recovery", RFC 816, July 1982.
  • [RFC4943] Roy, S., Durand, A., and J. Paugh, "IPv6 Neighbor Discovery On-Link Assumption Considered Harmful", RFC 4943, September 2007.
  • [RFC5382] Guha, S., Biswas, K., Ford, B., Sivakumar, S., and P. Srisuresh, "NAT Behavioral Requirements for TCP", BCP 142, RFC 5382, October 2008.
  • [Shneiderman] Shneiderman, B., "Response Time and Display Rate in Human Performance with Computers", ACM Computing Surveys, 1984.
  • [Stevens] Stevens, W., "TCP/IP Illustrated, Volume 1: The Protocols", Addison-Wesley, 1994.
  • [Stevens2] Wright, G. and W. Stevens, "TCP/IP Illustrated, Volume 2: The Implementation", Addison-Wesley, 1994.
  • [Thadani] Thadani, A., "Interactive User Productivity", IBM Systems Journal, No. 1, 1981.
  • [Zuquete] Zuquete, A., "Improving the functionality of SYN cookies", 6th IFIP Communications and Multimedia Security Conference (CMS 2002), 2002.
  • [v6-ON] Roy, S., Durand, A., and J. Paugh, "Issues with Dual Stack IPv6 on by Default", Work in Progress, July 2004.

Author's Address 🦋

Fernando Gont Universidad Tecnologica Nacional / Facultad Regional Haedo Evaristo Carriego 2644 Haedo, Provincia de Buenos Aires 1706 Argentina

Phone: +54 11 4650 8472 EMail: fernando@gont.com.ar URI: http://www.gont.com.ar

That's the whole thing, bestie! 🌈✨

📦 where this page comes from