RFC 6093

On the Implementation of the TCP Urgent Mechanism

πŸ“… published January 2011, from ietf

RFC 6093: On the Implementation of the TCP Urgent Mechanism πŸŽ€

Okay bestie, this one is by F. Gont (UTN/FRH) and A. Yourtchenko (Cisco) ✨. It's a Standards Track document, so it's an Internet Standards Track document πŸ’…. It was published in January 2011 πŸ¦‹ with ISSN 2070-1721, and it updates RFC 793, RFC 1011, and RFC 1122 🌸.

Abstract

Okay so this doc is literally about how real TCP implementations handle urgent indications πŸ”₯. It also looks at how some widely deployed middleboxes change what end systems actually see. The authors update the specs so they match what's really happening out there πŸ’–, raise awareness that urgent indications aren't super reliable on the Internet πŸ™ˆ, and recommend against using them (but there's advice for apps that still do) 🫢.

Status of This Memo

This is an Internet Standards Track document. ✨

This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community, got public review, and was approved for publication by the Internet Engineering Steering Group (IESG). If you want more on Internet Standards, check out Section 2 of RFC 5741 πŸ“š.

For the current status, any errata, and how to send feedback, go to http://www.rfc-editor.org/info/rfc6093 🧚.

Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved. πŸ’œ

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication. Please read these carefully, bestie, because they explain your rights and restrictions for this document πŸ“œ. Code Components pulled out of this document must include Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions. They come without warranty, as described in the Simplified BSD License 🌈.

Table of Contents

1. Introduction ....................................................3
2. Specification of the TCP Urgent Mechanism .......................3
   2.1. Semantics of Urgent Indications ............................3
   2.2. Semantics of the Urgent Pointer ............................4
   2.3. Allowed Length of "Urgent Data" ............................4
3. Current Implementation Practice of the TCP Urgent Mechanism .....5
   3.1. Semantics of Urgent Indications ............................5
   3.2. Semantics of the Urgent Pointer ............................5
   3.3. Allowed Length of "Urgent Data" ............................6
   3.4. Interaction of Middleboxes with TCP Urgent Indications .....6
4. Updating RFC 793, RFC 1011, and RFC 1122 ........................6
5. Advice to New Applications Employing TCP ........................7
6. Advice to Applications That Make Use of the Urgent Mechanism ....7
7. Security Considerations .........................................7
8. Acknowledgements ................................................8
9. References ......................................................8
   9.1. Normative References .......................................8
   9.2. Informative References .....................................8
   Appendix A.  Survey of the Processing of TCP Urgent
                Indications by Some Popular TCP Implementations ......10
   A.1. FreeBSD ...................................................10
   A.2. Linux .....................................................10
   A.3. NetBSD ....................................................10
   A.4. OpenBSD ...................................................11
   A.5. Cisco IOS software ........................................11
   A.6. Microsoft Windows 2000, Service Pack 4 ....................11
   A.7. Microsoft Windows 2008 ....................................11
   A.8. Microsoft Windows 95 ......................................11

1. Introduction ✨

Let's get into it. This doc looks at how some current TCP implementations process urgent indications, and how some widely deployed middleboxes affect what hosts end up processing πŸ‘€. It updates RFC 793 [RFC0793], RFC 1011 [RFC1011], and RFC 1122 [RFC1122] so they match what current implementations actually do with urgent indications πŸ’…. It also gives advice to applications that use the urgent mechanism, and raises awareness about how reliable urgent indications are on today's Internet πŸ™ˆ.

Given all that, plus the potential interoperability issues with the super common default mode, it is strongly recommended that applications do not use urgent indications 😭. Urgent indications are still kept as a mandatory part of TCP anyway, to support the few legacy apps that rely on them πŸ•°οΈ. Even those apps are expected to have trouble in environments with middleboxes πŸ’€.

Section 2 covers what the current IETF specs say about urgent indications. Section 3 covers what TCP implementations actually do with them πŸ”. Section 4 updates RFC 793 [RFC0793], RFC 1011 [RFC1011], and RFC 1122 [RFC1122] so they match current practice. Section 5 gives advice to new applications using TCP when it comes to the urgent mechanism 🌱. Section 6 gives advice to existing applications that use or rely on it 🫢.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119] πŸ“–.

2. Specification of the TCP Urgent Mechanism 🌸

2.1. Semantics of Urgent Indications

Okay so TCP has an "urgent mechanism" 🚨. It lets the sending user stimulate the receiving user to accept some "urgent data", and it lets the receiving TCP tell the receiving user when all the currently known "urgent data" has been read ✨.

The urgent mechanism lets a point in the data stream be marked as the end of urgent information πŸ“. Whenever that point is ahead of RCV.NXT at the receiving TCP, that TCP has to tell the user to go into "urgent mode". When the receive sequence number catches up to the urgent pointer, TCP has to tell the user to go back into "normal mode" [RFC0793]. This means, for example, that data received as "normal data" can turn into "urgent data" if an urgent indication shows up in a later TCP segment, before the TCP user consumes that data πŸŒ€.

The URG control flag says the "Urgent Pointer" field is meaningful, and it has to be added to the segment sequence number to get the urgent pointer. If the flag is absent, there's no "urgent data" outstanding [RFC0793] 🏳️.

The TCP urgent mechanism is not a way to send "out-of-band" data. The so-called "urgent data" should be delivered "in-line" to the TCP user 🧡.

2.2. Semantics of the Urgent Pointer

There's some confusion in RFC 793 [RFC0793] about what the Urgent Pointer means πŸ˜΅β€πŸ’«. Section 3.1 (page 17) of RFC 793 [RFC0793] says the Urgent Pointer "communicates the current value of the urgent pointer as a positive offset from the sequence number in this segment. The urgent pointer points to the sequence number of the octet following the urgent data. This field is only be interpreted in segments with the URG control bit set" (sic). But Section 3.9 (page 56) of RFC 793 [RFC0793], when it describes processing the SEND call in the ESTABLISHED and CLOSE-WAIT states, says "If the urgent flag is set, then SND.UP <- SND.NXT-1 and set the urgent pointer in the outgoing segments". Two different vibes, bestie πŸ™ƒ.

RFC 1011 [RFC1011] cleared this up, stating that "Page 17 is wrong. The urgent pointer points to the last octet of urgent data (not to the first octet of non-urgent data)". Then RFC 1122 [RFC1122] formally updated RFC 793 by stating, in Section 4.2.2.4 (page 84), that "the urgent pointer points to the sequence number of the LAST octet (not LAST+1) in a sequence of urgent data" 🎯.

2.3. Allowed Length of "Urgent Data"

RFC 793 [RFC0793] lets TCP peers send "urgent data" of any length, since the urgent mechanism just gives a pointer to an interesting point in the data stream πŸ“. Section 4.2.2.4 (page 84) of RFC 1122 [RFC1122] says it outright: "A TCP MUST support a sequence of urgent data of any length" πŸ’―.

3. Current Implementation Practice of the TCP Urgent Mechanism πŸ› οΈ

3.1. Semantics of Urgent Indications

As we covered in Section 2, the urgent mechanism just lets a point in the data stream be marked as the end of urgent information. It does not give you a way to send "out-of-band" data πŸ™…β€β™€οΈ.

Unfortunately, basically every TCP implementation handles urgent indications differently 😀. By default, the last byte of "urgent data" is delivered "out of band" to the application. That means it doesn't come through as part of the normal data stream [UNPv1]. For example, the "out-of-band" byte gets read when an application issues a recv(2) system call with the MSG_OOB flag set 🫠.

Most implementations have a socket option (SO_OOBINLINE) that lets an application override that (broken) default. With it, "urgent data" is delivered "in line" to the application, which gives you the semantics the IETF specs intended πŸŽ€.

3.2. Semantics of the Urgent Pointer

Every popular implementation the authors could test reads the TCP Urgent Pointer the way Section 3.1 of RFC 793 describes 😭. So even though RFC 1122 formally updated RFC 793 to clear up the ambiguity, that clarification never made it into actual implementations. Basically, virtually all implementations default to the Section 3.1 semantics of RFC 793 πŸ’€.

Some operating systems have a system-wide toggle to override this and read the Urgent Pointer the way RFC 1122 clarified. The catch is that this toggle has been found to be inconsistent πŸ™ˆ. For example, Linux has the sysctl tcp_stdurg (i.e., net.ipv4.tcp_stdurg). When it's set, it supposedly changes the system to read the TCP Urgent Pointer as RFC 1122 specifies. However, this sysctl only changes the semantics for incoming segments, not outgoing ones πŸ€¦β€β™€οΈ. So if the sysctl is set, an application might not be able to talk to itself when the TCP sender and receiver are both running on the same host πŸŒ€.

3.3. Allowed Length of "Urgent Data" ✨

okay so RFC 1122 (Section 4.2.2.4, page 84) says "A TCP MUST support a sequence of urgent data of any length" πŸ’… which sounds super generous, right? but here's the tea πŸ«–: in real life, every implementation that treats urgent stuff as "out-of-band" data πŸ™ˆ only keeps a buffer of one single byte for the "last byte of urgent data" 😭. so if more urgent indications show up before the app reads that pending out-of-band byte, boom, the old byte gets overwritten by the new one. rip πŸ’€

to stop "urgent data" from getting yeeted, some implementations queue every received "urgent byte" 🧚 so even if another urgent indication arrives before the app consumes the pending "urgent data", nothing has to get thrown away. iconic... except ngl some of these have been known to not enforce any limits on how much "urgent data" they queue πŸ™ƒ, which makes them open to trivial resource exhaustion attacks [CPNI-TCP]. not it 🚩

the vibe check is simple: those implementations are broken 🫠. the TCP urgent mechanism is not a mechanism for delivering "out-of-band" data. period πŸ’–


3.4. Interaction of Middleboxes with TCP Urgent Indications πŸ¦‹

so after the NIDS evasion techniques based on TCP urgent indications were published [phrack] πŸ•΅οΈβ€β™€οΈ, some middleboxes started clearing the urgent indications. they clear the URG flag and set the Urgent Pointer to zero πŸ™…β€β™€οΈ. this makes the "urgent data" become "in line" (meaning accessible by the read(2) call or the recv(2) call without the MSG_OOB flag) for those TCP implementations that treat the urgent mechanism as a facility for delivering "out-of-band" data (as described in Section 3.1) 😬. an example of such a middlebox is the Cisco PIX firewall [Cisco-PIX].

bestie, this should discourage applications from depending on urgent indications for their correct operation πŸŽ€, because urgent indications may not be reliable in the current Internet. main character energy means not relying on the thing that might get dropped 🌸


4. Updating RFC 793, RFC 1011, and RFC 1122 🧁

okay so, as long as both the TCP sender and the TCP receiver implement the same semantics for the Urgent Pointer, there is no functional difference between having the Urgent Pointer point to "the sequence number of the octet following the urgent data" and having it point to "the last octet of urgent data" πŸ€·β€β™€οΈ. and since all known implementations interpret the Urgent Pointer as pointing to "the sequence number of the octet following the urgent data" ✨, we're hereby updating RFC 793 [RFC0793], RFC 1011 [RFC1011], and RFC 1122 [RFC1122] so that "the urgent pointer points to the sequence number of the octet following the urgent data" (in segments with the URG control bit set) πŸ’…. this way it accommodates virtually all existing TCP implementations, which is so healthy for the ecosystem 🫢


5. Advice to New Applications Employing TCP 🌈

bestie, listen up πŸ‘‚. because of the issues we just covered in Section 3.2 and Section 3.4, new applications SHOULD NOT employ the TCP urgent mechanism 🚫. however, TCP implementations MUST still include support for the urgent mechanism so that existing applications can still use it 🫑. the old apps deserve their moment too πŸ₯Ή


6. Advice to Applications That Make Use of the Urgent Mechanism πŸ‘‘

even though applications SHOULD NOT employ the urgent mechanism, if an application still decides to use it, it MUST set the SO_OOBINLINE socket option. that way "urgent data" is delivered in line, as the IETF specifications intended πŸ“œβœ¨

also, applications that still decide to use the urgent mechanism need to be designed for correct operation even when the URG flag is cleared by middleboxes πŸ™ˆ. no cap, build for the chaos πŸ”₯


7. Security Considerations πŸ’€

⚠️ okay this section is SERIOUS, so read it with full focus 🫣

multiple factors can affect the data flow that actually gets delivered to an application when the TCP urgent mechanism is employed 😱. for example, there are two possible interpretations of the semantics of the Urgent Pointer in current implementations (depending on the value of the tcp_stdurg sysctl) πŸ”€. then there's the possibility of implementing the urgent mechanism as an "out-of-band" (OOB) facility (versus "in-band" as the IETF specifications intended) πŸŒ€. and middleboxes (such as packet scrubbers) or the end-systems themselves could cause the "urgent data" to be processed "in line" 🫠.

this might make it difficult for a Network Intrusion Detection System (NIDS) to track the application-layer data transferred to the destination system πŸ•΅οΈβ€β™€οΈ, and it could lead to false negatives or false positives in the NIDS [CPNI-TCP] [phrack]. yikes, the chaos is real 😭

probably the best way to avoid the security implications of TCP "urgent data" is to avoid having applications use the TCP urgent mechanism altogether πŸ™Œ. packet scrubbers could probably be configured to clear the URG bit and set the Urgent Pointer to zero, which would basically cause the "urgent data" to be put "in line" 🧹. however, this might cause interoperability problems or undesired behavior in applications that rely on the TCP urgent mechanism, such as Telnet [RFC0854] and FTP [RFC0959] πŸ₯Ί. so tread carefully, babe 🫢


8. Acknowledgements πŸ’œ

the authors of this document would like to thank (in alphabetical order) Jari Arkko, Ron Bonica, David Borman, Dave Cridland, Ralph Droms, Wesley Eddy, John Heffner, Alfred Hoenes, Alexey Melnikov, Keith Moore, Carlos Pignataro, Tim Polk, Anantha Ramaiah, Joe Touch, Michael Welzl, Dan Wing, and Alexander Zimmermann for giving valuable feedback on earlier versions of this document 🫢🫢

Fernando would like to thank David Borman and Joe Touch for a fruitful discussion about the TCP urgent mechanism at IETF 73 (Minneapolis) πŸ™οΈβœ¨

Fernando Gont's attendance at IETF meetings was supported by ISOC's "Fellowship to the IETF" program 🌟

finally, Fernando Gont wishes to express deep and heartfelt gratitude to Jorge Oscar Gont and Nelida Garcia for their precious motivation and guidance πŸ₯ΉπŸ’–


9. References πŸ“š

9.1. Normative References

[RFC0793]     Postel, J., "Transmission Control Protocol", STD 7, RFC
              793, September 1981.

[RFC1011]     Reynolds, J. and J. Postel, "Official Internet
              protocols", RFC 1011, May 1987.

[RFC1122]     Braden, R., Ed., "Requirements for Internet Hosts -
              Communication Layers", STD 3, RFC 1122, October 1989.

[RFC2119]     Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

9.2. Informative References

[CPNI-TCP]    Gont, F., "Security Assessment of the Transmission
              Control Protocol (TCP)", "http://www.cpni.gov.uk/
              Docs/tn-03-09-security-assessment-TCP.pdf", 2009.

[Cisco-PIX]   Cisco PIX, "http://www.cisco.com/en/US/docs/security/
              asa/asa70/command/reference/tz.html#wp1288756".

[FreeBSD]     The FreeBSD project, "http://www.freebsd.org".

[Linux]       The Linux Project, "http://www.kernel.org".

[NetBSD]      The NetBSD project, "http://www.netbsd.org".

[OpenBSD]     The OpenBSD project, "http://www.openbsd.org".

[RFC0854]     Postel, J. and J. Reynolds, "Telnet Protocol
              Specification", STD 8, RFC 854, May 1983.

[RFC0959]     Postel, J. and J. Reynolds, "File Transfer Protocol",
              STD 9, RFC 959, October 1985.

[UNPv1]       Stevens, W., "UNIX Network Programming, Volume 1.
              Networking APIs: Sockets and XTI", Prentice Hall PTR,
              1997.

[Windows2000] Microsoft Windows 2000, "http://technet.microsoft.com/
              en-us/library/bb726981(printer).aspx".

[Windows95]   Microsoft Windows 95, "ftp://ftp.demon.co.uk/pub/
              mirrors/win95netfaq/faq-c.html".

[phrack]      Ko, Y., Ko, S., and M. Ko, "NIDS Evasion Method named
              "SeolMa"", Phrack Magazine, Volume 0x0b, Issue 0x39,
              Phile #0x03 of 0x12 http://www.phrack.org/
              issues.html?issue=57&id=3#article, 2001.

A.1. FreeBSD 😈

FreeBSD 8.0 [FreeBSD] interprets the semantics of the urgent pointer as specified in Section 4 of this document. it does not provide any sysctl to override this behavior πŸ™…β€β™€οΈ.

FreeBSD provides the SO_OOBINLINE socket option that, when set, causes TCP "urgent data" to remain "in line". that means it will be accessible by the read(2) call or the recv(2) call without the MSG_OOB flag ✨.

FreeBSD supports only one byte of "urgent data". that means only the byte preceding the Urgent Pointer is considered "urgent data" 1️⃣.

A.2. Linux 🐧

Linux 2.6.15-53-386 [Linux] interprets the semantics of the urgent pointer as specified in Section 4 of this document. it provides the net.ipv4.tcp_stdurg sysctl to override this behavior, so it can interpret the Urgent Pointer as specified in RFC 1122 [RFC1122]. however, this sysctl only affects the processing of incoming segments (the Urgent Pointer in outgoing segments will still be set as specified in Section 4 of this document) πŸ€“.

Linux provides the SO_OOBINLINE socket option that, when set, causes TCP "urgent data" to remain "in line". that means it will be accessible by the read(2) call or the recv(2) call without the MSG_OOB flag πŸ’–.

Linux supports only one byte of "urgent data". that means only the byte preceding the Urgent Pointer is considered "urgent data" ☝️.

A.3. NetBSD 🐑

NetBSD 5.0.1 [NetBSD] interprets the semantics of the urgent pointer as specified in Section 4 of this document. it does not provide any sysctl to override this behavior πŸ™…β€β™€οΈ.

NetBSD provides the SO_OOBINLINE socket option that, when set, causes TCP "urgent data" to remain "in line". that means it will be accessible by the read(2) call or the recv(2) call without the MSG_OOB flag 🌊.

NetBSD supports only one byte of "urgent data". that means only the byte preceding the Urgent Pointer is considered "urgent data" 1️⃣.

A.4. OpenBSD 🐑

OpenBSD 4.2 [OpenBSD] interprets the semantics of the urgent pointer as specified in Section 4 of this document. it does not provide any sysctl to override this behavior πŸ™…β€β™€οΈ.

OpenBSD provides the SO_OOBINLINE socket option that, when set, causes TCP "urgent data" to remain "in line". that means it will be accessible by the read(2) or recv(2) calls without the MSG_OOB flag 🫧.

OpenBSD supports only one byte of "urgent data". that means only the byte preceding the Urgent Pointer is considered "urgent data" 1️⃣.

A.5. Cisco IOS software 🌐

Cisco IOS Software Releases 12.2(18)SXF7 and 12.4(15)T7 interpret the semantics of the urgent pointer as specified in Section 4 of this document 🧠.

the behavior is consistent with having the SO_OOBINLINE socket option turned on, i.e., the data is processed "in line" 🫢.

A.6. Microsoft Windows 2000, Service Pack 4 πŸͺŸ

Microsoft Windows 2000 [Windows2000] interprets the semantics of the urgent pointer as specified in Section 4 of this document. it provides the TcpUseRFC1122UrgentPointer system-wide variable to override this behavior, so it can interpret the Urgent Pointer as specified in RFC 1122 [RFC1122] πŸŽ›οΈ.

tests performed with a sample server application compiled using the cygwin environment showed that the default behavior is to return the "urgent data" "in line" πŸ§ͺ✨.

A.7. Microsoft Windows 2008 πŸͺŸ

Microsoft Windows 2008 interprets the semantics of the urgent pointer as specified in Section 4 of this document πŸ’….

A.8. Microsoft Windows 95 πŸͺŸ

okay so bestie, Windows 95 reads the urgent pointer the way Section 4 of this document describes ✨ it's giving spec-compliant energy. BUT there's a little escape hatch πŸŽ€ a system-wide variable called BSDUrgent that overrides that behavior, and with it on, the Urgent Pointer gets interpreted the way RFC 1122 [RFC1122] describes πŸ’…

and the tea β˜•: Windows 95 only supports one byte of "urgent data" 😭 that's the byte right before the Urgent Pointer, and that's it. not me being like "wait, really?" πŸ™ˆ [Windows95]


Authors' Addresses πŸ’Œ

okay now let's meet the brains behind this 🧠✨ first up, Fernando Gont 🌸

Fernando Gont
Universidad Tecnologica Nacional / Facultad Regional Haedo
Evaristo Carriego 2644
Haedo, Provincia de Buenos Aires  1706
Argentina

Phone: +54 11 4650 8472
EMail: fernando@gont.com.ar
URI:   http://www.gont.com.ar

and the other half of the duo, Andrew Yourtchenko πŸ¦‹

Andrew Yourtchenko
Cisco
De Kleetlaan, 7
Diegem  B-1831
Belgium

Phone: +32 2 704 5494
EMail: ayourtch@cisco.com

that's the whole thing, bestie 🫢🌈

πŸ“¦ where this page comes from