RFC 4413
TCP/IP Field Behavior
TCP/IP Field Behavior 🎀
Okay bestie, here's the title block tea 🫖✨
- RFC 4413, filed under the Network Working Group 🌸
- Authors: M. West and S. McCann, both from Siemens/Roke Manor Research 💜
- Category: Informational 📚
- Date: March 2006 🗓️
- The title block doesn't list anything this RFC obsoletes or updates 🦋
Status of This Memo 💅
This memo is here to give the Internet community some info 📖 It does not specify an Internet standard of any kind, so it's not a standard, not even a little. Distribution of this memo is unlimited 🌈
Copyright Notice 🔥
Copyright (C) The Internet Society (2006). 💖
Abstract 🦋
This memo is all about TCP/IP field behavior in the context of header compression 🧚 Header compression works because most header fields don't change randomly from packet to packet. Lots of them stay static, or change in a pretty predictable way 📈 When you design a header compression scheme, understanding how each field behaves in detail is super important 🔍 RFC 3095 has an example of this kind of analysis, and this memo does a similar job for compressing TCP/IP headers 💜
Table of Contents 🌸
1. Introduction ....................................................3
2. General classification ..........................................4
2.1. IP Header Fields ...........................................5
2.1.1. IPv6 Header Fields ....................................5
2.1.2. IPv4 Header Fields ....................................7
2.2. TCP Header Fields .........................................10
2.3. Summary for IP/TCP ........................................11
3. Classification of Replicable Header Fields .....................11
3.1. IPv4 Header (Inner and/or Outer) ..........................12
3.2. IPv6 Header (inner and/or outer) ..........................14
3.3. TCP Header ................................................14
3.4. TCP Options ...............................................15
3.5. Summary of Replication ....................................16
4. Analysis of Change Patterns of Header Fields ...................16
4.1. IP Header .................................................19
4.1.1. IP Traffic-Class / Type-Of-Service (TOS) .............19
4.1.2. ECN Flags ............................................19
4.1.3. IP Identification ....................................20
4.1.4. Don't Fragment (DF) flag .............................22
4.1.5. IP Hop-Limit / Time-To-Live (TTL) ....................22
4.2. TCP Header ................................................23
4.2.1. Sequence Number ......................................23
4.2.2. Acknowledgement Number ...............................24
4.2.3. Reserved .............................................25
4.2.4. Flags ................................................25
4.2.5. Checksum .............................................26
4.2.6. Window ...............................................26
4.2.7. Urgent Pointer .......................................27
4.3. Options ...................................................27
4.3.1. Options Overview .....................................28
4.3.2. Option Field Behavior ................................29
5. Other Observations .............................................36
5.1. Implicit Acknowledgements .................................36
5.2. Shared Data ...............................................36
5.3. TCP Header Overhead .......................................37
5.4. Field Independence and Packet Behavior ....................37
5.5. Short-Lived Flows .........................................37
5.6. Master Sequence Number ....................................38
5.7. Size Constraint for TCP Options ...........................38
6. Security Considerations ........................................39
7. Acknowledgements ...............................................39
8. References .....................................................40
8.1. Normative References ......................................40
8.2. Informative References ....................................411. Introduction ✨
Okay so this document describes the format of the TCP/IP header and the header field behavior, meaning how fields vary within a TCP flow 🦋 It's all presented in the context of header compression 🗜️
The IP header behaves a little differently from what RFC 3095 [31] described for UDP and RTP, so it's included here too 💅
This document borrows a lot of the classification text from RFC 3095 [31], instead of stuffing in a bunch of references to that document 🫶
Following the format from RFC 3095 [31], TCP/IP header fields get classified and analyzed in two steps 🔍 First up is the general classification in Section 2, where fields are sorted based on stable knowledge and assumptions. This general classification doesn't account for how changing fields behave, since that depends on the implementation and the application being used 🙈 Section 3 looks at how field values can be used to optimize short-lived flows 🏃♀️ Then Section 4 does a more detailed analysis of change characteristics. Finally, Section 5 wraps up with conclusions about how the various header fields should be handled by the header compression scheme to optimize compression 🎀
A big question this analysis raises is: what 'baseline' definition of all possible TCP/IP implementations should we use? 🤔 This review is based on analyzing TCP implementations that are actually deployed today and support mechanisms standardized by the IETF 💜
The general requirement for transparency is also a big deal 👀 A bunch of recent proposals for TCP extensions use some of the previously 'reserved' bits in the TCP packet header 😭 So a 'reserved' bit can't be assumed to always be zero. It may change 🌈 Ideally, the compression profile should accommodate that 🫶
Some reserved bits are available for future expansion 🚀 Nobody can predict how those bits will be used, but the expectation is that they will get used at some point 🔮 Given that, a compression scheme can optimize for today's situation, but it should be able to support any arbitrary usage of the reserved bits 💖 That said, you can't optimize for usage patterns that haven't been defined yet 🙃
2. General classification 💅
The following definitions (and some text) are copied from RFC 3095 [31], Appendix A. Differences in IP field behavior between RFC 3095 [31] (that is, IP/UDP/RTP behavior for audio and video applications) and this document have been identified 🎧📹
Here's the deal with "session": we define it as a TCP packet stream, which is a series of packets with the same IP addresses and port numbers 🧶 A packet flow is defined by certain fields (see STATIC-DEF, below) and can be thought of as a subset of a session 🌊 See [31] for a fuller discussion of splitting sessions into streams of packets for header compression 📚
At a general level, the header fields are separated into 5 classes 👑
INFERRED: These fields hold values that can be inferred from other values (for example, the size of the frame carrying the packet) 🧮 So the compression scheme doesn't have to handle them at all ✨STATIC: These fields are expected to stay constant for the lifetime of the packet stream 🧊 Static information has to be communicated once, somehow 📨STATIC-DEF: These are STATIC fields whose values define a packet stream 🧬 They're generally handled as STATIC 🫶STATIC-KNOWN: These STATIC fields are expected to have well-known values, so they don't need to be communicated at all 🥳CHANGING: These fields are expected to vary randomly within a limited value set or range, or in some other way 🌀
In this section, each IP and TCP header field gets assigned to one of these classes 🗂️ For every field except the ones classified as CHANGING, the reasons for the classification are also explained 💭 In Section 4, CHANGING fields get a closer look and are classified based on their expected change behavior 🔎
2.1. IP Header Fields 🌸
2.1.1. IPv6 Header Fields ✨
Okay bestie, here's the IPv6 header breakdown 🎀. Every field gets a class, and the class tells the compressor how much it can skip sending 💅.
+---------------------+-------------+----------------+
| Field | Size (bits) | Class |
+---------------------+-------------+----------------+
| Version | 4 | STATIC |
| DSCP* | 6 | ALTERNATING |
| ECT flag* | 1 | CHANGING |
| CE flag* | 1 | CHANGING |
| Flow Label | 20 | STATIC-DEF |
| Payload Length | 16 | INFERRED |
| Next Header | 8 | STATIC |
| Hop Limit | 8 | CHANGING |
| Source Address | 128 | STATIC-DEF |
| Destination Address | 128 | STATIC-DEF |
+---------------------+-------------+----------------+
* Differs from RFC 3095 [31]. (The DSCP, ECT,
and CE flags were amalgamated into the Traffic
Class octet in RFC 3095).Figure 1. IPv6 Header Fields
Version 🌸 This field tells you which IP version is in use. Packets with different values here have to go to different IP stacks, so every packet in one stream has to be the same IP version. That's why it's STATIC.
Flow Label 🦋 This one can be used to spot packets that belong to a particular packet stream. If nobody's using it, its value should be zero. If it is being used, every packet in the same stream has to carry the same value, because it's one of the things that defines the stream. So it's STATIC-DEF.
Payload Length 🧚 The packet length (and so the payload length) is expected to come from the link layer. That makes it INFERRED, so nothing needs to be sent for it.
Next Header 💜 This one usually stays the same across all packets in a stream, since it says what header comes next. It only changes when extension headers are sometimes missing. That's the reason it's STATIC. The class is inherited from RFC 3095 [31]. Honestly though, the next header is really decided by whatever header follows, so it might be more accurate to call it an inference. It depends on how the compression scheme is implemented 🤔.
Source and Destination Addresses 🍓 These are part of defining a stream, so they have to stay constant for every packet in it. Both are STATIC-DEF. Ngl, this is a bit of a simplified view. In this document the IP addresses are tied to the transport layer connection and treated as part of what defines a flow. More complicated flow separation could definitely be considered too (see RFC 3095 [31] for more on this). When tunneling is happening, the addresses in the outer tunnel headers are also assumed to be STATIC-DEF.
The total size of the fields in each class looks like this 💖:
+--------------+--------------+
| Class | Size (octets)|
+--------------+--------------+
| INFERRED | 2 |
| STATIC | 1.5 |
| STATIC-DEF | 34.5 |
| STATIC-KNOWN | 0 |
| CHANGING | 2 |
+--------------+--------------+Figure 2: Field sizes
2.1.2. IPv4 Header Fields 🦋
IPv4 gets the same treatment, just with its own quirks 🌈. Check out this table:
+---------------------+-------------+----------------+
| Field | Size (bits) | Class |
+---------------------+-------------+----------------+
| Version | 4 | STATIC |
| Header Length | 4 | STATIC-KNOWN |
| DSCP* | 6 | ALTERNATING |
| ECT flag* | 1 | CHANGING |
| CE flag* | 1 | CHANGING |
| Packet Length | 16 | INFERRED |
| Identification | 16 | CHANGING |
| Reserved flag* | 1 | CHANGING |
| Don't Fragment flag*| 1 | CHANGING |
| More Fragments flag | 1 | STATIC-KNOWN |
| Fragment Offset | 13 | STATIC-KNOWN |
| Time To Live | 8 | CHANGING |
| Protocol | 8 | STATIC |
| Header Checksum | 16 | INFERRED |
| Source Address | 32 | STATIC-DEF |
| Destination Address | 32 | STATIC-DEF |
+---------------------+-------------+----------------+
* Differs from RFC 3095 [31]. (The DSCP, ECT
and CE flags were amalgamated into the TOS
octet in RFC 3095; the DF flag behavior is
considered later; the reserved field is
discussed below).Figure 3. IPv4 Header Fields
Version 🌸 Same deal as before. It says which IP version this is. Packets with different values have to be handled by different IP stacks, so all packets in a stream must share the same version. That's why it's STATIC.
Header Length 🍓 As long as there are no options in the IP header, the header length is constant and well known. If options were present, the field would be STATIC, but here it's assumed there are no options. So it's STATIC-KNOWN.
Packet Length 🧚 The link layer is expected to provide the packet length, so this one is INFERRED.
Flags 🔥 Okay, flags are where it gets spicy 🌶️.
The Reserved flag must be set to zero, as defined in RFC 791 [1]. RFC 3095 [31] classifies it as STATIC-KNOWN. But reserved fields might get used someday, and it would be a bad idea to pick an encoding that blocks a compression profile from handling that future change. So CHANGING is used instead. (A compression profile can still optimise for today's situation, where the value is known to be 0 💅.)
The More Fragments (MF) flag is expected to be zero, since fragmentation is ideally not expected. Still, some scenarios (like some tunneling architectures) do cause fragmentation. In general though, fragmentation isn't expected to be common on the Internet, thanks to a combination of initial MSS negotiation and later path-MTU discovery. RFC 3095 [31] points out that for RTP, only the first fragment contains the transport layer protocol header, and later fragments would need a different profile to compress. The same is obviously true for TCP. If fragmentation did happen, the first fragment would by definition be relatively large, so the header overhead would be minimized. The later fragments would be compressed with another profile. So it's considered undesirable to optimise for fragmentation when doing header compression. The More Fragments flag is therefore classified as STATIC-KNOWN.
Fragment Offset 💜 If no fragmentation occurs, the fragment offset is always zero, so the field is STATIC-KNOWN. Even if fragmentation were considered further, only the first fragment would contain the TCP header, and that packet's fragment offset would still be zero.
Protocol 💖 This usually has the same value in all packets of a stream, since it encodes the type of the subsequent header.
It only changes when the sequence of headers changes (like when an extension header gets inserted or deleted, or a tunnel header is added or removed). That's why it's STATIC. Whether a change like that should make the packets count as a new flow for header compression is up to profile design. ROHC profiles must be able to cope with extension headers and tunneling, but picking a strategy is outside the scope of this document 🧠.
Header Checksum 🌈 The header checksum exists to protect individual hops from processing a corrupted header. Once almost all of the IP header info has been compressed away, keeping this checksum doesn't make much sense. It can be regenerated on the decompressor side instead, so it's INFERRED.
Here's the tea ☕. The TCP checksum doesn't protect the whole TCP/IP header. It only covers the TCP pseudo-header and the payload. ROHC [31] takes a different approach and uses a CRC to verify the uncompressed header. Since the complete TCP/IP header needs validating, computing the TCP checksum over the whole payload costs a lot, and the TCP checksum has known weaknesses [37], an additional check is necessary. So it's highly desirable that some additional checksum (such as a CRC) is used to validate correct decompression 🫶.
Source and Destination Addresses 👑 These are part of defining a stream, so they must stay constant across every packet in it. Both are STATIC-DEF.
The total size of the fields in each class is as follows 🎀:
+--------------+--------------+
| Class | Size (octets)|
+--------------+--------------+
| INFERRED | 4 |
| STATIC* | 1.5 |
| STATIC-DEF | 8 |
| STATIC-KNOWN*| 2.25 |
| CHANGING* | 4.25 |
+--------------+--------------+
* Differs from RFC 3095 [31]Figure 4. Field sizes
2.2. TCP Header Fields ✨
okay so here's the TCP header breakdown 🎀 every field gets a class, and the class tells you how it behaves across packets in a flow 💅
+---------------------+-------------+----------------+
| Field | Size (bits) | Class |
+---------------------+-------------+----------------+
| Source Port | 16 | STATIC-DEF |
| Destination Port | 16 | STATIC-DEF |
| Sequence Number | 32 | CHANGING |
| Acknowledgement Num | 32 | CHANGING |
| Data Offset | 4 | INFERRED |
| Reserved | 4 | CHANGING |
| CWR flag | 1 | CHANGING |
| ECE flag | 1 | CHANGING |
| URG flag | 1 | CHANGING |
| ACK flag | 1 | CHANGING |
| PSH flag | 1 | CHANGING |
| RST flag | 1 | CHANGING |
| SYN flag | 1 | CHANGING |
| FIN flag | 1 | CHANGING |
| Window | 16 | CHANGING |
| Checksum | 16 | CHANGING |
| Urgent Pointer | 16 | CHANGING |
| Options | 0(-352) | CHANGING |
+---------------------+-------------+----------------+
Figure 5: TCP header fields- Source and Destination ports 🌸
these two are part of how a stream gets defined, so they must stay constant for every packet in that stream 🔒 that's why they're STATIC-DEF (defined-static, bestie, iykyk) ✨
- Data Offset 🦋
this is how many 4 octet words are in the TCP header, which tells you where the data starts 💖 it's always a multiple of 4 octets, and you can rebuild it from the length of the options, so it doesn't need to be sent explicitly 🙈 that's why it's INFERRED 💅
2.3. Summary for IP/TCP 🌈
okay, let's put IP and TCP together and see the whole picture 👑
+----------------+----------------+----------------+
| Class \ IP ver | IPv6 (octets) | IPv4 (octets) |
+----------------+----------------+----------------+
| INFERRED | 2 + 4 bits | 4 + 4 bits |
| STATIC | 1 + 4 bits | 1 + 4 bits |
| STATIC-DEF | 38 + 4 bits | 12 |
| STATIC-KNOWN | - | 2 + 2 bits |
| CHANGING | 17 + 4 bits | 19 + 6 bits |
+----------------+----------------+----------------+
| Totals | 60 | 40 |
+----------------+----------------+----------------+
(Excludes options, which are all classified as CHANGING).
Figure 6. Overall field sizes(Excludes options, which are all classified as CHANGING, so yes, options are the messy one 🙃)
3. Classification of Replicable Header Fields 🧚
okay this section is the good stuff ✨ when multiple flows overlap in time or happen one after another really close together, their header values are super similar 🫶 that shared sameness gets stored in the compression context, so it should be possible to use what's common between flows to get a better compression ratio 📈 to do that, you have to understand which header fields are 'replicable' 💜
the big idea is 'replication' 🔁 you take an existing context, use it as a baseline, and copy it to start a brand new context 🍓 fields that match get initialised automatically 🎉 fields that changed get updated or overwritten using values from the initialisation packet that kicked off the replication 💅 this section looks at how fields compare across different flows 👀
heads up, though: replication is based on whole contexts, not just field values, so stuff the compressor creates for the context can get copied too 🤓 which one applies depends on the compression protocol (ROHC profile) you're using 🎀
here's a quick look at how the TCP/IP fields behave across 'replicable' packet streams 🦋
'N/A': the field doesn't need to be considered in replication, because it's inferred or known 'a priori' (so it doesn't show up in the context) 🙈'No': the field can't be replicated because its change pattern between two flows isn't correlated 💀'Yes': the field may be replicated 🫶 this doesn't guarantee the value will match across two candidate streams, only that replication might help improve the compression ratio ✨ specific encoding methods can make the compression more efficient 🔥
3.1. IPv4 Header (Inner and/or Outer) 💖
+-----------------------+---------------+------------+
| Field | Class | Replicable |
+-----------------------+---------------+------------+
| Version | STATIC | N/A |
| Header Length | STATIC-KNOWN | N/A |
| DSCP | ALTERNATING | No (1) |
| ECT flag | CHANGING | No (2) |
| CE flag | CHANGING | No (2) |
| Packet Length | INFERRED | N/A |
| Identification | CHANGING | Yes (3) |
| Reserved flag | CHANGING | No (4) |
| Don't Fragment flag | CHANGING | Yes (5) |
| More Fragments flag | STATIC-KNOWN | N/A |
| Fragment Offset | STATIC-KNOWN | N/A |
| Time To Live | CHANGING | Yes |
| Protocol | STATIC | N/A |
| Header Checksum | INFERRED | N/A |
| Source Address | STATIC-DEF | Yes |
| Destination Address | STATIC-DEF | Yes |
+-----------------------+---------------+------------+
Figure 7: IPv4 header(1) the DSCP gets marked based on what the application needs 💅 if you can assume replicable connections are in the same diffserv class, the DSCP will probably be replicable too 🌈 the DSCP can be set by the sender or by any packet marker, so one flow can have a bunch of DSCP values at different points in the network 🙃 header compression works on a point-to-point link though, so it expects a pretty stable value 🧚 if re-marking happens based on a meter's state, the value can change mid-flow ⚡ overall, we still expect supporting DSCP replication to be useful for header compression 💜
(2) the ECN bits can't be replicated (though the ECN nonce scheme [19] is expected to be used) 🔐 it seems likely that all TCP flows between ECN-capable hosts will use ECN, so whether or not ECN gets used between the same endpoints might be replicable 👀 see also note (4) ✨
(3) the replicable context for this field includes the IP-ID, NBO, and RND flags (as described in ROHC RTP) 🦋 this shows that replication works on the context, not just the header field values 🎀 so it has to be looked at based on the exact compression applied to each field 🫶
(4) since the Reserved Flag's future behaviour can't be predicted, it's not considered replicable 🙈 that said, the reserved flag's behaviour between the same endpoints will probably be similar 💖 so any choice of packet formats based on that behaviour might carry over to the new flow 🌸 for packet formats, this can probably be a compressor-local decision 🍓
(5) in theory the DF bit may be replicable 🤔 but that's not guaranteed, and in practice it's unlikely to be useful 💀 from a header compression view, figuring out whether a 1-bit flag should be replicated or specified explicitly probably takes more bits than just sending the flag's value 😭 we don't rule out DF replication though 🌈
3.2. IPv6 Header (inner and/or outer) 🧚♀️
+-----------------------+---------------+------------+
| Field | Class | Replicable |
+-----------------------+---------------+------------+
| Version | STATIC | N/A |
| Traffic Class | CHANGING | Yes (1) |
| ECT flag | CHANGING | No (2) |
| CE flag | CHANGING | No (2) |
| Flow Label | STATIC-DEF | N/A |
| Payload Length | INFERRED | N/A |
| Next Header | STATIC | N/A |
| Hop Limit | CHANGING | Yes |
| Source Address | STATIC-DEF | Yes |
| Destination Address | STATIC-DEF | Yes |
+-----------------------+---------------+------------+
(1) See comment about DSCP field for IPv4, above.
(2) See comment about ECT and CE flags for IPv4, above.
Figure 8. IPv6 Header(1) see the DSCP note for IPv4 above, it's the same vibe 💅 (2) see the ECT and CE flags note for IPv4 above 🫶
3.3. TCP Header 🦋
Okay bestie, the TCP header is where things get spicy 🌶️ Here's the breakdown:
+-----------------------+---------------+------------+
| Field | Class | Replicable |
+-----------------------+---------------+------------+
| Source Port | STATIC-DEF | Yes (1) |
| Destination Port | STATIC-DEF | Yes (1) |
| Sequence Number | CHANGING | No (2) |
| Acknowledgement Number| CHANGING | No |
| Data Offset | INFERRED | N/A |
| Reserved Bits | CHANGING | No (3) |
| Flags | | |
| CWR | CHANGING | No (4) |
| ECE | CHANGING | No (4) |
| URG | CHANGING | No |
| ACK | CHANGING | No |
| PSH | CHANGING | No |
| RST | CHANGING | No |
| SYN | CHANGING | No |
| FIN | CHANGING | No |
| Window | CHANGING | Yes |
| Checksum | CHANGING | No |
| Urgent Pointer | CHANGING | Yes (5) |
+-----------------------+---------------+------------+Figure 9: TCP Header 💅
(1) Okay so on the server side, the port number is likely to be a well-known value 🌸 On the client side, the stack usually picks the port number automatically. Whether it's replicable totally depends on how the stack chooses it. Most implementations use a simple scheme that sequentially picks the next available port number, but it may not be desirable to rely on this behavior 🙈
(2) With the recommendation (and expected deployment) of TCP Initial Sequence Number randomization, defined in RFC 1948 [10], it's going to be impossible to share the sequence number 💀 So this field won't be regarded as replicable.
(3) See comment (4) for the IPv4 header, above.
(4) See comment (2) on ECN flags for the IPv4 header, above.
(5) The urgent pointer is very rarely used 🧚 This means that, in practice, the field may be considered replicable.
3.4. TCP Options 🎀
Let's talk options, because these are giving main character energy ✨
+---------------------------+--------------+------------+
| Option | SYN-only (1) | Replicable |
+---------------------------+--------------+------------+
| End of Option List | No | No (2) |
| No-Operation | No | No (2) |
| Maximum Segment Size | Yes | Yes |
| Window Scale | Yes | Yes |
| SACK-Permitted | Yes | Yes |
| SACK | No | No |
| Timestamp | No | No |
+---------------------------+--------------+------------+Figure 10. TCP Options 🍓
(1) This tells you whether the option only shows up in SYN packets. Options that are not 'SYN-only' may appear in any packet. Lowkey, many TCP options are only used in SYN packets 👀 Some options, like MSS, Window Scale, and SACK-Permitted, tend to have the same value among replicable packet streams, which is so convenient 🫶
Thus, to support context sharing, the compressor should maintain such TCP options in the context (even though they only appear in the SYN segment).
(2) Since these options have fixed values, they could be regarded as replicable. However, the only interesting thing to convey about these options is their presence 🦋 If it is known that such an option exists, its value is defined.
3.5. Summary of Replication 🌈
So what's the tea? From everything above, there are reasonable grounds for exploiting redundancy between flows as well as between packets within a flow 💜 Just think about how amazing it would be to elide the source and destination addresses for a repeated connection between two IPv6 endpoints 😭 There's also a cost (in terms of complexity and robustness) for replicating contexts, and this must be considered when one decides what constitutes an appropriate solution. No cap, it's a trade-off.
Finally, note that the use of replication requires that the compressor have a suitable degree of confidence that the source data is present and correct at the decompressor 🔥 This may place some restrictions on which of the 'changing' fields, in particular, can be utilised during replication.
4. Analysis of Change Patterns of Header Fields ✨
To design suitable mechanisms for efficient compression of all header fields, their change patterns must be analyzed 💅 For this reason, an extended classification is done based on the general classification in 2, considering the fields that were labeled CHANGING in that classification.
The CHANGING fields are separated into five different subclasses:
- STATIC: These are fields that were classified as CHANGING on a general basis, but that are classified as STATIC here due to certain additional assumptions.
- SEMISTATIC: These fields are STATIC most of the time. However, occasionally the value changes but reverts to its original value after a known number of packets 🔁
- RARELY-CHANGING (RC): These are fields that change their values occasionally and then keep their new values 🌸
- ALTERNATING: These fields alternate between a small number of different values 🎀
- IRREGULAR: These, finally, are the fields for which no useful change pattern can be identified 💀
To further expand the classification possibilities without increasing complexity, the classification can be done either according to the values of the field and/or according to the values of the deltas for the field 🧚
When the classification is done, other details are also stated regarding possible additional knowledge about the field values and/or field deltas, according to the classification. For fields classified as STATIC or SEMISTATIC, the value of the field could be not only STATIC but also well-KNOWN a priori (two states for SEMISTATIC fields). For fields with non-irregular change behavior, it could be known that changes are usually within a LIMITED range compared to the maximal change for the field. For other fields, the values are completely UNKNOWN 🙈
Figure 11 classifies all the CHANGING fields on the basis of their expected change patterns. (4) refers to IPv4 fields and (6) refers to IPv6 🦋
+------------------------+-------------+-------------+-------------+
| Field | Value/Delta | Class | Knowledge |
+========================+=============+=============+=============+
| DSCP(4) / Tr.Class(6) | Value | ALTERNATING | UNKNOWN |
+------------------------+-------------+-------------+-------------+
| IP ECT flag(4) | Value | RC | UNKNOWN |
+------------------------+-------------+-------------+-------------+
| IP CE flag(4) | Value | RC | UNKNOWN |
+------------------------+-------------+-------------+-------------+
| Sequential | Delta | STATIC | KNOWN |
| -----------+-------------+-------------+-------------+
| IP Id(4) Seq. jump | Delta | RC | LIMITED |
| -----------+-------------+-------------+-------------+
| Random | Value | IRREGULAR | UNKNOWN |
+------------------------+-------------+-------------+-------------+
| IP DF flag(4) | Value | RC | UNKNOWN |
+------------------------+-------------+-------------+-------------+
| IP TTL(4) / Hop Lim(6) | Value | ALTERNATING | LIMITED |
+------------------------+-------------+-------------+-------------+
| TCP Sequence Number | Delta | IRREGULAR | LIMITED |
+------------------------+-------------+-------------+-------------+
| TCP Acknowledgement Num| Delta | IRREGULAR | LIMITED |
+------------------------+-------------+-------------+-------------+
| TCP Reserved | Value | RC | UNKNOWN |
+------------------------+-------------+-------------+-------------+
| TCP flags | | | |
| ECN flags | Value | IRREGULAR | UNKNOWN |
| CWR flag | Value | IRREGULAR | UNKNOWN |
| ECE flag | Value | IRREGULAR | UNKNOWN |
| URG flag | Value | IRREGULAR | UNKNOWN |
| ACK flag | Value | SEMISTATIC | KNOWN |
| PSH flag | Value | IRREGULAR | UNKNOWN |
| RST flag | Value | IRREGULAR | UNKNOWN |
| SYN flag | Value | SEMISTATIC | KNOWN |
| FIN flag | Value | SEMISTATIC | KNOWN |
+------------------------+-------------+-------------+-------------+
| TCP Window | Value | ALTERNATING | KNOWN |
+------------------------+-------------+-------------+-------------+
| TCP Checksum | Value | IRREGULAR | UNKNOWN |
+------------------------+-------------+-------------+-------------+
| TCP Urgent Pointer | Value | IRREGULAR | KNOWN |
+------------------------+-------------+-------------+-------------+
| TCP Options | Value | IRREGULAR | UNKNOWN |
+------------------------+-------------+-------------+-------------+Figure 11: Classification of CHANGING Fields 👑
The following subsections discuss the various header fields in detail. Note that Table 1 and the discussion below do not consider changes caused by loss or reordering before the compression point 🫶
4.1. IP Header 🌸
4.1.1. IP Traffic-Class / Type-Of-Service (TOS) 💖
Okay so, the Traffic-Class (IPv6) or Type-Of-Service/DSCP (IPv4) field might totally change while a packet stream is running, bestie. 🌸 This section looks at a bunch of RFCs that tweaked the original RFC 791 [1] 🧚
The TOS byte started out in RFC 791 [1] as 3 bits of precedence, then 3 bits of TOS, then 2 reserved bits that were defined to be zero. 🍓 RFC 1122 [21] stretched it to 5 bits of TOS, but nobody defined what the extra 2 bits were supposed to mean 🤷♀️ Then RFC 1349 [23] made the 4th bit of TOS mean 'minimize monetary cost' 💸 The big glow-up came with RFC 2474 [14], which obsoleted RFC 1349 [23]. It reworked the octet into 6 bits of DSCP (DiffServ Code Point) plus 2 unused bits ✨ And most recently, RFC 2780 [30] said the 2 reserved bits in the TOS or traffic class octet can be used for experimental ECN stuff 🔥
So the proposal is to treat the TOS (or traffic class) octet as 6 bits of DSCP and 2 extra bits. Those 2 bits are expected to be zero or to carry ECN data. For future-proofing, it's way better to assume ECN is in use, especially for TCP 💅
Legacy stacks are still gonna be around for a long time, so the profile has to work with them too 🫶 To keep things simple, we treat it as 6 bits of TOS info and 2 bits of ECN data, so the fields always look structured the same way 🎀
The DSCP (same as TOS in ROHC RTP) isn't expected to change often, though it could change mid-flow, like if there's a route change 🦋
4.1.2. ECN Flags 🌈
First up, we describe the ECN flags the way RFC 2481 [15] and RFC 3168 [18] do. After that there's a suggested update that would change how the flags behave 👑
In RFC 2481 [15] there are two separate flags: the ECT (ECN Capable Transport) flag and the CE (Congestion Experienced) flag. If the TCP stack negotiates ECT, it'll be '1' on all data packets and '0' on all 'pure acknowledgement' packets. So the ECT flag is tied to what the TCP stack is doing 🧠 Whether that can be exploited for compression? Honestly, not clear yet 🤔
The CE flag only matters if ECT is '1'. The sender sets it to '0', and an ECN-capable router in the network can flip it to '1' to say "hey, congestion here" 🚦 So CE gets set randomly, with a probability that depends on how congested the network is and where the compressor sits on the path. A compressor near the receiver in a congested network will see CE set a lot, but one near the sender will rarely, if ever, see CE set to '1' 😭
A recent experimental proposal [19] has a different take on these 2 bits. It treats them together as 4 codepoints, like this:
00 Not ECN capable
01 ECN capable, no congestion (known as ECT(0))
10 ECN capable, no congestion (known as ECT(1))
11 Congestion experiencedUsing 2 codepoints for ECT lets the sender spot when a receiver isn't reliably echoing congestion info, which is lowkey genius 🧚
For compression, this update means ECT(0) and ECT(1) are equally likely (for an ECN-capable flow), and '11' will show up pretty rarely. We already talked about how likely a congestion indication is when we covered the CE flag 💜
It's suggested that, for compression purposes, ECN with nonces be assumed as the baseline. BUT the compression scheme still has to be able to compress the original ECN scheme transparently 🙈
4.1.3. IP Identification 🎀
Okay, the Identification field (IP ID) of the IPv4 header is how you know which fragments belong to one datagram when they get put back together ✨ The IPv4 spec doesn't say exactly how these values get assigned. It just says each packet should get an IP ID that's unique for the source-destination pair and protocol for as long as the datagram (or any of its fragments) could still be alive in the network. So there are lots of ways to assign them, and we've sorted them into three classes 🦋
- Sequential jump 💅 This is the most common policy in today's IP stacks. One IP ID counter gets used for all packet streams. When the sender runs several streams at once, the IP ID can jump by more than one between packets in a single stream. The values are pretty predictable and need fewer bits to send than random ones, and the packet-to-packet jump (based on how many outgoing streams are active and how often they send) usually stays limited 📈
- Random 🎲 Some IP stacks use a pseudo-random number generator. That means no correlation between the IDs of consecutive datagrams, so there's no way to predict the next one. For header compression, this means the IP ID field has to be sent uncompressed with every datagram, which adds two extra octets of header 😤 IP stacks in cellular terminals that need the best header compression efficiency should avoid this policy 🚫
- Sequential 👑 This policy keeps a separate counter for each outgoing packet stream, so the IP ID goes up by one for each packet in the stream, except when it wraps around. That makes the delta constant and well known a priori. It's the most desirable policy for header compression, but honestly it's not used nearly as much as it should be 🥺
To avoid violating RFC 791 [1], packets sharing the same IP address pair and IP protocol number can't use the same IP ID values. So sequential implementations have to make the ID number spaces disjoint for packet streams of the same IP protocol going between the same pair of nodes. There are a few ways to do that, and they all add occasional jumps, which makes the policy less than perfectly sequential. For header compression, fewer jumps are better 🌟
Note that the ID is an IPv4 mechanism, so it's not a problem for IPv6 💖 For IPv4, there are three ways to handle it. First, there's the inefficient-but-reliable option, where the ID field is sent as-is in every packet. That adds two octets to the compressed headers, and it's the best choice if the sender uses random assignment 🫠 Second, there can be more flexible mechanisms that need fewer bits for ID handling, as long as the sender uses sequential jump assignment. Those will probably need even more bits if the sender uses random assignment. So knowing the sender's policy can help you pick between the first two options 🧐 Finally, even for IPv4, header compression could be designed without any extra ID info in the compressed headers. To use those schemes, you have to know which assignment policy the sender is using. That might not be possible to figure out, which means these solutions are pretty uncertain to apply. Still, designers of IPv4 stacks for cellular terminals should use an assignment policy close to sequential 🧚
For TCP compression, the IP ID behaves essentially the same way. The difference is that in RFC 3095 [31], the IP ID is generally inferred from the RTP Sequence Number. In TCP there's no obvious field to play that 'master sequence number' role 🙈
From a busy server, the observed behavior can get pretty erratic 😵💫 This is where sharing the IP compression context across several flows (between the same endpoints) could be really useful. But that only makes a real difference when there are lots of flows between one machine and the server. If context sharing is on the table, it's better to share the IP part of the context 💜
4.1.4. Don't Fragment (DF) flag 🌸
Path-MTU discovery (RFC 1191 for IPv4 [6] and RFC 1981 for IPv6 [11]) is widely used for TCP, but barely used for UDP packet streams 🦋 It uses a DF flag value of '1' to detect whether fragmentation is needed along the end-to-end path and to probe the smallest MTU on the path. Hosts using this technique can be expected to send all packets with DF set to '1', although a host can end PMTU discovery by clearing the DF bit to '0'. So for compression, we expect this field value to stay stable 💅
4.1.5. IP Hop-Limit / Time-To-Live (TTL) ⭐
The Hop-Limit (IPv6) or Time-To-Live (IPv4) field is expected to stay constant for the lifetime of a packet stream, or to bounce between a limited number of values because of route changes 🛣️
4.2. TCP Header 🦋
Any discussion of how compressible the TCP fields are borrows heavily from RFC 1144 [22] 🫶 The premise of how the compression gets done is a little different, though, and the protocol has evolved a bit since then 💖
4.2.1. Sequence Number ✨
okay bestie, understanding how the sequence and acknowledgement numbers behave is essential for any TCP compression scheme 💅
At the simplest level, the sequence number is pretty easy to describe 🌸 But there are a bunch of complicating factors we also need to think about 🙈
For transferring in-sequence data packets, the sequence number goes up by between 0 and an upper limit 🎀 That limit is defined by the MSS (Maximum Segment Size) and, if it's being used, by Path-MTU discovery.
There are common MSS values, but they can be pretty variable and unpredictable for any given flow 🦋 With that variability plus the range of window sizes, it's hard (compared with the RTP case, for example) to pick a "one size fits all" encoding for the sequence number. (The same argument applies equally to the acknowledgement number.) 💀
Okay, so the increment of the sequence number in a packet is the size of that packet's data payload, including the SYN and FIN flags 🧚 This is exactly the relationship that RFC 1144 [22] exploits to compress the sequence number in the most efficient case. This technique may not be directly applicable to a robust solution, but it may be a useful relationship to think about ✨
However, at any point on the path (wherever a compressor might be deployed), the sequence number can be anywhere within a range defined by the TCP window 👑 That range is a combination of a few things: buffer space at the sender, the advertised buffer size at the receiver, and TCP congestion control algorithms. Missing packets or retransmissions can make the TCP sequence number fluctuate within the limits of this window 😭
It would be lovely to predict the sequence number with some regularity 🥺 However, that also seems to be difficult to do. For example, during bulk data transfer, the sequence number tends to go up by 1 MSS per packet (assuming no packet loss). Higher layer values have an impact too: sequence number behavior has been observed with an 8 kbyte repeating pattern -- 5 segments of 1460 bytes followed by 1 segment of 892 bytes 🍓 The implementation of TCP and how buffers are managed within a protocol stack can affect the behavior of the sequence number as well.
It may be possible for the compressor to track the TCP window, which would let it bound the size of these jumps 🔥
For interactive flows (for example, telnet), the sequence number changes by small, irregular amounts 💜 In this case, the Nagle algorithm [3] commonly applies, coalescing small packets where possible to reduce the basic header overhead. This may also mean that predictable changes in the sequence number are less likely to occur. The Nagle algorithm is an optimisation and is not required to be used (applications can disable it). However, it is turned on by default in all common TCP implementations 🌈
Also, note that the SYN and FIN flags (which have to be acknowledged) each consume 1 byte of sequence space 🫶
4.2.2. Acknowledgement Number 🦋
Much of the information about the sequence number applies equally to the acknowledgement number. There are some important differences though, so pay attention 👀
For bulk data transfers, there tends to be 1 acknowledgement for every 2 data segments. The algorithm is specified in RFC 2581 [16]. An ACK need not always be sent immediately on receipt of a data segment, but it must be sent within 500ms, and it should be generated for at least every second full-size segment (MSS) of received data 💅 So it follows that the delta for the acknowledgement number is roughly twice that of the sequence number. This isn't always the case, though, so the discussion about sequence number irregularity should be applied here too.
As an aside, a common implementation bug is 'stretch ACKs' [33] (acknowledgements may be generated less frequently than every two full-size data segments). This pattern can also occur following loss on the return path 💀
Since the acknowledgement number is cumulative, dropped packets in the forward path will leave the acknowledgement number constant for a while in the reverse direction. Then, retransmission of a dropped segment can cause a substantial jump in the acknowledgement number 🎀 These jumps are bounded by the TCP window, just like the jumps in sequence number.
In the acknowledgement case, information about the advertised received window gives a bound to the size of any ACK jump ✨
4.2.3. Reserved 🫠
This field is reserved, so you might expect it to be zero. That can no longer be assumed, because of future-proofing. It's only a matter of time before someone suggests a use for the flag 🔮
4.2.4. Flags 🚩
- ECN-E (Explicit Congestion Notification)
'1' to echo the CE bit in the IP header. It will be set in several consecutive headers (until 'acknowledged' by CWR). If ECN nonces are used, there will be a 'nonce-sum' (NS) bit in the flags as well. Again, transparency of the reserved bits is crucial for future-proofing this compression scheme. From an efficiency/compression standpoint, the NS bit will either be unused (always '0') or randomly changing. The nonce sum is the 1-bit sum of the ECT codepoints, as described in [19] 🧠
- CWR (Congestion Window Reduced)
'1' to signal that the congestion window was reduced on ECN. It will generally be set in individual packets. The flag is set once per loss event. So the probability of it being set is proportional to the degree of congestion in the network, but it's less likely to be set than the CE flag 📉
- ECE (Echo Congestion Experience)
If 'congestion experienced' is signaled in a received IP header, this is echoed through the ECE bit in segments sent by the receiver until it's acknowledged by seeing the CWR bit set. Clearly, in periods of high congestion and/or long RTT, this flag will frequently be set to '1' 😤
During connection open (SYN and SYN/ACK packets), the ECN bits have special meaning:
- CWR and ECN-E are both set with SYN to indicate desire to use ECN.
- CWR only is set in SYN-ACK, to agree to ECN.
(The difference in bit-patterns for the negotiation is such that it will work with broken stacks that reflect the value of reserved bits 🛠️)
- URG (Urgent Flag)
'1' to indicate urgent data (which is unlikely with any flag other than ACK) 🚨
- ACK (Acknowledgement)
'1' for all except the initial 'SYN' packet 🤝
- PSH (Push Function Field)
Generally accepted to be randomly '0' or '1'. However, it may be biased more to one value than the other (this is largely caused by the implementation of the stack) 🎲
- RST (Reset Connection)
'1' to reset a connection (unlikely with any flag other than ACK) 💥
- SYN (Synchronize Sequence Number)
'1' for the SYN/SYN-ACK, only at the start of a connection 🌱
- FIN (End of Data: FINished)
'1' to indicate 'no more data' (unlikely with any flag other than ACK) 🏁
4.2.5. Checksum 🔒
Carried as the end-to-end check for the TCP data. See RFC 1144 [22] for a discussion of why this should be carried. A header compression scheme should not rely upon the TCP checksum for robustness, though, and should apply appropriate error-detection mechanisms of its own. The TCP checksum has to be considered randomly changing 🎰
4.2.6. Window 🪟
This may oscillate randomly between 0 and the receiver's window limit (for the connection) 🌊
In practice, the window will either not change or alternate between a relatively small number of values. Particularly when the window is closing (its value is getting smaller), the change in window is likely to be related to the segment size, but it's not clear that this necessarily offers any compression advantage. When the window is opening, remember the effect of 'Silly-Window Syndrome' avoidance. It prevents the window from opening by small amounts that would encourage the sender to clock out small segments 🙅♀️
When you're thinking about which fields might change in a sequence of TCP segments, note that the receiver can generate 'window update' segments in which only the window advertisement changes 🪄
4.2.7. Urgent Pointer 📍
From a compression point of view, the Urgent Pointer is interesting because it's an example where 'semantically identical' compression is not the same as 'bitwise identical' 🤯 The reason is that the value of the Urgent Pointer is only valid if the URG flag is set.
However, the TCP checksum must be passed transparently, in order to maintain its end-to-end integrity checking property. Since the TCP checksum includes the Urgent Pointer in its coverage, this enforces bitwise transparency of the Urgent Pointer. So the issue of 'semantic' vs. 'bitwise' identity comes down to this note: the Urgent Pointer must be compressed in a way that preserves its value 💖
If the URG flag is set, the Urgent Pointer indicates the end of the urgent data and can therefore point anywhere in the window. It may be set (and changing) over several segments. Also, urgent data is rarely used, since it isn't a particularly clean way of managing out-of-band data 🙃
4.3. Options 🎁
Options occupy space at the end of the TCP header. All options are included in the checksum. An option may begin on any byte boundary. The TCP header must be padded with zeros to make the header length a multiple of 32 bits.
Optional header fields are identified by an option kind field. Options 0 and 1 are exactly one octet, which is their kind field. All other options have their one-octet kind field, followed by a one-octet length field, followed by length-2 octets of option data 🌷
4.3.1. Options Overview ✨
Okay so IANA is the official keeper of the TCP options list 📋, and it's the one source of truth you should trust. You can find it at [20]. Figure 12 shows what was allocated at the time of publication, and any new option gets its own 'kind' value from IANA 💅. Where it applies, the associated RFC is cited too 🎀
+----+-------+------------------------------------+----------+-----+
|Kind|Length | Meaning | RFC | Use |
| |octets | | | |
+----+-------+------------------------------------+----------+-----+
| 0 | - | End of Option List | RFC 793 | * |
| 1 | - | No-Operation | RFC 793 | * |
| 2 | 4 | Maximum Segment Size | RFC 793 | * |
| 3 | 3 | WSopt - Window Scale | RFC 1323 | * |
| 4 | 2 | SACK Permitted | RFC 2018 | * |
| 5 | N | SACK | RFC 2018 | * |
| 6 | 6 | Echo (obsoleted by option 8) | RFC 1072 | |
| 7 | 6 | Echo Reply (obsoleted by option 8) | RFC 1072 | |
| 8 | 10 | TSopt - Time Stamp Option | RFC 1323 | * |
| 9 | 2 | Partial Order Connection Permitted | RFC 1693 | |
| 10 | 3 | Partial Order Service Profile | RFC 1693 | |
| 11 | 6 | CC | RFC 1644 | |
| 12 | 6 | CC.NEW | RFC 1644 | |
| 13 | 6 | CC.ECHO | RFC 1644 | |
| 14 | 3 | Alternate Checksum Request | RFC 1146 | |
| 15 | N | Alternate Checksum Data | RFC 1146 | |
| 16 | | Skeeter | | |
| 17 | | Bubba | | |
| 18 | 3 | Trailer Checksum Option | | |
| 19 | 18 | MD5 Signature Option | RFC 2385 | |
| 20 | | SCPS Capabilities | | |
| 21 | | Selective Negative Acks | | |
| 22 | | Record Boundaries | | |
| 23 | | Corruption experienced | | |
| 24 | | SNAP | | |
| 25 | | Unassigned (released 12/18/00) | | |
| 26 | | TCP Compression Filter | | |
+----+-------+------------------------------------+----------+-----+
Figure 12. Common TCP OptionsThe 'use' column has a * on the options most likely to show up in TCP flows 👀. Also, RFC 1072 [4] has been obsoleted by RFC 1323 [7], even though the original bit usage is defined only in RFC 1072 🫶
4.3.2. Option Field Behavior 🧚
Generally speaking, basically every option field gets classified as changing 🌪️. This section walks through the behaviour of each option referenced in an RFC, sorted by 'kind' indicator 💜
0: End of Option List
This option code marks the end of the option list 🍓. It might not line up with the end of the TCP header according to the Data Offset field 🙈. It's used at the end of all options, not after each one, and it only needs to be used if the end of the options wouldn't otherwise match the end of the TCP header. Defined in RFC 793 [2].
There's no data attached to this option, so a compression scheme just has to be able to encode that it's there. Also, since this option marks the end and TCP options are 4-octet aligned, there may be octets of padding (defined to be '0' in [2]) after it ✨
1: No-Operation
This option code can go between options, for example to line up the start of a later option on a word boundary 🧽. There's no guarantee that senders will use it, so receivers must be prepared to process options even if they don't begin on a word boundary RFC 793 [2]. There's no data attached to this option either, so a compression scheme only needs to encode its presence. That can be done by noting the option simply keeps a certain alignment, so compression only needs to convey that alignment. This way, padding can just be removed 💅
2: Maximum Segment Size
If this option is present, it tells you the maximum segment size that may be used to send a packet to this end-host 📏. This field must only be sent in the initial connection request (i.e., in segments with the SYN control bit set). If this option isn't used, any segment size is allowed RFC 793 [2].
This option is super common 💖. The segment size is a 16-bit quantity. In theory it could take any value, but there are a number of values that show up a lot. For example, 1460 bytes is very common for TCP/IPv4 over Ethernet (though with the increased prevalence of tunnels, smaller values such as 1400 have become more popular). 536 bytes is the default MSS value. This may let common values get encoded more efficiently 🎀
3: Window Scale Option (WSopt)
This option may be sent in a SYN segment by the TCP end-host to:
- (1) indicate that the sending TCP end-host is prepared to perform both send and receive window scaling, and
- (2) communicate a scale factor to be applied to its receive window.
The scale factor is encoded logarithmically as a power of 2 (presumably to be implemented by binary shifts) 🔥. Note that the window in the SYN segment itself is never scaled (RFC 1072 [4]). This option may be sent in an initial segment (i.e., in a segment with the SYN bit on and the ACK bit off). It may also be sent in later segments, but only if a Window Scale option was received in the initial segment. A Window Scale option in a segment without a SYN bit should be ignored. The Window field in a SYN segment itself is never scaled (RFC 1323 [7]).
The use of window scaling does not affect the encoding of any other field during the lifetime of the flow. Only the encoding of the window scaling option itself is important. The window scale must be between 0 and 14 (inclusive). Generally, smaller values would be expected (a window scale of 14 allows for a 1Gbyte window, which is extremely large) 👑
4: SACK-Permitted
This option may be sent in a SYN by a TCP that has been extended to receive (and presumably to process) the SACK option once the connection has opened RFC 2018 [12]. There's no data in this option; all that is required is for the presence of the option to be encoded 🌸
5: SACK
This option is used to convey extended acknowledgment information over an established connection 🦋. Specifically, it's sent by a data receiver to tell the data transmitter about non-contiguous blocks of data that have been received and queued. The data receiver waits for data in later retransmissions to fill the gaps in sequence space between these blocks. At that point, the data receiver acknowledges the data, normally by advancing the left window edge in the Acknowledgment Number field of the TCP header. It's important to understand that the SACK option will not change the meaning of the Acknowledgment Number field, whose value will still specify the left window edge, i.e., one byte beyond the last sequence number of fully received data (RFC 2018 [12]) 🧚
If SACK has been negotiated (through an exchange of SACK-Permitted options), then this option may occur when dropped segments are noticed by the receiver. Because this identifies ranges of blocks within the receiver's window, it can be viewed as a base value with a number of offsets. The base value (the left edge of the first block) can be viewed as an offset from the TCP acknowledgement number. There can be up to 4 SACK blocks in a single option. SACK blocks may occur in a number of segments (if there is more out-of-order data 'on the wire'), and this will typically extend the size of or add to the existing blocks 💫
Alternative proposals such as DSACK RFC 2883 [17] don't fundamentally change the behaviour of the SACK block, from the point of view of the information contained within it 🫶
6: Echo
This option carries information that the receiving TCP may send back in a subsequent TCP Echo Reply option (see below) 🔁. A TCP may send the TCP Echo option in any segment, but only if a TCP Echo option was received in a SYN segment for the connection. When the TCP echo option is used for RTT measurement, it will be included in data segments, and the four information bytes will define the time at which the data segment was transmitted, in any format convenient to the sender (see RFC 1072 [4]) ⏱️
The Echo option is generally not used in practice -- it's obsoleted by the Timestamp option 🥲. However, for transparency it's desirable that a compression scheme be able to transport it. (However, there's no benefit in attempting any treatment more sophisticated than viewing it as a generic 'option'.)
7: Echo Reply
A TCP that receives a TCP Echo option containing four information bytes will return these same bytes in a TCP Echo Reply option 🪞. This TCP Echo Reply option must be returned in the next segment (e.g., an ACK segment) that is sent. If more than one Echo option is received before a reply segment is sent, the TCP must choose only one of the options to echo, ignoring the others; specifically, it must choose the newest segment with the oldest sequence number (see RFC 1072 [4]) 🌈
The Echo Reply option is generally not used in practice -- it is obsoleted by the Timestamp option 💀. However, for transparency it's desirable that a compression scheme be able to transport it. (However, there's no benefit in attempting any more sophisticated treatment than viewing it as a generic 'option'.)
8: Timestamps
Okay bestie, this is the timestamps option, and it's giving main character 🕰️✨ It carries two four-byte fields. The first is TSval, the current value of the sending TCP's timestamp clock. The second is TSecr, the timestamp echo reply. TSecr only counts if the ACK bit is set in the TCP header 💅 When it's valid, it echoes a timestamp that the remote TCP sent in its TSval. When TSecr isn't valid, its value must be zero 🙈 Usually TSecr comes from the most recent Timestamps option received, but there are exceptions, which we'll get into below 🫶
A TCP can send Timestamps (TSopt) in an initial segment, meaning one with the SYN bit set and no ACK bit. It can put TSopt in other segments only if it got a TSopt in that initial segment for the connection (see RFC 1323 [7]) 🎀 Timestamps are super common. If they get exchanged during connection setup, then they're expected on every segment after that. If that exchange doesn't happen, they just won't show up for the rest of the flow 😭
Here's the thing: the value is a timestamp, so it's logical that you don't need to send the whole thing. There's no obvious pattern of increments you could count on though, so it's not that simple 🧚
The big reason to use timestamps is detecting sequence space wrap-around, which is PAWS (Protection Against Wrapped Sequence-number, see RFC 1323 [7]) 👑 It's not expected to be a serious concern on the links where TCP header compression would be deployed, but the integrity of this option still has to be kept intact 💜 RFC 3150 [32] talks about this issue. Also, the proposed Eifel algorithm [35] uses timestamps, so right now it's recommended that timestamps be used for cellular-type links [34] 📱✨
On the compression side, the resolution range for the timestamp suggested in RFC 1323 [7] is super wide, from 1ms to 1s per 'tick' 🌈 Combine that with the maybe-wide variation in RTT and it's hard to pick a set of encodings that's optimal in every case 😭
9: Partial Order Connection (POC) permitted
This option is a simple indicator that the two peer transport entities pass to each other to set up the POC protocol 🦋 See RFC 1693 [9].
Partial Order Connection barely gets used on the Internet right now, so the only requirement is that the header compression scheme can encode it 💅
10: POC service profile
This option carries the info the protocol needs to do its job, the kind of stuff you'd usually find in a TCP segment's header 🍓 Again, Partial Order Connection sees little (or no) use in the current Internet, so the only requirement is that the header compression scheme can encode it 🌸
11: Connection Count (CC)
This option is part of TCP Accelerated Open (TAO), which basically skips the TCP Three-Way Handshake (3WHS) ⭐ TAO adds a 32-bit incarnation number called a "connection count" (CC), carried in a TCP option in every segment 💖 Each direction of an open connection gets its own CC value. The implementation hands out monotonically increasing CC values to successive connections it opens, whether actively or passively (see RFC 1644 [8]). CC sees little (or no) use in the current Internet, so the only requirement is that the header compression scheme can encode it 🫶
12: CC.NEW
TAO only stays correct if clients generate monotonically increasing CC values for successive connection initiations 🔥 When a server gets a CC.NEW, it invalidates its cache entry and does a 3WHS instead. See RFC 1644 [8]. CC.NEW sees little (or no) use in the current Internet, so the only requirement is that the header compression scheme can encode it 🙈
13: CC.ECHO
When a server host sends a segment, it echoes the connection count from the initial one in a CC.ECHO option. The client host uses that to validate the segment (see RFC 1644 [8]) 🧚 CC.ECHO also sees little (or no) use in the current Internet, so the only requirement is that the header compression scheme can encode it 💜
14: Alternate Checksum Request
A TCP can send this option in a SYN segment to say it's ready to both generate and receive checksums using an alternate algorithm 👑 During communication, the alternate checksum replaces the regular TCP checksum in the checksum field of the TCP header. If the alternate checksum needs more than 2 octets to transmit, then either the checksum can move into a TCP Alternate Checksum Data Option and the TCP header's checksum field gets sent as zero, or the data can be split between the header field and the option 🎀 Alternate checksums are computed over the same data as the regular TCP checksum; see RFC 1146 [5].
Again, this option sees little (or no) use in the current Internet, so the only requirement is that the header compression scheme can encode it. It would only show up in connection setup (SYN) packets. Even if it were used, it wouldn't affect how the checksum gets handled, since the checksum should be carried transparently anyway 🌈
15: Alternate Checksum Data
This field only gets used when the negotiated alternate checksum is longer than 16 bits 😭 Those checksums won't fit in the TCP header's checksum field, so at least part of them has to go in an option. Whether the checksum gets split between the header field and the option, or the whole thing goes in the option, is decided checksum by checksum 🦋 The length of this option depends on which alternate checksum algorithm the connection uses; see RFC 1146 [5].
If an alternative checksum got negotiated during setup, this option might appear on all later packets, but only if they need to carry the checksum data 💖 In practice, this option is never actually seen, so the only requirement is that the header compression scheme can encode it ✨
16 - 18:
These non-RFC option types aren't considered in this document 🌸
19: MD5 Digest
Every segment sent on a TCP connection that's meant to be protected against spoofing will contain the 16-byte MD5 digest, made by running the MD5 algorithm over a concatenated block of data [13] 🔐💅
When a signed segment arrives, the receiver must validate it by calculating its own digest from the same data, using its own key, and then comparing the two digests. If the comparison fails, the segment must be dropped, and it must not send any response back to the sender 💀 Logging the failure is probably advisable 📝
Unlike other TCP extensions (e.g., the Window Scale option [7]), the absence of this option in the SYN-ACK segment must not cause the sender to stop sending signatures. This negotiation usually happens to avoid some TCP implementations misbehaving when they get options in non-SYN segments. That's not a problem for this option, because the SYN-ACK sent during connection negotiation won't be signed and will get ignored. The connection will never be made, so non-SYN segments with options never get sent. Even more important, sending signatures has to be fully under the control of the application, not at the mercy of a remote host that doesn't understand the option 🙅♀️
MD5 digest info should, like any cryptographically secure data, be incompressible 🔒 So the compression scheme must simply carry this option transparently, if it shows up.
20 - 26;
These non-RFC option types aren't considered in this document. That just means their behaviour isn't described in detail, since a compression scheme isn't expected to be optimised for them 🙃 Still, any unrecognised option must be carried by a TCP compression scheme transparently, so things keep working well when new or rare options show up 🌟
The list above covers the options known when this was written, but more options are expected to be defined later 🔮 It's important that a header compression scheme can handle any future options. The processing of options that aren't defined yet can't be optimised, but unknown options should at the very least be carried transparently 🫧
The current model for TCP options is that an option gets negotiated in the SYN exchange and then used from there on, if the negotiation works 🤝 That leads to some assumptions about when options show up, like only on packets with the SYN flag set, or on every packet. Where those assumptions hold, it may be possible to optimise compression of options a little bit. But it's seen as undesirable to be that constrained, since there's no guarantee that option handling and negotiation will stay the same in the future 🙈 Also, a compressor may not process the SYN packets of a flow, so it can't be assumed to know which options were negotiated 👀
5. Other Observations ✨
5.1. Implicit Acknowledgements
There may be a small number of cues for 'implicit acknowledgements' in a TCP flow 🕵️♀️ Even if the compressor only sees the data transfer direction, for example, seeing a packet without the SYN flag set implies that the SYN packet has been received.
There's a clear requirement that deploying compression be topologically independent 🗺️ This means it's not actually possible to be sure that seeing a data packet at the compressor guarantees the decompressor correctly received the SYN packet, since the SYN packet may have taken a different path 😭
Still, there may be other cues that can be used in certain circumstances to improve compression efficiency 💖
5.2. Shared Data
okay so there are two distinct deployments bestie 🦋 (i) where the forward (data) and reverse (ACK) path are both carried over a common link, and (ii) where the forward (data) and reverse (ACK) path are carried over different paths, with a specific link carrying packets for only one direction of communication.
in the first case 💜 a compressor and decompressor could be colocated. it may then be possible for the compressor and decompressor at each end of the link to exchange information. this could lead to possible optimizations ✨
for example, acknowledgement numbers are generally taken from the sequence numbers in the opposite direction. since an acknowledgement cannot be generated for a packet that has not passed across the link, this offers an efficient way of encoding acknowledgements 🧚
5.3. TCP Header Overhead
for a TCP bulk data-transfer, the overhead of the TCP header does not form a large proportion of the data packet (e.g., < 3% for a 1460 octet packet), particularly compared to the typical RTP voice case 🎀 spectral efficiency is clearly an important goal. however, extracting every last bit of compression gain offers only marginal benefit at a considerable cost in complexity. this trade-off, of efficiency and complexity, must be addressed in the design of a TCP compression profile ⭐
however, in the acknowledgement direction (i.e., for 'pure' acknowledgement headers), the overhead could be said to be infinite (since there is no data being carried). this is why optimizations for the acknowledgement path may be considered useful 💀
there are a number of schemes for manipulating TCP acknowledgements to reduce the ACK bandwidth. many of these are documented in [33] and [32]. most of these schemes are entirely compatible with header compression, without requiring any particular support. while it is not expected that a compression scheme will be optimised for experimental options, it is useful to consider these when developing header compression schemes, and vice versa. a header compression scheme must be able to support any option (including ones as yet undefined) 🫶
5.4. Field Independence and Packet Behavior
it should be apparent that direct comparisons with the highly 'packet'-based view of RTP compression are hard 🙈 RTP header fields tend to change regularly per-packet, and many fields (IPv4 IP ID, RTP sequence number, and RTP timestamp, for example) typically change in a dependent manner. however, TCP fields, such as sequence number, tend to change more unpredictably, partly because of the influence of external factors (size of TCP windows, application behavior, etc.). also, the field values tend to change independently. overall, this makes compression more challenging and makes it harder to select a set of encodings that can successfully trade off efficiency and robustness 😭
5.5. Short-Lived Flows
it is hard to see what can be done to improve performance for a single, unpredictable, short-lived connection 🌸 however, there are commonly cases where there will be multiple TCP connections between the same pair of hosts. as a particular example, consider web browsing (this is more the case with HTTP/1.0 [25] than with HTTP/1.1 [26]).
when a connection closes, either it is the last connection between that pair of hosts or it is likely that another connection will open within a relatively short space of time. in this case, the IP header part of the context (i.e., those fields characterised in Section 2.1) will probably be almost identical. certain aspects of the TCP context may also be similar 👑
support for context replication is discussed in more detail in Section 3. overall, support for sub-context sharing or initializing one context from another offers useful optimizations for a sequence of short-lived connections ✨
note that, although TCP is connection oriented, it is hard for a compressor to tell whether a TCP flow has finished. for example, even in the 'bi-directional' link case, seeing a FIN and the ACK of the FIN at the compressor/decompressor does not mean that the FIN cannot be retransmitted. thus, it may be more useful to think about initializing a new context from an existing one, rather than re-using an existing one 🍓
as mentioned previously in Section 4.1.3, the IP header can clearly be shared between any transport-layer flows between the same two end-points. there may be limited scope for initialisation of a new TCP header from an existing one. the port numbers are the most obvious starting point 🌈
5.6. Master Sequence Number
as pointed out earlier, in Section 4.1.3, there is no obvious candidate for a 'master sequence number' in TCP. moreover, it is noted that such a master sequence number is only required to allow a decompressor to acknowledge packets in bi-directional mode. it can also be seen that such a sequence number would not be required for every packet 💅
while the sequence number only needs to be 'sparse', it is clear that there is a requirement for an explicitly added sequence number. there are no obvious ways to guarantee the unique identity of a packet other than by adding such a sequence number (sequence and acknowledgement numbers can both remain the same, for example) 🦋
5.7. Size Constraint for TCP Options
as can be seen from the above analysis, most TCP options, such as MSS, WSopt, or SACK-Permitted, may appear only on a SYN segment. every implementation should (and we expect that most will) ignore unknown options on SYN segments 🔥 TCP options will be sent on non-SYN segments only when an exchange of options on the SYN segments has indicated that both sides understand the extension. other TCP options, such as MD5 Digest or Timestamp, also tend to be sent when the connection is initiated (i.e., in the SYN packet).
the total header size is also an issue. the TCP header specifies where segment data starts with a 4-bit field that gives the total size of the header (including options) in 32-bit words. this means that the total size of the header plus option must be less than or equal to 60 bytes. this leaves 40 bytes for options 🧚💖
6. Security Considerations 🔐
since this document only describes TCP field behavior, it raises no direct security concerns 🫶
this memo is intended to be used to aid the compression of TCP/IP headers. where authentication mechanisms such as IPsec AH [24] are used, it is important that compression be transparent. where encryption methods such as IPsec ESP [27] are used, the TCP fields may not be visible, preventing compression. this is serious, bestie, so keep it in mind when you pick your mechanisms 😭💀
7. Acknowledgements 🌷
many IP and TCP RFCs (hopefully all of which have been collated below), together with header compression schemes from RFC 1144 [22], RFC 3544 [36], and RFC 3095 [31], and of course the detailed analysis of RTP/UDP/IP in RFC 3095, have been sources of ideas and knowledge. further background information can also be found in [28] and [29] ✨
this document also benefited from discussion on the ROHC mailing list and in various corridors (virtual or otherwise) about many key issues. special thanks go to Qian Zhang, Carsten Bormann, and Gorry Fairhurst 👑
Qian Zhang and Hongbin Liao contributed the extensive analysis of shareable header fields 💜
any remaining misrepresentation or misinterpretation of information is entirely the fault of the authors 🙈
8. References 📚
8.1. Normative References
[1] Postel, J., "Internet Protocol", STD 5, RFC 791, September 1981.
[2] Postel, J., "Transmission Control Protocol", STD 7, RFC 793, September 1981.
[3] Nagle, J., "Congestion control in IP/TCP internetworks", RFC 896, January 1984.
[4] Jacobson, V. and R. Braden, "TCP extensions for long-delay paths", RFC 1072, October 1988.
[5] Zweig, J. and C. Partridge, "TCP alternate checksum options", RFC 1146, March 1990.
[6] Mogul, J. and S. Deering, "Path MTU discovery", RFC 1191, November 1990.
[7] Jacobson, V., Braden, B., and D. Borman, "TCP Extensions for High Performance", RFC 1323, May 1992.
[8] Braden, B., "T/TCP -- TCP Extensions for Transactions Functional Specification", RFC 1644, July 1994.
[9] Connolly, T., Amer, P., and P. Conrad, "An Extension to TCP: Partial Order Service", RFC 1693, November 1994.
[10] Bellovin, S., "Defending Against Sequence Number Attacks", RFC 1948, May 1996.
[11] McCann, J., Deering, S., and J. Mogul, "Path MTU Discovery for IP version 6", RFC 1981, August 1996.
[12] Mathis, M., Mahdavi, J., Floyd, S., and A. Romanow, "TCP Selective Acknowledgment Options", RFC 2018, October 1996.
[13] Heffernan, A., "Protection of BGP Sessions via the TCP MD5 Signature Option", RFC 2385, August 1998.
[14] Nichols, K., Blake, S., Baker, F., and D. Black, "Definition of the Differentiated Services Field (DS Field) in the IPv4 and IPv6 Headers", RFC 2474, December 1998.
[15] Ramakrishnan, K. and S. Floyd, "A Proposal to add Explicit Congestion Notification (ECN) to IP", RFC 2481, January 1999.
[16] Allman, M., Paxson, V., and W. Stevens, "TCP Congestion Control", RFC 2581, April 1999.
[17] Floyd, S., Mahdavi, J., Mathis, M., and M. Podolsky, "An Extension to the Selective Acknowledgement (SACK) Option for TCP", RFC 2883, July 2000.
[18] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition of Explicit Congestion Notification (ECN) to IP", RFC 3168, September 2001.
[19] Spring, N., Wetherall, D., and D. Ely, "Robust Explicit Congestion Notification (ECN) Signaling with Nonces", RFC 3540, June 2003.
8.2. Informative References ✨
okay so these are the "go read this if you want the full story" refs, bestie 📚 they're not required to implement the spec, but they're the receipts behind a lot of the ideas. Here's the whole list, in the order it's written 💅
- [20] IANA, "IANA", IANA TCP options, February 1998, <http://www.iana.org/assignments/tcp-parameters>.
- [21] Braden, R., "Requirements for Internet Hosts - Communication Layers", STD 3, RFC 1122, October 1989.
- [22] Jacobson, V., "Compressing TCP/IP headers for low-speed serial links", RFC 1144, February 1990.
- [23] Almquist, P., "Type of Service in the Internet Protocol Suite", RFC 1349, July 1992.
- [24] Kent, S. and R. Atkinson, "IP Authentication Header", RFC 2402, November 1998.
- [25] Berners-Lee, T., Fielding, R., and H. Nielsen, "Hypertext Transfer Protocol -- HTTP/1.0", RFC 1945, May 1996.
- [27] Kent, S. and R. Atkinson, "IP Encapsulating Security Payload (ESP)", RFC 2406, November 1998.
- [26] Fielding, R., Gettys, J., Mogul, J., Nielsen, H., and T. Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1", RFC 2068, January 1997.
- [28] Degermark, M., Nordgren, B., and S. Pink, "IP Header Compression", RFC 2507, February 1999.
- [29] Casner, S. and V. Jacobson, "Compressing IP/UDP/RTP Headers for Low-Speed Serial Links", RFC 2508, February 1999.
- [30] Bradner, S. and V. Paxson, "IANA Allocation Guidelines For Values In the Internet Protocol and Related Headers", BCP 37, RFC 2780, March 2000.
- [31] Bormann, C., Burmeister, C., Degermark, M., Fukushima, H., Hannu, H., Jonsson, L-E., Hakenberg, R., Koren, T., Le, K., Liu, Z., Martensson, A., Miyazaki, A., Svanbro, K., Wiebke, T., Yoshimura, T., and H. Zheng, "RObust Header Compression (ROHC): Framework and four profiles: RTP, UDP, ESP, and uncompressed", RFC 3095, July 2001.
- [32] Dawkins, S., Montenegro, G., Kojo, M., and V. Magret, "End-to-end Performance Implications of Slow Links", BCP 48, RFC 3150, July 2001.
- [33] Balakrishnan, Padmanabhan, V., Fairhurst, G., and M. Sooriyabandara, "TCP Performance Implications of Network Path Asymmetry", RFC 3449, December 2002.
- [34] Inamura, H., Montenegro, G., Ludwig, R., Gurtov, A., and F. Khafizov, "TCP over Second (2.5G) and Third (3G) Generation Wireless Networks", RFC 3481, February 2003.
- [35] Ludwig, R. and M. Meyer, "The Eifel Detection Algorithm for TCP", RFC 3522, April 2003.
- [36] Engan, M., Casner, S., Bormann, C., and T. Koren, "IP Header Compression over PPP", RFC 3544, July 2003.
- [37] Karn, P., Bormann, C., Fairhurst, G., Grossman, D., Ludwig, R., Mahdavi, J., Montenegro, G., Touch, J., and L. Wood, "Advice for Internet Subnetwork Designers", BCP 89, RFC 3819, July 2004.
Authors' Addresses 🎀
these are the two humans who wrote it, and where to find them. ngl the Roke Manor crew were very organised 🌸
Mark A. West
Siemens/Roke Manor Research
Roke Manor Research Ltd.
Romsey, Hants SO51 0ZN
UK
Phone: +44 (0)1794 833311
EMail: mark.a.west@roke.co.uk
URI: http://www.roke.co.ukStephen McCann
Siemens/Roke Manor Research
Roke Manor Research Ltd.
Romsey, Hants SO51 0ZN
UK
Phone: +44 (0)1794 833341
EMail: stephen.mccann@roke.co.uk
URI: http://www.roke.co.ukFull Copyright Statement 💜
okay the legal bit, but make it cute 🧚 the Internet Society owns the copyright for this one, dated 2006. The document is covered by the rights, licenses and restrictions in BCP 78, and apart from what that says, the authors keep all their rights 💖
Now the important bit, and it's giving serious disclaimer energy 💀 The document and everything in it is handed over "AS IS", and the contributor, the organisation they represent or sponsor (if there is one), the Internet Society and the IETF all say no to every warranty, express or implied. That includes no promise that using the info won't infringe anyone's rights, and no implied promise of merchantability or fitness for a particular purpose:
THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Intellectual Property 🦋
the IETF doesn't say whether any IPR (intellectual property rights) claimed to cover this tech is valid or how far it reaches. It also doesn't say whether licences would be available, and it hasn't checked for any rights itself 🙈 If you want the rules on RFC rights, look in BCP 78 and BCP 79 ✨
Copies of IPR disclosures sent to the IETF Secretariat, any licence assurances, and the results of any attempt to get a general licence or permission for implementers or users are all available from the IETF's online IPR repository at http://www.ietf.org/ipr 🌈
And the IETF invites anyone who knows about copyrights, patents, patent applications, or other proprietary rights that might cover technology needed to implement this standard to tell them. Send that info to ietf-ipr@ietf.org 📬
Acknowledgement 🫶
the RFC Editor function is funded by the IETF Administrative Support Activity (IASA). Thank you to them for keeping the lights on ✨👑
📦 where this page comes from
- ietf: rfc4413 2006-03
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms