RFC 1323
TCP Extensions for High Performance
Part 1 of 8 🎀
okay bestie, this is RFC 1323: TCP Extensions for High Performance 💖 It came out in May 1992 from the Network Working Group, written by V. Jacobson (LBL), R. Braden (ISI), and D. Borman (Cray Research). It's an IAB standards track protocol, and it obsoletes RFC 1072 and RFC 1185 🫶
Status of This Memo ✨
so this RFC is an IAB standards track protocol for the Internet community, and it's asking for discussion and suggestions for improvements 🙌 for the current standardization state and status of this protocol, check the latest edition of the "IAB Official Protocol Standards" 📚 and distribution of this memo is unlimited 🌈
Abstract 💅
this memo is a set of TCP extensions that level up performance over large bandwidth*delay product paths and give you reliable operation over very high-speed paths 🚀 it adds new TCP options for scaled windows and timestamps, and they're designed to play nicely with TCPs that don't implement the extensions 🤝 the timestamps power two mechanisms: RTTM (Round Trip Time Measurement) and PAWS (Protect Against Wrapped Sequences) 🧚 selective acknowledgments are not included in this memo 🙈
this memo combines and supersedes RFC-1072 and RFC-1185, and adds more clarification and more detailed specification ✍️ Appendix C summarizes the changes from the earlier RFCs 💜
Table of Contents 🧚
1. Introduction ................................................. 2
2. TCP Window Scale Option ...................................... 8
3. RTTM -- Round-Trip Time Measurement .......................... 11
4. PAWS -- Protect Against Wrapped Sequence Numbers ............. 17
5. Conclusions and Acknowledgments .............................. 25
6. References ................................................... 25
APPENDIX A: Implementation Suggestions ........................... 27
APPENDIX B: Duplicates from Earlier Connection Incarnations ...... 27
APPENDIX C: Changes from RFC-1072, RFC-1185 ...................... 30
APPENDIX D: Summary of Notation .................................. 31
APPENDIX E: Event Processing ..................................... 32
Security Considerations .......................................... 37
Authors' Addresses ............................................... 371. Introduction ✨
okay so TCP was designed to run reliably over basically any transmission medium, no matter the transmission rate, delay, corruption, duplication, or reordering of segments 🦋 production TCP implementations can currently adapt to transfer rates from 100 bps up to 10**7 bps, and round-trip delays from 1 ms up to 100 seconds 🌸 recent work on TCP performance has shown it can work well over a whole range of Internet paths, from 800 Mbit/sec I/O channels all the way down to 300 bit/sec dial-up modems [Jacobson88a] 😭
the introduction of fiber optics means transmission speeds keep getting higher, and the fastest paths are moving out of the territory TCP was originally engineered for 🏎️ this memo defines a set of modest extensions to TCP that stretch its application domain to match that growing network capability 💅 it is based upon and obsoletes RFC-1072 [Jacobson88b] and RFC-1185 [Jacobson90b] 🎀
there is no one-line answer to the question: "How fast can TCP go?" 🤷♀️ there are two separate kinds of issues, performance and reliability, and each one depends on different parameters 💖 we'll go through each in turn 🧚
1.1 TCP Performance 🚀
TCP performance depends not on the transfer rate itself, but on the product of the transfer rate and the round-trip delay 🤯 this "bandwidth*delay product" measures the amount of data that would "fill the pipe" 🪣 it's the buffer space required at sender and receiver to get maximum throughput on the TCP connection over the path, i.e., the amount of unacknowledged data TCP must handle to keep the pipeline full 💦 TCP performance problems show up when the bandwidth*delay product is large ⚠️ we call an Internet path operating in this region a "long, fat pipe", and a network containing this path an "LFN" (pronounced "elephan(t)") 🐘 iconic 👑
High-capacity packet satellite channels (e.g., DARPA's Wideband Net) are LFN's 🛰️ for example, a DS1-speed satellite channel has a bandwidth*delay product of 10**6 bits or more, which corresponds to 100 outstanding TCP segments of 1200 bytes each 🤓 terrestrial fiber-optical paths fall into the LFN class too 🌈 for example, a cross-country delay of 30 ms at a DS3 bandwidth (45Mbps) also exceeds 10**6 bits ✨
there are three fundamental performance problems with the current TCP over LFN paths, and we're going to tackle them one at a time 🫶
(1) Window Size Limit
the TCP header uses a 16 bit field to report the receive window size to the sender 🙃 therefore, the largest window that can be used is 2**16 = 65K bytes 💀
to get around this, Section 2 of this memo defines a new TCP option, "Window Scale", that allows windows larger than 2**16 🎉 this option defines an implicit scale factor, which gets used to multiply the window size value found in a TCP header to get the true window size 🧠 main character energy 🌟
(2) Recovery from Losses
packet losses in an LFN can have a catastrophic effect on throughput 😭 until recently, properly-operating TCP implementations would drain the data pipeline with every packet loss and need a slow-start action to recover 🫠 then came the Fast Retransmit and Fast Recovery algorithms [Jacobson90c], and their combined effect is to recover from one packet loss per window without draining the pipeline 🔥 the catch is that more than one packet loss per window typically results in a retransmission timeout, plus the resulting pipeline drain and slow start 🙈
expanding the window size to match the capacity of an LFN means the probability of more than one packet per window being dropped goes up too 📈 that could be absolutely devastating for TCP throughput over an LFN 💀 and if a congestion control mechanism based on some form of random dropping got introduced into gateways, randomly spaced packet drops would become common, possibly increasing the probability of dropping more than one packet per window 🎲
to generalize Fast Retransmit/Fast Recovery so it can handle multiple packets dropped per window, selective acknowledgments are required 🧩 unlike the normal cumulative acknowledgments of TCP, selective acknowledgments give the sender a complete picture of which segments are queued at the receiver and which haven't arrived yet 🖼️ there's some evidence in favor of selective acknowledgments that has been published [NBS85], and they've been included in a bunch of experimental Internet protocols: VMTP [Cheriton88], NETBLT [Clark87], and RDP [Velten84], and they were proposed for OSI TP4 [NBS85] 📚 but in the non-LFN regime, selective acknowledgments reduce the number of packets retransmitted without otherwise improving performance, which makes their complexity questionable 🤔 still, selective acknowledgments are expected to become much more important in the LFN regime 🚀
RFC-1072 defined a new TCP SACK option to send a selective acknowledgment 🙌 however, there are important technical issues still to be worked out about both the format and semantics of the SACK option 🛠️ so SACK has been omitted from this package of extensions 😢 it's hoped that SACK can "catch up" during the standardization process 🏃♀️💨
(3) Round-Trip Measurement
TCP does reliable data delivery by retransmitting segments that aren't acknowledged within some retransmission timeout (RTO) interval ⏱️ accurate dynamic determination of an appropriate RTO is essential to TCP performance 🎯 the RTO is determined by estimating the mean and variance of the measured round-trip time (RTT), i.e., the time between sending a segment and receiving an acknowledgment for it [Jacobson88a] 📏
Section 4 introduces a new TCP option, "Timestamps", and then defines a mechanism using it that lets nearly every segment, retransmissions included, get timed at negligible computational cost ⚡ we use the mnemonic RTTM (Round Trip Time Measurement) for this mechanism, to tell it apart from other uses of the Timestamps option 🏷️
1.2 TCP Reliability 🛡️
now we turn from performance to reliability 🫶 high transfer rate enters TCP performance through the bandwidth*delay product 🚄 however, high transfer rate alone can threaten TCP reliability by violating the assumptions behind the TCP mechanism for duplicate detection and sequencing 😱
an especially serious kind of error can come from an accidental reuse of TCP sequence numbers in data segments 🙀 imagine an "old duplicate segment", e.g., a duplicate data segment that got delayed in Internet queues, gets delivered to the receiver at the wrong moment, so that its sequence numbers fall somewhere within the current window 👻 there would be no checksum failure to warn of the error, and the result could be undetected corruption of the data 💔 reception of an old duplicate ACK segment at the transmitter could be only slightly less serious: it's likely to lock up the connection so that no further progress can be made, forcing an RST on the connection 🔒
TCP reliability depends on the existence of a bound on the lifetime of a segment, the "Maximum Segment Lifetime" or MSL 🧮 an MSL is generally required by any reliable transport protocol, since every sequence number field must be finite, and therefore any sequence number may eventually be reused 🔁 in the Internet protocol suite, the MSL bound is enforced by an IP-layer mechanism, the "Time-to-Live" or TTL field 🕰️
duplication of sequence numbers might happen in either of two ways 👯
- Sequence number wrap-around on the current connection
A TCP sequence number contains 32 bits 🔢 at a high enough transfer rate, the 32-bit sequence space may get "wrapped" (cycled) within the time a segment is delayed in queues 🌀
- Earlier incarnation of the connection
say a connection terminates, either by a proper close sequence or due to a host crash, and the same connection (i.e., using the same pair of sockets) is immediately reopened 🔄 a delayed segment from the terminated connection could fall within the current window for the new incarnation and be accepted as valid 😳 not me accepting an old segment like that 💅
Okay so duplicates from earlier incarnations, aka Case (2), get avoided by enforcing the current fixed MSL of the TCP spec, which Section 5.3 and Appendix B explain 💅. But case (1), avoiding reuse of sequence numbers within the same connection, needs an MSL bound that depends on the transfer rate. At high enough rates, we need a brand new mechanism 😭🔥
More specifically, if the max effective bandwidth TCP can push over a particular path is B bytes per second, then this constraint has to hold for error-free operation:
2**31 / B > MSL (secs) [1]Here's the table of Twrap = 2**31/B in seconds for some important bandwidths B 🍓
Network B*8 B Twrap
bits/sec bytes/sec secs
_______ _______ ______ ______
ARPANET 56kbps 7KBps 3*10**5 (~3.6 days)
DS1 1.5Mbps 190KBps 10**4 (~3 hours)
Ethernet 10Mbps 1.25MBps 1700 (~30 mins)
DS3 45Mbps 5.6MBps 380
FDDI 100Mbps 12.5MBps 170
Gigabit 1Gbps 125MBps 17It's clear that wrap-around of the sequence space is not a problem for 56kbps packet switching or even 10Mbps Ethernets 🦋 But at DS3 and FDDI speeds, Twrap is comparable to the 2 minute MSL assumed by the TCP specification [Postel81]. Moving toward gigabit speeds, Twrap gets too small to be reliably enforced by the Internet TTL mechanism 💀
The 16-bit window field of TCP limits the effective bandwidth B to 2**16/RTT, where RTT is the round-trip time in seconds [McKenzie89] 👑 If the RTT is big enough, that cap keeps B to a value that meets the constraint [1] for a large MSL value. Say you've got a transcontinental backbone with an RTT of 60ms (thanks, laws of physics 🙄). With the bandwidth*delay product capped at 64KB by the TCP window size, B is limited to 1.1MBps, no matter how high the theoretical transfer rate of the path is. That works out to cycling the sequence number space in Twrap= 2000 secs, which is safe in today's Internet ✨
It's important to get that the culprit isn't the larger window, it's the high bandwidth 😤 Take a (very large) FDDI LAN with a diameter of 10km. Using the speed of light, the RTT across the ring is (2*10**4)/(3*10**8) = 67 microseconds, and the delay*bandwidth product is then 833 bytes. A TCP connection across this LAN using a window of only 833 bytes will run at the full 100mbps and can wrap the sequence space in about 3 minutes, which is very close to the MSL of TCP. So high speed alone can cause a reliability problem with sequence number wrap-around, even without extended windows 🏎️
Watson's Delta-T protocol [Watson81] has network-layer mechanisms for precise enforcement of an MSL 🧚 The IP mechanism for MSL enforcement is loosely defined and even more loosely implemented in the Internet, though. So it's unwise to depend on active enforcement of MSL for TCP connections, and it's unrealistic to imagine setting MSLs smaller than the current values (e.g., 120 seconds specified for TCP) 🙈
A possible fix for cycling the sequence space would be to increase the size of the TCP sequence number field. For example, the sequence number field (and the acknowledgment field too) could be expanded to 64 bits. This could be done either by changing the TCP header or by adding an option 🎀
Section 5 presents a different mechanism, which we call PAWS (Protect Against Wrapped Sequence numbers), to extend TCP reliability to transfer rates well beyond the foreseeable upper limit of network bandwidths 🚀 PAWS uses the TCP Timestamps option defined in Section 4 to protect against old duplicates from the same connection.
1.3 Using TCP options ✨
The extensions defined in this memo all use new TCP options. Two possible issues come up with that: (1) compatibility and (2) overhead 🫶
First up, compatibility. We have to pay careful attention to interoperating with existing implementations. The only TCP option defined previously, MSS, is only allowed on a SYN segment. Every implementation should (and we expect most will) ignore unknown options on SYN segments. But some buggy TCP implementations might crash the first time they see an option on a non-SYN segment 💀 So for each of the extensions defined below, TCP options only get sent on non-SYN segments once an exchange of options on the SYN segments has shown that both sides understand the extension. Furthermore, an extension option only gets sent in a <SYN,ACK> segment if the corresponding option was received in the initial <SYN> segment 🙌
Next, overhead. A question may come up about the bandwidth and processing overhead for TCP options. Options that show up on SYN segments aren't likely to cause a performance concern. Opening a TCP connection already requires a bunch of special-case code, so processing options probably won't add much cost 🍓
On the other hand, a Timestamps option may appear in any data or ACK segment, adding 12 bytes to the 20-byte TCP header. We believe the bandwidth saved by cutting unnecessary retransmissions will more than pay for that extra header bandwidth 🦋
There's also the question of processing overhead for parsing the variable byte-aligned format of options, especially on a RISC-architecture CPU. To deal with that, Appendix A has a recommended layout of the options in TCP headers to get reasonable data field alignment. In the spirit of Header Prediction, a TCP can quickly test for this layout and, if it checks out, take a fast path. Hosts that use this canonical layout will effectively treat the options as a set of fixed-format fields appended to the TCP header. But to keep the philosophical and protocol framework of TCP options intact, a TCP has to be prepared to parse an arbitrary options field too, just with less efficiency 🧚
Finally, most of the mechanisms in this memo matter for LFN's and/or very high-speed networks 🚄 For low-speed networks, it might be a performance optimization to NOT use these mechanisms. A TCP vendor who cares about optimal performance over low-speed paths might consider turning these extensions off for those paths, or let a user or installation manager disable them 🌸
2. TCP WINDOW SCALE OPTION 💖
2.1. Introduction 🎀
okay so the window scale extension glow-ups the TCP window to 32 bits 💅 and then uses a scale factor to squeeze that 32-bit value into the 16-bit Window field of the TCP header (SEG.WND in RFC-793). The scale factor gets carried in a brand new TCP option called Window Scale. This option is sent only in a SYN segment (a segment with the SYN bit on), which means the window scale is fixed in each direction when a connection is opened 🔒.
ngl there was another option: put the window scale in every single TCP segment 🙄. But sending it only when the scale factor changed would be incorrect, because a TCP option in an acknowledgement segment won't be delivered reliably (unless the ACK just happens to be piggy-backed on data going the other way) 😬. Fixing the scale at open time has the advantage of lower overhead, but the disadvantage is that the scale factor cannot be changed during the connection 🥲.
The maximum receive window (and therefore the scale factor) comes from the maximum receive buffer space 🧠. In a typical modern implementation that buffer space is set by default, but a user program can override it before a TCP connection is opened ✨. Since that determines the scale factor, window scaling doesn't need any new user interface 🙌.
2.2. Window Scale Option 🍓
The three-byte Window Scale option MAY be sent in a SYN segment by a TCP. It has two purposes: (1) indicate that the TCP is prepared to do both send and receive window scaling, and (2) communicate a scale factor to be applied to its receive window. Thus, a TCP that is prepared to scale windows SHOULD send the option, even if its own scale factor is 1. The scale factor is limited to a power of two and encoded logarithmically, so it may be implemented by binary shift operations 🧮.
TCP Window Scale Option (WSopt):
Kind: 3 Length: 3 bytes
+---------+---------+---------+
| Kind=3 |Length=3 |shift.cnt|
+---------+---------+---------+This option is an offer, not a promise 🤝. Both sides MUST send Window Scale options in their SYN segments to enable window scaling in either direction. If window scaling is enabled, then the TCP that sent this option will right-shift its true receive-window values by shift.cnt bits for transmission in SEG.WND. The value shift.cnt MAY be zero (offering to scale, while applying a scale factor of 1 to the receive window).
This option MAY be sent in an initial <SYN> segment (i.e., a segment with the SYN bit on and the ACK bit off). It MAY also be sent in a <SYN,ACK> segment, but only if a Window Scale option was received in the initial <SYN> segment. A Window Scale option in a segment without a SYN bit SHOULD be ignored 🚫.
The Window field in a SYN (i.e., a <SYN> or <SYN,ACK>) segment itself is never scaled 🙅♀️.
2.3. Using the Window Scale Option 🧁
A model implementation of window scaling goes like this, using the notation of RFC-793 [Postel81] 📝:
- All windows are treated as 32-bit quantities for storage in the connection control block and for local calculations. This includes the send-window (
SND.WND) and the receive-window (RCV.WND) values, as well as the congestion window. - The connection state is augmented by two window shift counts,
Snd.Wind.ScaleandRcv.Wind.Scale, to be applied to the incoming and outgoing window fields, respectively. - If a TCP receives a
<SYN>segment containing a Window Scale option, it sends its own Window Scale option in the<SYN,ACK>segment. - The Window Scale option is sent with
shift.cnt = R, where R is the value that the TCP would like to use for its receive window. - Upon receiving a SYN segment with a Window Scale option containing
shift.cnt = S, a TCP setsSnd.Wind.Scaleto S and setsRcv.Wind.Scaleto R; otherwise, it sets bothSnd.Wind.ScaleandRcv.Wind.Scaleto zero. - The window field (
SEG.WND) in the header of every incoming segment, with the exception of SYN segments, is left-shifted bySnd.Wind.Scalebits before updatingSND.WND:
SND.WND = SEG.WND << Snd.Wind.Scale(assuming the other conditions of RFC793 are met, and using the "C" notation "<<" for left-shift).
- The window field (
SEG.WND) of every outgoing segment, with the exception of SYN segments, is right-shifted byRcv.Wind.Scalebits:
SEG.WND = RCV.WND >> Rcv.Wind.Scale.TCP determines if a data segment is "old" or "new" by testing whether its sequence number is within 2**31 bytes of the left edge of the window, and if it is not, discarding the data as "old". To insure that new data is never mistakenly considered old and vice-versa, the left edge of the sender's window has to be at most 2**31 away from the right edge of the receiver's window. Similarly with the sender's right edge and receiver's left edge. Since the right and left edges of either the sender's or receiver's window differ by the window size, and since the sender and receiver windows can be out of phase by at most the window size, the above constraints imply that 2 * the max window size MUST be less than 2**31, or
max window < 2**30Since the max window is 2**S (where S is the scaling shift count) times at most 2**16 - 1 (the maximum unscaled window), the maximum window is guaranteed to be < 2*30 if S <= 14. Thus, the shift count MUST be limited to 14 (which allows windows of 2**30 = 1 Gbyte) 💪. If a Window Scale option is received with a shift.cnt value exceeding 14, the TCP SHOULD log the error but use 14 instead of the specified value 🪵.
The scale factor applies only to the Window field as transmitted in the TCP header; each TCP using extended windows will maintain the window values locally as 32-bit numbers. For example, the "congestion window" computed by Slow Start and Congestion Avoidance is not affected by the scale factor, so window scaling will not introduce quantization into the congestion window 🌈.
3. RTTM: ROUND-TRIP TIME MEASUREMENT 🦋
3.1. Introduction 🔍
Accurate and current RTT estimates are necessary to adapt to changing traffic conditions and to avoid an instability known as "congestion collapse" [Nagle84] in a busy network. However, accurate measurement of RTT may be difficult both in theory and in implementation 😵💫.
Many TCP implementations base their RTT measurements upon a sample of only one packet per window. While this yields an adequate approximation to the RTT for small windows, it results in an unacceptably poor RTT estimate for an LFN. If we look at RTT estimation as a signal processing problem (which it is), a data signal at some frequency, the packet rate, is being sampled at a lower frequency, the window rate. This lower sampling frequency violates Nyquist's criteria and may therefore introduce "aliasing" artifacts into the estimated RTT [Hamming77] 📉.
A good RTT estimator with a conservative retransmission timeout calculation can tolerate aliasing when the sampling frequency is "close" to the data frequency. For example, with a window of 8 packets, the sample rate is 1/8 the data frequency -- less than an order of magnitude different. However, when the window is tens or hundreds of packets, the RTT estimator may be seriously in error, resulting in spurious retransmissions 😬.
If there are dropped packets, the problem becomes worse 💀. Zhang [Zhang86], Jain [Jain86] and Karn [Karn87] have shown that it is not possible to accumulate reliable RTT estimates if retransmitted segments are included in the estimate. Since a full window of data will have been transmitted prior to a retransmission, all of the segments in that window will have to be ACKed before the next RTT sample can be taken. This means at least an additional window's worth of time between RTT measurements and, as the error rate approaches one per window of data (e.g., 10**-6 errors per bit for the Wideband satellite network), it becomes effectively impossible to obtain a valid RTT measurement 🛰️.
A solution to these problems, which actually simplifies the sender substantially, is as follows: using TCP options, the sender places a timestamp in each data segment, and the receiver reflects these timestamps back in ACK segments. Then a single subtract gives the sender an accurate RTT measurement for every ACK segment (which will correspond to every other data segment, with a sensible receiver). We call this the RTTM (Round-Trip Time Measurement) mechanism 👑.
It is vitally important to use the RTTM mechanism with big windows; otherwise, the door is opened to some dangerous instabilities due to aliasing. Furthermore, the option is probably useful for all TCP's, since it simplifies the sender 🔥.
3.2. TCP Timestamps Option ⏰
TCP is a symmetric protocol, allowing data to be sent at any time in either direction, and therefore timestamp echoing may occur in either direction. For simplicity and symmetry, we specify that timestamps always be sent and echoed in both directions. For efficiency, we combine the timestamp and timestamp reply fields into a single TCP Timestamps Option 🧩.
TCP Timestamps Option (TSopt):
Kind: 8
Length: 10 bytes
+-------+-------+---------------------+---------------------+
|Kind=8 | 10 | TS Value (TSval) |TS Echo Reply (TSecr)|
+-------+-------+---------------------+---------------------+
1 1 4 4The Timestamps option carries two four-byte timestamp fields. The Timestamp Value field (TSval) contains the current value of the timestamp clock of the TCP sending the option 🕰️.
The Timestamp Echo Reply field (TSecr) is only valid if the ACK bit is set in the TCP header; if it is valid, it echoes a timestamp value that was sent by the remote TCP in the TSval field of a Timestamps option. When TSecr is not valid, its value MUST be zero 0️⃣. The TSecr value will generally be from the most recent Timestamp option that was received; however, there are exceptions that are explained below 👀.
A TCP MAY send the Timestamps option (TSopt) in an initial <SYN> segment (i.e., segment containing a SYN bit and no ACK bit), and MAY send a TSopt in other segments only if it received a TSopt in the initial <SYN> segment for the connection 🤙.
3.3. The RTTM Mechanism 🎯
The timestamp value to be sent in TSval is to be obtained from a (virtual) clock that we call the "timestamp clock". Its values MUST be at least approximately proportional to real time, in order to measure actual RTT ⏱️✨.
The one-way example is my favourite, bestie 💅 Picture A and B chatting, with A taking a big 60-time-unit nap in the middle 😴
TCP A TCP B
<A,TSval=1,TSecr=120> ------>
<---- <ACK(A),TSval=127,TSecr=1>
<B,TSval=5,TSecr=127> ------>
<---- <ACK(B),TSval=131,TSecr=5>
. . . . . . . . . . . . . . . . . . . . . .
<C,TSval=65,TSecr=131> ------>
<---- <ACK(C),TSval=191,TSecr=65>
(etc)Each TSecr is just whatever TSval came in last, and the dotted line is that 60-unit pause where A had nothing to say 🫠 That silence makes B's RTT look way bigger than it really is, since TSecr=131 shows up in data segment C. So in one-way flows, RTTM in the reverse direction measures a number that's inflated by the gaps in sending 🙈 Good news, though: there's a rule that stops the inflation, and it's giving main character energy ✨
A TSecr value received in a segment is used to update the averaged RTT measurement only if the segment acknowledges some new data, i.e., only if it advances the left edge of the send window.
Since TCP B isn't sending any data, segment C doesn't acknowledge anything new when it lands at B. So that inflated RTTM measurement never gets used to update B's RTTM measurement 🦋 Iykyk, and now you do too 🫶
3.4. Which Timestamp to Echo
Okay so, if more than one Timestamps option shows up before a reply segment goes out, the TCP MUST choose just one of the TSvals to echo and ignore the rest ✨ To keep the receiver's state small (aka the number of unprocessed TSvals hanging around), the receiver SHOULD be required to hold on to at most one timestamp in the connection control block 🧚
There are three situations to consider, and each one is its own little drama 🎀
(A) Delayed ACKs.
Lots of TCPs only acknowledge every Kth segment out of a group arriving close together. This policy is generally called "delayed ACKs" 🌸 The data-sender TCP MUST measure the effective RTT, including the extra time from those delayed ACKs, or else it'll retransmit unnecessarily 😭 So when delayed ACKs are in use, the receiver SHOULD reply with the TSval field from the earliest unacknowledged segment 💅
(B) A hole in the sequence space (segment(s) have been lost).
The sender keeps sending until the window is full, and the receiver may be generating ACKs as those out-of-order segments arrive (for example, to help "fast retransmit") 👀
A lost segment is probably a sign of congestion, so in that situation the sender SHOULD be conservative about retransmission 🥺 It's also better to overestimate the RTT than to underestimate it 💜 So an ACK for an out-of-order segment should contain the timestamp from the most recent segment that advanced the window ✨ The same thing happens if the network re-orders segments 🔁
(C) A filled hole in the sequence space.
The segment that fills the hole is the most recent measurement of the network characteristics, and that's the one we want 👑 An RTT computed from an earlier segment would probably include the sender's retransmit time-out, which would badly bias the sender's average RTT estimate 😬 So the timestamp from the latest segment (the one that filled the hole) MUST be echoed 🫶
An algorithm that covers all three cases is laid out in the rules below for Timestamps option processing on a synchronized connection 🌈
- The connection state gets augmented with two 32-bit slots. TS.Recent holds a timestamp to be echoed in TSecr whenever a segment is sent, and Last.ACK.sent holds the ACK field from the last segment sent. Last.ACK.sent will equal RCV.NXT except when ACKs have been delayed 🧁
- If Last.ACK.sent falls within the range of sequence numbers of an incoming segment:
SEG.SEQ <= Last.ACK.sent < SEG.SEQ + SEG.LENthen the TSval from the segment is copied to TS.Recent. Otherwise, the TSval is ignored 🙅♀️
- When a TSopt is sent, its TSecr field is set to the current TS.Recent value 💖
Let's walk through these rules with some examples! A, B, C... are data segments in successive blocks of sequence numbers, and ACK(A),... are the matching acknowledgment segments. Notice ACK(A) has the same sequence number as B 🦋 We only show one direction of timestamp echoing, for clarity 🙈
- Packets arrive in sequence, and some of the ACKs are delayed. By Case (A), the timestamp from the oldest unacknowledged segment gets echoed 🌸
TS.Recent
<A, TSval=1> ------------------->
1
<B, TSval=2> ------------------->
1
<C, TSval=3> ------------------->
1
<---- <ACK(C), TSecr=1>
(etc)- Packets arrive out of order, and every packet is acknowledged. By Case (B), the timestamp from the last segment that advanced the left window edge is echoed until the missing segment shows up. Once it does, it gets echoed according to Case (C) 🔥 The same sequence would happen if segments B and D were lost and retransmitted..
TS.Recent
<A, TSval=1> ------------------->
1
<---- <ACK(A), TSecr=1>
1
<C, TSval=3> ------------------->
1
<---- <ACK(A), TSecr=1>
1
<B, TSval=2> ------------------->
2
<---- <ACK(C), TSecr=2>
2
<E, TSval=5> ------------------->
2
<---- <ACK(C), TSecr=2>
2
<D, TSval=4> ------------------->
4
<---- <ACK(E), TSecr=4>
(etc)4. PAWS: PROTECT AGAINST WRAPPED SEQUENCE NUMBERS ✨
4.1. Introduction
Section 4.2 describes a simple mechanism for rejecting old duplicate segments that might corrupt an open TCP connection 🛡️ We call it PAWS (Protect Against Wrapped Sequence numbers). PAWS operates within a single TCP connection, using state saved in the connection control block 💜 Section 4.3 and Appendix C talk about what PAWS means for avoiding old duplicates from previous incarnations of the same connection 🎀
4.2. The PAWS Mechanism
PAWS uses the same TCP Timestamps option as the RTTM mechanism we just met, and it assumes every received TCP segment (data and ACK segments included) carries a timestamp SEG.TSval whose values are monotone non-decreasing in time ⏰ The big idea is that a segment can be thrown out as an old duplicate if it arrives with a SEG.TSval that's less than some timestamp recently received on this connection 🗑️
In both PAWS and RTTM, the "timestamps" are 32-bit unsigned integers in a modular 32-bit space. So "less than" is defined the same way it is for TCP sequence numbers, and the same implementation techniques apply 🧮 If s and t are timestamp values, then s < t if 0 < (t - s) < 2**31, computed in unsigned 32-bit arithmetic ✨
The incoming timestamps saved for this comparison MUST guarantee a value that is monotone increasing 📈 We could save the timestamp from the segment that last advanced the left edge of the receive window (the most recent in-sequence segment). But instead we choose TS.Recent from Section 3.4 for the RTTM mechanism, since using a common value for both PAWS and RTTM makes the implementation of both simpler 🧠 As Section 3.4 explained, TS.Recent only differs from the timestamp of the last in-sequence segment in the case of delayed ACKs, and so it's off by less than one window. Either choice protects against sequence number wrap-around 🫶
RTTM was specified symmetrically, so TSval timestamps ride along in both data and ACK segments and get echoed in TSecr fields on returning ACK or data segments 🔁 PAWS submits all incoming segments to the same test, so it protects against duplicate ACK segments as well as data segments 💅 (An alternative un-symmetric algorithm would protect against old duplicate ACKs: the sender of data would reject incoming ACK segments whose TSecr values were less than the TSecr saved from the last segment whose ACK field advanced the left edge of the send window. That algorithm was deemed to lack economy of mechanism and symmetry 🙃)
TSval timestamps sent on {SYN} and {SYN,ACK} segments are used to initialize PAWS 🚀 PAWS protects against old duplicate non-SYN segments, and against duplicate SYN segments received while there is a synchronized connection. Duplicate {SYN} and {SYN,ACK} segments received when there is no connection get discarded by the normal 3-way handshake and sequence number checks of TCP 🧚
It is RECOMMENDED that RST segments NOT carry timestamps, and that RST segments be acceptable regardless of their timestamp 🌷 Old duplicate RST segments should be exceedingly unlikely, and their cleanup function SHOULD take precedence over timestamps 🧹
4.2.1. Basic PAWS Algorithm
The PAWS algorithm requires the following processing to be performed on all incoming segments for a synchronized connection 🎀
R1) If there is a Timestamps option in the arriving segment and SEG.TSval < TS.Recent and if TS.Recent is valid (see later discussion), then treat the arriving segment as not acceptable 🚫
Send an acknowledgement in reply as specified in RFC-793 page 69 and drop the segment 💌
Note: it is necessary to send an ACK segment in order to retain TCP's mechanisms for detecting and recovering from half-open connections. For example, see Figure 10 of RFC-793. 🫶
R2)okay bestie, if the segment is outside the window, reject it. That's normal TCP processing 💅R3)if an arriving segment satisfiesSEG.SEQ <= Last.ACK.sent(see Section 3.4), then record its timestamp inTS.Recent⏰R4)if an arriving segment is in-sequence (i.e., at the left window edge), accept it normally ✨R5)otherwise, treat it as a normal in-window, out-of-sequence TCP segment (e.g., queue it for later delivery to the user) 🎀
Steps R2, R4, and R5 are the normal TCP processing steps specified by RFC-793. Literally just the classic TCP vibes 🫶
It's so important to know that the timestamp only gets checked when a segment first shows up at the receiver. That's true whether it's in-sequence or has to be queued for later delivery 👀 Here's the example, iykyk:
Suppose the segment sequence: A.1, B.1, C.1, ..., Z.1 has
been sent, where the letter indicates the sequence number
and the digit represents the timestamp. Suppose also that
segment B.1 has been lost. The timestamp in TS.TStamp is
1 (from A.1), so C.1, ..., Z.1 are considered acceptable
and are queued. When B is retransmitted as segment B.2
(using the latest timestamp), it fills the hole and causes
all the segments through Z to be acknowledged and passed
to the user. The timestamps of the queued segments are
*not* inspected again at this time, since they have
already been accepted. When B.2 is accepted, TS.Stamp is
set to 2.This rule keeps performance reasonable under loss 🌸 A full window of data is in transit at all times. After a loss, a full window less one packet will show up out-of-sequence and get queued at the receiver (e.g., up to ~2**30 bytes of data). The timestamp option must not result in discarding this data, bestie, that's the whole point 💖
In certain unlikely circumstances, the algorithm of rules R1-R4 could lead to discarding some segments unnecessarily. Check out this example:
Suppose again that segments: A.1, B.1, C.1, ..., Z.1 have
been sent in sequence and that segment B.1 has been lost.
Furthermore, suppose delivery of some of C.1, ... Z.1 is
delayed until AFTER the retransmission B.2 arrives at the
receiver. These delayed segments will be discarded
unnecessarily when they do arrive, since their timestamps
are now out of date.This case is very unlikely to occur 🙈 If the retransmission was triggered by a timeout, then some of the segments C.1, ... Z.1 must have been delayed longer than the RTO time. That's presumably an unlikely event, or there would be a ton of spurious timeouts and retransmissions. If B's retransmission was triggered by the "fast retransmit" algorithm (i.e., by duplicate ACKs), then the queued segments that caused those ACKs must have already been received.
Even if a segment were delayed past the RTO, the Fast Retransmit mechanism [Jacobson90c] will cause the delayed packets to be retransmitted at the same time as B.2. That avoids an extra RTT, so the performance penalty is very small 🦋
We know of no case with a significant probability of occurrence in which timestamps will cause performance degradation by unnecessarily discarding segments. No notes, no drama 💅
4.2.2. Timestamp Clock ⏱️
It's important to understand that the PAWS algorithm does not require clock synchronization between sender and receiver. The sender's timestamp clock is used to stamp the segments, and the sender uses the echoed timestamp to measure RTT's. However, the receiver treats the timestamp as simply a monotone-increasing serial number, without any necessary connection to its clock. From the receiver's viewpoint, the timestamp is acting as a logical extension of the high-order bits of the sequence number 🧚
The receiver algorithm does place some requirements on the frequency of the timestamp clock:
- (a) The timestamp clock must not be "too slow."
It must tick at least once for each 231 bytes sent. In fact, in order to be useful to the sender for round trip timing, the clock should tick at least once per window's worth of data. And even with the RFC-1072 window extension, 231 bytes must be at least two windows.
To make this more quantitative, any clock faster than 1 tick/sec will reject old duplicate segments for link speeds of ~8 Gbps. A 1ms timestamp clock will work at link speeds up to 8 Tbps (8*10**12) bps! 🚀
- (b) The timestamp clock must not be "too fast."
Its recycling time must be greater than MSL seconds. Since the clock (timestamp) is 32 bits and the worst-case MSL is 255 seconds, the maximum acceptable clock frequency is one tick every 59 ns.
However, it is desirable to establish a much longer recycle period. That helps handle outdated timestamps on idle connections (see Section 4.2.3), and it relaxes the MSL requirement for preventing sequence number wrap-around. With a 1 ms timestamp clock, the 32-bit timestamp will wrap its sign bit in 24.8 days. Thus, it will reject old duplicates on the same connection if MSL is 24.8 days or less. This appears to be a very safe figure. An MSL of 24.8 days or longer can probably be assumed by the gateway system without requiring precise MSL enforcement by the TTL value in the IP layer 💜
Based upon these considerations, we choose a timestamp clock frequency in the range 1 ms to 1 sec per tick. This range also matches the requirements of the RTTM mechanism, which doesn't need much more resolution than the granularity of the retransmit timer (e.g., tens or hundreds of milliseconds) 🍓
The PAWS mechanism also puts a strong monotonicity requirement on the sender's timestamp clock. How you implement the timestamp clock to meet this requirement depends on the system hardware and software. Some options:
- Some hosts have a hardware clock that is guaranteed to be monotonic between hardware resets.
- A clock interrupt may be used to simply increment a binary integer by 1 periodically.
- The timestamp clock may be derived from a system clock that is subject to being abruptly changed, by adding a variable offset value. This offset is initialized to zero. When a new timestamp clock value is needed, the offset can be adjusted as necessary to make the new value equal to or larger than the previous value (which was saved for this purpose).
4.2.3. Outdated Timestamps 🕰️
If a connection stays idle long enough for the timestamp clock of the other TCP to wrap its sign bit, then the value saved in TS.Recent becomes too old. As a result, the PAWS mechanism will cause all subsequent segments to be rejected, freezing the connection (until the timestamp clock wraps its sign bit again). Frozen, bestie 🥶
With the chosen range of timestamp clock frequencies (1 sec to 1 ms), the time to wrap the sign bit will be between 24.8 days and 24800 days. A TCP connection that's idle for more than 24 days and then comes back to life is exceedingly unusual. Still, it's undesirable in principle to place any limitation on TCP connection lifetimes 🌈
We therefore require that an implementation of PAWS include a mechanism to "invalidate" the TS.Recent value when a connection is idle for more than 24 days. (An alternative would be to send keepalive segments at a very low rate, but still more often than the wrap-around time for timestamps, e.g., once a day. This would impose negligible overhead. However, the TCP specification has never included keepalives, so the solution based upon invalidation was chosen.) 🫠
Note that a TCP doesn't know the frequency, and therefore the wraparound time, of the other TCP, so it must assume the worst. The validity of TS.Recent needs to be checked only if the basic PAWS timestamp check fails, i.e., only if SEG.TSval < TS.Recent. If TS.Recent is found to be invalid, then the segment is accepted, regardless of the failure of the timestamp check, and rule R3 updates TS.Recent with the TSval from the new segment 👑
To detect how long the connection has been idle, the TCP may update a clock or timestamp value associated with the connection whenever TS.Recent is updated, for example. The details will be implementation-dependent 🛠️
4.2.4. Header Prediction 🏎️
"Header prediction" [Jacobson90a] is a high-performance transport protocol implementation technique that matters most for high-speed links. It optimizes the code for the most common case: receiving a segment correctly and in order. Using header prediction, the receiver asks the question, "Is this segment the next in sequence?" That question can be answered in fewer machine instructions than the question, "Is this segment within the window?" Main character energy for sure ✨
Adding header prediction to our timestamp procedure leads to the following recommended sequence for processing an arriving TCP segment:
H1)Check timestamp (same as step R1 above)H2)Do header prediction: if segment is next in sequence and if there are no special conditions requiring additional processing, accept the segment, record its timestamp, and skip H3.H3)Process the segment normally, as specified in RFC-793. This includes dropping segments that are outside the window and possibly sending acknowledgments, and queueing in-window, out-of-sequence segments.
Another possibility would be to interchange steps H1 and H2. That means performing the header prediction step H2 FIRST, and performing H1 and H3 only when header prediction fails. This could be a performance improvement, since the timestamp check in step H1 is very unlikely to fail, and it requires interval arithmetic on a finite field, which is a relatively expensive operation. Doing this check on every single segment goes against the whole philosophy of header prediction 🙅♀️ We believe this change might reduce CPU time for TCP protocol processing by up to 5-10% on high-speed networks 🔥
However, putting H2 first would create a hazard. A segment from 2**32 bytes in the past might arrive at exactly the wrong time and be accepted mistakenly by the header-prediction step. The following reasoning has been introduced [Jacobson90b] to show that the probability of this failure is negligible 🥹
okay so say every segment is equally likely to show up as an old duplicate 👀. then the chance that an old duplicate lands exactly on the left window edge is the maximum segment size (MSS) divided by the size of the sequence space. this ratio must be less than 2**-16, since MSS must be < 2**16. for example, it'll be (2**12)/(2**32) = 2**-20 for an FDDI link 🌈. but ngl, the older a segment is, the less likely it is to still be floating around the Internet, and under any reasonable model of segment lifetime the probability of an old duplicate exactly at the left window edge must be much smaller than 2**-16 💅.
the 16 bit TCP checksum also allows a basic unreliability of one part in 2**16 🙈. a protocol mechanism whose reliability exceeds the reliability of the TCP checksum should be considered "good enough", i.e., it won't contribute significantly to the overall error rate. so we believe we can ignore the problem of an old duplicate being accepted by doing header prediction before checking the timestamp ✨.
however, this probabilistic argument is not universally accepted, and the consensus at present is that the performance gain does not justify the hazard in the general case. it is therefore recommended that H2 follow H1 🫶.
4.3. Duplicates from Earlier Incarnations of Connection 🦋
the PAWS mechanism protects against errors due to sequence number wrap-around on high-speed connection. segments from an earlier incarnation of the same connection are also a potential cause of old duplicate errors. in both cases, the TCP mechanisms to prevent such errors depend upon the enforcement of a maximum segment lifetime (MSL) by the Internet (IP) layer (see Appendix of RFC-1185 for a detailed discussion). unlike the case of sequence space wrap-around, the MSL required to prevent old duplicate errors from earlier incarnations does not depend upon the transfer rate. if the IP layer enforces the recommended 2 minute MSL of TCP, and if the TCP rules are followed, TCP connections will be safe from earlier incarnations, no matter how high the network speed. thus, the PAWS mechanism is not required for this case 🌸.
we may still ask whether the PAWS mechanism can provide additional security against old duplicates from earlier connections, allowing us to relax the enforcement of MSL by the IP layer. Appendix B explores this question, showing that further assumptions and/or mechanisms are required, beyond those of PAWS. this is not part of the current extension 🧚.
5. CONCLUSIONS AND ACKNOWLEDGMENTS 🎀
this memo presented a set of extensions to TCP to provide efficient operation over large-bandwidth*delay-product paths and reliable operation over very high-speed paths. these extensions are designed to provide compatible interworking with TCP's that do not implement the extensions 🫶.
these mechanisms are implemented using new TCP options for scaled windows and timestamps. the timestamps are used for two distinct mechanisms: RTTM (Round Trip Time Measurement) and PAWS (Protect Against Wrapped Sequences) 💖.
the Window Scale option was originally suggested by Mike St. Johns of USAF/DCA. the present form of the option was suggested by Mike Karels of UC Berkeley in response to a more cumbersome scheme defined by Van Jacobson. Lixia Zhang helped formulate the PAWS mechanism description in RFC-1185 👑.
finally, much of this work originated as the result of discussions within the End-to-End Task Force on the theoretical limitations of transport protocols in general and TCP in particular. more recently, task force members and other on the end2end-interest list have made valuable contributions by pointing out flaws in the algorithms and the documentation. the authors are grateful for all these contributions 🥹💜.
6. REFERENCES 📚
- [Clark87] Clark, D., Lambert, M., and L. Zhang, "NETBLT: A Bulk Data Transfer Protocol", RFC 998, MIT, March 1987.
- [Garlick77] Garlick, L., R. Rom, and J. Postel, "Issues in Reliable Host-to-Host Protocols", Proc. Second Berkeley Workshop on Distributed Data Management and Computer Networks, May 1977.
- [Hamming77] Hamming, R., "Digital Filters", ISBN 0-13-212571-4, Prentice Hall, Englewood Cliffs, N.J., 1977.
- [Cheriton88] Cheriton, D., "VMTP: Versatile Message Transaction Protocol", RFC 1045, Stanford University, February 1988.
- [Jacobson88a] Jacobson, V., "Congestion Avoidance and Control", SIGCOMM '88, Stanford, CA., August 1988.
- [Jacobson88b] Jacobson, V., and R. Braden, "TCP Extensions for Long-Delay Paths", RFC-1072, LBL and USC/Information Sciences Institute, October 1988.
- [Jacobson90a] Jacobson, V., "4BSD Header Prediction", ACM Computer Communication Review, April 1990.
- [Jacobson90b] Jacobson, V., Braden, R., and Zhang, L., "TCP Extension for High-Speed Paths", RFC-1185, LBL and USC/Information Sciences Institute, October 1990.
- [Jacobson90c] Jacobson, V., "Modified TCP congestion avoidance algorithm", Message to end2end-interest mailing list, April 1990.
- [Jain86] Jain, R., "Divergence of Timeout Algorithms for Packet Retransmissions", Proc. Fifth Phoenix Conf. on Comp. and Comm., Scottsdale, Arizona, March 1986.
- [Karn87] Karn, P. and C. Partridge, "Estimating Round-Trip Times in Reliable Transport Protocols", Proc. SIGCOMM '87, Stowe, VT, August 1987.
- [McKenzie89] McKenzie, A., "A Problem with the TCP Big Window Option", RFC 1110, BBN STC, August 1989.
- [Nagle84] Nagle, J., "Congestion Control in IP/TCP Internetworks", RFC 896, FACC, January 1984.
- [NBS85] Colella, R., Aronoff, R., and K. Mills, "Performance Improvements for ISO Transport", Ninth Data Comm Symposium, published in ACM SIGCOMM Comp Comm Review, vol. 15, no. 5, September 1985.
- [Postel81] Postel, J., "Transmission Control Protocol - DARPA Internet Program Protocol Specification", RFC 793, DARPA, September 1981.
- [Velten84] Velten, D., Hinden, R., and J. Sax, "Reliable Data Protocol", RFC 908, BBN, July 1984.
- [Watson81] Watson, R., "Timer-based Mechanisms in Reliable Transport Protocol Connection Management", Computer Networks, Vol. 5, 1981.
- [Zhang86] Zhang, L., "Why TCP Timers Don't Work Well", Proc. SIGCOMM '86, Stowe, Vt., August 1986.
APPENDIX A: IMPLEMENTATION SUGGESTIONS 💅
the following layouts are recommended for sending options on non-SYN segments, to achieve maximum feasible alignment of 32-bit and 64-bit machines 🍓.
+--------+--------+--------+--------+
| NOP | NOP | TSopt | 10 |
+--------+--------+--------+--------+
| TSval timestamp |
+--------+--------+--------+--------+
| TSecr timestamp |
+--------+--------+--------+--------+APPENDIX B: DUPLICATES FROM EARLIER CONNECTION INCARNATIONS 💀✨
okay so bestie, there are two cases we need to look at 👀: (1) a system crashes (and loses its connection state) and then restarts, and (2) the same connection gets closed and reopened while the host keeps its state 🫶. We'll cover each one in the next two sections.
B.1 System Crash with Loss of State
okay so TCP's quiet time of one MSL on startup is what handles losing connection state in a crash/restart 🌙. if you want the full explanation, check out "When to Keep Quiet" in the TCP protocol spec [Postel81] 📖. the MSL needed here does not depend on transfer speed 🚀. the current TCP MSL of 2 minutes seems acceptable as an operational compromise, bc a lot of host systems take that long to boot after a crash 😭.
however, the timestamp option can be used to ease the MSL requirements (or to give extra security against data corruption) 🔐. if timestamps are being used AND the timestamp clock can be guaranteed to be monotonic over a crash/restart, meaning the first value of the sender's timestamp clock after the restart is guaranteed to be greater than the last value before it, then a quiet time is unnecessary 💅.
to totally dispense with the quiet time, the host clock would need to be synced to a time source that stays stable over the crash/restart period, with an accuracy of one timestamp clock tick or better 🕰️. but we can back off from this strict requirement and use approximate clock sync instead 🙌. say the clock is always re-synchronized to within N timestamp clock ticks, and booting (extended with a quiet time, if necessary) takes more than N ticks. that guarantees monotonicity of the timestamps, which can then be used to reject old duplicates even without an enforced MSL 🔥.
B.2 Closing and Reopening a Connection
when a TCP connection closes, a delay of 2*MSL in TIME-WAIT state ties up the socket pair for 4 minutes (see Section 3.5 of [Postel81]. apps built on TCP that close one connection and open a new one (like an FTP data transfer connection using Stream mode) have to pick a brand new socket pair each time 🎀. the TIME-WAIT delay serves two different purposes 👑:
(a) Implement the full-duplex reliable close handshake of TCP.
the proper time to delay the final close step isn't really related to the MSL 🙅♀️. it depends on the RTO for the FIN segments, and therefore on the RTT of the path 🛣️. (it could be argued that the side sending a FIN knows what degree of reliability it needs, so it should be able to determine the length of the TIME-WAIT delay for the FIN's recipient. this could be accomplished with an appropriate TCP option in FIN segments.)
although there is no formal upper-bound on RTT, common network engineering practice makes an RTT greater than 1 minute very unlikely. thus, the 4 minute delay in TIME-WAIT state works satisfactorily to provide a reliable full-duplex TCP close ✅. again, note that this is independent of MSL enforcement and network speed.
the TIME-WAIT state could cause an indirect performance problem if an application needed to repeatedly close one connection and open another at a very high frequency, since the number of available TCP ports on a host is less than 2**16 😩. however, high network speeds are not the major contributor to this problem; the RTT is the limiting factor in how quickly connections can be opened and closed. therefore, this problem will be no worse at high transfer speeds 🙃.
(b) Allow old duplicate segments to expire.
to replace this function of TIME-WAIT state, a mechanism would have to operate across connections 🧐. PAWS is defined strictly within a single connection: the last timestamp, TS.Recent, is kept in the connection control block and discarded when a connection is closed.
an additional mechanism could be added to TCP: a per-host cache of the last timestamp received from any connection 🗂️. this value could then be used in the PAWS mechanism to reject old duplicate segments from earlier incarnations of the connection, if the timestamp clock can be guaranteed to have ticked at least once since the old connection was open. this would require that the TIME-WAIT delay plus the RTT together must be at least one tick of the sender's timestamp clock. such an extension is not part of the proposal of this RFC 🫠.
note that this is a variant on the mechanism proposed by Garlick, Rom, and Postel [Garlick77], which required each host to maintain connection records containing the highest sequence numbers on every connection. using timestamps instead, it's only necessary to keep one quantity per remote host, no matter how many simultaneous connections go to that host 🧠✨.
APPENDIX C: CHANGES FROM RFC-1072, RFC-1185 🦋
the protocol extensions defined in this document differ in several important ways from the ones in RFC-1072 and RFC-1185. here's the tea ☕:
- (a) SACK has been deferred to a later memo.
- (b) The detailed rules for sending timestamp replies (see Section 3.4) differ in important ways. The earlier rules could result in an under-estimate of the RTT in certain cases (packets dropped or out of order) 😬.
- (c) The same value TS.Recent is now shared by the two distinct mechanisms RTTM and PAWS. This simplification became possible because of change (b) 🧩.
- (d) An ambiguity in RFC-1185 was resolved in favor of putting timestamps on ACK as well as data segments. This supports the symmetry of the underlying TCP protocol 🪞.
- (e) The echo and echo reply options of RFC-1072 were combined into a single Timestamps option, to reflect the symmetry and to simplify processing 💖.
- (f) The problem of outdated timestamps on long-idle connections, discussed in Section 4.2.2, was realized and resolved 🙌.
- (g) RFC-1185 recommended that header prediction take precedence over the timestamp check. Based upon some scepticism about the probabilistic arguments given in Section 4.2.4, it was decided to recommend that the timestamp check be performed first 🤔.
- (h) The spec was modified so that the extended options will be sent on
<SYN,ACK>segments only when they are received in the corresponding<SYN>segments. This provides the most conservative possible conditions for interoperation with implementations without the extensions 🛡️.
in addition to these substantive changes, the present RFC attempts to specify the algorithms unambiguously by presenting modifications to the Event Processing rules of RFC-793; see Appendix E 📝.
APPENDIX D: SUMMARY OF NOTATION 📋
the following notation has been used in this document, so here's your cheat sheet 💜:
Options
WSopt: TCP Window Scale Option
TSopt: TCP Timestamps Option
Option Fields
shift.cnt: Window scale byte in WSopt.
TSval: 32-bit Timestamp Value field in TSopt.
TSecr: 32-bit Timestamp Reply field in TSopt.
Option Fields in Current Segment
SEG.TSval: TSval field from TSopt in current segment.
SEG.TSecr: TSecr field from TSopt in current segment.
SEG.WSopt: 8-bit value in WSopt
Clock Values
my.TSclock: Local source of 32-bit timestamp values
my.TSclock.rate: Period of my.TSclock (1 ms to 1 sec).
Per-Connection State Variables
TS.Recent: Latest received Timestamp
Last.ACK.sent: Last ACK field sent
Snd.TS.OK: 1-bit flag
Snd.WS.OK: 1-bit flag
Rcv.Wind.Scale: Receive window scale power
Snd.Wind.Scale: Send window scale powerAPPENDIX E: EVENT PROCESSING 🌟
the event processing rules pick up right where this part leaves off, so stay tuned, bestie 🦄✨
OPEN Call ✨
Okay bestie, so for the OPEN call you pick an initial send sequence number (ISS) and then send a SYN segment that looks like this:
<SEQ=ISS><CTL=SYN><TSval=my.TSclock><WSopt=Rcv.Wind.Scale>...
SEND Call 💅
CLOSED STATE (i.e., TCB does not exist) 🌸
...
LISTEN STATE 🎀
If the foreign socket is specified, then the connection goes from passive to active, and you select an ISS. Send a SYN segment containing the options <TSval=my.TSclock> and <WSopt=Rcv.Wind.Scale>. Set SND.UNA to ISS and SND.NXT to ISS+1. Then enter SYN-SENT state. ...
SYN-SENT STATE 🌷
SYN-RECEIVED STATE 🌷
...
ESTABLISHED STATE 💖
CLOSE-WAIT STATE 💖
Chop the buffer into segments and send them with a piggybacked acknowledgment (the acknowledgment value is RCV.NXT). ...
If the urgent flag is set ...
If the Snd.TS.OK flag is set, then include the TCP Timestamps option <TSval=my.TSclock,TSecr=TS.Recent> in each data segment.
Scale the receive window for transmission in the segment header:
SEG.WND = (SND.WND >> Rcv.Wind.Scale).SEGMENT ARRIVES 🦋
...
If the state is LISTEN then
First check for an RST
...
Second check for an ACK
...
Third check for a SYN
If the SYN bit is set, check the security. If the ...
...
If the SEG.PRC is less than the TCB.PRC then continue.
Check for a Window Scale option (WSopt). If one is found, save SEG.WSopt in Snd.Wind.Scale and set the Snd.WS.OK flag on. Otherwise, set both Snd.Wind.Scale and Rcv.Wind.Scale to zero and clear the Snd.WS.OK flag.
Check for a TSopt option. If one is found, save SEG.TSval in the variable TS.Recent and turn on the Snd.TS.OK bit.
Set RCV.NXT to SEG.SEQ+1, set IRS to SEG.SEQ, and any other control or text should be queued for processing later. ISS should be selected and a SYN segment sent of the form:
<SEQ=ISS><ACK=RCV.NXT><CTL=SYN,ACK>If the Snd.WS.OK bit is on, include a WSopt option <WSopt=Rcv.Wind.Scale> in this segment. If the Snd.TS.OK bit is on, include a TSopt <TSval=my.TSclock,TSecr=TS.Recent> in this segment. Last.ACK.sent is set to RCV.NXT.
SND.NXT is set to ISS+1 and SND.UNA to ISS. The connection state should be changed to SYN-RECEIVED. Note that any other incoming control or data (combined with SYN) will be processed in the SYN-RECEIVED state, but processing of SYN and ACK should not be repeated. If the listen was not fully specified (i.e., the foreign socket was not fully specified), then the unspecified fields should be filled in now.
Fourth, other text or control
...
If the state is SYN-SENT then
First check the ACK bit
...
Fourth check the SYN bit
...
If the SYN bit is on and the security/compartment and precedence are acceptable, then RCV.NXT is set to SEG.SEQ+1, IRS is set to SEG.SEQ, and any acknowledgements on the retransmission queue which are thereby acknowledged should be removed.
Check for a Window Scale option (WSopt). If one is found, save SEG.WSopt in Snd.Wind.Scale. Otherwise, set both Snd.Wind.Scale and Rcv.Wind.Scale to zero.
Check for a TSopt option. If one is found, save SEG.TSval in the variable TS.Recent and turn on the Snd.TS.OK bit in the connection control block. If the ACK bit is set, use my.TSclock - SEG.TSecr as the initial RTT estimate.
If SND.UNA > ISS (our SYN has been ACKed), change the connection state to ESTABLISHED and form an ACK segment:
<SEQ=SND.NXT><ACK=RCV.NXT><CTL=ACK>Then send it. If the Snd.Echo.OK bit is on, include a TSopt option <TSval=my.TSclock,TSecr=TS.Recent> in this ACK segment. Last.ACK.sent is set to RCV.NXT.
Data or controls which were queued for transmission may be included. If there are other controls or text in the segment, then continue processing at the sixth step below, where the URG bit is checked. Otherwise, return.
Otherwise, enter SYN-RECEIVED and form a SYN,ACK segment:
<SEQ=ISS><ACK=RCV.NXT><CTL=SYN,ACK>Then send it. If the Snd.Echo.OK bit is on, include a TSopt option <TSval=my.TSclock,TSecr=TS.Recent> in this segment. If the Snd.WS.OK bit is on, include a WSopt option <WSopt=Rcv.Wind.Scale> in this segment. Last.ACK.sent is set to RCV.NXT.
If there are other controls or text in the segment, queue them for processing after the ESTABLISHED state has been reached, and return.
Fifth, if neither of the SYN or RST bits is set, then drop the segment and return.
Otherwise,
First, check sequence number 🔥
Applies to these states:
- SYN-RECEIVED STATE
- ESTABLISHED STATE
- FIN-WAIT-1 STATE
- FIN-WAIT-2 STATE
- CLOSE-WAIT STATE
- CLOSING STATE
- LAST-ACK STATE
- TIME-WAIT STATE
Segments are processed in sequence. Initial tests on arrival are used to discard old duplicates, but further processing is done in SEG.SEQ order. If a segment's contents straddle the boundary between old and new, only the new parts should be processed.
Rescale the received window field:
TrueWindow = SEG.WND << Snd.Wind.Scale,and use "TrueWindow" in place of SEG.WND in the following steps.
Check whether the segment contains a Timestamps option and whether bit Snd.TS.OK is on. If so:
- If SEG.TSval < TS.Recent, then test whether the connection has been idle for less than 24 days. If both are true, then the segment is not acceptable, so follow the steps below for an unacceptable segment.
- If SEG.SEQ is equal to Last.ACK.sent, then save SEG.ECopt in the variable TS.Recent.
There are four cases for the acceptability test for an incoming segment:
...
If an incoming segment is not acceptable, an acknowledgment should be sent in reply (unless the RST bit is set, in which case drop the segment and return):
<SEQ=SND.NXT><ACK=RCV.NXT><CTL=ACK>Last.ACK.sent is set to SEG.ACK of the acknowledgment. If the Snd.Echo.OK bit is on, include the Timestamps option <TSval=my.TSclock,TSecr=TS.Recent> in this ACK segment. Set Last.ACK.sent to SEG.ACK and send the ACK segment. After sending the acknowledgment, drop the unacceptable segment and return.
...
Fifth check the ACK field.
If the ACK bit is off, drop the segment and return.
If the ACK bit is on:
...
ESTABLISHED STATE
If SND.UNA < SEG.ACK =< SND.NXT then set SND.UNA <- SEG.ACK. Also compute a new estimate of round-trip time. If the Snd.TS.OK bit is on, use my.TSclock - SEG.TSecr. Otherwise, use the elapsed time since the first segment in the retransmission queue was sent. Any segments on the retransmission queue which are thereby entirely acknowledged...
...
Seventh, process the segment text.
Applies to these states:
- ESTABLISHED STATE
- FIN-WAIT-1 STATE
- FIN-WAIT-2 STATE
...
Send an acknowledgment of the form:
<SEQ=SND.NXT><ACK=RCV.NXT><CTL=ACK>If the Snd.TS.OK bit is on, include the Timestamps option <TSval=my.TSclock,TSecr=TS.Recent> in this ACK segment. Set Last.ACK.sent to SEG.ACK of the acknowledgment and send it. This acknowledgment should be piggy-backed on a segment being transmitted if possible without incurring undue delay.
...
Security Considerations 🔥
Security issues are not discussed in this memo. 💀
Authors' Addresses 🎀
Van Jacobson
University of California
Lawrence Berkeley Laboratory
Mail Stop 46A
Berkeley, CA 94720
Phone: (415) 486-6411
EMail: van@CSAM.LBL.GOVBob Braden
University of Southern California
Information Sciences Institute
4676 Admiralty Way
Marina del Rey, CA 90292
Phone: (310) 822-1511
EMail: Braden@ISI.EDUDave Borman
Cray Research
655-E Lone Oak Drive
Eagan, MN 55121
Phone: (612) 683-5571
Email: dab@cray.com📦 where this page comes from
- ietf: rfc1323 1992-05
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms