RFC 1644
T/TCP -- TCP Extensions for Transactions Functional Specification
RFC 1644 🎀 T/TCP: TCP Extensions for Transactions, Functional Specification
Network Working Group 🌸 this one's from ISI, written by R. Braden.
- Request for Comments: 1644 ✨
- Author: R. Braden, ISI 💅
- Category: Experimental 🧪
- Date: July 1994 📅
- Obsoletes / Updates: nothing listed in the header, so it doesn't replace or patch any other RFC 🫶
Status of This Memo
Okay so this memo is an Experimental Protocol for the Internet community 🧪. It's basically asking everyone to talk about it and suggest improvements 💬. It is NOT an Internet Standard, so nobody's forcing you to implement it. And distribution is unlimited, so share it with everyone 🌈.
Abstract
The memo specifies T/TCP, an experimental TCP extension built for efficient transaction-oriented (request/response) service 💖. It's backwards-compatible, and it could fill the gap between regular connection-oriented TCP and the datagram-based UDP 🦋. Iconic gap-filling energy 👑.
This work was supported in part by the National Science Foundation under Grant Number NCR-8922231 🙏.
Table of Contents
1. INTRODUCTION .................................................. 2
2. OVERVIEW ..................................................... 3
2.1 Bypassing the Three-Way Handshake ........................ 4
2.2 Transaction Sequences .................................... 6
2.3 Protocol Correctness ..................................... 8
2.4 Truncating TIME-WAIT State ............................... 12
2.5 Transition to Standard TCP Operation ..................... 14
3. FUNCTIONAL SPECIFICATION ..................................... 17
3.1 Data Structures .......................................... 17
3.2 New TCP Options .......................................... 17
3.3 Connection States ........................................ 19
3.4 T/TCP Processing Rules ................................... 25
3.5 User Interface ........................................... 28
4. IMPLEMENTATION ISSUES ........................................ 30
4.1 RFC-1323 Extensions ...................................... 30
4.2 Minimal Packet Sequence .................................. 31
4.3 RTT Measurement .......................................... 31
4.4 Cache Implementation ..................................... 32
4.5 CPU Performance .......................................... 32
4.6 Pre-SYN Queue ............................................ 33
6. ACKNOWLEDGMENTS .............................................. 34
7. REFERENCES ................................................... 34
APPENDIX A. ALGORITHM SUMMARY ................................... 35
Security Considerations .......................................... 38
Author's Address ................................................. 381. INTRODUCTION ✨
Okay so TCP was designed around the virtual circuit model 🧵, which is all about streaming data. But there's another super common vibe: client-server interaction, where you send a request message and get a response message back 📨. This request/response pattern powers application-layer protocols doing transaction processing or remote procedure calls 🛠️, and it's also used by a bunch of network control and management protocols (think DNS and SNMP) 🔧.
Right now, lots of Internet programs that need request/response communication use UDP 💌. The problem is, when they need transport features like reliable delivery, they have to basically build their own private transport protocol up at the application layer 😭. Not it.
Request/response, or "transaction-oriented", communication has these features:
- (a) The fundamental interaction is a request followed by a response 💖.
- (b) An explicit open or close phase may impose excessive overhead 🙄.
- (c) At-most-once semantics is REQUIRED 🚫. That means a transaction MUST NOT be "replayed" as the result of a duplicate request packet 🙅♀️.
- (d) The minimum transaction latency for a client SHOULD be RTT + SPT, where RTT is the round-trip time and SPT is the server processing time ⏱️.
- (e) In favorable circumstances, a reliable request/response handshake SHOULD be achievable with exactly one packet in each direction 🦋. The dream 🌈.
This memo is about T/TCP, a backwards-compatible extension of TCP that gives you efficient transaction-oriented service alongside regular virtual-circuit service 💅. T/TCP gives you everything in the list above except for (e) 💀. So the minimum exchange for T/TCP is three segments, not one. Still slay though 👑.
In this memo, "transaction" means an elementary request/response packet sequence 📦. It is NOT meant to imply the semantics people usually associate with application-layer transaction processing, like 3-phase commits 🙈. T/TCP can be the transport layer underneath that kind of application-layer service 🧚, but T/TCP's own semantics are limited to transport-layer stuff like reliable, ordered delivery and at-most-once operation 🫶.
An earlier memo [RFC-1379] presented the concepts behind T/TCP 📖. But whether these ideas are actually useful in the real world depends on practical things like implementation complexity and performance 🛠️. So this memo gives a functional specification for one particular embodiment of the ideas in RFC-1379 🎀. Fair warning though, the specific algorithms here are a later evolution than RFC-1379. Appendix A in RFC-1379 explained why truncating TIME-WAIT state was tricky 😵💫. Then experience with an RFC-1379 implementation on a workstation showed that piling up TCB's in TIME-WAIT state is an intolerable problem ⚠️. That necessity led to a simple solution for truncating TIME-WAIT state, which is described in this memo 💜.
Section 2 introduces the T/TCP extensions, and section 3 has the complete T/TCP specification 📚. Section 4 covers some implementation issues, and Appendix A has an algorithmic summary 🧾. This document assumes you're already familiar with the standard TCP specification [STD-007] 🤓.
2. OVERVIEW 🌷
TCP is super symmetric between the two ends of a connection 🪞. T/TCP keeps that symmetry too. For example, T/TCP supports TCP's symmetric simultaneous open from both sides (see Section 2.3 below) 🤝. BUT transaction sequences use T/TCP in a very unsymmetrical way 🫣. So it's handy to use the terms "client host" and "server host" for the host that initiates a connection and the host that responds 🙌.
The goal of T/TCP is to let each transaction, i.e., each request/response sequence, be efficiently done as a single incarnation of a TCP connection 🎯. Standard TCP has two performance problems for transaction-oriented communication, and they are giving major problems 😤. First, a TCP connection is opened with a "3-way handshake", which MUST complete successfully before any data can be transferred 🤝. That handshake adds an extra RTT (round trip time) to the latency of a transaction 🐌.
The second problem is that closing a TCP connection leaves one or both ends in TIME-WAIT state for a time of 2*MSL, where MSL is the maximum segment lifetime (defined to be 120 seconds) ⏳. TIME-WAIT state seriously limits how fast successive transactions can happen between the same (host,port) pair 🚦, because a new incarnation of the connection can't be opened until the TIME-WAIT delay expires 😩. RFC-1379 explained why the alternative approach, using a different user port for each transaction between a pair of hosts, also limits the transaction rate 🙃. Reason one: the 16-bit port space limits the rate to 2**16/240 transactions per second 🔢. Reason two, the more practical one: an excessive amount of kernel space would get eaten up by TCP state blocks sitting in TIME-WAIT state [RFC-1379] 💀.
T/TCP fixes these two performance problems for transactions by (1) bypassing the 3-way handshake (3WHS) and (2) shortening the delay in TIME-WAIT state ✨. Main character energy 👑.
2.1. Bypassing the Three-Way Handshake 🦋
T/TCP introduces a 32-bit incarnation number called a "connection count" (CC) 🔢. It rides along in a TCP option in each segment 📮. Each direction of an open connection gets its own distinct CC value 🫶. And a T/TCP implementation gives monotonically increasing CC values to successive connections it opens, whether actively or passively 📈.
T/TCP uses the monotonic property of CC values in initial <SYN> segments to bypass the 3WHS 🪄. This mechanism is called TCP Accelerated Open (TAO) 💫. Under TAO, a host caches a small amount of state per remote host 🗂️. Specifically, a T/TCP host acting as a server keeps a cache containing the last valid CC value it received from each different client host 💾. If an initial <SYN> segment (a segment with a SYN bit but no ACK bit) from a particular client host carries a CC value larger than the cached value for that host, the monotonic property of CC's means the <SYN> segment MUST be new, so it can be accepted immediately ✅. Otherwise, the server host can't tell whether the <SYN> segment is an old duplicate or just got delivered out of order 🤷♀️. So it runs a normal 3WHS to validate the <SYN> 🔍. TAO is an optimization, and the normal TCP mechanism is the fallback 🛟. Good backup plan.
The CC value in non-<SYN> segments protects against old duplicate segments from earlier incarnations of the same connection (we call these 'antique duplicates' for short) 🏺. For short connections like transactions, these CC values let the TIME-WAIT state delay be safely discussed in Section 2.3 🕰️.
T/TCP defines three new TCP options, each carrying one 32-bit CC value 📦. They're named CC, CC.NEW, and CC.ECHO 🏷️. The CC option is the one that's normally used. CC.NEW and CC.ECHO have special jobs, as follows 👀.
- (a) CC.NEW: Correctness of the TAO mechanism requires that clients generate monotonically increasing CC values for successive connection initiations 📈. These values can come from a simple global counter 🧮. There are certain circumstances (discussed below in Section 2.2) when the client knows monotonicity MAY be violated 😬. In that case, it sends a CC.NEW instead of a CC option in the initial <SYN> segment 📩. When the server receives a CC.NEW, it invalidates its cache entry and does a 3WHS 🧹.
- (b) CC.ECHO: When a server host sends a <SYN,ACK> segment, it echoes the connection count from the initial <SYN> in a CC.ECHO option 🔁. The client host uses that to validate the <SYN,ACK> segment ✅.
Figure 1 shows the TAO mechanism bypassing a 3WHS 🎨. The cached CC values, denoted by cache.CC[host], are shown on each side 👀. The server host compares the new CC value x in segment #1 against x0, its cached value for client host A 🆚. This comparison is called the "TAO test" 🧪. Since x > x0, the <SYN> MUST be new and can be accepted immediately 🚀. So the data in the segment can be delivered to user process B, and the cached value gets updated 💅. If the TAO test fails (x <= x0), the server host would do a normal three-way handshake to validate the <SYN> segment, but the cache would NOT be updated 🙅♀️. Honestly that's the most logical thing ever 🧠.
TCP A (Client) TCP B (Server)
_______________ ______________
cache.CC[A]
V
[ x0 ] #1 --> <SYN, data1, CC=x> --> (TAO test OK (x > x0) =>
data1->user_B and
cache.CC[A]= x; )
[ x ]
#2 <-- <SYN, ACK(data1), data2, CC=y, CC.ECHO=x> <--
(data2->user_A;)
Figure 1. TAO: Three-Way Handshake is Bypassedokay so the CC value x gets echoed back in a CC.ECHO option inside the <SYN,ACK> segment (#2), and the client side uses that echo to validate the segment 🫶 once segment #2 passes, its data2 gets delivered to the client user process ✨. segment #2 also carries B's CC value, and A uses that to validate non-SYN segments from B. we'll get into that in Section 2.4 💅
implementing the T/TCP extensions means the connection control block (TCB) gets two new CC values per connection, called TCB.CCsend and TCB.CCrecv (or CCsend and CCrecv for short) 🎀. in the sequence from Figure 1, host A ends up with TCB.CCsend = x and TCB.CCrecv = y, and host B has it the other way around 🦋. any segment that shows up with a CC option whose SEG.CC value is different from TCB.CCsend gets rejected as an antique duplicate 💀 no thanks, bestie.
2.2 Transaction Sequences ✨
T/TCP runs the TAO mechanism from the last section to handle a whole transaction sequence 💖. Figure 2 shows the minimal transaction, where the request and response data each fit into a single segment. it takes three segments and finishes in one round-trip time (RTT) 🚀. if the TAO test fails on segment #1, B queues data1 and the FIN for later, then sends a <SYN,ACK> segment back to A so they can do a normal 3WHS. not a disaster, just slower 🙈
TCP A (Client) TCP B (Server)
_______________ ______________
CLOSED LISTEN
#1 SYN-SENT* --> <SYN,data1,FIN,CC=x> --> CLOSE-WAIT*
(TAO test OK)
(data1->user_B)
<-- LAST-ACK*
#2 TIME-WAIT <-- <SYN,ACK(FIN),data2,FIN,CC=y,CC.ECHO=x>
(data2->user_A)
#3 TIME-WAIT --> <ACK(FIN),CC=x> --> CLOSED
(timeout)
CLOSED
Figure 2: Minimal T/TCP Transaction SequenceT/TCP needs some extra connection states, like the SYN-SENT*, CLOSE-WAIT*, and LAST-ACK* states you can see in Figure 2 🌈 Section 3.3 has the details on these new states, so go check it out 🧚
to get that minimal 3-segment sequence from Figure 2, the server host has to hold off on acknowledging segment #1 so the response can ride along on segment #2 🍓. if the application takes longer than that delay to come up with the response, TCP B's normal retransmission mechanism sends an acknowledgment anyway, which stops TCP A from retransmitting 😭. Figure 3 shows what happens with a slow server application. the sequence in Figure 3 still has a 3-way handshake, but the TAO mechanism lets the request data get accepted right away, so the client still gets minimum latency ✨ the TAO magic is real, periodt 👑
TCP A (Client) TCP B (Server)
_______________ ______________
CLOSED LISTEN
#1 SYN-SENT* --> <SYN,data1,FIN,CC=x> --> CLOSE-WAIT*
(TAO test OK =>
data1->user_B)
(timeout)
#2 FIN-WAIT-1 <-- <SYN,ACK(FIN),CC=y,CC.ECHO=x> <-- CLOSE-WAIT*
#3 FIN-WAIT-1 --> <ACK(SYN),FIN,CC=x> --> CLOSE-WAIT
#4 TIME-WAIT <-- <ACK(FIN),data2,FIN,CC=y> <-- LAST-ACK
(data2->user_A)
#5 TIME_WAIT --> <ACK(FIN),CC=x> --> CLOSED
(timeout)
CLOSED
Figure 3: Acknowledgment Timeout in Server2.3 Protocol Correctness ✨
this section fills in more of the TAO mechanism and gives an informal sketch of why T/TCP works 🧠 no cap, this is the part that makes it all click 💜
CC values are 32-bit integers 🔢. the TAO test uses the same kind of modular arithmetic that TCP uses to compare sequence numbers. we assume the boundary between y < z and z < y for two CC values y and z happens when they differ by 2**31, meaning half the total CC space 🌸
the essential requirement for T/TCP correctness is this 👇
CC values must advance at a rate slower than 2**31 [R1]
counts per 2*MSLMSL means the maximum segment lifetime on the Internet 🌐. [R1] is super easy to meet with a 32-bit CC. for example, it allows 10**6 transactions per second with the very generous MSL of 1000 seconds [RFC-1379] 🤯 that is way more than current operating systems and network latency can actually hit, so we're good 💅
for now, assume that successive connections from client A to server B only carry monotonically increasing CC values 📈. so if x(i) and x(i+1) are CC values in two successive initial <SYN> segments from the same host, then x(i+1) > x(i) ✨. given [R1], the CC space can't wrap around within the range of segments that could be outstanding at one time 🙌. that means the <SYN> segments from a given host that haven't passed their MSL must have an ordered set of CC values:
x(1) < x(2) < x(3) ... < x(n),where the modular comparisons get swapped out for plain arithmetic comparisons 🧮. x(n) is the most recent acceptable <SYN>, and the server caches it 📦. if the server host gets a <SYN> with a CC option value y where y > x(n), that <SYN> has to be newer, because an antique duplicate SYN with a CC value greater than x(n) would have exceeded its MSL and vanished already 👻. so monotonic CC values plus the TAO test stop antique <SYN>s from being replayed by mistake 🛡️ iconic behaviour.
there are two ways a client can end up generating non-monotonic CC values 😬: (a) the client crashed and restarted, so the generated CC values jump backwards, or (b) the generated CC values wrapped around the finite space. wraparound can happen because CC generation is global across all connections 🌀. say host A sends a transaction to B, then sends more than 2**31 transactions to other hosts, and then sends another transaction to B. from B's point of view, CC just jumped backwards compared to its cached value 🙃
in either case, the server only sees the CC value jump backwards after at least one MSL has passed since the last <SYN> segment from that same client host ⏳. in case (a), a client host restart, this happens because T/TCP keeps TCP's explicit "Quiet Time" of one MSL interval [STD-007]. in case (b), wraparound, [R1] makes sure at least one MSL has to pass before the CC space wraps around 🧘. so there's no way a TAO test succeeds by mistake because of either kind of non-monotonicity. put simply, there's no chance of replays due to TAO 🫶
now, CC values jumping backwards won't cause an error, but they can hurt performance because of unnecessary 3WHS's 😤. that happens because the generated CC values jump backwards through about half their range, so every TAO test after that fails until the generated values catch back up to the cached value 🐢. to avoid this slowdown, a client host sends a CC.NEW option instead of a CC option when it restarts or the CC values wrap around 💫. receiving CC.NEW forces a 3WHS, but once that 3WHS completes successfully, the server cache gets updated to the new CC value ✨. to detect wraparound, the client has to cache the last CC value it sent to each server, so it keeps cache.CCsent[B] for each server B 📝. if that cached value is undefined, or if it's larger than the next CC value the client generates, the client sends a CC.NEW instead of a CC option in the next SYN segment 💅
Figure 4 shows the scenario for the first transaction from A to B after client host A crashed and recovered 🌷. a similar sequence happens if x isn't greater than cache.CCsent[B], meaning the generated CC values wrapped around. because segment #1 has a CC.NEW option, the server host invalidates the cache entry and does a 3WHS. it still sets B's TCB.CCrecv for this connection to x, though 🎯. TCP B uses this CCrecv value to validate the <ACK> segment (#3) that completes the 3WHS. receiving this segment updates cache.CC[A], since the entry was undefined before 🌟. (if a 3WHS always updated the cache, out-of-order SYN segments could make the cached value jump backwards, which could allow replays 😱.) finally, the CC.ECHO option in the <SYN,ACK> segment #2 defines A's cache.CCsent entry 🩷
this algorithm waits to update cache.CCsent[] until the <SYN> has been ACK'd 🕰️. that lets the undefined cache.CCsent value act as a "first-time switch" for reliable resynchronization of the cached value at the server after a crash or wraparound 🔁
when we say "cache", we mean the value can be thrown away at any time without causing erroneous behaviour, though it might slow things down 🗑️
(a) if a server host receives an initial <SYN> from client A but has no cached value cache.CC[A], the server just forces a 3WHS to validate the <SYN> segment 🤝
(b) if a client host has no cached value cache.CCsent[B] when it needs to send an initial <SYN> segment, it simply sends a CC.NEW option in the segment. this forces a 3WHS at the server 💜
TCP A (Client) TCP B (Server)
_______________ ______________
cache.CCsent[B] cache.CC[A]
V V
(Crash and restart)
[ ?? ] [ x0 ]
#1 --> <SYN, data1,CC.NEW=x> --> (invalidate cache;
queue data1;
3-way handshake)
[ ?? ] [ ?? ]
#2 <-- <SYN, ACK(data1),CC=y,CC.ECHO=x> <--
(cache.CCsent[B]= x;)
[ x ] [ ?? ]
#3 --> <ACK(SYN),CC=x> --> data1->user_B;
cache.CC[A]= x;
[ x ] [ x ]
Figure 4. Client Host RestartingOkay so, we've only checked that the TAO trick is correct for skipping the 3WHS so far 💅 But we also have to protect a connection from antique duplicate non-SYN segments (old packets that wander in super late and ghost you 👻). In standard TCP, that protection is one of the jobs of the TIME-WAIT state delay. (The other job is the TCP full-duplex close semantics, which we need to keep. More on that in Section 2.5.) To get a high rate of transaction processing, we have to be able to cut that TIME-WAIT delay short without being exposed to antique duplicate segments [RFC-1379] ✨
For short connections (like transactions), the CC values on each direction can guard against antique duplicate non-SYN segments 🎀 Here "short" means lasting less than MSL. Say a connection uses the CC values TCB.CCsend = x and TCB.CCrecv = y. By the requirement [R1], neither x nor y can be reused for a new connection from the same remote host for at least 2*MSL. If the connection has been around for less than MSL, its CC values won't be reused for longer than MSL, so every antique duplicate carrying that CC value has to vanish before it's reused. So for "short" connections, we can guard against antique non-SYN segments by just checking the CC value in the segment against TCB.CCrecv. Note that this check doesn't use the monotonic property of the CC values, only the fact that they don't cycle in less than 2*MSL. And again, the quiet time at system restart protects against errors caused by a crash that loses state 🫶
If the connection lasts longer than MSL, safety from old duplicates still needs a 2*MSL TIME-WAIT delay 🙈 So truncating TIME-WAIT is only possible for short connections. (Shankar and Lee [ShankarLee93] noticed this problem too 🌸) This difference between long and short connections makes a slightly complex service model for applications using T/TCP. An application has two strategies for multiple connections. For "short" connections, it should use a fixed port pair and the T/TCP mechanism to get rapid, efficient transaction processing. For connections whose durations are on the order of MSL or longer, it should use a different user port for each successive connection, like unmodified TCP does today. The latter strategy causes excessive overhead (from TCBs in TIME-WAIT state) if it's applied to high-frequency short connections. If an app picks the wrong one, its attempt to open a new connection may fail with a "busy" error 💀 If connection durations may range between long and short, an application may have to be able to switch strategies when one fails.
2.4. Truncating TIME-WAIT State ✨
Truncation of TIME-WAIT state is necessary for high transaction rates 🚀 As Figure 2 shows, a standard transaction leaves the client end of the connection in TIME-WAIT state. This section explains the protocol implications of truncating TIME-WAIT state, and when it's allowed (that is, when the connection has been in existence for less than MSL). In this case, the client host should be able to interrupt TIME-WAIT state to start a new incarnation of the same connection (same host and ports, obviously). This will send an initial <SYN> segment 🦋
It's possible for the new <SYN> to arrive at the server before the retransmission state from the previous incarnation is gone, as shown in Figure 5. Here the final <ACK> (segment #3) from the previous incarnation is lost, leaving retransmission state at B. However, the client received segment #2 and thinks the transaction completed successfully, so it can start a new transaction by sending <SYN> segment #4. When this <SYN> arrives at the server host, it must implicitly acknowledge segment #2, signal success to the server application, delete the old TCB, and create a new TCB, as shown in Figure 5. Still assuming the new <SYN> is known to be valid, the server host marks the new connection half-synchronized and delivers data3 to the server application. (The details of how this is accomplished are presented in Section 3.3.) 👑
The earlier discussion of the TAO mechanism assumed the previous incarnation was closed before a new <SYN> arrived at the server. However, TAO can't be used to validate the <SYN> if there's still state from the previous incarnation, as shown in Figure 5. In that case, it would be exceedingly awkward to do a 3WHS if the TAO test failed. Luckily, a modified version of the TAO test can still be performed, using the state in the earlier TCB instead of the cached state 🧚
(A) If the <SYN> segment contains a CC or CC.NEW option, the value SEG.CC from this option is compared with TCB.CCrecv, the CC value in the still-existing state block of the previous incarnation. If SEG.CC > TCB.CCrecv, the new <SYN> segment must be valid.
(B) Otherwise, the <SYN> is an old duplicate and is simply discarded.
Truncating TIME-WAIT state may be looked at as composing an extended state machine that joins the state machines of the two incarnations, old and new. It may be described by introducing new intermediate states (which we call I-states), with transitions that join the two diagrams and share some state from each. I-states are detailed in Section 3.3. 🦄
Notice also segment #2' in Figure 5. TCP's mechanism for recovering from half-open connections (see Figure 10 of [STD-007]) causes TCP A to send a RST when 2' arrives, which would incorrectly make B think the previous transaction didn't complete successfully. The half-open recovery mechanism must be defeated in this case, by A ignoring segment #2'. 🙅♀️
TCP A (Client) TCP B (Server)
_______________ ______________
CLOSED LISTEN
#1 --> <...,FIN,CC=x> --> LAST-ACK*
#2 <-- <...ACK(FIN),data2,FIN,CC=y,CC.ECHO=x> <--- LAST-ACK*
TIME-WAIT
(data2->user_A)
#3 TIME-WAIT --> <ACK(FIN),CC=x> --> X (DROP)
(New Active Open) (New Passive Open)
#4 SYN-SENT* --> <SYN, data3,CC=z> ...
LISTEN-LA
#2' (discard) <-- <...ACK(FIN),data2,FIN,CC=y> <--- (retransmit)
#4 SYN-SENT* ... <SYN,data3,CC=z> --> ESTABLISHED*
SYN OK (see text) =>
{Ack seg #2;
Delete old TCB;
Create new TCB;
data3 -> user_B;
cache.CC[A]= z;}Figure 5: Truncating TIME-WAIT State: SYN as Implicit ACK
2.5. Transition to Standard TCP Operation
T/TCP includes all normal TCP semantics, and it keeps working exactly like TCP when the particular assumptions for transactions don't hold 🌈 There's no limit on the size of an individual transaction, and T/TCP should merge seamlessly from pure transaction operation (as shown in Figure 2) to pure streaming mode for sending large files. All the sequences shown in [STD-007] are still valid, and the inherent symmetry of TCP is preserved 💜
Figure 6 shows a possible sequence when the request and response messages each need two segments. Segment #2 is a non-SYN segment that contains a TCP option. To avoid compatibility problems with existing TCP implementations, the client side should send segment #2 only if cache.CCsent[B] is defined, that is, only if host A knows that host B plays the new game 🎮
TCP A (Client) TCP B (Server)
_______________ ______________
CLOSED LISTEN
#1 SYN-SENT* --> <SYN,data1,CC=x> --> ESTABLISHED*
(TAO test OK =>
data1-> user)
#2 SYN-SENT* --> <data2,FIN,CC=x> --> CLOSE-WAIT*
(data2-> user)
CLOSE-WAIT*
#3 FIN-WAIT-2 <-- <SYN,ACK(FIN),data3,CC=y,CC.ECHO=x> <--
(data3->user)
#4 TIME_WAIT <-- <ACK(FIN),data4,FIN,CC=y> <-- LAST-ACK*
(data4->user)
#5 TIME-WAIT --> <ACK(FIN),CC=x> --> CLOSEDFigure 6. Multi-Packet Request/Response Sequence
Figure 7 shows a more complex example: one possible sequence with TAO combined with simultaneous open and close. This can be compared with Figure 8 of [STD-007] 🔥
TCP A TCP B
_______________ ______________
CLOSED CLOSED
#1 SYN-SENT* --> <SYN,data1,FIN,CC=x> ...
#2 CLOSING* <-- <SYN,data2,FIN,CC=y> <-- SYN-SENT*
(TAO test OK =>
data2->user_A
#3 CLOSING* --> <FIN,ACK(FIN),CC=x,CC.ECHO=y> ...
#1' ... <SYN,data1,FIN,CC=x> --> CLOSING*
(TAO test OK =>
data1->user_B)
#4 TIME-WAIT <-- <FIN,ACK(FIN),CC=y,CC.ECHO=x> <-- CLOSING*
#5 TIME-WAIT --> <ACK(FIN),CC=x> ...
#3' ... <FIN,ACK(FIN),CC=x,CC.ECHO=y> --> TIME-WAIT
#6 TIME-WAIT <-- <ACK(FIN),CC=y> <--- TIME-WAIT
#5' TIME-WAIT ... <ACK(FIN),CC=x> --> TIME-WAIT
(timeout) (timeout)
CLOSED CLOSEDFigure 7: Simultaneous Open and Close
3. FUNCTIONAL SPECIFICATION ✨
3.1. Data Structures
A connection count is an unsigned 32-bit integer, with the value zero excluded. Zero is used to denote an undefined value 🍓
A host maintains a global connection count variable CCgen, and each connection control block (TCB) contains two new connection count variables, TCB.CCsend and TCB.CCrecv. Whenever a TCB is created for the active or passive end of a new connection, CCgen is incremented by 1 and placed in TCB.CCsend of the TCB. However, if the previous CCgen value was 0xffffffff (-1), then the next value should be 1. TCB.CCrecv is initialized to zero (undefined) 💖
okay so T/TCP is giving the per-host cache realness 🗂️✨ each entry for a foreign host fh holds two CC values, cache.CC[fh] and cache.CCsent[fh], and it can hold other stuff too (more on that in Sections 4.3 and 4.4 💅). now the MSS drama: per [STD-007], a TCP is not permitted to send a segment bigger than the default size of 536 unless it got a bigger value in an MSS (Maximum Segment Size) option 😭. so the client could be stuck on 536 bytes for every single request, which is so not iconic. the fix is that a T/TCP may cache the MSS option values it got from remote hosts, and a TCP is allowed to use a cached MSS value for the initial SYN segment 🎀
when the client sends an initial <SYN> segment with data, it doesn't have a send window for the server yet 🙈 not a big deal though! we just define a default initial window, and our current suggestion is 4K 🌸 that non-zero default should be conditioned on having a cached connection count for the foreign host, so data only rides along on the initial SYN if cache.CC[foreign host] is non-zero. bestie, that's the gatekeeping 💅
in TCP the window gets adjusted dynamically for congestion control and avoidance [Jacobson88] 🦋 but a particular path might not be able to take an initial burst of 4096 bytes without congestive losses. if that turns out to be a problem, it should be possible to cache the congestion threshold for the path and use that value to figure out the max size of the initial packet burst a request creates ✨
3.2. New TCP Options
three new TCP options are defined: CC, CC.NEW, and CC.ECHO 🌈 each one carries a connection count SEG.CC. the complete rules for sending and processing them are in Section 3.4 below, so hold that thought 🧚
CC Option
- Kind:
11 - Length:
6
+--------+--------+--------+--------+--------+--------+
|00001011|00000110| Connection Count: SEG.CC |
+--------+--------+--------+--------+--------+--------+
Kind=11 Length=6this option may be sent in an initial SYN segment, and it may be sent in other segments if a CC or CC.NEW option has been received for this incarnation of the connection 🍓 its SEG.CC value is the TCB.CCsend value from the sender's TCB.
CC.NEW Option
- Kind:
12 - Length:
6
+--------+--------+--------+--------+--------+--------+
|00001100|00000110| Connection Count: SEG.CC |
+--------+--------+--------+--------+--------+--------+
Kind=12 Length=6this option may be sent instead of a CC option in an initial <SYN> segment (i.e., SYN but not ACK bit), to indicate that the SEG.CC value may not be larger than the previous value 🔥 its SEG.CC value is the TCB.CCsend value from the sender's TCB.
CC.ECHO Option
- Kind:
13 - Length:
6
+--------+--------+--------+--------+--------+--------+
|00001101|00000110| Connection Count: SEG.CC |
+--------+--------+--------+--------+--------+--------+
Kind=13 Length=6this option must be sent (in addition to a CC option) in a segment containing both a SYN and an ACK bit, if the initial SYN segment contained a CC or CC.NEW option 😭 its SEG.CC value is the SEG.CC value from the initial SYN.
a CC.ECHO option should be sent only in a <SYN,ACK> segment, and it should be ignored if it shows up in any other segment 🙈
3.3. Connection States
T/TCP needs new connection states and state transitions 🦋 figure 8 shows the resulting finite state machine, and see [RFC-1379] for a detailed development. if you remove all the state names ending in stars from Figure 8, the diagram reduces to the standard TCP state machine (see Figure 6 of [STD-007]), with two exceptions:
- STD-007 shows a direct transition from SYN-RECEIVED to FIN-WAIT-1 state when the user issues a CLOSE call. this transition is kinda suspect 💀 a more accurate state machine would seem to need the intermediate SYN-RECEIVED* state shown in Figure 8.
- in STD-007, a user CLOSE call in SYN-SENT state causes a direct transition to CLOSED state. the extended diagram of Figure 8 forces the connection to open before it closes, since calling CLOSE to end the request in SYN-SENT state is totally normal behaviour for a transaction client 💅 in the case that no data has been sent in SYN-SENT state, it is reasonable for a user CLOSE call to go straight into CLOSED state and delete the TCB.
each of the new states in Figure 8 has a starred name, made by slapping a star onto a standard TCP state ⭐ each starred state has a simple relationship with its unstarred buddy:
- SYN-SENT* and SYN-RECEIVED* differ from the SYN-SENT and SYN-RECEIVED states, respectively, in recording the fact that a FIN needs to be sent.
- the other starred states indicate that the connection is half-synchronized (hence, a SYN bit needs to be sent).
________ g ________
| |<------------| |
| CLOSED |------------>| LISTEN |
|________| h ------|________|
| / | |
| / i| j|
| / | |
a| a'/ | _V______ ________
| / j | |ESTAB- | e' | CLOSE- |
| / -----------|-->| LISHED*|------------>| WAIT*|
| / / | |________| |________|
| / / | | | | |
| / / | | c| d'| c|
____V_V_ / _______V | __V_____ | __V_____
| SYN- | b' | SYN- |c | |ESTAB- | e | | CLOSE- |
| SENT |------>|RECEIVED|---|->| LISHED|----------|->| WAIT |
|________| |________| | |________| | |________|
| | | | | |
| | | | __V_____ |
| | | | | LAST- | |
d'| d'| d'| d| | ACK* | |
| | | | |________| |
| | | | | |
| | | | ________ |c' |d
| k | | FIN- | | e''' | | | |
| -------|-->| WAIT-1*|---|------>|CLOSING*| | |
| / | |________| | |________| | |
| / | | | | | |
| / | c'| | c'| | |
___V___ / ____V___ V_____V_ ____V___ V____V__
| SYN- | b'' | SYN- | c | FIN- | e'' | | | LAST- |
| SENT* |---->|RECEIVD*|---->| WAIT-1 |---->|CLOSING | | ACK |
|________| |________| |________| |________| |________|
| | |
f| f| f'|
___V____ ____V___ ___V____
| FIN- | e |TIME- | T | |
| WAIT-2 |---->| WAIT |-->| CLOSED |
|________| |________| |________|
Figure 8A: Basic T/TCP State Diagram ________________________________________________________________
| |
| Label Event / Action |
| _____ ________________________ |
| |
| a Active OPEN / create TCB, snd SYN |
| a' Active OPEN / snd SYN |
| b rcv SYN [no TAO]/ snd ACK(SYN) |
| b' rcv SYN [no TAO]/ snd SYN,ACK(SYN) |
| b'' rcv SYN [no TAO]/ snd SYN,FIN,ACK(SYN) |
| c rcv ACK(SYN) / |
| c' rcv ACK(SYN) / snd FIN |
| d CLOSE / snd FIN |
| d' CLOSE / snd SYN,FIN |
| e rcv FIN / snd ACK(FIN) |
| e' rcv FIN / snd SYN,ACK(FIN) |
| e'' rcv FIN / snd FIN,ACK(FIN) |
| e''' rcv FIN / snd SYN,FIN,ACK(FIN) |
| f rcv ACK(FIN) / |
| f' rcv ACK(FIN) / delete TCB |
| g CLOSE / delete TCB |
| h passive OPEN / create TCB |
| i (= b') rcv SYN [no TAO]/ snd SYN,ACK(SYN) |
| j rcv SYN [TAO OK] / snd SYN,ACK(SYN) |
| k rcv SYN [TAO OK] / snd SYN,FIN,ACK(SYN) |
| T timeout=2MSL / delete TCB |
| |
| |
| Figure 8B. Definition of State Transitions |
|________________________________________________________________|this simple correspondence leads to an alternative state model, which makes it easy to slot the new states into an existing implementation 🧩 each state in the extended FSM is defined by the triplet:
(old_state, SENDSYN, SENDFIN)where old_state is a standard TCP state, and SENDFIN and SENDSYN are Boolean flags (see Figure 9) 💜 the SENDFIN flag gets turned on (on the client side) by a SEND(... EOF=YES) call, to indicate that a FIN should be sent in a state that wouldn't otherwise send a FIN ✨ the SENDSYN flag gets turned on when the TAO test succeeds, to indicate that the connection is only half synchronized, so a SYN will be sent in a state that wouldn't otherwise send a SYN 🫶
Okay bestie, this is where the SENDFIN and SENDSYN flags come in, and they're lowkey the stars of the show 💅
________________________________________________________________
| |
| New state: Old_state: SENDSYN: SENDFIN: |
| __________ __________ ______ ______ |
| |
| SYN-SENT* => SYN-SENT FALSE TRUE |
| |
| SYN-RECEIVED* => SYN-RECEIVED FALSE TRUE |
| |
| ESTABLISHED* => ESTABLISHED TRUE FALSE |
| |
| CLOSE-WAIT* => CLOSE-WAIT TRUE FALSE |
| |
| LAST-ACK* => LAST-ACK TRUE FALSE |
| |
| FIN-WAIT-1* => FIN-WAIT-1 TRUE FALSE |
| |
| CLOSING* => CLOSING TRUE FALSE |
| |
| |
| Figure 9: Alternative State Definitions |
|________________________________________________________________|Here's a more complete rundown of these boolean variables, because they're doing a lot of work 🧚
SENDFIN 🎀
SENDFIN gets switched on by the SEND(...EOF=YES) call, and it gets switched back off when FIN-WAIT-1 is entered. It can only be on in the SYN-SENT* and SYN-RECEIVED* states.
SENDFIN has two effects. First, it makes a FIN get sent on the last segment of data from the user. Second, it makes the SYN-SENT[*] and SYN-RECEIVED[*] states move straight to FIN-WAIT-1, skipping ESTABLISHED state. Main character skip 🦋
SENDSYN 🌸
The SENDSYN flag gets turned on when an initial SYN segment comes in and passes the TAO test. It gets turned off when the SYN is acknowledged, which specifically means there's no RST or SYN bit and SEG.UNA < SND.ACK.
SENDSYN has three effects. First, it makes the SYN bit get set in segments sent with the initial sequence number (ISN). Second, it causes a direct transition from LISTEN state to ESTABLISHED* if there's no FIN bit, or to CLOSE-WAIT* otherwise. Finally, it lets data get received and processed (passed up to the application) even when the segment doesn't have an ACK bit. Iconic behaviour 👑
Okay so now the state model. According to the state model of the basic TCP specification [STD-007], the server side must explicitly issue a passive OPEN call, which creates a TCB in LISTEN state, before an initial SYN can be accepted. To make TIME-WAIT truncation fit inside this model, we need to add five "I-states" 🫶 They're shown in Figure 10: LISTEN-LA, LISTEN-LA*, LISTEN-CL, LISTEN-CL*, and LISTEN-TW.
These are "bridge states" between the state diagrams of two successive incarnations. Here D is the duration of the previous connection, which means the elapsed time since that connection opened. The transitions labelled with lower-case letters are taken from Figure 8.
______________________________
| P: Passive OPEN / |
| |
| Q: Rcv SYN, special TAO test | d'| d|
| (see text) / Delete TCB, | ________ ___V____ |
| create TCB, snd SYN | |LISTEN- | P | LAST- | |
| | | LA* |<-----| ACK* | |
| Q': (same as Q) if D < MSL | |________| |________| |
| | | | | |
| R: Rcv ACK(FIN) / Delete TCB,| Q| c'| c'| |
| create TCB | | | | |
| | | ___V____ V______V
| S': Active OPEN if D < MSL / | | |LISTEN- | P | LAST- |
| Delete TCB, create TCB, | | | LA |<-----| ACK |
| snd SYN. | | |________| |________|
|______________________________| | | | |
| Q| R| f|
________ ________ | | | |
e''' | | P |LISTEN- | | | V V
---->|CLOSING*|----->| CL* | | | LISTEN CLOSED
|________| |________| | |
| | Q| | |
c'| c'| V V V
| | ESTABLISHED*
____V___ V_______
e'' | | P |LISTEN- |
---->|CLOSING |------>| CL |
|________| |________|
| R| Q|
f| V V
| LISTEN ESTABLISHED*
____V___ _________
e |TIME- | P | LISTEN- |
---->| WAIT |------------->| TW |
|________| |_________|
/ | | | |
S'/ T| T| Q'| |S'
| _____V_ h _____V__ | V
| | |-------->| | | SYN-SENT
| | CLOSED |<--------| LISTEN | |
| |________| ------|________| |
| | / | j| |
| a| a'/ i| V V
| | / | ESTABLISHED*
V V V V
SYN-SENT ...
Figure 10: I-States for TIME-WAIT TruncationFortunately, lots of TCP implementations use a different user interface model, where the user can issue a generic passive open ("listen") call. After that, whenever a matching initial SYN arrives, a new TCB in LISTEN state gets created automatically. With this user model, the I-states of Figure 10 aren't needed at all, which is honestly such a relief 🙌
Let's look at an example. Say an initial SYN segment arrives for a connection that's in LAST-ACK state. If this segment carries a CC option and SEG.CC is greater than TCB.CCrecv in the existing TCB, then the "q" transition from Figure 10 can happen directly from LAST-ACK. The previous TCB gets processed as if an ACK(FIN) had arrived, which tells the user that the CLOSE succeeded and deletes the TCB. Then the new SYN segment gets processed again, this time against a new TCB that's created automatically. The same idea can be used to avoid implementing any of the I-states 💖
3.4. T/TCP Processing Rules ✨
This section sums up the rules for sending and processing the T/TCP options. Let's go 🚀
INITIALIZATION
I1: All cache entries cache.CC[*] and cache.CCsent[*] are undefined (zero) when a host system initializes, and CCgen is set to a non-zero value.
I2: A new TCB gets initialized with TCB.CCrecv = 0 and TCB.CCsend set to the current CCgen value. Then CCgen is incremented. If the result is zero, CCgen is incremented again.
SENDING SEGMENTS
S1: Sending initial <SYN> segment
An initial <SYN> segment is sent with either a CC option or a CC.NEW option. If cache.CCsent[fh] is undefined, or if TCB.CCsend < cache.CCsent[fh], then the option CC.NEW(TCB.CCsend) is sent and cache.CCsent[fh] is set to zero. Otherwise, the option CC(TCB.CCsend) is sent and cache.CCsent[fh] is set to CCsend.
S2: Sending <SYN,ACK> segment
If the sender's TCB.CCrecv is non-zero, then a <SYN,ACK> segment is sent with both a CC(TCB.CCsend) option and a CC.ECHO (TCB.CCrecv) option.
S3: Sending non-SYN segment
A non-SYN segment is sent with a CC(TCB.CCsend) option if the TCB.CCrecv value is non-zero, or if the state is SYN-SENT or SYN-SENT* and cache.CCsent[fh] is non-zero. That last condition is required so CC options get sent in the segments after the first one of a multi-segment request message (check out segment #2 in Figure 6, it's a whole vibe 🍓).
RECEIVING INITIAL <SYN> SEGMENT
Suppose a server host receives a segment containing a SYN bit but no ACK bit, in LISTEN, SYN-SENT, or SYN-SENT* state.
R1.1: If the <SYN> segment contains a CC or CC.NEW option, then SEG.CC is stored into TCB.CCrecv of the new TCB.
R1.2: If the segment contains a CC option, and the local cache entry cache.CC[fh] is defined, and SEG.CC > cache.CC[fh], then the TAO test is passed and the connection is half-synchronized in the incoming direction. The server host replaces the cache.CC[fh] value with SEG.CC, passes any data in the segment to the user, and processes a FIN bit if one is present.
Acknowledgment of the SYN is delayed so it can piggyback on a response segment 🎀
R1.3: If SEG.CC <= cache.CC[fh] (the TAO test has failed), or if cache.CC[fh] is undefined, or if there's no CC option (but possibly a CC.NEW option), the server host carries on with normal TCP processing. If the connection was in LISTEN state, the host runs a 3-way handshake using the standard TCP rules. In the SYN-SENT or SYN-SENT* state (that's the simultaneous open case), TCP sends ACK(SYN) and enters SYN-RECEIVED state.
R1.4: If there's no CC option (but possibly a CC.NEW option), then the server host sets cache.CC[fh] to undefined (zero). Receiving an ACK for a SYN (after applying rule R1.3) will update cache.CC[fh], by rule R3.
Now suppose an initial <SYN> segment containing a CC or CC.NEW option arrives in an I-state. That means a state named LISTEN-xx, where xx is one of TW, LA, L8, CL, or CL*. 🙈
R1.5: If the state is LISTEN-TW, then the duration of the current connection gets compared with MSL. If duration > MSL, then send a RST:
<SEQ=0><ACK=SEG.SEQ+SEG.LEN><CTL=RST,ACK>Then drop the packet and return.
R1.6: Do a special TAO test by comparing SEG.CC with TCB.CCrecv.
If SEG.CC is greater, then processing happens as if an ACK(FIN) had arrived. That means signalling the application that the previous close completed successfully, and deleting the previous TCB. Then a new TCB in LISTEN state gets created, and the SYN segment is reprocessed against the new TCB.
Otherwise, the segment gets silently discarded.
RECEIVING <SYN,ACK> SEGMENT
Suppose a client host receives a <SYN,ACK> segment for a connection in SYN-SENT or SYN-SENT* state.
R2.1: If SEG.ACK is not acceptable (see [STD-007]) and cache.CCsent[fh] is non-zero, then simply drop the segment without sending a RST. (The new SYN that the client is (re-)transmitting will eventually acknowledge any outstanding data and FIN at the server. Sweet, it all works out 🌈)
R2.2: If the segment contains a CC.ECHO option whose SEG.CC is different from TCB.CCsend, then the segment is unacceptable and gets dropped. 💀
R2.3: okay so, if cache.CCsent[fh] is zero, it gets set to TCB.CCsend 🌸
R2.4: if the segment has a CC option, its SEG.CC gets stored into TCB.CCrecv of the TCB 💜
RECEIVING <ACK> SEGMENT IN SYN-RECEIVED STATE
R3.1: if a segment has a CC option whose SEG.CC is different from TCB.CCrecv, then the segment is unacceptable and gets dropped 🙅♀️
R3.2: otherwise, the 3-way handshake has completed successfully on the server side ✨ If the segment has a CC option and cache.CC[fh] is zero, then cache.CC[fh] gets replaced by TCB.CCrecv 🎀
RECEIVING OTHER SEGMENT
R4: any other segment received with a CC option is unacceptable if its SEG.CC is different from TCB.CCrecv. BUT a RST segment is exempt from this test 🫶
OPEN REQUEST
To allow truncation of TIME-WAIT state, the following changes are made in the state diagram for OPEN requests (see Figure 10) 🦋
O1.1: a new passive open request is allowed in any of the states LAST-ACK, LAST-ACK*, CLOSING, CLOSING*, or TIME-WAIT. This causes a transition to the corresponding I-state (see Figure 10), which keeps the previous state, including the retransmission queue and timer 🧚
O1.2 a new active open request is allowed in TIME-WAIT or LISTEN-TW state, IF the elapsed time since the current connection opened is less than MSL 👑 The result is to delete the old TCB and create a new one, send a new SYN segment, and enter SYN-SENT or SYN-SENT* state (depending on whether or not the SYN segment has a FIN bit) 💅
Finally, T/TCP has a provision to improve performance for the case of a client that "sprays" transactions rapidly using many different server hosts and/or ports 🌈 If TCB.CCrecv in the TCB is non-zero (and we're still assuming the connection duration is less than MSL), then the TIME-WAIT delay may be set to min(K*RTO, 2*MSL). Here RTO is the measured retransmission timeout time, and the constant K is currently specified to be 8 ⭐
3.5. User Interface 🎀
[STD-007] defines a prototype user interface ("transport service") that implements the virtual circuit service model [STD-007, Section 3.8] 💖 One addition to this interface is required for transaction processing: a new Boolean flag "end-of-file" (EOF), added to the SEND call. A generic SEND call becomes:
Send
Format: SEND (local connection name, buffer address,
byte count, PUSH flag, URGENT flag, EOF flag [,timeout])The following text would be added to the description of SEND in [STD-007]:
If the EOF (End-Of-File) flag is set, any remaining queued data is pushed and the connection is closed. Just as with the CLOSE call, all data being sent is delivered reliably before the close takes effect, and data may continue to be received on the connection after completion of the SEND call. 🍓
Figure 8A shows a skeleton sequence of user calls by which a client could initiate a transaction 🧚 The SEND call initiates a transaction request to the foreign socket (host and port) specified in the passive OPEN call. The predicate "recv_EOF" tests whether or not a FIN has been received on the connection. This might be implemented using the STATUS command of [STD-007], or it might be implemented by some operating-system-dependent mechanism. When recv_EOF returns TRUE, the connection has been completely closed and the client end of the connection is in TIME-WAIT state 😭
__________________________________________________________________
| |
| |
| OPEN(local_port, foreign_socket, PASSIVE) -> conn_name; |
| |
| SEND(conn_name, request_buffer, length, |
| PUSH=YES, URG=NO, EOF=YES); |
| |
| while (not recv_EOF(conn_name)) { |
| |
| RECEIVE(conn_name, reply_buffer, length) -> count; |
| |
| <Process reply_buffer.> |
| } |
| |
| |
| Figure 8A: Client Side User Interface |
|__________________________________________________________________|If a client is going to send a rapid series of such requests to the same foreign_socket, it should use the same local_port for all of them 🫧 This will allow truncation of TIME-WAIT state. Otherwise, it could leave local_port wild, allowing TCP to choose successive local ports for each call, realizing that each transaction may leave behind a significant control block overhead in the kernel 💀
Figure 8B shows a basic sequence of server calls 🌷 The server application waits for a request to arrive, then reads and processes it until a FIN arrives (recv_EOF returns TRUE). At this time, the connection is half-closed. The SEND call used to return the reply completes the close in the other direction. It should be noted that the use of SEND(... EOF=YES) in Figure 4B instead of a SEND, CLOSE sequence is only an optimization. It allows piggybacking the FIN in order to minimize the number of segments. It should have little effect on transaction latency 🦋
__________________________________________________________________
| |
| |
| OPEN(local_port, ANY_SOCKET, PASSIVE) -> conn_name; |
| |
| <Wait for connection to open.> |
| |
| STATUS(conn_name) -> foreign_socket |
| |
| while (not recv_EOF(conn_name)) { |
| |
| RECEIVE(conn_name, request_buffer, length) -> count; |
| |
| <Process request_buffer.> |
| } |
| |
| <Compute reply and store into reply_buffer.> |
| |
| SEND(conn_name, reply_buffer, length, |
| PUSH=YES, URG=NO, EOF=YES); |
| |
| |
| Figure 8B: Server Side User Interface |
|__________________________________________________________________|4. IMPLEMENTATION ISSUES ✨
4.1. RFC-1323 Extensions 🎀
okay so there's this recently-proposed set of TCP enhancements [RFC-1323] that adds a Timestamps option carrying two 32-bit timestamp values 🕰️. It's used to measure round-trip time (RTT) super accurately 💅. The same option also powers "PAWS" (Protect Against Wrapped Sequence), which stops erroneous data delivery when old duplicate segments and sequence number reuse collide at very high bandwidths 🚀. literally a bodyguard for your bytes 🫶 The transaction approach in this memo doesn't depend on RFC-1323, but implementing RFC-1323 is desirable for all TCP's 💖.
The RFC-1323 extensions and the T/TCP extensions share some implementation issues 👯. Both need TCP headers to carry options. Fitting options into TCP headers means you have to change how the maximum segment size gets figured out, so you don't accidentally cause IP fragmentation 😬. Both also need an extra state variable in the TCB, which may or may not be a headache depending on your implementation 🙈.
4.2. Minimal Packet Sequence 📦
Most TCP implementations will need a few small tweaks to get the minimal packet sequence for a transaction shown in Figure 2 ✨.
Lots of TCP implementations delay acknowledgments for some of the data segments, to cut down on acknowledgment segments and to piggyback on the reverse data flow (usually character echoes) 🍓. To get minimal packet exchanges for transactions, you have to delay the acknowledgment of some control bits in a similar way 🧚. Specifically, the <SYN,ACK> segment that's about to go out in ESTABLISHED* or CLOSE-WAIT* state should be delayed. The amount of delay is set by the minimum RTO at the transmitter. It's a parameter of the protocol, not of the application 🤓. The proposal is to reuse the same delay parameter (and the same mechanism, if possible) that's already used for delaying data acknowledgments 💜.
To get the FIN piggy-backed on the reply data (segment #3 in Figure 2), implementations that have an implied PUSH=YES on all SEND calls will need to augment the user interface so that PUSH=NO can be set for transactions 🎀.
4.3. RTT Measurement ⏱️
Transactions bring new wrinkles into measuring round-trip times [Jacobson88] 🌀.
- (a) With the minimal 3-segment exchange, there can be exactly one RTT measurement in each direction for each transaction. Dynamic estimation of RTT can't happen within a single transaction, so it has to happen across successive transactions. That's why caching the measured RTT and RTT variance values is essential for transaction processing 🗂️. In normal virtual circuit communication, caching is only desirable.
- (b) When a transaction wraps up, the RTT and RTT variance kept in the cache must be some average of previous values combined with the values measured during the transaction that's completing. The big question is the time constant for this average, since the dynamics for transactions are quite different from file transfers, for example 🤔.
- (c) An RTT measurement by the client will yield:
T = RTT + min(SPT, ATO),where SPT (server processing time) was defined in the introduction, and ATO is the timeout period for sending a delayed ACK. So the measured RTT includes SPT, which can be arbitrarily variable. The resulting variability of the measured T, though, cannot exceed ATO. (In a popular TCP implementation, for example, ATO = 200ms, so the variance of SPT makes a relatively small contribution to the variance of RTT 🌈.)
- (d) Transactions sample the RTT at random times, which are determined by the client and server applications rather than by network dynamics 🎲. When there are long pauses between transactions, cached path properties will be poor predictors of current values in the network 🥶.
So the dynamics of RTT measurement for transactions differ from those for virtual circuits 🦋. RTT measurements should work correctly for very short connections, but reduce to the current TCP algorithms for long-lasting connections. Further study of this issue is needed 📚.
4.4. Cache Implementation 🗃️
This extension requires a per-host cache of connection counts 💾. The cache may also hold smoothed RTT, RTT variance, congestion avoidance threshold, and MSS values. Depending on implementation details, it may be simplest to build a fresh cache for these values. Another option is to use the routing cache that should already be included in the host [RFC-1122] 🛣️.
Implementation of the cache can be simplified because it's only consulted when a connection is established. After that, the CC values relevant to the connection are kept in the TCB. So a cache entry can be safely reused during the lifetime of a connection, with no need for locking 🔓✨.
4.5. CPU Performance 🏎️
TCP implementations are customarily optimized for streaming data at high speeds, not for opening or closing connections 💨. Jacobson's Header Prediction algorithm [Jacobson90] handles the simple common cases of in-sequence data and ACK segments when streaming data. To provide good performance for transactions, an implementation might be able to do an analogous "header prediction" specifically for the minimal request and response segments 🔮.
The overhead of UDP provides a lower bound on the overhead of TCP-based transaction processing. It probably won't be possible to reach that bound for TCP transactions, since opening a TCP connection creates a significant amount of state that UDP doesn't require 🫠.
McKenney and Dove [McKenney92] have pointed out that TCP transaction processing can stress the performance of the demultiplexing algorithm, meaning the algorithm used to look up the TCB when a segment arrives 🔍. They advocate hash-table techniques rather than a linear search. The effect may be especially acute for a transaction client using the extended TCP described here, because TCB's are left behind in TIME-WAIT state 😱. A high rate of transactions from a given client leaves a large number of TCB's in TIME-WAIT state until their timeouts expire. With a linear search, all of those control blocks must be traversed just to discover that the new association doesn't exist 🙃. In that situation, a hash table lookup should not degrade severely due to transactions 💖.
4.6. Pre-SYN Queue 📥
Suppose segment #1 in Figure 4 is lost in the network. When segment #2 arrives in LISTEN state, it's ignored by the TCP rules (see [STD-007] p.66, "fourth other text and control"), so it has to be retransmitted 😩. The server side could queue any ACK-less data segments received in LISTEN state and "replay" the segments in this queue when a SYN segment does arrive. A data segment received with an ACK bit, which is the normal case for existing TCP's, would still generate an RST segment 🙅.
Queueing segments in LISTEN state is different from queueing out-of-order segments after the connection is synchronized 🧐. In LISTEN state, the sequence number corresponding to the left window edge isn't known yet, so the segment can't be trimmed to fit within the window before it's queued. In fact, no processing should be done on a queued segment while the connection is still in LISTEN state. Therefore, a new "pre-SYN queue" would be needed 🆕. A timeout would also be required to flush the Pre-SYN Queue in case a SYN segment was never received ⏰.
Although implementing a pre-SYN queue isn't difficult in BSD TCP, its limited contribution to throughput probably doesn't justify the effort 🤷.
6. ACKNOWLEDGMENTS 🙏
I am so grateful to Dave Clark for pointing out bugs in RFC-1379 and for helping me clarify the model 👑. I also want to thank Greg Minshall, whose probing questions led to further elucidation of the issues in T/TCP 🫶.
7. REFERENCES 📚
- [Jacobson88] Jacobson, V., "Congestion Avoidance and Control", ACM SIGCOMM '88, Stanford, CA, August 1988.
- [Jacobson90] Jacobson, V., "4BSD Header Prediction", Comp Comm Review, v. 20, no. 2, April 1990.
- [McKenney92] McKenney, P., and K. Dove, "Efficient Demultiplexing of Incoming TCP Packets", ACM SIGCOMM '92, Baltimore, MD, October 1992.
- [RFC-1122] Braden, R., Ed., "Requirements for Internet Hosts -- Communications Layers", STD-3, RFC-1122, USC/Information Sciences Institute, October 1989.
- [RFC-1323] Jacobson, V., Braden, R., and D. Borman, "TCP Extensions for High Performance, RFC-1323, LBL, USC/Information Sciences Institute, Cray Research, February 1991.
- [RFC-1379] Braden, R., "Transaction TCP -- Concepts", RFC-1379, USC/Information Sciences Institute, September 1992.
- [ShankarLee93] Shankar, A. and D. Lee, "Modulo-N Incarnation Numbers for Cache-Based Transport Protocols", Report CS-TR-3046/UIMACS-TR-93-24, University of Maryland, March 1993.
- [STD-007] Postel, J., "Transmission Control Protocol - DARPA Internet Program Protocol Specification", STD-007, RFC-793, USC/Information Sciences Institute, September 1981.
APPENDIX A. ALGORITHM SUMMARY ✨
okay bestie, this is the cheat sheet 🧚 Appendix A is the whole T/TCP rulebook squished into one place, so you can see every rule at once. first up, the symbols we're gonna be using 💅
Options
CC(SEG.CC): TCP Connection Count (CC) Option
CC.NEW(SEG.CC): TCP CC.NEW option
CC.ECHO(SEG.CC): TCP CC.ECHO option
Here SEG.CC is option value in segment.
Per-Connection State Variables in TCB
CCsend: CC value to be sent in segments
CCrecv: CC value to be received in segments
Elapsed: Duration of connection
Global Variables:
CCgen: CC generator variable
cache.CC[fh]: Cache entry: Last CC value received.
cache.CCsent[fh]: Cache entry: Last CC value sent.so CCgen is the global generator, CCsend and CCrecv live in each connection's TCB, and the cache entries are keyed by fh 🦋
now the main event, the pseudo-code summary 🔥
PSEUDO-CODE SUMMARY:
Passive OPEN => {
Create new TCB;
}
Active OPEN => {
<Create new TCB>
CCrecv = 0;
CCsend = CCgen;
If (CCgen == 0xffffffff) then Set CCgen = 1;
else Set CCgen = CCgen + 1.
<Send initial {SYN} segment (see below)>
}a passive OPEN just makes a TCB, but an active OPEN does a whole thing: it zeroes CCrecv, copies CCgen into CCsend, bumps the generator (wrapping back to 1 after 0xffffffff), and then sends the initial SYN 💖
Send initial {SYN} segment => {
If (cache.CCsent[fh] == 0 OR CCsend < cache.CCsent[fh] ) then {
Include CC.NEW(CCsend) option in segment;
Set cache.CCsent[fh] = 0;
}
else {
Include CC(CCsend) option in segment;
Set cache.CCsent[fh] = CCsend;
}
}so the initial SYN picks between the two options depending on what we've already sent to that host. if the cache is empty or our new CC is smaller, we send CC.NEW and reset the cache to 0. otherwise it's the regular CC option and we update the cache 🌸
Send {SYN,ACK} segment => {
If (CCrecv != 0) then
Include CC(CCsend), CC.ECHO(CCrecv) options in segment.
}the SYN,ACK only gets the CC and CC.ECHO options if we've actually received a CC value (CCrecv isn't 0) 🫶
Receive {SYN} segment in LISTEN, SYN-SENT, or SYN-SENT* state => {
If state == LISTEN then {
CCrecv = 0;
CCsend = CCgen;
If (CCgen == 0xffffffff) then Set CCgen = 1;
else Set CCgen = CCgen + 1.
}
If (Segment contains CC option OR
Segment contains CC.NEW option) then
Set CCrecv = SEG.CC.
if (Segment contains CC option AND
cache.CC[fh] != 0 AND
SEG.CC > cache.CC[fh] ) then { /* TAO Test OK */
Set cache.CC[fh] = CCrecv;
<Mark connection half-synchronized>
<Process data and/or FIN and return>
}
If (Segment does not contain CC option) then
Set cache.CC[fh] = 0;
<Do normal TCP processing and return>.
}this one's a whole storyline 🎀 if we're in LISTEN we pick up fresh CC numbers first. then, if the SYN has CC or CC.NEW, we grab its value. the TAO test (the /* TAO Test OK */ bit) passes when there's a CC option, the cache isn't 0, and SEG.CC is bigger than the cached value. if it passes, we update the cache, mark the connection half-synchronized, and process any data or FIN. if the SYN has no CC option at all, the cache gets zeroed. then it's normal TCP from there 💅
Receive {SYN} segment in LISTEN-TW, LISTEN-LA, LISTEN-LA*, LISTEN-CL,
or LISTEN-CL* state => {
If ( (Segment contains CC option AND CCrecv != 0 ) then {
If (state = LISTEN-TW AND Elapsed > MSL ) then
<Send RST, drop segment, and return>.
if (SEG.CC > CCrecv ) then {
<Implicitly ACK FIN and data in retransmission queue>;
<Close and delete TCB>;
<Reprocess segment>.
/* Expect to match new TCB
* in LISTEN state.
*/
}
}
else
<Drop segment>.
}ngl this is the drama section 💀 for the TIME-WAIT-ish states, if the SYN has a CC and we already have a CCrecv, we check the timing first. if we're in LISTEN-TW and Elapsed is past MSL, we send a RST and drop it. otherwise, if SEG.CC beats CCrecv, we ACK the old FIN and data, close and delete the old TCB, and reprocess the segment so it can match a fresh TCB in LISTEN. no CC or no CCrecv means we just drop the segment 🙈
Receive {SYN,ACK} segment => {
if (Segment contains CC.ECHO option AND
SEG.CC != CCsend) then
<Send a reset and discard segment>.
if (Segment contains CC option) then {
Set CCrecv = SEG.CC.
if (cache.CC[fh] is undefined) then
Set cache.CC[fh] = CCrecv.
}
}when a SYN,ACK comes in, if it echoes a CC that doesn't match our CCsend, it's a no-go: we send a reset and throw the segment away. if it has a CC option, we save it to CCrecv, and if the cache is undefined we fill it in too 🦋
Send non-SYN segment => {
if (CCrecv != 0 OR
(cache.CCsent[fh] != 0 AND
state is SYN-SENT or SYN-SENT*)) then
Include CC(CCsend) option in segment.
}for regular non-SYN segments, we slap a CC(CCsend) option on it when CCrecv isn't 0, or when we're in SYN-SENT or SYN-SENT* and the cache has something in it 🌈
Receive non-SYN segment in SYN-RECEIVED state => {
if (Segment contains CC option AND RST bit is off) {
if (SEG.CC != CCrecv) then
<Segment is unacceptable; drop it and send an
ACK segment, as in normal TCP processing>.
if (cache.CC[fh] is undefined) then
Set cache.CC[fh] = CCrecv.
}
}in SYN-RECEIVED, a non-SYN segment with a CC option and the RST bit off has to match CCrecv. if it doesn't, it's unacceptable: drop it and send an ACK, same as normal TCP. if the cache is undefined, we fill it with CCrecv ✨
Receive non-SYN segment in (state >= ESTABLISHED) => {
if (Segment contains CC option AND RST bit is off) {
if (SEG.CC != CCrecv) then
<Segment is unacceptable; drop it and send an
ACK segment, as in normal TCP processing>.
}
}same vibe once we're ESTABLISHED or beyond: CC option plus RST bit off means SEG.CC has to match CCrecv, or it's unacceptable and gets dropped with an ACK sent back, as normal TCP does 👑
Security Considerations
okay this part is short but we have to say it: this memo does not discuss security issues at all 😭 so there's no security analysis in here, and nothing in this summary should be read as making any claim about security either way 🔥
Author's Address
Bob Braden
University of Southern California
Information Sciences Institute
4676 Admiralty Way
Marina del Rey, CA 90292
Phone: (310) 822-1511
EMail: Braden@ISI.EDUand that's the whole RFC, bestie 🎀💖 Bob Braden's name is on it, so go show your friends 🦋
📦 where this page comes from
- ietf: rfc1644 1994-07
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms