RFC 2140
TCP Control Block Interdependence
🎀 Quick heads-up, bestie: RFC 2140 doesn't have a section 4.1.3. Its headings aren't numbered at all, so I didn't invent one. Everything else is here, with the technical bits kept exact.
Status of this Memo 🌸
Okay so this memo is giving information for the Internet community 💅. It does not specify an Internet standard of any kind, so nobody has to implement anything from it. Distribution of this memo is unlimited, so share it with everyone 🫶.
Abstract ✨
Okay this doc makes the case for interdependent TCP control blocks 🦋. The idea is that part of the TCP state gets shared among similar concurrent connections, or across similar connection instances. TCP state is a whole mix of stuff: connection state, current round-trip time estimates, congestion control information, and process information 🧁. Right now all of that lives per-connection in the TCP control block, but the paper argues it should be shared across connections to the same host 💜. The goal is better transient transport performance while staying backward-compatible with existing implementations 🔥.
This document is a product of the LSAM project at ISI 🎀.
Introduction 💖
So TCP is a connection-oriented reliable transport protocol layered over IP [9] 🌈. Each TCP connection keeps its own state, usually in a data structure called the TCP Control Block (TCB) 📦. The TCB holds info about the connection state, its associated local process, and feedback parameters about how the connection's transmission is going ✨. As originally specified and usually implemented, the TCB is maintained on a per-connection basis, and this doc talks about what that choice means 🙈.
The author argues for an alternate approach that shares some of this state across similar connection instances and among similar simultaneous connections 🧚. The payoff is better transient performance, especially for the many short-lived and simultaneous connections you get on the World-Wide Web [1] 🕸️. Fun detail: these changes only touch TCB initialization, so the long-term behavior of TCP after a connection is established has no effect 💅.
The TCP Control Block (TCB) 🦋
Okay, so a TCB is associated with each connection, which just means each association of a pair of applications across the network 💫. The TCB can be summarized as containing [9]:
Local process state
pointers to send and receive buffers
pointers to retransmission queue and current segment
pointers to Internet Protocol (IP) PCB
Per-connection shared state
macro-state
connection state
timers
flags
local and remote host numbers and ports
micro-state
send and receive window state (size*, current number)
round-trip time and variance
cong. window size*
cong. window size threshold*
max windows seen*
MSS#
round-trip time and variance#The per-connection info gets split into macro-state and micro-state, terminology borrowed from [5] 🍓. Macro-state describes the finite state machine, and we include the endpoint numbers and components (timers, flags) that help maintain that state 🧠. This also covers the protocol for establishing and maintaining shared state about the connection. Micro-state describes the protocol after a connection is established, and it's what keeps the data transfer reliable and congestion-controlled 🔥.
There are two more classes of shared micro-state that are more about host-pairs than application pairs 👑. One class is clearly host-pair dependent (marked #, like MSS and RTT) 🌷. The other is host-pair dependent only in its aggregate (marked *, like congestion window info and current window sizes) 💜.
TCB Interdependence 🌈
Okay this part is so iconic 😭. The idea that some TCB state is host-pair specific rather than application-pair dependent isn't new at all, and it's a pretty common engineering choice in layered protocol implementations ✨. Sharing RTT info among protocols layered over IP, including UDP and TCP, was discussed in [8] 🎀. T/TCP uses caches to keep TCB info across instances, like smoothed RTT, RTT variance, congestion avoidance threshold, and MSS [3] 🦋. These values sit alongside the connection counts T/TCP uses to speed up data delivery before the full three-way handshake during an OPEN 🧚.
The whole goal is to aggregate TCB components where they reflect one association, which is the host-pair, instead of artificially splitting those components up by connection 💖.
At least one current T/TCP implementation saves the MSS and aggregates the RTT parameters across multiple connections, but skips caching the congestion window information [4], as originally specified in [2] 🙈. There may be other values worth caching, like current window size, so new connections get full access to accumulated channel resources 🫶.
Now here's the key bit 🔑: there are two cases of TCB interdependence. Temporal sharing happens when the TCB of an earlier (now CLOSED) connection to a host gets used to initialize some parameters of a new connection to that same host ⏳. Ensemble sharing happens when a currently active connection to a host gets used to initialize another (concurrent) connection to that host 👯. T/TCP documents only considered the temporal case, but this memo considers both 💅.
An Example of Temporal Sharing 💎
Temporal sharing of cached TCB data has been implemented in the SunOS 4.1.3 T/TCP extensions [4] and the FreeBSD port of the same [7] 🍒. Like we said before, only the MSS and RTT parameters are cached, as originally specified in [2]. Later discussion of T/TCP suggested adding congestion control parameters to this cache [3] 🌸.
The cache gets accessed in two ways: it's read to initialize new TCBs, and written when more current per-host state is available 📖. New TCBs are initialized like this. Note that snd_cwnd reuse isn't implemented yet, though it was discussed in the T/TCP concepts [2] 🙈:
TEMPORAL SHARING - TCB Initialization
Cached TCB New TCB
----------------------------------------
old-MSS old-MSS
old-RTT old-RTT
old-RTTvar old-RTTvar
old-snd_cwnd old-snd_cwnd (not yet impl.)Most cached TCB values get updated when a connection closes 🔒. The one exception is MSS, which gets updated whenever the MSS option is received in a TCP header 💫.
TEMPORAL SHARING - Cache Updates
Cached TCB Current TCB when? New Cached TCB
---------------------------------------------------------------
old-MSS curr-MSS MSSopt curr-MSS
old-RTT curr-RTT CLOSE old += (curr - old) >> 2
old-RTTvar curr-RTTvar CLOSE old += (curr - old) >> 2
old-snd_cwnd curr-snd_cwnd CLOSE curr-snd_cwnd (not yet impl.)MSS caching is super simple: reported values get cached, and the most recent value is used 🧁. The cache is updated whenever the MSS option is received, so it always has the most recent MSS value from any connection 🌈. The cache is only consulted at connection establishment and isn't otherwise updated, which means MSS options don't affect current connections 🙅♀️. The default MSS is never saved, since only reported MSS values update the cache, so you need an explicit override to reduce the MSS 🔧.
RTT values get updated by a more complicated mechanism [3], [8] 🧮. Dynamic RTT estimation needs a sequence of RTT measurements, even though a single T/TCP transaction might not collect enough samples 🥺. So the cached RTT (and its variance) is an average of its previous value with the contents of the currently active TCB for that host, when a TCB closes ⚖️. RTT values only update when a connection closes. Also, the way the RTT values get averaged isn't the same as the way they're computed within a connection, so the cached value might not be appropriate 🤷♀️.
For temporal sharing, the cache only needs updating when a connection closes, because the cached values won't be used to initialize a new TCB yet ⏰. For ensemble sharing it's different, as discussed below 👀.
Other TCB variables can also be cached between sequential instances, like the congestion control window information 🪟. Old cache values can be overwritten with the current TCB estimates, or a MAX or MIN function can merge the results, depending on how optimistic or pessimistic you want to be about the reused values 😤. For example, the congestion window can be reused if there are no concurrent connections 💖.
An Example of Ensemble Sharing ✨
okay bestie, so sharing cached TCB data across concurrent connections needs extra care 🫶 some of the shared state is really an aggregate thing. MSS and RTT values can just be copied over 💖 but copying congestion window info? not so simple 🙈 for now we're only going to cover the MSS and RTT rules 🎀
ENSEMBLE SHARING - TCB Initialization
Cached TCB New TCB
----------------------------------
old-MSS old-MSS
old-RTT old-RTT
old-RTTvar old-RTTvar ENSEMBLE SHARING - Cache Updates
Cached TCB Current TCB when? New Cached TCB
-----------------------------------------------------------
old-MSS curr-MSS MSSopt curr-MSS
old-RTT curr-RTT update rtt_update(old,curr)
old-RTTvar curr-RTTvar update rtt_update(old,curr)Okay so for ensemble sharing, the TCB info needs to get cached as early as possible 🦋 sometimes even before a connection closes. Otherwise, if you open a bunch of concurrent connections and none of them close before the others open, you literally get zero TCB sharing 💀 The optimistic move is updating the cached data as early as you can, not just when a connection is closing. Some T/TCP implementations already do this for MSS when the TCP MSS header option comes in [4] 💅 it just isn't addressed specifically in the concepts or functional specification [2][3] 🌸
In current T/TCP, RTT values only get updated after a CLOSE, which means concurrent sessions get nothing out of it 😭 As we mentioned in the temporal case, averaging values across concurrent connections means you have to fold in new RTT measurements 🧚 The work needed to update that aggregate average should be kept as small as possible, but the result should be the same as if every value had been measured inside one single connection ✨ The function "rtt_update" in the ensemble sharing table is what handles this. It runs whenever the RTT would have been updated in the individual TCP connection 👑 So the cache ends up holding the shared RTT variables, which means they don't need to live in the TCB anymore [8] 🎀
Congestion window aggregation is way messier in the concurrent case 🥺 With an ensemble of connections, we have to figure out how that ensemble would have shared the congestion window, so we can derive the initial values for new TCBs 🔥 Here's the thing: concurrent connections between two hosts (usually) share the same network paths, so they also share whatever capacity is available along that path 🌈 When it comes to congestion, the set of connections might behave as if it were multiplexed before TCP, like all the data was part of one single connection 💜 So the current window sizes would keep a constant sum, assuming there's enough offered load 💖 This would be going beyond caching and into truly sharing state, unlike the RTT case ⭐
We have to pause and say: any assumption about this sharing can be wrong, including this one 🙈 In current implementations, new congestion windows start at an initial value of one segment, so the sum of the current windows goes up for every new connection 😬 That can cause real trouble when several connections share a highly congested link, like with trans-Atlantic Web access 🌊
There are a few ways to initialize the congestion window in a new TCB among an ensemble of current connections to a host, as shown below 🦋 Current TCP implementations start it at one segment [9], and T/TCP hinted that it should start at the old window size [3] 💅 With the first option, the assumption is that new connections should behave as conservatively as possible 🫶 With the second, there's zero accommodation for concurrent aggregate behavior 💀
Either way, the sum of the window sizes can go up instead of staying put 😭 Another option is to give each pending connection its "fair share" of the available congestion window, then let the connections balance out from there 🎀 The assumption here is that new connections are basically implicit requests for an equal share of the available link bandwidth, and that share should be granted at the expense of the current connections 👑 This may or may not be the right function, so we're proposing it gets examined further 🔍
ENSEMBLE SHARING - TCB Initialization
Some Options for Sharing Window-size
Cached TCB New TCB
-----------------------------------------------------------------
old-snd_cwnd (current) one segment
(T/TCP hint) old-snd_cwnd
(proposed) old-snd_cwnd/(N+1)
subtract old-snd_cwnd/(N+1)/N
from each concurrent ENSEMBLE SHARING - Cache Updates
Cached TCB Current TCB when? New Cached TCB
----------------------------------------------------------------
old-snd_cwnd curr-snd_cwnd update (adjust sum as appropriate)Compatibility Issues 🎀
Okay so current TCP implementations don't use TCB caching, except for T/TCP variants [4][7] 🧚 New connections just use the default initial values for every non-instantiated TCB variable 💅 That means each connection works out its own RTT measurements, MSS value, and congestion info from scratch 🔄 Eventually those values get updated for each connection 🌸
For the congestion and current window info, the initial values might not match the long-term aggregate behavior of a set of concurrent connections 😬 Say a single connection has a window of 4 segments. New connections assume initial windows of 1 segment (the minimum), but the existing connection's window doesn't shrink to make room for that extra load 💀 So connections end up interfering with each other 😭 We've actually seen this on trans-Atlantic links, where concurrent connections carrying Web traffic collide because their initial windows are too big, even when they're set to one segment 🌊
Because this proposal tries to anticipate the aggregate steady-state values of TCB state across a group or over time, it should avoid the transient effects of new connections 🦋 Plus, since it looks at the ensemble and temporal properties of those aggregates, it should also keep the transients of short-lived or multiple concurrent connections from hurting overall network performance 💖 We are running analysis and experiments to validate these assumptions 🔬✨
Performance Considerations 💅
Here we're trying to optimize the transient behavior of TCP without messing with its long-term properties 🫶 The biggest cost is maintaining the cached values, or using per-host state instead of per-connection state 💸 Where performance does take a hit, though, the per-host info can be kept in per-connection copies (like it is now), because higher performance should mean less interference between concurrent connections 🚀
Sharing TCB state should only happen at connection establishment and close (to update the cache), so overhead stays low, transient behavior gets optimized, and the effect on the steady-state stays minimal 🌈 It's possible that sharing state during a connection, like the RTT or window-size variables, could be useful too, as long as the implementation cost isn't high 🧠
Implications 🔮
Okay there are a few implications to baking TCB interdependence into TCP implementations 👑 First, it might mean you don't need application-layer multiplexing for performance boosts [6] 💜 Protocols like persistent-HTTP dodge connection reestablishment costs by serializing or multiplexing a bunch of per-host connections over one TCP connection 🎀 That skips TCP's per-connection OPEN handshake, and it also skips recomputing MSS, RTT, and congestion windows 🧚 By avoiding the so-called "slow-start restart," performance can get optimized 🚀 Our proposal gives you the MSS, RTT, and OPEN handshake avoidance of T/TCP, plus the "slow-start restart avoidance" of multiplexing, without needing a multiplexing mechanism at the application layer 🙌 Multiplexing is going to get complicated once quality-of-service mechanisms (e.g., "integrated services scheduling") show up later ⚡
Second, we're trying to push some of the TCP implementation down from the traditional transport layer (in the ISO model [10]) to the network layer 🌍 This acknowledges that some state we currently keep per-connection is actually per-path, which we simplify as per-host-pair 🫧 Transport protocols usually manage per-application-pair associations (per stream), and network protocols manage per-path associations (routing) 🗺️ So round-trip time, MSS, and congestion info is handled more appropriately in a network-layer way: aggregated across concurrent connections and shared across connection instances 🦋
An earlier version of RTT sharing suggested implementing RTT state at the IP layer, not the TCP layer [8] 🧐 Our observations are about sharing state among TCP connections, which avoids some of the difficulties with an IP-layer solution 💅 One of those difficulties is figuring out which prior outgoing packet goes with an incoming packet, so you can infer the RTT from the exchange 🙈 Because RTTs are still determined inside the TCP layer, this is way simpler than doing it at the IP layer 🌸 It's a case where information should be computed at the transport layer but shared at the network layer ✨
We also want to note that per-host-pair associations aren't the only place these techniques apply 🌈 TCBs could be shared in a similar way between hosts on a LAN, since the predominant path can be LAN-LAN instead of host-host 🏠
There may be other info that can be shared between concurrent connections too 👀 For example, if another connection has just tried to expand its window size and failed, a connection may hold off on trying the same thing for a while 🫶 The idea is that existing TCP implementations infer the behavior of all competing connections, including those on the same host or LAN 🕵️♀️ One possible optimization is making that implicit feedback explicit, through extended information in the per-host TCP area 💜🔥
Security Considerations 🔒
okay bestie, this part gets a little serious 💀 but I'll make it make sense. These suggested enhancements don't add any new ramifications for direct attacks 🫶. They can be open to denial-of-service attacks if they aren't secured some other way. Like, an application can open a connection and set its window size to 0, and that literally denies service to any other connection between those same hosts 😭.
TCB sharing might be vulnerable to DoS attacks wherever the TCB is shared: between connections inside one host, or between hosts if TCB sharing is implemented on the LAN (check the Implications section 💅). Some shared TCB parameters are only used to create new TCBs, and others are shared among the TCBs of ongoing connections. New connections can join that ongoing set, e.g., to optimize send window size across a group of connections to the same host 🧚.
Attacks on parameters used only for initialization only affect the transient performance of a TCP connection 🦋. For short connections, the performance hit can get really close to a denial-of-service attack, ngl. For example, if an application changes its TCB to have a false and small window size, later connections would be slow until their window grew appropriately 💀.
The solution is to limit the effect of compromised TCB values 🛡️. TCBs count as compromised when they're modified directly by an application or sent between hosts through unauthenticated means (e.g., by using a dirty flag). TCBs that aren't compromised by application modification don't have any unique security ramifications 💜. Note that the proposed parameters for TCB sharing aren't currently modifiable by an application 🙈.
All shared TCBs MUST be validated against default minimum parameters before they're used for new connections. This validation won't impact performance, because it only happens at TCB initialization 🌸. This limits the effect of attacks on new connections to reducing the benefit of TCB sharing, which leaves you with the current default TCP performance. For ongoing connections, the effect of incoming packets on shared information should be both limited and validated against constraints before use. This is a beneficial precaution for existing TCP implementations as well 🎀.
TCBs modified by an application SHOULD not be shared, unless the new connection sharing the compromised information has been given explicit permission to use such information by the connection API 🚫. No mechanism for that indication currently exists, but it could be supported by an augmented API ✨. This sharing restriction SHOULD be implemented in both the host and the LAN. Sharing on a LAN SHOULD utilize authentication to prevent undetected tampering of shared TCB parameters 🔐. These restrictions limit the security impact of modified TCBs both for connection initialization and for ongoing connections 👑.
Finally, shared values MUST be limited to performance factors only 🚨. Other information, such as TCP sequence numbers, is already known to compromise security when shared, so absolutely not 😭🙅♀️.
Acknowledgements 🙏
The author wants to thank the members of the High-Performance Computing and Communications Division at ISI, notably Bill Manning, Bob Braden, Jon Postel, Ted Faber, and Cliff Neuman, for all their help developing this memo 🫶.
References 📚
[1] Berners-Lee, T., et al., "The World-Wide Web," Communications of
the ACM, V37, Aug. 1994, pp. 76-82.
[2] Braden, R., "Transaction TCP -- Concepts," RFC-1379,
USC/Information Sciences Institute, September 1992.
[3] Braden, R., "T/TCP -- TCP Extensions for Transactions Functional
Specification," RFC-1644, USC/Information Sciences Institute,
July 1994.
[4] Braden, B., "T/TCP -- Transaction TCP: Source Changes for Sun OS
4.1.3,", Release 1.0, USC/ISI, September 14, 1994.
[5] Comer, D., and Stevens, D., Internetworking with TCP/IP, V2,
Prentice-Hall, NJ, 1991.
[6] Fielding, R., et al., "Hypertext Transfer Protocol -- HTTP/1.1,"
Work in Progress.
[7] FreeBSD source code, Release 2.10, <http://www.freebsd.org/>.
[8] Jacobson, V., (mail to public list "tcp-ip", no archive found),
1986.
[9] Postel, Jon, "Transmission Control Protocol," Network Working
Group RFC-793/STD-7, ISI, Sept. 1981.
[10] Tannenbaum, A., Computer Networks, Prentice-Hall, NJ, 1988.Author's Address 💌
Joe Touch
University of Southern California/Information Sciences Institute
4676 Admiralty Way
Marina del Rey, CA 90292-6695
USA
Phone: +1 310-822-1511 x151
Fax: +1 310-823-6714
URL: http://www.isi.edu/~touch
Email: touch@isi.edu📦 where this page comes from
- ietf: rfc2140 1997-04
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms