RFC 3135

RFC 3135: Performance Enhancing Proxies Intended to Mitigate Link-Related Degradations 🎀

Okay bestie, let's get into RFC 3135 🦋. It was published in June 2001 by the Network Working Group, and it's an Informational doc, which means it's sharing knowledge with the Internet community. It's not a standard. The authors are J. Border from Hughes Network Systems, M. Kojo from the University of Helsinki, J. Griner from the NASA Glenn Research Center, G. Montenegro from Sun Microsystems, Inc., and Z. Shelby from the University of Oulu. Its title block doesn't list anything it obsoletes or updates, so it's a standalone survey 💅.

Status of this Memo ✨

This memo provides information for the Internet community. It does not specify an Internet standard of any kind. Distribution of this memo is unlimited.

Copyright Notice 🌸

Copyright (C) The Internet Society (2001). All Rights Reserved.

Abstract 💖

This doc is a survey of Performance Enhancing Proxies (PEPs) that people often use to fix degraded TCP performance caused by the quirks of specific link environments, like satellite, wireless WAN, and wireless LAN. It describes the different types of PEPs and the mechanisms they use to boost performance, with a big focus on proxies that work with TCP. It also explains why people build and use them, plus some of the consequences of using them, especially for the Internet as a whole 🌈.

Table of Contents 📋

1. Introduction  . . . . . . . . . . . . . . . . . . . . . . . . .  3
2. Types of Performance Enhancing Proxies  . . . . . . . . . . . .  4
2.1 Layering . . . . . . . . . . . . . . . . . . . . . . . . . . .  4
2.1.1 Transport Layer PEPs . . . . . . . . . . . . . . . . . . . .  5
2.1.2 Application Layer PEPs . . . . . . . . . . . . . . . . . . .  5
2.2 Distribution . . . . . . . . . . . . . . . . . . . . . . . . .  6
2.3 Implementation Symmetry  . . . . . . . . . . . . . . . . . . .  6
2.4 Split Connections  . . . . . . . . . . . . . . . . . . . . . .  7
2.5 Transparency . . . . . . . . . . . . . . . . . . . . . . . . .  8
3. PEP Mechanisms  . . . . . . . . . . . . . . . . . . . . . . . .  9
3.1 TCP ACK Handling . . . . . . . . . . . . . . . . . . . . . . .  9
3.1.1 TCP ACK Spacing  . . . . . . . . . . . . . . . . . . . . . .  9
3.1.2 Local TCP Acknowledgements . . . . . . . . . . . . . . . . .  9
3.1.3 Local TCP Retransmissions  . . . . . . . . . . . . . . . . .  9
3.1.4 TCP ACK Filtering and Reconstruction . . . . . . . . . . . . 10
3.2 Tunneling  . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.3 Compression  . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.4 Handling Periods of Link Disconnection with TCP  . . . . . . . 11
3.5 Priority-based Multiplexing  . . . . . . . . . . . . . . . . . 12
3.6 Protocol Booster Mechanisms  . . . . . . . . . . . . . . . . . 13
4. Implications of Using PEPs  . . . . . . . . . . . . . . . . . . 14
4.1 The End-to-end Argument  . . . . . . . . . . . . . . . . . . . 14
4.1.1 Security . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4.1.1.1 Security Implications  . . . . . . . . . . . . . . . . . . 15
4.1.1.2 Security Implication Mitigations . . . . . . . . . . . . . 16
4.1.1.3 Security Research Related to PEPs  . . . . . . . . . . . . 16
4.1.2 Fate Sharing . . . . . . . . . . . . . . . . . . . . . . . . 16
4.1.3 End-to-end Reliability . . . . . . . . . . . . . . . . . . . 17
4.1.4 End-to-end Failure Diagnostics . . . . . . . . . . . . . . . 19
4.2 Asymmetric Routing . . . . . . . . . . . . . . . . . . . . . . 19
4.3 Mobile Hosts . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.4 Scalability  . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.5 Other Implications of Using PEPs . . . . . . . . . . . . . . . 21
5. PEP Environment Examples  . . . . . . . . . . . . . . . . . . . 21
5.1 VSAT Environments  . . . . . . . . . . . . . . . . . . . . . . 21
5.1.1 VSAT Network Characteristics . . . . . . . . . . . . . . . . 22
5.1.2 VSAT Network PEP Implementations . . . . . . . . . . . . . . 23
5.1.3 VSAT Network PEP Motivation  . . . . . . . . . . . . . . . . 24
5.2 W-WAN Environments . . . . . . . . . . . . . . . . . . . . . . 25
5.2.1 W-WAN Network Characteristics  . . . . . . . . . . . . . . . 25
5.2.2 W-WAN PEP Implementations  . . . . . . . . . . . . . . . . . 26
5.2.2.1 Mowgli System  . . . . . . . . . . . . . . . . . . . . . . 26
5.2.2.2 Wireless Application Protocol (WAP)  . . . . . . . . . . . 28
5.2.3 W-WAN PEP Motivation . . . . . . . . . . . . . . . . . . . . 29
5.3 W-LAN Environments . . . . . . . . . . . . . . . . . . . . . . 30
5.3.1 W-LAN Network Characteristics  . . . . . . . . . . . . . . . 30
5.3.2 W-LAN PEP Implementations: Snoop . . . . . . . . . . . . . . 31
5.3.3 W-LAN PEP Motivation . . . . . . . . . . . . . . . . . . . . 33
6. Security Considerations . . . . . . . . . . . . . . . . . . . . 34
7. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 34
8. Acknowledgements  . . . . . . . . . . . . . . . . . . . . . . . 34
9. References  . . . . . . . . . . . . . . . . . . . . . . . . . . 35
10. Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . 39
Appendix A - PEP Terminology Summary . . . . . . . . . . . . . . . 41
Full Copyright Statement . . . . . . . . . . . . . . . . . . . . . 45

1. Introduction ✨

Okay so, TCP [RFC0793] is the transport layer protocol that tons of Internet and intranet apps rely on 🍓. The thing is, in certain environments, TCP and other higher layer protocols get held back by the link characteristics of that environment 😭.

This doc is a survey of Performance Enhancing Proxy (PEP) mitigation techniques, aka ways to fix that slowdown 🧚. A PEP helps Internet protocols run better on network paths where regular performance takes a hit because of the characteristics of a link or subnetwork along the path. This doc is informational, so it doesn't tell you whether to use PEPs or not, no judgment 🙈. Separate standards track recommendations for mitigating TCP performance over links with high error rates, links with low bandwidth, and so on have been developed or are in progress by the Performance Implications of Link Characteristics WG (PILC) [PILCWEB] 💜.

Link design choices can have a huge impact on the performance and efficiency of the Internet 🔥. But not every link characteristic, like high latency, can be fixed by choices in link layer design. And the cost of compensating for some characteristics might be way too high for certain technologies 💀. The techniques in this survey are applied to link technologies that already exist. When new link technologies are designed, they should be designed so these techniques aren't required, if that's at all possible 🌸.

This doc does not push PEPs as a general solution. Honestly, we believe the end-to-end principle should stay the prevailing approach in designing Internet protocols 👑. PEPs should only be used in specific environments and circumstances where end-to-end mechanisms that give similar performance boosts aren't available. In any environment where someone might consider a PEP for better performance, an end user (or, in some cases, the responsible network administrator) should know the PEP is there, and the choice to use PEP functionality should be under the end user's control. This matters most if a PEP would interfere with end-to-end use of IP layer security mechanisms, or otherwise have undesirable effects in some situations 🔐. That way the user can always choose end-to-end IP, but of course, without whatever performance boost the PEP might have given 🎀.

This survey doesn't make recommendations either way about using PEPs 🦋. Standards track recommendations have been or are being developed within the IETF for individual link characteristics, like links with high error rates, links with low bandwidth, links with asymmetric bandwidth, and so on, by the PILC working group [PILCWEB] 💅.

The rest of this doc is organized like this. Section 2 gives an overview of the different kinds of PEP implementations ✨. Section 3 covers some of the mechanisms PEPs may use to improve performance 🛠️. Section 4 talks about some of the implications of using PEPs, especially for the global Internet 🌍. And finally, Section 5 walks through example environments where PEPs get used: satellite very small aperture terminal (VSAT) environments, mobile wireless WAN (W-WAN) environments, and wireless LAN (W-LAN) environments 🛰️. There's also a summary of PEP terminology in an appendix (Appendix A) 📚.

2. Types of Performance Enhancing Proxies 🌈

There are many types of Performance Enhancing Proxies, and each one gets used in different environments to tackle different link characteristics that mess with protocol performance 💁‍♀️. Also, enhancing performance isn't limited to throughput. Other performance-related stuff, like how usable a link is, can get addressed too. For example, [M-TCP] deals with keeping TCP connections alive during periods of disconnection in wireless networks 🫶.

The following sections describe some of the key characteristics that set different types of PEPs apart ✨.

2.1 Layering 🧁

Honestly, a PEP can function at any protocol layer, but it usually only works at one or two layers 💖. This doc focuses on PEPs that function at the transport layer or the application layer, since those are the most common for boosting performance over links with tricky characteristics 🚀. A PEP can also operate below the network layer, meaning at the link layer, but this doc barely pays attention to those. Link layer mechanisms can be, and typically are, implemented transparently to the network and higher layers, so nothing above the link layer needs to change 🦋.

Also, some PEPs work across several protocol layers by using protocol info and possibly changing how the protocol operates at more than one layer. For a PEP like that, it can be hard to say exactly which layer(s) it operates on 🌀. Lowkey mind-bending, I know 🙈.

2.1.1 Transport Layer PEPs

Okay so transport layer PEPs live at the transport level 🦋 They might be aware of what kind of application is riding on the transport layer, but at most they only use that info to influence how they handle the transport protocol. They don't modify the application protocol in any way and just let it operate end-to-end, which is so iconic 💅 Most transport layer PEP implementations interact with TCP, and those are called TCP Performance Enhancing Proxies (TCP PEPs) ✨ For example, if ACKs tend to bunch together and cause annoying data segment bursts, a TCP PEP can simply modify the ACK spacing to improve performance. On the flip side, in an environment with a large bandwidth*delay product, a TCP PEP might alter the behavior of the TCP connection by generating local acknowledgments to TCP data segments, so the connection's throughput goes up 🚀

The term TCP spoofing is sometimes used as a synonym for TCP PEP functionality, but ngl it's not the same thing 🙈 TCP spoofing more accurately describes intercepting a TCP connection in the middle and terminating it as if the interceptor is the intended destination. That's a characteristic of many TCP PEP implementations, but it's not a characteristic of all of them, so don't let the names fool you 💖

2.1.2 Application Layer PEPs

Application layer PEPs operate above the transport layer 🌸 Different kinds of application layer proxies are already widely used across the Internet, like Web caches and relay Mail Transfer Agents (MTA). They typically try to improve performance or service availability and reliability in general, in a way that works in any environment, and they don't necessarily include any optimizations specific to certain link characteristics.

Application layer PEPs, on the other hand, can be built to improve both application protocol and transport layer performance for a particular application running over a particular type of link 🔥 An application layer PEP can have the same functionality as the matching regular proxy for that application (e.g., relay MTA or Web caching proxy), but with link-specific optimizations of the application protocol operation added on top. Main character energy ✨

Some application protocols have extraneous round trips, overly verbose headers, and/or inefficient header encoding, which can really hurt performance, especially with long delay and slow links 😭 This unnecessary overhead can be reduced, in general or for a particular type of link, by using an application layer PEP at an intermediate node. Some examples of application layer PEPs that have been shown to improve performance on slow wireless WAN links are described in [LHKR96] and [CTC+97] 🧚

2.2 Distribution

A PEP implementation may be integrated, meaning it comprises a single PEP component implemented within a single node, or distributed, meaning it comprises two or more PEP components, typically implemented across multiple nodes 🍓 An integrated PEP implementation is a single point where performance enhancement is applied. For example, a single PEP component might be implemented to provide impedance matching at the point where wired and wireless links meet.

A distributed PEP implementation is generally used to surround a particular link where performance enhancement is wanted 🌈 For example, a PEP implementation for a satellite connection may be distributed between two PEPs, one located at each end of the satellite link 🛰️

2.3 Implementation Symmetry

A PEP implementation may be symmetric or asymmetric 💜 Symmetric PEPs use identical behavior in both directions, meaning the actions the PEP takes don't depend on which interface a packet was received on. Asymmetric PEPs operate differently in each direction. The direction can be defined in terms of the link (e.g., from a central site to a remote site) or in terms of protocol traffic (e.g., the direction of TCP data flow, often called the TCP data channel, or the direction of TCP ACK flow, often called the TCP ACK channel). An asymmetric PEP implementation is generally used at a point where the characteristics of the links on each side of the PEP differ, or with asymmetric protocol traffic 👑 For example, an asymmetric PEP might be placed at the intersection of wired and wireless networks, or an asymmetric application layer PEP might be used for the request-reply type of HTTP traffic. A PEP implementation may also be both symmetric and asymmetric at the same time with regard to different mechanisms it employs. (PEP mechanisms are described in Section 3.)

Whether a PEP implementation is symmetric or asymmetric is independent of whether it is integrated or distributed. In other words, a distributed PEP implementation might operate symmetrically at each end of a link, meaning the two PEPs function identically 👯 On the other hand, a distributed PEP implementation might operate asymmetrically, with a different PEP implementation at each end of the link. Again, this is usually used with asymmetric links. For example, on a link with an asymmetric amount of bandwidth available in each direction, the PEP on the end forwarding traffic in the direction with a large amount of bandwidth might focus on locally acknowledging TCP traffic to use that available bandwidth. At the same time, the PEP on the end forwarding traffic in the direction with very little bandwidth might focus on reducing the amount of TCP acknowledgement traffic forwarded across the link, so the link doesn't get congested 🫶

2.4 Split Connections

A split connection TCP implementation terminates the TCP connection received from an end system and establishes a corresponding TCP connection to the other end system 🎀 In a distributed PEP implementation, this is typically done so a third connection between the two PEPs can be optimized for the link. That might be a TCP connection optimized for the link, or it might be another protocol, for example, a proprietary protocol running on top of UDP. Also, the distributed implementation might use a separate connection between the proxies for each TCP connection, or it might multiplex the data from multiple TCP connections across a single connection between the PEPs 🔀

In an integrated PEP split connection TCP implementation, the PEP again terminates the connection from one end system and originates a separate connection to the other end system. [I-TCP] documents an example of a single PEP split connection implementation 📚

Lots of integrated PEPs use a split connection implementation to address a mismatch in TCP capabilities between two end systems 🙃 For example, the TCP window scaling option [RFC1323] can extend the maximum amount of TCP data that can be "in flight" (i.e., sent and awaiting acknowledgement). This is useful for filling a link that has a high bandwidth*delay product. If one end system is capable of using scaled TCP windows but the other isn't, the end system that isn't capable can set up its connection with a PEP on its side of the high bandwidth*delay link. The split connection PEP then sets up a TCP connection with window scaling over the link to the other end system 💅

Split connection TCP implementations can effectively leverage TCP performance enhancements that are optimal for a particular link but which cannot necessarily be employed safely over the global Internet 🌍

Note that using split connection PEPs doesn't necessarily rule out simultaneous use of IP for end-to-end connectivity 🙌 If a split connection is managed per application or per connection and is under the control of the end user, the user can decide whether a particular TCP connection or application uses the split connection PEP or operates end-to-end. When a PEP is employed on a last hop link, that end user control is relatively easy to implement ✨

In effect, application layer proxies for TCP-based applications are split connection TCP implementations, with end systems using PEPs as a service related to a particular application 🦋 Therefore, all transport (TCP) layer enhancements available with split connection TCP implementations can also be employed with application layer PEPs in conjunction with application layer enhancements. Stacked benefits, periodt 💖

2.5 Transparency

Another key characteristic of a PEP is its degree of transparency 👀 PEPs may operate totally transparently to the end systems, transport endpoints, and/or applications involved (in a connection), requiring no modifications to the end systems, transport endpoints, or applications.

On the other hand, a PEP implementation may require modifications to both ends in order to be used. In between, a PEP implementation may require modifications to only one of the ends involved. Either of these kinds of PEP implementations is non-transparent, at least to the layer requiring modification 🙈

It's sometimes useful to think about the degree of transparency of a PEP implementation at four levels: transparency with respect to the end systems (network-layer transparent PEP), transparency with respect to the transport endpoints (transport-layer transparent PEP), transparency with respect to the applications (application-layer transparent PEP), and transparency with respect to the users 🧚 For example, a user who subscribes to a satellite Internet access service may be aware that the satellite terminal is providing a performance enhancing service, even though the TCP/IP stack and the applications on the user's PC aren't aware of the PEP that implements it 🛰️

Note that transparency is not the same issue as maintaining end-to-end semantics 🧠 For example, a PEP implementation that simply uses a TCP ACK spacing mechanism maintains the end-to-end semantics of the TCP connection, while a split connection TCP PEP implementation may not. Yet both can be implemented transparently to the transport endpoints at both ends. The implications of not maintaining the end-to-end semantics, in particular the end-to-end semantics of TCP connections, are discussed in Section 4 ⭐

3. PEP Mechanisms

An obvious key characteristic of a PEP implementation is the mechanism(s) it uses to improve performance 🔧 Some examples of PEP mechanisms are described in the following subsections. A PEP implementation might implement more than one of these mechanisms 💜

3.1 TCP ACK Handling

Many TCP PEP implementations are based on TCP ACK manipulation 🎀 The handling of TCP acknowledgments can differ significantly between different TCP PEP implementations. The following subsections describe various TCP ACK handling mechanisms. Many implementations combine some of these mechanisms and possibly employ some additional mechanisms as well 🌸

3.1.1 TCP ACK Spacing

In environments where ACKs tend to bunch together, ACK spacing is used to smooth out the flow of TCP acknowledgments traversing a link 🌊 This improves performance by eliminating bursts of TCP data segments that the TCP sender would otherwise send due to back-to-back arriving TCP acknowledgments [BPK97] 🫶

3.1.2. Local TCP Acknowledgements ✨

okay so some PEP implementations let the PEP locally acknowledge TCP data segments it receives 🫶 this is literally so useful over paths with a big bandwidth*delay product because it speeds up TCP slow start and lets the sending TCP open up its congestion window way faster 🚀 local (negative) acknowledgments are also often used to trigger local, faster error recovery on links with serious error rates 💀 (see Section 3.1.3, obviously 💅)

local acks happen automatically with split connection TCP implementations 🦋 and when they're in use, the TCP PEP is the one stuck with the job of recovering any data that gets dropped after the PEP already acknowledged it. no pressure 🙈

3.1.3. Local TCP Retransmissions 🔁

a TCP PEP can locally retransmit data segments that get lost on the path between itself and the receiving end system 🌸 the goal is faster recovery from lost data, and it's giving main character energy 👑 to pull this off, the PEP can use acknowledgments coming back from the receiving end system, plus suitable timeouts, to figure out when it's time to retransmit locally 🧚

TCP PEPs that send local acknowledgments to the sending end system are required to use local retransmissions towards the receiving end system 🎀 so they come as a set, iykyk 😭

some PEPs do local retransmissions even when they aren't using local acknowledgments to change TCP connection performance 🍓 Basic Snoop [SNOOP] is a well-known example of this 💜 Snoop caches the TCP data segments it receives and forwards 📦 then it keeps an eye on the end-to-end acknowledgments coming back from the receiving TCP end system, watching for duplicate acknowledgments (DUPACKs) 👀 when DUPACKs show up, Snoop locally retransmits the lost TCP data segments straight from its cache ✨ and it hides those DUPACKs from the sending TCP end system until acknowledgments for new data arrive 🫠 Snoop also has an option to use local negative acknowledgments to trigger local TCP retransmissions 🔥 one way to do that is applying TCP selective acknowledgments locally on the error-prone link (see Section 5.3 for details 🌈)

3.1.4. TCP ACK Filtering and Reconstruction 🪞

on paths with really lopsided bandwidth, the TCP ACKs flowing in the low-speed direction can get congested if the asymmetry ratio is high enough 😩 the ACK filtering and reconstruction mechanism fixes this by filtering the ACKs on one side of the link and then rebuilding the deleted ACKs on the other side 🪄 the mechanism, and the whole problem of TCP ACK congestion on highly asymmetric links, get the deep dive in [RFC2760] and [BPK97] 📚 go read those if you're curious bestie 🤓

3.2. Tunneling 🚇

a Performance Enhancing Proxy can encapsulate messages 📦 so they can be carried across a particular link, or forced to traverse a particular path 🛤️ a PEP at the far end of the encapsulation tunnel strips off the tunnel wrappers before final delivery to the receiving end system 🎁 a tunnel might be used by a distributed split connection TCP implementation as the way to carry the connection between the distributed PEPs 🧩 tunnels can also help force TCP connections that use asymmetric routing to go through the end points of a distributed PEP implementation 🦋

3.3. Compression 🗜️

loads of PEP implementations include support for one or more forms of compression 💖 in some cases compression is literally the only mechanism used for performance improvement 🤯 compression cuts down the number of bytes that need to be sent across a link, which is useful in general and can be super important for bandwidth-limited links 🫶 the perks include better link efficiency and higher effective link utilization, reduced latency and improved interactive response time, decreased overhead, and a reduced packet loss rate over lossy links ✨

where it makes sense, link layer compression gets used 🔗 TCP and IP header compression are also really common with PEP implementations 💅 [RFC1144] describes a widely deployed method for compressing TCP headers 🎀 other header compression algorithms are described in [RFC2507], [RFC2508] and [RFC2509] 🌸

payload compression is also desirable, and it's getting more important with today's huge focus on Internet security 🔐 here's the tea: network (IP) layer (and above) security mechanisms turn IP payloads into random bit streams 🌀 that defeats link layer compression, because the redundant "information" gets removed or hidden 😭 so compressing the payload has to happen before the security mechanisms are applied 🙏 [RFC2393] defines a framework where common compression algorithms can be applied to arbitrary IP segment payloads 🧠 but [RFC2393] compression isn't always applicable, sadly 💔 lots of IP payloads (like images, audio, video, and "zipped" files being transferred) are already compressed 📸 and when security mechanisms like TLS [RFC2246] are applied above the network (IP) layer, the data is already encrypted (and maybe compressed too), so the redundancy is removed or hidden again 🙈 the extra transport or network layer compression that's left will only compact headers, which are small and may already be covered by their own separate compression algorithms 🤏

with application layer PEPs, you can use application-specific compression 🎯 an application-specific (or content-specific) compression mechanism is usually way more efficient than any generic compression mechanism 🏆 for example, a distributed Web PEP implementation might use a more efficient binary encoding of HTTP headers 🧃 or a PEP can use lossy compression that reduces the image quality of online images on Web pages according to end user instructions 🖼️ that means fewer bytes go over a slow link, which means a faster response time for the user [LHKR96] 🚀

3.4. Handling Periods of Link Disconnection with TCP 📡

periods of link disconnection or link outages happen all the time with some wireless links 😤 during these periods, a TCP sender doesn't get the acknowledgments it's expecting ❌ when the retransmit timer expires, TCP closes its congestion window, with all of the related drawbacks 💀 a TCP PEP can keep an eye on the traffic going from the TCP sender toward the TCP receiver behind the disconnected link 👀 the TCP PEP holds on to the last ACK, so it can shut down the TCP sender's window by sending that last ACK with a window set to zero 🪟 that way, the TCP sender goes into persist mode 🧊

to make this work in both directions with an integrated TCP PEP implementation, the TCP receiver behind the disconnected link must be aware of the current state of the connection, and when a disconnection happens, it must be able to freeze all of its timers ⏸️ [M-TCP] implements this kind of operation ✨ another option is that the disconnected link is surrounded by a distributed PEP pair 👯

in split connection TCP implementations, a period of link disconnection can easily be hidden from the end host on the other side of the PEP 🫥 so the TCP connection doesn't break even if the disconnection lasts a super long time 🥹 if the TCP PEP can't forward data because the link is down, it stops receiving data 🛑 normal TCP flow control then stops the TCP sender from sending more than the TCP advertised window the PEP allowed 🚦 so the PEP and its counterpart behind the disconnected link can use a modified TCP version that keeps the state and all unacknowledged data segments through the disconnection, and then does local recovery once the link is back 🔄 whether the disconnection is hidden from the application and user depends on what the user is actually using the TCP connection for 🤷‍♀️

3.5. Priority-based Multiplexing 🚦

putting priority-based multiplexing on a slow and expensive link can seriously improve the performance and usability of that link for selected applications or connections 💎

picture a user stuck behind a slow link 🐢 if there are simultaneous data transfers, the link feels way more usable when urgent transfers (like interactive connections) get a shorter response time than less urgent background transfers ⚡ and if the interactive connections send enough data to keep the slow link fully utilized, the background transfers might need to be fully suspended for a while so the interactive ones get timely delivery 🫶

in-flight TCP segments of an end-to-end TCP connection (with low priority) can't be delayed for a long time 😬 otherwise the TCP timer at the sending end expires, and performance gets suboptimal 📉 BUT this can be controlled together with a split connection TCP PEP, by assigning different priorities to different connections (or applications) 🎛️ a split connection PEP implementation lets the PEP in an intermediate node delay the data delivery of a lower-priority TCP flow for an unlimited period of time, just by rescheduling the order in which it forwards data from different flows to the destination host behind the slow link 🗓️ this doesn't hurt the delayed TCP flow, because normal TCP flow control handles suspending the flow between the TCP sender and the PEP whenever the PEP isn't forwarding its data, and resumes it once the PEP decides to keep going 🪄 it can be extra helpful if the protocol stacks on both sides of the slow link use priority-based scheduling of connections too 🤝

with a PEP like this, plus user-controlled priorities, the user can give higher priority to selected interactive connections and get a much shorter response time on them 🏃‍♀️💨 even when there are simultaneous low-priority bulk transfers that, in regular end-to-end operation, would eat almost all of the slow link's bandwidth 🍽️ those bulk transfers then just chill and proceed during the idle periods of the interactive connections 😌 so the user can keep a slow and expensive link (like a wireless WAN) fully utilized 💰

other priority-based mechanisms can be applied on shared wireless links with more than two terminals 📶 shared wireless mediums are becoming a weak link in Internet QoS architectures, so lots of people turn to PEPs to get extra priority levels across a shared wireless medium [SHEL00] 🙌 these PEPs are distributed across all nodes of the shared wireless medium 🕸️ for example, in an 802.11 WLAN, this PEP is implemented in the access point (base station) and in each mobile host 📱 one PEP then uses distributed queuing techniques to coordinate the traffic classes of all nodes 🧠 this is also sometimes called subnet bandwidth management 💅 see [BBKT97] for an example of queuing techniques that can be used to achieve this 📖 and this technique can be implemented either above or below the IP layer 🪜 priority treatment can typically be specified either by the user or by marking the (IPv4) ToS or (IPv6) Traffic Class IP header field 🏷️✨

3.6 Protocol Booster Mechanisms 🚀

Okay so the research in [FMSBMR98] shows there are a bunch more PEP mechanisms, and they're called protocol boosters 💅. Some of them are specific to UDP flows, and that's giving very niche but iconic 🦋. For example, a PEP may apply asymmetrical methods such as extra UDP error detection. The 16 bit UDP checksum is optional, so it's typically not computed 🙈. BUT, for links with errors, the checksum could be super beneficial ✨. So a PEP can add this checksum to outgoing UDP packets. Main character energy 👑.

Symmetrical mechanisms have also been developed 🌈. A Forward Erasure Correction (FZC) mechanism can be used with real-time and multicast traffic 🎀. The encoding PEP adds a parity packet over a block of packets. Then, when it's received, the parity gets removed and the missing data is regenerated 🫶. A jitter control mechanism can be implemented, but it comes at the expense of extra latency 😭. A sending PEP can add a timestamp to outgoing packets. The receiving PEP then delays packets so it can reproduce the correct interval 🧚.

4. Implications of Using PEPs 🌸

The following sections describe some of the implications of using Performance Enhancing Proxies, so let's get into it bestie 💜.

4.1. The End-to-end Argument

As indicated in [RFC1958], the end-to-end argument [SRC84] is one of the architectural principles of the Internet 🍓. The basic argument is that, as a first principle, certain required end-to-end functions can only be correctly performed by the end systems themselves. Most of the potential negative implications of using PEPs are related to the possibility of breaking the end-to-end semantics of connections. This is one of the main reasons why PEPs are not recommended for general use 💀.

As indicated in Section 2.5, not all PEP implementations break the end-to-end semantics of connections. Correctly designed PEPs do not attempt to replace any application level end-to-end function. They only attempt to add performance optimizations to a subpath of the end-to-end path between the application endpoints, and that can be consistent with the end-to-end argument ✨. However, a user or network administrator adding a PEP to their network configuration should be aware of the potential end-to-end implications related to the mechanisms used by that particular PEP implementation 🔥.

4.1.1. Security 🔐

In most cases, security applied above the transport layer can be used with PEPs, especially transport layer PEPs 🎀. However, today only a limited number of applications include support for transport (or higher) layer security 🙈. Network (IP) layer security (IPsec) [RFC2401], on the other hand, can generally be used by any application, transparently to the application. It's giving reliable 💅.

4.1.1.1. Security Implications

The most detrimental negative implication of breaking the end-to-end semantics of a connection is that it disables end-to-end use of IPsec 😭. In general, a user or network administrator must choose between using PEPs and using IPsec. If IPsec is employed end-to-end, PEPs implemented on intermediate nodes in the network cannot examine the transport or application headers of IP packets. That's because encryption of IP packets via IPsec's ESP header (in either transport or tunnel mode) renders the TCP header and payload unintelligible to the PEPs 🔒. Without being able to examine those headers, a PEP may not function optimally or at all 💀.

If a PEP implementation is non-transparent to the users and the users trust the PEP in the middle, IPsec can be used separately between each end system and the PEP. However, in most cases this is an undesirable or unacceptable alternative, because end systems cannot trust PEPs in general 🙅‍♀️. It's also not as secure as end-to-end security. (For example, the traffic is exposed in the PEP when it is decrypted to be processed.) And it can lead to potentially misleading security level assumptions by the end systems. If the two end systems negotiate different levels of security with the PEP, the end system that negotiated the stronger level may not be aware that a lower level of security is being provided for part of the connection 😳. The PEP could be implemented to prevent this by being smart enough to force the same level of security to each end system, but that increases the complexity of the PEP implementation (and it still isn't as secure as end-to-end security) 🧠.

With a transparent PEP implementation, it's difficult for the end systems to trust the PEP because they may not be aware of its existence 👀. Even if the user is aware of the PEP, setting up acceptable security associations with the PEP while maintaining the PEP's transparent nature is problematic, if not impossible 💔.

Note that even when a PEP implementation does not break the end-to-end semantics of a connection, the PEP implementation may still be unable to function in the presence of IPsec. For example, it's difficult to do ACK spacing if the PEP cannot reliably determine which IP packets contain ACKs of interest 🫠. In any case, the authors are currently not aware of any PEP implementations, transparent or non-transparent, that provide support for end-to-end IPsec, except in a case where the PEPs are implemented on the end hosts 🥺.

4.1.1.2. Security Implication Mitigations

There are some steps that can be taken to let IPsec and PEPs coexist ✨. If an end user can select the use of IPsec for some traffic and not for other traffic, PEP processing can be applied to the traffic sent without IPsec. Of course, the user must then do without security for that traffic, or provide security for it via other means (for example, by using transport layer security) 🔑. However, even when this is possible, significant complexity may need to be added to the configuration of the end system 😵‍💫.

Another alternative is to implement IPsec between the two PEPs of a distributed PEP implementation. This at least protects the traffic between the two PEPs. (The issue of trusting the PEPs does not change.) In the case where the PEP implementation is not transparent to the user, (assuming that the user trusts the PEPs,) the user can configure their end system to use the PEPs as the end points of an IPsec tunnel 🛡️. An IPsec tunnel could even potentially be used between the end system and a PEP to protect traffic on that part of the path. But all of this adds complexity 🌀. And it still doesn't eliminate the risk of the traffic being exposed in the PEP itself, since the traffic is received from one IPsec tunnel, processed, and then forwarded (even if it's forwarded through another IPsec tunnel) 😱.

4.1.1.3. Security Research Related to PEPs

There is research underway investigating ways of changing the implementation of IPsec to be more friendly to the use of PEPs 🔬. One approach being actively looked at is multi-layer IP security. [Zhang00] describes a method that allows TCP headers to be encrypted as one layer (with the PEPs in the path of the TCP connections included in the security associations used to encrypt the TCP headers), while the TCP payload is encrypted end-to-end as a separate layer. This still involves trusting the PEP, but to a much lesser extent 🥰. However, a drawback is that it adds a significant amount of complexity to the IP security implementation. Given the existing complexity of IPsec, this drawback is a serious impediment to standardizing the multi-layer IP security idea, and it is very unlikely this approach will be adopted as a standard any time soon 😩. Therefore, relying on this type of approach will likely involve non-standard protocols (and the associated risk of doing so) 🚧.

4.1.2. Fate Sharing 🤝

Another important aspect of the end-to-end argument is fate sharing. If a failure occurs in the network, the ability of the connection to survive it depends on how much state is maintained on behalf of the connection in the network, and on whether that state is self-healing. If no connection-specific state resides in the network, or such state is self-healing (as in regular end-to-end operation), then a failure in the network will break the connection only if there is no alternate path through the network between the end systems. And if there is no path, both end systems can detect this. However, if the connection depends on some state being stored in the network (e.g., in a PEP), then a failure in the network (e.g., the node containing a PEP crashes) causes this state to be lost, forcing the connection to terminate even if an alternate path through the network exists 💔.

The importance of this aspect of the end-to-end argument with respect to PEPs depends on both the PEP implementation and the types of applications being used. Sometimes coincidentally, but more often by design, PEPs are used in environments where there is no alternate path between the end systems. So a failure of the intermediate node containing a PEP would result in the termination of the connection in any case 🫣. And even when this isn't the case, the risk of losing the connection under regular end-to-end operation may exist, since the connection could break for some other reason, for example a long enough link outage of a last-hop wireless link to the end host 📡. Therefore, users may choose to accept the risk of a PEP crashing in order to take advantage of the performance gains the PEP offers 🏎️. The important thing is that accepting this risk should be under the control of the user (i.e., the user should always have the option to choose end-to-end operation) and, if the user chooses to use the PEP, the user should be aware of the implications a PEP failure has for the applications being used 🫶✨.

4.1.3 End-to-end Reliability 💌

okay so this one is about reliability, and bestie, it's giving "trust but verify." 🫶 Another part of the end-to-end argument is acknowledging that data arrived end-to-end, so you get reliable delivery from one end to the other. An app that wants reliable end-to-end delivery has to build its own end-to-end check and recovery at the application level. According to the end-to-end argument, that is the only way to get reliable end-to-end operation right. If you skip it, the app is breaking the end-to-end argument 🙈. So a properly built app can never fully rely on the transport layer (like TCP) or any other communication subsystem to deliver reliably end-to-end. Not even TCP, sorry TCP 😭💅.

Why? Three reasons, and I'm obsessed with how clean they are ✨.

First, a TCP connection can break down for some reason, and then data gets lost. The app has to recover that data itself 🔥.

Second, TCP's checksum might be considered inadequate, which means corrupted data can slip past TCP without TCP noticing [Pax99]. So the app needs its own check for data corruption 💀.

Third, a TCP acknowledgement only means the data reached the TCP implementation on the other end system. It does not promise the data reached the application on that end system 👀. That's why a well designed application must use an application layer acknowledgement to make sure application layer data really got through end-to-end 🎀.

Now, none of this means TCP is useless, btw. A reliable transport protocol (that's TCP) still lets you do really important stuff, like congestion control, efficiently for an application 🌸. So it's not "TCP bad," it's "TCP is a great helper but not your whole safety net" 🧚.

Now for PEPs. If a PEP acknowledges application data too early (before it gets an application ACK from the other endpoint), end-to-end reliability can't be guaranteed 😱. The good news is that application layer PEPs typically don't do this. They don't send an application ACK to the sender until they've received an application ACK from the receiver ✨. Transport layer PEPs, including TCP PEPs, generally don't mess with end-to-end application layer acknowledgments either, because they let applications run end-to-end 🦋. Still, whoever deploys the PEP (the user and/or network administrator) has to understand how it works so they can understand the risks to end-to-end reliability 🧐.

Some Internet apps don't really run end-to-end in normal operation, so they don't get that end-to-end reliability guarantee in the first place 🫠. Email is the classic example. Internet email delivery often goes through relay Mail Transfer Agents, which are relay Simple Mail Transfer Protocol (SMTP) servers 📬. Here's how it goes: an originating MTA (SMTP server) sends the mail message to a relay MTA. The relay MTA receives the message, stores it in non-volatile storage (like on disk), and then sends an application level acknowledgement. After that, the relay MTA takes "full responsibility" for delivering the message to the destination SMTP server (maybe through another relay MTA). It keeps trying to forward the message for a pretty long time, typically around 5 days 🗓️. This doesn't give a 100% guarantee of delivery, but reliability is considered "good enough" 🤷‍♀️.

An application layer PEP for this kind of app can acknowledge application data (like a mail message) without really lowering reliability, as long as the PEP follows the same procedure as the regular proxy (like a relay MTA) 🎀. And again, the user and/or network administrator using a PEP like this needs to understand how it operates so they know the reliability risks that come with it 💅.

4.1.4 End-to-end Failure Diagnostics 🔍

okay, next up is diagnostics, and it's the "why is my connection dead??" chapter 😭. Another part of the end-to-end argument is being able to support end-to-end failure diagnostics when problems happen. If a network problem breaks a connection, the endpoints will notice the failure through timeouts ⏰. But a PEP sitting between the two endpoints can delay (sometimes a lot!) the moment one or both endpoints detect the failure 🙃. (Of course, some PEPs are intentionally designed to hide these kinds of failures, as described in Section 3.4 🫣.) How bad the delay is depends on the apps being used. It could be totally fine, or just a little annoying for the end user, or it could go all the way up to hurting mission critical business functions by slowing down switchovers to alternate communication paths 😱.

Debugging tools can get affected by PEPs too! 🛠️ PING (described in [RFC792] and [RFC2151]) is often used to test connectivity. But PING is based on ICMP instead of TCP (it's implemented with ICMP Echo and Reply commands at the network layer), so the network might route PING traffic around the PEP 🙈. That means PING could say an end-to-end path exists between two hosts when it doesn't actually exist for TCP traffic 💀. And even when PING does go through the PEP, what it tells you gets altered. If PING passes transparently through the PEP, PING gives no sign that the PEP exists. And because PING traffic isn't processed the same way TCP traffic is, it may not accurately show the network delay that TCP traffic is experiencing 📉. On the flip side, if the PEP terminates the PING and answers it on behalf of the end host, then the PING only tells you about connectivity to the PEP 🫥. Traceroute (also described in [RFC2151]) gets affected by the PEP too 🧭.

4.2 Asymmetric Routing 🔄

okay routing time, and this one is a lot 🧩. Deploying a PEP usually means traffic to and from the end hosts has to go through the intermediate node(s) where the PEPs live 🛣️. Some networks can't do that, or it might mean the intermediate node has to sit several hops away from the target link edge. That's impractical in many cases and can lead to non-optimal routing 😬.

Here's the plot twist: this restriction doesn't apply to every PEP 🦋. For example, a PEP that's just doing ACK spacing only needs to see one direction of the traffic flow (the direction the ACKs are flowing in) ✨. ACK spacing can be done without ever seeing the actual data flow 🧚.

4.3 Mobile Hosts 📱

main character moment: mobile hosts 🏃‍♀️. In environments where a PEP serves mobile hosts, extra problems can show up, because PEP related state information may need to move to a new PEP node during a handoff 🤯.

When a mobile host moves, it goes through handovers 📶. If the intermediate node and home for the serving PEP changes because of a handover, any state information the PEP keeps that's needed for continuous operation has to be transferred to the new intermediate node. That keeps the connection working 💖. But this takes extra work and overhead, and it may not be possible fast enough, especially if the host moves a lot across the cell boundaries of a wireless network 🌀. And if the mobile host moves to another IP network, routing to and from it may have to change so traffic goes through a new PEP node 🗺️.

Right now, mobility is a bigger deal for W-LAN networks than for W-WAN networks 📡. Currently, a W-WAN base station usually doesn't give the mobile host its connection point to the wireline Internet. (A W-WAN base station may not even have an IP stack, lol 💀.) Instead, the W-WAN network handles mobility while the connection point to the wireline Internet stays the same as the host moves. So PEP state handover usually isn't needed in most W-WAN networks when the host moves 🥰. But that's generally not true for W-LAN networks, and even with W-WAN networks, the user and/or network administrator using a PEP needs to be aware of how the W-WAN base stations and the PEP work together, in case W-WAN PEP state handoff becomes necessary in the future 🔮.

4.4 Scalability 📈

real talk, PEPs are kind of thirsty for resources 😩. Because a PEP usually processes packet information above the IP layer, it needs more processing power per packet than a router does. So PEPs will always be (at least) one step behind routers when it comes to total throughput they can support 🐢. (Processing above the IP layer is also harder to implement in hardware, fyi 🔧.) On top of that, since most PEP implementations need per connection state, PEP memory requirements are generally way higher than a router's 🧠. That means a PEP implementation may have a limit on how many connections it can support, while a router has no such limit 🚧.

More processing power and memory mean scalability issues come with PEPs 🤓. Putting a PEP on a high speed link, or on a link that supports a huge number of connections, may need network topology changes beyond just plugging the PEP into the traffic path 🧩. For example, if one PEP can only handle half the traffic on a link, you may need multiple PEPs running in parallel, which adds complexity to the network configuration to split traffic between them 🥴.

4.5 Other Implications of Using PEPs 🌈

okay wrap-up vibes for this section ✨. This document describes some significant implications of using Performance Enhancing Proxies. But the list here isn't necessarily exhaustive 🙃. Some other possible implications include using PEPs in multi-homing environments and using PEPs with respect to Quality of Service (QoS) transparency 🎀. For example, there may be potential interaction with the priority-based multiplexing mechanism described in Section 3.5 and the use of differentiated services [RFC2475] 🤝.

So, users and network administrators who want to deploy a PEP should look at more than just the implications listed in this document. They should also look at the overall impact (positive and negative) the PEP will have on their applications and network infrastructure, both now and later, when new applications get added or the network infrastructure needs changes 🔭💜.

5. PEP Environment Examples 🌍

the fun part is coming 🎉. The following sections describe examples of environments where PEPs are currently used to improve performance. The examples show how the various PEP types and PEP mechanisms described earlier in the document are used, and they help explain why those were developed and why people use them 🧡.

5.1. VSAT Environments 🛰️

okay so VSATs are currently run on geosynchronous satellites 🛰️✨ The usual setup is a star topology, bestie. There's one big hub earth station in the middle, and VSATs sit at the remote sites around it. Data going from the hub out to the remote sites travels on an outroute 📡 Data going the other way, from remote sites back to the hub, goes over one or more inroutes. VSAT links are highly asymmetric, and the outroute is typically much bigger than the inroute 💅 (You can have several inroutes per outroute, but any single VSAT can only use one inroute at a time, which is what makes the link asymmetric 🙈)

VSAT networks are mostly used for private networks (aka intranets) for enterprises like corporations with sites spread all over the place 🏢 They're rarely, if ever, used for Internet connectivity, except at the edge of the Internet, meaning as the last hop 🌐 Usually the connection to the Internet happens at the VSAT hub site, using firewalls and, when needed, NAT [RFC2663] devices 🔥

5.1.1. VSAT Network Characteristics 📊

okay so for TCP performance, VSAT networks show a subset of the satellite characteristics written up in [RFC2488] 💖

Long feedback loops 🔁

Propagation delay from sender to receiver in a geosynchronous satellite network can range from 240 to 280 milliseconds, depending on where the sending and receiving sites are in the satellite footprint 🛰️ That makes the round trip time, just from propagation delay, at least 480 milliseconds 😭 Queueing delay and delay from shared channel access methods can sometimes push the total delay up to a few seconds, literally 💀

Large bandwidth*delay products 📦

VSAT networks can support capacity from a few kilobits per second all the way up to multiple megabits per second 🚀 Combine that with the already long round trip time and TCP has to keep a huge number of packets "in flight" to fully use the satellite link 🦋

Asymmetric capacity ⚖️

Like we said above, the outroute of a VSAT network is usually way bigger than an inroute 👑 Even with multiple inroutes in a network, one VSAT can only access one inroute at a time. So the incoming (outroute) and outgoing (inroute) capacity for a VSAT is often super asymmetric 🙃 Outroute capacity has been growing in recent years, and ratios of 400 to 1 or greater are becoming more and more common 📈 With a TCP maximum segment size of 1460 bytes and delayed acknowledgments [RFC1122] in use, the ratio of IP packet bytes for data to IP packet bytes for ACKs is only (3000 to 40) 75 to 1 🥲

So inroute capacity for carrying ACKs can have a big impact on TCP performance 🎀 (The issue of asymmetric link impact on TCP performance gets a deeper dive in [BPK97] 📚)

With respect to the other satellite characteristics listed in [RFC2488], VSAT networks typically don't suffer from intermittent connectivity or variable round trip times ✨ They also usually include a lot of error correction coding, so the bit error rate is very low during clear sky conditions, approaching the bit error rate of a typical terrestrial network 🌤️ In severe weather the bit error rate may increase significantly, but those conditions are rare (when you look at overall network availability) 🌧️ VSAT networks are generally engineered to keep working in those conditions, but they aren't built to optimize performance during them 🫶

5.1.2. VSAT Network PEP Implementations 🧩

Performance Enhancing Proxies (PEPs) for VSAT networks mostly focus on boosting throughput for stuff like FTP and HTTP web page retrievals 🌸 To a lesser degree, they also work on improving interactive response time for small transactions ⭐

There isn't one dominant PEP implementation for VSAT networks 😌 Each VSAT vendor tends to make their own version of PEP functionality, built into their other VSAT product features 💅 [HNS] and [SPACENET] describe VSAT products with integrated PEP capabilities 📝 There are also third party PEP implementations made to work with VSAT networks. These run on nodes outside the VSAT network, at the hub and remote sites 🏃‍♀️ NettGain [FLASH] and Venturi [FOURELLE] are examples of these products 🌈 VSAT network PEP implementations generally share these characteristics:

• They focus on improving TCP performance 💜
• They use an asymmetric distributed implementation 🧠
• They use a split connection approach with local acknowledgments and local retransmissions 🔁
• They support some form of compression to reduce the amount of bandwidth required, with emphasis on saving inroute bandwidth 🗜️

The key differentiators between VSAT network PEP implementations are:

• The maximum throughput they attempt to support, which is mainly a function of the amount of buffer space they use 🪣
• The protocol used over the satellite link. Some implementations use a modified version of TCP, while others use a proprietary protocol running on top of UDP 🎭
• The type of compression used. Third party VSAT network PEP implementations generally focus on application (e.g., HTTP) specific compression algorithms, while PEP implementations integrated into the VSAT network generally focus on link specific compression 🎀

PEP implementations integrated into a VSAT product are generally transparent to the end systems 🫥 Third party PEP implementations used with VSAT networks usually need configuration changes in the remote site end systems so TCP packets get routed to the remote site proxies, but they don't need changes at the hub site end systems 🛠️ In some cases, the PEP implementation is actually built transparently into the end system node itself, using a "bump in the stack" approach 🧚 In all cases though, using a PEP is non-transparent to the user. The user is aware when a PEP implementation is being used to boost performance 👀

⚠️ Heads up: even when the PEP is hidden from the end systems, the user can still tell it's there, so don't assume it's invisible just because the applications don't notice it.

5.1.3. VSAT Network PEP Motivation 💡

Since the early stages of their deployment, VSAT networks have supported local termination of a protocol (e.g., SDLC and X.25) on each side of the satellite link. This hides the satellite link from the applications using the protocol 🙌 So when LAN capabilities were added to VSAT networks, VSAT customers expected, and honestly demanded, similar techniques for improving the performance of IP based traffic, especially TCP traffic 🔥

As mentioned in Section 5.1, VSAT networks are mainly used for intranets, with Internet connectivity limited to and closely controlled at the hub site of the VSAT network 🏠 So VSAT customers aren't as affected (or at least they perceive they aren't) by the Internet related implications of using PEPs as other technologies are 🤷‍♀️ What matters more to VSAT customers is optimizing the network itself 💖 VSAT customers in general prefer that the network does the optimizing, rather than making changes to their own equipment, like enabling the TCP scaled window option 🙅‍♀️ They'd rather optimize their end system configuration for local communications related to their mission critical functions, and let the VSAT network hide the satellite link as much as possible 🧸 VSAT network vendors have also used PEP functionality to offer value added "services" to customers, like extending the useful life of older equipment, including older, "non-modern" TCP stacks ⏳

Of course, as the line between intranets and the Internet keeps fading, the implications of using PEPs start to matter more for VSAT networks 🌊 For example, twelve years ago security wasn't a major concern, because the equipment cost of intercepting VSAT traffic was relatively high 💸 Now, technology has advanced and that cost is much less prohibitive 😬 Because PEP functionality in VSAT networks prevents the use of IPsec, customers have to rely on higher layer security mechanisms like TLS, or on proprietary security mechanisms built into the VSAT networks themselves. That's because many applications currently can't make use of the standardized higher layer security mechanisms (or simply don't) 🔐 This, in turn, affects the cost of the VSAT network, and affects how well customers can make use of Internet based capabilities 💅

5.2. W-WAN Environments 📱

In mobile wireless WAN (W-WAN) environments, the wireless link is typically used as the last-hop link to the end user 📶 W-WANs include networks such as GSM [GSM], GPRS [GPRS],[BW97], CDPD [CDPD], IS-95 [CDMA], RichoNet, and PHS 🌏 Many of these networks, but not all, were designed to provide mobile telephone voice service in the first place and include data services as well, or they evolved from a mobile telephone network 📞

5.2.1. W-WAN Network Characteristics 🌀

W-WAN links typically show some combination of these link characteristics:

• Low bandwidth (on some links, the available bandwidth might be as low as a few hundred bits/sec) 🐢
• High latency (a minimum round-trip delay close to one second is not exceptional) 😩
• High BER resulting in frame or packet losses, or long variable delays due to local link-layer error recovery 💥
• Some W-WAN links have a lot of internal buffer space, which tends to accumulate data, resulting in increased round-trip delay due to long (and variable) queuing delays 🪣
• On some W-WAN links, users may share common channels for their data packet delivery, which in turn may cause unexpected delays to a user's packet delivery due to simultaneous use of the same channel resources by other users 🙃
• Unexpected link disconnections (or intermittent link outages) may occur frequently, and the period of disconnection may last a very long time 💀
• (Re)setting the link-connection up may take a long time (several tens of seconds or even minutes) ⏳
• The W-WAN network typically takes care of terminal mobility: the connection point to the Internet is retained while the user moves with the mobile host 🚶‍♀️✨
• The use of most W-WAN links is expensive. Many service providers apply time-based charging 💸

5.2.2. W-WAN PEP Implementations 🦋

Performance Enhancing Proxies for W-WAN environments generally focus on improving interactive response time, but they also aim to improve throughput, mainly by reducing the transfer volume over the inherently slow link in various ways 🐌💨 To do this, enhancements are typically applied at almost all protocol layers 🧱🌈

5.2.2.1 Mowgli System 🦋

okay so the Mowgli system [KRA94] is one of the OG approaches to deal with the problems that come with low bandwidth W-WAN links 😭 it's literally a classic in this space 💅

the indirect approach in Mowgli isn't limited to one layer like a lot of other split connection approaches. it involves all protocol layers 🔥 the basic architecture is split TCP (UDP is also supported) plus full support for application layer proxies, with a distributed PEP approach. an application layer proxy pair may be added between a client and server: the agent (local proxy) on a mobile host, and the proxy on an intermediate node that gives the mobile host its connection to the wireline Internet. such a pair may be either explicit or fully transparent to the applications, but it is, at all times, under end-user control. so the user gets to pick which traffic goes through the PEP and choose end-to-end IP for everything else 🎀

to let legacy applications run unmodified and without recompilation (no cap, this is the iconic part ✨), the socket layer on the mobile host is slightly modified. it connects the apps that are set up to go through the PEP to a local agent, and it keeps the original TCP/IP socket semantics. there are two types of application layer agent-proxy pairs you can configure for mobile host use 🌸

a generic pair works with any application and simply provides split transport service, with some optional generic enhancements like compression. an application-specific pair can be retailed for any application or a group of applications that are able to take leverage on the same kind of enhancements. a good example is the Mowgli WWW system, which improves the user perceived response time of Web browsing a lot 😍 mainly by reducing the transfer volume and the number of round trips over the wireless link [LAKLR95], [LHKR96].

Mowgli also gives you an option to replace the TCP/IP core protocols on the last-hop link with a custom protocol tuned for low-bandwidth W-WAN links [KRLKA97]. this protocol is designed to provide the same transport service, with similar semantics, as regular TCP and UDP provide, but it uses a different protocol implementation that can freely apply any appropriate protocol mechanisms without being constrained by the current TCP/IP packet format or protocol operation 🧚 as this protocol is required to operate over a single logical link only, it could partially combine the protocol control information and protocol operation of the link, network, and transport layers. it can also run on top of various link services: different raw link services, PPP, IP, or even a single TCP connection used as a link service, with "TCP multiplexing" implemented over it. in all other cases, except when the protocol is configured to operate on top of raw (wireless) link service, IP may co-exist with the custom protocol, allowing simultaneous end-to-end IP delivery for the traffic not traversing through the PEP implementation 💖

the custom protocol can also run in different operation modes that turn certain protocol functions on or off depending on the underlying link service. for example, if the underlying link service provides reliable data delivery, the checksum and the window-based error recovery can be turned off, which cuts the protocol overhead. only a very simple recovery mechanism is needed to recover from an unexpected link disconnection. this is why the design could use an extremely efficient header encoding (only 1-3 bytes per packet in a typical case), reduce the number of round trips significantly, and easily add features that are useful with low-bandwidth W-WAN links ✨ those features include:

• suspending protocol operation during periods of link disconnection or link outage, with fast start once the link is operational again
• priority-based multiplexing of user data over the W-WAN link, so interactive applications get link capacity in a timely manner even when bandwidth-intensive background transfers are happening
• link-level flow control, to stop data from piling up in the W-WAN link internal buffers

if desired, regular TCP/IP transport can be used on the last-hop link instead, possibly with corresponding protocol modifications in TCP (and UDP) that tune it to be more suitable for W-WAN links 🦋

5.2.2.2 Wireless Application Protocol (WAP) 📱

the Mowgli system was designed to support mobile hosts attached to the Internet over constrained links, but it didn't address the specific challenges of low-end mobile devices 🙈 many mobile wireless devices are power, memory, and processing constrained, and the communication links to these devices have lower bandwidth and less stable connections. these limitations led designers to develop the Wireless Application Protocol (WAP). WAP specifies an application framework and network protocols meant to work across differing narrowband wireless network technologies, bringing Internet content and advanced data services to low-end digital cellular phones and other mobile wireless terminals, such as pagers and PDAs 💜

the WAP model consists of a WAP client (mobile terminal), a WAP proxy, and an origin server. it requires a WAP proxy between the WAP client and the server on the Internet. WAP uses a layered, scalable architecture [WAPARCH], specifying the following five protocol layers to be used between the terminal and the proxy: Application Layer (WAE) [WAPWAE], Session Layer (WSP) [WAPWSP], Transaction Layer (WTP) [WAPWTP], Security Layer (WTLS) [WAPWTLS], and Transport Layer (WDP) [WAPWDP]. standard Internet protocols are used between the proxy and the origin server. if the origin server includes WAP proxy functionality, it is called a WAP Server 🌈

in a typical scenario, a WAP client sends an encoded WAP request to a WAP proxy. the WAP proxy translates the WAP request into a WWW (HTTP) request, performing the required protocol conversions, and submits it to a standard web server on the Internet. after the web server responds to the WAP proxy, the response is encoded into a more compact binary format to decrease the size of the data over the air. this encoded response is then forwarded to the WAP client [WAPPROXY] 📲

WAP operates over a variety of bearer datagram services. when communicating over these bearer services, the WAP transport layer (WDP) is always used between the WAP client and WAP proxy, and it provides port addressed datagram service to the higher WAP layers. if the bearer service supports IP (e.g., GSM-CSD, GSM-GPRS, IS-136, CDPD), UDP is used as the datagram protocol. however, if the bearer service does not support IP (e.g., GSM-SMS, GSM-USSD, GSM Cell Broadcast, CDMS-SMS, TETRA-SDS), WDP implements the required datagram protocol as an adaptation layer between the bearer network and the protocol stack 🧩

the use of the other layers depends on the port number. WAP has registered a set of well-known ports with IANA. the port number the application selects for communication between a WAP client and proxy defines the other layers to be used at each end 🔐 the security layer, WTLS, provides privacy, data integrity and authentication. its functionality is similar to TLS 1.0 [RFC2246], extended with datagram support, an optimized handshake, and dynamic key refreshing. if the origin server includes WAP proxy functionality, it might be used to facilitate end-to-end security solutions. otherwise, it provides security between the mobile terminal and the proxy.

the transaction layer, WTP, is message based, without connection establishment and tear down. it supports three types of transaction classes: an unconfirmed request (unidirectional), a reliable (confirmed) request (unidirectional), and a reliable (confirmed) request-reply transaction. data is carried in the first packet, and the 3-way handshake is eliminated to reduce latencies 🏃‍♀️ in addition, acknowledgments, retransmission, and flow control are provided. it allows more than one outstanding transaction at a time. it handles the bearer dependence of a transfer, e.g., it selects timeout values and packet sizes according to the bearer. unfortunately, WTP uses fixed retransmission timers and does not include congestion control, which is a potential problem area as the use of WAP increases [RFC3002] 😬

the session layer, WSP, supports binary encoded HTTP 1.1 with some extensions, such as long living sessions with suspend/resume facility and state handling, header caching, and push facility. on top of the architecture sits the application environment (WAE) 👑

5.2.3 W-WAN PEP Motivation 💅

as indicated in Section 5.2.1, W-WAN networks typically offer very low bandwidth connections with high latency, relatively frequent periods of link disconnection, and they are usually expensive to use 💸 therefore, the transfer volume and extra round-trips, such as those associated with TCP connection setup and teardown, must be reduced. the slow W-WAN link should be efficiently shielded from excess traffic and global (wired) Internet congestion to make Internet access usable and economical. furthermore, interactive traffic must be transmitted in a timely manner even if there are other simultaneous bandwidth intensive (background) transfers, and during periods with connectivity the link must be kept fully utilized, due to the expense of use. in addition, the (long) periods of link disconnection must not abort active (bulk data) transfers, if an end-user so desires 🥺

as (all) applications cannot be made mobility/W-WAN aware in a short time frame, or maybe ever, support for mobile W-WAN use should be implemented in a way that allows most applications, at least those running on fixed Internet hosts, to continue their operation unmodified 🫶

5.3 W-LAN Environments ✨

wireless LANs (W-LAN) are typically organized in a cellular topology, where an access point with a W-LAN transceiver controls a single cell 📡 a cell is defined in terms of the coverage area of the base station. the access points are directly connected to the wired network. the access point in each cell is responsible for forwarding packets to and from the hosts located in that cell. often, the hosts with W-LAN transceivers are mobile. when such a mobile host moves from one cell to another, the responsibility for forwarding packets between the wired network and the mobile host must be transferred to the access point of the new cell. this is known as a handoff 🦋 many W-LAN systems also support an operation mode enabling ad-hoc networking. in this mode, access points are not necessarily needed, but hosts with W-LAN transceivers can communicate directly with other hosts within the transceiver's transmission range 🌸

5.3.1 W-LAN Network Characteristics ✨

okay so wireless LANs are giving speed 🏎️💨. Most of them hit around 1 Mbps to 11 Mbps right now, but higher speeds up to 54 Mbps or even faster are expected to become common later. The round-trip delay is only a few milliseconds or tens of milliseconds, which is honestly so fast 😭. Examples of W-LANs include IEEE 802.11, HomeRF, and Hiperlan. Wireless personal area networks (WPAN) like Bluetooth can use the same PEP techniques too 🦋.

BUT the vibe is a little chaotic 🙈. W-LANs are error-prone because of bit errors, collisions, and link outages. Consecutive packet losses can also happen during handoffs. Most W-LAN MAC protocols do low-level retransmissions, which is a whole shield for the upper layers because it hides most of the losses 🛡️. Still, the losses that can't be fixed, the retransmission latency, and the link outages all still reach the upper layers. TCP performance over a W-LAN, or over any network path that includes a W-LAN link, is likely to suffer because of this ngl 💀.

Here's the tea ☕. TCP thinks these packet losses mean the network is congested, so the TCP sender shrinks its congestion window. It often has to time out to recover from the back-to-back losses. The result is often end-to-end performance that is unacceptably bad 😩.

5.3.2 W-LAN PEP Implementations: Snoop 🧚

Berkeley's Snoop protocol [SNOOP] is a TCP-specific approach, and it's so iconic 👑. A TCP-aware module called a Snoop agent sits at the W-LAN base station, which acts as the last-hop router to the mobile host. Snoop wants to keep the TCP end-to-end semantics intact, which is very main character energy 💅. The Snoop agent watches every packet that goes through the base station in either direction and keeps soft state for each TCP connection. It is an asymmetric PEP implementation, meaning it treats TCP data and ACK channels differently, and it also treats the uplink (from the mobile host) differently from the downlink (to the mobile host) TCP segments 🌈.

For a data transfer going to a mobile host, the Snoop agent caches the unacknowledged TCP data segments that it forwards to the TCP receiver, and it keeps an eye on the matching ACKs. It does two things:

1. It retransmits any lost data segments locally, using local timers and TCP duplicate ACKs to spot the loss, instead of waiting for the TCP sender to do it end-to-end 🔁.
2. It suppresses the duplicate ACKs on their way from the mobile host back to the sender, which stops fast retransmit and congestion avoidance from kicking in at the sender 🙅‍♀️.

Suppressing the duplicate ACKs is required, and here's why it matters 🫶. The Snoop agent retransmits a packet locally, so without suppression the TCP sender would make unnecessary fast retransmits. Picture a system with the Snoop agent, a TCP sender S that sends packets to receiver R through a base station BS. Say S sends packets A, B, C, D, E (in that order), and BS forwards them to the wireless receiver R. Now say the first transmission of packet B gets lost to errors on the wireless link. R receives packets A, C, D, E, and then B, in that order. Packets C, D, and E each trigger duplicate ACKs. When S gets three duplicate ACKs, it triggers fast retransmit, which means it retransmits and shrinks its congestion window 📉. The Snoop agent also retransmits B locally once it gets three duplicate ACKs. So the fast retransmit at S still happens even though the wireless link already did a local retransmit, and that drags throughput down 😤. Snoop fixes this by dropping the TCP duplicate ACKs appropriately at BS 🎀.

For a data transfer from a mobile host, the Snoop agent detects packet losses on the wireless link by watching the data segments it forwards. Then it uses either Negative Acknowledgements (NAK) locally or Explicit Loss Notifications (ELN) to tell the mobile sender that the loss wasn't caused by congestion. That lets the sender retransmit without setting off the normal congestion control procedures 💖. To make this work, changes at the mobile host are required 🛠️.

When a Snoop agent uses NAKs to tell the TCP sender about losses on the wireless link, one way to do it is to use the Selective Acknowledgment (SACK) option of TCP [RFC2018]. This requires enabling SACK processing at the mobile host 📝. The Snoop agent sends a TCP SACK when it detects a hole in the transmission sequence from the mobile host, or when it hasn't received any new packets from the mobile host for a certain amount of time ⏰. This approach depends on the advisory nature of SACKs. The mobile sender is advised to retransmit the missing segments that the SACK points out, but it must not assume successful end-to-end delivery of the segments acknowledged with SACK, because those segments might still get lost later on the path to the receiver 🫠. Instead, the sender must wait for a cumulative ACK to arrive 🙏.

When the ELN mechanism is used to tell the mobile sender about packet losses, Snoop uses one of the 'unreserved' bits in the TCP header for ELN [SNOOPELN] 🔥. The Snoop agent keeps track of the holes that match segments lost over the wireless link. When a (duplicate) ACK matching a hole in the sequence space arrives from the TCP receiver, the Snoop agent sets the ELN bit on that ACK to show the loss is unrelated to congestion, and then forwards the ACK to the TCP sender. Once the sender gets a certain number of (duplicate) ACKs with ELN (a configurable variable at the mobile host, e.g., two), it retransmits the missing segment without doing any congestion control measures 🚀.

⚠️ Using one of the six bits reserved for future use in the TCP header for ELN is dangerous, bestie. It exercises checks that might not be correctly implemented in TCP stacks, and it may expose bugs 💀.

A scheme like Snoop is only needed if the chance of a fast retransmit caused by wireless errors is non-negligible 🤔. If the wireless link uses link-layer recovery for lost data, then this scheme is not beneficial, sorry not sorry 🙅‍♀️. And if the TCP window tends to stay smaller than four segments, for example because of congestion-related losses on the wired network, the probability that the Snoop agent gets a chance to locally retransmit a lost packet is small. That's because at least three duplicate ACKs are needed to trigger the local retransmission, but with a small window the Snoop agent may not be able to forward three new packets after the lost one, so it can't induce the three duplicate ACKs it needs 😭. On the flip side, when the TCP window is large enough, Snoop can give a significant performance boost compared with standard TCP 📈.

To fix the small-window problem, Snoop proposes that a TCP sender be allowed to transmit a new data segment for each duplicate ACK it receives, as long as the number of duplicate ACKs stays below the threshold for TCP fast retransmission (three duplicate ACKs) 🧠. If the new segment reaches the receiver, it generates another duplicate ACK, which lets the sender transmit yet another data segment. This keeps going until enough duplicate ACKs pile up to trigger TCP fast retransmission. This proposal is the same as the "Limited Transfer" proposal [RFC3042], which was recently forwarded to the standards track 🎉. The catch is that to benefit from this solution, it needs to be deployed on TCP senders, so it isn't ready for use in a short time frame ⏳.

Snoop needs the intermediate node (the base station) to examine and operate on the traffic between the mobile host and the other end host on the wired Internet. So Snoop does not work if the IP traffic is encrypted 🔐. Possible solutions involve:

• making the Snoop agent a party to the security association between the client and the server;
• IPsec tunneling mode, terminated at the Snooping base station.

However, these techniques require that users trust base stations 🫣.

Snoop also requires that both the data and the corresponding ACKs traverse the same base station 🚦. Plus, the Snoop agent may duplicate efforts by the link layer, since it retransmits the TCP data segments "at the transport layer" across the wireless link 🙃. (Snoop has been described by its designers as a TCP-aware link layer. This is the right approach: the link and network layers can be much more aware of each other than strict layering suggests 💜.)

5.3.3 W-LAN PEP Motivation 🌸

Wireless LANs suffer from an error-prone wireless channel 😵‍💫. The errors are typically bursty, and channel conditions can change rapidly because of mobility and environmental changes. Packets get dropped from bit errors or during handovers, and periods of link outage can happen too. Even though the typical MAC does retransmissions, dropped packets, outages, and retransmission latency can still have serious performance implications for IP performance, especially for TCP 😩.

PEPs can be used to ease problems caused by packet losses, to protect TCP from link outages, and to add priority multiplexing ✨. Techniques like Snoop are integrated right into access points, while priority and compression schemes are distributed across the W-LAN 🫶.

6. Security Considerations 🔐

The use of Performance Enhancing Proxies introduces several issues that impact security. First, (as described in detail in Section 4.1.1,) using PEPs and using IPsec is generally mutually exclusive 🙅‍♀️. Unless the PEP is also both capable and trusted to be the endpoint of an IPsec tunnel (and the use of an IPsec tunnel is deemed good enough security for the applicable threat model), a user or network administrator must choose between improved performance and network layer security 😭. In some cases, transport (or higher) layer security can be used together with a PEP to reduce the impact of not having network layer security. But support by applications for transport (or higher) layer security is far from ubiquitous 💀.

The PEP itself also needs to be protected from attack. First, even when IPsec tunnels are used with the PEP, the PEP is a point in the network where traffic is exposed 👀. And because of where it sits in the network, the PEP is an ideal platform from which to launch a denial of service or man-in-the-middle attack 🚨. (Taking the PEP out of action is also a potential denial of service attack in itself.) Therefore, the PEP must be protected (e.g., by a firewall) or must protect itself from improper access by an attacker, just like any other device that resides in a network 🛡️💅.

7. IANA Considerations 🎀

This document is an informational overview document and, as such, does not introduce new nor modify existing name or number spaces managed by IANA 📚.

8. Acknowledgements 🫶

This document grew out of the Internet-Draft "TCP Performance Enhancing Proxy Terminology", RFC 2757 "Long Thin Networks", and work done in the IETF TCPSAT working group 🌈. The authors are indebted to the active members of the PILC working group. In particular, Joe Touch and Mark Allman gave us invaluable feedback on various aspects of the document, and Magdolna Gerendai gave us essential help on the WAP example 💖.

Okay bestie, we're at the reading list 📚✨ These are the receipts for everything the doc talked about, so I kept every single entry exactly as written. Then we get to meet the authors 🫶

9. References ✨

Here's the whole bibliography, word for word, so you can go hunt down any paper or spec that caught your eye 🔍

   [BBKT97]    P. Bhagwat, P. Bhattacharya, A. Krishma, S.K. Tripathi,
               "Using channel state dependent packet scheduling to
               improve TCP throughput over wireless LANs," ACM Wireless
               Networks, March 1997, pp. 91 - 102.  Available at:
               http://www.acm.org/pubs
               /articles/journals/wireless/1997-3-1/p91-bhagwat/p91-
               bhagwat.pdf

   [BPK97]     H. Balakrishnan, V.N. Padmanabhan, R.H. Katz, "The
               Effects of Asymmetry on TCP Performance," Proc. ACM/IEEE
               Mobicom, Budapest, Hungary, September 1997.

   [BW97]      G. Brasche, B. Walke, "Concepts, Services, and Protocols
               of the New GSM Phase 2+ general Packet Radio Service,"
               IEEE Communications Magazine, Vol. 35, No. 8, August
               1997.

   [CDMA]      Electronic Industry Alliance (EIA)/Telecommunications
               Industry Association (TIA), IS-95: Mobile Station-Base
               Station Compatibility Standard for Dual-Mode Wideband
               Spread Spectrum Cellular System, 1993.

   [CDPD]      Wireless Data Forum, CDPD System Specification, Release
               1.1, 1995.

   [CTC+97]    H. Chang, C. Tait, N. Cohen, M. Shapiro, S. Mastrianni,
               R. Floyd, B. Housel, D. Lindquist, "Web Browsing in a
               Wireless Environment: Disconnected and Asynchronous
               Operation in ARTour Web Express," Proc. MobiCom'97,
               Budapest, Hungary, September 1997.

   [FMSBMR98]  D.C. Feldmeier, A.J. McAuley, J.M. Smith, D.S. Bakin,
               W.S. Marcus, T.M. Raleigh, "Protocol Boosters," IEEE
               Journal on Selected Areas of Communication, Vol. 16, No.
               3, April 1998.

   [FLASH]     Flash Networks Ltd., performance boosting products
               technology vendor based in Holmdel, New Jersey.  Website
               at http://www.flashnetworks.com.

   [FOURELLE]  Fourelle Systems, performance boosting products
               technology vendor based in Santa Clara, California.
               Website at http://www.fourelle.com.

   [GPRS]      ETSI, "General Packet Radio Service (GPRS): Service
               Description, Stage 2," GSM03.60, v.6.1.1, August 1998.

   [GSM]       M. Rahnema, "Overview of the GSM system and protocol
               architecture," IEEE Communications Magazine, Vol. 31, No.
               4, pp. 92-100, April 1993.

   [HNS]       Hughes Network Systems, Inc., VSAT technology vendor
               based in Germantown, Maryland.  Website at
               http://www.hns.com.

   [I-TCP]     A. Bakre, B.R. Badrinath, "I-TCP: Indirect TCP for Mobile
               Hosts," Proc. 15th International Conference on
               Distributed Computing Systems (ICDCS), May 1995.

   [KRA94]     M. Kojo, K. Raatikainen, T. Alanko, "Connecting Mobile
               Workstations to the Internet over a Digital Cellular
               Telephone Network," Proc. Workshop on Mobile and Wireless
               Information Systems (MOBIDATA), Rutgers University, NJ,
               November 1994.  Revised version published in Mobile
               Computing, pp. 253-270, Kluwer, 1996.

   [KRLKA97]   M. Kojo, K. Raatikainen, M. Liljeberg, J. Kiiskinen, T.
               Alanko, "An Efficient Transport Service for Slow Wireless
               Telephone Links," IEEE Journal on Selected Areas of
               Communication, Vol. 15, No. 7, September 1997.

   [LAKLR95]   M. Liljeberg, T. Alanko, M. Kojo, H. Laamanen, K.
               Raatikainen, "Optimizing World-Wide Web for Weakly-
               Connected Mobile Workstations: An Indirect Approach,"
               Proc. of the 2nd Int. Workshop on Services in Distributed
               and Networked Environments, Whistler, Canada, pp. 132-
               139, June 1995.

   [LHKR96]    M. Liljeberg, H. Helin, M. Kojo, K. Raatikainen, "Mowgli
               WWW Software: Improved Usability of WWW in Mobile WAN
               Environments," Proc. IEEE Global Internet 1996
               Conference, London, UK, November 1996.

   [M-TCP]     K. Brown, S. Singh, "M-TCP: TCP for Mobile Cellular
               Networks," ACM Computer Communications Review Volume
               27(5), 1997.  Available at
               ftp://ftp.ece.orst.edu/pub/singh/papers/mtcp.ps.gz.

   [Pax99]     V. Paxson, "End-to-End Internet Packet Dynamics,"
               IEEE/ACM Transactions on Networking, Vol. 7, No. 3, 1999,
               pp. 277-292.

   [PILCWEB]   http://pilc.grc.nasa.gov.

   [RFC0792]   Postel, J., "Internet Control Message Protocol", STD 5,
               RFC 792, September 1981.

   [RFC0793]   Postel, J., "Transmission Control Protocol", STD 7, RFC
               793, September 1981.

   [RFC1122]   Braden, R., "Requirements for Internet Hosts --
               Communications Layers", STD 3, RFC 1122, October 1989.

   [RFC1144]   Jacobson, V., "Compressing TCP/IP Headers for Low-Speed
               Serial Links", RFC 1144, February 1990.

   [RFC1323]   Jacobson, V., Braden, R. and D. Borman, "TCP Extensions
               for High Performance", RFC 1323, May 1992.

   [RFC1958]   Carpenter, B., "Architectural Principles of the
               Internet", RFC 1958, June 1996.

   [RFC2018]   Mathis, M., Mahdavi, J., Floyd, S. and A. Romanow, "TCP
               Selective Acknowledgment Options", RFC 2018, October
               1996.

   [RFC2151]   Kessler, G. and S. Shepard, "A Primer On Internet and
               TCP/IP Tools and Utilities", FYI 30, RFC 2151, June 1997.

   [RFC2246]   Dierk, T. and E. Allen, "TLS Protocol Version 1," RFC
               2246, January 1999.

   [RFC2393]   Shacham, A., Monsour, R., Pereira, R. and M. Thomas, "IP
               Payload Compression Protocol (IPcomp)", RFC 2393,
               December 1998.

   [RFC2401]   Kent, S., and R. Atkinson, "Security Architecture for the
               Internet Protocol", RFC 2401, November 1998.

   [RFC2475]   Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z.
               and W. Weiss, "An Architecture for Differentiated
               Services", RFC 2475, December 1998.

   [RFC2488]   Allman, M., Glover, D. and L. Sanchez, "Enhancing TCP
               Over Satellite Channels using Standard Mechanisms", BCP
               28, RFC 2488, January 1999.

   [RFC2507]   Degermark, M., Nordgren, B. and S. Pink, "IP Header
               Compression", RFC 2507, February 1999.

   [RFC2508]   Casner, S. and V. Jacobson, "Compressing IP/UDP/RTP
               Headers for Low-Speed Serial Links", RFC 2508, February
               1999.

   [RFC2509]   Engan, M., Casner, S. and C. Bormann, "IP Header
               Compression over PPP", RFC 2509, February 1999.

   [RFC2663]   Srisuresh, P. and Y. Holdrege, "IP Network Address
               Translator (NAT) Terminology and Considerations", RFC
               2663, August 1999.

   [RFC2760]   Allman, M., Dawkins, S., Glover, D., Griner, J.,
               Henderson, T., Heidemann, J., Kruse, H., Ostermann, S.,
               Scott, K., Semke, J., Touch, J. and D. Tran, "Ongoing TCP
               Research Related to Satellites", RFC 2760, February 2000.

   [RFC3002]   Mitzel, D., "Overview of 2000 IAB Wireless
               Internetworking Workshop", RFC 3002, December 2000.

   [RFC3042]   Allman, M., Balakrishnan, H. and S. Floyd, "Enhancing
               TCP's Loss Recovery Using Limited Transmit", RFC 3042,
               January 2001.

   [SHEL00]    Z. Shelby, T. Saarinen, P. Mahonen, D. Melpignano, A.
               Marshall, L. Munoz, "Wireless IPv6 Networks - WINE," IST
               Mobile Summit, Ireland, October 2000.

   [SNOOP]     H. Balakrishnan, S. Seshan, E. Amir, R. Katz, "Improving
               TCP/IP Performance over Wireless Networks," Proc. 1st ACM
               Conference on Mobile Communications and Networking
               (Mobicom), Berkeley, California, November 1995.

   [SNOOPELN]  H. Balakrishnan, R. Katz, "Explicit Loss Notification and
               Wireless Web Performance," Proc. IEEE Globecom 1998,
               Internet Mini-Conference, Sydney, Australia, November
               1998.

   [SPACENET]  Spacenet, VSAT technology vendor based in Mclean,
               Virginia.  Website at http://www.spacenet.com.

   [SRC84]     J.H. Saltzer, D.P. Reed, D.D. Clark, "End-To-End
               Arguments in System Design," ACM TOCS, Vol. 2, No. 4, pp.
               277-288, November 1984.

   [WAPARCH]   Wireless Application Protocol Architecture Specification,
               April 1998, http://www.wapforum.org.

   [WAPPROXY]  Wireless Application Protocol Push Proxy Gateway Service
               Specification, August 1999, http://www.wapforum.org.

   [WAPWAE]    Wireless Application Protocol Wireless Application
               Environment Overview, March 2000,
               http://www.wapforum.org.

   [WAPWDP]    Wireless Application Protocol Wireless Datagram Protocol
               Specification, February 2000, http://www.wapforum.org.

   [WAPWSP]    Wireless Application Protocol Wireless Session Protocol
               Specification, May 2000, http://www.wapforum.org.

   [WAPWTLS]   Wireless Application Protocol Wireless Transport Layer
               Security Specification, February 2000,
               http://www.wapforum.org.

   [WAPWTP]    Wireless Application Protocol Wireless Transaction
               Protocol Specification, February 2000,
               http://www.wapforum.org.

   [Zhang00]   Y. Zhang, B. Singh, "A Multi-Layer IPsec Protocol," Proc.
               proceedings of 9th USENIX Security Symposium, Denver,
               Colorado, August 2000.  Available at
               http://www.wins.hrl.com/people/ygz/papers/usenix00.html.

10. Authors' Addresses 🎀

Here are the people who wrote this doc, with their contact details so you can reach out if something's unclear 🫶 I kept every line exactly as it was, typos and all.

   Questions about this document may be directed to:

   John Border
   Hughes Network Systems
   11717 Exploration Lane
   Germantown, Maryland  20876

   Phone: +1-301-548-6819
   Fax:   +1-301-548-1196
   EMail: border@hns.com
   Markku Kojo
   Department of Computer Science
   University of Helsinki
   P.O. Box 26 (Teollisuuskatu 23)
   FIN-00014 HELSINKI
   Finland

   Phone: +358-9-1914-4179
   Fax:   +358-9-1914-4441
   EMail: kojo@cs.helsinki.fi


   Jim Griner
   NASA Glenn Research Center
   MS: 54-5
   21000 Brookpark Orad
   Cleveland, Ohio  44135-3191

   Phone: +1-216-433-5787
   Fax:   +1-216-433-8705
   EMail: jgriner@grc.nasa.gov


   Gabriel Montenegro
   Sun Microsystems Laboratories, Europe
   29, chemin du Vieux Chene
   38240 Meylan, FRANCE

   Phone: +33 476 18 80 45
   EMail: gab@sun.com


   Zach Shelby
   University of Oulu
   Center for Wireless Communications
   PO Box 4500
   FIN-90014
   Finland

   Phone: +358-40-779-6297
   EMail: zach.shelby@ee.oulu.fi

Appendix A - PEP Terminology Summary ✨

okay bestie, this appendix is basically a PEP glossary, and it's a little bit chaotic because a lot of these words mean something different in PEP land than in regular networking 💅 Some of them are the same words you'd use anyway, but with a PEP twist, so read these carefully 🧚

ACK filtering 🫶 Removing acknowledgments to stop a low speed link from getting congested. This is usually used on paths that include a super asymmetric link. People also call it ACK reduction. See Section 3.1.4.

ACK spacing 🦋 Delaying the forwarding of acknowledgments so they're spaced out properly. For example, this helps keep TCP data from coming in big bursty clumps. See Section 3.1.1.

application layer PEP 🌸 A Performance Enhancing Proxy operating above the transport layer. It might be aimed at improving application protocol performance, transport protocol performance, or both. Described in detail in Section 2.1.2.

asymmetric link 🌈 A link with different rates for the forward channel (the one carrying data segments) and the back, or return, channel (the one carrying ACKs). Very main character energy for a link, honestly 👑

available bandwidth ✨ The total capacity of a link that's free to carry information at any given moment. It can be lower than the raw bandwidth because of competing traffic 😭

bandwidth utilization 🍓 The amount of information that's actually delivered over a link in a given period. This is usually written as a percent of the link's raw bandwidth. ngl it's the honest version of "how full is this link really" 💜

gateway 💀 This one has a few meanings depending on the context, so don't panic:

• An access point to a particular link;
• A device that can start and end connections on behalf of a user or end system (for example, a firewall or proxy).

It's not necessarily a router, but it could be one 🔥

in flight (data) 🎀 Data that's been sent but not acknowledged yet. More precisely, it's data that was sent and the sender still hasn't received the acknowledgement for it. Waiting vibes 🙈

link layer PEP 🦋 A Performance Enhancing Proxy operating below the network layer.

local acknowledgement 🫶 When an entity sitting in the path between two end systems generates acknowledgments, so the sending system can send more data without waiting for the end-to-end ACKs. This is described in the TCP context in Section 3.1.2. It's giving middleman who keeps things moving 🏃‍♀️

performance enhancing proxy 👑 An entity in the network that acts on behalf of an end system or user (with or without that end system or user knowing about it) in order to enhance protocol performance. Section 2 describes the different types of performance enhancing proxies. Section 3 describes the mechanisms they use to make things faster ✨

raw bandwidth 🌸 The total capacity of an unloaded link that's available to carry information. This is the "empty road" number 🛣️

Snoop 🧚 A TCP-aware link layer developed for wireless packet radio and cellular networks. It works by caching segments at a wireless base station. If the base station sees duplicate acknowledgments for a segment it has cached, it retransmits the missing segment. While it's doing that, it suppresses the duplicate acknowledgment stream that would be forwarded back to the sender, until the wireless receiver starts acknowledging new data. This is described in detail in Section 5.3.2 and [SNOOP]. Snoop is literally the sneakiest helper and I'm obsessed 🙈💜

split connection 🍓 A connection that gets terminated before it reaches the intended destination end system, so another connection can be started toward that end system. This lets different parts of the path of the originally intended connection use different connection characteristics. See Section 2.4 🌈

TCP PEP 🎀 A Performance Enhancing Proxy operating at the transport layer with TCP. It's aimed at improving TCP performance 💅

TCP splitting 🦋 Using one or more split TCP connections to improve TCP performance.

TCP spoofing 👀 Sometimes used as a synonym for TCP PEP, but more accurately, TCP spoofing refers to using mechanisms that are transparent (to the TCP stacks in the end systems) to improve TCP performance. See Section 2.1.1. Not me being confused by the name, it's a whole vibe 🙃

transparent ✨ In the context of a PEP, transparent means it doesn't require changes to be made to the end systems, transport endpoints, and/or applications involved in a connection. See Section 2.5 for a more detailed explanation 🫶

transport layer PEP 🌸 A Performance Enhancing Proxy operating at the transport layer. Described in detail in Section 2.1.1.

tunneling 🧚 In the context of PEPs, tunneling is the process of wrapping a packet for transmission over a particular link between two PEPs. See Section 3.2 📦

WAP 💖 The Wireless Application Protocol specifies an application framework and network protocols intended to work across differing narrow-band wireless network technologies. See Section 5.2.2.2 🌈

---

Full Copyright Statement 🎀

okay so this part is the legal stuff, and I'm going to keep every word of it because it matters 🙏 Copyright and permissions are not something to vibe-rewrite 💀

Copyright (C) The Internet Society (2001). All Rights Reserved.

This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English.

The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns.

This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

---

Acknowledgement 🫶

Funding for the RFC Editor function is currently provided by the Internet Society. Thank you to everyone keeping the RFC Editor running, you're the real MVPs 👑✨