RFC 4821

Packetization Layer Path MTU Discovery

๐Ÿ“… published March 2007, from ietf

Packetization Layer Path MTU Discovery ๐ŸŽ€

okay bestie, this is RFC 4821 ๐Ÿฆ‹

  • Authors: M. Mathis and J. Heffner, both from PSC ๐Ÿ’–
  • Stream: Network Working Group
  • Request for Comments: 4821
  • Category: Standards Track โœจ
  • Date: March 2007 ๐ŸŒธ
  • Obsoletes / Updates: the header has no Obsoletes or Updates line. The intro says it does not update RFC 1191 or RFC 1981 ๐Ÿงš

Status of This Memo ๐Ÿ’…

so this doc is an Internet standards track protocol for the Internet community, and it's literally asking for discussion and suggestions ๐Ÿซถ For the standardization state and status, check the current edition of the "Internet Official Protocol Standards" (STD 1). Distribution of this memo is unlimited, so share it with everyone ๐Ÿ’œ

Copyright (C) The IETF Trust (2007).

Abstract โœจ

this is about a sturdy way to do Path MTU Discovery (PMTUD) ๐ŸŒˆ where TCP, or some other Packetization Layer, pokes at an Internet path by sending progressively bigger packets ๐Ÿ“ It's an extension of RFC 1191 and RFC 1981, which cover ICMP-based Path MTU Discovery for IP versions 4 and 6, respectively ๐Ÿ‘‘

Table of Contents ๐Ÿฆ‹

1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
2.  Overview . . . . . . . . . . . . . . . . . . . . . . . . . . .  3
3.  Terminology  . . . . . . . . . . . . . . . . . . . . . . . . .  6
4.  Requirements . . . . . . . . . . . . . . . . . . . . . . . . .  9
5.  Layering . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
  5.1.  Accounting for Header Sizes  . . . . . . . . . . . . . . . 10
  5.2.  Storing PMTU Information . . . . . . . . . . . . . . . . . 11
  5.3.  Accounting for IPsec . . . . . . . . . . . . . . . . . . . 12
  5.4.  Multicast  . . . . . . . . . . . . . . . . . . . . . . . . 12
6.  Common Packetization Properties  . . . . . . . . . . . . . . . 13
  6.1.  Mechanism to Detect Loss . . . . . . . . . . . . . . . . . 13
  6.2.  Generating Probes  . . . . . . . . . . . . . . . . . . . . 13
7.  The Probing Method . . . . . . . . . . . . . . . . . . . . . . 14
  7.1.  Packet Size Ranges . . . . . . . . . . . . . . . . . . . . 14
  7.2.  Selecting Initial Values . . . . . . . . . . . . . . . . . 16
  7.3.  Selecting Probe Size . . . . . . . . . . . . . . . . . . . 17
  7.4.  Probing Preconditions  . . . . . . . . . . . . . . . . . . 18
  7.5.  Conducting a Probe . . . . . . . . . . . . . . . . . . . . 18
  7.6.  Response to Probe Results  . . . . . . . . . . . . . . . . 19
    7.6.1.  Probe Success  . . . . . . . . . . . . . . . . . . . . 19
    7.6.2.  Probe Failure  . . . . . . . . . . . . . . . . . . . . 19
    7.6.3.  Probe Timeout Failure  . . . . . . . . . . . . . . . . 20
    7.6.4.  Probe Inconclusive . . . . . . . . . . . . . . . . . . 20
  7.7.  Full-Stop Timeout  . . . . . . . . . . . . . . . . . . . . 20
  7.8.  MTU Verification . . . . . . . . . . . . . . . . . . . . . 21
8.  Host Fragmentation . . . . . . . . . . . . . . . . . . . . . . 22
9.  Application Probing  . . . . . . . . . . . . . . . . . . . . . 23
10. Specific Packetization Layers  . . . . . . . . . . . . . . . . 23
  10.1. Probing Method Using TCP . . . . . . . . . . . . . . . . . 23
  10.2. Probing Method Using SCTP  . . . . . . . . . . . . . . . . 25
  10.3. Probing Method for IP Fragmentation  . . . . . . . . . . . 26
  10.4. Probing Method Using Applications  . . . . . . . . . . . . 27
11. Security Considerations  . . . . . . . . . . . . . . . . . . . 28
12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 28
  12.1. Normative References . . . . . . . . . . . . . . . . . . . 28
  12.2. Informative References . . . . . . . . . . . . . . . . . . 29
Appendix A.  Acknowledgments . . . . . . . . . . . . . . . . . . . 31

1. Introduction โœจ

okay so this doc describes Packetization Layer Path MTU Discovery (PLPMTUD) ๐Ÿฆ‹ It's an extension of the existing Path MTU Discovery methods in [RFC1191] and [RFC1981]. If ICMP messages aren't around, the right MTU gets figured out by starting with small packets and probing with bigger and bigger ones ๐Ÿ’– Most of the algorithm lives above IP, in the transport layer (like TCP) or another "Packetization Protocol" that's in charge of working out packet boundaries ๐ŸŒธ

this doc does not update RFC 1191 or RFC 1981 ๐Ÿ™ˆ BUT since it works fine without ICMP, it kind of, implicitly, relaxes some of the requirements in those docs ๐Ÿ‘‘

the methods here lean on features of existing protocols ๐Ÿงš They work for lots of transport protocols over IPv4 and IPv6 ๐ŸŒˆ They don't need help from the lower layers (other than being consistent about which packet sizes are acceptable) or from peers ๐Ÿซถ Since only senders use them, implementation variants won't cause interoperability problems ๐Ÿ’…

for clarity, we go with TCP and IPv6 terminology across the board ๐Ÿ“ The terminology section also gives the matching IPv4 terms and concepts, and a few spots call out the details that differ between IPv4 and IPv6 โœจ

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. ๐Ÿ”ฅ

this doc comes out of the Path MTU Discovery (PMTUD) working group of the IETF, and it borrows heavily from RFC 1191 and RFC 1981 for terminology, ideas, and some of the text ๐Ÿ’œ

2. Overview โœจ

okay so PLPMTUD (Packetization Layer Path MTU Discovery) is a way for TCP or other Packetization Protocols to dynamically figure out the MTU of a path by probing with progressively larger packets ๐Ÿ” it works best when it's paired with the ICMP-based Path MTU Discovery mechanism from RFC 1191 and RFC 1981, but it fixes a lot of the robustness problems of the classic techniques because it doesn't depend on ICMP messages actually getting delivered ๐Ÿ’…

this method is for TCP and other transport- or application-level protocols that choose their own packet boundaries (like segment sizes) and have an acknowledgment structure that tells the sender, accurately and quickly, which packets got lost ๐Ÿฆ‹

the general plan is that the Packetization Layer finds the right Path MTU by probing the path with bigger and bigger packets. if a probe packet makes it through, the effective Path MTU gets raised to the probe size ๐ŸŒˆ

if a probe goes missing (with or without an ICMP Packet Too Big message), that's read as an MTU limit and not as a congestion signal ๐Ÿ™ˆ and in this one case only, the Packetization Protocol is allowed to retransmit whatever data is missing without touching the congestion window. main character energy for sure โœจ

if there's a timeout or more packets are lost during probing, the probe counts as inconclusive (the lost probe doesn't necessarily mean it was bigger than the Path MTU, bestie ๐Ÿซถ). those losses get treated like any other congestion indication, so window or rate adjustments are mandatory under the relevant congestion control standards [RFC2914]. probing can resume after a delay that depends on what kind of failure was detected โณ

PLPMTUD uses a searching technique to find the Path MTU. every conclusive probe narrows the search range, either by raising the lower limit when a probe succeeds or lowering the upper limit when one fails, so it converges on the true Path MTU ๐ŸŽฏ for most transport layers, the search should stop once the range is narrow enough that a bigger effective Path MTU isn't worth the search overhead it takes to find it ๐Ÿ’œ

the most likely (and least serious) probe failure is link congestion-related losses while probing. in that case, it's fine to retry a probe of the same size as soon as the Packetization Layer has fully adapted to the congestion and recovered from the losses ๐ŸŒธ in other cases, extra losses or timeouts point to problems with the link or the Packetization Layer, and it's desirable to wait longer depending on how severe the error is ๐Ÿ˜ญ

an optional verification process can be used to catch situations where raising the MTU raises the packet loss rate ๐Ÿ‘€ for example, if a link is striped across several physical channels with inconsistent MTUs, a probe might get delivered even though it's too large for some of those channels. raising the Path MTU to the probe size in that case can cause severe packet loss and abysmal performance ๐Ÿ’€ after raising the MTU, the new size can be verified by monitoring the loss rate ๐Ÿ“

PLPMTUD also brings some flexibility to classical Path MTU Discovery. it can be configured to do just ICMP black hole recovery to make classical PMTUD more robust, or at the other extreme, all ICMP processing can be turned off and PLPMTUD can completely replace classical Path MTU Discovery ๐Ÿงš

classical Path MTU Discovery can hit protocol failures (connection hangs) if ICMP Packet Too Big (PTB) messages aren't delivered or processed for some reason [RFC2923]. with PLPMTUD, classical Path MTU Discovery can be modified to include extra consistency checks without raising the risk of connection hangs from spurious failures of those extra checks ๐Ÿ”ง changes like that to classical Path MTU Discovery are beyond the scope of this document ๐Ÿ™…โ€โ™€๏ธ

in the limiting case, all ICMP PTB messages might be unconditionally ignored, and PLPMTUD can be the only method used to discover the Path MTU. in that setup, PLPMTUD basically parallels congestion control ๐ŸŽ€ an end-to-end transport protocol adjusts properties of the data stream (window size or packet size) and uses packet losses to figure out whether those adjustments were appropriate. this feels more philosophically consistent with the end-to-end principle of the Internet than relying on ICMP messages that contain transcribed headers from multiple protocol layers ๐Ÿซถ

most of the difficulty in implementing PLPMTUD comes from the fact that it has to be implemented in several different places within a single node ๐Ÿงฉ in general, each Packetization Protocol needs its own PLPMTUD implementation. also, the natural way to share Path MTU information between concurrent or subsequent connections is a path information cache in the IP layer, so the various Packetization Protocols need a way to access and update that shared cache ๐Ÿ—‚๏ธ this memo describes PLPMTUD in terms of its primary subsystems without fully describing how they're put together into a complete implementation โœจ

the vast majority of implementation details in this document are recommendations based on experience with earlier versions of Path MTU Discovery ๐Ÿ“š they're motivated by a desire to maximize PLPMTUD's robustness in the less-than-ideal network conditions that exist in the real world ๐ŸŒ

this document does not contain a complete description of an implementation. it only sketches details that don't affect interoperability with other implementations and that have strong externally imposed optimality criteria (like the MTU searching and caching heuristics) ๐Ÿง  other details are spelled out explicitly because there's an obvious alternative implementation that doesn't work well in some (possibly subtle) case ๐Ÿ™ˆ

Section 3 provides a complete glossary of terms ๐Ÿ“–

Section 4 describes the details of PLPMTUD that affect interoperability with other standards or Internet protocols ๐Ÿค

Section 5 describes how to partition PLPMTUD into layers, and how to manage the path information cache in the IP layer ๐Ÿฐ

Section 6 describes the general Packetization Layer properties and features needed to implement PLPMTUD ๐Ÿ› ๏ธ

Section 7 describes how to use probes to search for the Path MTU ๐Ÿ”Ž

Section 8 recommends using IPv4 fragmentation in a configuration that mimics IPv6 functionality, to minimize future problems migrating to IPv6 ๐Ÿฆ‹

Section 9 describes a programming interface for implementing PLPMTUD in applications that choose their own packet boundaries, and for tools to diagnose path problems that interfere with Path MTU Discovery ๐Ÿฉบ

Section 10 discusses implementation details for specific protocols, including TCP ๐Ÿ’–


3. Terminology ๐ŸŽ€

we use the following terms in this document:

IP: either IPv4 [RFC0791] or IPv6 [RFC2460] ๐ŸŒ

Node: a device that implements IP ๐Ÿ“ฑ

Upper layer: a protocol layer immediately above IP. examples are transport protocols like TCP and UDP, control protocols like ICMP, routing protocols like OSPF, and Internet or lower-layer protocols being "tunneled" over (i.e., encapsulated in) IP, like IPX, AppleTalk, or IP itself ๐Ÿš‡

Link: a communication facility or medium over which nodes can communicate at the link layer, i.e., the layer immediately below IP. examples are Ethernets (simple or bridged), PPP links, X.25, Frame Relay, or Asynchronous Transfer Mode (ATM) networks, and Internet (or higher) layer "tunnels", like tunnels over IPv4 or IPv6 ๐Ÿ”— occasionally we use the slightly more general term "lower layer" for this concept ๐Ÿซง

Interface: a node's attachment to a link ๐Ÿ”Œ

Address: an IP layer identifier for an interface or a set of interfaces ๐Ÿ 

Packet: an IP header plus payload ๐Ÿ“ฆ

MTU: Maximum Transmission Unit, the size in bytes of the largest IP packet, including the IP header and payload, that can be transmitted on a link or path. note that this could more properly be called the IP MTU, to be consistent with how other standards organizations use the acronym MTU ๐Ÿ“

Link MTU: the Maximum Transmission Unit, i.e., maximum IP packet size in bytes, that can be conveyed in one piece over a link. be aware that this definition is different from the definition used by other standards organizations ๐Ÿ‘€

for IETF documents, link MTU is uniformly defined as the IP MTU over the link. this includes the IP header, but excludes link layer headers and other framing that isn't part of IP or the IP payload. ๐Ÿšจ

other standards organizations generally define link MTU to include the link layer headers, so keep that in mind, bestie ๐Ÿ’…

Path: the set of links traversed by a packet between a source node and a destination node ๐Ÿ›ฃ๏ธ

Path MTU, or PMTU: the minimum link MTU of all the links in a path between a source node and a destination node ๐Ÿ

Classical Path MTU Discovery: the process described in RFC 1191 and RFC 1981, in which nodes rely on ICMP Packet Too Big (PTB) messages to learn the MTU of a path ๐Ÿ•ฐ๏ธ

Packetization Layer: the layer of the network stack that segments data into packets ๐Ÿง

Effective PMTU: the current estimated value for PMTU used by a Packetization Layer for segmentation ๐Ÿ“

PLPMTUD: Packetization Layer Path MTU Discovery, the method described in this document, which is an extension to classical PMTU Discovery ๐Ÿฆ„

PTB (Packet Too Big) message: an ICMP message reporting that an IP packet is too large to forward. this is the IPv6 term that corresponds to the IPv4 ICMP "Fragmentation Needed and DF Set" message ๐Ÿ“จ

Flow: a context in which MTU Discovery algorithms can be invoked. this is naturally an instance of a Packetization Protocol, for example, one side of a TCP connection ๐ŸŒŠ

MSS: the TCP Maximum Segment Size [RFC0793], the maximum payload size available to the TCP layer. this is typically the Path MTU minus the size of the IP and TCP headers ๐Ÿงฎ

Probe packet: a packet that is being used to test a path for a larger MTU ๐Ÿ•ต๏ธโ€โ™€๏ธ

Probe size: the size of a packet being used to probe for a larger MTU, including IP headers ๐Ÿ“

Probe gap: the payload data that will be lost and need to be retransmitted if the probe is not delivered ๐Ÿ•ณ๏ธ

Leading window: any unacknowledged data in a flow at the time a probe is sent โช

Trailing window: any data in a flow sent after a probe, but before the probe is acknowledged โฉ

Search strategy: the heuristics used to choose successive probe sizes to converge on the proper Path MTU, as described in Section 7.3 ๐Ÿงญ

Full-stop timeout: a timeout where none of the packets transmitted after some event are acknowledged by the receiver, including any retransmissions. this is taken as an indication of some failure condition in the network, such as a routing change onto a link with a smaller MTU. this is described in more detail in Section 7.7 ๐Ÿ›‘

4. Requirements โœจ

okay bestie, this is the rulebook section ๐Ÿ’… first up, all links MUST enforce their MTU, which means any link that might sometimes deliver packets bigger than its rated MTU MUST consistently discard them ๐Ÿ™ˆ no random oversized surprises, periodt.

back in the distant past there were a few network devices that didn't enforce MTU but also couldn't deliver oversized packets reliably ๐Ÿ’€ like some early bit-wise Ethernet repeaters that would forward packets of any size but couldn't do it dependably because their hardware data clock wasn't stable enough. this is the only requirement PLPMTUD puts on the lower layers ๐Ÿซถ and it's spelled out on purpose so nobody standardises or deploys some future tech that doesn't play nice with PLPMTUD. not on my watch ๐Ÿ˜ค

all hosts SHOULD use IPv4 fragmentation in a mode that copies IPv6 functionality โœจ all fragmentation SHOULD happen on the host, and all IPv4 packets, fragments included, SHOULD have the DF bit set so the network won't fragment them again ๐Ÿฆ‹ see Section 8 for the rest of that tea.

the requirements below only apply to implementations that include PLPMTUD ๐ŸŽ€

to use PLPMTUD, a Packetization Layer MUST have a loss reporting mechanism that gives the sender timely and accurate info about which packets got lost in the network ๐Ÿ“ฌ no vibes-based guessing allowed.

normal congestion control algorithms MUST stay in effect under all conditions, except when ONLY an isolated probe packet is detected as lost ๐ŸŒธ in that one specific case, the usual congestion (window or data rate) reduction SHOULD be suppressed. if any other data loss shows up, standard congestion control MUST take place ๐Ÿ’œ no skipping that.

suppressed congestion control MUST be rate limited so it happens less often than the worst-case loss rate for TCP congestion control at a comparable data rate over the same path (aka less than the "TCP-friendly" loss rate [tcp-friendly]) ๐Ÿ“ this SHOULD be enforced by requiring a minimum headway between a suppressed congestion adjustment (from a failed probe) and the next probe attempt. that headway equals one round-trip time for each packet the congestion window allows ๐Ÿงš we dig into this more in Section 7.6.2, so stay tuned, it's a whole arc โœจ

whenever the MTU goes up, the congestion state variables MUST be rescaled so the window size in bytes (or data rate in bytes per second) doesn't go up ๐Ÿ“ˆ whenever the MTU goes down (like when processing ICMP PTB messages), the congestion state variable SHOULD be rescaled so the window size in packets doesn't go up ๐ŸŒˆ

if PLPMTUD updates the MTU for a particular path, all Packetization Layer sessions that share that path representation (as described in Section 5.2) SHOULD be notified so they can use the new MTU and make the required congestion control adjustments ๐Ÿ”ฅ everyone gets the group chat update.

all implementations MUST include mechanisms that let applications selectively send packets larger than the current effective Path MTU but smaller than the first-hop link MTU ๐Ÿ‘‘ this is needed so an application can run PLPMTUD over a connectionless protocol, and so diagnostic tools can work without relying on the operating system's Path MTU Discovery implementation. see Section 9 for more on that โœจ

implementations MAY use different heuristics to pick the initial effective Path MTU for each protocol ๐Ÿฆ‹ connectionless protocols and protocols that don't support PLPMTUD SHOULD have their own default initial effective Path MTU, which can be set more conservatively (smaller) than the initial value used by TCP and other protocols that are well suited to PLPMTUD ๐ŸŒธ there SHOULD also be per-protocol and per-route limits on the initial effective Path MTU (eff_pmtu) and the upper searching limit (search_high). see Section 7.2 for the deets ๐Ÿ’…


5. Layering ๐Ÿงš

Packetization Layer Path MTU Discovery is easiest to build when its jobs are split between layers ๐Ÿซถ the IP layer is the best place to keep shared state, collect ICMP messages, track IP header sizes, and manage the MTU info that the link layer interfaces hand over ๐Ÿ’œ BUT the procedures PLPMTUD uses for probing and verifying the Path MTU are super tightly coupled to Packetization Layer stuff, like data recovery and congestion control state machines ๐Ÿ“

this layering approach is a direct extension of the advice in the current PMTUD specs, RFC 1191 and RFC 1981 ๐ŸŽ€

5.1. Accounting for Header Sizes

PLPMTUD works across multiple layers, so it needs a way to account for header sizes at every layer between IP and the Packetization Layer (inclusive) ๐ŸŒˆ when sending non-probe packets, it's enough for the Packetization Layer to make sure there's an upper bound on the final IP packet size, so it doesn't go over the current effective Path MTU โœจ every Packetization Layer taking part in classical Path MTU Discovery already has this requirement ๐Ÿ’…

when conducting a probe, the Packetization Layer MUST determine the probe packet's final size, IP headers included ๐Ÿ”ฅ this one's specific to PLPMTUD, and meeting it may need extra inter-layer communication in existing implementations ๐Ÿ™ˆ

5.2. Storing PMTU Information

this memo uses the idea of a "flow" to set the scope of the Path MTU Discovery algorithms ๐Ÿฆ‹ for lots of implementations, a flow naturally maps to one instance of each protocol (aka each connection or session). in those implementations, the algorithms in this doc run inside each session, for each protocol ๐Ÿ‘‘ the observed PMTU (eff_pmtu in Section 7.1) MAY be shared between different flows that have a common path representation ๐Ÿ’œ

OR, PLPMTUD could be built so its whole state is tied to the path representations instead ๐Ÿงš that version could use multiple connections or sessions for each probe sequence. this approach is likely to converge way faster in some environments, like when an app uses lots of small connections that are each too short to finish Path MTU Discovery on their own ๐ŸŒธ

within a single implementation, different protocols can use either approach ๐Ÿ“ because of protocol-specific limits on generating probes (Section 6.2) and the MTU searching algorithm (Section 7.3), it may not be feasible for different Packetization Layer protocols to share PLPMTUD state โœจ so some protocols might share probing state, while others can only share the observed PMTU, and then the protocols will have different convergence properties ๐ŸŒˆ

the IP layer SHOULD be used to store the cached PMTU value and other shared state, like MTU values reported by ICMP PTB messages ๐Ÿซถ ideally, this shared state should be tied to one specific path that packets take between the source and destination nodes ๐Ÿ”ฅ BUT in most cases a node won't have enough info to fully and accurately identify that path ๐Ÿ™ˆ so a node has to associate a PMTU value with some local representation of a path, and it's up to the implementation to choose that representation ๐Ÿ’…

an implementation MAY use the destination address as its local representation of a path ๐Ÿ’œ the PMTU value for a destination would be the minimum PMTU learned across all the paths in use to that destination ๐Ÿฆ‹ the set of paths in use to one destination is expected to be small, often just a single path ๐ŸŽ€ this gets optimally sized packets on a per-destination basis, and it fits nicely with the host conceptual model in [RFC2461]: a PMTU value could be stored with the matching entry in the destination cache ๐Ÿงš since NATs and other middleboxes may show different PMTUs at one IP address at the same time, the minimum value SHOULD be stored ๐Ÿ‘‘

network or subnet numbers MUST NOT be used as representations of a path, because there's no general mechanism for the remote host to figure out the network mask ๐Ÿšซ๐Ÿ’€

for source-routed packets (aka packets with an IPv6 routing header, or an IPv4 Loose Source and Record Route (LSRR) or Strict Source and Record Route (SSRR) option), the source route MAY further qualify the local representation of a path โœจ an implementation MAY use source route info as part of that local representation ๐ŸŒธ

if IPv6 flows are in use, an implementation MAY use the 3-tuple of the Flow Label and the source and destination addresses [RFC2460][RFC3697] as its local representation of a path ๐Ÿฆ‹ this could theoretically get optimally sized packets on a per-flow basis, which is finer-grained than MTU values kept per destination ๐Ÿ’–

5.3. Accounting for IPsec

this document doesn't take a stance on where IP Security (IPsec) [RFC2401] sits, since it logically lives between IP and the Packetization Layer ๐Ÿ” a PLPMTUD implementation can treat IPsec as part of IP or as part of the Packetization Layer, as long as the accounting is consistent inside the implementation ๐ŸŒˆ if IPsec is treated as part of the IP layer, each security association to a remote node may need to be treated as its own separate path ๐Ÿงš if IPsec is treated as part of the Packetization Layer, the IPsec header size MUST be included in the Packetization Layer's header size calculations ๐Ÿ’…

5.4. Multicast

with a multicast destination address, copies of a packet may travel many different paths to reach many different nodes ๐ŸŒ so the local representation of the "path" to a multicast destination has to stand in for a potentially huge set of paths ๐Ÿซถ

at minimum, an implementation MAY keep a single MTU value to use for all multicast packets originating from the node ๐Ÿ’œ this MTU SHOULD be small enough that it's expected to be less than the Path MTU of every path in the multicast tree ๐Ÿฆ‹ if a Path MTU smaller than the configured multicast MTU is learned via unicast means, the multicast MTU MAY be reduced to that value ๐ŸŒธ this approach is likely to result in smaller packets than necessary for many paths, so it's a bit of a compromise ๐Ÿ“

if the application using multicast gets complete delivery reports (unlikely, since this requirement scales poorly), PLPMTUD MAY be implemented in multicast protocols so that the smallest path MTU learned across a group becomes the effective MTU for that group ๐Ÿ‘‘โœจ

6. Common Packetization Properties ๐ŸŽ€

this section covers general Packetization Layer properties and characteristics needed to implement PLPMTUD ๐Ÿฆ‹ it also covers some implementation issues that are common to all Packetization Layers ๐Ÿ’–

6.1. Mechanism to Detect Loss ๐Ÿ”

okay so loss detection is SO important, bestie ๐Ÿ˜ญ PLPMTUD makes its MTU adjustments based on the losses it detects. If the loss notification is delayed or kinda inaccurate, you end up with wrong MTU decisions or really slow convergence ๐Ÿ’€. The mechanism has to be able to tell the isolated loss of just a probe apart from other losses in the probe's leading and trailing windows. That's precision main character energy ๐Ÿ’…

the best option is an explicit loss detection mechanism, like a Selective Acknowledgment (SACK) scoreboard [RFC3517] or an ACK Vector [RFC4340], to tell real losses apart from reordered data ๐Ÿง  implicit mechanisms like TCP Reno style duplicate acknowledgments counting are sufficient too โœจ

PLPMTUD can also be implemented in protocols that rely on timeouts as their main loss recovery mechanism ๐Ÿ•ฐ๏ธ however, timeouts SHOULD NOT be used as the primary mechanism for loss indication unless there are no other alternatives ๐Ÿ™…โ€โ™€๏ธ


6.2. Generating Probes ๐ŸŽˆ

okay there are a bunch of ways to tweak Packetization Layers so they can generate probes ๐Ÿง and each technique comes with different overheads in three areas: how hard the probe packet is to generate (the PL implementation complexity and extra data motion), how much extra network capacity the probes eat, and the overhead of recovering from failed probes (both network and protocol overheads) ๐Ÿ˜ฉ

some protocols might be extended to allow arbitrary padding with dummy data ๐Ÿซง this makes the implementation SO much simpler because probing can happen without any participation from higher layers. And if the probe fails, the missing data (the "probe gap") is guaranteed to fit within the current MTU when it gets retransmitted ๐Ÿ™Œ this is probably the most appropriate method for protocols that support arbitrary length options or multiplexing within the protocol itself ๐Ÿ’–

a lot of Packetization Layer protocols can carry pure control messages (no data from higher protocol layers at all), and those can be padded to arbitrary lengths ๐ŸŽ€ the SCTP PAD chunk is an example of this (see Section 10.2). The big perk here is that nothing needs to be retransmitted if the probe gets lost ๐Ÿฅฐ

these techniques do NOT work for TCP ๐Ÿ™ˆ TCP has no separate length field or other mechanism to tell padding apart from real payload data. So with TCP, the only approach is to send additional payload data in an over-sized segment. There are at least two variants of this, discussed in Section 10.1 ๐Ÿฆ‹

in a few cases there may be no reasonable mechanisms to generate probes within the Packetization Layer protocol itself. As a last resort, it may be possible to rely on an adjunct protocol, such as ICMP ECHO ("ping"), to send probe packets ๐Ÿ“ see Section 10.3 for more on this approach.


7. The Probing Method ๐Ÿงช

this section is where we get into the nitty gritty ๐Ÿค“ it covers the details of the MTU probing method, including how to send probes and how to process the error indications you need to search for the Path MTU ๐Ÿ”Ž

7.1. Packet Size Ranges ๐Ÿ“

okay so this document describes the probing method using three state variables ๐Ÿ‘‘

  • search_low: The smallest useful probe size, minus one. The network is expected to be able to deliver packets of size search_low.
  • search_high: The greatest useful probe size. Packets of size search_high are expected to be too large for the network to deliver.
  • eff_pmtu: The effective PMTU for this flow. This is the largest non-probe packet permitted by PLPMTUD for the path.

here's the visual ๐Ÿ‘€

               search_low          eff_pmtu         search_high
                   |                   |                  |
           ...------------------------->

               non-probe size range
                   <-------------------------------------->
                               probe size range

                                 Figure 1

when transmitting non-probes, the Packetization Layer SHOULD create packets of a size less than or equal to eff_pmtu ๐Ÿ“ฆ

when transmitting probes, the Packetization Layer MUST select a probe size that is larger than search_low and smaller than or equal to search_high ๐ŸŽฏ

when probing upward, eff_pmtu always equals search_low ๐Ÿชœ in other states, like initial conditions, after ICMP PTB message processing, or following PLPMTUD on another flow sharing the same path representation, eff_pmtu may be different from search_low. Normally, eff_pmtu will be greater than or equal to search_low and less than search_high. It is generally expected but not required that the probe size will be greater than eff_pmtu ๐Ÿค

for initial conditions when there's no information about the path, eff_pmtu may be greater than search_low ๐ŸŒค๏ธ the initial value of search_low SHOULD be conservatively low, but performance may be better if eff_pmtu starts at a higher, less conservative, value. See Section 7.2 โœจ

if eff_pmtu is larger than search_low, it is explicitly permitted to send non-probe packets larger than search_low ๐Ÿ’ซ when one of those packets gets acknowledged, it's effectively an "implicit probe" and search_low SHOULD be raised to the size of the acknowledged packet ๐Ÿ“ˆ BUT if an "implicit probe" is lost, it MUST NOT be treated as a probe failure the way a true probe would be ๐Ÿšซ if eff_pmtu is too large, this condition will only be detected with ICMP PTB messages or black hole discovery (see Section 7.7) ๐Ÿ•ณ๏ธ


7.2. Selecting Initial Values ๐ŸŒฑ

the initial value for search_high SHOULD be the largest possible packet that might be supported by the flow ๐Ÿ”๏ธ this may be limited by the local interface MTU, by an explicit protocol mechanism such as the TCP MSS option, or by an intrinsic limit such as the size of a protocol length field. In addition, the initial value for search_high MAY be limited by a configuration option to prevent probing above some maximum size. Search_high is likely to be the same as the initial Path MTU as computed by the classical Path MTU Discovery algorithm ๐Ÿงฎ

it is RECOMMENDED that search_low be initially set to an MTU size that is likely to work over a very wide range of environments ๐ŸŒ given today's technologies, a value of 1024 bytes is probably safe enough. The initial value for search_low SHOULD be configurable โš™๏ธ

okay real talk, properly functioning Path MTU Discovery is critical to the robust and efficient operation of the Internet ๐ŸŒ any major change (like the ones described in this document) has the potential to be very disruptive if it causes any unexpected changes in protocol behaviors ๐Ÿ˜ฌ the selection of the initial value for eff_pmtu decides how much a PLPMTUD implementation's behavior looks like classical PMTUD in cases where the classical method is sufficient ๐Ÿชž

a conservative configuration would be to set eff_pmtu to search_high, and rely on ICMP PTB messages to set eff_pmtu down as appropriate. In this configuration, classical PMTUD is fully functional and PLPMTUD is only invoked to recover from ICMP black holes through the procedure described in Section 7.7 ๐Ÿ›Ÿ

in some cases, where it's known that classical PMTUD is likely to fail (for example, if ICMP PTB messages are administratively disabled for security reasons), using a small initial eff_pmtu will avoid the costly timeouts required for black hole detection โณ the trade-off is that using a smaller than necessary initial eff_pmtu might cause reduced performance ๐Ÿ“‰

note that the initial eff_pmtu can be any value in the range search_low to search_high ๐ŸŽจ an initial eff_pmtu of 1400 bytes might be a good compromise because it would be safe for nearly all tunnels over all common networking gear, and yet close to the optimal MTU for the majority of paths in the Internet today ๐Ÿซถ this might be improved by using some statistics of other recent flows. For example, the initial eff_pmtu for a flow might be set to the median of the probe size for all recent successful probes ๐Ÿ“Š

since the cost of PLPMTUD is dominated by the protocol specific overheads of generating and processing probes, it's probably desirable for each protocol to have its own heuristics for selecting the initial eff_pmtu ๐Ÿงฉ it's especially important that connectionless protocols, and other protocols that may not receive clear indications of ICMP black holes, use conservative (smaller) initial values for eff_pmtu, as described in Section 10.3 ๐Ÿ’œ

there SHOULD be per-protocol and per-route configuration options to override initial values for eff_pmtu and other PLPMTUD state variables ๐ŸŽ›๏ธ


7.3. Selecting Probe Size ๐ŸŽฏ

the probe may have a size anywhere in the "probe size range" described above ๐ŸŒˆ but a number of factors affect the selection of an appropriate size. A simple strategy might be to do a binary search, halving the probe size range with each probe. However, for some protocols, such as TCP, failed probes are more expensive than successful ones, since data in a failed probe will need to be retransmitted ๐Ÿ’ธ for such protocols, a strategy that raises the probe size in smaller increments might have lower overhead. For many protocols, both at and above the Packetization Layer, the benefit of increasing MTU sizes may follow a step function, so it's not advantageous to probe within certain regions at all ๐Ÿชœ

as an optimization, it may be appropriate to probe at certain common or expected MTU sizes, for example, 1500 bytes for standard Ethernet, or 1500 bytes minus header sizes for tunnel protocols ๐Ÿ›œ

some protocols may use other mechanisms to choose the probe sizes ๐Ÿงธ for example, protocols that have certain natural data block sizes might simply assemble messages from a number of blocks until the total size is smaller than search_high, and if possible larger than search_low ๐Ÿงฑ

each Packetization Layer MUST determine when probing has converged, which means when the probe size range is small enough that further probing is no longer worth its cost ๐Ÿ when probing has converged, a timer SHOULD be set. When the timer expires, search_high should be reset to its initial value (described above) so that probing can resume ๐Ÿ”„ thus, if the path changes and increases the Path MTU, the flow will eventually take advantage of it ๐Ÿฆ‹

the value for this timer MUST NOT be less than 5 minutes and is recommended to be 10 minutes, per RFC 1981. ๐ŸŒธโœจ

7.4. Probing Preconditions โœจ

okay bestie, before a flow sends a probe, it MUST meet at least these conditions ๐Ÿ’…

  • It has no outstanding probes or losses.
  • If the last probe failed or was inconclusive, then the probe timeout has expired (see Section 7.6.2).
  • The available window is greater than the probe size.
  • For a protocol using in-band data for probing, enough data is available to send the probe.

ok so the timely loss detection stuff in most protocols has its own pre-conditions that SHOULD be satisfied before a probe goes out ๐Ÿงš TCP Fast Retransmit isn't robust unless there are enough segments after the probe, so the sender SHOULD have enough data queued and enough receiver window to send the probe plus at least Tcprexmtthresh [RFC2760] additional segments. this can totally block probing in some protocol states, like when you're too close to the end of a connection or the window is tiny ๐Ÿฅบ

protocols MAY delay sending non-probes so they can pile up enough data to meet the pre-conditions for probing ๐ŸŒธ the delayed sending algorithm SHOULD use some self-scaling technique to keep that delay in check. for example, the returning ACKs can be used to stop the window from dropping by more than the amount of data the probe needs. main character self-regulation โœจ

7.5. Conducting a Probe ๐Ÿฆ‹

once a probe size in the right range has been picked and the preconditions above are met, the Packetization Layer MAY conduct a probe. to do that, it builds a probe packet whose size, including the outermost IP headers, equals the probe size. after sending, it waits for a response, and that response lands in one of these buckets:

  • Success: The probe is acknowledged as having been received by the remote host.
  • Failure: A protocol mechanism indicates that the probe was lost, but no packets in the leading or trailing window were lost.
  • Timeout failure: A protocol mechanism indicates that the probe was lost, and no packets in the leading window were lost, but is unable to determine whether any packets in the trailing window were lost. For example, loss is detected by a timeout, and go-back-n retransmission is used.
  • Inconclusive: The probe was lost in addition to other packets in the leading or trailing windows.

7.6. Response to Probe Results ๐ŸŽ€

when a probe is done, the result SHOULD be processed as follows, sorted by the probe's result type ๐Ÿ’œ

7.6.1. Probe Success ๐ŸŒˆ

when the probe gets delivered, it's a sign the Path MTU is at least as big as the probe size ๐Ÿ”ฅ set search_low to the probe size. if the probe size is larger than eff_pmtu, raise eff_pmtu to the probe size. the probe size might be smaller than eff_pmtu if the flow hasn't been using the full MTU of the path because something else is holding it back, like not much data in an interactive session ๐Ÿซถ

note that if a flow's packets get routed over multiple paths, or over a path with a non-deterministic MTU, delivery of one probe packet does not indicate that all packets of that size will get through ๐Ÿ˜ญ to be robust in that case, the Packetization Layer SHOULD conduct MTU verification as described in Section 7.8.

7.6.2. Probe Failure ๐Ÿ’€

when only the probe is lost, that's treated as a sign the Path MTU is smaller than the probe size ๐Ÿ™ˆ in this case alone, the loss SHOULD NOT be interpreted as congestion signal.

if there are no other indications, set search_high to the probe size minus one. eff_pmtu might be larger than the probe size if the flow hasn't been using the full MTU of the path because some other limit is in play, like available data in an interactive session. if eff_pmtu is larger than the probe size, eff_pmtu MUST be reduced to no larger than search_high, and SHOULD be reduced to search_low, since eff_pmtu has been determined to be invalid. it's giving the same energy as after a full-stop timeout (see Section 7.7).

if an ICMP PTB message is received matching the probe packet, then search_high and eff_pmtu MAY be set from the MTU value indicated in the message. note that the ICMP message may be received either before or after the protocol loss indication ๐Ÿงพ

a probe failure event is the one situation under which the Packetization Layer SHOULD ignore loss as a congestion signal. there's some small risk that suppressing congestion control might have unanticipated consequences (even for one isolated loss), so it is REQUIRED that probe failure events be less frequent than the normal period for losses under standard congestion control. specifically, after a probe failure event and suppressed congestion control, PLPMTUD MUST NOT probe again until an interval that is larger than the expected interval between congestion control events. see Section 4 for details. the simplest estimate of the interval to the next congestion event is the same number of round trips as the current congestion window in packets. ๐Ÿ›‘

7.6.3. Probe Timeout Failure โฐ

if the loss was detected with a timeout and repaired with go-back-n retransmission, then congestion window reduction will be necessary ๐Ÿ“‰ the relatively high price of a failed probe in this case may merit a longer time interval until the next probe. a time interval that is five times the non-timeout failure case (Section 7.6.2) is RECOMMENDED. ๐Ÿ“

7.6.4. Probe Inconclusive ๐Ÿค”

the presence of other losses near the loss of the probe may indicate that the probe was lost due to congestion rather than due to an MTU limitation. in this case, the state variables eff_pmtu, search_low, and search_high SHOULD NOT be updated, and the same-sized probe SHOULD be attempted again as soon as the probing preconditions are met (i.e., once the packetization layer has no outstanding unrecovered losses) ๐Ÿงš at this point, re-probing is especially appropriate, since the flow's congestion window will be at its lowest point, minimizing the probability of congestive losses. smart move, honestly ๐Ÿ’œ

7.7. Full-Stop Timeout ๐Ÿ›‘

under all conditions, a full-stop timeout (also known as a "persistent timeout" in other documents) SHOULD be taken as an indication of some significantly disruptive event in the network, like a router failure or a routing change to a path with a smaller MTU ๐Ÿ˜ฑ for TCP, this happens when the R1 timeout threshold described by [RFC1122] expires.

if there is a full-stop timeout and there was not an ICMP message indicating a reason (PTB, Net unreachable, etc., or the ICMP message was ignored for some reason), the RECOMMENDED first recovery action is to treat this as a detected ICMP black hole as defined in [RFC2923] ๐Ÿ•ณ๏ธ

the response to a detected black hole depends on the current values for search_low and eff_pmtu. if eff_pmtu is larger than search_low, set eff_pmtu to search_low. otherwise, set both eff_pmtu and search_low to the initial value for search_low. upon additional successive timeouts, search_low and eff_pmtu SHOULD be halved, with a lower bound of 68 bytes for IPv4 and 1280 bytes for IPv6. even lower lower bounds MAY be permitted to support limited operation over links with MTUs that are smaller than permitted by the IP specifications ๐Ÿฆ‹

7.8. MTU Verification ๐Ÿ”

it's possible for a flow to simultaneously traverse multiple paths, but an implementation will only be able to keep a single path representation for the flow ๐ŸŒธ if the paths have different MTUs, storing the minimum MTU of all paths in the flow's path representation will result in correct behavior. if ICMP PTB messages are delivered, then classical PMTUD will work correctly in this situation. slay โœจ

if ICMP delivery fails, breaking classical PMTUD, the connection will rely solely on PLPMTUD. in this case, PLPMTUD may fail as well, since it assumes a flow traverses a path with a single MTU ๐Ÿ˜ญ a probe with a size greater than the minimum but smaller than the maximum of the Path MTUs may be successful. however, upon raising the flow's effective PMTU, the loss rate will significantly increase. the flow may still make progress, but the resultant loss rate is likely to be unacceptable. for example, when using two-way round-robin striping, 50% of full-sized packets would be dropped ๐Ÿ’€

striping like this is often operationally undesirable for other reasons (e.g., due to packet reordering), and it's usually avoided by hashing each flow to a single path. still, to increase robustness, an implementation SHOULD implement some form of MTU verification, so that if increasing eff_pmtu results in a sharp increase in loss rate, it will fall back to using a lower MTU ๐Ÿซถ

a RECOMMENDED strategy would be to save the value of eff_pmtu before raising it. then, if loss rate rises above a threshold for a period of time (e.g., loss rate is higher than 10% over multiple retransmission timeout (RTO) intervals), the new MTU is considered incorrect ๐Ÿšจ the saved value of eff_pmtu SHOULD be restored, and search_high reduced in the same manner as in a probe failure. PLPMTUD implementations SHOULD implement MTU verification. bestie, that's the whole verification arc ๐ŸŒˆ

8. Host Fragmentation โœจ

Packetization Layers SHOULD avoid sending messages that will require fragmentation [Kent87] [frag-errors]. however, entirely preventing fragmentation isn't always possible ๐Ÿ™ˆ some Packetization Layers, like a UDP application outside the kernel, may be unable to change the size of messages it sends, which means datagram sizes can exceed the Path MTU.

IPv4 let these applications send packets without the DF bit set. oversized packets without the DF bit set would get fragmented in the network or the sending host when they hit a link with an MTU smaller than the packet. in some cases, packets could be fragmented more than once if there were cascaded links with progressively smaller MTUs ๐ŸŒ€ this approach is NOT RECOMMENDED. ngl, it's a vibe to avoid.

it is RECOMMENDED that IPv4 implementations use a strategy that mimics IPv6 functionality ๐Ÿ’– when an application sends datagrams larger than the effective Path MTU, they SHOULD be fragmented to the Path MTU in the host IP layer, even if they're smaller than the MTU of the first link, directly attached to the host. the DF bit SHOULD be set on the fragments, so they won't get fragmented again in the network. this technique minimizes the likelihood that applications will rely on IPv4 fragmentation in a way that can't be implemented in IPv6. at least one major operating system already uses this strategy ๐Ÿ‘‘ Section 9 describes some exceptions to this rule when the application is sending oversized packets for probing or diagnostic purposes.

since protocols that don't implement PLPMTUD are still subject to problems due to ICMP black holes, it may be desirable to limit these protocols to "safe" MTUs likely to work on any path (e.g., 1280 bytes) ๐Ÿงธ allow any protocol implementing PLPMTUD to operate over the full range supported by the lower layer.

note that IP fragmentation divides data into packets, so it is minimally a Packetization Layer. however, it does not have a mechanism to detect lost packets, so it cannot support a native implementation of PLPMTUD. fragmentation-based PLPMTUD requires an adjunct protocol as described in Section 10.3. ๐Ÿงฉ

9. Application Probing โœจ

okay bestie, this section is about apps doing their own PLPMTUD probing ๐Ÿฆ‹ All implementations MUST include a mechanism where applications using connectionless protocols can send their own probes. This is necessary to implement PLPMTUD in an application protocol (see Section 10.4) or to build diagnostic tools for debugging PMTUD problems ๐Ÿ” There MUST be a mechanism that lets an application send datagrams bigger than eff_pmtu, the operating system's estimate of the Path MTU, without them getting fragmented. If these are IPv4 packets, they MUST have the DF bit set ๐Ÿ’…

Most operating systems give you two modes for sending datagrams ๐Ÿ™„ One silently fragments packets that are too big, and the other rejects packets that are too big. Neither one works for PLPMTUD in an application or for diagnosing Path MTU Discovery problems, so a third mode is REQUIRED, where the datagram gets sent even if it's larger than the current Path MTU estimate ๐ŸŒธ

Implementing PLPMTUD in an application also needs a way for the app to tell the operating system how the probe turned out (see Section 7.6). Or it can directly update search_low, search_high, and eff_pmtu (see Section 7.1) ๐Ÿงš

Diagnostic apps are super useful for finding PMTUD problems, like the ones a defective router can cause when it sends ICMP PTB messages with the wrong size info ๐Ÿ˜ญ The fastest way to track these down is a tool that can send probes of any specified size and collect and display all the ICMP PTB messages that come back ๐Ÿ”ฅ


10. Specific Packetization Layers

okay so every Packetization Layer protocol must consider all of the issues from Section 6 ๐Ÿซถ For a lot of protocols this is pretty straightforward. This section walks through the details for implementing PLPMTUD with a couple of protocols, and the hope is that these examples are enough for implementers to adapt the idea to other protocols ๐ŸŽ€

10.1. Probing Method Using TCP

ngl this one is kinda tricky ๐Ÿ’€ TCP has no way to tell in-band data apart from padding, so TCP has to make its probes by segmenting data in the right way. There are two approaches to segmentation: overlapping and non-overlapping ๐Ÿ“

With the non-overlapping method, the data gets segmented so the probe and any segments after it don't overlap. If the probe is lost, the "probe gap" ends up being a full probe size minus headers. The data in that gap has to be retransmitted using several smaller segments ๐Ÿ˜ฉ

             TCP sequence number

           t   <---->

           i         <-------->           (probe)
           m                   <---->
           e
                         .
                         .                (probe lost)
                         .

                     <---->               (probe gap retransmitted)
                           <-->

                                 Figure 2

There's also an alternate approach, where the data sent after the probe overlaps the probe so the probe gap is the same length as the current MSS ๐Ÿ‘‘ When the probe succeeds, this adds overhead because some data gets sent twice, but after a lost probe you only have to retransmit one segment. When a probe succeeds, there will probably be some duplicate acknowledgments because of the duplicate data ๐Ÿ˜ญ It's important that these duplicate ACKs don't trigger Fast Retransmit. So an implementation using this approach SHOULD limit the probe size to three times the current MSS (which causes at most 2 duplicate acknowledgments), or it should adjust its duplicate acknowledgment threshold for data right after a successful probe ๐Ÿ’…

             TCP sequence number

           t   <---->
           i         <-------->           (probe)
           m               <---->

           e                     <---->

                         .
                         .                (probe lost)
                         .

                     <---->               (probe gap retransmitted)

                                 Figure 3

Which segmentation method to use should come down to whatever is simplest and most efficient for a given TCP implementation ๐Ÿฆ‹

10.2. Probing Method Using SCTP

In the Stream Control Transmission Protocol (SCTP) [RFC2960], the app writes messages to SCTP, and SCTP splits the data into smaller "chunks" that are suitable for transmission through the network ๐Ÿงš Each chunk gets its own Transmission Sequence Number (TSN). Once a TSN has been transmitted, SCTP can't change the chunk size. SCTP's multi-path support normally requires SCTP to pick a chunk size so its messages fit the smallest PMTU across all paths ๐ŸŒˆ It's not required, but implementations may bundle several data chunks together to make bigger IP packets for paths with a larger PMTU. Also, SCTP has to independently probe the PMTU on each path to the peer ๐Ÿซถ

The RECOMMENDED way to generate probes is to add a chunk that's only padding to an SCTP message. The PAD chunk defined in [RFC4820] SHOULD be attached to a minimum length HEARTBEAT (HB) chunk to build a probe packet ๐Ÿ’– This method is fully compatible with all current SCTP implementations โœจ

SCTP MAY also probe with a method similar to the TCP one described above, using inline data ๐ŸŽ€ This has the advantage that successful probes have no extra overhead. The catch is that failed probes will need data to be retransmitted, which may hurt flow performance ๐Ÿ˜ฌ

10.3. Probing Method for IP Fragmentation

okay this part is spicy ๐ŸŒถ๏ธ There are a few protocols and apps that normally send large datagrams and rely on IP fragmentation to deliver them. People have known for a long time that this has some undesirable consequences [Kent87]. More recently, it came to light that IPv4 fragmentation isn't robust enough for general use on today's Internet ๐Ÿ’€ The 16-bit IP identification field isn't big enough to stop frequent mis-associated IP fragments, and the TCP and UDP checksums aren't enough to stop the resulting corrupted data from reaching higher protocol layers [frag-errors] ๐Ÿ˜ญ

As mentioned in Section 8, datagram protocols (like UDP) might rely on IP fragmentation as a Packetization Layer. But using IP fragmentation to implement PLPMTUD is a problem, because the IP layer has no way to tell whether the packets were actually delivered to the far node without the app directly taking part ๐Ÿ™ˆ

To support IP fragmentation as a Packetization Layer under an unmodified application, an implementation SHOULD rely on the Path MTU sharing described in Section 5.2, plus an adjunct protocol to probe the Path MTU ๐Ÿซถ There are a bunch of protocols that could do this, like ICMP ECHO and ECHO REPLY, or "traceroute" style UDP datagrams that trigger ICMP messages. Using ICMP ECHO and ECHO REPLY probes both the forward and return paths, so the sender can only take advantage of the minimum of the two. Other methods that probe only the forward path are preferred if they're available ๐Ÿงš

All of these approaches have a few potential robustness problems. The most likely failures come from losses that have nothing to do with MTU (for example, nodes that discard some protocol types). These non-MTU-related losses can stop PLPMTUD from raising the MTU, which forces IP fragmentation to use a smaller MTU than necessary. Since these failures aren't likely to cause interoperability problems, they're pretty benign ๐ŸŒธ

Other, more serious failure modes do exist, though ๐Ÿ˜ฑ For example, middle boxes or upper-layer routers might choose different paths for different protocol types or sessions. In that kind of environment, adjunct protocols may legitimately see a different Path MTU than the primary protocol. If the adjunct protocol finds a larger MTU than the primary protocol, PLPMTUD may select an MTU that the primary protocol can't use. Even though this is a potentially serious problem, lots of people would see this situation as incorrect, so there will be strong motivation to fix it ๐Ÿ”ฅ

Since connectionless protocols might not keep enough state to effectively diagnose MTU black holes, it would be more robust to err on the side of using too small an initial MTU (for example, 1 kByte or less) before probing a path to measure the MTU ๐Ÿฆ‹ For this reason, implementations that use IP fragmentation SHOULD use an initial eff_pmtu, selected as described in Section 7.2, except they use a separate global control for the default initial eff_mtu for connectionless protocols ๐ŸŒˆ

Connectionless protocols also bring an extra problem for maintaining the path information cache: there are no events for connection establishment and tear-down to manage the cache with ๐Ÿ˜ฉ A natural approach is to keep an immutable cache entry for the "default path," which has an eff_pmtu fixed at the initial value for connectionless protocols. The adjunct Path MTU Discovery protocol would get invoked once the number of fragmented datagrams to any particular destination hits some configurable threshold (for example, 5 datagrams). A new path cache entry gets created when the adjunct protocol updates eff_pmtu, and it's deleted based on a timer or a Least Recently Used cache replacement algorithm ๐Ÿ’œ

10.4. Probing Method Using Applications

the disadvantages of relying on IP fragmentation plus an adjunct protocol for Path MTU Discovery can be overcome by implementing Path MTU Discovery inside the application itself, using the application's own protocol ๐Ÿ‘‘ The app needs a suitable way to generate probes, and an accurate, timely way to figure out whether they were lost ๐Ÿซถ

Ideally, the application protocol includes a lightweight echo function that confirms message delivery, plus a way to pad messages out to the desired probe size, where the padding isn't echoed back ๐Ÿ’– This combination (kind of like the SCTP HB plus PAD) is RECOMMENDED, because an application can measure the MTU of each direction separately on a path with asymmetrical MTUs โœจ

For protocols that can't implement PLPMTUD with "echo plus pad," there are often other ways to generate probes ๐ŸŒธ For example, the protocol might have a variable length echo that effectively measures the minimum MTU of both the forward and return paths. Or there may be a way to add padding to regular messages that carry real application data. There may also be other ways to segment application data to generate probes, or, as a last resort, it may be possible to extend the protocol with new message types specifically to support MTU discovery ๐Ÿงš

Note that if you need to add new message types to support PLPMTUD, the most general approach is to add ECHO and PAD messages. These give the most possible flexibility in how an application-specific PLPMTUD implementation interacts with other applications and protocols on the same end system ๐ŸŽ€

All application probing techniques need the ability to send messages larger than the current eff_pmtu described in Section 9 ๐ŸŒˆ

11. Security Considerations ๐Ÿ”๐Ÿ’–

okay bestie, this section is SO important so pay attention ๐Ÿ‘€ first things first, under all conditions, the PLPMTUD procedures in this doc are at least as secure as the current standard Path MTU Discovery procedures in RFC 1191 and RFC 1981. no cap, that's a big promise ๐Ÿ’…

next, PLPMTUD is built to work robustly without any ICMP or other messages from the network ๐Ÿฆ‹ so it can be configured to ignore all ICMP messages, either globally or per application. in that setup, it can only be attacked if the attacker can figure out which probe packets to target and make them get lost ๐Ÿ™ˆ attacking PLPMTUD does hurt performance, but not as much as attacking congestion control by making arbitrary packets get lost. honestly, an attacker would do way more damage by totally wrecking specific protocols, like DNS ๐Ÿ’€

also, packetization protocols can share state with each other ๐Ÿซถ so if one of them (especially an application) was hostile to the other protocols on the same host, it could hurt their performance by shrinking the effective MTU. so if a packetization protocol is untrusted, it SHOULD NOT be allowed to write to shared state.

wait, I need to be careful here: that last rule is a "SHOULD NOT" in the original, and it's the only BCP 14 keyword in this section, so keep it exactly like that! โš ๏ธ

12. References ๐Ÿ“š

12.1. Normative References ๐ŸŒธ

  • [RFC0791] Postel, J., "Internet Protocol", STD 5, RFC 791, September 1981. โœจ
  • [RFC1191] Mogul, J. and S. Deering, "Path MTU discovery", RFC 1191, November 1990. โœจ
  • [RFC1981] McCann, J., Deering, S., and J. Mogul, "Path MTU Discovery for IP version 6", RFC 1981, August 1996. โœจ
  • [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. โœจ
  • [RFC2460] Deering, S. and R. Hinden, "Internet Protocol, Version 6 (IPv6) Specification", RFC 2460, December 1998. โœจ
  • [RFC0793] Postel, J., "Transmission Control Protocol", STD 7, RFC 793, September 1981. โœจ
  • [RFC3697] Rajahalme, J., Conta, A., Carpenter, B., and S. Deering, "IPv6 Flow Label Specification", RFC 3697, March 2004. โœจ
  • [RFC2960] Stewart, R., Xie, Q., Morneault, K., Sharp, C., Schwarzbauer, H., Taylor, T., Rytina, I., Kalla, M., Zhang, L., and V. Paxson, "Stream Control Transmission Protocol", RFC 2960, October 2000. โœจ
  • [RFC4820] Tuexen, M., Stewart, R., and P. Lei, "Padding Chunk and Parameter for the Stream Control Transmission Protocol (SCTP)", RFC 4820, March 2007. โœจ

12.2. Informative References ๐Ÿงš

  • [RFC2760] Allman, M., Dawkins, S., Glover, D., Griner, J., Tran, D., Henderson, T., Heidemann, J., Touch, J., Kruse, H., Ostermann, S., Scott, K., and J. Semke, "Ongoing TCP Research Related to Satellites", RFC 2760, February 2000. ๐Ÿ›ฐ๏ธ
  • [RFC1122] Braden, R., "Requirements for Internet Hosts - Communication Layers", STD 3, RFC 1122, October 1989. ๐Ÿ–ฅ๏ธ
  • [RFC2923] Lahey, K., "TCP Problems with Path MTU Discovery", RFC 2923, September 2000. ๐Ÿ•ต๏ธ
  • [RFC2401] Kent, S. and R. Atkinson, "Security Architecture for the Internet Protocol", RFC 2401, November 1998. ๐Ÿ”’
  • [RFC2914] Floyd, S., "Congestion Control Principles", BCP 41, RFC 2914, September 2000. ๐Ÿšฆ
  • [RFC2461] Narten, T., Nordmark, E., and W. Simpson, "Neighbor Discovery for IP Version 6 (IPv6)", RFC 2461, December 1998. ๐Ÿ˜๏ธ
  • [RFC3517] Blanton, E., Allman, M., Fall, K., and L. Wang, "A Conservative Selective Acknowledgment (SACK)-based Loss Recovery Algorithm for TCP", RFC 3517, April 2003. ๐Ÿ”
  • [RFC4340] Kohler, E., Handley, M., and S. Floyd, "Datagram Congestion Control Protocol (DCCP)", RFC 4340, March 2006. ๐ŸŒŠ
  • [Kent87] Kent, C. and J. Mogul, "Fragmentation considered harmful", Proc. SIGCOMM '87 vol. 17, No. 5, October 1987. ๐Ÿ“œ
  • [tcp-friendly] Mahdavi, J. and S. Floyd, "TCP-Friendly Unicast Rate-Based Flow Control", Technical note sent to the end2end-interest mailing list, January 1997, <http://www.psc.edu/networking/papers/tcp_friendly.html>. ๐Ÿฆ„
  • [frag-errors] Heffner, J., "IPv4 Reassembly Errors at High Data Rates", Work in Progress, December 2007. ๐Ÿงฉ

Appendix A. Acknowledgments ๐Ÿซถ๐ŸŒˆ

Many ideas and even some of the text come directly from RFC 1191 and RFC 1981. ๐Ÿ’œ

Many people made significant contributions to this document, including: Randall Stewart for SCTP text, Michael Richardson for material from an earlier ID on tunnels that ignore DF, Stanislav Shalunov for the idea that pure PLPMTUD parallels congestion control, and Matt Zekauskas for maintaining focus during the meetings. Thanks to the early implementors: Kevin Lahey, John Heffner, and Rao Shoaib, who provided concrete feedback on weaknesses in earlier versions. Thanks also to all of the people who made constructive comments in the working group meetings and on the mailing list. We are sure we have missed many deserving people. ๐Ÿฅน

Matt Mathis and John Heffner are supported in this work by a grant from Cisco Systems, Inc. ๐Ÿ’ธ

Authors' Addresses ๐Ÿก

Matt Mathis Pittsburgh Supercomputing Center 4400 Fifth Avenue Pittsburgh, PA 15213 USA

Phone: 412-268-3319 EMail: mathis@psc.edu

John W. Heffner Pittsburgh Supercomputing Center 4400 Fifth Avenue Pittsburgh, PA 15213 US

Phone: 412-268-2329 EMail: jheffner@psc.edu

Copyright (C) The IETF Trust (2007).

This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights.

This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property ๐Ÿงพ

The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79.

Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr.

The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org.

Acknowledgement ๐Ÿ’–

Funding for the RFC Editor function is currently provided by the Internet Society. ๐ŸŒธ

๐Ÿ“ฆ where this page comes from