RFC 9268

IPv6 Minimum Path MTU Hop-by-Hop Option

📅 published August 2022, from ietf

RFC 9268: IPv6 Minimum Path MTU Hop-by-Hop Option 🦋

okay bestie, let's get into this one 💖 It's an Experimental RFC (so, not a full Standards Track thing, just vibes-checking the idea 🧪), written by R. Hinden of Check Point Software and G. Fairhurst of the University of Aberdeen. It came out in August 2022 and it's published as an IETF product with IESG approval, but it's explicitly NOT a standard yet 🙈. It doesn't obsolete anything and it doesn't update anything either, so it's its own little main character 👑

Abstract ✨

Okay so the big idea is a brand-new IPv6 Hop-by-Hop Option 🎀 The point is to record the smallest Path MTU (PMTU) along the forward path from a source host to a destination host. Then the value gets sent back to the source through the return Path MTU field in the Option, so the sender knows what size to use. It's giving efficiency 💅

Status of This Memo 🌸

This document is not an Internet Standards Track specification. It's published for examination, experimental implementation, and evaluation, so people can poke at it 🧚

It does define an Experimental Protocol for the Internet community. It's an IETF product, it represents IETF consensus, got public review, and was approved by the Internet Engineering Steering Group (IESG). Not every IESG-approved doc is up for any level of Internet Standard, though, so see Section 2 of RFC 7841 for the deets 🍓

You can find the current status, errata, and feedback info at https://www.rfc-editor.org/info/rfc9268 ✨

Copyright (c) 2022 IETF Trust and the persons identified as the document authors. All rights reserved 🔥

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they stand on the publication date. Please review those carefully, because they spell out your rights and restrictions for this doc 🫶 Code Components pulled out of this document must include Revised BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come without warranty as described in the Revised BSD License 💀

Table of Contents 🌈

1.  Introduction
  1.1.  Example Operation
  1.2.  Use of the IPv6 Hop-by-Hop Options Header
2.  Motivation and Problem Solved
3.  Requirements Language
4.  Applicability Statements
5.  IPv6 Minimum Path MTU Hop-by-Hop Option
6.  Router, Host, and Transport Layer Behaviors
  6.1.  Router Behavior
  6.2.  Host Operating System Behavior
  6.3.  Transport Layer Behavior
    6.3.1.  Including the Option in an Outgoing Packet
    6.3.2.  Validation of the Packet that Includes the Option
    6.3.3.  Receiving the Option
    6.3.4.  Using the Rtn-PMTU Field
    6.3.5.  Detecting Path Changes
    6.3.6.  Detection of Dropping Packets that Include the Option
7.  IANA Considerations
8.  Security Considerations
  8.1.  Router Option Processing
  8.2.  Network-Layer Host Processing
  8.3.  Validating Use of the Option Data
  8.4.  Direct Use of the Rtn-PMTU Value
  8.5.  Using the Rtn-PMTU Value as a Hint for Probing
  8.6.  Impact of Middleboxes
9.  Experiment Goals
10. Implementation Status
11. References
  11.1.  Normative References
  11.2.  Informative References
Appendix A.  Examples of Usage
Acknowledgments
Authors' Addresses

1. Introduction 🧚‍♀️

Okay so this spec defines a new IPv6 Hop-by-Hop (HBH) Option that records the minimum Maximum Transmission Unit (MTU) along the forward path between a source and a destination host 🦋 The source host makes a packet with this Option and sets the Min-PMTU field to the MTU of the outbound link that will carry the packet toward the destination 💅

At every later hop where the Option gets processed, the router compares the Min-PMTU field in the Option with the MTU of its outgoing link. If the link's MTU is less than the Min-PMTU, it rewrites the Option Data with the smaller value 🙈 When the packet reaches the destination host, the host can send the minimum Reported MTU for the path back to the source using the Rtn-PMTU field in the Option. The source can then feed that value into the method that sets the Path MTU (PMTU) used by upper-layer protocols 🫶

The IPv6 Minimum Path MTU Hop-by-Hop (MinPMTU HBH) Option is built to work with the packet sizes you can specify in the IPv6 header. The biggest packet size that fits in an IPv6 header is 65,535 octets (2^16) ✨

This method could finish Path MTU Discovery (PMTUD) in a single round-trip time, even over paths where the links keep getting smaller MTUs 🔥

The mechanism here is about unicast only. It doesn't describe multicast, which is left for future work 🌸

1.1. Example Operation 🎀

The figure below shows how the method works. In this setup, the path from source host to destination host has three links: the source has a link MTU of MTU-S, the link between routers R1 and R2 has an MTU of 9000 bytes, and the last link to the destination has an MTU of MTU-D 🧚

      +--------+         +----+        +----+         +-------+
      |        |         |    |        |    |         |       |
      | Sender +---------+ R1 +--------+ R2 +-------- + Dest. |
      |        |         |    |        |    |         |       |
      +--------+  MTU-S  +----+  9000B +----+  MTU-D  +-------+

      Figure 1: An Example Path between the Source Host and the
                   Destination Host

Three scenarios get covered 💖

  • Scenario 1 says every link has a 9000 byte MTU, and both routers support the method. The initial Min-PMTU never changes along the path, so the PMTU is 9000 bytes 🍓
  • Scenario 2 says the link between R2 and the destination host (MTU-D) has an MTU of 1500 bytes. That's the smallest one. Router R2 updates the Min-PMTU to 1500 bytes, and the method correctly updates the PMTU to 1500 bytes too. If there were an even smaller MTU further down the path that also supports the method, it would get caught as well 😭
  • Scenario 3 says the router right before the smallest link (R2) doesn't support the method, and the link to the destination host (MTU-D) has an MTU of 1500 bytes. So R2 never updates the Min-PMTU to 1500 bytes, and the method fails to find the real PMTU 💀

In Scenarios 2 and 3, a lower PMTU also goes undetected if PMTUD was being used and the ICMPv6 Packet Too Big (PTB) message never made it back to the sender [RFC8201] 🙈

These scenarios are summed up in the table below. An "H" in the R1 and/or R2 columns means that router understands the MinPMTU HBH Option 🌈

+===+========+========+====+====+==========+=======================+
|   | MTU-S  | MTU-D  | R1 | R2 | Rec PMTU | Note                  |
+===+========+========+====+====+==========+=======================+
| 1 | 9000 B | 9000 B | H  | H  | 9000 B   | Endpoints attempt to  |
|   |        |        |    |    |          | use a 9000 B PMTU.    |
+---+--------+--------+----+----+----------+-----------------------+
| 2 | 9000 B | 1500 B | H  | H  | 1500 B   | Endpoints attempt to  |
|   |        |        |    |    |          | use a 1500 B PMTU.    |
+---+--------+--------+----+----+----------+-----------------------+
| 3 | 9000 B | 1500 B | H  | -  | 9000 B   | Endpoints attempt to  |
|   |        |        |    |    |          | use a 9000 B PMTU but |
|   |        |        |    |    |          | need to implement a   |
|   |        |        |    |    |          | method to fall back   |
|   |        |        |    |    |          | to discover and use a |
|   |        |        |    |    |          | 1500 B PMTU.          |
+---+--------+--------+----+----+----------+-----------------------+

  Table 1: Three Scenarios That Arise from Using the Path Shown in
                          Figure 1

1.2. Use of the IPv6 Hop-by-Hop Options Header 🦋

Per [RFC8200], IPv6 lets nodes optionally process the Hop-by-Hop header. From Section 4 of [RFC8200], here's the exact vibe:

The Hop-by-Hop Options header is not inserted or deleted, but may be examined or processed by any node along a packet's delivery path, until the packet reaches the node (or each of the set of nodes, in the case of multicast) identified in the Destination Address field of the IPv6 header. The Hop-by-Hop Options header, when present, must immediately follow the IPv6 header. Its presence is indicated by the value zero in the Next Header field of the IPv6 header. NOTE: While [RFC2460] required that all nodes must examine and process the Hop-by-Hop Options header, it is now expected that nodes along a packet's delivery path only examine and process the Hop-by-Hop Options header if explicitly configured to do so.

The Hop-by-Hop Option in this doc is designed to take advantage of exactly that behavior 💅 Nodes that don't support this Option SHOULD ignore it. That can mean the Min-PMTU value doesn't account for every link along the path 🙈

(part 1 of 5, more to come bestie 💖)

2. Motivation and Problem Solved ✨

okay bestie, let's talk about why any of this matters 🥺 Path MTU Discovery on the Internet is lowkey giving chaos right now. The mechanisms in [RFC8201] don't work well everywhere 😭 They fail in a bunch of situations, like when nodes in the middle of the network don't send ICMPv6 PTB messages, or only send rate-limited ones, or don't have a return path to the source host. So tons of transport connections just default to smaller packets, like 1280 bytes, and it's SO hard to use paths with a bigger PMTU even when they exist 💔 Apps that send big packets get forced into IPv6 fragmentation [RFC8200], and that can make Internet communication less reliable [RFC8900]. Not it. 🙅‍♀️

Encapsulations and tunnels also eat into the payload size a transport protocol can use 🍓 Some use cases add extra overhead too. Like, Network Virtualization Using Generic Routing Encapsulation (NVGRE) [RFC7637] wraps Layer 2 (L2) packets in an outer IP header and doesn't allow IP fragmentation. Bestie, that's a lot 💀

Bigger packets can also make hosts faster 🚀 because packet processing doesn't get capped by the packet rate. For example, hitting wire speed on a 10G link with 1280 byte packets takes about 977K packets per second (pps), but only about 139K pps for 9000 byte packets. Iconic math 🧮

The goal of this document is to fix all this without needing ICMPv6 PTB messages from nodes in the middle of the network 🦋 Instead, the destination host learns the Minimum Path MTU and sends that info back to the source host. This is expected to work better than the current mechanisms based on [RFC8201] ✨

A similar idea got proposed for IPv4 way back in 1988, in [RFC1063] by Jeff Mogul, C. Kent, Craig Partridge, and Keith McCloghire 🌸 It was later obsoleted in 1990 by [RFC1191], which is the current deployed approach to Path MTU Discovery. This document is different because it uses the Hop-by-Hop Option of IPv6. It doesn't replace PMTUD [RFC8201], Packetization Layer Path MTU Discovery (PLPMTUD) [RFC4821], or Datagram Packetization Layer PMTU Discovery (DPLPMTUD) [RFC8899]. It's designed to compliment these methods 🫶

3. Requirements Language 🎀

okay so these are the BCP 14 key words: "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL". They're interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they show up in all capitals, just like here 🫧

4. Applicability Statements 💅

The Path MTU Option is made for environments where you have control over the hosts and nodes that connect them, and where there's more than one MTU size in use. Think data centers, and paths between data centers, so hosts can better take advantage of a path that's able to support a large PMTU ✨

The Option is so sufficiently simple that it can run on a router's fast path 🏎️ A successful experiment depends on both host and router vendors implementing it, and on operators deploying it 🧑‍💻 The contained use case of connections within and between data centers could be what drives deployment 💖

The method could also be useful in other environments, including the general Internet, and it gives you an edge when this Hop-by-Hop Option is supported on all paths 🌈 It's more robust when it probes the path using packets that don't carry application data, and when it's paired with a method like Packetization Layer PMTUD [RFC4821] or Datagram Packetization Layer PMTU Discovery (DPLPMTUD) [RFC8899] 🫧

5. IPv6 Minimum Path MTU Hop-by-Hop Option 🦋

The Minimum Path MTU Hop-by-Hop Option has the following format, bestie 👑

          Option    Option    Option
           Type    Data Len   Data
         +--------+--------+--------+--------+---------+-------+-+
         |BBCTTTTT|00000100|     Min-PMTU    |     Rtn-PMTU    |R|
         +--------+--------+--------+--------+---------+-------+-+

Figure 2: Format of the Minimum Path MTU Hop-by-Hop Option

Here's the breakdown, field by field 🧩 Option Type (see Section 4.2 of [RFC8200]):

     BB     00   Skip over this Option and continue processing.

     C       1   Option Data can change en route to the packet's final
                 destination.

     TTTTT 10000 Option Type assigned from IANA [IANA-HBH].

     Length:  4  The size of the value field in Option Data
                 field supports PMTU values from 0 to 65,534
                 octets, the maximum size represented by the
                 Path MTU Option.


     Min-PMTU: n 16-bits.  The minimum MTU recorded along the path
                 in octets, reflecting the smallest link MTU that
                 the packet experienced along the path.
                 A value less than the IPv6 minimum link
                 MTU [RFC8200] MUST be ignored.

     Rtn-PMTU: n 15-bits.  The returned Path MTU field, carrying the 15
                 most significant bits of the latest received Min-PMTU
                 field for the forward path.  The value zero means that
                 no Reported MTU is being returned.

     R        n  1-bit.  R-Flag.   Set by the source to signal that
                 the destination host should include the received
                 Rtn-PMTU field updated by the reported Min-PMTU value
                 when the destination host is to send a PMTU Option back
                 to the source host.
NOTE: The encoding of the final two octets (Rtn-PMTU and R-Flag) could be implemented by a mask of the latest received Min-PMTU value with 0xFFFE, discarding the right-most bit and then performing a logical 'OR' with the R-Flag value of the sender. This encoding fits in the minimum-sized Hop-by-Hop Option header. 🎀

6. Router, Host, and Transport Layer Behaviors 🌷

6.1. Router Behavior

Routers that aren't configured to support Hop-by-Hop Options are not expected to examine or process the contents of this Option [RFC8200] 🙈

Routers that support Hop-by-Hop Options but aren't configured to support this Option SHOULD skip over this Option and continue to process the header [RFC8200] 🫶

Routers that support this Option MUST compare the value of the Min-PMTU field with the MTU configured for the outgoing link. If the MTU of the outgoing link is less than the Min-PMTU, the router rewrites the Min-PMTU in the Option to use the smaller value. (The router processing is performed without checking the valid range of the Min-PMTU or the Rtn-PMTU fields.) 🔍

A router MUST ignore and MUST NOT change the Rtn-PMTU field or the R-Flag in the Option. 🚫

6.2. Host Operating System Behavior

The PMTU entry associated with the destination in the host's destination cache [RFC4861] SHOULD be updated after detecting a change using the IPv6 Minimum Path MTU Hop-by-Hop Option. This cached value can be used by other flows that share the host's destination cache 💜

The value in the host destination cache SHOULD be used by PLPMTUD to select an initial PMTU for a flow. The cached PMTU is only increased by PLPMTUD when the Packetization Layer determines the path actually supports a larger PMTU [RFC4821] [RFC8899] 📈

When requested to send an IPv6 packet with the MinPMTU HBH Option, the source host includes the Option in an outgoing packet. The source host MUST fill the Min-PMTU field with the MTU configured for the link over which it will send the packet on the next hop towards the destination host 🛣️

When a host includes the Option in a packet it sends, the host SHOULD set the Rtn-PMTU field to the previously cached value of the received Minimum Path MTU for the flow in the Rtn-PMTU field (see Section 6.3.3). If this value is not set (for example, because there is no cached reported Min-PMTU value), the Rtn-PMTU field value MUST be set to zero. 🧠

The source host MAY request the destination host to return the reported Min-PMTU value by setting the R-Flag in the Option of an outgoing packet. The R-Flag SHOULD NOT be set when the MinPMTU HBH Option was sent solely to provide requested feedback on the return Path MTU, so each response doesn't generate another response 🔁

The destination host controls when to send a packet with this Option in response to an R-Flag, as well as which packets to include it in. The destination host MAY limit the rate at which it sends these packets ⏱️

A destination host only sets the R-Flag if it wishes the source host to also return the discovered PMTU value for the path from the destination to the source 💌

The normal sequence of operation of the R-Flag, using the terminology from the diagram in Figure 1, goes like this:

  1. The source sends a probe to the destination. The sender sets the R-Flag.
  2. The destination responds by sending a probe including the received Min-PMTU as the Rtn-PMTU. A destination that does not wish to probe the return path sets the R-Flag to 0. ✨

6.3. Transport Layer Behavior

This Hop-by-Hop Option is intended to be used with a Path MTU Discovery method 🌈

PLPMTUD [RFC8899] uses probe packets for two distinct functions:

  • Probe packets are used to confirm connectivity. Such probes can be of any size up to the Packetization Layer Path MTU (PLPMTU). These probe packets are sent to solicit a response using the path to the remote node. These probe packets can carry the Hop-by-Hop PMTU Option, providing the final size of the packet does not exceed the current PLPMTU. After validating that the packet originates from the path (Section 4.6.1 of [RFC8899]), the PLPMTUD method can use the reported size from the Hop-by-Hop Option as the next search point when it resumes the search algorithm. (This use resembles the use of the PTB_SIZE information in Section 4.6.2 of [RFC8899].) 🔎
  • A second use of probe packets is to explore if a path supports a packet size greater than the current PLPMTU. If this probe packet is successfully delivered (as determined by the source host), then the PLPMTU is raised to the size of the successful probe. These probe packets do not usually set the Path MTU Hop-by-Hop Option. See Section 1.2 of [RFC8899]. Section 4.1 of [RFC8899] also describes ways that a probe packet can be constructed, depending on whether the probe packets carry application data. 🧪

The PMTU Hop-by-Hop Option probe can be sent on packets that include application data but needs to be robust to potential loss of the packet (i.e., with the possibility that retransmission might be needed if the packet is lost). 🔄

Using a PMTU probe on packets that do not carry application data will avoid the need for loss recovery if a router on the path drops packets that set this Option. (This avoids the transport needing to retransmit a lost packet that includes this Option.) This is the normal default format for both uses of probes. 🫧

6.3.1. Including the Option in an Outgoing Packet ✨

okay so the upper-layer protocol can ask for the MinPMTU HBH Option to get tucked into an outgoing IPv6 packet 💅 A transport protocol (or upper-layer protocol) can only put it on the specific packets it uses to test the path 🧚 It doesn't need to show up on every packet in a flow, bestie.

NOTE: Slapping this Option on a big packet (like one bigger than the current PMTU) is not likely to be useful 😭 That packet would just get dropped by any link with a smaller MTU along the path, so the Min-PMTU info never reaches the destination host. Not the vibe.

Discussion:

  • For TCP, the Option could go in a packet carrying a TCP segment sent after the connection is established 🦋 A segment without data could be used so you don't have to retransmit that data if the probe packet gets lost. The value you find can be used to inform PLPMTUD [RFC4821] 💖
    • > NOTE: A TCP SYN can also negotiate the Maximum Segment Size (MSS), which acts as an upper limit to the packet size a TCP sender can send. If this Option went into a TCP SYN, it could make it more likely that the SYN segment gets lost when routers on the path drop packets with this Option (see Section 6.3.6) 😬 That could mess with the result of racing Options [TAPS-ARCH] or feature negotiation. Hard pass, lowkey.
  • Datagram transport protocols (e.g., UDP) are harder to pin down 🌸 Applications using them range from super short-lived (low data-volume) exchanges all the way to longer (bulk) exchanges of packets between the source and destination hosts [RFC8085].
  • Simple-exchange protocols (i.e., low data-volume applications [RFC8085] that only send one or a few packets per transaction) might assume the PMTU is symmetrical 👑 That means the PMTU is the same in both directions, or at least not smaller on the return path. That shortcut doesn't hold up when the paths are not symmetric 🙈
  • The MinPMTU HBH Option can be used with ICMPv6 [RFC4443] 🍓 This needs a response from the remote node, so it's restricted to use with ICMPv6 echo messages. The MinPMTU HBH Option could give extra info about the PMTU a path might support, and that could work as a diagnostic tool for measuring the PMTU of a path ✨ As with other uses, the actual supported PMTU is only confirmed after you get a response to a follow-up probe of the PMTU size.
  • A datagram transport can use DPLPMTUD [RFC8899] 🎀 For example, QUIC (see Section 14.3 of [RFC9000]) can use DPLPMTUD to figure out whether the path to a destination will support a desired maximum datagram size. When using the IPv6 MinPMTU HBH Option, the Option could be added to an extra QUIC PMTU probe that is of minimal size (or one no larger than the currently supported PMTU size) 🧁 Once the Rtn-PMTU value in the MinPMTU HBH Option has been learned, DPLPMTUD can be triggered to test for a larger PLPMTU using an appropriately sized PLPMTU probe packet (see Section 5.3.1 of [RFC8899]).
  • Using this Option with DNS and DNSSEC over UDP is expected to work for paths where the PMTU is symmetric 🧚 The DNS server will learn the PMTU from the DNS query messages. If the Rtn-PMTU value is smaller, a large DNSSEC response might get dropped, and then the known problems with PMTUD will show up 💀 DNS and DNSSEC over transport protocols that can carry the PMTU ought to work.
  • This method can also be used with anycast to discover the PMTU of the path, but whoever uses it needs to be aware that the anycast binding might change 👀

6.3.2. Validation of the Packet that Includes the Option 🛡️

An upper-layer protocol (e.g., transport endpoint) using this Option needs to protect itself from data injection attacks by off-path devices [RFC8085] 😤 It needs a way to make sure the information in the Option Data was provided by a node on the path 🫶 That checks that the packet is part of an existing flow, using context available at the upper layer. For example, a TCP connection or UDP application that keeps track of the related state and uses a randomized ephemeral port gives you this basic validation against off-path data injection; see Section 5.1 of [RFC8085]. IPsec [RFC4301] and TLS [RFC8446] give you even more assurance, iykyk 🔐

The upper layer discards any received packet when packet validation fails. When packet validation fails, the upper layer MUST also discard the associated Option Data from the MinPMTU HBH Option without any further processing 🚫

6.3.3. Receiving the Option 📬

For a connection-oriented upper-layer protocol, caching the received Min-PMTU could be done by saving the value in the connection context at the transport layer 🌈 A connectionless upper layer (e.g., one using UDP) needs the upper-layer protocol to cache the value for each flow it uses.

A destination host that receives a MinPMTU HBH Option with the R-Flag SHOULD include the MinPMTU HBH Option in the next outgoing IPv6 packet for the corresponding flow 💜

A simple mechanism could only include this Option (with the Rtn-PMTU field set) the first time the Option is received, or when it signals a change in the Minimum Path MTU. This limits how many packets get sent, including the Option packets ✨ However, this doesn't give you robustness to packet loss or recovery after a sender loses state, so it's a tradeoff, bestie.

Discussion:

  • Some upper-layer protocols send packets less often than the host receives them 🐢 That means less frequent feedback of the received Rtn-PMTU value. However, a host always sends the most recent Rtn-PMTU value 💅

6.3.4. Using the Rtn-PMTU Field 🧐

The Rtn-PMTU field gives an indication of the PMTU from on-path routers. It does not necessarily reflect the actual PMTU between the source and destination hosts, so you gotta be careful using the Rtn-PMTU value 😭 Specifically:

  • The actual PMTU can be lower than the Rtn-PMTU value because a router on the path that did not process the Option didn't update the Min-PMTU field 🙃
  • The actual PMTU may be lower than the Rtn-PMTU value because there is a Layer 2 device with a lower MTU 🔧
  • The actual PMTU may be larger than the Rtn-PMTU value because of a corrupted, delayed, or misordered response 🌀 A source host MUST ignore a Rtn-PMTU value larger than the MTU configured for the outgoing link 🚫
  • The path might have changed between the time the probe was sent and the time the Rtn-PMTU value was received 🛤️

IPv6 requires that every link in the Internet have an MTU of 1280 octets or greater 💖 A node MUST ignore a Rtn-PMTU value less than 1280 octets [RFC8200] 🙅‍♀️

To avoid accidentally dropping packets that exceed the actual PMTU (e.g., Scenario 3 in Section 1.1), the source host can hold off on increasing the PMTU until a probe packet at the size of the Rtn-PMTU value has been successfully acknowledged by the upper layer 🧘 That confirms the path supports the larger PMTU. This probing makes things more robust, but it adds one extra path round-trip time before the PMTU gets updated ⏳ This use is kinda like PTB messages in Section 4.6 of DPLPMTUD [RFC8899]. The big difference is that a PTB message can only try to lower the PMTU, whereas this Option could trigger a probe packet to try to increase it 📈

Section 5.2 of [RFC8201] gives guidance on caching PMTU information and how it relates to IPv6 flow labels 🏷️ Implementations should consider the impact of Equal-Cost Multipath (ECMP) [RFC6438], specifically whether a PMTU ought to be maintained for each transport endpoint or for each network address 🤔

6.3.5. Detecting Path Changes 🔍

Path characteristics can change, and the actual PMTU could go up or down over time 🎢 For instance, that can happen after a path change, when packets are forwarded over a link with a different MTU than before. To bound the delay in discovering an increase in the actual PMTU, a host with a link MTU larger than the current PMTU SHOULD periodically send the MinPMTU HBH Option with the R-bit set 💫 DPLPMTUD gives recommendations on how this could be implemented (see Section 5.3 of [RFC8899]). Since the Option uses less capacity than a full-sized probe packet, there can be an advantage in using it to detect a change in path characteristics 🌟

6.3.6. Detection of Dropping Packets that Include the Option 🚨

There is evidence that some middleboxes drop packets that include Hop-by-Hop Options 😩 For example, a firewall might drop a packet that carries an unknown extension header or Option. This practice is expected to decrease as an Option becomes more widely used 🌱 It could result in an ICMPv6 message being generated that indicates the problem, and that could be used to (temporarily) suspend use of this Option.

A middlebox that silently discards a packet with this Option results in any packet using the Option being dropped 💀 This dropping can be avoided with appropriate configuration in a controlled environment, such as within a data center, but it needs to be considered for Internet usage. Section 6.2 recommends that this Option is not used on packets where loss might adversely impact performance 🙏

7. IANA Considerations 🌷

IANA has registered an IPv6 Hop-by-Hop Option type in the "Destination Options and Hop-by-Hop Options" registry within the "Internet Protocol Version 6 (IPv6) Parameters" registry group [IANA-HBH]. This assignment is shown in Section 5. 📝

8. Security Considerations 🚨🔐

This section discusses the security considerations, and it's serious business, bestie 😤 It first reviews router Option processing. It then reviews host processing when receiving this Option at the network layer. It then considers two ways the Option Data can be processed, followed by two approaches for using the Option Data. Finally, it discusses middlebox implications related to use on the general Internet.

8.1. Router Option Processing 🖥️

This Option shares the characteristics of all other IPv6 Hop-by-Hop Options: if it's not supported at line rate, it could be used to degrade the performance of a router 📉 This Option, while simple, is no different from other uses of IPv6 Hop-by-Hop Options.

It is common for routers to ignore the Hop-by-Hop Option header or to drop packets containing a Hop-by-Hop Option header 🙈 Routers implementing IPv6 according to [RFC8200] only examine and process the Hop-by-Hop Options header if they're explicitly configured to do so ⚙️

8.2. Network-Layer Host Processing 🛡️

okay bestie, this is the security tea ☕ A malicious attacker can forge a packet aimed at a host that carries the MinPMTU HBH Option. By design, the network is allowed to modify the fields of this IP Option 😬

For comparison, the ICMPv6 PTB message used in Path MTU Discovery [RFC8201] and the source host have an inherent trust relationship with the destination host that includes this Option. That trust can help verify the Option ✨ ICMPv6 PTB messages come from any router on the path to the destination host. The source host has no prior knowledge of those routers, except for the first hop router 🧭

Receiving this packet means extra processing. The network stack parses the packet before it gets delivered to the upper-layer protocol, and this network-layer Option processing is normally finished before any upper-layer protocol delivery checks happen 🔍

The network layer doesn't normally have enough info to validate that the packet carrying an Option came from the destination (or an on-path node). It also usually doesn't have enough context to demultiplex the packet and figure out which transport flow it belongs to. So any changes from receiving the Option can apply to all flows between a pair of endpoints. Main character problems 💀

These considerations are no different from other uses of Hop-by-Hop Options, and this is the use case for PMTUD. The next section describes a mitigation for this attack 🫶

8.3. Validating Use of the Option Data 🔐

Transport protocols should be designed to protect against data injection attacks from off-path devices, and the mechanisms should be described in the Security Considerations section of each transport specification (see Section 5.1 of "UDP Usage Guidelines" [RFC8085]). For example, a TCP or UDP application that maintains the related state and uses a randomized ephemeral port gets basic protection 🎀 TLS [RFC8446] or IPsec [RFC4301] give cryptographic authentication 🔒 An upper-layer protocol that validates each received packet discards any packet when that validation fails. In this case, the host MUST also discard the associated Option Data from the MinPMTU HBH Option without further processing (Section 6.3). No exceptions, periodt 💅

A network node on the path can see all packets it forwards. By observing the network packet payload, the node might be able to construct a packet that the destination host might validate. Such a node could also drop or limit the flow in other ways that could be even more disruptive 😭 Authenticating the packet, for example with IPsec [RFC4301] or TLS [RFC8446], mitigates this attack. Also note that the authentication style of the Authentication Header (AH) [RFC4302] authenticates the payload and outer IPv6 header, but it does not check Hop-by-Hop Options that change on route 🙈

8.4. Direct Use of the Rtn-PMTU Value 🚫

The simplest way to use the Rtn-PMTU value is to update the PMTU directly with it. Hot take: this creates a set of security issues when the Option carries malicious data 💀

  • A direct update of the PMTU using the Rtn-PMTU value could let an attacker inflate or reduce the host PMTU for the destination. Forcing a reduction in the PMTU can make network use less efficient, might increase the number of packets or fragments needed to send the same volume of payload data, and can stop an unfragmented datagram larger than the PMTU from being sent. Increasing the PMTU can make a path silently drop packets (described as a black hole in [RFC8899]) when the source host sends packets larger than the actual PMTU. That keeps going until the PMTU gets updated again 🕳️
  • The method can be used to solicit a response from the destination host. A malicious attacker could forge a packet that causes the destination to add the Option to a packet sent to the source host. A forged value of Rtn-PMTU in the Option Data might also impact the remote endpoint, as described in the previous bullet. That keeps going until a valid MinPMTU HBH Option is received. This attack could be mitigated by limiting how often the MinPMTU HBH Option is sent in reply to incoming packets that carry the Option 🛑

8.5. Using the Rtn-PMTU Value as a Hint for Probing 🔎

Another way to use the Rtn-PMTU value is to indirectly trigger a probe that checks whether the path supports a PMTU of size Rtn-PMTU. This approach needs context for the flow, so it assumes an upper-layer protocol that validates the packet carrying the Option (see Section 8.3). That's the case when it's used together with DPLPMTUD [RFC8899]. A set of security considerations shows up when an Option carries malicious data 👀

  • If the forged packet carries a validated Option with a non-zero Rtn-PMTU field, the upper-layer protocol could use the information in the Rtn-PMTU field. An Rtn-PMTU larger than the current PMTU can trigger a probe for a new size 📈
  • If the forged packet carries a non-zero Min-PMTU field, the upper-layer protocol would change the cached information about the path from the source. The cached information at the destination host will be overwritten when the host receives another packet that includes a MinPMTU HBH Option corresponding to the flow 🧠
  • Processing of the Option could cause a destination host to add the MinPMTU HBH Option to a packet sent to the source host. This Option will carry an Rtn-PMTU value that could have been updated by the forged packet. The impact on the source host of receiving this resembles what was discussed previously 🔁

8.6. Impact of Middleboxes 📦

There is evidence that some middleboxes drop packets that include Hop-by-Hop Options. For example, a firewall might drop a packet that carries an unknown extension header or Option. This practice is expected to decrease as the Option becomes more widely used. Methods to address this are discussed in Section 6.3.6 ✨

When a forged packet causes a packet that includes the MinPMTU HBH Option to be sent, and the return path doesn't forward packets with this Option, the packet will be dropped (see Section 6.3.6). This attack is mitigated by validating the Option Data before use and by limiting the rate of responses generated. An upper layer could further reduce the impact by responding to an R-Flag by including the Option in a packet that doesn't carry application data 🌸

9. Experiment Goals 🧪

This section describes the experimental goals of this specification 🎯

A successful deployment of the method depends on several components being implemented and deployed:

  • Support in the sending node (see Section 6.2). This also requires corresponding support in upper-layer protocols (see Section 6.3).
  • Router support in nodes (see Section 6.1). The IETF continues to provide recommendations on the use of IPv6 Hop-by-Hop Options, for example, see Section 2.2.2 of [RFC9099]. This document does not update the way router implementations configure support for Hop-by-Hop Options.
  • Support in the receiving node (see Section 6.3.3).

Experience from deployment is an expected input to any decision to move this specification from Experimental to IETF Standards Track. Appropriate inputs might include:

  • reports of implementation experience,
  • measurements of the number of paths where the method can be used, or
  • measurements showing the benefit realized, or the implications of using specific methods over specific paths.

10. Implementation Status 🛠️

At the time this document was published, there were two known implementations of the Path MTU Hop-by-Hop Option. These are:

  • Wireshark dissector. This is shipping in production in Wireshark version 3.2 [WIRESHARK].
  • A prototype in the open source version of the FD.io Vector Packet Processing (VPP) technology [VPP]. At the time this document was published, the source code can be found [VPP_SRC].

11. References 📚

11.1. Normative References

[IANA-HBH] IANA, "Destination Options and Hop-by-Hop Options",
           <https://www.iana.org/assignments/ipv6-parameters/>.

[RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
           Requirement Levels", BCP 14, RFC 2119,
           DOI 10.17487/RFC2119, March 1997,
           <https://www.rfc-editor.org/info/rfc2119>.

[RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
           2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
           May 2017, <https://www.rfc-editor.org/info/rfc8174>.

[RFC8200]  Deering, S. and R. Hinden, "Internet Protocol, Version 6
           (IPv6) Specification", STD 86, RFC 8200,
           DOI 10.17487/RFC8200, July 2017,
           <https://www.rfc-editor.org/info/rfc8200>.

[RFC8201]  McCann, J., Deering, S., Mogul, J., and R. Hinden, Ed.,
           "Path MTU Discovery for IP version 6", STD 87, RFC 8201,
           DOI 10.17487/RFC8201, July 2017,
           <https://www.rfc-editor.org/info/rfc8201>.

11.2. Informative References

Appendix A. Examples of Usage ✨

Okay bestie, this appendix is basically a little picture book 📖 of how the MinPMTU HBH Option gets used by a source running DPLPMTUD to figure out the PLPMTU that a path actually supports. 🧚 The vibe: there's a path where the on-path router has an outgoing MTU of d'. The source starts out sending packets of size a, then uses DPLPMTUD to level up the size step by step, hitting b, c, d, e, and so on (whatever the search algorithm picks). The search wraps up with a PLPMTU that's at least d and at most d'. Iconic. 👑

Example one is DPLPMTUD without the MinPMTU HBH Option. DPLPMTUD searches with probe packets that keep growing. Probes of size e get sent, and they're bigger than the actual PMTU, so no bueno. In this example, PTB messages never come back from the routers, and the repeated failed probes eventually finish the search phase. Data packets never go out bigger than the last probe that got confirmed. (ACKs of data packets are left out of the diagram, so don't be confused 🙈.)

Figure 3

     ----Packets of data size a ------------------------------>
     ----Probe size b ---------------------------------------->
      <---------------------------------- ACK of probe --------
     ----Packets of data size b ------------------------------>
     ----Probe size c ---------------------------------------->
      <---------------------------------- ACK of probe --------
     ----Packets of data size c ------------------------------>
     ----Probe size d ---------------------------------------->
      <---------------------------------- ACK of probe --------
     ----Packets of data size d ------------------------------>
      <---------------------------------- ACK of probe --------
     ...
     ----Probe size e --------------X
             X----ICMPv6 PTB d' ----|
     ----Packets of data size d ------------------------------>
     ----Probe size e --------------X (again)
             X----ICMPv6 PTB d' ----|
     ----Packets of data size d ------------------------------>
     ...
     etc. until MaxProbes are unsuccessful and search phase completes.
     ----Packets of data size d ------------------------------>

Example two is DPLPMTUD with the MinPMTU HBH Option set on a connectivity probe packet. 🎀 The IPv6 Option goes end to end, and a router on the path updates the Min-PMTU to d'. That value comes back in a response that also sets the MinPMTU HBH Option. Once the source gets the Rtn-PMTU value, DPLPMTUD immediately sends a probe at the target size d'. If that probe gets confirmed for the path, the PLPMTU gets updated, and the source can use data packets up to size d'. (The search algorithm is still allowed to keep probing to see if the path can handle something bigger 👀.) Data packets never go out bigger than the last confirmed probe size d'.

Figure 4

        ----Packets of data size a ------------------------------>
        ----Connectivity probe with MinPMTU-
                                    +--updated to minPMTU=d'----->
         <-----------------ACK with Rtn-PMTU=d'--------------------
        ----Packets of data size a ------------------------------>
        ----Probe size d' --------------------------------------->
         <---------------------------------- ACK of probe ---------
        -----Packets of data size d' ---------------------------->
        Search phase completes.
        -----Packets of data size d' ---------------------------->

The final example is DPLPMTUD with the MinPMTU HBH Option set on a connectivity probe packet, but this one shows what happens when that connectivity probe gets dropped. 😭 In this case, the packet carrying the MinPMTU HBH Option never arrives. DPLPMTUD goes back to searching with probes of increasing size, and bumps the PLPMTU whenever a probe gets confirmed. When a probe goes over the actual PMTU, an ICMPv6 PTB message comes back saying PTB_SIZE is d'. DPLPMTUD then immediately sends a probe at the target size d'. If that probe gets confirmed, the PLPMTU is updated and the source can use data packets up to size d'. If the ICMPv6 PTB message never shows up, the DPLPMTU ends up being the last confirmed probe size, which is d. 💀

Figure 5

        ----Packets of data size a ------------------------------->
        ----Connectivity probe with MinPMTU --------X
        ----Packets of data size a ------------------------------->
        ----Probe size b ----------------------------------------->
         <---------------------------------- ACK of probe --------
        ----Packets of data size b ------------------------------->
        ----Probe size c ----------------------------------------->
         <---------------------------------- ACK of probe --------
        ----Packets of data size c ------------------------------->
        ----Probe size d ----------------------------------------->
         <---------------------------------- ACK of probe --------
        ----Packets of data size d ------------------------------->
        ----Probe size e ------------X
         <--ICMPv6 PTB PTB_SIZE d' --|
        ----Packets of data size d ------------------------------->
        ----Probe size d' using target set by PTB_SIZE ----------->
         <---------------------------------- ACK of probe --------
         Search phase completes.
        ----Packets of data size d' ------------------------------>

Figure 5

The number of probe rounds depends on how many steps the search algorithm needs, and it's usually bigger for a bigger PMTU. 🌈

Acknowledgments 🫶

Huge thanks to Tom Herbert, Tom Jones, Fred Templin, Ole Troan, Tianran Zhou, Jen Linkova, Brian Carpenter, Peng Shuping, Mark Smith, Fernando Gont, Michael Dougherty, Erik Kline, and the other members of the 6MAN Working Group, who gave helpful comments. You're all literal legends 💖.

Authors' Addresses 🦋

Robert M. Hinden
Check Point Software
959 Skyway Road
San Carlos, CA 94070
United States of America
Email: bob.hinden@gmail.com


Godred Fairhurst
University of Aberdeen
School of Engineering
Fraser Noble Building
Aberdeen
AB24 3UE
United Kingdom
Email: gorry@erg.abdn.ac.uk
📦 where this page comes from