RFC 1191

Path MTU discovery

๐Ÿ“… published November 1990, from ietf

Path MTU Discovery ๐ŸŽ€

okay bestie, this is RFC 1191, all about Path MTU Discovery ๐Ÿฆ‹. It was written by J. Mogul at DECWRL and S. Deering at Stanford University, and it's dated November 1990. It's an IAB Standards Track protocol, and it obsoletes RFC 1063 ๐Ÿ’….


Status of this Memo

ok so this RFC specifies a protocol on the IAB Standards Track for the Internet community, and it asks for discussion and suggestions for improvements ๐Ÿงš. For the current standardization state and status, check the current edition of the "IAB Official Protocol Standards" ๐Ÿ“š. Distribution of this memo is unlimited ๐Ÿ’–.


Table of Contents ๐Ÿ’…

   Status of this Memo                                             1
   Abstract                                                        2
   Acknowledgements                                                2
   1. Introduction                                                 2
   2. Protocol overview                                            3
   3. Host specification                                           4
       3.1. TCP MSS Option                                         5
   4. Router specification                                         6
   5. Host processing of old-style messages                        7
   6. Host implementation                                          8
       6.1. Layering                                               9
       6.2. Storing PMTU information                              10
       6.3. Purging stale PMTU information                        11
       6.4. TCP layer actions                                     13
       6.5. Issues for other transport protocols                  14
       6.6. Management interface                                  15
   7. Likely values for Path MTUs                                 15
       7.1. A better way to detect PMTU increases                 16
   8. Security considerations                                     18
   References                                                     18
   Authors' Addresses                                             19


                             List of Tables

   Table 7-1:   Common MTUs in the Internet                       17

Abstract ๐Ÿฆ‹

this memo describes a technique for dynamically discovering the maximum transmission unit (MTU) of an arbitrary internet path ๐ŸŒˆ. It makes a small change to the way routers generate one type of ICMP message โœจ. For a path that goes through a router that hasn't been changed like that, this technique might not find the correct Path MTU ๐Ÿ™ˆ. But it will always pick a Path MTU that's as accurate as the one current practice picks, and in a lot of cases way more accurate ๐Ÿ”ฅ.


Acknowledgements ๐Ÿซถ

this proposal came out of the IETF MTU Discovery Working Group ๐Ÿ’œ.

The mechanism was first suggested by Geof Cooper [2], who wrote out all the basic ideas in two short paragraphs, the stuff the Working Group took months to reinvent ๐Ÿ˜ญ.


1. Introduction ๐ŸŒธ

ok picture this: one IP host has a huge pile of data to send to another host ๐Ÿ“ฆ. That data gets sent as a bunch of IP datagrams. It's usually best if those datagrams are the biggest size that doesn't need fragmentation anywhere along the path from source to destination ๐Ÿ’…. (For the case against fragmentation, see [5].) This datagram size is called the Path MTU (PMTU), and it equals the smallest MTU of any hop on the path ๐ŸŽ€. The problem is that the current Internet protocol suite has no standard way for a host to discover the PMTU of an arbitrary path ๐Ÿ’€.

Note: The Path MTU is what [1] calls the "Effective MTU for sending" (EMTU_S). A PMTU belongs to a path, which is a particular combination of IP source and destination address and maybe a Type-of-service (TOS) ๐Ÿงš.

current practice [1] is to use the lesser of 576 and the first-hop MTU as the PMTU for any destination that isn't on the same network or subnet as the source ๐Ÿ˜ญ. This is lowkey wasteful, because lots of paths have a PMTU greater than 576 ๐Ÿ™„. A host sending datagrams way smaller than the Path MTU allows is wasting Internet resources and probably getting worse throughput than it could ๐Ÿ’”. Also, current practice doesn't stop fragmentation in every case, because some paths have a PMTU under 576 ๐Ÿšซ.

future routing protocols are expected to be able to provide accurate PMTU information within a routing area, though maybe not across multi-level routing hierarchies ๐Ÿ—บ๏ธ. It isn't clear how soon that will be available everywhere, so for the next several years the Internet needs a simple mechanism that discovers PMTUs without wasting resources, and that works before all hosts and routers have been modified ๐Ÿซถ.


2. Protocol overview โœจ

in this memo we describe a technique for using the Don't Fragment (DF) bit in the IP header to dynamically discover a path's PMTU ๐Ÿฆ‹. The basic idea is that a source host starts out assuming the PMTU of a path is the (known) MTU of its first hop, and it sends all datagrams on that path with the DF bit set ๐ŸŽฏ. If any datagram is too big to be forwarded without fragmentation by some router along the path, that router throws it away and sends back an ICMP Destination Unreachable message with a code meaning "fragmentation needed and DF set" [7] ๐Ÿ™…โ€โ™€๏ธ. When the source gets one of those (from now on called a "Datagram Too Big" message), it lowers its assumed PMTU for the path ๐Ÿ“‰.

The PMTU discovery process ends when the host's estimate of the PMTU is low enough that its datagrams can get through without fragmentation โœ…. Or the host can end discovery by no longer setting the DF bit in its datagram headers ๐Ÿคทโ€โ™€๏ธ. It might do that, for example, if it's okay with datagrams being fragmented in some situations ๐Ÿงฉ. Normally the host keeps setting DF in all datagrams, so that if the route changes and the new PMTU is lower, it gets discovered ๐Ÿ‘€.

Here's the catch ๐Ÿ˜ค: the Datagram Too Big message, as it's currently specified, doesn't say what the MTU of the hop was that rejected the datagram, so the source host can't tell exactly how much to lower its assumed PMTU ๐Ÿ˜ฉ. To fix that, we propose using a header field in the Datagram Too Big message that's not used right now, to report the MTU of the hop that's causing the problem ๐Ÿ”ง. This is the only change specified for routers to support PMTU Discovery ๐Ÿ’….

A path's PMTU can change over time because the routing topology changes ๐ŸŒŠ. Decreases in PMTU are detected by Datagram Too Big messages, except on paths where the host has stopped setting the DF bit ๐Ÿ‘€. To detect increases in a path's PMTU, a host periodically raises its assumed PMTU (and if it had stopped setting DF, it starts again) ๐Ÿ“ˆ. This almost always ends with datagrams getting discarded and Datagram Too Big messages being generated, because in most cases the path's PMTU hasn't changed ๐Ÿ˜ฌ. So it should be done infrequently ๐Ÿ•ฐ๏ธ.

Since this mechanism basically guarantees that a host won't get any fragments from a peer doing PMTU Discovery, it might help when interoperating with certain hosts that (improperly) can't reassemble fragmented datagrams ๐Ÿงšโ€โ™€๏ธ.


3. Host specification ๐Ÿ‘‘

when a host gets a Datagram Too Big message, it MUST reduce its estimate of the PMTU for the relevant path, based on the value of the Next-Hop MTU field in the message (see section 4) ๐ŸŽ€. We don't specify the exact behavior of a host here, because different applications can have different requirements, and different implementation architectures can favor different strategies ๐Ÿง .

We do require that after receiving a Datagram Too Big message, a host MUST attempt to avoid getting more of those messages in the near future ๐Ÿšง. The host can either reduce the size of the datagrams it sends along the path, or stop setting the Don't Fragment bit in those datagrams' headers ๐ŸŽ€. Obviously, the first option might keep triggering Datagram Too Big messages for a while, but since each of those messages (and the dropped datagrams they respond to) uses up Internet resources, the host MUST force the PMTU Discovery process to converge ๐Ÿ.

Hosts using PMTU Discovery MUST detect decreases in Path MTU as fast as possible ๐Ÿƒโ€โ™€๏ธ๐Ÿ’จ. Hosts MAY detect increases in Path MTU too ๐ŸŒˆ. But doing that means sending datagrams bigger than the current estimated PMTU, and the PMTU most likely hasn't gone up, so this MUST be done at infrequent intervals ๐Ÿ•ฐ๏ธ. An attempt to detect an increase (by sending a datagram larger than the current estimate) MUST NOT be done less than 5 minutes after a Datagram Too Big message has been received for the given destination, or less than 1 minute after a previous, successful attempted increase โณ. We recommend setting these timers to twice their minimum values (10 minutes and 2 minutes, respectively) ๐Ÿ’ก.

Hosts MUST be able to deal with Datagram Too Big messages that don't include the next-hop MTU, because it isn't feasible to upgrade every router on the Internet in any finite time ๐ŸŒ. You can spot a Datagram Too Big message from an unmodified router because the newly-defined Next-Hop MTU field is zero ๐Ÿ”. (This is required by the ICMP specification [7], which says that "unused" fields must be zero.) In section 5, we talk about possible strategies a host can follow when it gets an old-style Datagram Too Big message (one sent by an unmodified router) ๐Ÿงฉ.

A host MUST never reduce its estimate of the Path MTU below 68 octets ๐Ÿ›‘.

A host MUST not increase its estimate of the Path MTU because of what's in a Datagram Too Big message ๐Ÿ™…โ€โ™€๏ธ. A message claiming the Path MTU went up might be a stale datagram that's been floating around the Internet ๐Ÿ‘ป, a fake packet injected as part of a denial-of-service attack ๐Ÿšจ, or the result of having multiple paths to the destination ๐Ÿ”€. Not me trusting that one ๐Ÿ’….

3.1. TCP MSS Option โœจ

okay so a host doing PMTU Discovery has to obey the rule that it not send IP datagrams larger than 576 octets unless it has permission from the receiver ๐Ÿ™…โ€โ™€๏ธ. For TCP connections, that means a host must not send datagrams bigger than 40 octets plus the Maximum Segment Size (MSS) its peer sent ๐Ÿ’Œ

Note: The TCP MSS is defined to be the relevant IP datagram size minus 40 [9]. The default of 576 octets for the maximum IP datagram size yields a default of 536 octets for the TCP MSS. ๐Ÿงฎ

Section 4.2.2.6 of "Requirements for Internet Hosts -- Communication Layers" [1] says:

Some TCP implementations send an MSS option only if the destination host is on a non-connected network. However, in general the TCP layer may not have the appropriate information to make this decision, so it is preferable to leave to the IP layer the task of determining a suitable MTU for the Internet path.

Actually, a lot of TCP implementations always send an MSS option, but they set the value to 536 if the destination is non-local ๐Ÿ™ƒ. This was correct back when the Internet was full of hosts that didn't follow the rule about not sending datagrams larger than 576 octets to non-local destinations. Now that most hosts do follow that rule, it's unnecessary to limit the TCP MSS option to 536 for non-local peers ๐ŸŒธ

Moreover, doing this stops PMTU Discovery from finding PMTUs larger than 576, so hosts SHOULD no longer lower the value they send in the MSS option ๐Ÿšซ๐Ÿ“‰. The MSS option should be 40 octets less than the size of the largest datagram the host is able to reassemble (MMS_R, as defined in [1]). In many cases, that will be the architectural limit of 65495 (65535 - 40) octets ๐Ÿ›๏ธ. A host MAY send an MSS value derived from the MTU of its connected network (the maximum MTU over its connected networks, for a multi-homed host). This should not cause problems for PMTU Discovery, and it may dissuade a broken peer from sending enormous datagrams ๐Ÿ™ˆ

Note: At the moment, we see no reason to send an MSS greater than the maximum MTU of the connected networks, and we recommend that hosts do not use 65495. It is quite possible that some IP implementations have sign-bit bugs that would be tickled by unnecessary use of such a large MSS. ๐Ÿ›

4. Router specification ๐Ÿ›ฃ๏ธ

When a router is unable to forward a datagram because it exceeds the MTU of the next-hop network and its Don't Fragment bit is set, the router is required to return an ICMP Destination Unreachable message to the source of the datagram, with the Code indicating "fragmentation needed and DF set" ๐Ÿšจ. To support the Path MTU Discovery technique specified in this memo, the router MUST include the MTU of that next-hop network in the low-order 16 bits of the ICMP header field that is labelled "unused" in the ICMP specification [7]. The high-order 16 bits remain unused, and MUST be set to zero. So the message looks like this ๐Ÿ‘€:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |   Type = 3    |   Code = 4    |           Checksum            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |           unused = 0          |         Next-Hop MTU          |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |      Internet Header + 64 bits of Original Datagram Data      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The value carried in the Next-Hop MTU field is:

The size in octets of the largest datagram that could be forwarded, along the path of the original datagram, without being fragmented at this router. The size includes the IP header and IP data, and does not include any lower-level headers.

This field will never contain a value less than 68, since every router "must be able to forward a datagram of 68 octets without fragmentation" [8] ๐Ÿซถ


5. Host processing of old-style messages ๐Ÿ•ฐ๏ธ

In this section we outline several possible strategies a host could follow after receiving a Datagram Too Big message from an unmodified router (i.e., one where the Next-Hop MTU field is zero). This section is not part of the protocol specification ๐Ÿ“

The simplest thing a host can do is assume the PMTU is the minimum of its currently-assumed PMTU and 576, and stop setting the DF bit in datagrams sent on that path. So the host falls back to the same PMTU it would pick under current practice (see section 3.3.3 of "Requirements for Internet Hosts -- Communication Layers" [1]). This strategy has the advantage that it terminates quickly, and it does no worse than existing practice ๐Ÿƒโ€โ™€๏ธ. It fails, however, to avoid fragmentation in some cases, and to make the most efficient use of the internetwork in other cases ๐Ÿ˜ฌ

More sophisticated strategies involve "searching" for an accurate PMTU estimate, by continuing to send datagrams with the DF bit while varying their sizes ๐Ÿ”. A good search strategy is one that obtains an accurate estimate of the Path MTU without causing many packets to be lost in the process ๐Ÿ’…

Several possible strategies apply algorithmic functions to the previous PMTU estimate to generate a new one. For example, you could multiply the old estimate by a constant (say, 0.75). We do NOT recommend this, because it either converges far too slowly, or it substantially underestimates the true PMTU ๐ŸŒ

A more sophisticated approach is to do a binary search on the packet size. This converges somewhat faster, although it still takes 4 or 5 steps to converge from an FDDI MTU to an Ethernet MTU. A serious disadvantage is that it requires a complex implementation in order to recognize when a datagram has made it to the other end (indicating that the current estimate is too low). We also do not recommend this strategy ๐Ÿ™…โ€โ™€๏ธ

One strategy that seems to work really well starts from the observation that there are, in practice, relatively few MTU values in use on the Internet. So rather than blindly searching through arbitrarily chosen values, we can search only the ones that are likely to show up ๐ŸŽฏ. Plus, since designers tend to choose MTUs in similar ways, it's possible to collect groups of similar MTU values and use the lowest value in each group as our search "plateau". (It is clearly better to underestimate an MTU by a few per cent than to overestimate it by one octet.) ๐Ÿง 

In section 7, we describe how we arrived at a table of representative MTU plateaus for use in PMTU estimation. With this table, convergence is as good as binary search in the worst case, and it's way better in common cases (for example, it takes only two round-trip times to go from an FDDI MTU to an Ethernet MTU) ๐Ÿš€. Since the plateaus lie near powers of two, if an MTU isn't represented in this table, the algorithm will not underestimate it by more than a factor of 2 โœจ

Any search strategy needs some "memory" of previous estimates in order to choose the next one. One approach is to use the currently-cached estimate of the Path MTU, but there's actually better information available in the Datagram Too Big message itself ๐Ÿ’ก. All ICMP Destination Unreachable messages, including this one, contain the IP header of the original datagram, and that header contains the Total Length of the datagram that was too big to be forwarded without fragmentation. Since this Total Length may be less than the current PMTU estimate, but is still larger than the actual PMTU, it may be a good input to the method for choosing the next PMTU estimate ๐ŸŽ€

Note: routers based on implementations derived from 4.2BSD Unix send an incorrect value for the Total Length of the original IP datagram. The value sent by these routers is the sum of the original Total Length and the original Header Length (expressed in octets). Since it is impossible for the host receiving such a Datagram Too Big message to know whether it was sent by one of these routers, the host must be conservative and assume that it was. If the Total Length field returned is not less than the current PMTU estimate, it must be reduced by 4 times the value of the returned Header Length field. โš ๏ธ

The strategy we recommend, then, is to use as the next PMTU estimate the greatest plateau value that is less than the returned Total Length field (corrected, if necessary, according to the Note above) ๐Ÿ‘‘


6. Host implementation ๐Ÿ’ป

In this section we discuss how PMTU Discovery is implemented in host software. This is not a specification, but rather a set of suggestions ๐Ÿ’ญ The issues include:

  • What layer or layers implement PMTU Discovery?
  • Where is the PMTU information cached?
  • How is stale PMTU information removed?
  • What must transport and higher layers do?

6.1. Layering ๐Ÿง

In the IP architecture, the choice of what size datagram to send is made by a protocol at a layer above IP. We refer to such a protocol as a "packetization protocol". Packetization protocols are usually transport protocols (for example, TCP), but they can also be higher-layer protocols (for example, protocols built on top of UDP) ๐Ÿ“ฆ

Implementing PMTU Discovery in the packetization layers simplifies some of the inter-layer issues, but it has several drawbacks. The implementation may have to be redone for each packetization protocol, it becomes hard to share PMTU information between different packetization layers, and the connection-oriented state maintained by some packetization layers may not easily extend to save PMTU information for long periods ๐Ÿ˜ค

We therefore believe that the IP layer should store PMTU information and that the ICMP layer should process received Datagram Too Big messages ๐Ÿงš. The packetization layers must still be able to respond to changes in the Path MTU by changing the size of the datagrams they send, and they must also be able to specify that datagrams are sent with the DF bit set. We do not want the IP layer to simply set the DF bit in every packet, since a packetization layer (perhaps a UDP application outside the kernel) may be unable to change its datagram size ๐Ÿ™…โ€โ™€๏ธ. Protocols involving intentional fragmentation are inelegant, but they are sometimes successful (NFS being the primary example), and we do not want to break such protocols ๐Ÿ’œ

To support this layering, packetization layers require an extension of the IP service interface defined in [1]:

A way to learn of changes in the value of MMS_S, the "maximum send transport-message size", which is derived from the Path MTU by subtracting the minimum IP header size.

6.2. Storing PMTU information โœจ

Okay bestie, so in general the IP layer should be keeping a little notebook ๐Ÿ““ of every PMTU value it learns, and each value gets tied to a specific path. A path is identified by a source address, a destination address and an IP type-of-service. (Some implementations don't record the source address of paths, and that's acceptable for single-homed hosts, since those only have one possible source address ๐Ÿซถ)

Note: Some paths may be further distinguished by different security classifications. The details of such classifications are beyond the scope of this memo. ๐Ÿ”’

The obvious place to store this association is as a field in the routing table entries. A host won't have a route for every possible destination, but it should be able to cache a per-host route for every active destination. (This requirement is already imposed by the need to process ICMP Redirect messages, so it's not new drama ๐Ÿ’…)

When the first packet is sent to a host for which no per-host route exists, a route is chosen either from the set of per-network routes, or from the set of default routes. The PMTU fields in these route entries should be initialized to be the MTU of the associated first-hop data link, and they must never be changed by the PMTU Discovery process. (PMTU Discovery only creates or changes entries for per-host routes, so the per-network and default ones are off-limits ๐Ÿ™…โ€โ™€๏ธ) Until a Datagram Too Big message is received, the PMTU associated with the initially-chosen route is presumed to be accurate. Vibes until proven otherwise โœจ

When a Datagram Too Big message is received, the ICMP layer determines a new estimate for the Path MTU. It does this either from a non-zero Next-Hop MTU value in the packet, or using the method described in section 5. If a per-host route for this path doesn't exist, then one is created (almost as if a per-host ICMP Redirect is being processed, and the new route uses the same first-hop router as the current route ๐Ÿฆ‹). If the PMTU estimate associated with the per-host route is higher than the new estimate, then the value in the routing entry is changed.

The packetization layers must be notified about decreases in the PMTU. Any packetization layer instance (for example, a TCP connection) that is actively using the path must be notified if the PMTU estimate is decreased. Everyone using the path needs the tea โ˜•

Note: even if the Datagram Too Big message contains an Original Datagram Header that refers to a UDP packet, the TCP layer must be notified if any of its connections use the given path. ๐ŸŒธ

Also, the instance that sent the datagram that elicited the Datagram Too Big message should be notified that its datagram has been dropped, even if the PMTU estimate has not changed. That way it can retransmit the dropped datagram ๐Ÿ”

Note: The notification mechanism can be analogous to the mechanism used to provide notification of an ICMP Source Quench message. In some implementations (such as 4.2BSD-derived systems), the existing notification mechanism is not able to identify the specific connection involved, and so an additional mechanism is necessary. ๐Ÿ™ƒ Alternatively, an implementation can avoid the use of an asynchronous notification mechanism for PMTU decreases by postponing notification until the next attempt to send a datagram larger than the PMTU estimate. In this approach, when an attempt is made to SEND a datagram with the DF bit set, and the datagram is larger than the PMTU estimate, the SEND function should fail and return a suitable error indication. This approach may be more suitable to a connectionless packetization layer (such as one using UDP), which (in some implementations) may be hard to "notify" from the ICMP layer. In this case, the normal timeout-based retransmission mechanisms would be used to recover from the dropped datagrams. ๐Ÿงš

It's really important to understand that notifying the packetization layer instances using the path about the change in the PMTU is distinct from notifying a specific instance that a packet has been dropped. The latter should be done as soon as practical (i.e., asynchronously from the point of view of the packetization layer instance). The former may be delayed until a packetization layer instance wants to create a packet. Retransmission should be done for only those packets that are known to be dropped, as indicated by a Datagram Too Big message. Don't go resending everything, periodt ๐Ÿ›‘


6.3. Purging stale PMTU information ๐Ÿงน

Internetwork topology is dynamic, and routes change over time. The PMTU discovered for a given destination may be wrong if a new route comes into use, so PMTU information cached by a host can become stale. It's giving outdated ๐Ÿ•ฐ๏ธ

Because a host using PMTU Discovery always sets the DF bit, if the stale PMTU value is too large, this will be discovered almost immediately once a datagram is sent to the given destination. No such mechanism exists for realizing that a stale PMTU value is too small, so an implementation should "age" cached values. When a PMTU value has not been decreased for a while (on the order of 10 minutes), the PMTU estimate should be set to the first-hop data-link MTU, and the packetization layers should be notified of the change. This will cause the complete PMTU Discovery process to take place again. Fresh start, new era ๐ŸŒˆ

Note: an implementation should provide a means for changing the timeout duration, including setting it to "infinity". For example, hosts attached to an FDDI network which is then attached to the rest of the Internet via a slow serial line are never going to discover a new non-local PMTU, so they should not have to put up with dropped datagrams every 10 minutes. ๐Ÿข

An upper layer MUST not retransmit datagrams in response to an increase in the PMTU estimate, since this increase never comes in response to an indication of a dropped datagram. Don't resend just because the number went up, bestie ๐Ÿ™…โ€โ™€๏ธ

One approach to implementing PMTU aging is to add a timestamp field to the routing table entry. This field is initialized to a "reserved" value, indicating that the PMTU has never been changed. Whenever the PMTU is decreased in response to a Datagram Too Big message, the timestamp is set to the current time. โฐ

Once a minute, a timer-driven procedure runs through the routing table. For each entry whose timestamp is not "reserved" and is older than the timeout interval:

  • The PMTU estimate is set to the MTU of the associated first hop.
  • Packetization layers using this route are notified of the increase.

PMTU estimates may disappear from the routing table if the per-host routes are removed. This can happen in response to an ICMP Redirect message, or because certain routing-table daemons delete old routes after several minutes. Also, on a multi-homed host a topology change may result in the use of a different source interface. When this happens, if the packetization layer is not notified then it may continue to use a cached PMTU value that is now too small, which is a whole mess ๐Ÿ˜ญ One solution is to notify the packetization layer of a possible PMTU change whenever a Redirect message causes a route change, and whenever a route is simply deleted from the routing table. ๐Ÿซถ

Note: a more sophisticated method for detecting PMTU increases is described in section 7.1. ๐Ÿ‘€

6.4. TCP layer actions ๐ŸŽ€

The TCP layer must track the PMTU for the destination of a connection, and it should not send datagrams that would be larger than this. A simple implementation could ask the IP layer for this value (using the GET_MAXSIZES interface described in [1]) each time it created a new segment, but this could be inefficient. Moreover, TCP implementations that follow the "slow-start" congestion-avoidance algorithm [4] typically calculate and cache several other values derived from the PMTU. It may be simpler to receive asynchronous notification when the PMTU changes, so that these variables may be updated. Less asking, more being told ๐Ÿ’…

A TCP implementation must also store the MSS value received from its peer (which defaults to 536), and not send any segment larger than this MSS, regardless of the PMTU. In 4.xBSD-derived implementations, this requires adding an additional field to the TCP state record. ๐Ÿ“

Finally, when a Datagram Too Big message is received, it implies that a datagram was dropped by the router that sent the ICMP message. It is sufficient to treat this as any other dropped segment, and wait until the retransmission timer expires to cause retransmission of the segment. If the PMTU Discovery process requires several steps to estimate the right PMTU, this could delay the connection by many round-trip times. Ugh, slow ๐ŸŒ

Alternatively, the retransmission could be done in immediate response to a notification that the Path MTU has changed, but only for the specific connection specified by the Datagram Too Big message. The datagram size used in the retransmission should, of course, be no larger than the new PMTU. ๐Ÿฆ‹

Note: One MUST not retransmit in response to every Datagram Too Big message, since a burst of several oversized segments will give rise to several such messages and hence several retransmissions of the same data. If the new estimated PMTU is still wrong, the process repeats, and there is an exponential growth in the number of superfluous segments sent! ๐Ÿ’€ This means that the TCP layer must be able to recognize when a Datagram Too Big notification actually decreases the PMTU that it has already used to send a datagram on the given connection, and should ignore any other notifications. ๐Ÿ™ˆ

Modern TCP implementations incorporate "congestion advoidance" and "slow-start" algorithms to improve performance [4]. Unlike a retransmission caused by a TCP retransmission timeout, a retransmission caused by a Datagram Too Big message should not change the congestion window. It should, however, trigger the slow-start mechanism (i.e., only one segment should be retransmitted until acknowledgements begin to arrive again). Gentle restart, no panic ๐Ÿง˜โ€โ™€๏ธ

TCP performance can be reduced if the sender's maximum window size is not an exact multiple of the segment size in use (this is not the congestion window size, which is always a multiple of the segment size). In many system (such as those derived from 4.2BSD), the segment size is often set to 1024 octets, and the maximum window size (the "send space") is usually a multiple of 1024 octets, so the proper relationship holds by default. If PMTU Discovery is used, however, the segment size may not be a submultiple of the send space, and it may change during a connection. This means that the TCP layer may need to change the transmission window size when PMTU Discovery changes the PMTU value. The maximum window size should be set to the greatest multiple of the segment size (PMTU - 40) that is less than or equal to the sender's buffer space size. Math is the mood ๐Ÿงฎโœจ

PMTU Discovery does not affect the value sent in the TCP MSS option, because that value is used by the other end of the connection, which may be using an unrelated PMTU value. Different ends, different vibes ๐Ÿซถ

6.5. Issues for other transport protocols ๐Ÿฆ‹

okay so some transport protocols (like ISO TP4 [3]) are not allowed to repacketize when they retransmit ๐Ÿ’€. Once a datagram of a certain size has been sent, its contents can't be split into smaller datagrams for the retry. In that case the original datagram SHOULD be retransmitted without the DF bit set, so it's allowed to get fragmented as needed to reach its destination. Subsequent datagrams, when they're transmitted for the first time, SHOULD be no larger than the Path MTU allows, and SHOULD have the DF bit set โœจ.

The Sun Network File System (NFS) uses a Remote Procedure Call (RPC) protocol [11] that, in many cases, sends datagrams that must be fragmented even for the first-hop link ๐Ÿ™ˆ. This might improve performance in certain cases, but it is known to cause reliability and performance problems, especially when the client and server are separated by routers.

We recommend that NFS implementations use PMTU Discovery whenever routers are involved ๐Ÿ’œ. Most NFS implementations allow the RPC datagram size to be changed at mount-time (indirectly, by changing the effective file system block size), but might require some modification to support changes later on.

Also, since a single NFS operation cannot be split across several UDP datagrams, certain operations (primarily, those operating on file names and directories) require a minimum datagram size that may be larger than the PMTU. NFS implementations SHOULD NOT reduce the datagram size below this threshold, even if PMTU Discovery suggests a lower value. (Of course, in this case datagrams SHOULD NOT be sent with DF set ๐ŸŒธ.)


6.6. Management interface ๐ŸŽ€

We suggest that an implementation provide a way for a system utility program to:

  • Specify that PMTU Discovery not be done on a given route.
  • Change the PMTU value associated with a given route.

The former can be accomplished by associating a flag with the routing entry. When a packet is sent via a route with this flag set, the IP layer leaves the DF bit clear no matter what the upper layer requests ๐Ÿงš.

These features might be used to work around an anomalous situation, or by a routing protocol implementation that is able to obtain Path MTU values.

The implementation SHOULD also provide a way to change the timeout period for aging stale PMTU information โฐ.


7. Likely values for Path MTUs ๐Ÿ“

The algorithm recommended in section 5 for "searching" the space of Path MTUs is based on a table of values that severely restricts the search space. We describe here a table of MTU values that, as of this writing, represents all major data-link technologies in use in the Internet ๐ŸŒˆ.

In table 7-1, data links are listed in order of decreasing MTU, and grouped so that each set of similar MTUs is associated with a "plateau" equal to the lowest MTU in the group. (The table also includes some entries not currently associated with a data link, and gives references where available.) Where a plateau represents more than one MTU, the table shows the maximum inaccuracy associated with the plateau, as a percentage.

We do not expect that the values in the table, especially for higher MTU levels, are going to be valid forever. The values given here are an implementation suggestion, NOT a specification or requirement. Implementors SHOULD use up-to-date references to pick a set of plateaus, because it is important that the table not contain too many entries or the process of searching for a PMTU might waste Internet resources. Implementors SHOULD also make it convenient for customers without source code to update the table values in their systems (for example, the table in a BSD-derived Unix kernel could be changed using a new "ioctl" command) ๐Ÿ”ฅ.

Note: It might be a good idea to add a few table entries for values equal to small powers of 2 plus 40 (for the IP and TCP headers), where no similar values exist, since this seems to be a reasonably non-arbitrary way of choosing arbitrary values. The table might also contain entries for values slightly less than large powers of 2, in case MTUs are defined near those values (it is better in this case for the table entries to be low than to be high, or else the next lowest plateau may be chosen instead).

7.1. A better way to detect PMTU increases ๐Ÿ‘‘

Section 6.3 suggests detecting increases in the PMTU value by periodically increasing the PMTU estimate to the first-hop MTU. Since it is likely that this process will simply "rediscover" the current PMTU estimate, at the cost of several dropped datagrams, it SHOULD NOT be done often ๐Ÿ˜ญ.

A better approach is to periodically increase the PMTU estimate to the next-highest value in the plateau table (or the first-hop MTU, if that is smaller). If the increased estimate is wrong, at most one round-trip time is wasted before the correct value is rediscovered. If the increased estimate is still too low, a higher estimate will be attempted somewhat later.

Because it may take several such periods to discover a significant increase in the PMTU, we recommend that a short timeout period SHOULD be used after the estimate is increased, and a longer timeout be used

Plateau    MTU    Comments                      Reference
------     ---    --------                      ---------
           65535  Official maximum MTU          RFC 791
           65535  Hyperchannel                  RFC 1044
65535
32000             Just in case
           17914  16Mb IBM Token Ring           ref. [6]
17914
           8166   IEEE 802.4                    RFC 1042
8166
           4464   IEEE 802.5 (4Mb max)          RFC 1042
           4352   FDDI (Revised)                RFC 1188
4352 (1%)
           2048   Wideband Network              RFC 907
           2002   IEEE 802.5 (4Mb recommended)  RFC 1042
2002 (2%)
           1536   Exp. Ethernet Nets            RFC 895
           1500   Ethernet Networks             RFC 894
           1500   Point-to-Point (default)      RFC 1134
           1492   IEEE 802.3                    RFC 1042
1492 (3%)
           1006   SLIP                          RFC 1055
           1006   ARPANET                       BBN 1822
1006
           576    X.25 Networks                 RFC 877
           544    DEC IP Portal                 ref. [10]
           512    NETBIOS                       RFC 1088
           508    IEEE 802/Source-Rt Bridge     RFC 1042
           508    ARCNET                        RFC 1051
508 (13%)
           296    Point-to-Point (low delay)    RFC 1144
296
68                Official minimum MTU          RFC 791

                Table 7-1:  Common MTUs in the Internet

after the PMTU estimate is decreased because of a Datagram Too Big message. For example, after the PMTU estimate is decreased, the timeout SHOULD be set to 10 minutes; once this timer expires and a larger MTU is attempted, the timeout can be set to a much smaller value (say, 2 minutes). In no case SHOULD the timeout be shorter than the estimated round-trip time, if this is known ๐Ÿ’–.


8. Security considerations ๐Ÿ’€

This Path MTU Discovery mechanism makes possible two denial-of-service attacks, both based on a malicious party sending false Datagram Too Big messages to an Internet host. Bestie, this is serious ๐Ÿ˜ค.

In the first attack, the false message indicates a PMTU much smaller than reality. This SHOULD NOT entirely stop data flow, since the victim host SHOULD NOT set its PMTU estimate below the absolute minimum, but at 8 octets of IP data per datagram, progress could be slow ๐Ÿข.

In the other attack, the false message indicates a PMTU greater than reality. If believed, this could cause temporary blockage as the victim sends datagrams that will be dropped by some router. Within one round-trip time, the host would discover its mistake (receiving Datagram Too Big messages from that router), but frequent repetition of this attack could cause lots of datagrams to be dropped. A host, however, SHOULD NOT raise its estimate of the PMTU based on a Datagram Too Big message, so it SHOULD NOT be vulnerable to this attack ๐Ÿ›ก๏ธ.

A malicious party could also cause problems if it could stop a victim from receiving legitimate Datagram Too Big messages, but in this case there are simpler denial-of-service attacks available ๐Ÿ™ƒ.


References ๐Ÿ“š

[1] R. Braden, ed. Requirements for Internet Hosts -- Communication Layers. RFC 1122, SRI Network Information Center, October, 1989.

[2] Geof Cooper. IP Datagram Sizes. Electronic distribution of the TCP-IP Discussion Group, Message-ID <8705240517.AA01407@apolling.imagen.uucp>.

[3] ISO. ISO Transport Protocol Specification: ISO DP 8073. RFC 905, SRI Network Information Center, April, 1984.

[4] Van Jacobson. Congestion Avoidance and Control. In Proc. SIGCOMM '88 Symposium on Communications Architectures and Protocols, pages 314-329. Stanford, CA, August, 1988.

[5] C. Kent and J. Mogul. Fragmentation Considered Harmful. In Proc. SIGCOMM '87 Workshop on Frontiers in Computer Communications Technology. August, 1987.

[6] Drew Daniel Perkins. Private Communication.

[7] J. Postel. Internet Control Message Protocol. RFC 792, SRI Network Information Center, September, 1981.

[8] J. Postel. Internet Protocol. RFC 791, SRI Network Information Center, September, 1981.

[9] J. Postel. The TCP Maximum Segment Size and Related Topics. RFC 879, SRI Network Information Center, November, 1983.

[10] Michael Reilly. Private Communication.

[11] Sun Microsystems, Inc. RPC: Remote Procedure Call Protocol. RFC 1057, SRI Network Information Center, June, 1988.


Authors' Addresses ๐Ÿซถ

Jeffrey Mogul
Digital Equipment Corporation Western Research Laboratory
100 Hamilton Avenue
Palo Alto, CA  94301

Phone: (415) 853-6643
EMail: mogul@decwrl.dec.com


Steve Deering
Xerox Palo Alto Research Center
3333 Coyote Hill Road
Palo Alto, CA  94304

Phone: (415) 494-4839
EMail: deering@xerox.com
๐Ÿ“ฆ where this page comes from