RFC 9486

IPv6 Options for In Situ Operations, Administration, and Maintenance (IOAM)

📅 published September 2023, from ietf

RFC 9486: IPv6 Options for In Situ OAM (IOAM) 💖

Okay bestie, this one is about IOAM options for IPv6 ✨. It's an Internet Standards Track document from the IETF, published September 2023. The editors are S. Bhandari (Thoughtspot) and F. Brockners (Cisco), both marked Ed. for editor. It's listed under ISSN 2070-1721. The header doesn't list anything it Obsoletes or Updates, so there's no drama there 🙈.

Abstract ✨

okay so IOAM is basically how you write telemetry and operational notes into the packet itself while it's zooming along a path between two points in the network 📦🦋. This doc explains how IOAM Data-Fields get wrapped up inside IPv6 💅.

Status of This Memo 📜

This is an Internet Standards Track document, and it's a product of the Internet Engineering Task Force (IETF) 👑. It represents the consensus of the IETF community. It went through public review and got approved for publication by the Internet Engineering Steering Group (IESG). If you want more on Internet Standards, check Section 2 of RFC 7841.

Info about the current status of this doc, any errata, and how to send feedback lives at https://www.rfc-editor.org/info/rfc9486.

Copyright (c) 2023 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication. Please read these carefully, because they spell out your rights and restrictions for this document. Code Components extracted from this document must include Revised BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come without warranty, as described in the Revised BSD License.

Table of Contents 🗺️

   1.  Introduction
   2.  Conventions
     2.1.  Requirements Language
     2.2.  Abbreviations
   3.  In situ OAM Metadata Transport in IPv6
   4.  IOAM Deployment in IPv6 Networks
     4.1.  Considerations for IOAM Deployment and Implementation in
           IPv6 Networks
     4.2.  IOAM-Domains Bounded by Hosts
     4.3.  IOAM-Domains Bounded by Network Devices
   5.  Security Considerations
     5.1.  Applicability of Authentication Header (AH)
   6.  IANA Considerations
   7.  References
     7.1.  Normative References
     7.2.  Informative References
   Acknowledgements
   Contributors
   Authors' Addresses

1. Introduction 🌸

So, IOAM records operational and telemetry information in the packet while it traverses a path between two points in the network. The IOAM concepts, the nomenclature, and the IOAM Data-Fields are all defined in [RFC9197]. This document is the one that explains how IOAM Data-Fields get encapsulated in IPv6 [RFC8200], and it also covers the deployment requirements for networks that use IPv6-encapsulated IOAM Data-Fields 🧚.

The words "encapsulation" and "decapsulation" mean the same thing here as they do in [RFC9197]. An IOAM encapsulating node puts one or more IOAM Option-Types into packets that have IOAM enabled for them 💜.

2. Conventions 📚

2.1. Requirements Language

Here's the key word rundown 💅. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.

2.2. Abbreviations

Here are the abbreviations used in this document 🍓:

   E2E:       Edge-to-Edge

   IOAM:      In situ Operations, Administration, and Maintenance as
              defined in [RFC9197]

   OAM:       Operations, Administration, and Maintenance

   POT:       Proof of Transit

3. In situ OAM Metadata Transport in IPv6 🦋

IOAM in IPv6 is used to level up diagnostics of IPv6 networks 📈. It works alongside other mechanisms that do the same job, like the "IPv6 Performance and Diagnostic Metrics (PDM) Destination Option" described in [RFC8250].

At the time this document was written, a few implementations of IOAM for IPv6 already existed. For example, there's IOAM for IPv6 in the Linux Kernel, supported from kernel version 5.15 onward, at https://github.com/torvalds/linux/commit/7c804e91df523a37c29e183ea2b10ac73c3a4f3d. There's also IOAM for IPv6 in Vector Packet Processing (VPP), at https://docs.fd.io/vpp/17.04/ioam_ipv6_doc.html 🔥.

IOAM Data-Fields can be encapsulated with two types of extension headers in IPv6 packets: either the hop-by-hop options header or the destination options header. Multiple options with the same option type MAY appear in the same hop-by-hop options or destination options header, as long as they have distinct content 🫶.

An IPv6 packet that carries IOAM data in an extension header can also have other extension headers, as long as they're compliant with [RFC8200] ✨.

Here's the option format, straight from the diagram 👑:

   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |  Option-Type  |  Opt Data Len |   Reserved    | IOAM Opt-Type |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+
   |                                                               |  |
   .                                                               .  I
   .                                                               .  O
   .                                                               .  A
   .                                                               .  M
   .                                                               .  .
   .                          Option Data                          .  O
   .                                                               .  P
   .                                                               .  T
   .                                                               .  I
   .                                                               .  O
   .                                                               .  N
   |                                                               |  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+

        Figure 1: IPv6 Hop-by-Hop and Destination Option Format for
                         Carrying IOAM Data- Fields

Now the fields, one by one 🍭:

  • Option-Type: an 8-bit option type identifier, as defined in Section 6.
  • Opt Data Len: an 8-bit unsigned integer. It's the length of this option in octets, not counting the first 2 octets.
  • Reserved: an 8-bit field. It MUST be set to zero by the source 🙅‍♀️.
  • IOAM Option-Type: shortened to "IOAM Opt-Type" in the diagram above. It's an 8-bit field, as defined in Section 4.1 of [RFC9197].
  • Option Data: a variable-length field. The data depends on the Option-Type, which is detailed below.
    • Pre-allocated Trace Option: the IOAM Pre-allocated Trace Option-Type, defined in Section 4.4 of [RFC9197], is represented as an IPv6 option in the hop-by-hop extension header:
      • Option-Type: 0x31 (8-bit identifier of the IPv6 Option-Type for IOAM).
      • IOAM Type: IOAM Pre-allocated Trace Option-Type.
    • Proof of Transit Option-Type: the IOAM POT Option-Type, defined in Section 4.5 of [RFC9197], is represented as an IPv6 option in the hop-by-hop extension header:
      • Option-Type: 0x31 (8-bit identifier of the IPv6 Option-Type for IOAM).
      • IOAM Type: IOAM POT Option-Type.
    • Edge-to-Edge Option: the IOAM E2E Option, defined in Section 4.6 of [RFC9197], is represented as an IPv6 option in the destination extension header:
      • Option-Type: 0x11 (8-bit identifier of the IPv6 Option-Type for IOAM).
      • IOAM Type: IOAM E2E Option-Type.
    • Direct Export (DEX) Option: the IOAM Direct Export Option-Type, defined in Section 3.2 of [RFC9326], is represented as an IPv6 option in the hop-by-hop extension header:
      • Option-Type: 0x11 (8-bit identifier of the IPv6 Option-Type for IOAM).
      • IOAM Type: IOAM Direct Export (DEX) Option-Type.

Alignment is a big deal here 📏. All the IOAM IPv6 options defined in this document have alignment requirements. Specifically, they all need to be aligned on multiples of 4 bytes. That way, the fields specified in [RFC9197] sit at a multiple-of-4 offset from the start of the hop-by-hop and destination options header 💖.

IPv6 options can be at most 255 octets long 🙈. So the total length of IOAM Option-Types, including all their data fields, is also capped at 255 octets when they're encapsulated into IPv6.

4. IOAM Deployment in IPv6 Networks 🌈

4.1. Considerations for IOAM Deployment and Implementation in IPv6 Networks

IOAM deployments in IPv6 networks MUST take the following considerations and requirements into account 🔥.

C1: IOAM MUST be deployed in an IOAM-Domain. An IOAM-Domain is a set of nodes that use IOAM, and it's bounded by its perimeter or edge. The nodes forming an IOAM-Domain may be connected to the same physical infrastructure (like a service provider's network). They may also be remotely connected to each other (like an enterprise VPN or an overlay). It's expected that all nodes in an IOAM-Domain are managed by the same administrative entity. For more details on IOAM-Domains, please refer to [RFC9197] 🧠.

C2: Implementations of IOAM MUST ensure that adding IOAM Data-Fields does not alter the way routers forward packets or the forwarding decisions they make 🚦. Packets with added IOAM information must follow the same path within the domain as an identical packet without IOAM information would, even when Equal-Cost Multipath (ECMP) is in play. This matters a lot for deployments where IOAM Data-Fields only get added "on-demand". So, implementations of IOAM MUST ensure that ECMP behavior is the same for packets with and without IOAM Data-Fields. For IOAM to work in IPv6 networks, IOAM MUST be explicitly enabled per interface on every node within the IOAM-Domain. Unless a particular interface is explicitly enabled (in other words, explicitly configured) for IOAM, a router MUST ignore IOAM Options 🙅‍♀️.

C3: To keep packets intact in an IOAM-Domain, the Maximum Transmission Unit (MTU) of transit routers and switches must be configured to a value that does not lead to an "ICMP Packet Too Big" error message being sent to the originator and the packet being dropped. The PMTU tolerance range must be identified, and IOAM encapsulation operations or data field insertion must not exceed this range. Control of the MTU is critical to the proper operation of IOAM 😭. The PMTU tolerance must be identified through configuration, and IOAM operations must not push the packet size beyond PMTU.

C4: [RFC8200] rules out inserting IOAM data directly into the original IPv6 header of in-flight packets 🙃. IOAM deployments that don't encapsulate or decapsulate IOAM on the host, but still want to do that on transit nodes, MUST add an additional IPv6 header to the original packet. The IOAM data then goes into this additional IPv6 header 🦋.

4.2. IOAM-Domains Bounded by Hosts

okay so when the IOAM-Domain is bounded by hosts 🖥️, the hosts do the encapsulation and decapsulation themselves 💅 that means hosts put the IOAM Data-Fields straight into the IPv6 header, or pull them straight back out. IOAM data rides along in IPv6 packets as hop-by-hop or destination options, exactly as this document specifies ✨

4.3. IOAM-Domains Bounded by Network Devices

now for when the IOAM-Domain is bounded by network devices 🌐. Routers are the classic example, and they form the edge of the IOAM-Domain 🧚. The network devices handle the IOAM Data-Field encapsulation and decapsulation, and they wrap the IOAM Data-Fields in an additional, outer, IPv6 header that carries them. Very main character energy 👑

5. Security Considerations 🔐

ngl this section is the serious one, so pay attention bestie 🥹 This document describes how IOAM Data-Fields get encapsulated in IPv6. For general IOAM security stuff, check [RFC9197]. The security considerations for each specific IOAM Data-Field case (that's Trace, POT, and E2E) are also described and defined in [RFC9197].

Since this document adds new options for IPv6, the security considerations of [RFC8200] and [RFC8250] apply too 🫶

From a network-protection point of view, there's an assumed trust model. Any node that adds IOAM to a packet, removes IOAM from a packet, or modifies IOAM Data-Fields of a packet is assumed to be allowed to do so. By default, packets that include IPv6 extension headers with IOAM information MUST NOT be leaked through the boundaries of the IOAM-Domain 🚫

IOAM-Domain boundary routers MUST filter any incoming traffic from outside the IOAM-Domain that contains IPv6 extension headers with IOAM information. They MUST also filter any outgoing traffic leaving the IOAM-Domain that contains IPv6 extension headers with IOAM information. Non-negotiable, periodt 💀

In the general case, an IOAM node only adds, removes, or modifies an IPv6 extension header with IOAM information if the directive to do so comes from a trusted source and the directive is validated. Trust, but verify 🔍

Problems may occur if the above behaviors are not implemented, or if the assumed trust model is violated (for example, through a security breach). That's a whole disaster 😭 In addition to the security considerations discussed in [RFC9197], the security considerations associated with IPv6 extension headers listed in [RFC9098] apply.

5.1. Applicability of Authentication Header (AH)

the network devices in an IOAM-Domain are trusted to add, update, and remove IOAM options according to the constraints specified in [RFC8200] 🔒 The IOAM-Domain does not rely on the AH as defined in [RFC4302] to secure IOAM options. Using IOAM options with AH and processing them that way is not defined in this document. Future documents may define the use of IOAM with AH and its processing, so keep an eye out 👀🦋

6. IANA Considerations 📋

IANA has assigned the IPv6 Option-Types from the "Destination Options and Hop-by-Hop Options" subregistry of "Internet Protocol Version 6 (IPv6) Parameters" at https://www.iana.org/assignments/ipv6-parameters/ 🍓

+=======+===================+===================+===========+
| Hex   | Binary Value      | Description       | Reference |
| Value +=====+=====+=======+                   |           |
|       | act | chg | rest  |                   |           |
+=======+=====+=====+=======+===================+===========+
| 0x11  | 00  | 0   | 10001 | IOAM Destination  | RFC 9486  |
|       |     |     |       | Option and IOAM   |           |
|       |     |     |       | Hop-by-Hop Option |           |
+-------+-----+-----+-------+-------------------+-----------+
| 0x31  | 00  | 1   | 10001 | IOAM Destination  | RFC 9486  |
|       |     |     |       | Option and IOAM   |           |
|       |     |     |       | Hop-by-Hop Option |           |
+-------+-----+-----+-------+-------------------+-----------+

                             Table 1

7. References 📚

7.1. Normative References

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, https://www.rfc-editor.org/info/rfc2119.

[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, https://www.rfc-editor.org/info/rfc8174.

[RFC9197] Brockners, F., Ed., Bhandari, S., Ed., and T. Mizrahi, Ed., "Data Fields for In Situ Operations, Administration, and Maintenance (IOAM)", RFC 9197, DOI 10.17487/RFC9197, May 2022, https://www.rfc-editor.org/info/rfc9197.

[RFC9326] Song, H., Gafni, B., Brockners, F., Bhandari, S., and T. Mizrahi, "In Situ Operations, Administration, and Maintenance (IOAM) Direct Exporting", RFC 9326, DOI 10.17487/RFC9326, November 2022, https://www.rfc-editor.org/info/rfc9326.

7.2. Informative References

[IPV6-RECORD-ROUTE] Kitamura, H., "Record Route for IPv6 (RR6) Hop-by-Hop Option Extension", Work in Progress, Internet-Draft, draft-kitamura-ipv6-record-route-00, 17 November 2000, https://datatracker.ietf.org/doc/html/draft-kitamura-ipv6-record-route-00.

[RFC4302] Kent, S., "IP Authentication Header", RFC 4302, DOI 10.17487/RFC4302, December 2005, https://www.rfc-editor.org/info/rfc4302.

[RFC8200] Deering, S. and R. Hinden, "Internet Protocol, Version 6 (IPv6) Specification", STD 86, RFC 8200, DOI 10.17487/RFC8200, July 2017, https://www.rfc-editor.org/info/rfc8200.

[RFC8250] Elkins, N., Hamilton, R., and M. Ackermann, "IPv6 Performance and Diagnostic Metrics (PDM) Destination Option", RFC 8250, DOI 10.17487/RFC8250, September 2017, https://www.rfc-editor.org/info/rfc8250.

[RFC9098] Gont, F., Hilliard, N., Doering, G., Kumari, W., Huston, G., and W. Liu, "Operational Implications of IPv6 Packets with Extension Headers", RFC 9098, DOI 10.17487/RFC9098, September 2021, https://www.rfc-editor.org/info/rfc9098.

Acknowledgements 🫶

The authors would like to thank Tom Herbert, Éric Vyncke, Nalini Elkins, Srihari Raghavan, Ranganathan T S, Karthik Babu Harichandra Babu, Akshaya Nadahalli, Stefano Previdi, Hemant Singh, Erik Nordmark, LJ Wobker, Mark Smith, Andrew Yourtchenko, and Justin Iurman for the comments and advice 💖 For the IPv6 encapsulation, this document leverages concepts described in [IPV6-RECORD-ROUTE]. The authors would like to acknowledge the work done by the author Hiroshi Kitamura and the people involved in writing it 🌸

Contributors 🌈

This document was the collective effort of several authors. The text and content were contributed by the editors and the coauthors listed below ✨

Carlos Pignataro Cisco Systems, Inc. 7200-11 Kit Creek Road Research Triangle Park, NC 27709 United States of America Email: cpignata@cisco.com

Hannes Gredler RtBrick Inc. Email: hannes@rtbrick.com

John Leddy Email: john@leddy.net

Stephen Youell JP Morgan Chase 25 Bank Street London E14 5JP United Kingdom Email: stephen.youell@jpmorgan.com

Tal Mizrahi Huawei Network.IO Innovation Lab Israel Email: tal.mizrahi.phd@gmail.com

Aviv Kfir Mellanox Technologies, Inc. 350 Oakmead Parkway, Suite 100 Sunnyvale, CA 94085 United States of America Email: avivk@mellanox.com

Barak Gafni Mellanox Technologies, Inc. 350 Oakmead Parkway, Suite 100 Sunnyvale, CA 94085 United States of America Email: gbarak@mellanox.com

Petr Lapukhov Facebook 1 Hacker Way Menlo Park, CA 94025 United States of America Email: petr@fb.com

Mickey Spiegel Barefoot Networks, an Intel company 4750 Patrick Henry Drive Santa Clara, CA 95054 United States of America Email: mickey.spiegel@intel.com

Suresh Krishnan Kaloom Email: suresh@kaloom.com

Rajiv Asati Cisco Systems, Inc. 7200 Kit Creek Road Research Triangle Park, NC 27709 United States of America Email: rajiva@cisco.com

Mark Smith PO BOX 521 Heidelberg VIC 3084 Australia Email: markzzzsmith+id@gmail.com

Authors' Addresses 🦋

Shwetha Bhandari (editor) Thoughtspot 3rd Floor, Indiqube Orion 24th Main Rd, Garden Layout, HSR Layout Bangalore 560 102 Karnataka India Email: shwetha.bhandari@thoughtspot.com

Frank Brockners (editor) Cisco Systems, Inc. Hansaallee 249, 3rd Floor 40549 Duesseldorf Germany Email: fbrockne@cisco.com

📦 where this page comes from