RFC 9452

Network Service Header (NSH) Encapsulation for In Situ OAM (IOAM) Data

📅 published August 2023, from ietf

RFC 9452: Network Service Header (NSH) Encapsulation for In Situ OAM (IOAM) Data 🎀

Okay bestie, this one is by F. Brockners, Ed. (Cisco) and S. Bhandari, Ed. (Thoughtspot), published August 2023 🌸 It's on the Standards Track (ISSN 2070-1721), so it's a real Internet standard, not just vibes. The header doesn't list any RFCs it obsoletes or updates, so it's standing on its own 💅

Abstract

Okay so IOAM (In situ Operations, Administration, and Maintenance) is all about recording and collecting operational and telemetry info while a packet travels along a path between two points in the network 🦋 This document explains how IOAM-Data-Fields get encapsulated with the Network Service Header (NSH). It's giving packet diary 📓✨

Status of This Memo

This is an Internet Standards Track document 🌈

It's a product of the Internet Engineering Task Force (IETF) and represents the consensus of the IETF community. It got public review and was approved for publication by the Internet Engineering Steering Group (IESG). If you want more info on Internet Standards, check Section 2 of RFC 7841 💜

For the current status, any errata, and how to send feedback, go to https://www.rfc-editor.org/info/rfc9452 🔗

Copyright (c) 2023 IETF Trust and the persons identified as the document authors. All rights reserved 🫶

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they stood on the publication date. Please actually read these, because they spell out your rights and restrictions for this doc 📜 Any Code Components pulled out of this document must include the Revised BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come without warranty, as the Revised BSD License says 💅

Table of Contents

   1.  Introduction
   2.  Conventions
   3.  IOAM Encapsulation with NSH
   4.  IANA Considerations
   5.  Security Considerations
   6.  References
     6.1.  Normative References
     6.2.  Informative References
   Appendix A.  Discussion of the IOAM-Encapsulation Approach
   Acknowledgments
   Contributors
   Authors' Addresses

1. Introduction ✨

IOAM, defined in [RFC9197], records and collects OAM information while a packet moves through a particular network domain 🛣️ "In situ" just means the OAM data gets added onto the data packets themselves, instead of being sent in packets that exist only for OAM 🍓

This document defines how IOAM-Data-Fields get carried as part of the Network Service Header (NSH) encapsulation [RFC8300], for the Service Function Chaining (SFC) Architecture [RFC7665]. The IOAM-Data-Fields themselves are defined in [RFC9197] 💖

2. Conventions 🎀

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here 🧠

Abbreviations used in this document:

  • IOAM: In situ Operations, Administration, and Maintenance
  • MD: NSH Metadata, see [RFC7665]
  • NSH: Network Service Header
  • OAM: Operations, Administration, and Maintenance
  • SFC: Service Function Chaining
  • TLV: Type, Length, Value

3. IOAM Encapsulation with NSH 🦋

The NSH is defined in [RFC8300] 💜 IOAM-Data-Fields are carried as the NSH payload, using a Next Protocol header that comes right after the NSH headers. Then an IOAM header holding the IOAM-Data-Fields gets added 🧚

The IOAM-Data-Fields MUST follow the definitions for their corresponding IOAM Option-Types (for example, see Section 4 of [RFC9197] and Section 3.2 of [RFC9326]). In an administrative domain where IOAM is used, inserting the IOAM header into NSH is enabled at the NSH tunnel endpoints. Those endpoints are also set up to act as the encapsulating and decapsulating nodes for IOAM 🎯

The operator MUST ensure that SFC-aware nodes along the Service Function Path support IOAM. Otherwise, packets might get dropped (see the last paragraph of this section, plus Section 2.2 of [RFC8300]) 💀 The IOAM transit nodes (for example, a Service Function Forwarder (SFF)) MUST process all the IOAM headers that are relevant based on their configuration 🔥 For a discussion of the deployment-related stuff around IOAM-Data-Fields, see [RFC9378] 📚

Here's the layout, Figure 1 👑

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+
   |Ver|O|U|    TTL    |   Length  |U|U|U|U|MD Type| NP = 0x06  |  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  N
   |          Service Path Identifier              | Service Index |  S
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  H
   |                            ...                                |  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+
   |  IOAM-Type    | IOAM HDR Len  |    Reserved   | Next Protocol |  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  I
   !                                                               |  O
   !                                                               |  A
   ~                 IOAM Option and Optional Data Space           ~  M
   |                                                               |  |
   |                                                               |  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+
   |                                                               |
   |                                                               |
   |                 Payload + Padding (L2/L3/...)                 |
   |                                                               |
   |                                                               |
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Figure 1 ✨

The NSH header and fields are defined in [RFC8300]. The O bit MUST be handled by following the rules in [RFC9451] 🫶 The "NSH Next Protocol" value (called NP in the diagram above) is 0x06 💅

Here's what the IOAM-related fields in NSH mean 🌸

  • IOAM-Type: an 8-bit field that defines the IOAM Option-Type, as listed in the "IOAM Option-Type" registry specified in [RFC9197].
  • IOAM HDR Len: an 8-bit field holding the length of the IOAM header in multiples of 4 octets. It includes the IOAM-Type and IOAM HDR Len fields themselves.
  • Reserved bits: these are there for future use. The reserved bits MUST be set to 0x0 on transmission and ignored on receipt 🙈
  • Next Protocol: an 8-bit unsigned integer that determines the type of header following IOAM. The semantics of this field are identical to the Next Protocol field in [RFC8300].
  • IOAM Option and Optional Data Space: this holds the IOAM-Data-Fields, as specified by the IOAM-Type field. The IOAM-Data-Fields are defined for the corresponding IOAM Option-Type (for example, see Section 4 of [RFC9197] and Section 3.2 of [RFC9326]). They are always aligned by 4 octets, so there is no padding field 🧁

Multiple IOAM Option-Types MAY be included within the NSH encapsulation. For example, if an NSH encapsulation has two IOAM Option-Types before a data payload, the Next Protocol field of the first IOAM option will contain 0x06, while the Next Protocol field of the second IOAM Option-Type will contain the "NSH Next Protocol" number that indicates the type of the data payload 🧩 Whether the IOAM Active and Loopback flags [RFC9322] apply is outside the scope of this document, and it may be specified in the future 🔮

If the IOAM Incremental Trace Option-Type is used, an SFC-aware node that acts as an IOAM transit node needs to adjust the IOAM HDR Len field accordingly. See Section 4.4 of [RFC9197] 🪄

Per Section 2.2 of [RFC8300], packets with unsupported Next Protocol values SHOULD be silently dropped by default. So if a packet with IOAM hits an NSH-based forwarding node (like an SFF) that doesn't support the IOAM header, it SHOULD drop the packet 🚫 How to maintain and notify about these events is outside the scope of this document 🤷‍♀️

6.2. Informative References

[RFC7665] Halpern, J., Ed. and C. Pignataro, Ed., "Service Function Chaining (SFC) Architecture", RFC 7665, DOI 10.17487/RFC7665, October 2015, <https://www.rfc-editor.org/info/rfc7665>.

[RFC9322] Mizrahi, T., Brockners, F., Bhandari, S., Gafni, B., and M. Spiegel, "In Situ Operations, Administration, and Maintenance (IOAM) Loopback and Active Flags", RFC 9322, DOI 10.17487/RFC9322, November 2022, <https://www.rfc-editor.org/info/rfc9322>.

[RFC9326] Song, H., Gafni, B., Brockners, F., Bhandari, S., and T. Mizrahi, "In Situ Operations, Administration, and Maintenance (IOAM) Direct Exporting", RFC 9326, DOI 10.17487/RFC9326, November 2022, <https://www.rfc-editor.org/info/rfc9326>.

[RFC9378] Brockners, F., Ed., Bhandari, S., Ed., Bernier, D., and T. Mizrahi, Ed., "In Situ Operations, Administration, and Maintenance (IOAM) Deployment", RFC 9378, DOI 10.17487/RFC9378, April 2023, <https://www.rfc-editor.org/info/rfc9378>.

Appendix A. Discussion of the IOAM-Encapsulation Approach 🦋

okay so this appendix is the "why did we do it this way" section, and honestly it's my favourite bit 🤓✨ it lists a few different ways you could wrap IOAM inside NSH and explains why the team picked the one they did.

the encapsulation has to be friendly to BOTH hardware and software forwarders 💅 and it needs to work for big networks that want to use lots of IOAM-Data-Fields at the same time. no cap, that's a lot to ask 😭

  • Hardware- and software-friendly implementation 🔥

hardware forwarders are happiest with an encapsulation that doesn't make them keep looking up fields inside the packet over and over. every lookup that depends on the result of another lookup eats up gates and time, and nobody wants that, so you keep it to a minimum ✨ this is why flat TLV structures are preferred over nested ones 🙈 the IOAM-Data-Fields come in a few categories (trace, proof-of-transit, and edge-to-edge), and each of those defines its own TLV structure. a hardware-friendly approach would NOT wrap those three categories inside yet another TLV, and would just carry the options as a serial sequence instead 💜

  • Total length of the IOAM-Data-Fields 📏

the total length can get pretty chunky if you use a bunch of different IOAM-Data-Fields AND the path is long 😳 for example, say an operator wants the IOAM Trace Option-Type and captures node-id, app_data, egress and ingress interface-id, timestamp seconds, and timestamp nanoseconds at every hop. that's 20 octets added to the packet at every hop. now if that deployment has a maximum path length of 15 hops in the IOAM domain, you're looking at a max of 300 octets encapsulated in the packet. yikes, bestie 💀

okay so there are a few different ways you could encapsulate IOAM-Data-Fields in NSH:

  1. Encapsulation as "NSH MD Type 2" (see [RFC8300], Section 2.5) 🎀

each IOAM Option-Type (trace, proof-of-transit, and edge-to-edge) would get its own type, and the different IOAM-Data-Fields would be TLVs sitting inside that particular option type. NSH MD Type 2 does support variable length metadata, but its length field is 6 bits, which gives a maximum of 256 (2^6 x 4) octets. that's giving a little too cramped 🥲

  1. Encapsulation using the "Next Protocol" field. 🌸

each IOAM Option-Type (trace, proof-of-transit, and edge-to-edge) would get its own "next protocol" value.

  1. Encapsulation using the "Next Protocol" field. 🧚

a single NSH protocol type code point would be allocated for IOAM, and then a "sub-type" field would say which IOAM options type (trace, proof-of-transit, edge-to-edge) is being carried.

and the third one is the winner 👑 it avoids that extra layer of TLV-nesting you'd get from NSH MD Type 2, and it doesn't cap IOAM data at 256 octets, so very large deployments are totally supported. main character energy, honestly 🦋💖

Acknowledgments

the authors want to thank Éric Vyncke, Nalini Elkins, Srihari Raghavan, Ranganathan T S, Karthik Babu Harichandra Babu, Akshaya Nadahalli, Stefano Previdi, Hemant Singh, Erik Nordmark, LJ Wobker, Andrew Yourtchenko, Greg Mirsky, and Mohamed Boucadair for their comments and advice 🫶💜

Contributors

these people contributed significantly to the content of this document and should be considered coauthors 🌈 they're all iconic ✨

Vengada Prasad Govindan

Cisco Systems, Inc.
Email: venggovi@cisco.com

Carlos Pignataro

Cisco Systems, Inc.
7200-11 Kit Creek Road
Research Triangle Park, NC 27709
United States of America
Email: cpignata@cisco.com

Hannes Gredler

RtBrick Inc.
Email: hannes@rtbrick.com

John Leddy

Email: john@leddy.net

Stephen Youell

JP Morgan Chase
25 Bank Street
London
E14 5JP
United Kingdom
Email: stephen.youell@jpmorgan.com

Tal Mizrahi

Huawei Network.IO Innovation Lab
Israel
Email: tal.mizrahi.phd@gmail.com

David Mozes

Email: mosesster@gmail.com

Petr Lapukhov

Facebook
1 Hacker Way
Menlo Park, CA 94025
United States of America
Email: petr@fb.com

Remy Chang

Barefoot Networks
2185 Park Boulevard
Palo Alto, CA 94306
United States of America

Authors' Addresses 🎀

Frank Brockners (editor) 💅

Cisco Systems, Inc.
3rd Floor
Hansaallee 249
40549 Duesseldorf
Germany
Email: fbrockne@cisco.com

Shwetha Bhandari (editor) ✨

Thoughtspot
3rd Floor, Indiqube Orion
24th Main Rd, Garden Layout, HSR Layout
Bangalore 560 102
Karnataka
India
Email: shwetha.bhandari@thoughtspot.com
📦 where this page comes from