RFC 9772

Active Operations, Administration, and Maintenance (OAM) for Use in Generic Network Virtualization Encapsulation (Geneve)

📅 published May 2025, from ietf

Active Operations, Administration, and Maintenance (OAM) for Use in Generic Network Virtualization Encapsulation (Geneve) ✨

Okay bestie, this one is all about OAM (the Operations, Administration, and Maintenance toolkit) for Geneve 🦋, the flexible overlay protocol that wraps up packets so they can ride across the underlay network. It's giving "keeping your virtual networks healthy and happy" 💅

Title block facts 📋

  • RFC number: 9772
  • Authors: G. Mirsky (Ericsson), S. Boutros (Ciena), D. Black (Dell), S. Pallagatti (VMware)
  • Category: Standards Track, so this is an Internet Standards Track document 👑
  • ISSN: 2070-1721
  • Date: May 2025 🌸
  • Obsoletes / Updates: the original title block doesn't list any, so there's nothing to obsolete or update here 🫶

Abstract 🎀

Geneve is a super flexible and extensible network virtualization overlay protocol. Its whole job is to encapsulate packets so they can travel across the physical underlay networks 🧚. This document lays out the requirements and a framework for OAM in Geneve networks, covering the OAM functions you need to monitor, diagnose, and troubleshoot overlays so they keep running properly and performing well. It's basically a guide for implementing OAM mechanisms inside Geneve, so network operators can keep their virtualized networks reliable and efficient 💖

Status of This Memo 📜

This is an Internet Standards Track document ✨

This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community, and it went through public review and got approved for publication by the Internet Engineering Steering Group (IESG) 🔥. For more on Internet Standards, check out Section 2 of RFC 7841 🤓.

Want the current status, any errata, or a way to send feedback? Head to https://www.rfc-editor.org/info/rfc9772 🦋

Copyright (c) 2025 IETF Trust and the persons identified as the document authors. All rights reserved 🌈

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they stand on the date of publication. Please read these carefully, because they spell out your rights and restrictions for this document 🫶. Code Components pulled out of this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions, and they come without warranty, as described in the Revised BSD License ⚠️.

Table of Contents 🗺️

  • 1. Introduction
    • 1.1. Conventions Used in This Document
      • 1.1.1. Requirements Language
      • 1.1.2. Acronyms
  • 2. The Applicability of Active OAM Protocols in Geneve Networks
    • 2.1. Requirements for Active OAM Protocols in Geneve Networks
    • 2.2. Defect Detection and Troubleshooting in Geneve Network with Active OAM
      • 2.2.1. Echo Request and Echo Reply in Geneve Tunnel
    • 2.3. Active OAM Encapsulation in Geneve
  • 3. IANA Considerations
  • 4. Security Considerations
  • 5. References
    • 5.1. Normative References
    • 5.2. Informative References
  • Acknowledgments
  • Authors' Addresses

1. Introduction 🌸

Geneve [RFC8926] is built for lots of network virtualization scenarios. It can encapsulate multiple protocols, like Ethernet and IPv4/IPv6, and it carries metadata inside the Geneve message itself ✨

OAM protocols give you the fault management and performance monitoring you need for healthy network operation 💅. Active OAM protocols, as defined in [RFC7799], work by injecting specially built packets into the network 🚀. Examples include ICMP and ICMPv6 ([RFC0792] and [RFC4443] respectively), Bidirectional Forwarding Detection (BFD) [RFC5880], and the Simple Two-way Active Measurement Protocol (STAMP) [RFC8762] 🧚.

Here's the important bit 🔑: if you want your performance metrics or detected failures to be accurately tied to a particular Geneve flow, those OAM test packets need to share fate. In other words, they have to be in-band with the overlay data packets of that monitored flow as they cross the underlay network 🫶. In this document, "in-band OAM" means the following:

  • In-band OAM is an active or hybrid OAM method, as defined in [RFC7799], that traverses the same set of links and interfaces and gets the same Quality of Service treatment as the monitored object 🎀. Here, the monitored object is either the entire Geneve tunnel or one specific tenant flow inside that tunnel 💜.

Section 2.1 lists the general requirements for active OAM protocols in a Geneve overlay network 📋. IP encapsulation meets these requirements and works well for wrapping up active OAM protocols inside a Geneve overlay 🦋. Active OAM messages get swapped between two Geneve tunnel endpoints. Each endpoint can be the tenant-facing interface of the Network Virtualization Edge (NVE), or some other device acting as a Geneve tunnel endpoint 🌈. Testing end-to-end between tenants is out of scope 🙈. To keep things simple, this document uses an NVE to represent the Geneve tunnel endpoint. For detailed definitions and descriptions of an NVE, see [RFC7365] and [RFC8014] 🧚

The IP encapsulation of Geneve OAM defined here applies to an overlay service by introducing a Management Virtual Network Identifier (VNI). It can be combined with different values of the Protocol Type field in the Geneve header, like Ethertypes for IPv4 or IPv6 💖. Other types of OAM encapsulation in Geneve are outside the scope of this document 🫶.

1.1. Conventions Used in This Document 📚

1.1.1. Requirements Language 🔥

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here 💅.

1.1.2. Acronyms 🍓

  • Geneve: Generic Network Virtualization Encapsulation
  • NVO3: Network Virtualization over Layer 3
  • OAM: Operations, Administration, and Maintenance
  • VNI: Virtual Network Identifier
  • BFD: Bidirectional Forwarding Detection
  • STAMP: Simple Two-way Active Measurement Protocol
  • NVE: Network Virtualization Edge

2. The Applicability of Active OAM Protocols in Geneve Networks 🦋

2.1. Requirements for Active OAM Protocols in Geneve Networks 💜

OAM protocols, whether they're doing fault management or performance monitoring, exist to give you reliable info. That info lets you detect a failure, identify the defect, and localize it, so you can apply corrective actions and keep the impact on service small 🛠️. Several OAM protocols do these jobs, and they all need demultiplexing at the receiving Geneve instance 🌸. To make the correlation between what the monitored Geneve tunnel is experiencing and the state of the OAM protocol more accurate, the OAM encapsulation has to meet these requirements:

  • Requirement 1: Geneve OAM test packets MUST share the same fate as the data traffic of the monitored Geneve tunnel. Specifically, the OAM test packets MUST be in-band with the monitored traffic and follow the same overlay and transport paths as packets carrying data payloads in the forward direction, meaning from the ingress toward the egress endpoint(s) of the OAM test 🚀.

An OAM protocol MAY be used to monitor an entire Geneve tunnel. In that case, test packets could be in-band relative to a subset of tenant flows carried over the Geneve tunnel 🧚. If the goal is to monitor the conditions experienced by a particular tenant's flow, the test packets MUST be in-band with that specific flow inside the Geneve tunnel 💖. Both scenarios get the full treatment in Section 2.2 🎀.

  • Requirement 2: The encapsulation of OAM control messages and data packets in the underlay network MUST be indistinguishable 👀.
  • Requirement 3: The presence of an OAM control message in a Geneve packet MUST be unambiguously identifiable to Geneve functionality, such as at endpoints of Geneve tunnels 🔑.
  • Requirement 4: OAM test packets MUST NOT be forwarded to a tenant system 🙅.

A test packet generated by an active OAM protocol, whether it's for defect detection or performance measurement, MUST be in-band with the tunnel or data flow being monitored, as Requirement 1 says 🫶. Where multiple paths through the domain are available, underlay transport nodes can be programmed to use characteristic information to balance load across the known paths 🌈. It's essential that test packets follow the same route, meaning they traverse the same set of nodes and links as a data packet of the monitored flow 🧐. That's why the following requirement supports OAM packet fate-sharing with the data flow:

  • Requirement 5: There MUST be a way to encode entropy information into the underlay forwarding scheme so that OAM packets take the same data-flow paths as the transit traffic flows 👑.

Part 1 of 3 complete! 🌈✨ More coming, so stay tuned, bestie 🫶

2.2. Defect Detection and Troubleshooting in Geneve Network with Active OAM ✨

Okay bestie, this section is all about catching and chasing down defects in a Geneve network using active OAM 🔍 It covers two scenarios, so let's look at the example setup first 🦋

   +--------+                                             +--------+
   | Tenant +--+                                     +----| Tenant |
   | VNI 28 |  |                                     |    | VNI 35 |
   +--------+  |          ................           |    +--------+
               |  +----+  .              .  +----+   |
               |  | NVE|--.              .--| NVE|   |
               +--| A  |  .              .  | B  |---+
                  +----+  .              .  +----+
                  /       .              .
                 /        .     Geneve   .
   +--------+   /         .    Network   .
   | Tenant +--+          .              .
   | VNI 35 |             .              .
   +--------+             ................
                                 |
                               +----+
                               | NVE|
                               | C  |
                               +----+
                                 |
                                 |
                       =====================
                         |               |
                     +--------+      +--------+
                     | Tenant |      | Tenant |
                     | VNI 28 |      | VNI 35 |
                     +--------+      +--------+

                  Figure 1: An Example of a Geneve Domain

Figure 1 is our example Geneve domain, and it's giving full network drama 💀

Scenario one: there's a communication problem between Network Virtualization Edge (NVE) device A and NVE C. The operator checks things out and finds that the forwarding in the IP underlay network is working just fine 🙈 But the Geneve connection is still unstable for every NVE A and NVE C tenant. The good news is that detecting, troubleshooting, and localizing the problem can be done regardless of the VNI value 💜

Scenario two: traffic on VNI 35 between NVE A and NVE B is totally fine, and so is VNI 28 between NVE A and NVE C. However, traffic on VNI 35 between NVE A and NVE C is having issues, like excessive packet loss 😭

For scenario one, you can detect and investigate with any VNI value, whether or not it connects tenant systems. To conform to Requirement 4, OAM test packets SHOULD be transmitted on a VNI that doesn't have any tenants 🫶 That kind of Geneve tunnel is dedicated to carrying only control and management data between the tunnel endpoints, so it's called a "Geneve control channel", and that VNI is called the "Management VNI" 🌸 A configured VNI MAY be used to identify the control channel, but it is RECOMMENDED that the default value 1 be used as the Management VNI. We'll get into how test packets are encapsulated using the Management VNI in Section 2.3 ✨

The control channel of a Geneve tunnel MUST NOT carry tenant data. Since no tenants are connected to the control channel, a system that supports this specification MUST NOT forward a packet received over the control channel to any tenant 🚫 A packet received over the control channel by a system that supports this specification MUST be forwarded if and only if it is sent onto the control channel of the concatenated Geneve tunnel. Otherwise, it MUST be terminated locally 🎀

Here's the spicy part: the Management VNI SHOULD be terminated on the tenant-facing side of the Geneve encapsulation/decapsulation functionality, not the DC-network-facing side (per definitions in Section 4 of [RFC8014]). That way, Geneve encap/decap functionality is included in its scope 🧚 This means an active OAM packet, like an ICMP echo request, gets decapsulated the same way as any other received Geneve packet. In this example, the resulting ICMP packet is handed to the NVE's local management functionality, which processes it and generates an ICMP echo reply. That echo reply is encapsulated in Geneve (as specified in Section 2.3) and sent back to the NVE that sent the echo request 🔁 One advantage is that a repeated ICMP echo request/reply test could catch an intermittent problem in Geneve encap/decap hardware. That problem would go untested if the Management VNI were handled as a "special case" at the DC-network-facing interface 👑

Scenario two happens when a test packet is sent using the VNI value tied to the monitored service flow. Doing that means the test packet gets the same network treatment as the tenant's packets 💅 An example of how that works is discussed in [RFC9521] 🌈

2.2.1. Echo Request and Echo Reply in Geneve Tunnel

ICMP and ICMPv6 ([RFC0792] and [RFC4443] respectively) were mentioned above, and they're examples of active OAM protocols 🍓 They provide the required on-demand defect detection and failure localization. ICMP control messages sit right after the inner IP header that's encapsulated in Geneve. ICMP extensions for Geneve networks use the mechanisms defined in [RFC4884] ✨

2.3. Active OAM Encapsulation in Geneve

Active OAM over a Management VNI in the Geneve network uses an IP encapsulation 🦋 Protocols like BFD [RFC5880] and STAMP [RFC8762] use UDP transport. The destination UDP port number in the inner UDP header (Figure 2) identifies the OAM protocol. This approach is well-known and has been used in MPLS networks, for example [RFC8029] 💖 To use IP encapsulation for an active OAM protocol, the Protocol Type field of the Geneve header MUST be set to 0x0800 (IPv4) or 0x86DD (IPv6). [RFC9521] describes how IP encapsulation is used for BFD 🎀

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   ~                        Outer IPvX Header                      ~
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   ~                        Outer UDP Header                       ~
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   ~                          Geneve Header                        ~
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   ~                        Inner IPvX Header                      ~
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   ~                        Inner UDP Header                       ~
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   ~                        Active OAM Packet                      ~
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

      Figure 2: An Example of Geneve IP/UDP Encapsulation of an Active
                                 OAM Packet

Figure 2 is the layer-by-layer picture of how the packet gets wrapped 🧁

Inner IP header:

  • Destination IP: The IP address MUST be set to the loopback address 127.0.0.1/32 for the IPv4 version. For IPv6, the address MUST be selected from the Dummy IPv6 Prefix 100:0:0:1::/64 [RFC9780]. A source-only IPv6 address is used as the destination to generate an exception and a reply message to the request message received 🌟
  • Source IP: IP address of the NVE. 💜
  • TTL or Hop Limit: MUST be set to 255 per [RFC5082]. The receiver of an active OAM Geneve packet with IP/UDP encapsulation MUST drop packets whose TTL/Hop Limit is not 255. 🔥

3. IANA Considerations 🎀

This document has no IANA actions 🌸

4. Security Considerations 🔐

Okay, this part is serious, so listen up 👀 As part of a Geneve network, active OAM inherits the security considerations discussed in [RFC8926]. On top of that, a system MUST provide control to limit the rate of Geneve OAM packets punted to the Geneve control plane for processing, so that the control plane doesn't get overloaded. That's a hard requirement, not a suggestion 😭 OAM in Geneve packets uses the General TTL Security Mechanism [RFC5082], and any packet received with an inner TTL / Hop Count other than 255 MUST be discarded 🚨

5. References 📚

5.1. Normative References

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, <https://www.rfc-editor.org/info/rfc2119>. 🦋

[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, <https://www.rfc-editor.org/info/rfc8174>. 💖

[RFC8926] Gross, J., Ed., Ganga, I., Ed., and T. Sridhar, Ed., "Geneve: Generic Network Virtualization Encapsulation", RFC 8926, DOI 10.17487/RFC8926, November 2020, <https://www.rfc-editor.org/info/rfc8926>. 🌈

[RFC9780] Mirsky, G., Mishra, G., and D. Eastlake 3rd, "Bidirectional Forwarding Detection (BFD) for Multipoint Networks over Point-to-Multipoint MPLS Label Switched Paths (LSPs)", RFC 9780, DOI 10.17487/RFC9780, May 2025, <https://www.rfc-editor.org/info/rfc9780>. ⭐

5.2. Informative References ✨

okay bestie, this is the bonus reading list 📚💖 none of these are needed to get the core spec, but they're the context behind the whole vibe, so keep them close 🫶

Acknowledgments 🙏💜

The authors want to give a huge shoutout to Donald E. Eastlake 3rd 🌸 his suggestions made the document way more readable, and that's iconic behaviour from a legend 👑

Authors' Addresses 📮✨

  • Greg Mirsky 🦋
    • Ericsson
    • Email: gregimirsky@gmail.com
  • Sami Boutros 🍓
    • Ciena
    • Email: sboutros@ciena.com
  • David Black 🌈
    • Dell
    • Email: david.black@dell.com
  • Santosh Pallagatti 🧚
    • VMware
    • Email: santosh.pallagatti@gmail.com
📦 where this page comes from