RFC 9259

Operations, Administration, and Maintenance (OAM) in Segment Routing over IPv6 (SRv6)

📅 published June 2022, from ietf

RFC 9259 🎀 Operations, Administration, and Maintenance (OAM) in Segment Routing over IPv6 (SRv6)

okay bestie, the title block is the tea ☕ and it's giving official IETF Standards Track 💅 Here's the lineup:

  • Stream: Internet Engineering Task Force (IETF)
  • Category: Standards Track
  • ISSN: 2070-1721
  • Date: June 2022
  • Authors: Z. Ali (Cisco Systems), C. Filsfils (Cisco Systems), S. Matsushima (Softbank), D. Voyer (Bell Canada), M. Chen (Huawei)
  • Obsoletes / Updates: this header doesn't list any Obsoletes or Updates entries 🫶

Abstract 🦋

okay so this doc is about how the IPv6 ping and traceroute tools you already know can work in a Segment Routing over IPv6 (SRv6) network 🌸 It also defines the OAM flag (O-flag) in the Segment Routing Header (SRH), which lets you do controllable and predictable flow sampling from segment endpoints ✨ On top of that, it covers how a centralized monitoring system can run a path continuity check between any nodes inside an SRv6 domain. main character energy, fr 💜

Status of This Memo 🌈

this one is an Internet Standards Track document 👑

it's a product of the Internet Engineering Task Force (IETF), so it reflects what the IETF community agreed on. it got public review and the Internet Engineering Steering Group (IESG) signed off on it for publication 🔥 if you want more on Internet Standards, check out Section 2 of RFC 7841.

for the latest on status, errata, and how to send feedback, go to https://www.rfc-editor.org/info/rfc9259. 🍓

Copyright (c) 2022 IETF Trust and the persons identified as the document authors. All rights reserved. 🎀

this document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they stand on the publication date. please actually read these, bestie, because they spell out your rights and restrictions for this doc 🙈 and if you pull Code Components out of this document, they must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions, and they come with no warranty, per the Revised BSD License.

Table of Contents 🧚

1.  Introduction
  1.1.  Requirements Language
  1.2.  Abbreviations
  1.3.  Terminology and Reference Topology
2.  OAM Mechanisms
  2.1.  OAM Flag in the Segment Routing Header
    2.1.1.  OAM Flag Processing
  2.2.  OAM Operations
3.  Security Considerations
4.  Privacy Considerations
5.  IANA Considerations
6.  References
  6.1.  Normative References
  6.2.  Informative References
Appendix A.  Illustrations
  A.1.  Ping in SRv6 Networks
    A.1.1.  Pinging an IPv6 Address via a Segment List
    A.1.2.  Pinging a SID
  A.2.  Traceroute in SRv6 Networks
    A.2.1.  Traceroute to an IPv6 Address via a Segment List
    A.2.2.  Traceroute to a SID
  A.3.  Hybrid OAM Using the OAM Flag
  A.4.  Monitoring of SRv6 Paths
Acknowledgements
Contributors
Authors' Addresses

1. Introduction ✨

okay so SRv6 [RFC8402] basically just adds a new kind of Routing Extension Header 💅 which means the IPv6 OAM tools you already have can be reused in an SRv6 network. this doc shows how the existing IPv6 ping and traceroute mechanisms work there. it covers pinging an SRv6 Segment Identifier (SID) to check that the SID is reachable and locally programmed on the target node 🦋 it also covers tracerouting to an SRv6 SID for hop-by-hop fault localization, plus path tracing to a SID 🔥

this doc also brings in upgrades to OAM for SRv6 that give you controllable and predictable flow sampling from segment endpoints. it uses things like the IP Flow Information Export (IPFIX) protocol [RFC7011] to make that happen 🌸 the big one is the O-flag in the SRH, which is a marking bit in user packets that triggers telemetry data collection and export at the segment endpoints 💜

last thing: it explains how the centralized OAM technique in [RFC8403] can stretch to SRv6 so you can run a path continuity check between any nodes in an SRv6 domain ✨ specifically, a centralized monitoring system can keep an eye on arbitrary SRv6 paths by sending loopback probes that start and end at that same monitoring system 👑

1.1. Requirements Language

the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here. 🎀

1.2. Abbreviations

these are the abbreviations used in this doc, babe 🧚

  • SID: Segment Identifier
  • SL: Segments Left
  • SR: Segment Routing
  • SRH: Segment Routing Header [RFC8754]
  • SRv6: Segment Routing over IPv6 [RFC8402]
  • PSP: Penultimate Segment Pop [RFC8986]
  • USP: Ultimate Segment Pop [RFC8986]
  • ICMPv6: Internet Control Message Protocol for the Internet Protocol version 6 [RFC4443]
  • IS-IS: Intermediate System to Intermediate System
  • OSPF: Open Shortest Path First [RFC2328]
  • IGP: Interior Gateway Protocol (e.g., OSPF and IS-IS)
  • BGP-LS: Border Gateway Protocol - Link State [RFC8571]

1.3. Terminology and Reference Topology

this section has the terminology and a simple topology that the whole doc uses for illustrations 🌈 the picture is below, so let's get into it 🫶

+--------------------------| N100 |---------------------------------+
|                                                                   |
|  ====== link1====== link3------ link5====== link9------   ======  |
   ||N1||------||N2||------| N3 |------||N4||------| N5 |---||N7||
   ||  ||------||  ||------|    |------||  ||------|    |---||  ||
   ====== link2====== link4------ link6======link10------   ======
      |            |                      |                   |
   ---+--          |       ------         |                 --+---
   |CE1 |          +-------| N6 |---------+                 |CE2 |
   ------            link7 |    | link8                     ------
                           ------

                     Figure 1: Reference Topology

in the reference topology, here's the tea 🍓

  • Node j has an IPv6 loopback address 2001:db8:L:j::/128. 💖
  • Nodes N1, N2, N4, and N7 are SRv6-capable nodes. 💅
  • Nodes N3, N5, and N6 are IPv6 nodes that are not SRv6-capable nodes. They're called "non-SRv6-capable nodes" 🙈
  • CE1 and CE2 are Customer Edge devices with any data plane capability (e.g., IPv4, IPv6, and L2). 🌸
  • A SID at node j with locator block 2001:db8:K::/48 and function U is written as 2001:db8:K:j:U::. 🔥
  • Node N100 is a controller. 👑
  • The IPv6 address of the nth link between nodes i and j, on the i side, is written as 2001:db8:i:j:in::. For example, in Figure 1, the IPv6 address of link6 (the second link between nodes N3 and N4) at node N3 is 2001:db8:3:4:32::. And the IPv6 address of link5 (the first link between nodes N3 and N4) at node N3 is 2001:db8:3:4:31:: ✨
  • 2001:db8:K:j:Xin:: is explicitly allocated as the End.X SID at node j toward neighbor node i via the nth link between nodes i and j. For example, 2001:db8:K:2:X31:: is End.X at node N2 toward node N3 via link3 (the first link between N2 and N3). And 2001:db8:K:4:X52:: is End.X at node N4 toward node N5 via link10 (the second link between N4 and N5). Check out [RFC8986] if you want the full End.X SID scoop 🦋
  • A SID list is written as <S1, S2, S3>, where S1 is the first SID to visit, S2 is the second, and S3 is the last SID to visit along the SR path 💜
  • (SA,DA) (S3, S2, S1; SL)(payload) represents an IPv6 packet with:
    • an IPv6 header with source address SA, destination address DA, and SRH as the next header 🎀
    • an SRH with SID list <S1, S2, S3> and SegmentsLeft = SL 🌈
    • (payload), which is the payload of the packet 🫶

okay but the < > and () symbols are NOT the same thing, so pay attention 🙈 <S1, S2, S3> is a SID list where S1 is first and S3 is last to traverse. (S3, S2, S1; SL) is that same SID list, but encoded in SRH format, where the rightmost SID in the SRH is the first one and the leftmost SID is the last one 🧚 when you're talking about an SR Policy at a high level, the <S1, S2, S3> notation is simpler. when you're looking at detailed packet behavior, the (S3, S2, S1; SL) notation is the better pick ✨

2. OAM Mechanisms 💅

this section defines the OAM enhancements for SRv6 networks 🦋

2.1. OAM Flag in the Segment Routing Header

[RFC8754] describes the Segment Routing Header (SRH) and how SR-capable nodes use it 🌸 the SRH has an 8-bit Flags field 💜

this doc defines the following bit in the SRH Flags field to carry the O-flag 👑

              0 1 2 3 4 5 6 7
             +-+-+-+-+-+-+-+-+
             |   |O|         |
             +-+-+-+-+-+-+-+-+

where:

  • O-flag: OAM flag in the SRH Flags field defined in [RFC8754] 🔥

2.1.1. OAM Flag Processing

Okay so the O-flag in the SRH is basically a tiny marking bit 💅 that gets set on user packets to trigger telemetry data collection and export at the segment endpoints ✨

An SR domain ingress edge node encapsulates packets going through the SR domain, as defined in [RFC8754] 🧚. The SR domain ingress edge node MAY use the O-flag in the SRH to mark the packet, which triggers telemetry collection and export at the segment endpoints. Depending on local configuration, the ingress edge node may also run a classification and sampling mechanism to decide which packets get marked with the O-flag in the SRH. How that classification and sampling works is outside the scope of this document 🙈

This document doesn't specify which data elements need to be exported, or the configurations for them. It also doesn't define any formats for exporting those data elements. That said, without loss of generality, this document assumes that the IP Flow Information Export (IPFIX) protocol [RFC7011] is used to export traffic flow information from network devices to a controller for monitoring and analytics 📊. It also assumes, again without loss of generality, that the requested information elements are configured by the management plane through data set templates (e.g., as in IPFIX [RFC7012]) 🎀.

Implementing the O-flag is OPTIONAL 🌸. If a node doesn't support the O-flag, it just ignores it when it receives the packet. If a node does support the O-flag, it can optionally advertise that via control plane protocol(s).

The following is appended to line S01 of the pseudocode associated with the SID S (as defined in Section 4.3.1.1 of [RFC8754]) when N receives a packet destined to S, S is a local SID, and the O-flag is processed:

      S01.1. IF the O-flag is set and local configuration permits
             O-flag processing {
                a. Make a copy of the packet.
                b. Send the copied packet, along with a timestamp,
                to the OAM process for telemetry data collection
                and export.      ;; Ref1
                }
      Ref1: To provide an accurate timestamp, an implementation should
      copy and record the timestamp as soon as possible during packet
      processing. Timestamp and any other metadata are not carried in
      the packet forwarded to the next hop.

Please note that O-flag processing happens before the regular processing of the local SID S 👀. Specifically, line S01.1 of the pseudocode in this document gets inserted between lines S01 and S02 of the pseudocode in Section 4.3.1.1 of [RFC8754].

Based on the requested information elements configured by the management plane through data set templates [RFC7012], the OAM process exports those elements. The information elements can include parts of the packet header and/or parts of the packet payload for flow identification. The OAM process uses information elements defined in IPFIX [RFC7011] and Packet Sampling (PSAMP) [RFC5476] to export the requested sections of the mirrored packets 🦋.

If the penultimate segment of a segment list is a PSP SID, telemetry data from the ultimate segment cannot be requested. That's because, when the penultimate segment is a PSP SID, the SRH is removed at the penultimate segment, and the O-flag is not processed at the ultimate segment 😭.

The processing node MUST rate-limit the number of packets punted to the OAM process to a configurable rate 🚦. This keeps the OAM and telemetry collection processes from hurting performance. Failing to implement the rate limit can lead to a denial-of-service attack, as detailed in Section 3 💀.

The OAM process MUST NOT process the copy of the packet or respond to any Upper-Layer header (like ICMP, UDP, etc.) payload, so the datagram doesn't get evaluated multiple times 🙅‍♀️.

The OAM process is expected to be located on the routing node processing the packet. The specification of the OAM process or the external controller operations is beyond the scope of this document, but the OAM process SHOULD NOT be topologically distant from the routing node, because that is likely to create significant security and congestion issues 🚨. How to correlate the data collected from different nodes at an external controller is also outside the scope of this document. Appendix A illustrates use of the O-flag for implementing a hybrid OAM mechanism, where the "hybrid" classification is based on [RFC7799] 🌈.

2.2. OAM Operations

IPv6 OAM operations can be performed for any SRv6 SID whose behavior allows Upper-Layer header processing for an applicable OAM payload (e.g., ICMP, UDP) 💖.

Pinging an SRv6 SID is used to verify that the SID is reachable and locally programmed at the target node 🏓. Traceroute to a SID is used for hop-by-hop fault localization, as well as path tracing to a SID 🗺️. Appendix A illustrates the ICMPv6-based ping and UDP-based traceroute mechanisms for pinging and tracerouting an SRv6 SID. This document only illustrates ICMPv6-based ping and UDP-based traceroute to an SRv6 SID, but the procedures are equally applicable to other OAM mechanisms that probe an SRv6 SID, such as Bidirectional Forwarding Detection (BFD) [RFC5880], Seamless BFD (S-BFD) [RFC7880], and Simple Two-way Active Measurement Protocol (STAMP) probe message processing [STAMP-SR] ✨. Specifically, as long as local configuration allows Upper-Layer header processing of the applicable OAM payload for SRv6 SIDs, the existing IPv6 OAM techniques can be used to target a probe to a (remote) SID 🎯.

IPv6 OAM operations can be performed with the target SID in the IPv6 destination address, either without an SRH or with an SRH where the target SID is the last segment. In general, OAM operations to a target SID may not exercise all of its processing, depending on its behavior definition 🤔. For example, pinging an End.X SID [RFC8986] only validates that the SID is locally programmed at the target node. It does not validate switching to the correct outgoing interface 🙃. To exercise the behavior of a target SID, the OAM operation should construct the probe in a manner similar to a data packet that exercises the SID behavior, i.e., include that SID as a transit SID either in an SRH or in the IPv6 DA of an outer IPv6 header, or as appropriate based on the definition of the SID behavior 💅.

3. Security Considerations

[RFC8754] defines the notion of an SR domain and the use of the SRH within the SR domain 🛡️. The use of OAM procedures described in this document is restricted to an SR domain. For example, similar to SID manipulation, O-flag manipulation is not considered a threat within the SR domain. Procedures for securing an SR domain are defined in Sections 5.1 and 7 of [RFC8754].

As noted in Section 7.1 of [RFC8754], compromised nodes within the SR domain may mount attacks 😱. The O-flag may be set by an attacking node attempting a denial-of-service attack on the OAM process at the segment endpoint node. An implementation that correctly implements the rate limiting described in Section 2.1.1 is not susceptible to that denial-of-service attack 🙌. Additionally, SRH flags are protected by the Hashed Message Authentication Code (HMAC) TLV, as described in Section 2.1.2.1 of [RFC8754]. Once an HMAC is generated for a segment list with the O-flag set, it can be used for an arbitrary amount of traffic using that segment list with the O-flag set 🔐.

The security properties of the channel used to send exported packets marked by the O-flag depend on the specific OAM processes used. An on-path attacker able to observe this OAM channel could conduct traffic analysis or, potentially, eavesdropping (depending on the OAM configuration) on the telemetry for the entire SR domain from that vantage point 👀💀.

This document doesn't impose any additional security challenges beyond the security threats described in [RFC4884], [RFC4443], [RFC0792], [RFC8754], and [RFC8986] 🫶.

4. Privacy Considerations

The per-packet marking capabilities of the O-flag give a granular mechanism for collecting telemetry 🔍. When an operator deploys this collection with the knowledge and consent of users, it enables a variety of diagnostics and monitoring that support the OAM and security operations needed for resilient network operations 🌸. However, this collection mechanism also gives operators an explicit protocol mechanism for surveillance and pervasive monitoring done contrary to users' consent 😤. That's a lot of responsibility, bestie.

5. IANA Considerations

IANA has registered the following in the "Segment Routing Header Flags" subregistry in the "Internet Protocol Version 6 (IPv6) Parameters" registry 📝:

                     +=====+=============+===========+
                     | Bit | Description | Reference |
                     +=====+=============+===========+
                     | 2   | O-flag      | RFC 9259  |
                     +-----+-------------+-----------+

                                  Table 1

6. References

6.1. Normative References

6.2. Informative References 📚

okay so these are the bonus reads, bestie 🎀 they aren't required to implement the spec, but they're the receipts behind the ideas. ICMP, OSPF, BFD, IPFIX, segment routing, all the chaotic-good stuff 💅

Appendix A. Illustrations 🎨

this appendix is the fun part, where we see how the existing IPv6 OAM tools can be used in an SRv6 network 🦋 it also shows an OAM mechanism for doing controllable and predictable flow sampling from segment endpoints, and it explains how the centralized OAM technique in [RFC8403] can be extended for SRv6 ✨

A.1. Ping in SRv6 Networks 🏓

the existing reachability checks along the shortest path keep working with zero changes 💖 any IPv6 node (SRv6-capable or non-SRv6-capable) can initiate, transit, and egress a ping packet. iconic behaviour, honestly 😭

here are some extra ICMPv6 ping use cases in SRv6 networks 👀

A.1.1. Pinging an IPv6 Address via a Segment List

okay so if an SRv6-capable ingress node wants to ping an IPv6 address via an arbitrary segment list <S1, S2, S3>, it needs to start an ICMPv6 ping with an SR header containing the SID list <S1, S2, S3> 🧚 the topology is in Figure 1. the user pings from node N1 to a loopback of node N5 via segment list <2001:db8:K:2:X31::, 2001:db8:K:4:X52::>. the SID behavior in this example is End.X, as described in [RFC8986], but the procedure is equally applicable to any other (transit) SID type 💅

here's the sample output for that ping from N1 to the loopback address of N5 via the same segment list 👇

    > ping 2001:db8:L:5:: via segment list 2001:db8:K:2:X31::,
           2001:db8:K:4:X52::

    Sending 5, 100-byte ICMPv6 Echos to B5::, timeout is 2 seconds:
    !!!!!
    Success rate is 100 percent (5/5), round-trip min/avg/max = 0.625
    /0.749/0.931 ms

Figure 2: Sample Ping Output at an SRv6-Capable Node

all transit nodes handle the echo request like any other data packet carrying an SR header, so they don't need any change 🫶 the egress node doesn't need a change either to process the ICMPv6 echo request. for example, in the Figure 2 scenario:

  • Node N1 starts an ICMPv6 ping packet with the SRH like this: (2001:db8:L:1::, 2001:db8:K:2:X31::) (2001:db8:L:5::, 2001:db8:K:4:X52::, 2001:db8:K:2:X31::, SL=2, NH = ICMPv6)(ICMPv6 Echo Request).
  • Node N2, which is an SRv6-capable node, performs the standard SRH processing. specifically, it executes the End.X behavior indicated by the 2001:db8:K:2:X31:: SID and forwards the packet on link3 to node N3 🔥
  • Node N3, which is a non-SRv6-capable node, performs the standard IPv6 processing. specifically, it forwards the echo request based on DA 2001:db8:K:4:X52:: in the IPv6 header 🌸
  • Node N4, which is an SRv6-capable node, performs the standard SRH processing. specifically, it observes the End.X behavior (2001:db8:K:4:X52::) and forwards the packet on link10 towards node N5. if 2001:db8:K:4:X52:: is a PSP SID, the penultimate node (node N4) does not, should not, and cannot differentiate between the data packets and OAM probes. specifically, if 2001:db8:K:4:X52:: is a PSP SID, node N4 executes the SID like any other data packet with DA = 2001:db8:K:4:X52:: and removes the SRH 🍓
  • the echo request packet at node N5 arrives as an IPv6 packet with or without an SRH. if node N5 receives the packet with an SRH, it skips SRH processing (SL=0). either way, node N5 performs the standard ICMPv6 processing on the echo request and replies with the echo reply message to node N1. the echo reply message is IP routed 🦋

A.1.2. Pinging a SID

the ping mechanism above can also be used to do SID reachability checks and to validate that the SID is locally programmed at the target node 🔍 here's how that works. this example uses ping to an End SID, as described in [RFC8986], but the procedure is equally applicable to pinging any other SID behaviors 💜

say the user wants to ping a remote SID 2001:db8:K:4::, via 2001:db8:K:2:X31::, from node N1. the ICMPv6 echo request gets processed at the individual nodes along the path like this 👑

  • Node N1 starts an ICMPv6 ping packet with the SRH like this: (2001:db8:L:1::, 2001:db8:K:2:X31::) (2001:db8:K:4::, 2001:db8:K:2:X31::; SL=1; NH=ICMPv6)(ICMPv6 Echo Request).
  • Node N2, which is an SRv6-capable node, performs the standard SRH processing. specifically, it executes the End.X behavior indicated by the 2001:db8:K:2:X31:: SID on the echo request packet. if 2001:db8:K:2:X31:: is a PSP SID, node N4 executes the SID like any other data packet with DA = 2001:db8:K:2:X31:: and removes the SRH ✨
  • Node N3, which is a non-SRv6-capable node, performs the standard IPv6 processing. specifically, it forwards the echo request based on DA = 2001:db8:K:4:: in the IPv6 header 🧚
  • When node N4 receives the packet, it processes the target SID (2001:db8:K:4::) 🙈
  • If the target SID (2001:db8:K:4::) is not locally instantiated and does not represent a local interface, the packet is discarded 💀
  • If the target SID (2001:db8:K:4::) is locally instantiated or represents a local interface, the node processes the Upper-Layer header. as part of that Upper-Layer header processing, node N4 responds to the ICMPv6 echo request message with an echo reply message. the echo reply message is IP routed 🌈

A.2. Traceroute in SRv6 Networks 🗺️

the existing traceroute mechanisms, along the shortest path, keep working without any modification 🎀 any IPv6 node (SRv6-capable or a non-SRv6-capable) can initiate, transit, and egress a traceroute probe. the vibes are immaculate 🌸

here are some extra traceroute use cases in SRv6 networks 👀

A.2.1. Traceroute to an IPv6 Address via a Segment List ✨

Okay so, if an SRv6-capable ingress node wants to traceroute to an IPv6 address via an arbitrary segment list <S1, S2, S3>, it needs to launch a traceroute probe with an SR header carrying the SID list <S1, S2, S3> 🦋. It's basically a breadcrumb trail 🍞. In this example, the user runs a traceroute from node N1 to a loopback of node N5 via the segment list <2001:db8:K:2:X31::, 2001:db8:K:4:X52::>. The SID behavior here is End.X, as described in [RFC8986], but the procedure works just as well for any other (transit) SID type 💅. Figure 3 has the sample output for the traceroute request 💖.

   > traceroute 2001:db8:L:5:: via segment list 2001:db8:K:2:X31::,
                2001:db8:K:4:X52::

   Tracing the route to 2001:db8:L:5::
   1  2001:db8:2:1:21:: 0.512 msec 0.425 msec 0.374 msec
      DA: 2001:db8:K:2:X31::,
      SRH:(2001:db8:L:5::, 2001:db8:K:4:X52::, 2001:db8:K:2:X31::, SL=2)
   2  2001:db8:3:2:31:: 0.721 msec 0.810 msec 0.795 msec
      DA: 2001:db8:K:4:X52::,
      SRH:(2001:db8:L:5::, 2001:db8:K:4:X52::, 2001:db8:K:2:X31::, SL=1)
   3  2001:db8:4:3::41:: 0.921 msec 0.816 msec 0.759 msec
      DA: 2001:db8:K:4:X52::,
      SRH:(2001:db8:L:5::, 2001:db8:K:4:X52::, 2001:db8:K:2:X31::, SL=1)
   4  2001:db8:5:4::52:: 0.879 msec 0.916 msec 1.024 msec
      DA: 2001:db8:L:5::

         Figure 3: Sample Traceroute Output at an SRv6-Capable Node

In this output, every hop's info comes from the ICMPv6 "Time Exceeded" or "Destination Unreachable" responses 📦. And those responses are IP routed 🚗.

The info for link3 is returned by node N3, which is a non-SRv6-capable node. Even so, the ingress node can still display SR header contents as the packet travels through that non-SRv6-capable node 🤯. Why? The "Time Exceeded" ICMPv6 message can contain as much of the invoking packet as possible, as long as the ICMPv6 packet doesn't go over the minimum IPv6 MTU [RFC4443]. The SR header gets included in these ICMPv6 messages even when they come from non-SRv6-capable transit nodes that aren't running SRv6 software. Specifically, a node generating an ICMPv6 message that contains a copy of the invoking packet doesn't need to understand the extension header(s) in that invoking packet. Main character energy for the routers, honestly 👑.

The segment list info for the first hop is returned by node N2, which is an SRv6-capable node. Just like with the second hop, the ingress node can display SR header contents for the first hop 🌸.

There is no difference in processing of the traceroute probe at an SRv6-capable and a non-SRv6-capable node. Similarly, both SRv6-capable and non-SRv6-capable nodes may use the address of the interface on which probe was received as the source address in the ICMPv6 response 🎀. ICMPv6 extensions defined in [RFC5837] can be used to display info about the IP interface through which the datagram would have been forwarded had it been forwardable, the IP next hop it would have been forwarded to, the IP interface it arrived on, and the sub-IP component of the IP interface it arrived on. The details are giving ✨.

The IP address of the interface where the traceroute probe was received is super useful 🔍. It can also help verify whether the SIDs 2001:db8:K:2:X31:: and 2001:db8:K:4:X52:: are executed correctly by nodes N2 and N4, respectively. Specifically, the info for the second hop contains the incoming interface address 2001:db8:2:3:31:: at node N3. That matches the expected interface bound to End.X behavior 2001:db8:K:2:X31:: (link3). Similarly, the info for the fourth hop contains the incoming interface address 2001:db8:4:5::52:: at node N5. That matches the expected interface bound to End.X behavior 2001:db8:K:4:X52:: (link10). Chef's kiss 👩‍🍳💋.

A.2.2. Traceroute to a SID 🧚

The mechanism for tracerouting an IPv6 address via a segment list (from the previous section) can also be used to traceroute a remote SID behavior, and the example below shows how 💜. The example uses traceroute to an End SID, as described in [RFC8986], but the procedure works for tracerouting any other SID behavior too.

Please note that the traceroute to a SID is shown using UDP probes 🙈. However, the procedure applies equally to other implementations of the traceroute mechanism. A UDP-encoded message that traceroutes a SID would use the UDP ports assigned by IANA for "traceroute use" 📝.

Say the user wants to traceroute a remote SID 2001:db8:K:4::, via 2001:db8:K:2:X31::, from node N1. The traceroute probe gets processed at each node along the path like this:

  • Node N1 initiates a traceroute probe packet as follows: (2001:db8:L:1::, 2001:db8:K:2:X31::) (2001:db8:K:4::, 2001:db8:K:2:X31::; SL=1; NH=UDP)(Traceroute probe). The first traceroute probe is sent with the hop-count value set to 1. The hop-count value goes up by 1 for each subsequent traceroute probe 🚀.
  • When node N2 receives the packet with hop-count = 1, it processes the hop-count expiry. Specifically, node N2 responds with the ICMPv6 message with type "Time Exceeded" and code "hop limit exceeded in transit". The ICMPv6 response is IP routed 🔁.
  • When node N2 receives the packet with hop-count > 1, it performs the standard SRH processing. Specifically, it executes the End.X behavior indicated by the 2001:db8:K:2:X31:: SID on the traceroute probe. If 2001:db8:K:2:X31:: is a PSP SID, node N2 executes the SID like any other data packet with DA = 2001:db8:K:2:X31:: and removes the SRH 🧼.
  • When node N3, which is a non-SRv6-capable node, receives the packet with hop-count = 1, it processes the hop-count expiry. Specifically, node N3 responds with the ICMPv6 message with type "Time Exceeded" and code "Hop limit exceeded in transit". The ICMPv6 response is IP routed 💌.
  • When node N3, which is a non-SRv6-capable node, receives the packet with hop-count > 1, it performs the standard IPv6 processing. Specifically, it forwards the traceroute probe based on DA 2001:db8:K:4:: in the IPv6 header 📬.
  • When node N4 receives the packet with DA set to the local SID 2001:db8:K:4::, it processes the End SID 🌟.
  • If the target SID (2001:db8:K:4::) is not locally instantiated and does not represent a local interface, the packet is discarded 🗑️.
  • If the target SID (2001:db8:K:4::) is locally instantiated or represents a local interface, the node processes the Upper-Layer header. As part of that Upper-Layer header processing, node N4 responds with the ICMPv6 message with type "Destination Unreachable" and code "Port Unreachable". The ICMPv6 response is IP routed 🎉.

Figure 4 shows a sample traceroute output for this example 🍓.

     > traceroute 2001:db8:K:4:X52:: via segment list 2001:db8:K:2:X31::

     Tracing the route to SID 2001:db8:K:4:X52::
     1  2001:db8:2:1:21:: 0.512 msec 0.425 msec 0.374 msec
        DA: 2001:db8:K:2:X31::,
        SRH:(2001:db8:K:4:X52::, 2001:db8:K:2:X31::; SL=1)
     2  2001:db8:3:2:21:: 0.721 msec 0.810 msec 0.795 msec
        DA: 2001:db8:K:4:X52::,
        SRH:(2001:db8:K:4:X52::, 2001:db8:K:2:X31::; SL=0)
     3  2001:db8:4:3:41:: 0.921 msec 0.816 msec 0.759 msec
        DA: 2001:db8:K:4:X52::,
        SRH:(2001:db8:K:4:X52::, 2001:db8:K:2:X31::; SL=0)

         Figure 4: Sample Output for Hop-by-Hop Traceroute to a SID

A.3. Hybrid OAM Using the OAM Flag ✨

okay bestie, this section shows off a hybrid OAM mechanism that uses the O-flag 🦋 Without loss of generality, we're assuming node N100 is a centralized controller 👑

This is not the same thing as "in situ OAM" from [RFC9197] 💅 In situ OAM records operational and telemetry info right inside the packet as it travels between two points in the network [RFC9197]. This illustration doesn't need any OAM data recorded in the packet at all 🌸

It also doesn't pick any formats for exporting data, and it doesn't say which data elements need exporting 🤷‍♀️ It does assume the system clocks on all nodes in the SR domain are synchronized ⏰

Here's the vibe. A user wants to monitor sampled IPv4 VPN 999 traffic going from CE1 to CE2 through a low-latency SR Policy P installed at node N1 🚀 To take the low-latency path, SR Policy P forces the packet through the segments 2001:db8:K:2:X31:: and 2001:db8:K:4:X52::. The VPN SID at node N7 for VPN 999 is 2001:db8:K:7:DT999::, which is a USP SID 💖 Nodes N1, N4, and N7 can process the O-flag, but node N2 can't 🙈 Node N100 is the centralized controller that can process and correlate the copies of packets sent from N1, N4, and N7, and it knows which nodes can handle O-flag processing 🧠 With a little help from N1, N4, and N7, N100 runs the hybrid OAM mechanism using the O-flag like this:

  • Packet P1 is sent from CE1 to node N1. The packet is:
P1: (IPv4 header)(payload)
  • Node N1 steers packet P1 through SR Policy P. Based on local configuration, node N1 also runs logic to sample traffic steered through SR Policy P for hybrid OAM purposes. Specification for the sampling logic is beyond the scope of this document 🍓 Say packet P1 is classified as a packet to be monitored via hybrid OAM. Node N1 sets the O-flag during the encapsulation required by SR Policy P. As part of setting the O-flag, node N1 also sends a timestamped copy of packet P1 to a local OAM process. The packet is:
P1: (2001:db8:L:1::, 2001:db8:K:2:X31::) (2001:db8:K:7:DT999::,
2001:db8:K:4:X52::, 2001:db8:K:2:X31::; SL=2; O-flag=1;
NH=IPv4)(IPv4 header)(payload)

The local OAM process sends a full or partial copy of packet P1 to node N100. The OAM process includes the recorded timestamp, additional OAM information (like incoming and outgoing interface), and any applicable metadata. Node N1 forwards the original packet towards the next segment, 2001:db8:K:2:X31:: 🚚

  • When node N2 receives the packet with the O-flag set, it ignores the O-flag, because node N2 isn't capable of processing it 🙃 Node N2 does the standard SRv6 SID and SRH processing instead. Specifically, it executes the End.X behavior [RFC8986] indicated by the 2001:db8:K:2:X31:: SID and forwards packet P1 over link3 towards node N3. The packet is:
P1: (2001:db8:L:1::, 2001:db8:K:4:X52::) (2001:db8:K:7:DT999::,
2001:db8:K:4:X52::, 2001:db8:K:2:X31::; SL=1; O-flag=1;
NH=IPv4)(IPv4 header)(payload)
  • When node N3 receives packet P1, it's a non-SRv6-capable node, so it does standard IPv6 processing. Specifically, it forwards packet P1 based on the DA 2001:db8:K:4:X52:: in the IPv6 header 📬
  • When node N4 receives packet P1, it processes the O-flag. The packet is:
P1: (2001:db8:L:1::, 2001:db8:K:4:X52::) (2001:db8:K:7:DT999::,
2001:db8:K:4:X52::, 2001:db8:K:2:X31::; SL=1; O-flag=1;
NH=IPv4)(IPv4 header)(payload)

As part of processing the O-flag, node N4 sends a timestamped copy of the packet to a local OAM process. Based on local configuration, the local OAM process sends a full or partial copy of packet P1 to node N100. The OAM process includes the recorded timestamp, additional OAM information (like incoming and outgoing interface, etc.), and any applicable metadata. Node N4 does the standard SRv6 SID and SRH processing on the original packet P1. Specifically, it executes the End.X behavior indicated by the 2001:db8:K:4:X52:: SID and forwards packet P1 over link10 towards node N5. The packet is:

P1: (2001:db8:L:1::, 2001:db8:K:7:DT999::) (2001:db8:K:7:DT999::,
2001:db8:K:4:X52::, 2001:db8:K:2:X31::; SL=0; O-flag=1;
NH=IPv4)(IPv4 header)(payload)
  • When node N5 receives packet P1, it's a non-SRv6-capable node, so it does standard IPv6 processing. Specifically, it forwards the packet based on the DA 2001:db8:K:7:DT999:: in the IPv6 header 🧭
  • When node N7 receives packet P1, it processes the O-flag. The packet is:
P1: (2001:db8:L:1::, 2001:db8:K:7:DT999::) (2001:db8:K:7:DT999::,
2001:db8:K:4:X52::, 2001:db8:K:2:X31::; SL=0; O-flag=1;
NH=IPv4)(IPv4 header)(payload)

As part of processing the O-flag, node N7 sends a timestamped copy of the packet to a local OAM process. The local OAM process sends a full or partial copy of packet P1 to node N100. The OAM process includes the recorded timestamp, additional OAM information (like incoming and outgoing interface, etc.), and any applicable metadata. Node N7 does the standard SRv6 SID and SRH processing on the original packet P1. Specifically, it executes the VPN SID indicated by 2001:db8:K:7:DT999:: and, based on a lookup in table 100, forwards packet P1 towards CE2. The packet is:

P1: (IPv4 header)(payload)
  • Node N100 processes and correlates the copies of the packets sent from nodes N1, N4, and N7 to find segment-by-segment delays and provide other hybrid OAM information related to packet P1. For segment-by-segment delay computation, it's assumed that clocks are synchronized across the SR domain 🧮
  • The process continues for any other sampled packets 🔁

A.4. Monitoring of SRv6 Paths 🛰️

Lately, network operators have shown interest in doing network OAM functions in a centralized way 👀 [RFC8403] describes exactly that kind of centralized OAM mechanism. Specifically, [RFC8403] describes a procedure for doing path continuity checks between any nodes within an SR domain from a centralized monitoring system. However, while [RFC8403] focuses on SR networks with an MPLS data plane, this document shows how the same concept can be used for path monitoring in an SRv6 network from a centralized controller 💜

In the reference topology in Figure 1, node N100 uses an IGP protocol like OSPF or IS-IS to get a view of the topology within the IGP domain. Node N100 can also use BGP-LS to get the complete view of an inter-domain topology. The controller uses that topology visibility to monitor the paths between the various endpoints 🗺️

Node N100 advertises an End SID [RFC8986] 2001:db8:K:100:1::. To monitor any arbitrary SRv6 paths, the controller can create a loopback probe that originates and terminates on node N100. To tell apart a failure in the monitored path from a loss of connectivity between the controller and the network, node N100 runs a suitable mechanism to monitor its connectivity to the monitored network 🔍

The following example shows loopback probes in which node N100 needs to verify the segment list <2001:db8:K:2:X31::, 2001:db8:K:4:X52::>:

  • Node N100 generates an OAM packet. The controller routes the probe packet towards the first segment, which is 2001:db8:K:2:X31:::
(2001:db8:L:100::, 2001:db8:K:2:X31::)(2001:db8:K:100:1::,
2001:db8:K:4:X52::, 2001:db8:K:2:X31::, SL=2)(OAM Payload)
  • Node N2 executes the End.X behavior indicated by the 2001:db8:K:2:X31:: SID and forwards the packet on link3 to node N3:
(2001:db8:L:100::, 2001:db8:K:4:X52::)(2001:db8:K:100:1::,
2001:db8:K:4:X52::, 2001:db8:K:2:X31::, SL=1)(OAM Payload)
  • Node N3, which is a non-SRv6-capable node, does standard IPv6 processing. Specifically, it forwards the packet based on the DA 2001:db8:K:4:X52:: in the IPv6 header 📦
  • Node N4 executes the End.X behavior indicated by the 2001:db8:K:4:X52:: SID and forwards the packet on link10 to node N5:
(2001:db8:L:100::, 2001:db8:K:100:1::)(2001:db8:K:100:1::,
2001:db8:K:4:X52::, 2001:db8:K:2:X31::, SL=0)(OAM Payload)
  • Node N5, which is a non-SRv6-capable node, does standard IPv6 processing. Specifically, it forwards the packet based on the DA 2001:db8:K:100:1:: in the IPv6 header 🛤️
  • Node N100 executes the standard SRv6 END behavior. It decapsulates the header and consumes the probe for OAM processing. The information in the OAM payload is used to detect missing probes, round-trip delay, and so on 🎉

The OAM payload type, and the information carried in the OAM probe, is a local implementation decision at the controller and is outside the scope of this document 🤫

Acknowledgements 🫶

The authors would like to thank Joel M. Halpern, Greg Mirsky, Bob Hinden, Loa Andersson, Gaurav Naik, Ketan Talaulikar, and Haoyu Song for their review comments 💕

Contributors 🌈

The following people contributed to this document:

Robert Raszuk Bloomberg LP Email: robert@raszuk.net

John Leddy Individual Email: john@leddy.net

Gaurav Dawra LinkedIn Email: gdawra.ietf@gmail.com

Bart Peirens Proximus Email: bart.peirens@proximus.com

Nagendra Kumar Cisco Systems, Inc. Email: naikumar@cisco.com

Carlos Pignataro Cisco Systems, Inc. Email: cpignata@cisco.com

Rakesh Gandhi Cisco Systems, Inc. Email: rgandhi@cisco.com

Frank Brockners Cisco Systems, Inc. Email: fbrockne@cisco.com

Darren Dukes Cisco Systems, Inc. Email: ddukes@cisco.com

Cheng Li Huawei Email: chengli13@huawei.com

Faisal Iqbal Individual Email: faisal.ietf@gmail.com

Authors' Addresses 💌

Zafar Ali Cisco Systems Email: zali@cisco.com

Clarence Filsfils Cisco Systems Email: cfilsfil@cisco.com

Satoru Matsushima Softbank Email: satoru.matsushima@g.softbank.co.jp

Daniel Voyer Bell Canada Email: daniel.voyer@bell.ca

Mach(Guoyi) Chen Huawei Email: mach.chen@huawei.com

📦 where this page comes from