RFC 9343

IPv6 Application of the Alternate-Marking Method

📅 published December 2022, from ietf

RFC 9343: IPv6 Application of the Alternate-Marking Method 🦋

Part 1 of 6 🎀

Okay bestie, first the vibes check 💅 This is RFC 9343, IPv6 Application of the Alternate-Marking Method, published by the IETF (Internet Engineering Task Force) in December 2022. It's a Standards Track document, so it's an actual spec and not just a vibe. The authors are G. Fioccola and T. Zhou from Huawei, M. Cociglio from Telecom Italia, F. Qin from China Mobile, and R. Pang from China Unicom. The ISSN is 2070-1721. The title block doesn't list any RFCs this one obsoletes or updates, so no drama there 🌸

Abstract ✨

Okay so this doc explains how the Alternate-Marking Method can be used as a passive performance measurement tool in an IPv6 domain. It defines an Extension Header Option to encode Alternate-Marking information in both the Hop-by-Hop Options Header and the Destination Options Header. Iconic 🔥

Status of This Memo 📜

This is an Internet Standards Track document.

This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community, and it went through public review and got approved for publication by the Internet Engineering Steering Group (IESG). If you want more on Internet Standards, check Section 2 of RFC 7841.

Info about the current status of this document, any errata, and how to give feedback lives at https://www.rfc-editor.org/info/rfc9343. Bookmark it 🔖

Copyright (c) 2022 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as in effect on the date of publication. Please actually read these, because they spell out your rights and restrictions for this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions, and they're provided without warranty as described in the Revised BSD License. Legal stuff, but it matters 🙈

Table of Contents 🧚

1.  Introduction
  1.1.  Terminology
  1.2.  Requirements Language
2.  Alternate-Marking Application to IPv6
  2.1.  Controlled Domain
    2.1.1.  Alternate-Marking Measurement Domain
3.  Definition of the AltMark Option
  3.1.  Data Fields Format
4.  Use of the AltMark Option
5.  Alternate-Marking Method Operation
  5.1.  Packet Loss Measurement
  5.2.  Packet Delay Measurement
  5.3.  Flow Monitoring Identification
  5.4.  Multipoint and Clustered Alternate Marking
  5.5.  Data Collection and Calculation
6.  Security Considerations
7.  IANA Considerations
8.  References
  8.1.  Normative References
  8.2.  Informative References
Acknowledgements
Authors' Addresses

1. Introduction ✨

[RFC9341] and [RFC9342] describe a passive performance measurement method that can measure packet loss, latency, and jitter on live traffic. Because it works by marking consecutive batches of packets, people usually call it the Alternate-Marking Method. Simple name, big energy 💖

This document defines how the Alternate-Marking Method can be used to measure performance metrics in IPv6. The whole point is to bring Alternate Marking to IPv6 so you can get detailed packet loss, delay, and delay variation measurements, both hop by hop and end to end. That means you can pinpoint exactly where the problems are in an IPv6 network 🦋

Alternate Marking is an on-path telemetry technique. It synchronizes measurements at different points in a network by switching the value of a marking bit, which divides the packet flow into batches. Each batch is a measurable entity that every network node along the path can recognize. If you count the packets in each batch and compare what different nodes measured, you can measure packet loss very precisely. The alternation of the marking bit values can also serve as a time reference for calculating delay and delay variation. The Alternate-Marking operation is described further in Section 5.

This document introduces a TLV (type-length-value) that can be encoded in the Options Headers (Hop-by-Hop or Destination), following [RFC8200], so the Alternate-Marking Method works in an IPv6 domain. Main character moment 👑

The Alternate-Marking Method MUST be applied to IPv6 only in a controlled environment, as further described in Section 2.1. [RFC8799] has more discussion of network behaviors that can be applied only within limited domains. Read that one if you're curious 🌈

The threat model for applying the Alternate-Marking Method in an IPv6 domain is covered in Section 6. Don't skip it 💀

1.1. Terminology

This document uses the terms related to the Alternate-Marking Method as defined in [RFC9341] and [RFC9342]. Same vocab, no new surprises 🧚

1.2. Requirements Language

Okay, the BCP 14 key words are the ones that set the rules. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here. Capitals = serious business 💅

2. Alternate-Marking Application to IPv6 💜

The Alternate-Marking Method needs a marking field. A few options could work, like IPv6 Extension Headers, the IPv6 Address, and the Flow Label. But you have to look at the drawbacks of every possibility, specifically:

  • reusing an existing Extension Header for Alternate Marking leads to a non-optimized implementation;
  • using the IPv6 destination address to encode the Alternate-Marking processing is very expensive; and
  • using the IPv6 Flow Label for Alternate Marking conflicts with the utilization of the Flow Label for load distribution purposes [RFC6438].

In the end, a Hop-by-Hop Option or a Destination Option is the best choice 🫶

The approach for applying Alternate Marking to IPv6 in this memo is compliant with [RFC8200]. It involves the following operations:

  • The source node is the only one that writes the Options Header to alternately mark the flow (for both the Hop-by-Hop and Destination Option). The intermediate nodes and the destination node MUST only read the marking values of the Option without modifying the Options Header. Hands off the header, nobody else touches it 🙈
  • In the case of a Hop-by-Hop Options Header carrying Alternate-Marking bits, the Options Header is not inserted or deleted on the path, but any node along the path can read it. The intermediate nodes may or may not be configured to support this Option, and the measurement can only be done on nodes configured to read the Option. As discussed further in Section 4, the presence of the Hop-by-Hop Option should not affect traffic throughput, both on nodes that don't recognize the Option and on nodes that do support it. Real talk though, there's a gap between theory and practice. In a real implementation, packets with a Hop-by-Hop Option can get skipped or processed in the slow path. Some proposals are trying to fix this and make Hop-by-Hop Options more practical (see [PROC-HBH-OPT-HEADER] and [HBH-OPTIONS-PROCESSING]), but those aspects are out of scope for this document 😭
  • In the case of a Destination Options Header carrying Alternate-Marking bits, it is not processed, inserted, or deleted by any node along the path until the packet reaches the destination node. Note that if there's also a Routing Header (RH), any visited destination in the route list can process the Options Header. Route list = more stops, more checks ✨

A Hop-by-Hop Options Header is also useful for telling routers on the path to process the Alternate Marking. As mentioned, though, routers will only examine this Option if they're properly configured 🍓

Optimizing both the implementation and the scaling of the Alternate-Marking Method is also considered, and there needs to be a way to identify flows. The Flow Monitoring Identification (FlowMonID) field, introduced in Section 5.3, goes in this direction and is used to identify a monitored flow. Think of it as the flow's ID badge 🎀

The FlowMonID is different from the Flow Label field of the IPv6 header [RFC6437]. A source uses the Flow Label to tag sequences of packets that the network should treat as a single flow. As [RFC6438] notes, it can be used for load balancing (LB) and equal-cost multipath (ECMP). Reusing the Flow Label to identify monitored flows is not considered, because it may change the application's intent and forwarding behavior. Also, the Flow Label can be changed en route, which could break the integrity of the measurement. Those reasons make the FlowMonID necessary for IPv6. The FlowMonID is designed and only used to identify the monitored flow. Flow Label and FlowMonID in the same packet are totally disjoint, have different scopes, identify flows based on different criteria, and are meant for different use cases. Two different jobs, zero drama 💅

The rationale for the FlowMonID is discussed further in Section 5.3. This 20-bit field makes it easy and flexible to identify the monitored flow, and it enables better measurement correlation and finer granularity, since it can be combined with the conventional TCP/IP 5-tuple to identify a flow. A key point covered in Section 5.3 is how the FlowMonID stays unique and how to disambiguate it if there's a collision. Stay tuned 👀

The next section highlights an important requirement for applying Alternate Marking to IPv6. The concept of the controlled domain is explained there and is treated as an essential precondition, as also highlighted in Section 6. Bestie, this is the one to pay attention to 🌸

2.1. Controlled Domain 🏰

Okay so IPv6 has way more flex than IPv4, and people have proposed some seriously innovative apps 🦋. But for security and compatibility reasons, a bunch of them have to stay inside a controlled environment. The Alternate-Marking application to IPv6 is in that group too, as we're assuming from here on. [RFC8799] has more examples of these limited domain solutions if you want to go down that rabbit hole 🕳️.

The IPv6 application of the Alternate-Marking Method MUST be deployed in a controlled domain. ✨ It's not common for user traffic to start and end inside the controlled domain, as also mentioned in Section 2.1.1 🙈. So this is typically only applicable in an overlay network. User traffic gets encapsulated at one domain border and decapsulated at the other, and the encapsulation carries the relevant extension header for Alternate Marking. This also means an implementation MUST filter packets that carry Alternate-Marking data and are entering or leaving the controlled domain. Main character boundary control 👑.

A controlled domain is a managed network where access is selected, monitored, and controlled by enforcing policies at the domain boundaries. The goal is to discard undesired external packets coming in and check the internal packets going out 🔐. It doesn't necessarily mean a single administrative domain or a single organization 💅. A controlled domain can be one administrative domain, or it can be made up of multiple administrative domains under a defined network management. Some scenarios end up with the Alternate-Marking Method spanning more than one domain. In those cases it is RECOMMENDED that the multiple domains form one whole controlled domain while crossing the external domain, by using IPsec [RFC4301] authentication and encryption or other VPN technology that provides full packet confidentiality and integrity protection 🫶. Basically, it has to be possible to control the domain boundaries, and you might need extra precautions if the traffic goes over the Internet 🌈.

The security considerations in Section 6 also highlight this requirement 🚨.

2.1.1. Alternate-Marking Measurement Domain 🎀

The Alternate-Marking measurement domain can overlap with the controlled domain, or it can be a subset of it 🧩. How the Alternate-Marking Method gets used depends on where the controlled domain boundaries are. Here's the breakdown:

  • The user equipment can be the starting or ending node only when/if it is fully managed and belongs to the controlled domain. In that case, the user-generated IPv6 packets contain the Alternate-Marking data. But in practice this is not common, because user equipment can't be totally secured in most cases 😬.
  • The Customer Premises Equipment (CPE) or the Provider Edge (PE) routers are most likely to be the starting or ending nodes, since they can be border routers of the controlled domain 🏠. For instance, the CPE connects the user's premises with the service provider's network. It only belongs to a controlled domain if the service provider manages it and extra security measures are taken to keep it trustworthy 🔒. The CPE or the PE can typically encapsulate a received packet in an outer IPv6 header that contains the Alternate-Marking data. They can also filter and drop packets from outside the domain that have inconsistent fields, so the relevant security rules at the domain boundaries actually work 🛡️. For example, a simple security check can be to insert the Alternate-Marking data if and only if the destination is within the controlled domain ✅.

3. Definition of the AltMark Option 🧚

Here's the definition of a TLV for the Extension Header Option. It carries the data fields dedicated to the Alternate-Marking Method 💜.

3.1. Data Fields Format 📦

The figure below shows the data fields format for the enhanced Alternate-Marking TLV (AltMark) 🦋. This AltMark data can be encapsulated in the IPv6 Options Headers, either Hop-by-Hop or Destination Option.

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
                                   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                                   |  Option Type  |  Opt Data Len |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |              FlowMonID                |L|D|     Reserved      |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Where:

  • Option Type: 8-bit identifier of the type of Option that needs to be allocated. Unrecognized Types MUST be ignored on processing 🙃. For the Hop-by-Hop Options Header or Destination Options Header, [RFC8200] defines how to encode the three high-order bits of the Option Type field. The two high-order bits specify the action that must be taken if the processing IPv6 node does not recognize the Option Type. For AltMark, these two bits MUST be set to 00, which means skip over this Option and keep processing the header ⏭️. The third-highest-order bit specifies whether the Option Data can change en route to the packet's final destination. For AltMark, the value of this bit MUST be set to 0, meaning the Option Data does not change en route 🧊. So since the three high-order bits of the AltMark Option are set to 000, nodes can simply skip this Option if they don't recognize it, and the data in this Option doesn't change en route. The source is the only one that can write it 💖.
  • Opt Data Len: 4. This is the length of the Option Data Fields of this Option, in bytes 📏.
  • FlowMonID: 20-bit unsigned integer. The FlowMon identifier is described in Section 5.3 🌸. As discussed further below, it was picked as 20 bits because it's a reasonable value and a good compromise when it comes to the chance of collision 🎲. It MUST be set pseudo-randomly by the source node or by a centralized controller ✨.
  • L: Loss flag for Packet Loss Measurement, as described in Section 5.1 📉.
  • D: Delay flag for Single Packet Delay Measurement, as described in Section 5.2 ⏱️.
  • Reserved: Reserved for future use. These bits MUST be set to zero on transmission and ignored on receipt 🫧.

4. Use of the AltMark Option 🌟

The AltMark Option is the best way to implement the Alternate-Marking Method 👑. It's carried by the Hop-by-Hop Options Header and the Destination Options Header. With a Destination Option, only the source and destination nodes process it: the source node inserts it, and the destination node processes it 💌. With the Hop-by-Hop Option, any node along the path may examine it, as long as that node is explicitly configured to do so 🔍.

It's important to highlight that the Option Layout can be used as either the Destination Option or the Hop-by-Hop Option, depending on the use case. Which one you pick depends on the chosen type of performance measurement 📊. In general you need both end-to-end and hop-by-hop measurements, and the Alternate-Marking methodology allows both kinds of performance measurement by definition 🙌. In many cases the end-to-end measurement won't be enough, and hop-by-hop measurement is required. To meet that need, the most complete choice is the Hop-by-Hop Options Header 🥇.

IPv6, as specified in [RFC8200], lets nodes optionally process Hop-by-Hop headers 🧐. Specifically, the Hop-by-Hop Options Header is not inserted or deleted. It may be examined or processed by any node along a packet's delivery path, until the packet reaches the node (or each of the set of nodes in the case of multicast) identified in the Destination Address field of the IPv6 header. Also, nodes along a packet's delivery path are expected to only examine and process the Hop-by-Hop Options Header if they're explicitly configured to do so 🛤️.

Another scenario is when a Routing Header is present. Both Hop-by-Hop Options and Destination Options Headers can be used when there's a Routing Header. Depending on where the Destination Options sit in the header chain (before or after the Routing Header, if any), the Destination Options Headers can be processed by either the intermediate routers listed in the Routing Header or the destination node 🗺️. For example, a type of Routing Header called the Segment Routing Header (SRH) has been defined in [RFC8754] for the Segment Routing over IPv6 (SRv6) data place. More details about the SRv6 application can be found in [SRv6-AMM] 💫.

To sum up, these tools let you control which nodes do the measurement 🎯:

  • Destination Option not preceding a Routing Header => measurement only by node in Destination Address
  • Hop-by-Hop Option => every router on the path with feature enabled
  • Destination Option preceding a Routing Header => every destination node in the route list

In general, Hop-by-Hop and Destination Options are the most suitable ways to implement Alternate Marking 🍓.

Now, it's worth mentioning that Hop-by-Hop Options are not strongly recommended in [RFC7045] and [RFC8200], unless there's a clear justification to standardize one. That's because nodes may be configured to ignore the Options Header, or to drop packets containing it, or to assign them to a slow processing path 🐢. For the AltMark Data Fields described in this document, the motivation to standardize a Hop-by-Hop Option is that it's needed for Operations, Administration, and Maintenance (OAM) 🛠️. An intermediate node can read it or not, and this doesn't affect the packet behavior. The source node is the only one that writes the Hop-by-Hop Option to alternately mark the flow. So performance measurement can be done for the nodes configured to read this Option, while the others simply aren't considered for the metrics 📈.

The Hop-by-Hop Option defined in this document is designed to take advantage of how Hop-by-Hop Options are processed 🧠. Nodes that don't support this Option would be expected to ignore it if they encounter it, following the procedures of [RFC8200]. In that case, the performance measurement may not account for all links and nodes along a path 🕸️. The Hop-by-Hop Option in this document is also designed to minimize throughput impact, both on nodes that don't recognize the Option and on nodes that do support it. The reason is that the three high-order bits of the Options Header defined here are 000. In theory, per [RFC8200] and [HBH-OPTIONS-PROCESSING], that means "skip if not recognized and data does not change en route". [RFC8200] also says nodes only examine and process the Hop-by-Hop Options Header if explicitly configured to do so. For these reasons, this Hop-by-Hop Option should not affect the throughput 🏎️. However, in practice, it's important to know that things may differ in the implementation. Packets with Hop-by-Hop Options can be forced onto the slow path. This is a general issue, though, as also explained in [HBH-OPTIONS-PROCESSING] 🐌. It's also worth mentioning that applying this to a controlled domain should avoid the risk of arbitrary nodes dropping packets with Hop-by-Hop Options 🚫.

5. Alternate-Marking Method Operation ✨

okay bestie, this section is where the method actually runs 🦋 [RFC9341] brings in a bunch of methods we can use, and this document adds one extra field to make deployment easier and way more scalable 💅

5.1. Packet Loss Measurement

Packet loss measurement is honestly so simple compared to the older mechanisms, as detailed in [RFC9341] 🫶 The packets in a flow get grouped into batches, and every packet in a batch gets the L bit (Loss flag) set to the same value. The source node can flip the L bit between 0 and 1 after a fixed number of packets or on a fixed timer, and which one you use depends on the implementation ⏰ Only the source node marks packets to create the batches. The intermediate nodes just read the marks and figure out which packets belong to which batch 🧚 By counting the packets in each batch and comparing the numbers that different nodes along the path measured, you can work out the packet loss in any single batch between any two nodes 📊 Each batch is a measurable chunk that every network node along the path can recognize 💖

So the source can use either a fixed number of packets or a fixed timer to make batches, but as [RFC9341] also explains, timer-based batches are the better choice because they're more deterministic than counter-based ones 🎀 Counter-based batches have no definitive rule, which is why [RFC9341] doesn't consider them ngl 🙈 A fixed timer gives you better control over the method. You can make the batches long enough to make collecting and comparing the measurements way easier ✨ In the implementation, each node can send its counters to the controller that does the calculation. It's also possible to exchange this information using other on-path techniques, but that is out of scope for this document 🌸

Packets with different L values may get swapped at batch boundaries, and when that happens, each marked packet MUST NOT be left guessing: it is required that each router can assign each marked packet to the right batch. Two things matter here: the clock error between network nodes and the network delay 😭 These can create offsets between batches and cause out-of-order packets. The timing math explained in Section 5 of [RFC9341] must be satisfied, and it accounts for the different causes of reordering, like clock error and network delay 🔥 The idea is to define a counting interval that's available so the counters stay stable and these problems don't happen. If you ignore the effects of network delay, the condition for using this method is that the clocks in different nodes MUST be synchronized to the same clock reference with an accuracy of +/- B/2 time units, where B is the fixed time duration of the batch. That way, every node can assign each marked packet to the right batch 👑 Usually, counters get read in the middle of the batch period so you're sure you're reading quiescent counters 🌈 Basically, the length of the batches MUST be chosen large enough so the method isn't affected by those factors. The batch length can be decided based on the specific deployment scenario 🍓

   L bit=1   ----------+           +-----------+           +----------
                       |           |           |           |
   L bit=0             +-----------+           +-----------+
              Batch n        ...      Batch 3     Batch 2     Batch 1
            <---------> <---------> <---------> <---------> <--------->

                                Traffic Flow
            ===========================================================>
   L bit   ...1111111111 0000000000 11111111111 00000000000 111111111...
            ===========================================================>

      Figure 1: Packet Loss Measurement and Single-Marking Methodology
                                Using L Bit

Quick note: the previous figure shows batch durations that stay stable over time 👀 In theory, you could change the batch length over time and between different flows for extra flexibility. But in practice, that could make it way harder to correlate the information 😩

5.2. Packet Delay Measurement

The same principle used for packet loss can also be applied to one-way delay measurement 💜 Delay metrics MAY be calculated using the following two possibilities:

  • Single-Marking Methodology: This approach uses only the L bit to calculate both packet loss and delay. In this case, the D flag MUST be set to zero on transmit and ignored by the monitoring points. The alternating L bit values can act as a time reference for calculating delay. Whenever the L bit changes and a new batch starts, a network node can store the timestamp of the first packet in the new batch. That timestamp can be compared with the timestamp of the first packet of the same batch on a second node to compute packet delay. But this measurement is only accurate if no packet loss occurs and there is no packet reordering at the batch edges 🙃 A different approach is based on the concept of the mean delay. The mean delay for each batch is calculated by considering the average arrival time of the packets for that batch. There are limitations here too, though: each node needs to collect all the timestamps and calculate the average timestamp for each batch. On top of that, the information is limited to a mean value 🥲
  • Double-Marking Methodology: This approach is more complete. It uses the L bit only to calculate packet loss, and the D bit (Delay flag) is fully dedicated to delay measurements 🌟 The idea is to use the first marking with the L bit to create the alternate flow, and then, within the batches identified by the L bit, a second marking selects the packets used for measuring delay. The D bit creates a new set of marked packets that are fully identified across the network, so a network node can store the timestamps of these packets. Those timestamps can be compared with the timestamps of the same packets on a second node to compute delay values for each packet 🦋 The most efficient and robust mode is to select a single double-marked packet for each batch. That way, there's no time gap between double-marked packets to worry about reordering them 🧠 For choosing which packet to double-mark, the same considerations from Section 5.1 apply here too, so the double-marked packet can be picked within the available counting interval that isn't affected by factors like clock errors. If a double-marked packet is lost, the delay measurement for that batch is simply discarded. That's not a big problem, because it's easy to recognize the problematic batch and skip the measurement for just that one 🫧 So, to get more delay information and handle out-of-order issues, this method is the preferred one 👑

In summary, Double Marking is better than Single Marking 💅 The two approaches give slightly different pieces of information, though, and the data consumer can combine them to build a more robust data set 🌸

Just like we said in Section 5.1 for the packet counters, the timestamps can be sent to the controller responsible for the calculation, or exchanged using other on-path techniques. But that's out of scope for this document 🎀

   L bit=1   ----------+           +-----------+           +----------
                       |           |           |           |
   L bit=0             +-----------+           +-----------+

   D bit=1         +          +          +          +            +
                   |          |          |          |            |
   D bit=0   ------+----------+----------+----------+------------+-----

                                Traffic Flow
            ===========================================================>
   L bit   ...1111111111 0000000000 11111111111 00000000000 111111111...

   D bit   ...0000010000 0000010000 00000100000 00001000000 000001000...
            ===========================================================>

         Figure 2: Double-Marking Methodology Using L Bit and D Bit

Likewise, the same goes for packet delay measurement (for both Single Marking and Double Marking), and this method can also be used to measure inter-arrival jitter 🧚✨

5.3. Flow Monitoring Identification 🎀

okay bestie, the Flow Monitoring Identification (FlowMonID) is how you say "THIS is the flow we're measuring" 💅 and it's required for a few big reasons:

  • First, it cuts down on per-node config 🧚. Otherwise, every node would need its own access control list (ACL) for each monitored flow, and that's a lot. A flow identifier also gives you a flexible granularity for defining flows, and you can even pair it with other identifiers (e.g., 5-tuple) ✨
  • Second, it simplifies counter handling 🔢. Hardware processing of flow tuples (and ACL matching) is challenging and often runs into performance issues, especially on tunnel interfaces 🙈
  • Third, it makes data export encapsulation and correlation easier for the collectors 🫶

The FlowMonID MUST only be used as a monitored flow identifier in order to determine a monitored flow within the measurement domain. Ngl, this is so important because it makes identification easy and gives you better correlation too 💖

The FlowMonID allocation procedure can be stateful or stateless 🦋. With a stateful approach, the FlowMonID historic information has to be stored and tracked so values stay unique within the domain. That can get complicated, so it's out of scope for this document. The stateless approach is described below, where FlowMonID values are pseudo-randomly generated 🌸

The value of 20 bits was picked for the FlowMonID because it's a good compromise 👑. It gives a low rate of ambiguous FlowMonIDs, which is acceptable for most applications. The disambiguation issue can be solved by tagging the pseudo-randomly generated FlowMonID with additional flow information. Specifically, it is RECOMMENDED to consider the 3-tuple FlowMonID, source, and destination addresses:

  • If the 20-bit FlowMonID is set independently and pseudo-randomly in a distributed way, collisions can happen 😭. The birthday problem from probability theory tells us that if the 20-bit FlowMonID is set independently and pseudo-randomly with no additional input entropy, there's a 50% chance of collision for 1206 flows. So for more entropy, the FlowMonID gets combined with source and destination addresses. Since there's a 1% chance of collision for 145 flows, you can monitor 145 concurrent flows per host pair with a 1% chance of collision 💅
  • If the 20-bit FlowMonID is set pseudo-randomly but in a centralized way, the controller can instruct the nodes so that FlowMonID uniqueness is guaranteed 🍓. With 20 bits, there are 1048576 combinations, and the controller should ensure that all the FlowMonID values are used without any collision. So by considering source and destination addresses together with the FlowMonID, you can monitor 1048576 concurrent flows per host pair ✨

A consistent approach MUST be used in the Alternate-Marking deployment so you don't end up mixing different ways of identifying flows 🚫. All the nodes along the path and involved in the measurement SHOULD use the same mode for identification. As mentioned, it is RECOMMENDED to use the FlowMonID for identification in combination with source and destination addresses to identify a flow. With that combo, you get 145 concurrent flows per host pair with a 1% chance of collision for pseudo-randomly generated FlowMonIDs, or 1048576 concurrent flows per host pair with a centralized controller 👑. Worth mentioning: the centralized control approach allows finer granularity, so it adds even more flexibility to flow identification 🌈

The FlowMonID field is set at the source node, which is the ingress point of the measurement domain, and it can be set in two ways 🧚:

  • It can be algorithmically generated by the source node, which can set it pseudo-randomly with some chance of collision. This approach can't guarantee uniqueness, because conflicts and collisions are possible. But since FlowMonID is meant to be used with source and destination addresses, the conflict probability goes down thanks to the FlowMonID space available for each endpoint pair (i.e., 145 flows with 1% chance of collision) 💜
  • It can be assigned by the central controller. The controller knows the network topology, so it can allocate values properly to avoid or minimize ambiguity and guarantee uniqueness 🔥. The controller can also verify there's no ambiguity between different pseudo-randomly generated FlowMonIDs on the same path. The conflict probability is really small because FlowMonID is coupled with source and destination addresses, and up to 1048576 flows can be monitored for each endpoint pair 💅. When all values in the FlowMonID space are used up, the centralized controller can keep track of and reassign values that old flows no longer use ♻️

If the FlowMonID is set by the source node, intermediate nodes can read the FlowMonIDs from packets in flight and act accordingly 🦋. If the FlowMonID is set by the controller, both options are feasible for intermediate nodes: they can learn by reading the packets, or they can be instructed by the controller 🫶.

Setting the FlowMonID at the source node may seem faster and more scalable than setting it through the controller. But it's supposed that the controller doesn't slow things down, since it can enable the Alternate-Marking Method and its parameters (like FlowMonID) together with flow instantiation, as further described in [BGP-SR-POLICY-IFIT] and [PCEP-IFIT] ✨

5.4. Multipoint and Clustered Alternate Marking 🌸

The Alternate-Marking Method can be extended to any kind of multipoint-to-multipoint paths 🌐. [RFC9341] only applies to point-to-point unicast flows, while the Clustered Alternate-Marking Method, introduced in [RFC9342], is valid for multipoint-to-multipoint unicast flows, anycast, and ECMP flows 💖

[RFC9342] describes the network clustering approach, which allows flexible and optimized performance measurement 🎀. A cluster is the smallest identifiable non-trivial subnetwork of the entire network graph that still satisfies the condition that the number of packets going in is the same as the number going out 🧩. With network clustering, you can partition the network into clusters at different levels to get the degree of detail you need 🔍.

For Multipoint Alternate Marking, FlowMonID can identify a multipoint-to-multipoint flow in general, not only a point-to-point flow 👑.

5.5. Data Collection and Calculation 📊

The nodes enabled to perform performance monitoring collect the values of the packet counters and timestamps 🕰️. There are several ways to implement data collection and calculation, but this document doesn't specify any of them 🤷‍♀️.

There are documents on the control plane mechanisms of Alternate Marking, e.g., [BGP-SR-POLICY-IFIT] and [PCEP-IFIT] 🧚‍♀️💜

6. Security Considerations ✨

okay bestie, let's talk security 🔐 this doc is about a measurement method, so it doesn't directly mess with Internet security or the apps running on it 💅 but ngl, implementing it means being super mindful of security and privacy 🫶

there are two kinds of security worries: harm caused by the measurements and harm to the measurements 👀

Harm caused by the measurement: Alternate Marking means the source node inserts an Options Header into IPv6 packets 📦 and that has to happen without messing with the quality of service the packets get, and it has to keep the routers doing the measuring stable and performing well 🏎️ like we talked about in Section 4, the AltMark Option was designed with throughput in mind, so it can run without hurting anyone's user experience 💖

Harm to the measurement: routers could alter the AltMark Option fields (like the marking or FlowMonID) 😤 or a sneaky attacker could add the AltMark Option to packets just to burn through the resources of network devices and the entities involved. remember, the source node is the only one that writes the Options Header 🙅‍♀️ intermediate and destination nodes just read it and never modify it. but an on-path attacker can still flip the flags (on purpose or by accident), sneak an Option into the packet flow, or delete one from it 💀 that could give the appearance of loss or delay, or invalidate the measurement by changing Option identifiers like FlowMonID. the malicious plan could be to make the network admin do stuff that isn't necessary, or to hide real issues in the network 🙈

because network nodes deliberately altering the bits of the AltMark Option, or injecting Options Headers as a Denial of Service (DoS) trick, could mess with the measurement, Alternate Marking MUST be applied in the context of a controlled domain, where the network nodes are locally administered and this type of attack can be avoided 🧱 for this reason, the method is not implemented on an end node that isn't fully managed and doesn't belong to the controlled domain. packets generated outside the controlled domain may eat up router resources by maliciously using the Hop-by-Hop Option, but this can be mitigated by filtering those packets at the controlled domain boundary. that works because if an end node isn't part of the controlled domain, it's not supposed to add the AltMark Hop-by-Hop Option, so it's easy to recognize 👀

An attacker outside the controlled domain can maliciously send packets with the AltMark Option 🙄 but if Alternate Marking isn't supported in the controlled domain, no problem happens. the AltMark Option gets treated like any other unrecognized Option and ignored, since the nodes aren't configured to deal with it. so the only effect is the increased packet size (by 48 bits). if Alternate Marking is supported in the controlled domain, though, the measurements need protecting, so external packets with the AltMark Option MUST be filtered. like any other Hop-by-Hop Options or Destination Options, AltMark Options entering or leaving the domain can be filtered, e.g., by using ACL extensions for filtering 🛡️

The flow identifier (FlowMonID), together with the two marking bits (L and D), makes up the AltMark Option. As explained in Section 5.3, there's a chance of collision if the FlowMonID is set pseudo-randomly, but there is a solution for this issue 💡 in general, this may not be a big deal. a low rate of ambiguous FlowMonIDs can be acceptable, since it doesn't cause significant harm to operators or their clients, and that harm may not justify the complications of avoiding it. but for large scale measurements, where a big number of flows could be monitored, the probability of a collision is higher, so disambiguating the FlowMonID field can be considered 🤓

The privacy concerns also need to be analyzed, even though the method only relies on info in the Options Header without releasing any user data 🕵️ from a confidentiality perspective, the AltMark Option doesn't contain user data, but its metadata can be used for network reconnaissance. that could compromise users' privacy by letting attackers collect info about network performance and network paths. the AltMark Option has two kinds of metadata: the marking bits (L and D) and the flow identifier (FlowMonID).

  • The marking bits are small bits of info exchanged between network nodes. because of that intrinsic characteristic, network reconnaissance through passive eavesdropping on data plane traffic is difficult. an attacker can't gain info about network performance from a single monitoring point. the only way an attacker could do it is by eavesdropping on multiple monitoring points at the same time, because they'd have to do the same kind of calculation and aggregation that Alternate Marking requires.
  • The FlowMonID field is the identifier of the monitored flow in the AltMark Option. it's way more sensitive for network reconnaissance, and it may allow a flow tracking type of attack, since an attacker could collect info about network paths 😬

Furthermore, in a pervasive surveillance attack, more info can be derived over time 👁️ but, as described later, applying Alternate Marking to a controlled domain helps mitigate all of the above privacy concerns 🌸

At the management plane, attacks can be set up by misconfiguring or maliciously configuring the AltMark Option. so AltMark Option configuration MUST be secured in a way that authenticates authorized users and verifies the integrity of configuration procedures. solutions to ensure the integrity of the AltMark Option are outside the scope of this document. also, attacks on the reporting of statistics between the monitoring points and the network management system (e.g., a centralized controller) can interfere with the proper functioning of the system. so the channels used to report back flow statistics MUST be secured.

As stated above, the precondition for applying Alternate Marking is that it MUST be applied in specific controlled domains, thus confining the potential attack vectors within the network domain 🏰 a limited administrative domain gives the network administrator the means to select, monitor, and control access to the network, which makes it a trusted domain. in this regard, it's expected to enforce policies at the domain boundaries, filtering both external packets with the AltMark Option entering the domain and internal packets with the AltMark Option leaving it. so the trusted domain is unlikely to be subject to hijacking, since packets with the AltMark Option are processed and used only within the controlled domain 👑

As stated, applying it to a controlled domain ensures control over packets entering and leaving the domain, but despite that, leakages may happen for different reasons, like a failure or a fault 😵 in that case, nodes outside the domain are expected to ignore packets with the AltMark Option, since they aren't configured to handle it, and they should not process it.

Additionally, the AltMark Option is carried by the Options Header, so it will have some impact on the packet sizes for the monitored flow and on the path MTU, since some packets might exceed the MTU 📏 however, the relatively small size (48 bits in total) of these Options Headers, plus its application to a controlled domain, helps mitigate the problem.

It's worth mentioning that the security concerns may change based on the specific deployment scenario and related threat analysis, which can lead to specific security solutions that are beyond the scope of this document 🔭 for example, the AltMark Option can be used as a Hop-by-Hop or Destination Option. with a Destination Option, the AltMark Option may traverse multiple administrative domains, so it isn't confined to a single one. in that case, a user who knows the risks may still want to use Alternate Marking for telemetry and test purposes, but the controlled domain must be composed of more than one administrative domain. to make that work, the inter-domain links need to be secured (e.g., by IPsec or VPNs) to avoid external threats and realize the whole controlled domain 🔐

It might theoretically be possible to modulate the marking or the other fields of the AltMark Option so they serve as a covert channel for an on-path observer 🫣 this may affect both the data and management planes, but, here too, the application to a controlled domain helps reduce the effects.

The Alternate-Marking application described in this document relies on a time synchronization protocol ⏰ so by attacking the time protocol, an attacker can potentially compromise the integrity of the measurement. a detailed discussion of the threats against time protocols and how to mitigate them is presented in [RFC7384]. Network Time Security (NTS), described in [RFC8915], is a mechanism that can be employed. also, the time distributed to the network nodes through the time protocol is centrally taken from an external accurate time source, such as an atomic clock or a GPS clock. by attacking the time source, it's possible to compromise the integrity of the measurement as well. there are security measures that can be taken to mitigate GPS spoofing attacks, and a network administrator should certainly employ solutions to secure the network domain 🛰️

7. IANA Considerations 🎀

IANA has allocated the Option Type in the "Destination Options and Hop-by-Hop Options" subregistry of the "Internet Protocol Version 6 (IPv6) Parameters" registry (<https://www.iana.org/assignments/ipv6-parameters/>) as follows 🍓

        +===========+===================+=============+===========+
        | Hex Value | Binary Value      | Description | Reference |
        +===========+=====+=====+=======+=============+===========+
        |           | act | chg | rest  |             |           |
        +===========+=====+=====+=======+=============+===========+
        | 0x12      | 00  | 0   | 10010 | AltMark     | RFC 9343  |
        +-----------+-----+-----+-------+-------------+-----------+

            Table 1: Destination Options and Hop-by-Hop Options
                                  Registry

8. References 📚

8.1. Normative References 📚

okay bestie, these are the must-reads 🫶 if you're implementing this stuff, you need every single one of these

8.2. Informative References 🔍

these ones are the extra reading, the deep-dive stuff that's super useful but not strictly required to get the core idea 🌸

Acknowledgements ✨

The authors would like to thank Bob Hinden, Ole Troan, Martin Duke, Lars Eggert, Roman Danyliw, Alvaro Retana, Eric Vyncke, Warren Kumari, Benjamin Kaduk, Stewart Bryant, C. A. Wood, Yoshifumi Nishida, Tom Herbert, Stefano Previdi, Brian Carpenter, Greg Mirsky, and Ron Bonica for their valuable comments and suggestions. 🫶💜

Authors' Addresses 💖

these are the humans who made it all happen, so here's where you can find them 🧚

Giuseppe Fioccola
Huawei
Riesstrasse, 25
80992 Munich
Germany
Email: giuseppe.fioccola@huawei.com
Tianran Zhou
Huawei
156 Beiqing Rd.
Beijing
100095
China
Email: zhoutianran@huawei.com
Mauro Cociglio
Telecom Italia
Email: mauro.cociglio@outlook.com
Fengwei Qin
China Mobile
32 Xuanwumenxi Ave.
Beijing
100032
China
Email: qinfengwei@chinamobile.com
Ran Pang
China Unicom
9 Shouti South Rd.
Beijing
100089
China
Email: pangran@chinaunicom.cn
📦 where this page comes from