RFC 9342
Clustered Alternate-Marking Method
RFC 9342: Clustered Alternate-Marking Method ✨
Okay bestie, let's dive into RFC 9342 🦋. It's a Standards Track doc from the IETF, published in December 2022, with ISSN 2070-1721. The editor is G. Fioccola from Huawei Technologies, and the co-authors are M. Cociglio (Telecom Italia), A. Sapio (Intel Corporation), R. Sisto (Politecnico di Torino), and T. Zhou (Huawei Technologies) 💜. It obsoletes RFC 8889, so that older spec is officially replaced by this one 🌸.
Abstract 🎀
So this doc takes the Alternate-Marking method and levels it up so it can measure any unicast flow whose packets might take several different paths through the network. That means multipoint-to-multipoint, which is giving big, complicated topology energy 🔥. The trick is a network clustering approach, which is why the technique is called "Clustered Alternate Marking" 💅. And yep, it obsoletes RFC 8889 💖.
Status of This Memo ⭐
This is an Internet Standards Track document 🌈.
It's a product of the Internet Engineering Task Force (IETF), so it represents the consensus of the IETF community. It went through public review and got approved for publication by the Internet Engineering Steering Group (IESG) ✨. If you want more info on Internet Standards, check out Section 2 of RFC 7841 🧚.
Info about the current status of this doc, any errata, and how to send feedback lives at https://www.rfc-editor.org/info/rfc9342 🍓.
Copyright Notice 🦋
Copyright (c) 2022 IETF Trust and the persons identified as the document authors. All rights reserved 💜.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, because they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions, and they're provided without warranty as described in the Revised BSD License 🙈.
Table of Contents 🌸
1. Introduction
1.1. Summary of Changes from RFC 8889
1.2. Requirements Language
2. Terminology
2.1. Correlation with RFC 5644
3. Flow Classification
4. Extension of the Method to Multipoint Flows
4.1. Monitoring Network
4.2. Network Packet Loss
5. Network Clustering
5.1. Algorithm for Clusters Partition
6. Multipoint Packet-Loss Measurement
7. Multipoint Delay and Delay Variation
7.1. Delay Measurements on a Multipoint-Paths Basis
7.1.1. Single-Marking Measurement
7.2. Delay Measurements on a Single-Packet Basis
7.2.1. Single- and Double-Marking Measurement
7.2.2. Hashing Selection Method
8. Synchronization and Timing
9. Recommendations for Deployment
10. A Closed-Loop Performance-Management Approach
11. Security Considerations
12. IANA Considerations
13. References
13.1. Normative References
13.2. Informative References
Appendix A. Example of Monitoring Network and Clusters Partition
Acknowledgements
Contributors
Authors' Addresses1. Introduction ✨
So the Alternate-Marking Method, as described in [RFC9341], works for a point-to-point path 💖. This doc's extension covers the most general case, which is a multipoint-to-multipoint path, and it gives you flexible, adaptive performance measurements in a managed network 🔥.
The Alternate-Marking methodology splits the packet flow into marking blocks, and the things you monitor are the packet counters and the timestamps for each marking period 🧚. In some setups you're monitoring a LOT of flows and nodes. Multipoint Alternate Marking exists to cut those numbers down and make performance monitoring more flexible when you don't need a super detailed analysis 🫶. For example, if you have n measurement points and m monitored flows, the packet counters for each time interval are on the order of n*m*2 (1 per color) 💀. The numbers of measurement points and flows can change depending on which part of the network you're watching (core, metro, access, and so on) and how granular you want to be (per service, per customer, etc.). So if both n and m are big, the counters blow up, and Multipoint Alternate Marking gives you a tool to control that 😭.
This approach only applies to unicast flows, not multicast 🌈. Broadcast, Unknown Unicast, and Multicast (BUM) traffic is out of scope here, because traffic replication isn't covered by the Multipoint Alternate-Marking Method 🙈. The method can also apply to anycast flows, and Equal-Cost Multipath (ECMP) paths can be monitored easily with this technique too ✨.
[RFC9341] applies to point-to-point unicast flows and BUM traffic 💜. For BUM traffic, the basic method of [RFC9341] can be applied link by link, so the multicast flow tree distribution can be split into separate unicast point-to-point links 🦋.
This document and its Clustered Alternate-Marking Method apply to multipoint-to-multipoint unicast flows, anycast, and ECMP flows 👑. So the Alternate-Marking Method can be extended to any kind of multipoint-to-multipoint path, and the network-clustering approach here is the formal way to make that happen, giving you flexible and optimized performance measurement support for network management in every situation 💅.
Without network clustering, you can only apply Alternate Marking to the whole network or per single flow 😬. With network clustering, you can partition the network into clusters at different levels to get the detail you need 🎀. Sometimes you can monitor a multipoint network just by watching its clusters, without digging in deep 🍓. If there's a problem (packet loss is measured or the delay is too high), you can enhance the filtering criteria and do a detailed analysis using a different combination of clusters, all the way down to a per-flow measurement as described in [RFC9341] 🔥.
This approach fits really well with the Closed-Loop Network and Software-Defined Network (SDN) paradigm, where the SDN orchestrator and SDN controllers are the brains of the network 🧠. They manage flow control to the switches and routers and can calibrate performance measurements based on how accurate you want them to be 🌸. An SDN controller application can orchestrate how accurately network performance monitoring is set up by applying Multipoint Alternate Marking as described in this document 💖.
It's important to underline that, since this is an extension of [RFC9341], this is a methodology document 🫶. The mechanism used to transmit the counters and timestamps is out of scope here 🙈.
This document assumes the blocks are created according to a fixed timer, as per [RFC9341] ⏰. Switching after a fixed number of packets is possible, but it's out of scope here 🦋.
Note that the fragmented packets' case can be handled with the Alternate-Marking methodology, and the same guidance given in Section 6 of [RFC9341] also applies to Multipoint Alternate Marking 🧚.
1.1. Summary of Changes from RFC 8889 💅
This document defines the Multipoint Alternate-Marking Method. It clears up some ambiguities and graduates from the experimental phase of the original spec in [RFC8889] 🎓.
The relevant changes are:
- Added recommendations about the different deployments when one or two flag bits are available for marking (Section 9) 🌈.
- Changed the structure to improve readability ✨.
- Removed the wording about experimenting with the method, plus considerations that no longer apply 🗑️.
- Revised the description of detailed aspects of the methodology, e.g., synchronization and timing ⏱️.
It's important to note that all these changes are totally backward compatible with [RFC8889], and no new technique has been introduced in this document compared to [RFC8889] 💜.
1.2. Requirements Language 🎀
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here 💖.
2. Terminology 🧚
The basic terms are identical to the ones found in Alternate Marking [RFC9341]. Just remember that [RFC9341] is valid for point-to-point unicast flows and BUM traffic 🍓.
The important new terms are explained below 👑:
Multipoint Alternate Marking: An extension to [RFC9341], valid for multipoint-to-multipoint unicast flows, anycast, and ECMP flows. It can also be called "Clustered Alternate Marking" ✨.
Flow definition: The concept of flow is generalized in this document 🌈. The identification fields are selected without any constraints, so a flow can be multipoint-to-multipoint, as a result of aggregating point-to-point flows 🦋.
Monitoring network: Identified with the nodes of the network that are the measurement points (MPs), and the links that connect MPs 💜. The monitoring network graph depends on the flow definition, so it can represent a specific flow or the entire network topology as an aggregate of all the flows 🔥. Each node of the monitoring network cannot be both a source and a destination of the flow 🙅.
Cluster: The smallest identifiable non-trivial subnetwork of the entire monitoring network graph that still satisfies the condition that the number of packets going in is the same as the number going out 💅. A cluster partition algorithm, such as the one found in Section 5.1, can be applied to split the monitoring network into clusters 🫶.
Multipoint metrics: Packet loss, delay, and delay variation are extended to multipoint flows 🌸. You can compute these metrics on the basis of multipoint paths, so the measurements can be tied to a cluster, a combination of clusters, or the entire monitored network 🎀. For delay and delay variation, you can also define the metrics on a single-packet basis, which means the multipoint path is used to easily couple packets between input and output nodes of a multipoint path ⭐.
The next section highlights the correlation with the terms used in [RFC5644] 🧚
2.1. Correlation with RFC 5644 🎀
okay so bestie, [RFC5644] is kinda limited 🥺 it only does active measurements using a single source packet or stream. And its scope is only observations of corresponding packets along the path (that's the spatial metric 🦋) and at one or more destinations (one-to-group) along the path.
So this memo is like, the glow-up 💅✨. The scope here is to define multiparty metrics for passive and hybrid measurements in a group-to-group topology with multiple sources and destinations. Main character energy 👑
[RFC5644] gave us metric names we can totally reuse 💖, but they need to be extended and rephrased so they fit the Alternate-Marking schema:
a. the multiparty metrics are not only one-to-group metrics but can also be group-to-group metrics;
b. the spatial metrics, used for measuring the performance of segments of a source-to-destination path, are applied here to clusters.
3. Flow Classification 🧚
okay so a unicast flow is identified by all the packets having a set of common characteristics 🌸. This definition is inspired by [RFC7011].
For example, if you consider a flow as all the packets sharing the same source IP address or the same destination IP address, the resulting pattern won't be point-to-point at all 🙈. It'll be point-to-multipoint or multipoint-to-point instead.
In general, a flow can be defined by a set of selection rules used to match a subset of the packets processed by the network device. These rules specify a set of Layer 3 and Layer 4 header fields (identification fields) and the relative values that must be found in matching packets 💅.
The choice of identification fields directly affects the type of paths the flow follows in the network 🦋. Like, you can relate a set of identification fields to the pattern of the resulting graphs, as listed in Figure 1.
A TCP 5-tuple usually identifies flows following either a single path or a point-to-point multipath (in the case of load balancing) 🎀. On the contrary, a single source address selects aggregate flows following a point-to-multipoint path, while a multipoint-to-point path can be the result of matching on a single destination address. And if a selection rule and its reverse are used for bidirectional measurements, they can correspond to a point-to-multipoint path in one direction and a multipoint-to-point path in the opposite direction 🔥.
So the flows to be monitored are selected into the monitoring points using packet selection rules, and those rules can also change the pattern of the monitored network 💖.
Note that, more generally, the flow can be defined at different levels based on the potential encapsulation 🌈. Additional conditions that are not in the packet header can also be included as part of the matching criteria ✨.
The Alternate-Marking Method is applicable only to a single path (and partially to a one-to-one multipath) 😭. So the extension proposed in this document is suitable for the most general case of multipoint-to-multipoint, which embraces all the other patterns in Figure 1 👑.
point-to-point single path
+------+ +------+ +------+
---<> R1 <>----<> R2 <>----<> R3 <>---
+------+ +------+ +------+
point-to-point multipath
+------+
<> R2 <>
/ +------+ \
/ \
+------+ / \ +------+
---<> R1 <> <> R4 <>---
+------+ \ / +------+
\ /
\ +------+ /
<> R3 <>
+------+
point-to-multipoint
+------+
<> R4 <>---
/ +------+
+------+ /
<> R2 <>
/ +------+ \
+------+ / \ +------+
---<> R1 <> <> R5 <>---
+------+ \ +------+
\ +------+
<> R3 <>
+------+ \
\ +------+
<> R6 <>---
+------+
multipoint-to-point
+------+
---<> R1 <>
+------+ \
\ +------+
<> R4 <>
/ +------+ \
+------+ / \ +------+
---<> R2 <> <> R6 <>---
+------+ / +------+
+------+ /
<> R5 <>
/ +------+
+------+ /
---<> R3 <>
+------+
multipoint-to-multipoint
+------+ +------+
---<> R1 <> <> R6 <>---
+------+ \ / +------+
\ +------+ /
<> R4 <>
+------+ \
+------+ \ +------+
---<> R2 <> <> R7 <>---
+------+ \ / +------+
\ +------+ /
<> R5 <>
/ +------+ \
+------+ / \ +------+
---<> R3 <> <> R8 <>---
+------+ +------+
Figure 1: Flow ClassificationThe case of unicast flow is considered in Figure 1 🍓. The anycast flow is also covered, since it is only a special case of a unicast flow if routing is stable throughout the measurement period. Furthermore, an ECMP flow is in scope by definition, since it is a point-to-multipoint unicast flow 🫶.
4. Extension of the Method to Multipoint Flows 🌈
By using the Alternate-Marking Method, only point-to-point paths can be monitored 🥲. To have an IP (TCP/UDP) flow that follows a point-to-point path, you generally have to define, with a specific value, 5 identification fields (IP Source, IP Destination, Transport Protocol, Source Port, and Destination Port).
Multipoint Alternate Marking enables performance measurement for multipoint flows selected by identification fields without any constraints (even the entire network production traffic) 🚀. It's also possible to use multiple marking points for the same monitored flow ✨.
4.1. Monitoring Network 🗺️
The monitoring network is deduced from the production network by identifying the nodes of the graph that are the measurement points and the links that are the connections between measurement points. It can be modeled as a set of nodes and a set of directed arcs that connect pairs of nodes 💜.
There are some techniques that can help with building the monitoring network (as an example, see [RFC9198]). In general, there are different options: the monitoring network can be obtained by considering all the possible paths for the traffic, or by periodically checking the traffic (e.g., daily, weekly, and monthly) and updating the graph as appropriate. But this is up to the Network Management System (NMS) configuration 🍓.
So a graph model of the monitoring network can be built according to the Alternate-Marking Method, where the monitored interfaces and links are identified. Only the measurement points and links where the traffic has flowed have to be represented in the graph 🦋.
A simple example of a monitoring network graph is shown in Appendix A 🎀.
Each monitoring point is characterized by the packet counter that refers only to a marking period of the monitored flow. Also, it is assumed that there is a monitoring point at all possible egress points of the multipoint monitored network 💅.
The same is also applicable for the delay, but it will be described in the following sections 🌸.
The rest of the document assumes that the traffic is going from left to right in order to simplify the explanation. But the analysis done for one direction applies equally to all directions 🧚.
4.2. Network Packet Loss 💀
Since all the packets of the considered flow leaving the network have previously entered the network, the number of packets counted by all the input nodes is always greater than, or equal to, the number of packets counted by all the output nodes 😭. It is assumed that routing is stable during the measurement period, while packet fragmentation must be handled as described in [RFC9341].
In the case of no packet loss occurring in the marking period, if all the input and output points of the network domain to be monitored are measurement points, the sum of the number of packets on all the ingress interfaces equals the number on egress interfaces for the monitored flow. In this circumstance, if no packet loss occurs, the intermediate measurement points only have the task of splitting the measurement 👑.
It is possible to define the network packet loss of one monitored flow for a single period. In a packet network, the number of lost packets is the number of packets counted by the input nodes minus the number of packets counted by the output nodes. This is true for every packet flow in each marking period 🔥.
The monitored network packet loss with n input nodes and m output nodes is given by:
PL = (PI1 + PI2 +...+ PIn) - (PO1 + PO2 +...+ POm)where:
- PL is the network packet loss (number of lost packets);
- PIi is the number of packets flowed through the i-th input node in this period; and
- POj is the number of packets flowed through the j-th output node in this period.
The equation is applied on a per-time-interval basis and a per-flow basis:
- The reference interval is the Alternate-Marking period, as defined in [RFC9341].
- The flow definition is generalized here. Indeed, as described before, a multipoint packet flow is considered, and the identification fields can be selected without any constraints 💖.
5. Network Clustering ✨
okay so the equation from Section 4.2 is already a total-loss queen 👑. It can tell you how many packets got lost across the whole monitored network, and it only needs the counters on the input and output nodes 🫶
but wait, it gets better. we can use the counters on all the other nodes too, to zoom in and find the smallest subnetworks where the losses are actually happening 🔍
so, as Section 2 defined it, a cluster is a non-trivial subnetwork of the whole monitoring network graph that still follows the rule that the packets going in equal the packets coming out, if no packet loss happens 🦋 a cluster should include all the arcs coming out of its input nodes and all the arcs going into its output nodes 💅 that way we count exactly the packets leaving an input node and coming back out at an output node, no matter which path they took 🌸
just like the whole monitoring network graph, the cluster is defined on a per-flow basis 💖 in a fully monitored network (where every network interface is monitored), each network device is one cluster, and each physical link is two clusters, one for each device 🧚
clusters can be different sizes depending on the flow-filtering criteria you use 🎀
also, clusters can sometimes be optionally simplified 🍓 for example, say two monitored interfaces are split by a single router (one is the input interface, the other is the output interface, and the router only has these two interfaces). instead of counting twice, once on the way in and once on the way out, you can just use one measurement point 🙈 in that case we don't care about the router's internal packet loss ✨
it's also worth saying that it might be handy to define clusters based on topology, so they work for every possible flow in the monitored network 🌈
and note that with translation or encapsulation, the cluster properties still have to stay invariant 💜
5.1. Algorithm for Clusters Partition
a simple algorithm can split the monitoring network into clusters 🧩 you do this for each direction separately, because a node can't be both a source and a destination 🙅♀️ the clusters partition is based on the monitoring network graph, which can be valid for one specific flow or be general and valid for the entire network topology 💅
it's a two-step algorithm:
- Group the links that have the same starting node;
- Join the grouped links that share at least one ending node.
since the links are unidirectional, step one means listing all the links as connections between two nodes and grouping the links that have the same starting node 🫶 you can start from any link and it will still work ✨ after that, step two means eventually joining the groups from step one by looking at their ending nodes 👑 if different groups have at least one ending node in common, they get put together and end up in the same set 🦋 once both steps are done, each set of links, together with its endpoint nodes, is one cluster 🎀
there's a simple example of the clusters partition in Appendix A 🌸
the algorithm works the same way for the more general multipoint-to-multipoint network as it does for the point-to-multipoint example 💖 but in a multipoint-to-multipoint network, the multiple sources MUST mark the traffic coherently and MUST be synchronized with all the other nodes according to the timing requirements in Section 8 🚨 this part is non-negotiable bestie 😭
once the clusters partition is done, you can calculate packet loss, delay, and delay variation on a cluster basis 🔥 the packet counters for each marking period let you work out the packet rate on a cluster basis too, so Committed Information Rate (CIR) and Excess Information Rate (EIR) can also be deduced on a cluster basis 💅
and obviously, if you combine some clusters into a new connected subnetwork, the packet-loss rule still holds 🙌 so you're also allowed to consider combinations of clusters whenever it suits you 🌈
this means that in a very large network, you don't have to set up detailed filter criteria to inspect the traffic 🧚 you can check a multipoint network and, if something's wrong, go deeper with a step-by-step cluster analysis, but only for the cluster or combination of clusters where the problem is happening 🔍
in summary, once a flow is defined, the clusters partition algorithm is based on topological information 🗺️ so it looks at all the possible links and nodes that could be crossed by the given flow, even if there's no traffic 💀 if the flow doesn't enter or go through all the nodes, the counters have a non-zero value for the nodes involved and a zero value for the nodes without traffic 🫶 but in the end, all the formulas still work ✨
the algorithm above is an iterative clustering algorithm, since it runs its steps in iterations 🔁 but you can also apply a recursive clustering algorithm, as detailed in [IEEE-ACM-TON-MPNPM] 🦋
the complete, mathematical breakdown of the possible clusters partition algorithms, including efficiency considerations and a comparison between the methods, is in the paper [IEEE-ACM-TON-MPNPM] 📚💜
6. Multipoint Packet-Loss Measurement 💅
the network packet loss from Section 4.2 is valid for the entire monitored flow, and it can easily be extended to each multipoint path 🌸 that includes the whole multipoint network, a cluster, or a combination of clusters 🧩 this way, you can calculate Multipoint Packet Loss that represents a multipoint path 🎀
the same equation from Section 4.2 works for any multipoint path, like a cluster or a combination of clusters 🫶 the number of packets is just the ones entering and leaving the multipoint path ✨
using the algorithm from Section 5.1, you can split the monitoring network into clusters 🧩 then you can measure packet loss on a cluster basis for each single period by looking at the counters of the input and output nodes that belong to that specific cluster 🔍 this can be done for every packet flow in each marking period 🦋
7. Multipoint Delay and Delay Variation 🌈
same energy here 💖 the same line of reasoning can be applied to delay and delay variation ⏱️ the delay measurement methods defined in [RFC9341] can be extended to multipoint flows 🧚 it's important to note that both delay and delay-variation measurements make sense on a multipoint path 💅 the delay variation is calculated from the same packets selected for measuring the delay 🫶
in general, you can do delay and delay-variation measurements either on multipoint paths or on single packets 👑
- Delay measurements on a multipoint-path basis mean the delay value represents an entire multipoint path (for example, the whole multipoint network, a cluster, or a combination of clusters) 🌸
- Delay measurements on a single-packet basis mean you can use a multipoint path just to easily pair up packets between the input and output nodes of that path, as described in the following sections 💖
7.1. Delay Measurements on a Multipoint-Paths Basis 🦋
7.1.1. Single-Marking Measurement 🎀
mean delay and mean delay-variation measurements can also be generalized to multipoint flows 🫶 you can compute the average one-way delay of packets in one block, a cluster, or the entire monitored network ✨
the average latency can be measured as the difference between the weighted averages of the mean timestamps of the sets of output and input nodes 🧮 that means that in the calculation, you can weigh the timestamps by the number of packets at each endpoint 💅
and since the one-way delay value represents a multipoint path, you can calculate the two-way delay of a multipoint path by adding up the one-way delays of the two directions, similar to [RFC9341] 🌈
7.2. Delay Measurements on a Single-Packet Basis 🧚
7.2.1. Single- and Double-Marking Measurement 💜
delay and delay-variation measurements linked to only one picked packet per period, whether single or double marked, can't be easily done in a multipoint scenario because of some limitations 🙈
- Single Marking based on the first or last packet of the interval doesn't work properly 💀 in a point-to-multipoint scenario, you can't tell which path the first packet of each block takes over the multipoint flow, so you can't correlate it 😭 this is also true for the general multipoint-to-multipoint scenario 🫠
- Double Marking or multiplexed marking does work, but only through statistical means 📊 in a point-to-multipoint scenario, if you select only a single packet with the second marking for each block, you can follow and calculate the delay for that picked packet ✨ but the measurement can only be done for a single path in each marking period 🦋 to go through all the paths of the multipoint flow, you can keep the measurement going for the following marking periods and hope to cover them all 🤞 in the general multipoint-to-multipoint case, you also have to account for the multiple source nodes, which make correlating the samples more complicated 🌀 in that case, it may be possible to select the second marked packet for just one source node at a time in each block, and cover the remaining source nodes one by one in the next marking periods 🎀
note that since the one-way delay measurement is done on a single-packet basis, you can always calculate the two-way delay, but it isn't immediate, because you need to couple the measurement on each single path with the opposite direction 🫶 in this case, the NMS can do the calculation 🖥️
if you do a delay measurement for more than one picked packet and for all the paths of the multipoint flow in the same marking period, then neither the Single- nor the Double-Marking Method is applicable in the multipoint scenario 🚫 the packets take different paths, and it gets really hard to correlate marked packets in a multipoint-to-multipoint path if there's more than one per period 😵💫
a desirable option is to monitor all the paths of a multipoint path at the same time in the same marking period 👑 for this, hashing can be used, as covered in the next section 🫶💖
7.2.2. Hashing Selection Method ✨
okay so sampling and filtering for IP packet selection got introduced in [RFC5474] and [RFC5475] 💅 the hash-based selection methods for delay measurement can work in a multipoint-to-multipoint path, and you can use them coupled to mean delay or totally standalone 🦋
[IEEE-NETWORK-PNPM] shows how to combine the hash method (see [RFC5474] and [RFC5475]) with the Alternate-Marking Method for point-to-point flows 🌸 this combo is called "Mixed Hashed Marking" because it's the marriage of the marking method and the hashing technique 💍 it only uses Single Marking, because it's assumed that Double Marking isn't used with hashing 🙈 pairing Single Marking with hashing lets you pick a simpler hash function, since the alternating blocks already give you time boundaries for the hashed samples 🧚 the marking batches anchor the hashed samples, which makes it way easier to match up the hashed packets along the path 🫶 for example, if a hashed sample gets lost, the damage stays inside its own block and doesn't mess up identifying the samples in the following blocks 😭
with hash-based sampling, the number of samples per block can swing a lot, because it depends on the packet rate, and that rate is variable 💀 a dynamic approach can help you keep an almost fixed number of samples in every marking period, and that's the better choice if you want regular measurements over time 🌈 in hash-based sampling, Alternate Marking creates the periods, so the hash-based samples get split into batches, and that lets you anchor each selected sample to its period 🎀 on top of that, with dynamic hash-based sampling you can adjust the length of the hash value to match the current packet rate, so the number of samples stays bounded in each marking period 👑
in a multipoint environment, hashing selection might be the answer for doing delay measurements on specific packets while getting around the limitations of Single- and Double-Marking 🔥
8. Synchronization and Timing ⏰
it's giving "timing matters" 😤 out-of-order packets happen and they have to be handled too, as described in [RFC9341] ✨
but in a multisource situation there's an extra issue to think about 👀 with a multipoint path, the egress nodes receive alternate-marked packets in random order from different ingress nodes, and that must not affect the measurement 💅
so if we look at a multipoint-to-multipoint path with more than one marking node, it's important to pin down the reference measurement interval 🦋 in general, the measurement interval used to describe the results is the one from the marking node that's most aligned with the start of the measurement, as shown in Figure 2 🌸
Note that the mark switching approach based on a fixed timer is considered in this document. 🧚
time -> start stop
T(R1) |-------------|
T(R2) |-------------|
T(R3) |------------|
Figure 2: Measurement Intervalin Figure 2, it's assumed that the node with the earliest clock (R1) identifies the right start and end times for the measurement 🎀 but that's just an assumption, and other possibilities could happen 🤷♀️ so in this case T(R1) is the measurement interval, and recognizing it is essential if you want to compare with other active, passive, or hybrid packet-loss metrics 💜
for the timing constraints of the method, [RFC9341] already covers two things that get taken into account: the clock error between network devices and the network delay between the measurement points 🫶
when we expand to a multipoint environment, there are more marking nodes marking traffic based on synchronized clock time 🌈 but because of different synchronization issues that can pop up, the marking batches can end up with different lengths and different offsets when they get mixed into a multipoint flow 😵💫 according to [RFC9341], the maximum clock skew between network devices is A 🍓 so the extra gap that appears between the multiple sources can be folded into A ✨
...BBBBBBBBB | AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA | BBBBBBBBB...
|<======================================>|
| L |
...=========>|<==================><==================>|<==========...
| L/2 L/2 |
|<====>| |<====>|
d | | d
|<========================>|
available counting interval
Figure 3: Timing Aspectsit's also assumed that the multipoint path can be modeled with a normal distribution 📈 if it can't, you'll need to reformulate based on whatever distribution it does follow 🙃 under that assumption, the guard band d is still defined as in [RFC9341] and it's calculated like this: d = A + D_avg + 3*D_stddev,
where A is the clock accuracy, D_avg is the average network delay, and D_stddev is the standard deviation of the delay 🦋
as shown in Figure 3 and according to [RFC9341], the condition the method needs to work properly is that the available counting interval must be > 0, which means: L - 2d > 0.
this formula has to be verified for each measurement point on the multipoint path 🔍
Note that the timing considerations are valid for both packet loss and delay measurements. 💖
9. Recommendations for Deployment 🚀
the methodology in the earlier sections can be used for a bunch of different performance measurement problems, as [RFC9341] also explains 🌟 [RFC8889] reports experimental examples, and [IEEE-NETWORK-PNPM] has some info on deployment experience too 🎀
there are different ways to deploy this, using one flag bit, two flag bits, or hashing selection 🧁
One flag: packet-loss measurement MUST be done as described in Section 6, applying the network clustering partition from Section 5 💅 delay measurement MUST be done with the mean delay calculation that represents the multipoint path, as described in Section 7.1.1 🦋 a Single-Marking Method based on the first/last packet of the interval can't be applied, as mentioned in Section 7.2.1 🙅♀️
Two flags: packet-loss measurement MUST be done as described in Section 6, applying the network clustering partition from Section 5 🌸 delay measurement SHOULD be done on a single-packet basis using the Double-Marking Method (Section 7.2.1) ✨ in this case the mean delay calculation (Section 7.1.1) MAY also be used as a representative value for a multipoint path 🧚 which one you pick depends on the kind of info you need, as explained further below 💜
One flag with hash-based selection: packet-loss measurement MUST be done as described in Section 6, applying the network clustering partition from Section 5 🫶 the hash-based selection methods introduced in Section 7.2.2 MUST be used for delay measurement 🔥
similar to [RFC9341], there are some operational guidelines to think about when you're deciding which recommendation to use (i.e., one flag or two flags or one flag with hash-based selection. 📝
- The Multipoint Alternate-Marking Method uses specific flags in the packet header, so the number of flags available to the implementation is a big deal 🚩 if only one flag is available, there's no other way 😤 if two flags are available, you can weigh the two-flag option against one flag with hash-based selection 💅
- The length of the Alternate-Marking period affects how often you measure, and you can set it based on the temporal sampling you need 🕰️ but you can't pick it totally freely, as explained in Section 8 🙈
- The Multipoint Alternate-Marking methods let you calculate packet loss, delay, and delay variation, but depending on the method (Single Marking, Double Marking, or hashing selection), you get different kinds of info 🌈 for example, to get measurements on a multipoint-paths basis, one flag can be used 🎀 to get measurements on a single-packet basis, two flags are preferred 👑 so the type of data your scenario needs is another thing to factor in 🍓
- The Multipoint Alternate-Marking Methods have different computational loads depending on the method used, so the computing resources available on the measurement points can also sway the choice 🧠 for example, mean delay calculation may need more processing, so it might not be the best pick if you're trying to keep the computational load down 😅
the experiment with Multipoint Alternate-Marking methodologies confirmed the benefits of the Alternate-Marking methodology [RFC9341], and its extension to the general case of multipoint-to-multipoint scenarios 🥹🎉
The Multipoint Alternate-Marking Method MUST only be applied to controlled domains, as per [RFC9341]. 🔒✨
10. A Closed-Loop Performance-Management Approach ✨
okay so the Multipoint Alternate-Marking framework in this document is seriously giving flexibility for Performance Management (PM) 💅 because it can cut down the order of magnitude of the packet counters 🔥. that means an SDN orchestrator can actually supervise, control, and manage PM even in huge networks 🌈 bestie, that's main character energy.
the monitoring network can be looked at as one whole thing, or it can be split into clusters 🧚, which are the smallest subnetworks (group-to-group segments). each cluster keeps the packet-loss property for its own subnetwork 🦋. and the clusters can get combined into new connected subnetworks at different levels, depending on how much detail you're after ✨.
an SDN controller or an NMS can calibrate performance measurements since they know the network topology 🫶. they can start without digging super deep. if something goes wrong (packet loss is measured or the delay is too high 😭), the filtering criteria can be reconfigured right away to partition the network using clusters and/or different combinations of clusters 💜. this way the problem gets localized into one specific cluster or one combination of clusters 🎀, and then you do a more detailed analysis step by step by successive approximation, all the way down to a point-to-point flow analysis. this is the so-called "closed loop" 🔁💖.
this approach can be called "network zooming" 🔍 and it can be done in two different ways:
- change the traffic filter and select more detailed flows;
- activate new measurement points by defining more specified clusters.
network zooming means some filters or rules get changed, so there's a transient time to wait once the new network configuration takes effect ⏳. the network orchestrator/controller can figure out how long that wait should be, based on the network conditions 🧠.
for example, if network zooming finds a performance problem for traffic coming from one specific source, we need to recognize the marked signal from that source node and its relative path 🛤️. for this, we can activate all the available measurement points and make the flow filter criteria more specific (i.e., 5-tuple) 🎯. or, as an alternative, it can be enough to select packets from that specific source for delay measurements. in that case, it's possible to apply the hashing technique, as mentioned in the previous sections 🌸.
[OPSAWG-IFIT-FRAMEWORK] defines an architecture where the centralized data collector and network management can apply the intelligent and flexible Alternate-Marking algorithm as described earlier 🧁.
as for [RFC9341], it's possible to classify the traffic and mark a portion of the total traffic 🏷️. for each period, the packet rate and bandwidth get calculated from the number of packets. this way the network orchestrator becomes aware if the traffic rate goes over limits 🚨. more precision can also be had by reducing the marking period 🔬; some implementations use a marking period of 1 sec or less ⚡.
on top of that, an SDN controller could also collect the measurement history 📚.
it's important to mention that the Multipoint Alternate-Marking framework also helps Traffic Visualization 👀. this methodology is super useful for figuring out which path or cluster is crossed by the flow 🗺️.
11. Security Considerations 🔐
okay this one is serious bestie 😤. this document specifies a method of performing measurements that does not directly affect Internet security or the applications that run on the Internet 🛡️. BUT implementing this method has to be mindful of security and privacy concerns, as explained in [RFC9341] 🙈💀.
12. IANA Considerations 📋
this document has no IANA actions 🌷.
13. References 📖
13.1. Normative References
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, <https://www.rfc-editor.org/info/rfc2119>.
[RFC5475] Zseby, T., Molina, M., Duffield, N., Niccolini, S., and F. Raspall, "Sampling and Filtering Techniques for IP Packet Selection", RFC 5475, DOI 10.17487/RFC5475, March 2009, <https://www.rfc-editor.org/info/rfc5475>.
[RFC5644] Stephan, E., Liang, L., and A. Morton, "IP Performance Metrics (IPPM): Spatial and Multicast", RFC 5644, DOI 10.17487/RFC5644, October 2009, <https://www.rfc-editor.org/info/rfc5644>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[RFC9341] Fioccola, G., Ed., Cociglio, M., Mirsky, G., Mizrahi, T., and T. Zhou, "Alternate-Marking Method", RFC 9341, DOI 10.17487/RFC9341, December 2022, <https://www.rfc-editor.org/info/rfc9341>.
13.2. Informative References
[IEEE-ACM-TON-MPNPM] Cociglio, M., Fioccola, G., Marchetto, G., Sapio, A., and R. Sisto, "Multipoint Passive Monitoring in Packet Networks", IEEE/ACM Transactions on Networking, Vol. 27, Issue 6, DOI 10.1109/TNET.2019.2950157, December 2019, <https://doi.org/10.1109/TNET.2019.2950157>.
[IEEE-NETWORK-PNPM] Mizrahi, T., Navon, G., Fioccola, G., Cociglio, M., Chen, M., and G. Mirsky, "AM-PM: Efficient Network Telemetry using Alternate Marking", IEEE Network, Vol. 33, Issue 4, DOI 10.1109/MNET.2019.1800152, July 2019, <https://doi.org/10.1109/MNET.2019.1800152>.
[OPSAWG-IFIT-FRAMEWORK] Song, H., Qin, F., Chen, H., Jin, J., and J. Shin, "A Framework for In-situ Flow Information Telemetry", Work in Progress, Internet-Draft, draft-song-opsawg-ifit- framework-19, 24 October 2022, <https://datatracker.ietf.org/doc/html/draft-song-opsawg- ifit-framework-19>.
[RFC5474] Duffield, N., Ed., Chiou, D., Claise, B., Greenberg, A., Grossglauser, M., and J. Rexford, "A Framework for Packet Selection and Reporting", RFC 5474, DOI 10.17487/RFC5474, March 2009, <https://www.rfc-editor.org/info/rfc5474>.
[RFC7011] Claise, B., Ed., Trammell, B., Ed., and P. Aitken, "Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of Flow Information", STD 77, RFC 7011, DOI 10.17487/RFC7011, September 2013, <https://www.rfc-editor.org/info/rfc7011>.
[RFC8889] Fioccola, G., Ed., Cociglio, M., Sapio, A., and R. Sisto, "Multipoint Alternate-Marking Method for Passive and Hybrid Performance Monitoring", RFC 8889, DOI 10.17487/RFC8889, August 2020, <https://www.rfc-editor.org/info/rfc8889>.
[RFC9198] Alvarez-Hamelin, J., Morton, A., Fabini, J., Pignataro, C., and R. Geib, "Advanced Unidirectional Route Assessment (AURA)", RFC 9198, DOI 10.17487/RFC9198, May 2022, <https://www.rfc-editor.org/info/rfc9198>.
Appendix A. Example of Monitoring Network and Clusters Partition 🧩
okay now the fun part 🎀. figure 4 shows a simple example of a monitoring network graph 🕸️:
+------+
<> R6 <>---
/ +------+
+------+ +------+ /
<> R2 <>---<> R4 <>
/ +------+ \ +------+ \
/ \ \ +------+
+------+ / +------+ \ +------+ <> R7 <>---
---<> R1 <>---<> R3 <>---<> R5 <> +------+
+------+ \ +------+ \ +------+ \
\ \ \ +------+
\ \ <> R8 <>---
\ \ +------+
\ \
\ \ +------+
\ <> R9 <>---
\ +------+
\
\ +------+
<> R10 <>---
+------+
Figure 4: Monitoring Network Graphin the monitoring network graph example, you can spot the clusters partition by applying this two-step algorithm described in Section 5.1 🔍.
the first step finds these groups:
- Group 1: (R1-R2), (R1-R3), (R1-R10)
- Group 2: (R2-R4), (R2-R5)
- Group 3: (R3-R5), (R3-R9)
- Group 4: (R4-R6), (R4-R7)
- Group 5: (R5-R8)
then the second step builds the clusters partition (and we can underline that Groups 2 and 3 connect together, since R5 is in common 🤝):
- Cluster 1: (R1-R2), (R1-R3), (R1-R10)
- Cluster 2: (R2-R4), (R2-R5), (R3-R5), (R3-R9)
- Cluster 3: (R4-R6), (R4-R7)
- Cluster 4: (R5-R8)
the flow direction considered here is from left to right ➡️. for the opposite direction, the same reasoning applies, and in this example you get the same clusters partition 🔄.
in the end, 4 clusters are obtained:
Cluster 1
+------+
<> R2 <>---
/ +------+
/
+------+ / +------+
---<> R1 <>---<> R3 <>---
+------+ \ +------+
\
\
\
\
\
\
\
\
\
\ +------+
<> R10 <>---
+------+
Cluster 2
+------+ +------+
---<> R2 <>---<> R4 <>---
+------+ \ +------+
\
+------+ \ +------+
---<> R3 <>---<> R5 <>---
+------+ \ +------+
\
\
\
\
\ +------+
<> R9 <>---
+------+
Cluster 3
+------+
<> R6 <>---
/ +------+
+------+ /
---<> R4 <>
+------+ \
\ +------+
<> R7 <>---
+------+
Cluster 4
+------+
---<> R5 <>
+------+ \
\ +------+
<> R8 <>---
+------+
Figure 5: Clusters Examplethere are clusters with more than two nodes as well as two-node clusters 🫧. in the two-node clusters, the loss is on the link (Cluster 4) 🔗. in the more-than-two-node clusters, the loss is on the cluster, but we can't know which link it is (Cluster 1, 2, or 3) 🤷♀️.
Acknowledgements 🫶
the authors would like to thank Martin Duke and Tommy Pauly for their assistance and their detailed and valuable reviews 💜.
Contributors 🌸
Greg Mirsky Ericsson Email: gregimirsky@gmail.com
Tal Mizrahi Huawei Technologies Email: tal.mizrahi.phd@gmail.com
Xiao Min ZTE Corp. Email: xiao.min2@zte.com.cn
Authors' Addresses ✨
okay bestie, here's the squad that made this doc happen 🫶 every name, address, and email is exactly as the RFC has it, so you can reach them 💌
Giuseppe Fioccola (editor)
Huawei Technologies
Riesstrasse, 25
80992 Munich
Germany
Email: giuseppe.fioccola@huawei.comMauro Cociglio
Telecom Italia
Email: mauro.cociglio@outlook.comAmedeo Sapio
Intel Corporation
4750 Patrick Henry Dr.
Santa Clara, CA 95054
United States of America
Email: amedeo.sapio@intel.comRiccardo Sisto
Politecnico di Torino
Corso Duca degli Abruzzi, 24
10129 Torino
Italy
Email: riccardo.sisto@polito.itTianran Zhou
Huawei Technologies
156 Beiqing Rd.
Beijing
100095
China
Email: zhoutianran@huawei.comand that's the whole RFC 9342 wrap-up 💜 thank u for riding through every part with me, you literally did amazing 🎀
📦 where this page comes from
- ietf: rfc9342 2022-12
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms