RFC 8889
Multipoint Alternate-Marking Method for Passive and Hybrid Performance Monitoring
Multipoint Alternate-Marking Method for Passive and Hybrid Performance Monitoring 🦋
RFC 8889 · Category: Experimental · August 2020 · ISSN 2070-1721
Okay bestie, this one is RFC 8889, published by the IETF in August 2020 🌸 It's an Experimental doc, so it's a protocol for testing and evaluation, not a standards-track spec. The editor is G. Fioccola from Huawei Technologies, with co-authors M. Cociglio (Telecom Italia), A. Sapio (Intel Corporation), and R. Sisto (Politecnico di Torino). The title block doesn't list anything this RFC obsoletes or updates ✨
Abstract
The Alternate-Marking method, as presented in RFC 8321, can only be applied to point-to-point flows, because it assumes that all the packets of the flow measured on one node are measured again by a single second node. This document generalizes and expands this methodology to measure any kind of unicast flow whose packets can follow several different paths in the network. In wider terms, that's a multipoint-to-multipoint network 🕸️ That's why the technique here is called "Multipoint Alternate Marking" 💖
Status of This Memo
This document is not an Internet Standards Track specification. It's published for examination, experimental implementation, and evaluation 🧪
It defines an Experimental Protocol for the Internet community. It's a product of the Internet Engineering Task Force (IETF) and represents the consensus of the IETF community. It got public review and was approved for publication by the Internet Engineering Steering Group (IESG). Not every IESG-approved doc is a candidate for any level of Internet Standard, so see Section 2 of RFC 7841 for the details 🙈
For info on the current status of this doc, any errata, and how to send feedback, check https://www.rfc-editor.org/info/rfc8889 ✨
Copyright Notice
Copyright (c) 2020 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as in effect on the date of publication. Please review these documents carefully, because they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they're provided without warranty as described in the Simplified BSD License 📜
Table of Contents 🗂️
1. Introduction
2. Terminology
2.1. Correlation with RFC 5644
3. Flow Classification
4. Multipoint Performance Measurement
4.1. Monitoring Network
5. Multipoint Packet Loss
6. Network Clustering
6.1. Algorithm for Clusters Partition
7. Timing Aspects
8. Multipoint Delay and Delay Variation
8.1. Delay Measurements on a Multipoint-Paths Basis
8.1.1. Single-Marking Measurement
8.2. Delay Measurements on a Single-Packet Basis
8.2.1. Single- and Double-Marking Measurement
8.2.2. Hashing Selection Method
9. A Closed-Loop Performance-Management Approach
10. Examples of Application
11. Security Considerations
12. IANA Considerations
13. References
13.1. Normative References
13.2. Informative References
Acknowledgements
Authors' Addresses1. Introduction 🎀
The Alternate-Marking method, as described in RFC 8321 [RFC8321], works for a point-to-point path. The extension in this doc covers the most general case, which is multipoint-to-multipoint paths, and it gives you flexible and adaptive performance measurements in a managed network 💅
Okay so the Alternate-Marking methodology in RFC 8321 [RFC8321] lets you sync up measurements at different points by chopping the packet flow into batches. That means you get coherent counters and can see what's happening in every marking period for each monitored flow ✨ The monitoring parameters are the packet counter and the timestamps of a flow for each marking period. More details on where Alternate Marking applies are in RFC 8321 [RFC8321], and implementation details are in the paper "AM-PM: Efficient Network Telemetry using Alternate Marking" [IEEE-Network-PNPM] 📄
Some Alternate-Marking setups have tons of monitored flows and nodes 😭 Multipoint Alternate Marking wants to bring those numbers down and make performance monitoring more flexible when you don't need a super detailed analysis. For example, if you've got n measurement points and m monitored flows, the packet counters for each time interval come out to n*m*2 (1 per color). How many measurement points and flows you end up with depends on the part of the network you're watching (core network, metro network, access network) and how granular you go (each service, each customer). So if both n and m are high, the packet counters blow up, and Multipoint Alternate Marking gives you a tool to control those parameters 🧮
This approach is only for unicast flows, not multicast. Broadcast, Unknown Unicast, and Multicast (BUM) traffic is out of scope here, because traffic replication isn't covered by the Multipoint Alternate-Marking method 🙅♀️ It can apply to anycast flows, and Equal-Cost Multipath (ECMP) paths can also be easily monitored with this technique 🌈
Short version: RFC 8321 [RFC8321] applies to point-to-point unicast flows and BUM traffic, while this doc and its Clustered Alternate-Marking method are valid for multipoint-to-multipoint unicast flows, anycast, and ECMP flows 🧚
So the Alternate-Marking method can be extended to any kind of multipoint-to-multipoint path. The network-clustering approach in this doc is the formal way to make that happen, and it gives you flexible, optimized performance measurement support for network management in every situation 💜
Without network clustering, you can only apply Alternate Marking to the whole network or per single flow. With network clustering, you can partition the network into clusters at different levels to get the degree of detail you need. In some cases you can monitor a multipoint network just by looking at the network clustering, without digging in deep. If there's a problem (packet loss is measured, or the delay is too high), you can specify the filtering criteria more narrowly to do a detailed analysis, using a different combination of clusters, all the way down to a per-flow measurement as described in RFC 8321 [RFC8321] 🔍
This approach fits really well with the Closed-Loop Network and Software-Defined Network (SDN) paradigm. The SDN orchestrator and the SDN controllers are the brains of the network. They manage flow control to the switches and routers and can calibrate the performance measurements depending on the accuracy you want 🧠 An SDN controller application can decide how accurately network performance monitoring is set up by applying the Multipoint Alternate Marking described in this doc 👑
Important: as an extension of RFC 8321 [RFC8321], this is a methodology document. So the mechanism used to transmit the counters and timestamps is out of scope here, and the implementation is open. There are several options, e.g., see "Enhanced Alternate Marking Method" [ENHANCED-ALTERNATE-MARKING] 🎁
Fragmentation check: the fragmented packets case can be managed with the Alternate-Marking methodology only if fragmentation happens outside the monitored portion of the network. This is always true for both RFC 8321 [RFC8321] and Multipoint Alternate Marking, as explained here 🚧
2. Terminology 📚
The basic terms are identical to the ones in Alternate Marking [RFC8321]. Just remember that RFC 8321 [RFC8321] is valid for point-to-point unicast flows and BUM traffic 🧠
These are the important new terms you need to know:
- Multipoint Alternate Marking: An extension to RFC 8321 [RFC8321], valid for multipoint-to-multipoint unicast flows, anycast, and ECMP flows. It can also be called Clustered Alternate Marking 🦋
- Flow definition: The concept of flow is generalized in this doc. The identification fields are selected without any constraints, and in general the flow can be a multipoint-to-multipoint flow, as a result of aggregate point-to-point flows 🌊
- Monitoring network: Identified with the nodes of the network that are the measurement points (MPs) and the links that connect the MPs. The monitoring network graph depends on the flow definition, so it can represent a specific flow or the entire network topology as an aggregate of all the flows 🗺️
- Cluster: The smallest identifiable subnetwork of the entire monitoring network graph that still satisfies the condition that the number of packets that go in is the same as the number that go out 🫧
- Multipoint metrics: Packet loss, delay, and delay variation are extended to the case of multipoint flows. You can compute these metrics on the basis of multipoint paths in order to associate the measurements to a cluster, a combination of clusters, or the entire monitored network 📊 For delay and delay variation, you can also define the metrics on a single-packet basis. That means the multipoint path is used to easily couple packets between the input and output nodes of a multipoint path 🎯
The next section highlights the correlation with the terms used in RFC 5644 [RFC5644] 👀
2.1. Correlation with RFC 5644
Real talk: RFC 5644 [RFC5644] is limited to active measurements using a single source packet or stream. Its scope is also limited to observations of corresponding packets along the path (spatial metric) and at one or more destinations (one-to-group) along the path 🙃
This memo is different. Its scope is to define multiparty metrics for passive and hybrid measurements in a group-to-group topology with multiple sources and destinations 🫶
RFC 5644 [RFC5644] introduces metric names that can be reused here, but they have to be extended and rephrased to apply to the Alternate-Marking schema:
a. the multiparty metrics are not only one-to-group metrics but can also be group-to-group metrics;
b. the spatial metrics, used for measuring the performance of segments of a source-to-destination path, are applied here to group-to-group segments (called clusters) 💖
3. Flow Classification ✨
okay so a unicast flow is basically every packet that shares a bunch of common traits 🦋 and this idea is inspired by RFC 7011 [RFC7011] 💅
here's the fun part 🫶 if you say a flow is "all the packets with the same source IP" or "all the packets with the same destination IP," the shape you get is NOT a neat point-to-point line. it's point-to-multipoint or multipoint-to-point, which is so much more chaotic than you'd think 😭
in general, a flow gets defined by selection rules that grab a subset of the packets a network device handles. these rules list Layer 3 and Layer 4 header fields (the identification fields) plus the values those fields have to hold in matching packets 🎀
and here's the tea ☕ the identification fields you pick totally decide what path the flow takes through the network. each set of fields maps to a certain graph pattern, which is listed in Figure 1 🌈
a TCP 5-tuple usually picks out flows that follow either one single path or a point-to-multipoint multipath (when load balancing is happening) 🌸 a single source address, on the other hand, selects aggregate flows that go point-to-multipoint, and matching on one destination address can give you a multipoint-to-point. and if you use a selection rule AND its reverse for bidirectional measurements, one direction can look like point-to-multipoint while the opposite direction looks like multipoint-to-point. it's giving two different personalities depending on which way you look 🙈
so the flows you want to monitor get selected at the monitoring points using packet selection rules, and those rules can also change the pattern of the network being monitored 🧚
also, more generally, a flow can be defined at different levels depending on the potential encapsulation 🍓 and extra conditions that aren't in the packet header can be part of the matching criteria too ✨
the Alternate-Marking method only works for a single path (and only partly for a one-to-one multipath), so the extension in this document is built for the most general case: multipoint-to-multipoint, which covers all the other patterns in Figure 1 👑
point-to-point single path
+------+ +------+ +------+
---<> R1 <>----<> R2 <>----<> R3 <>---
+------+ +------+ +------+
point-to-point multipath
+------+
<> R2 <>
/ +------+ \
/ \
+------+ / \ +------+
---<> R1 <> <> R4 <>---
+------+ \ / +------+
\ /
\ +------+ /
<> R3 <>
+------+
point-to-multipoint
+------+
<> R4 <>---
/ +------+
+------+ /
<> R2 <>
/ +------+ \
+------+ / \ +------+
---<> R1 <> <> R5 <>---
+------+ \ +------+
\ +------+
<> R3 <>
+------+ \
\ +------+
<> R6 <>---
+------+
multipoint-to-point
+------+
---<> R1 <>
+------+ \
\ +------+
<> R4 <>
/ +------+ \
+------+ / \ +------+
---<> R2 <> <> R6 <>---
+------+ / +------+
+------+ /
<> R5 <>
/ +------+
+------+ /
---<> R3 <>
+------+
multipoint-to-multipoint
+------+ +------+
---<> R1 <> <> R6 <>---
+------+ \ / +------+
\ +------+ /
<> R4 <>
+------+ \
+------+ \ +------+
---<> R2 <> <> R7 <>---
+------+ \ / +------+
\ +------+ /
<> R5 <>
/ +------+ \
+------+ / \ +------+
---<> R3 <> <> R8 <>---
+------+ +------+
Figure 1: Flow Classificationthe unicast case is the one pictured in Figure 1 💖 the anycast flow is in scope too, because nothing gets replicated and only one node from the anycast group receives the traffic, so it counts as a special case of unicast 🌷 and an ECMP flow is in scope by definition, since it's a point-to-multipoint unicast flow 💜
4. Multipoint Performance Measurement 🌈
the Alternate-Marking method can only monitor point-to-point paths 😤 so to get an IP (TCP/UDP) flow that follows a point-to-point path, you have to set a specific value for 5 identification fields: IP Source, IP Destination, Transport Protocol, Source Port, and Destination Port 🎀
Multipoint Alternate Marking changes the game 🔥 it lets you do performance measurement for multipoint flows selected by identification fields with no constraints at all (even the entire network's production traffic!). you can also use multiple marking points for the same monitored flow 🦋
4.1. Monitoring Network 🗺️
the monitoring network is worked out from the production network by finding the nodes of the graph that act as measurement points, plus the links that connect those measurement points 🌸
there are techniques that can help build the monitoring network (for example, see [ROUTE-ASSESSMENT]) 💅 generally you have a couple of options: the monitoring network can come from considering all the possible paths for the traffic, or from periodically checking the traffic (say daily, weekly, or monthly) and updating the graph as needed 🧚 which option you go with is up to the Network Management System (NMS) configuration ⭐
so a graph model of the monitoring network can be built with the Alternate-Marking method, and that means identifying the monitored interfaces and links 🫶 only the measurement points and links that the traffic actually flowed through need to show up in the graph 🙈
Figure 2 shows a simple example of a monitoring network graph 👑
+------+
<> R6 <>---
/ +------+
+------+ +------+ /
<> R2 <>---<> R4 <>
/ +------+ \ +------+ \
/ \ \ +------+
+------+ / +------+ \ +------+ <> R7 <>---
---<> R1 <>---<> R3 <>---<> R5 <> +------+
+------+ \ +------+ \ +------+ \
\ \ \ +------+
\ \ <> R8 <>---
\ \ +------+
\ \
\ \ +------+
\ <> R9 <>---
\ +------+
\
\ +------+
<> R10 <>---
+------+
Figure 2: Monitoring Network Grapheach monitoring point gets characterized by a packet counter, and that counter only refers to one marking period of the monitored flow 🍓
the same idea applies to delay too 🌈 but that gets covered in the following sections ✨
5. Multipoint Packet Loss 💀
here's the logic that makes this work 🔍 every packet of the flow that leaves the network had to enter the network first, so the number of packets counted by all the input nodes is always greater than or equal to the number counted by all the output nodes 🦋 noninitial fragments are not considered here 🙈
the assumption is that the Alternate-Marking method is in use 💖 when no packet loss happens during the marking period, and every input and output point of the network domain you want to monitor is a measurement point, then the sum of the packets on all the ingress interfaces equals the sum on all the egress interfaces for the monitored flow 🎀 in that situation, if there's no loss, the intermediate measurement points just have the job of splitting up the measurement 🧚
you can define the Network Packet Loss of one monitored flow for a single period 🌸 in a packet network, the number of lost packets is the number counted by the input nodes minus the number counted by the output nodes 💅 this holds for every packet flow in every marking period ✨
the monitored network packet loss, with n input nodes and m output nodes, is given by:
PL = (PI1 + PI2 +...+ PIn) - (PO1 + PO2 +...+ POm)where:
- PL is the network packet loss (number of lost packets)
- PIi is the number of packets that flowed through the i-th input node in this period
- POj is the number of packets that flowed through the j-th output node in this period
the equation gets applied on a per-time-interval basis and a per-flow basis 👑
- the reference interval is the Alternate-Marking period, as defined in RFC 8321 [RFC8321] 🌷
- the flow definition is generalized here 🌈 as we covered earlier, a multipoint packet flow is considered, and the identification fields can be selected with no constraints at all 🔥
6. Network Clustering ✨
okay so the previous equation can tell us the total number of packets lost across the whole monitored network, and it only needs the counters on the input and output nodes 💅
but wait, it gets even better 🫶 we can also use the counters sitting everywhere else in the network to zoom in and find the smallest subnetworks where the losses are actually happening. these little subnetworks are called "clusters" 🧚
a cluster graph is a subnetwork of the whole monitoring network graph that still satisfies the packet loss equation from the last section. the only difference is that PL now means the number of packets lost in the cluster. and just like the full monitoring network graph, a cluster is defined per flow 🦋
so a cluster has to include all the arcs coming out of its input nodes and all the arcs going into its output nodes. this is what makes it work: every packet that leaves an input node can be counted again at an output node, no matter which path it took. (and only those packets, so no extras sneaking in 💀)
in a completely monitored unidirectional network, where every network interface is monitored, each network device is its own cluster, and each physical link turns into two clusters, one for each device 🌸
clusters can be different sizes depending on the flow-filtering criteria you pick 👑
sometimes clusters can also be optionally simplified 🍓 for example, say two monitored interfaces are separated by one router, with one being the input interface and the other the output interface, and that router only has these two interfaces. instead of counting twice (once when packets enter and once when they leave), you can just consider a single measurement point. in this case we don't care about the router's internal packet loss at all 🙈
it's also worth pointing out that it might be handy to define clusters based on topological information, so they can be applied to all the possible flows in the monitored network ✨
6.1. Algorithm for Clusters Partition
there's a simple algorithm you can use to split the monitoring network into clusters 🎀 you do this for each direction separately. the clusters partition is based on the monitoring network graph, which can be valid for one specific flow or be general and valid for the whole network topology 💖
it's a two-step algorithm:
- Group the links where there is the same starting node;
- Join the grouped links with at least one ending node in common.
since the links are unidirectional, step one means listing all the links as connections between two nodes and grouping the different links that share the same starting node. you can start from any link and the procedure still works 🌈 then step two means joining the groups from step one by looking at their ending nodes. if different groups have at least one ending node in common, they get put together and belong to the same set. once both steps are done, each composed set of links, together with its endpoint nodes, is a cluster 🫶
now let's use this on our monitoring network graph example 🔥
step one gives us these groups:
- Group 1: (R1-R2), (R1-R3), (R1-R10)
- Group 2: (R2-R4), (R2-R5)
- Group 3: (R3-R5), (R3-R9)
- Group 4: (R4-R6), (R4-R7)
- Group 5: (R5-R8)
then step two builds the clusters partition. (look at Groups 2 and 3 connecting, since R5 is in common 👀✨)
- Cluster 1: (R1-R2), (R1-R3), (R1-R10)
- Cluster 2: (R2-R4), (R2-R5), (R3-R5), (R3-R9)
- Cluster 3: (R4-R6), (R4-R7)
- Cluster 4: (R5-R8)
the flow direction we're looking at here goes from left to right. for the opposite direction, you apply the same reasoning, and in this example you get the same clusters partition 🦋
so in the end, we get these 4 clusters:
Cluster 1
+------+
<> R2 <>---
/ +------+
/
+------+ / +------+
---<> R1 <>---<> R3 <>---
+------+ \ +------+
\
\
\
\
\
\
\
\
\ +------+
<> R10 <>---
+------+
Cluster 2
+------+ +------+
---<> R2 <>---<> R4 <>---
+------+ \ +------+
\
+------+ \ +------+
---<> R3 <>---<> R5 <>---
+------+ \ +------+
\
\
\
\
\ +------+
<> R9 <>---
+------+
Cluster 3
+------+
<> R6 <>---
/ +------+
+------+ /
---<> R4 <>
+------+ \
\ +------+
<> R7 <>---
+------+
Cluster 4
+------+
---<> R5 <>
+------+ \
\ +------+
<> R8 <>---
+------+
Figure 3: Clusters Examplesome clusters have more than two nodes, and some have just two 🧠 in the two-node clusters, the loss is on the link (that's Cluster 4). in the clusters with more than two nodes, the loss is on the cluster, but we can't tell which link it is (Cluster 1, 2, or 3) 🥺
so the packet loss calculation can be done on a cluster basis. the packet counters for each marking period let you calculate the packet rate on a cluster basis too, which means Committed Information Rate (CIR) and Excess Information Rate (EIR) could also be deduced per cluster 💅
by combining some clusters into a new connected subnetwork (called a "super cluster"), the packet-loss rule still holds 👑
this means that in a very large network, you don't need to configure detailed filter criteria to inspect the traffic. you can check a multipoint network and, if something goes wrong, dig in with a step-by-step cluster analysis, but only for the cluster or combination of clusters where the problem is happening 🎀
in summary, once a flow is defined, the algorithm for building the clusters partition is based on topological information. so it considers all the possible links and nodes the flow crosses, even if there's no traffic on them. if the flow doesn't enter or traverse all the nodes, the counters have a nonzero value for the nodes involved and a zero value for the nodes without traffic. but in the end, all the formulas are still valid 🌸
the algorithm described above is an iterative clustering algorithm, but it's also possible to apply a recursive clustering algorithm by using the node-node adjacency matrix representation [IEEE-ACM-ToN-MPNPM] 🧚
the complete and mathematical analysis of the possible clusters partition algorithms, including efficiency considerations and a comparison between the different methods, is in the paper [IEEE-ACM-ToN-MPNPM] 📚💜
7. Timing Aspects 🦋
timing matters a lot here, bestie ⏰ out-of-order packets happen, and they have to be handled too, as described in RFC 8321 [RFC8321]. but in a multisource situation, there's an extra issue to think about. with a multipoint path, the egress nodes receive alternate marked packets in random order from different ingress nodes, and this must not affect the measurement 😭
so if we analyze a multipoint-to-multipoint path with more than one marking node, it's important to figure out the reference measurement interval. in general, the measurement interval used to describe the results is the one from the marking node that lines up best with the start of the measurement, as shown in Figure 4 🌈
note that the mark switching approach based on a fixed timer is the one considered in this document ⭐
time -> start stop
T(R1) |-------------|
T(R2) |-------------|
T(R3) |------------|
Figure 4: Measurement Intervalin Figure 4, we assume that the node with the earliest clock (R1) identifies the right starting and ending times of the measurement. but that's just an assumption, and other things could happen 🙈 so in this case, T(R1) is the measurement interval, and recognizing it is essential if you want to compare with other active, passive, or hybrid Packet Loss metrics 💖
when we expand to multipoint-to-multipoint flows, we have to keep in mind that all source nodes mark the traffic, and that adds more complexity 😵💫
regarding the timing aspects of the methodology, RFC 8321 [RFC8321] already describes two contributions that are taken into account: the clock error between network devices and the network delay between measurement points 🫶
but we need to think about one more contribution now. since all source nodes mark the traffic, the source measurement intervals can have different lengths and different offsets. this mismatch m can be added to d, as shown in Figure 5 🔥
...BBBBBBBBB | AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA | BBBBBBBBB...
|<======================================>|
| L |
...=========>|<==================><==================>|<==========...
| L/2 L/2 |
|<=><===>| |<===><=>|
m d | | d m
|<====================>|
available counting interval
Figure 5: Timing Aspects for Multipoint Pathsso the misalignment between the marking source routers adds another constraint, and the value of m gets added to d (which already includes clock error and network delay) 🎀
so there are three different possible contributions to consider: clock error between network devices, network delay between measurement points, and misalignment between the marking source routers 🌸
in the end, the condition that has to be satisfied for the method to work properly is that the available counting interval must be > 0. which means:
L - 2m - 2d > 0.this formula needs to be verified for each measurement point on the multipoint path. here m is the misalignment between the marking source routers, while d, already introduced in RFC 8321 [RFC8321], takes into account clock error and network delay between network nodes. so the mismatch between measurement intervals has to satisfy this condition 💅✨
and finally, the timing considerations are valid for both packet loss and delay measurements 🧚💜
8. Multipoint Delay and Delay Variation ✨
okay so the same logic we just used for loss works for delay and delay variation too, bestie 🫶 Similar to the delay measurements in RFC 8321 [RFC8321], the marking batches anchor the samples to a particular period, and that period is the time reference you can use. 💖 Super important: both delay and delay-variation measurements make sense on a multipoint path. The delay variation is calculated from the same packets that were picked for the delay measurement. 🦋
so basically you can do delay and delay-variation measurements in two main ways:
- Multipoint-path delay measurements are about the whole path. The delay value represents an entire multipoint path (like the whole multipoint network, a cluster, or a combo of clusters). 🌈
- Single-packet delay measurements use the multipoint path just to easily pair up packets between the input and output nodes of that path. We'll get into this in the following sections. 🧚
8.1. Delay Measurements on a Multipoint-Paths Basis 💅
8.1.1. Single-Marking Measurement 🎀
okay this one is giving "averages, but make it multipoint" 💜 Mean delay and mean delay-variation measurements can be generalized to multipoint flows too. You can compute the average one-way delay of packets in one block, a cluster, or the entire monitored network. ✨
The average latency can be measured as the difference between the weighted averages of the mean timestamps of the output and input node sets. So in the calculation, you can weigh the timestamps by the number of packets for each endpoint. It's literally just math being iconic. 🔥
8.2. Delay Measurements on a Single-Packet Basis 🌸
8.2.1. Single- and Double-Marking Measurement 🙈
so delay and delay-variation measurements for just one picked packet per period (both single and double marked) can be done in a multipoint scenario, but there are some limitations, bestie 😭
Single marking based on the first or last packet of the interval would not work, because it would not be possible to agree on the first packet of the interval. 💀
Double marking or multiplexed marking would work, but each measurement would only tell you about the delay of a single path. However, by repeating the measurement multiple times, you can get info about all the paths in the multipoint flow. This works for a point-to-multipoint path, but it is more difficult for a multipoint-to-multipoint path because of the multiple source routers. 🦋
if we tried to measure more than one picked packet in the same marking period, especially for multipoint-to-multipoint measurements, neither single nor double marking is useful, because they wouldn't represent the entire flow. 🥺 The packets can take different paths with different delays, and it can be really hard to recognize marked packets in a multipoint-to-multipoint path, especially when there's more than one per period. It's giving chaos, ngl. 😵💫
A desirable option is to monitor all the paths of a multipoint path at the same time in the same marking period. For this purpose, hashing can be used, as covered in the next section. 💖
8.2.2. Hashing Selection Method 🍓
RFCs 5474 [RFC5474] and 5475 [RFC5475] introduce sampling and filtering techniques for IP packet selection. 🫶
okay so the hash-based selection methods for delay measurement can work in a multipoint-to-multipoint path, and you can use them either paired with mean delay or on their own. ✨
[ALTERNATE-MARKING] explains how to use the hash method (RFCs 5474 [RFC5474] and 5475 [RFC5475]) together with the Alternate-Marking method for point-to-point flows. It's also called Mixed Hashed Marking. Pairing a marking method with hashing is so useful because the marking batches anchor the hash-selected samples, which makes it way easier to correlate the hashed packets along the path. 👑
You can use a basic-hash or a dynamic-hash method. The basic approach has one big challenge: the frequency of sampled packets can vary a lot. That's why the dynamic approach was introduced for point-to-point flows, so you get a desired, almost fixed number of samples per measurement period. 🌈 With basic hash-based sampling, the number of samples can swing wildly because it depends on the packet rate, which changes. The dynamic approach keeps the sample count almost fixed in each marking period, which is a better option for regular measurements over time. 💅 In hash-based sampling, Alternate Marking is used to create periods, so the hash-based samples are split into batches. That lets you anchor the selected samples to their period. 🦋
In dynamic hash-based sampling, the length of the hash value adapts on the fly, which bounds the number of samples in each marking period. You do this by choosing the maximum number of samples (NMAX) to catch in a marking period. The algorithm starts with only a few hash bits, which lets it pick a bigger percentage of packets. For example, with 0 hash bits all packets are sampled, with 1 hash bit half of them are sampled, and so on. 🎀 When the number of selected packets reaches NMAX, a hash bit gets added. So the sampling now runs at half the original rate, and any packets already selected that don't match the new hash are discarded. This step can be repeated iteratively. 🔥
It's assumed that each sample includes the timestamp (used for delay measurement) and the hash value, so the management system can match the samples received from the two measurement points. 🫶 The dynamic process statistically converges at the end of a marking period, and the final number of selected samples is between NMAX/2 and NMAX. So the dynamic approach paces the sampling rate and bounds the number of sampled packets per sampling period. ✨
In a multipoint environment, the behavior is similar to a point-to-point flow. Specifically, for a multipoint-to-multipoint flow, the dynamic hash could be the solution for delay measurements on specific packets while overcoming the single- and double-marking limitations. 👑
The management system receives the samples, including the timestamps and the hash value, from all the MPs. This happens for both point-to-point and multipoint-to-multipoint flows. Then the longest hash used by the MPs is deduced and applied to couple timestamps either from the same packets of 2 MPs of a point-to-point path, or from the input and output MPs of a cluster (or a super cluster or the entire network). 🧚 But there are some considerations, bestie: if there isn't any packet loss, the set of input samples is always equal to the set of output samples. With packet loss, the set of output samples can be a subset of the input samples, but the method still works because, at the end, it's easy to couple the input and output timestamps of each caught packet using the hash. (In particular, the "unused part of the hash" should be different for each packet.) 🌸
So the basic hash is logically similar to the double-marking method, and on a point-to-point path, double marking and basic-hash selection are equivalent. 💖 The dynamic approach scales the number of measurements per interval. It might seem like double marking would also work well if we shortened the interval length, but that only works on a point-to-point path, not a multipoint path, where we can't couple the picked packets. 🥹 So in general, if you want delay measurements on a multipoint-to-multipoint path and want to select more than one packet per period, double marking can't be used, because you couldn't couple the picked packets between input and output nodes. On the other hand, you can do that with hashing selection. 🫧
9. A Closed-Loop Performance-Management Approach ✨
okay so the Multipoint Alternate-Marking framework in this document is lowkey a game changer for Performance Management (PM) 💅 because it can shrink the order of magnitude of the packet counters. 🔥 That means an SDN orchestrator can actually supervise, control, and manage PM even in huge networks. bestie, the vibes are immaculate 🦋
The monitoring network can be looked at as one whole thing, or split up into clusters, which are the smallest subnetworks (group-to-group segments). 🧚 Each subnetwork keeps the packet-loss property, so nothing gets lost in the vibe. 🌸 Clusters can also get stitched together into new connected subnetworks at different levels, depending on how much detail you're after. ✨
An SDN controller or a Network Management System (NMS) can calibrate performance measurements, since they know the network topology. 👑 They can start without digging super deep. 🙈 If something goes sideways (packet loss is measured or the delay is too high), the filtering criteria can be reconfigured immediately to partition the network using clusters and/or different combinations of clusters. 💖 That way the problem gets localized to one specific cluster or one combination of clusters, and you zoom in step by step by successive approximation, all the way down to a point-to-point flow analysis. This is the so-called "closed loop". 🔁
This approach is called "network zooming" and it can be done in two different ways:
- change the traffic filter and select more detailed flows; 💅
- activate new measurement points by defining more specified clusters. 🎀
Network zooming means some filters or rules get changed, so there's a transient time to wait once the new network configuration takes effect. ⏳ The Network Orchestrator/Controller can decide that wait time based on the network conditions. 🫶
For example, if network zooming finds a performance problem for traffic coming from a specific source, we need to recognize the marked signal from that source node and its relative path. 🦋 To do that, we can activate all the available measurement points and better specify the flow filter criteria (i.e., 5-tuple). 💜 Or, as an alternative, it can be enough to select packets from that specific source for delay measurements. In that case, it is possible to apply the hashing technique, as mentioned in the previous sections. 🌈
[IFIT-FRAMEWORK] defines an architecture where the centralized Data Collector and Network Management can apply the intelligent and flexible Alternate-Marking algorithm as previously described. 🧠✨
As for RFC 8321 [RFC8321], it is possible to classify the traffic and mark a portion of the total traffic. 🍓 For each period, the packet rate and bandwidth are calculated from the number of packets. That way the network orchestrator becomes aware if the traffic rate surpasses limits. 🚨 Also, more precision can be obtained by reducing the marking period; indeed, some implementations use a marking period of 1 sec or less. ⚡
On top of that, an SDN controller could also collect the measurement history. 📚
It's important to mention that the Multipoint Alternate Marking framework also helps Traffic Visualization. 🎨 Honestly, this methodology is so useful for identifying which path or cluster is crossed by the flow. 👀
10. Examples of Application 🌸
There are application fields where it may be useful to take the Multipoint Alternate Marking into consideration:
VPN: 💼 The IP traffic is selected on an IP-source basis in both directions. At the endpoint WAN interface, all the output traffic is counted in a single flow. The input traffic is composed of all the other flows aggregated by source address. So, by considering n endpoints, the monitored flows are n (each flow with 1 ingress point and (n-1) egress points) instead of n*(n-1) flows (each flow with 1 ingress point and 1 egress point). 🤯
Mobile Backhaul: 📱 LTE traffic is selected, in the Up direction, by the EnodeB source address and, in the Down direction, by the EnodeB destination address, because the packets are sent from the Mobile Packet Core to the EnodeB. So the monitored flow is only one per EnodeB in both directions. 🙌
Over The Top (OTT) services: 🎬 The traffic is selected, in the Down direction, by the source addresses of the packets sent by OTT servers. In the opposite direction (Up), it is selected by the destination IP addresses of the same servers. So the monitoring is based on a single flow per OTT server in both directions. 💫
Enterprise SD-WAN: 🏢 SD-WAN lets you connect remote branch offices to data centers and build higher-performance WANs. 🚀 A centralized controller is used to set policies and prioritize traffic. The SD-WAN takes those policies and the availability of network bandwidth into account when routing traffic. 🛣️ This helps ensure that application performance meets Service Level Agreements (SLAs). 📈 This methodology can also help with path selection for the WAN connection based on per-cluster and per-flow performance. 💅
Note that the preceding list is just an example and is not exhaustive. More applications are possible. 🌈
11. Security Considerations 🔐
This document specifies a method of performing measurements that does not directly affect Internet security or applications that run on the Internet. 🛡️ However, implementation of this method must be mindful of security and privacy concerns, as explained in RFC 8321 [RFC8321]. 😤 Please take this seriously, bestie. 🫶
12. IANA Considerations 📋
This document has no IANA actions. 🙅♀️
13. References 📚
13.1. Normative References
[RFC5474] Duffield, N., Ed., Chiou, D., Claise, B., Greenberg, A.,
Grossglauser, M., and J. Rexford, "A Framework for Packet
Selection and Reporting", RFC 5474, DOI 10.17487/RFC5474,
March 2009, <https://www.rfc-editor.org/info/rfc5474>.
[RFC5475] Zseby, T., Molina, M., Duffield, N., Niccolini, S., and F.
Raspall, "Sampling and Filtering Techniques for IP Packet
Selection", RFC 5475, DOI 10.17487/RFC5475, March 2009,
<https://www.rfc-editor.org/info/rfc5475>.
[RFC5644] Stephan, E., Liang, L., and A. Morton, "IP Performance
Metrics (IPPM): Spatial and Multicast", RFC 5644,
DOI 10.17487/RFC5644, October 2009,
<https://www.rfc-editor.org/info/rfc5644>.
[RFC8321] Fioccola, G., Ed., Capello, A., Cociglio, M., Castaldelli,
L., Chen, M., Zheng, L., Mirsky, G., and T. Mizrahi,
"Alternate-Marking Method for Passive and Hybrid
Performance Monitoring", RFC 8321, DOI 10.17487/RFC8321,
January 2018, <https://www.rfc-editor.org/info/rfc8321>.13.2. Informative References
[ALTERNATE-MARKING]
Mizrahi, T., Arad, C., Fioccola, G., Cociglio, M., Chen,
M., Zheng, L., and G. Mirsky, "Compact Alternate Marking
Methods for Passive and Hybrid Performance Monitoring",
Work in Progress, Internet-Draft, draft-mizrahi-ippm-
compact-alternate-marking-05, 6 July 2019,
<https://tools.ietf.org/html/draft-mizrahi-ippm-compact-
alternate-marking-05>.
[ENHANCED-ALTERNATE-MARKING]
Zhou, T., Fioccola, G., Lee, S., Cociglio, M., and W. Li,
"Enhanced Alternate Marking Method", Work in Progress,
Internet-Draft, draft-zhou-ippm-enhanced-alternate-
marking-05, 13 July 2020, <https://tools.ietf.org/html/
draft-zhou-ippm-enhanced-alternate-marking-05>.
[IEEE-ACM-ToN-MPNPM]
Cociglio, M., Fioccola, G., Marchetto, G., Sapio, A., and
R. Sisto, "Multipoint Passive Monitoring in Packet
Networks", IEEE/ACM Transactions on Networking vol. 27,
no. 6, pp. 2377-2390, DOI 10.1109/TNET.2019.2950157,
December 2019,
<https://doi.org/10.1109/TNET.2019.2950157>.
[IEEE-Network-PNPM]
Mizrahi, T., Navon, G., Fioccola, G., Cociglio, M., Chen,
M., and G. Mirsky, "AM-PM: Efficient Network Telemetry
using Alternate Marking", IEEE Network vol. 33, no. 4,
pp. 155-161, DOI 10.1109/MNET.2019.1800152, July 2019,
<https://doi.org/10.1109/MNET.2019.1800152>.
[IFIT-FRAMEWORK]
Song, H., Qin, F., Chen, H., Jin, J., and J. Shin, "In-
situ Flow Information Telemetry", Work in Progress,
Internet-Draft, draft-song-opsawg-ifit-framework-12, 14
April 2020, <https://tools.ietf.org/html/draft-song-
opsawg-ifit-framework-12>.
[RFC7011] Claise, B., Ed., Trammell, B., Ed., and P. Aitken,
"Specification of the IP Flow Information Export (IPFIX)
Protocol for the Exchange of Flow Information", STD 77,
RFC 7011, DOI 10.17487/RFC7011, September 2013,
<https://www.rfc-editor.org/info/rfc7011>.
[ROUTE-ASSESSMENT]
Alvarez-Hamelin, J., Morton, A., Fabini, J., Pignataro,
C., and R. Geib, "Advanced Unidirectional Route Assessment
(AURA)", Work in Progress, Internet-Draft, draft-ietf-
ippm-route-10, 13 August 2020,
<https://tools.ietf.org/html/draft-ietf-ippm-route-10>.Acknowledgements 🫶
The authors would like to thank Al Morton, Tal Mizrahi, and Rachel Huang for the precious contributions. 💖
Authors' Addresses 💌
Giuseppe Fioccola (editor) Huawei Technologies Riesstrasse, 25 80992 Munich Germany
Email: giuseppe.fioccola@huawei.com
Mauro Cociglio Telecom Italia Via Reiss Romoli, 274 10148 Torino Italy
Email: mauro.cociglio@telecomitalia.it
Amedeo Sapio Intel Corporation 4750 Patrick Henry Dr. Santa Clara, CA 95054 United States of America
Email: amedeo.sapio@intel.com
Riccardo Sisto Politecnico di Torino Corso Duca degli Abruzzi, 24 10129 Torino Italy
Email: riccardo.sisto@polito.it
📦 where this page comes from
- ietf: rfc8889 2020-08
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms