RFC 8321

Alternate-Marking Method for Passive and Hybrid Performance Monitoring

📅 published January 2018, from ietf

RFC 8321: Alternate-Marking Method for Passive and Hybrid Performance Monitoring ✨

okay so this one is an Experimental RFC from January 2018 (ISSN 2070-1721) 🧪 It doesn't obsolete or update any other RFC, so it's a standalone doc. The editor is G. Fioccola (Ed.) 👑, and the co-authors are A. Capello, M. Cociglio, and L. Castaldelli from Telecom Italia, M. Chen and L. Zheng from Huawei Technologies, G. Mirsky from ZTE, and T. Mizrahi from Marvell 🫶

Abstract

so the abstract basically says this doc describes a way to measure packet loss, delay, and jitter on live traffic 📡 The method is built on an Alternate-Marking (coloring) technique 🎨 There's a report in here that walks through an example and shows where the method can be used. The whole thing can be used in lots of situations, and depending on the application it could count as Passive or Hybrid 🦋

Status of This Memo

ngl this section is very serious 🙈 The document is not an Internet Standards Track spec. It's published for examination, experimental implementation, and evaluation 🧪

It does define an Experimental Protocol for the Internet community 💜 It's a product of the Internet Engineering Task Force (IETF), it represents the consensus of the IETF community, and it got public review and was approved for publication by the Internet Engineering Steering Group (IESG). Not every IESG-approved doc is a candidate for any level of Internet Standard, so see Section 2 of RFC 7841 for that 💅

For info on the current status, any errata, and how to give feedback, check https://www.rfc-editor.org/info/rfc8321 🌸

Copyright (c) 2018 IETF Trust and the persons identified as the document authors. All rights reserved 🎀

This doc is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they were in effect on the publication date. Please read these carefully, bestie, because they describe your rights and restrictions for this document 📜 Code Components pulled out of this doc must include Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come without warranty as described in the Simplified BSD License 🔥

Table of Contents

1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
  1.1.  Requirements Language . . . . . . . . . . . . . . . . . .   5
2.  Overview of the Method  . . . . . . . . . . . . . . . . . . .   5
3.  Detailed Description of the Method  . . . . . . . . . . . . .   6
  3.1.  Packet Loss Measurement . . . . . . . . . . . . . . . . .   6
    3.1.1.  Coloring the Packets  . . . . . . . . . . . . . . . .  11
    3.1.2.  Counting the Packets  . . . . . . . . . . . . . . . .  12
    3.1.3.  Collecting Data and Calculating Packet Loss . . . . .  13
  3.2.  Timing Aspects  . . . . . . . . . . . . . . . . . . . . .  13
  3.3.  One-Way Delay Measurement . . . . . . . . . . . . . . . .  15
    3.3.1.  Single-Marking Methodology  . . . . . . . . . . . . .  15
    3.3.2.  Double-Marking Methodology  . . . . . . . . . . . . .  17
  3.4.  Delay Variation Measurement . . . . . . . . . . . . . . .  18
4.  Considerations  . . . . . . . . . . . . . . . . . . . . . . .  18
  4.1.  Synchronization . . . . . . . . . . . . . . . . . . . . .  19
  4.2.  Data Correlation  . . . . . . . . . . . . . . . . . . . .  19
  4.3.  Packet Reordering . . . . . . . . . . . . . . . . . . . .  20
5.  Applications, Implementation, and Deployment  . . . . . . . .  21
  5.1.  Report on the Operational Experiment  . . . . . . . . . .  22
    5.1.1.  Metric Transparency . . . . . . . . . . . . . . . . .  24
6.  Hybrid Measurement  . . . . . . . . . . . . . . . . . . . . .  24
7.  Compliance with Guidelines from RFC 6390  . . . . . . . . . .  25
8.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  27
9.  Security Considerations . . . . . . . . . . . . . . . . . . .  27
10. References  . . . . . . . . . . . . . . . . . . . . . . . . .  28
  10.1.  Normative References . . . . . . . . . . . . . . . . . .  28
  10.2.  Informative References . . . . . . . . . . . . . . . . .  29
Acknowledgements  . . . . . . . . . . . . . . . . . . . . . . . .  32
Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  32

1. Introduction ✨

okay so nowadays most Service Providers' networks carry traffic with content that's super sensitive to packet loss [RFC7680], delay [RFC7679], and jitter [RFC3393] 😭

so Service Providers need methods and tools to monitor and measure network performance with decent accuracy 🎯 That's how they keep control of the quality of experience their customers actually feel. Performance monitoring also gives useful info for better network management, like isolating network problems and troubleshooting, etc. 🔧

A lot of work has been done on Operations, Administration, and Maintenance (OAM), which includes performance monitoring techniques, by Standards Developing Organizations (SDOs). [RFC7276] is a great overview of the existing OAM mechanisms defined in the IETF, ITU-T, and IEEE 📚 In the IETF, tons of work has gone into fault detection and connectivity verification, while only a minor effort has been put into performance monitoring so far 🥲 The IPPM WG has defined standard metrics for measuring network performance, but the methods developed in that WG mainly focus on Active measurement techniques. More recently, the MPLS WG defined mechanisms for measuring packet loss, one-way and two-way delay, and delay variation in MPLS networks [RFC6374], but their applicability to Passive measurements has some limitations, especially for pure connection-less networks 💀

so the lack of good tools to measure packet loss with the accuracy people want pushed the team to design a new method for performance monitoring of live traffic, one that's easy to implement and deploy 🦋 That's the method in this document. Basically, it's a Passive performance monitoring technique that's potentially applicable to any kind of packet-based traffic, including Ethernet, IP, and MPLS, both unicast and multicast 🌈 It focuses mainly on packet loss measurement, but it can easily be extended to one-way or two-way delay and delay variation measurements too 💖

The method was explicitly designed for Passive measurements, but it can also be used with Active probes. Passive measurements are usually easier for customers to understand, and they give much better accuracy, especially for packet loss measurements 🫶

RFC 7799 [RFC7799] defines Passive and Hybrid Methods of Measurement. Passive Methods are based solely on observations of an undisturbed and unmodified packet stream of interest. Hybrid Methods combine Active Methods and Passive Methods 🧚

so given those definitions, the Alternate-Marking Method could count as Hybrid or Passive, depending on the case 🎀 If the marking is done by changing existing field values in the packets (like the Differentiated Services Code Point (DSCP) field), it's Hybrid. If the marking field is dedicated, reserved, and included in the protocol specification, the Alternate-Marking technique can be considered Passive. Examples are the Synonymous Flow Label described in [SFL-FRAMEWORK] or the OAM Marking Bits described in [PM-MM-BIER] 🌸

The advantages of this method are:

  • easy implementation: it can be implemented with features already available on major routing platforms, as described in Section 5.1, or with an optimized implementation of the method for both legacy and newest technologies 🛠️
  • low computational effort: the extra processing load is negligible 🍓
  • accurate packet loss measurement: single packet loss granularity is achieved with a Passive measurement 🎯
  • potential applicability to any kind of packet-based or frame-based traffic: Ethernet, IP, MPLS, etc., and both unicast and multicast 🌍
  • robustness: the method can tolerate out-of-order packets, and it's not based on "special" packets whose loss could have a negative impact 💪
  • flexibility: all the timestamp formats are allowed, because they are managed out of band. The format, either the Network Time Protocol (NTP) [RFC5905] or the IEEE 1588 Precision Time Protocol (PTP) [IEEE-1588], depends on the precision you want 🕰️
  • no interoperability issues: the features needed to experiment with and test the method (as described in Section 5.1) are available on all current routing platforms. Both a centralized and a distributed solution can be used to harvest data from the routers 🫧

The method doesn't raise any specific need for protocol extension, but it could be improved further with some extensions to existing protocols 👑 The thing is, using Diffserv bits for coloring the packets might not be a viable solution in some cases, so a standard way to color packets for this specific application could be really beneficial 🔥

1.1. Requirements Language 💅

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here. ✨

2. Overview of the Method 🦋

okay so there are a bunch of ways to measure packet loss on a production traffic flow 💅 The most intuitive one is numbering the packets so every router that gets the flow can instantly spot a missing one. ngl it sounds simple but it's really not 😭 You have to stick a sequence number into each packet, and the devices have to pull it out and check it in real time. On live traffic that's a whole thing. If UDP is the transport, there's no sequence number to use at all 💀 And if you grab a higher-layer one (like the one in the RTP header), digging it out of every single packet and processing it live could totally overload the device 🫠

Another approach is to count the packets sent on one end, count the packets received on the other end, and compare the two numbers. This is way simpler to implement ✨ BUT the devices doing the measuring have to be in sync. To compare two counters, they have to refer to exactly the same set of packets. The flow is continuous and can't be paused when you want to read a counter, so figuring out exactly when to read it is tricky 🙈 One fix is to virtually chop the flow into consecutive blocks by periodically inserting a delimiter, so each counter covers the same block. The delimiter could be, for example, a special packet inserted artificially into the flow. Here's the catch though: delimiting with specific packets has some limits. It means generating extra packets, and the gear has to be able to process them. The method is also vulnerable to delimiting packets arriving out of order and, to a lesser extent, to their loss 👀

The method in this document follows the second approach, but it doesn't use extra packets to chop the flow into blocks. Instead, it "marks" the packets so that packets in the same block share the same color, while consecutive blocks get different colors 🌈 Each color change acts like a sort of auto-synchronization signal, which keeps the measurements taken by different devices along the path consistent with each other (see also [IP-MULTICAST-PM] and [OPSAWG-P3M], where this technique was introduced) 🫶

Figure 1 shows a very simple network and how the method can measure packet loss on different network segments. By turning on the measurement on several interfaces along the path, you can do link monitoring, node monitoring, or end-to-end monitoring 🔥 The method is flexible enough to measure packet loss on any segment of the network, so it can be used to isolate the faulty element 🎀

                               Traffic Flow
        ========================================================>
          +------+       +------+       +------+       +------+
      ---<>  R1  <>-----<>  R2  <>-----<>  R3  <>-----<>  R4  <>---
          +------+       +------+       +------+       +------+
          .              .      .              .       .      .
          .              .      .              .       .      .
          .              <------>              <------->      .
          .          Node Packet Loss      Link Packet Loss   .
          .                                                   .
          <--------------------------------------------------->
                           End-to-End Packet Loss

                     Figure 1: Available Measurements

3. Detailed Description of the Method ✨

okay this is the part where we get into the nitty-gritty 🤓 This section explains, in detail, how the method actually operates. The main focus is on measuring packet loss, since that's the core application of the method 💜 But the method's use for delay and jitter measurements is also considered, so we'll keep that in mind too 🧚

3.1. Packet Loss Measurement ✨

Okay so the big idea 💖 is that we chop traffic flows into consecutive blocks. Each block is something every network device on the path can recognize without any confusion 🦋. You count the packets in each block, compare the numbers different devices measured, and boom, you can see how many packets went missing in any single block between any two points.

Like we said earlier, the easiest way to make blocks is to "color" the traffic (two colors is enough, no cap) 🎀. Packets in different consecutive blocks get different colors. When the color flips, the old block ends and the new one starts. So every packet in a block shares the same color, and neighboring blocks never share one. How many packets end up in each block depends on what triggers the flip:

  • if the color switches after a fixed number of packets, then each block will contain the same number of packets (except for any losses); and
  • if the color switches according to a fixed timer, then the number of packets may be different in each block depending on the packet rate.

Here's what a flow looks like once it's split into colored blocks 🌈

A: packet with A coloring
B: packet with B coloring

         |           |           |           |           |
         |           |    Traffic Flow       |           |
 ------------------------------------------------------------------->
  BBBBBBB AAAAAAAAAAA BBBBBBBBBBB AAAAAAAAAAA BBBBBBBBBBB AAAAAAA
 ------------------------------------------------------------------->
    ...  |  Block 5  |  Block 4  |  Block 3  |  Block 2  |  Block 1
         |           |           |           |           |

Figure 2: Traffic Coloring

Figure 3 shows how this trick can measure link packet loss between two adjacent nodes 👑. Let's say we want to watch packet loss on the link between router R1 and router R2. The traffic gets colored alternately with two colors, A and B. Every time the color changes, the transition makes a sort of square-wave signal, like this:

Color A   ----------+           +-----------+           +----------
                    |           |           |           |
Color B             +-----------+           +-----------+
           Block n        ...      Block 3     Block 2     Block 1
         <---------> <---------> <---------> <---------> <--------->

                             Traffic Flow
         ===========================================================>
Color   ...AAAAAAAAAAA BBBBBBBBBBB AAAAAAAAAAA BBBBBBBBBBB AAAAAAA...
         ===========================================================>

Figure 3: Computation of Link Packet Loss

Traffic coloring can be done by R1 itself if the traffic isn't already colored 🧚. R1 needs two counters on its egress interface, C(A)R1 and C(B)R1. C(A)R1 counts the packets colored A, and C(B)R1 counts the ones colored B. While traffic is colored A, only C(A)R1 goes up and C(B)R1 stays put. When traffic is colored B, only C(B)R1 goes up. C(A)R1 and C(B)R1 can be used as reference values to figure out the packet loss from R1 to any other measurement point down the path. Router R2 similarly needs two counters on its ingress interface, C(A)R2 and C(B)R2, to count the packets it receives on that interface that are colored A and B. When an A block ends, you can compare C(A)R1 and C(A)R2 and calculate the packet loss within that block. When the next B block ends, you compare C(B)R1 with C(B)R2, and so on for every block after that 🫶

Likewise, by using two counters on the R2 egress interface, you can count the packets sent out of the R2 interface and use them as reference values to calculate the packet loss from R2 to any measurement point down R2 💅

Using a fixed timer for color switching gives you better control over the method 👑. The (time) length of the blocks can be picked big enough to make collecting and comparing measurements from different devices way easier. It's preferable not to read the counters right after the color switch, because some packets could arrive out of order and bump the counter for the previous block (color). So it's worth waiting a bit. A safe choice is to wait L/2 time units after the color switch (where L is the duration of each block) before reading the still counter of the previous color. That way, the chance of reading a running counter instead of a still one is minimized. The catch is that the longer the block lasts, the less often you can take a measurement 😭

The following table shows how the counters can be used to calculate packet loss between R1 and R2 🍓. The first column lists the sequence of traffic blocks, and the other columns hold the counters of A-colored and B-colored packets for R1 and R2. In this example, we assume the counter values get reset to zero whenever a block ends and its counter has been read. With that assumption, the table only shows relative values, which is the exact number of packets of each color within each block. If the counters weren't reset, the table would show cumulative values instead, but you could still work out the relative values just by taking the difference from the previous block of the same color.

The color is switched on the basis of a fixed timer (not shown in the table), so the number of packets in each block is different.

        +-------+--------+--------+--------+--------+------+
        | Block | C(A)R1 | C(B)R1 | C(A)R2 | C(B)R2 | Loss |
        +-------+--------+--------+--------+--------+------+
        | 1     | 375    | 0      | 375    | 0      | 0    |
        | 2     | 0      | 388    | 0      | 388    | 0    |
        | 3     | 382    | 0      | 381    | 0      | 1    |
        | 4     | 0      | 377    | 0      | 374    | 3    |
        | ...   | ...    | ...    | ...    | ...    | ...  |
        | 2n    | 0      | 387    | 0      | 387    | 0    |
        | 2n+1  | 379    | 0      | 377    | 0      | 2    |
        +-------+--------+--------+--------+--------+------+

Table 1: Evaluation of Counters for Packet Loss Measurements

During an A block (blocks 1, 3, and 2n+1), all the packets are A-colored. That means the C(A) counters go up to the number seen on the interface, while the C(B) counters stay at zero. On the flip side, during a B block (blocks 2, 4, and 2n), all the packets are B-colored: the C(A) counters are zero, and the C(B) counters go up.

When a block ends (because of the color switch), the relative counters stop incrementing. At that point you can read them, compare the values measured on routers R1 and R2, and calculate the packet loss within that block 🔥

Let's look at the table above. During the first block (A-colored), C(A)R1 and C(A)R2 both have the value 375. That matches the exact number of packets in the first block, so no loss. During the second block (B-colored), the R1 and R2 counters both have the value 388, which matches the number of packets in the second block, so again no loss. During the third and fourth blocks, the R1 and R2 counters are different, which means some packets got lost. In the example, one single packet (382-381) was lost during block three, and three packets (377-374) were lost during block four 💀

The method applied to R1 and R2 can be extended to any other router and used on more complex networks, as long as measurement is enabled on the path the observed traffic flow(s) follow 🌸

It's worth mentioning two different strategies you can use when implementing the method:

  • flow-based: the flow-based strategy is used when only a limited number of traffic flows need to be monitored. According to this strategy, only a subset of the flows is colored. Counters for packet loss measurements can be set up for each single flow, or for the set as a whole, depending on the granularity you want. A relevant problem with this approach is that you have to know in advance the path followed by the flows that are subject to measurement. Path rerouting and traffic load-balancing make the issue more complex, especially for unicast traffic. The problem is easier to solve for multicast traffic, where load-balancing is seldom used and static joins are frequently used to force traffic forwarding and replication.
  • link-based: measurements are performed on all the traffic on a link-by-link basis. The link could be a physical link or a logical link. Counters could be set up for the traffic as a whole or for each traffic class (in case you want to monitor each class separately), but in the second case, a couple of counters are needed for each class.

As mentioned, flow-based measurement requires identifying the flow to be monitored and discovering the path the selected flow follows 🦋. You can monitor a single flow or multiple flows grouped together, but in the grouped case, measurement is consistent only if all the flows in the group follow the same path. Also, if you measure by grouping many flows, you can't determine exactly which flow was affected by packet loss. To get measures per single flow, you have to configure counters for each specific flow. Once the flow(s) to be monitored are identified, the monitoring has to be configured on the proper nodes. Configuring the monitoring means setting up the rule to intercept the traffic and setting up the counters to count the packets.

To get just end-to-end monitoring, it's enough to enable monitoring on the first- and last-hop routers of the path. The mechanism is completely transparent to intermediate nodes and independent from the path the traffic flows follow 🧸 On the contrary, to monitor the flow hop by hop along its whole path, you have to enable monitoring on every node from the source to the destination. If the exact path a flow follows isn't known a priori (i.e., the flow has multiple paths to reach the destination), you have to enable the monitoring system on every path. Counters on interfaces the flow traverses will report a packet count, whereas counters on other interfaces will be null 🌈

3.1.1. Coloring the Packets

okay so coloring is the foundation of making packet blocks 🎨 it's giving "choose where to turn it on, then choose how to color the packets" ✨

for flow-based measurements, the flow you want to watch can be picked out with a set of selection rules (like header fields) that match only some of the packets 🎀 that way you control how many nodes get involved, which path the packets take, and how big the flows are 💅 you can have multiple coloring nodes, or just one that's way easier to manage and can't cause any conflicts 🫶 coloring in multiple nodes is totally allowed, but the coloring must change periodically between the nodes, following the timing stuff in Section 3.2 ⏰ so every node that's set up as a measurement point along the path should be able to tell the colored packets apart without any confusion 🔍 and [MULTIPOINT-ALT-MM] extends all of this to multipoint-to-multipoint flows 🦋 it can also be really useful to color the flow as close to the source as possible, because then you get an end-to-end measurement if there's also a measurement point on the last-hop router 😭👑

for link-based measurements, all the traffic has to be colored when it gets sent over the link 🔥 if it was already colored, it has to be re-colored, because the color needs to stay consistent on that link 💜 so every hop along the path must (re-)color the traffic 🌸 the color doesn't have to stay consistent across different links, which is lowkey a relief 🙈

coloring can be done by setting one specific bit in the packet header and flipping its value periodically ✨ how you pick the marking field depends on the application, and that's out of scope here 💖 but Section 5 does walk through some applications, so go check it out 🧚

3.1.2. Counting the Packets

for flow-based measurements, assuming only the source nodes do the coloring, the nodes between the source and the destination (destination included) have to count the colored packets they receive and forward 📊 you can turn this on at every router along the path, or only on some of them, depending on which part of the network you're watching (a single link, a certain metro area, the backbone, or the whole path) 🌈 since the color switches back and forth between two values, you need two counters, one for each value: one counter for packets with color A and one for packets with color B 💜 for each flow (or group of flows) you're monitoring, and for every interface where monitoring is Active, you need a couple of counters 🍓 for example, if you want to monitor three flows separately on a router with four interfaces involved, you need 24 counters (two counters for each of the three flows on each of the four interfaces) 🤯 and [MULTIPOINT-ALT-MM] extends the counting to multipoint-to-multipoint flows too 🦋

for link-based measurements it's basically the same vibe, except the coloring and counting happen link by link, at each endpoint of the link 🔗

now, timing your reads is a whole thing ⏱️ to count the exact number of packets in a block, the routers have to do the read once that block has ended 🛑 in other words, the counter for color A has to be read while the current block is color B, so you know the value is stable 🫶 there are two ways to do this. the general approach is to read the counters periodically, many times during a block's duration, and compare those readings with each other 👀 when the counter stops going up, that means the current block has ended, and you can safely work with its value ✅ or, if the coloring is done on a fixed timer, you can just set up the reads to line up with that timer 💅 for example, reading the color A counter once every period, in the middle of the following color B block, is a safe choice 🌷 you should also leave a sufficient margin between the end of a block and the read, so out-of-order packets get accounted for 🙌

3.1.3. Collecting Data and Calculating Packet Loss

okay here's the tea ☕ the nodes that are set up to do performance monitoring collect the counter values, but they can't directly use that info to measure packet loss, because they only have their own samples 🙃 so an external Network Management System (NMS) can be used to collect and process the data and do the packet loss calculation 📈 the NMS compares counter values from different nodes, and it can figure out whether any packets were lost (even just one!) and exactly where they went missing 🕵️‍♀️💀

the counter values need to be sent to the NMS as soon as they've been read 🚀 this can be done using SNMP or FTP, and it can work in either Push Mode or Polling Mode 💌 in Push Mode, each router periodically sends its info to the NMS 📤 in Polling Mode, it's the NMS that periodically polls the routers to gather the info 📥 either way, the NMS has to collect all the relevant values from all the routers within one cycle of the timer ⏰ no stragglers allowed 😤

it would also be possible to use a protocol that exchanges counter values between the two endpoints, so they can each do the packet loss calculation for each traffic direction 🔁✨

a possible approach for the performance measurement (PM) architecture is explained in [COLORING] 🧩 and [IP-FLOW-REPORT] introduces new information elements for IP Flow Information Export (IPFIX) [RFC7011] 🎀

3.2. Timing Aspects

okay this section is where the timing magic happens ⏳✨ this document introduces two color-switching methods: one based on a fixed number of packets, and one based on a fixed timer 🎀 but the fixed-timer method is the one to pick because it's more deterministic 💜 so that's the one the rest of the document sticks with 💅

clocks in network devices aren't accurate in general, so there's a clock error between the measurement points R1 and R2 🙈 but to make the methodology work, they must be synchronized to the same clock reference with an accuracy of +/- L/2 time units, where L is the fixed time duration of the block 🫶 that's how each router can assign every colored packet to the right batch 📦 the reason it works is that the minimum time distance between two packets of the same color that belong to different batches is L time units 🌈

in practice, clock errors aren't the only issue: the delay between measurement points matters too 🐢 each packet can be delayed differently, which can cause packets to arrive out of order at batch boundaries 😵 so, without considering clock error, we wait L/2 after the color switches to make sure we're reading a still counter ✋

in summary, there are two things we have to account for: the clock error between network devices, and the interval we need to wait so packets don't end up out of order because of network delay 💖

the following figure explains both issues 🖼️

   ...BBBBBBBBB | AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA | BBBBBBBBB...
                |<======================================>|
                |                   L                    |
   ...=========>|<==================><==================>|<==========...
                |       L/2                   L/2        |
                |<===>|                            |<===>|
                   d  |                            |   d
                      |<==========================>|
                       available counting interval

                         Figure 4: Timing Aspects

it is assumed that all network devices are synchronized to a common reference time with an accuracy of +/- A/2 🕰️ so the difference between the clock values of any two network devices is bounded by A 🫶

the guard band d is given by:

d = A + D_max - D_min,

where A is the clock accuracy, D_max is an upper bound on the network delay between the network devices, and D_min is a lower bound on the delay 🛡️

the available counting interval is L - 2d, which must be > 0 🌸

and the condition that must be satisfied, which is a requirement on the synchronization accuracy, is:

d < L/2

✨ the vibe check is: keep the guard band small enough, and you're golden 💅

3.3. One-Way Delay Measurement

the same principle used to measure packet loss can also be applied to one-way delay measurement 🚀 there are three alternatives, and we'll go through them in the next sections 🦋

quick note, though: for all the one-way delay alternatives in those next sections, if you sum the one-way delays of the two directions of a path, you can always measure the two-way delay (round-trip "virtual" delay) 🔁💜 it's literally math magic bestie 🧙‍♀️✨

3.3.1. Single-Marking Methodology 🦋

okay so the color flip can be your time reference for delay 🕰️✨ whenever the color changes (aka a new block just started), a network device can save the timestamp of the first packet in that new block. then you compare that timestamp with the same packet's timestamp on a second router and boom, packet delay 💅 in the example in Figure 2, R1 saves TS(A1)R1 when it sends block 1's first packet (A-colored), TS(B2)R1 when it sends block 2's first packet (B-colored), and so on for every block. R2 does the exact same thing on the receiving side and records TS(A1)R2, TS(B2)R2, etc. 🎀 because each timestamp belongs to one specific packet (the first packet of each block), you can be SURE the timestamps you compare are about the same packets 😤 so comparing TS(A1)R1 with TS(A1)R2 (and TS(B2)R1 with TS(B2)R2, and so on) gives you the delay between R1 and R2. want more measurements? just save more timestamps for other packets inside each block 🧚

okay but real talk, to compare timestamps from different routers coherently, the clocks on the network nodes have to be in sync ⏰🫶 also, a measurement only counts if no packets get lost AND packet misordering can be avoided. otherwise the first packet of a block on R1 might not be the first packet of that same block on R2 🙈 (like if that packet got lost between R1 and R2, or it shows up after the next one). yikes 💀

here's the table showing how timestamps turn into a delay between R1 and R2 🍓 the first column is the block sequence, and the other columns hold the timestamp of each block's first packet on R1 and R2. the delay is just the difference between timestamps. to keep things simple, every value is in milliseconds 💖

+-------+---------+---------+---------+---------+-------------+
| Block | TS(A)R1 | TS(B)R1 | TS(A)R2 | TS(B)R2 | Delay R1-R2 |
+-------+---------+---------+---------+---------+-------------+
| 1     | 12.483  | -       | 15.591  | -       | 3.108       |
| 2     | -       | 6.263   | -       | 9.288   | 3.025       |
| 3     | 27.556  | -       | 30.512  | -       | 2.956       |
|       | -       | 18.113  | -       | 21.269  | 3.156       |
| ...   | ...     | ...     | ...     | ...     | ...         |
| 2n    | 77.463  | -       | 80.501  | -       | 3.038       |
| 2n+1  | -       | 24.333  | -       | 27.433  | 3.100       |
+-------+---------+---------+---------+---------+-------------+

   Table 2: Evaluation of Timestamps for Delay Measurements

row 1 has the timestamps from R1 and R2 for block 1's first packet (A-colored), and the delay is the R2 timestamp minus the R1 timestamp ✨ row 2 does the same for block 2's first packet (B-colored) 🌈 by comparing timestamps from different nodes that refer to the SAME packets (identified by the color alternation), you can measure delay on different network segments. slay 👑

to keep the example simple, there's just one measurement per block, using only the first packet. you can get more by looking at more packets in the block. for example, you could take a timestamp every N packets, which gives you multiple delay measurements 💜 if you take it to the extreme, you could measure the delay of every single packet by taking and comparing each one's timestamp. technically possible, but impractical from an implementation point of view 😭

3.3.1.1. Mean Delay 🍰

remember how the method we just covered is sensitive to packets arriving out of order? 🙃 to fix that, we use a different approach based on the mean delay. the mean delay is the average arrival time of the packets within a single block. each network device locally stores a timestamp for every packet it receives in a block, then adds up all the timestamps and divides by the total number of packets received. boom, you've got the average arrival time for that block 🧁 subtract the average arrival times of two adjacent devices and you get the mean delay between those nodes ✨ when you compute the mean delay, the measurement error can pile up because you're accumulating the error from a lot of packets, but this method is robust to out-of-order packets AND packet loss (only a small error gets introduced) 🫶 it also hugely cuts down the number of timestamps the management system has to collect, since each network device only needs one per block 🗂️

the catch? it only gives you ONE measurement for the whole duration of the block (say, 5 minutes) 😤 and it doesn't tell you the minimum, maximum, or median delay values [RFC6703]. you can work around that by making blocks shorter (like from 5 minutes down to a few seconds), but that needs a highly optimized implementation of the method 🔥

3.3.2. Double-Marking Methodology 💅

the Single-Marking methodology for one-way delay measurement is sensitive to out-of-order reception of packets 🌀 the first fix we talked about was mean delay. the problem with mean delay is that it doesn't tell you anything about how delay values are distributed across the block. it would also be nice to have the minimum, maximum, and median delay values and, more broadly, to understand the statistical distribution of delay values 📊 so to get more info about delay AND handle out-of-order issues, we can use a different approach: the Double-Marking methodology 🦋

the basic idea is that the first marking creates the alternate flow, and inside that colored flow, a second marking selects the packets used for measuring delay/jitter ✨ the first marking is needed for packet loss and mean delay measurement. the second marking creates a new set of marked packets that are fully identified across the network, so a network device can store their timestamps. those timestamps get compared with the timestamps of the same packets on a second router to compute the delay of each packet 🎀 you can get more measurements just by changing how often the second marking happens. BUT the second marking frequency must not be too high, or you'll run into out-of-order issues 😬 there should also be a security time gap between packets with the second marking (e.g., at the very least, this gap could be the mean network delay calculated with the previous methodology). that gap helps avoid out-of-order issues and gives you a number of measurement packets that is rate independent 🧚 and if a second-marking packet gets lost, the delay measurement for that block is corrupted and should be discarded 💀

mean delay is calculated over ALL the packets in a sample, and it's a simple computation for the Single-Marking Method 🍓 sometimes the mean alone isn't enough to describe the sample, and you need more delay statistics, like percentiles, variance, and median delay values 📈 the conventional range (maximum minus minimum) should be avoided for several reasons, including that the maximum is unstable because it's heavily influenced by outliers 🙈 RFC 5481 [RFC5481], Section 6.5 points out that the 99.9th percentile of delay and delay variation is more useful to performance planners 👑 to fix this, the idea is to pair the mean delay for the whole batch with the Double-Marking Method, where a subset of the batch packets is picked for detailed delay calculation using a second marking. that way you can do a detailed analysis on those double-marked packets 🌸 also, there are classic algorithms for calculating median and variance, but they're out of scope for this document 💜 comparing the mean delay for the whole batch with the mean delay of those double-marked packets is super useful, because it tells you whether the Double-Marking measurements actually represent the delay trends 🔍✨

3.4. Delay Variation Measurement 🌈

just like one-way delay measurement (for both Single Marking and Double Marking), this method can also measure inter-arrival jitter. we use the definition from RFC 3393 [RFC3393] 🫶 for a Single-Marking Method, the color alternation can be the time reference for measuring delay variations. for Double Marking, the time reference is the second-marked packets 🎀 looking at the example in Figure 2, R1 stores the timestamp TS(A)R1 whenever it sends the first packet of a block, and R2 stores the timestamp TS(B)R2 whenever it receives the first packet of a block 🦋 the inter-arrival jitter can be easily worked out from one-way delay measurement by evaluating the delay variation between consecutive samples ✨

the concept of mean delay also works for delay variation: you evaluate the average variation of the interval between consecutive packets of the flow from R1 to R2 💖

4. Considerations 🧐

this section covers some considerations about the methodology 💅

4.1. Synchronization ⏰

the Alternate-Marking technique doesn't need strong synchronization, especially for packet loss and two-way delay measurement 🫶 only one-way delay measurement requires network devices to have synchronized clocks 🙌

color switching is the reference for ALL the network devices, and the only requirement is that every network device recognizes the right batch along the path 🌸

if the measurement period is L time units long, then all network devices must be synchronized to the same clock reference with an accuracy of +/- L/2 time units (not counting network delay) 😮‍💨 this level of accuracy means every network device consistently matches the color bit to the correct block 🎯 for example, if the color toggles every second (L = 1 second), then the clocks must be synchronized with an accuracy of +/- 0.5 second to a common time reference ⏱️

this synchronization requirement can be met even with a fairly inaccurate synchronization method ✨ that holds for packet loss and two-way delay measurement, but NOT for one-way delay measurement, where clock synchronization must be accurate 🙈

so a system that uses only packet loss and two-way delay measurement doesn't need synchronization at all 🎀 that's because the clock values on the network devices don't affect the two-way delay computation 💜

4.2. Data Correlation 💖

Okay so data correlation is basically the mechanism that lets you compare counters and timestamps so you can work out packet loss, delay, and delay variation. 🧚 There are a few ways to make it happen depending on the Alternate-Marking application and use case. Some possibilities are to:

  • use a centralized solution using NMS to correlate data; and
  • define a protocol-based distributed solution by introducing a new protocol or by extending the existing protocols (e.g., see RFC 6374 [RFC6374] or the Two-Way Active Measurement Protocol (TWAMP) as defined in RFC 5357 [RFC5357] or the One-Way Active Measurement Protocol (OWAMP) as defined in RFC 4656 [RFC4656]) in order to communicate the counters and timestamps between nodes. 🦋

In the following paragraphs, an example data correlation mechanism is explained and could be used independently of the adopted solutions. ✨

When data is collected on the upstream and downstream nodes, e.g., packet counts for packet loss measurement or timestamps for packet delay measurement, and is periodically reported to or pulled by other nodes or an NMS, a certain data correlation mechanism SHOULD be in use to help the nodes or NMS tell whether any two or more packet counts are related to the same block of markers or if any two timestamps are related to the same marked packet. 🌸

Hold up, the Alternate-Marking Method literally splits the packets of the measured flow into different measurement blocks, and each block can get a Block Number (BN) too. 💅 The BN is generated each time a node reads the data (packet counts or timestamps) and gets attached to each packet count and timestamp that's reported to or pulled by other nodes or NMSs. The value of a BN could be calculated as the modulo of the local time (when the data are read) and the interval of the marking time period. 🫶

So when the nodes or NMS see the same BNs attached to two packet counts, one from an upstream node and one from a downstream node, they know those two counts belong to the same block. 🔥 Like, these two packet counts belong to the same block of markers from the upstream and downstream nodes, periodt. 👑 The assumption behind this BN mechanism is that the measurement nodes are time synchronized, and that means they need a certain time synchronization capability (e.g., the Network Time Protocol (NTP) [RFC5905] or the IEEE 1588 Precision Time Protocol (PTP) [IEEE-1588]). 🌈 Synchronization aspects are further discussed in Section 4.1. 🎀

4.3. Packet Reordering

Okay, bestie, real talk: ECMP makes packet reordering super common in an IP network. 😭 The accuracy of a marking-based PM, especially packet loss measurement, can get affected by packet reordering, so take a look at this example:

Block   :    1    |    2    |    3    |    4    |    5    |...
--------|---------|---------|---------|---------|---------|---
Node R1 : AAAAAAA | BBBBBBB | AAAAAAA | BBBBBBB | AAAAAAA |...
Node R2 : AAAAABB | AABBBBA | AAABAAA | BBBBBBA | ABAAABA |...

Figure 5: Packet Reordering 🍓

In Figure 5, the packet stream for Node R1 isn't being reordered, so it can be safely assigned to interval blocks. 💜 But the packet stream for Node R2 is getting reordered, so looking at the packet with the marker of "B" in block 3, there is no safe way to tell whether it belongs to block 2 or block 4. 💀

In general, there's a need to assign packets with the marker of "B" or "A" to the right interval blocks. 🙈 Most of the packet reordering happens at the edge of adjacent blocks, and those are easy to handle if the interval of each block is sufficiently large. Then it can be assumed that the packets with different markers belong to the block that they are closer to. ✨ If the interval is small, it's difficult and sometimes impossible to determine which block a packet belongs to. 😭

Choosing a proper interval is important, but how to choose one is out of the scope of this document. 🦋 That said, an implementation SHOULD provide a way to configure the interval and allow a certain degree of packet reordering. 💖

5. Applications, Implementation, and Deployment 🌈

The methodology described in the previous sections can be applied in various situations. Basically, the Alternate-Marking technique could be used in many cases for performance measurement. 💅 The only requirement is to select and mark the flow to be monitored; in this way, packets are batched by the sender, and each batch is alternately marked so it can be easily recognized by the receiver. 🎀

Some recent Alternate-Marking Method applications are listed below: 🧚

  • IP Flow Performance Measurement (IPFPM): this application of the marking method is described in [COLORING]. As an example, in this document, the last reserved bit of the Flag field of the IPv4 header is proposed to be used for marking, while a solution for IPv6 could be to leverage the IPv6 extension header for marking. 🌸
  • OAM Passive Performance Measurement: In [RFC8296], two OAM bits from the Bit Index Explicit Replication (BIER) header are reserved for the Passive performance measurement marking method. [PM-MM-BIER] details the measurement for multicast service over the BIER domain. In addition, the Alternate-Marking Method could also be used in a Service Function Chaining (SFC) domain. Lastly, the application of the marking method to Network Virtualization over Layer 3 (NVO3) protocols is considered by [NVO3-ENCAPS]. 👑
  • MPLS Performance Measurement: RFC 6374 [RFC6374] uses the Loss Measurement (LM) packet as the packet accounting demarcation point. Unfortunately, this gives rise to a number of problems that may lead to significant packet accounting errors in certain situations. 😭 [MPLS-FLOW] discusses the desired capabilities for MPLS flow identification in order to perform a better in-band performance monitoring of user data packets. A method of accomplishing identification is Synonymous Flow Labels (SFLs) introduced in [SFL-FRAMEWORK], while [SYN-FLOW-LABELS] describes performance measurements in RFC 6374 with SFL. 🦋
  • Active Performance Measurement: [ALT-MM-AMP] describes how to extend the existing Active Measurement Protocol, in order to implement the Alternate-Marking methodology. [ALT-MM-SLA] describes an extension to the Cisco SLA Protocol Measurement-Type UDP-Measurement. 💜

An example of implementation and deployment is explained in the next section, just to clarify how the method can work. 🫶

5.1. Report on the Operational Experiment

Okay bestie, so this method is also called Packet Network Performance Monitoring (PNPM) 💅 and it was invented and engineered in Telecom Italia 🌸. The general description in this document actually came from the operational experiment 🧪. The core elements got tested for real, and what they learned is what shaped the formal Alternate-Marking Method we covered in the earlier sections ✨.

The methodology has been used experimentally on Telecom Italia's network 🌐. It's applied to multicast IPTV channels and other specific traffic flows with high QoS requirements, like Mobile Backhauling traffic running over a VPN MPLS 🎀. Iconic, right?

The technology was built using functions and tools that already exist on IP routers 🔧, and it's currently being used to monitor packet loss in some parts of Telecom Italia's network 📡. The method has also been evaluated for delay measurement in Telecom Italia's labs 🧫.

This section walks through how the experiment was run, especially how the features already on existing routing platforms can be used to apply the method. Think of it as a real-life example of implementation and deployment 🧚.

The operational test uses the flow-based strategy from Section 3 💖. The link-based strategy could also work, applied to a physical link or a logical link (for example, an Ethernet VLAN or an MPLS Pseudowire (PW)) 🦋.

Since a Service Provider (like Telecom Italia) ran the experiment on its own network, it had to lean on what the router functions already offer 👑. With current router implementations, only the QoS-related fields and features give the flexibility needed to set bits in the packet header 💅. If a Service Provider only uses the three most-significant bits of the DSCP field (which correspond to IP Precedence) for QoS classification and queuing, then the two least-significant bits of the DSCP field (bit 0 and bit 1) can be used to implement the method without messing with QoS policies 🙈. That's the approach the experiment took ✨. Bit 0 can identify flows that are subject to traffic monitoring (set to 1 if the flow is under monitoring, otherwise set to 0) 🔍. Bit 1 can be used for coloring the traffic (switching between the values 0 and 1, which correspond to colors A and B) and for creating the blocks 🎨.

The experiment treats a flow as all the packets sharing the same source IP address or the same destination IP address, depending on the direction 🧭. In practice, once the flow is defined, traffic coloring with the DSCP field can be set up by configuring an access-list on the router output interface 🛠️. That access-list catches the flow(s) you want to monitor and applies a policy that sets the DSCP field accordingly 💜. Because the coloring has to switch between the two values over time, the policy needs to be changed periodically 🔄. An automatic script handles that job on a fixed timer ⏰. The script is loaded onto the router and automates the basic operations the methodology needs 🤖.

After the traffic is colored with the DSCP field, every router on the path can do the counting 🔢. An access-list that matches specific DSCP values can count the packets of the monitored flow(s) 📊. The same access-list can be installed on all the routers along the path 🛤️. On top of that, network flow monitoring, such as what IPFIX [RFC7011] provides, can be used to recognize the timestamps of the first and last packet of a batch 🕰️. That opens the door to one of the alternatives for measuring delay that's detailed in Section 3.3 🌈.

In Telecom Italia's experiment, the timer is set to 5 minutes, so the script's sequence of actions also runs every 5 minutes ⏱️. That value turned out to be a good compromise between how often you measure and how stable the measurement stays (in other words, whether you can collect all the measurements that refer to the same block) 🫶.

For this experiment, the counters and any other data are collected by the automatic script, which sends them out to an NMS 📬. The NMS is responsible for calculating packet loss by comparing the counter values from the routers along the flow path(s) 🧮. A 5-minute timer for color switching is a safe choice for reading the counters, and it also lines up nicely with the NMS's reporting window 💅.

Heads up: using the DSCP field for marking means the method only works reliably inside a single management and operation domain ⚠️.

Lastly, the Telecom Italia experiment scales up to 1000 flows monitored together on a single router 🚀. An implementation on dedicated hardware scales further, but so far it's only been tested in labs 🧪.

5.1.1. Metric Transparency 🫧

Since this is a Service Provider application, the method can be applied to end-to-end services sold to customers 💼. So it's really important to highlight that the method MUST be transparent outside the Service Provider domain 🎀.

In Telecom Italia's implementation, the source node colors the packets using a policy that gets changed periodically by an automatic script, which alternates the DSCP field of the packets 🎨. The nodes between the source and the destination (destination included) use an access-list to count the colored packets they receive and forward 🔢.

The destination node has a big role too 👑. It intercepts the colored packets, and a policy restores the DSCP field of all the packets to its initial value 🪄. That's what makes the metric transparent: outside the part of the network under monitoring, the traffic flow stays exactly the same 🦋.

Thanks to this restoring technique, network elements outside the Alternate-Marking monitoring domain (for example, the two Provider Edge nodes of the Mobile Backhauling VPN MPLS) are totally unaware that packets were ever marked 🙈. So this restoring technique makes Alternate Marking completely transparent outside its monitoring domain 💖.

6. Hybrid Measurement 🧚

The method was explicitly designed for Passive measurements, but it can also be used with Active measurements 🌸. To get both end-to-end measurements and intermediate measurements (Hybrid measurements), two endpoints can exchange artificial traffic flows and apply Alternate Marking over those flows 🔁. At the intermediate points, the artificial traffic is handled the same way as real traffic and measured as specified before 📏. So applying the marking method can also simplify Active measurement, as explained in [ALT-MM-AMP] ✨.

7. Compliance with Guidelines from RFC 6390 ✨

okay so [RFC6390] is basically the rulebook 📖 for writing Performance Metrics for protocols above and below the IP layer, like IP-based apps running over reliable or datagram transport protocols. 💅

This doc isn't cooking up a brand-new Performance Metric 🙅‍♀️. It's a new Method of Measurement for a few metrics that are already standardized. Still, it's worth following the [RFC6390] guidelines so the whole method is described more completely and coherently 🫶. We used the Performance Metric Definition template from Section 5.4 of [RFC6390], plus the Dependencies from Section 5.5 of that same doc.

  • Metric Name / Metric Description: as we already said, this doc doesn't propose any new Performance Metrics 🙈. What it does is describe a novel method for measuring packet loss [RFC7680]. The same concept, with small differences, can also be used to measure delay [RFC7679] and jitter [RFC3393]. The doc mainly focuses on applying it to packet loss measurement.
  • Method of Measurement or Calculation: following the method from the earlier sections, the number of packets lost is calculated by subtracting the value of the counter on the source node from the value of the counter on the destination node 🧮. Both counters have to refer to the same color 🎨. The calculation happens when the counter values are in a steady state. That steady state is built right into the marking method, because the color alternates and the counters for each color sit still, one color at a time, for the whole marking period 🦋.
  • Units of Measurement: the method calculates and reports the exact number of packets that the source node sent but the destination node never received 📦.
  • Measurement Point(s) with Potential Measurement Domain: you can measure between adjacent nodes (per-link), or along a multi-hop path, as long as the traffic being measured actually follows that path 🛤️. On a multi-hop path, you can measure end-to-end and hop-by-hop, both 🙌.
  • Measurement Timing: the method has a constraint on how often you measure ⏱️. That's covered in Section 3.2, which says the marking period and the guard band interval are strictly tied to each other to avoid out-of-order issues 🌀. The reason is that a measurement needs the counter to be in a steady state, and that only happens while traffic is being colored with the alternate color. For example, the experiment of the method set the time interval to 5 minutes, but other optimized implementations can also use a marking period of a few seconds ⚡.
  • Implementation: the experiment uses two encodings of the DSCP field to color the packets 🌈. That means you can use policy configurations on the router to color the packets and set up the counter for each color to match 🎀. The path that the measured traffic follows should be known ahead of time, so you can configure the counters along that path and compare the right values 🧚.
  • Verification: the method has been tested and experimented with for packet loss and delay measurements, both in the lab and in the operational network 🔬. The testing used traffic generators together with precision test instruments and network emulators 🛠️.
  • Use and Applications: the method can measure packet loss with high precision on live traffic 💯. Also, by combining end-to-end and per-link measurements, you can pinpoint the single link that's experiencing loss events 🎯. Main character energy, honestly 👑.
  • Reporting Model: the counter values have to be sent to a centralized management system that does the calculations 🖥️. Each sample has to include a reference to the time interval it belongs to, so the management system can correlate everything correctly 🗂️. Samples should be sent while the corresponding counter is in a steady state (within a time interval). Otherwise, the sample value should be stored locally 💾.
  • Dependencies: the counter values have to be correlated to the time interval they refer to 🕰️. Also, because the experiment is based on DSCP values, there are big dependencies on how the DSCP field gets used 😭. It has to be possible to rely on unused DSCP values without affecting QoS-related configuration and behavior 🚦. And the intermediate nodes have to leave the DSCP field value alone so they don't mess up the measurement 🙅‍♀️.
  • Organization of Results: the Method of Measurement produces singletons 🍓.
  • Parameters: right now, the main parameter of the method is the time interval used to alternate the colors and read the counters ⏰.

8. IANA Considerations 🌸

Nothing for IANA to do here, bestie. This document has no IANA actions 🫶.

9. Security Considerations 🔐

This one is serious, so let's go 🚨. This document specifies a method for measurements inside a Service Provider's network. It wasn't built for conducting Internet measurements, so it doesn't directly affect Internet security or the applications that run on the Internet 🌐. Even so, anyone implementing this method has to be super mindful of security and privacy concerns 👀.

There are two types of security concerns: harm caused by the measurements, and harm to the measurements 😳.

  • Harm caused by the measurement: the measurements in this document are Passive, so no new packets get injected into the network. That means there's no new potential harm to the network itself or to data traffic 🙌. But the method does involve modifying a header or encapsulation of data packets on the fly 😬. That has to be done in a way that doesn't alter the quality of service the measured packets get, and that keeps the stability and performance of the routers doing the measurements intact 🛡️. One of the big security threats in OAM protocols is network reconnaissance, where an attacker figures out network performance by passively eavesdropping on OAM messages 🕵️‍♀️. The good news is that the marking bits are the only information exchanged between network devices. So passively eavesdropping on data-plane traffic doesn't let attackers learn anything about network performance 💅.
  • Harm to the Measurement: the measurements could be harmed by routers changing the marking of packets, or by an attacker injecting fake traffic 😱. Authentication techniques, like digital signatures, may be used where appropriate to guard against injected traffic attacks 🔑. Since routers (or other network devices) along the path could intentionally change the marking bits, as mentioned above, the mechanism in this document can only be applied in the context of a controlled domain 🏰. That means the routers (and other network devices) are locally administered, so this type of attack can be avoided 🙅‍♀️. Also, an attacker can't learn anything about network performance from a single monitoring point 🙅. They'd need synchronized monitoring points at multiple points along the path, because those points have to do the same kind of measurement and aggregation that Service Providers using Alternate Marking have to do 🧩.

The privacy concerns of network measurement are limited, because the method only relies on information in the header or encapsulation and doesn't release any user data 🤍. Header or encapsulation information is metadata, and metadata can be used to compromise user privacy, but the limited marking technique in this document seems unlikely to substantially increase those existing privacy risks 🌷. It might be theoretically possible to modulate the marking to work as a covert channel 🤔. But it would have a very low data rate if it's going to avoid messing with the measurement systems that monitor the marking 🐢.

Delay attacks are another potential threat for this document 🥺. Delay measurement uses one specific packet in each block, marked by a dedicated color bit. So a man-in-the-middle attacker could selectively add synthetic delay only to the delay-colored packets, causing systematic error in the delay measurements 😭. As discussed in earlier sections, the methods here rely on an underlying time synchronization protocol. So if an attacker goes after the time protocol, they could potentially compromise the integrity of the measurement 🕰️. A detailed discussion of threats against time protocols, and how to mitigate them, is in RFC 7384 [RFC7384] 📚.

10. References 📚

10.1. Normative References

[IEEE-1588] IEEE, "IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems", IEEE Std 1588-2008.

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, <https://www.rfc-editor.org/info/rfc2119>.

[RFC3393] Demichelis, C. and P. Chimento, "IP Packet Delay Variation Metric for IP Performance Metrics (IPPM)", RFC 3393, DOI 10.17487/RFC3393, November 2002, <https://www.rfc-editor.org/info/rfc3393>.

[RFC5905] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch, "Network Time Protocol Version 4: Protocol and Algorithms Specification", RFC 5905, DOI 10.17487/RFC5905, June 2010, <https://www.rfc-editor.org/info/rfc5905>.

[RFC7679] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, Ed., "A One-Way Delay Metric for IP Performance Metrics (IPPM)", STD 81, RFC 7679, DOI 10.17487/RFC7679, January 2016, <https://www.rfc-editor.org/info/rfc7679>.

[RFC7680] Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, Ed., "A One-Way Loss Metric for IP Performance Metrics (IPPM)", STD 82, RFC 7680, DOI 10.17487/RFC7680, January 2016, <https://www.rfc-editor.org/info/rfc7680>.

[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, <https://www.rfc-editor.org/info/rfc8174>.

10.2. Informative References 🧾✨

Okay bestie, here's the receipts drawer 💅 Every paper trail this RFC leans on for the informative side, with all the details. Some are still drafts (Work in Progress), so they're not final yet 🌸

  • [ALT-MM-AMP] Fioccola, G., Clemm, A., Bryant, S., Cociglio, M., Chandramouli, M., and A. Capello, "Alternate Marking Extension to Active Measurement Protocol", Work in Progress, draft-fioccola-ippm-alt-mark-active-01, March 2017.
  • [ALT-MM-SLA] Fioccola, G., Clemm, A., Cociglio, M., Chandramouli, M., and A. Capello, "Alternate Marking Extension to Cisco SLA Protocol RFC6812", Work in Progress, draft-fioccola-ippm-rfc6812-alt-mark-ext-01, March 2016.
  • [COLORING] Chen, M., Zheng, L., Mirsky, G., Fioccola, G., and T. Mizrahi, "IP Flow Performance Measurement Framework", Work in Progress, draft-chen-ippm-coloring-based-ipfpm-framework-06, March 2016.
  • [IP-FLOW-REPORT] Chen, M., Zheng, L., and G. Mirsky, "IP Flow Performance Measurement Report", Work in Progress, draft-chen-ippm-ipfpm-report-01, April 2016.
  • [IP-MULTICAST-PM] Cociglio, M., Capello, A., Bonda, A., and L. Castaldelli, "A method for IP multicast performance monitoring", Work in Progress, draft-cociglio-mboned-multicast-pm-01, October 2010.
  • [MPLS-FLOW] Bryant, S., Pignataro, C., Chen, M., Li, Z., and G. Mirsky, "MPLS Flow Identification Considerations", Work in Progress, draft-ietf-mpls-flow-ident-06, December 2017.
  • [MULTIPOINT-ALT-MM] Fioccola, G., Cociglio, M., Sapio, A., and R. Sisto, "Multipoint Alternate Marking method for passive and hybrid performance monitoring", Work in Progress, draft-fioccola-ippm-multipoint-alt-mark-01, October 2017.
  • [NVO3-ENCAPS] Boutros, S., Ganga, I., Garg, P., Manur, R., Mizrahi, T., Mozes, D., Nordmark, E., Smith, M., Aldrin, S., and I. Bagdonas, "NVO3 Encapsulation Considerations", Work in Progress, draft-ietf-nvo3-encap-01, October 2017.
  • [OPSAWG-P3M] Capello, A., Cociglio, M., Castaldelli, L., and A. Bonda, "A packet based method for passive performance monitoring", Work in Progress, draft-tempia-opsawg-p3m-04, February 2014.
  • [PM-MM-BIER] Mirsky, G., Zheng, L., Chen, M., and G. Fioccola, "Performance Measurement (PM) with Marking Method in Bit Index Explicit Replication (BIER) Layer", Work in Progress, draft-ietf-bier-pmmm-oam-03, October 2017.
  • [RFC4656] Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. Zekauskas, "A One-way Active Measurement Protocol (OWAMP)", RFC 4656, DOI 10.17487/RFC4656, September 2006, <https://www.rfc-editor.org/info/rfc4656>.
  • [RFC5357] Hedayat, K., Krzanowski, R., Morton, A., Yum, K., and J. Babiarz, "A Two-Way Active Measurement Protocol (TWAMP)", RFC 5357, DOI 10.17487/RFC5357, October 2008, <https://www.rfc-editor.org/info/rfc5357>.
  • [RFC5481] Morton, A. and B. Claise, "Packet Delay Variation Applicability Statement", RFC 5481, DOI 10.17487/RFC5481, March 2009, <https://www.rfc-editor.org/info/rfc5481>.
  • [RFC6374] Frost, D. and S. Bryant, "Packet Loss and Delay Measurement for MPLS Networks", RFC 6374, DOI 10.17487/RFC6374, September 2011, <https://www.rfc-editor.org/info/rfc6374>.
  • [RFC6390] Clark, A. and B. Claise, "Guidelines for Considering New Performance Metric Development", BCP 170, RFC 6390, DOI 10.17487/RFC6390, October 2011, <https://www.rfc-editor.org/info/rfc6390>.
  • [RFC6703] Morton, A., Ramachandran, G., and G. Maguluri, "Reporting IP Network Performance Metrics: Different Points of View", RFC 6703, DOI 10.17487/RFC6703, August 2012, <https://www.rfc-editor.org/info/rfc6703>.
  • [RFC7011] Claise, B., Ed., Trammell, B., Ed., and P. Aitken, "Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of Flow Information", STD 77, RFC 7011, DOI 10.17487/RFC7011, September 2013, <https://www.rfc-editor.org/info/rfc7011>.
  • [RFC7276] Mizrahi, T., Sprecher, N., Bellagamba, E., and Y. Weingarten, "An Overview of Operations, Administration, and Maintenance (OAM) Tools", RFC 7276, DOI 10.17487/RFC7276, June 2014, <https://www.rfc-editor.org/info/rfc7276>.
  • [RFC7384] Mizrahi, T., "Security Requirements of Time Protocols in Packet Switched Networks", RFC 7384, DOI 10.17487/RFC7384, October 2014, <https://www.rfc-editor.org/info/rfc7384>.
  • [RFC7799] Morton, A., "Active and Passive Metrics and Methods (with Hybrid Types In-Between)", RFC 7799, DOI 10.17487/RFC7799, May 2016, <https://www.rfc-editor.org/info/rfc7799>.
  • [RFC8296] Wijnands, IJ., Ed., Rosen, E., Ed., Dolganow, A., Tantsura, J., Aldrin, S., and I. Meilik, "Encapsulation for Bit Index Explicit Replication (BIER) in MPLS and Non-MPLS Networks", RFC 8296, DOI 10.17487/RFC8296, January 2018, <https://www.rfc-editor.org/info/rfc8296>.
  • [SFL-FRAMEWORK] Bryant, S., Chen, M., Li, Z., Swallow, G., Sivabalan, S., and G. Mirsky, "Synonymous Flow Label Framework", Work in Progress, draft-ietf-mpls-sfl-framework-00, August 2017.
  • [SYN-FLOW-LABELS] Bryant, S., Chen, M., Li, Z., Swallow, G., Sivabalan, S., Mirsky, G., and G. Fioccola, "RFC6374 Synonymous Flow Labels", Work in Progress, draft-ietf-mpls-rfc6374-sfl-01, December 2017.

Acknowledgements 🫶

The previous IETF specifications describing this technique were: [IP-MULTICAST-PM] and [OPSAWG-P3M].

The authors would like to thank Alberto Tempia Bonda, Domenico Laforgia, Daniele Accetta, and Mario Bianchetti for their contribution to the definition and the implementation of the method. 💜

The authors would also thank Spencer Dawkins, Carlos Pignataro, Brian Haberman, and Eric Vyncke for their assistance and their detailed and precious reviews. 🌈

Authors' Addresses 🧑‍💻

Giuseppe Fioccola (editor) 👑

Telecom Italia
Via Reiss Romoli, 274
Torino  10148
Italy

Email: giuseppe.fioccola@telecomitalia.it

Alessandro Capello 🦋

Telecom Italia
Via Reiss Romoli, 274
Torino  10148
Italy

Email: alessandro.capello@telecomitalia.it

Mauro Cociglio 🍓

Telecom Italia
Via Reiss Romoli, 274
Torino  10148
Italy

Email: mauro.cociglio@telecomitalia.it

Luca Castaldelli 🧚

Telecom Italia
Via Reiss Romoli, 274
Torino  10148
Italy

Email: luca.castaldelli@telecomitalia.it

Mach(Guoyi) Chen 🌸

Huawei Technologies

Email: mach.chen@huawei.com

Lianshu Zheng ⭐

Huawei Technologies

Email: vero.zheng@huawei.com

Greg Mirsky 💅

ZTE
United States of America

Email: gregimirsky@gmail.com

Tal Mizrahi 🎀

Marvell
6 Hamada St.
Yokneam
Israel

Email: talmi@marvell.com
📦 where this page comes from