RFC 8372

MPLS Flow Identification Considerations

📅 published May 2018, from ietf

RFC 8372: MPLS Flow Identification Considerations 🦋

Okay bestie, let's get the tea first 🫖 This one is Informational, which means it's a "here's what to think about" doc and not a Standards Track spec 💅 It came out in May 2018 and its ISSN is 2070-1721. The authors are S. Bryant (Huawei), C. Pignataro (Cisco), M. Chen (Huawei), Z. Li (Huawei), and G. Mirsky (ZTE Corp.). The header doesn't list anything it obsoletes or updates, so it's a standalone glow-up ✨

Abstract 🌸

This doc is all about the stuff you have to think about before building a solution for MPLS flow identification 🧚 The main reason anyone needs this is in-band performance monitoring, which means measuring the actual user data packets while MPLS is wrapping them up 🎀

Status of This Memo 💜

Quick reality check, bestie: this is not an Internet Standards Track spec. It's published just to share info 📚

It is a product of the IETF, so it reflects what the IETF community agrees on. It got public review and was approved for publication by the IESG. Not everything the IESG approves is eligible for any level of Internet Standard, though. If you want the details, check Section 2 of RFC 7841 🤓

For the current status, any errata, or to give feedback, go to https://www.rfc-editor.org/info/rfc8372 🍓

Copyright (c) 2018 IETF Trust and the persons identified as the document authors. All rights reserved. 🔒

This doc is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info), as they stood on the publication date. Please read them carefully, because they explain your rights and restrictions for this document 🫶 Any Code Components you pull out of this doc have to include the Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions. They come with no warranty, as the Simplified BSD License says ⚖️

Table of Contents 💖

1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
2.  Loss Measurement Considerations . . . . . . . . . . . . . . .   3
3.  Delay Measurement Considerations  . . . . . . . . . . . . . .   4
4.  Units of Identification . . . . . . . . . . . . . . . . . . .   4
5.  Types of LSP  . . . . . . . . . . . . . . . . . . . . . . . .   6
6.  Network Scope . . . . . . . . . . . . . . . . . . . . . . . .   7
7.  Backwards Compatibility . . . . . . . . . . . . . . . . . . .   7
8.  Data Plane  . . . . . . . . . . . . . . . . . . . . . . . . .   7
9.  Control Plane . . . . . . . . . . . . . . . . . . . . . . . .   9
10. Privacy Considerations  . . . . . . . . . . . . . . . . . . .   9
11. Security Considerations . . . . . . . . . . . . . . . . . . .   9
12. IANA Considerations . . . . . . . . . . . . . . . . . . . . .   9
13. Informative References  . . . . . . . . . . . . . . . . . . .  10
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . .  11
Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  11

1. Introduction ✨

Okay, so this section is the setup. It talks about everything you need to consider when you're building a solution for MPLS flow identification 🎀 Again, the big use case is in-band performance monitoring of MPLS flows when MPLS is encapsulating user data packets.

Why does anyone need to identify flows in MPLS networks? Because it's needed for stuff like figuring out packet loss and measuring packet delay 💅 A loss and delay measurement method for MPLS was already defined in [RFC6374]. When [RFC6374] is used to measure packet loss, it leans on injected Operations, Administration, and Maintenance (OAM) packets. Those mark where a group of packets starts and ends, which is the window you're measuring loss over 📦 If packets from one group get misordered relative to the next group, or if any of the packets being counted get misordered relative to the Loss Measurement packet, the loss measurement is going to be wrong. Not great, bestie 😭

[RFC6374] also didn't support different levels of flow granularity. It didn't really handle the multipoint cases either, where two or more ingress Label Switching Routers (LSRs) can send packets to one or more destinations 🌸

Now here's the vibe shift. Loss rates in normal operation are super low, and better link and transmission tech has made it way harder to measure loss with active methods that use synthetic traffic 📉 Add in tougher service-level requirements, and operators now need to measure loss on the real user data traffic using passive methods 🔍 Whatever technique gets deployed has to be transparent to the end user, and you have to assume they won't do anything active in the measurement process. It's really important that the flow identification technique is invisible to them, and that no trace of the measurement leaks into their network 🫣

One more thing: when there are multiple traffic sources, like in multipoint-to-point and multipoint-to-multipoint environments, the sink needs a way to tell apart packets from the different sources. In other words, a multipoint measurement model still needs to be developed 🧚‍♀️

2. Loss Measurement Considerations 💅

Modern networks that aren't oversubscribed usually drop very few packets 🌊 That means packet loss measurement is super sensitive to exactly which set of packets is being measured. Without some kind of coloring or batch marking, like the one proposed in [RFC8321], you might not be able to hit the accuracy you need when measuring loss on customer data traffic 😬 So if accurate loss measurement is required, it can be economically worth it, or even a technical must-have, to put some form of marking in the packets. That marking assigns each packet to a particular loss measurement counter 🔢

When that level of accuracy is needed and the traffic between a source-destination pair goes through Equal-Cost Multipath (ECMP), you need a demarcation mechanism to group the packets into batches 🍬 Once a batch has been correlated at both ingress and egress, the packet accounting mechanism can work on the batch that can be accounted for at both the packet ingress and the packet egress. ECMP makes accounting errors especially bad on Label Switched Paths (LSPs), because the transit time through the network is different for each ECMP path. That's because:

  1. the packets may go through different sets of LSRs;
  2. the packets may leave from different interfaces on different line cards on LSRs; and
  3. the packets may arrive at different interfaces on different line cards on LSRs.

Here's a tricky bit 🙈 If you change the identity label (the MPLS label normally used to identify the LSP, Virtual Private Network, Pseudowire, and so on) to show the batch, that can affect which path ECMP picks. When the ECMP path member is chosen by deep packet inspection, changing the batch by changing the identity label has no effect on the ECMP path. But if the path member is chosen by looking at an entropy label [RFC6790], then changing the batch identifier won't change the ECMP path either. ECMP is everywhere in multipoint-to-(multi)point networks, so you need some way to avoid the accounting errors ECMP introduces 🫠

3. Delay Measurement Considerations 🎀

Most of the delay measurement methods that exist today are active. They rely on an extra injected test packet to figure out the delay of a path ⏱️ With active measurement, the rate, number, and spacing of those injected packets can affect how accurate the results are. Because of ECMP (or link aggregation techniques), injected test packets can take different links from the ones the data traffic uses 🛤️ So measuring the delay of the real traffic is required.

For combined loss and delay measurements, both the loss considerations and the delay considerations apply 💖

4. Units of Identification 🦋

The most basic unit of identification is the identity of the node that processed the packet when it entered the MPLS network 🚪 The unit you actually need can change depending on the use case, whether that's accounting, performance measurement, or some other kind of packet observation 📊 Worth noting: you might need to impose identity at several different layers of the MPLS label stack 🧱

This document looks at identifying these traffic components:

  • Per source LSR: everything from one source is aggregated
  • Per group of LSPs chosen by an ingress LSR: an ingress LSP aggregates a group of LSPs (e.g., all the LSPs of a tunnel)
  • Per LSP: the basic form
  • Per flow [RFC6790] within an LSP: a fine-grained method

Quick note: you need fine-grained identity resolution when you want to do these operations on a flow that isn't easily identified by some other element in the label stack 🔬 That kind of fine-grained resolution might be possible with deep packet inspection. But that's not always possible, and you might want to cut processing costs by only doing it on entry to the network. Adding a suitable identifier to the packet for other network elements to reference keeps the processing other elements have to do to a minimum 🧠 A good example of a fine-grained case is traffic belonging to a certain service or coming from a specific source, especially if someone is looking into service level agreement or application performance issues 📈

So, broadly speaking, the identification requirements are:

  • There needs to be some way for an egress LSR to identify the ingress LSR with an appropriate degree of scope. Section 6 talks about this more 🔭
  • There needs to be a way to identify a specific LSP at the egress node. This covers the case where you're instrumenting multiple LSPs between the same pair of nodes. In that case, the identity of the ingress LSR isn't enough 🙅‍♀️
  • To save resources like labels, counters, and/or compute cycles, you may want to identify an LSP group so an operation can run on the group as one aggregate 🧃
  • There needs to be a way to identify a flow within an LSP. You need this when you're investigating a specific flow that got aggregated into an LSP 🕵️‍♀️

The unit of identification, and how you decide which packets make up a flow, depends on the application or use case. Those are out of scope for this document 👑

5. Types of LSP ✨

okay so we need to think about a bunch of LSP types 🦋 The two easiest ones to monitor are point-to-point LSPs and point-to-multipoint LSPs. For a point-to-point LSP, like one made with the Resource Reservation Protocol - Traffic Engineering (RSVP-TE) [RFC5420] signaling protocol or one that follows the MPLS Transport Profile (MPLS-TP) [RFC5654], you can figure out the ingress LSR just by peeking at the top label in the stack. That's because at any provider-edge (PE) or provider (P) router on the path, the top label is unique to the ingress-egress pair at every hop at a given layer in the LSP hierarchy 💅. As long as Penultimate Hop Popping (PHP) is turned off, the egress LSR can see who the ingress LSR is, so identifying the ingress LSR is basically a solved problem 🎀. BUT, and this is a big but 😭, you still can't pin down a specific flow without extra info stuck in the packet or pulled from some part of the packet payload.

For a point-to-multipoint LSP, and again with no PHP, the ingress LSR can also be inferred from the top label. The catch is that the top label alone may not be enough to identify the flow, so more information may need to be carried in the packet or gleaned from some aspect of the payload. When you design a solution, it's desirable to use one common flow identification solution for both point-to-point and point-to-multipoint LSP types 🌈. It's also desirable to use a common method of LSP group identification. In the cases above, a context label [RFC5331] needs to be used to provide the identity info you need, and it's a widely supported MPLS feature, so no stress 🙌.

Now the main character of the bunch: the multipoint-to-point LSP 👑. Here, the same label is normally used by multiple ingress or upstream LSRs, so the egress LSRs can't tell where a packet came from just by looking at the top label. That means a packet has to be able to explicitly carry any of the identity types described in Section 4. Multipoint-to-multipoint LSPs are the same story 💜: the same label is normally shared by multiple ingress or upstream LSRs, so source identification isn't possible from the top label at the egress LSRs, and the identity types from Section 4 are needed again. The one bit of good news is that the scope of the identity may be constrained to be unique within the set of multipoint-to-multipoint LSPs terminating on any common node ✨.

6. Network Scope 🌸

The scope of identification can be limited to the set of flows that are uniquely identifiable at an ingress LSR, or to some aggregation of those flows. An ingress LSR does not need to ask for help outside the MPLS protocol domain, so it can keep it all in the family 🧚.

If a solution sticks to carrying the required identity inside the MPLS label stack (instead of in some different associated data structure), then the choice of label and the label stack size put limits on things, and the identity scope ends up living inside that MPLS domain. For similar reasons, the identity scope of any component of an LSP is limited to the scope of that LSP 🍓.

7. Backwards Compatibility 🫶

Let's be real, it's super unlikely that every LSR in a network will be able to do all the identification methods we've been talking about 😅. So any flow identity solution has to be backwards compatible with MPLS gear that's already out there. The new feature can't break anything that already works on the legacy equipment. This matters even more when the rollout is incremental or when the feature isn't required on every LSR or every LSP. So the flow identification design must support LSRs that can identify the traffic components described in Section 4 living side by side with LSRs that can't 🦋. On top of that, the identification of those traffic components must be an optional feature that is disabled by default 🔒.

As a design simplification, a solution may require that all egress LSRs of a point-to-multipoint or multipoint-to-multipoint LSP support the identification type in use, so a single packet can be processed correctly by every egress device. The flip side of that is that either all egress LSRs are enabled to support the required identity type, or none of them are 💅.

8. Data Plane 💖

Okay, the MPLS data plane situation is kind of a lot 😭. There's a HUGE installed base of MPLS equipment. This gear usually stays in service for a long time, and in lots of cases, the hardware limits mean it just can't be upgraded to do new data-plane stuff. So changing the MPLS data plane is expensive, adds complexity to the network, and can seriously hurt how deployable a solution is if it needs those changes 💀. That's why MPLS users have set a very high bar for data-plane changes, and only a tiny number have actually been adopted. So the identification method must minimize changes to the MPLS data plane. Ideally, methods that need no data-plane changes should get preferential consideration ✨. If you do pick a method that changes the data plane, it needs a significant advantage over any method that makes no change, and that advantage needs to be carefully evaluated and documented 📝.

If a data-plane change turns out to be necessary, it should be (a) as small a change as possible and (b) a general-purpose method, so it can be reused for future applications 🌈. It's imperative that, as far as anyone can foresee, any necessary change to the MPLS data plane does not impose any foreseeable future limitation on it 🔥.

Stack size is a problem for lots of MPLS implementations, both because of hardware limits and because of the impact on networks and applications that carry a large number of small payloads 🍓. One MPLS payload can also be carried inside another. For example, one LSP can ride over another LSP, or a Pseudowire (PW) or similar multiplexing construct can ride over an LSP, and identification may be needed at both layers 🎀. The biggest worry is low-cost edge LSRs, which for cost reasons have a significant limit on how many Label Stack Entries (LSEs) they can impose or dispose 😩. So any method of identity must not consume an excessive number of unique labels, and it must not cause an excessive increase in the size of the label stack.

The MPLS data plane has two kinds of special-purpose labels: the original 16 reserved labels, and the much bigger set of special-purpose labels defined in [RFC7274]. The original reserved labels cost one LSE, and the newer special-purpose labels [RFC7274] cost two LSEs. Since there are only so many original reserved labels, it's core to the MPLS design philosophy that this scarce resource is only used when it's absolutely necessary 🙈. Using a special-purpose label to encode flow identity takes two label stack entries: one for the reserved label and one for the flow identity. Using extended special-purpose labels [RFC7274] takes a total of three label stack entries to encode the flow identity. The bigger set of [RFC7274] labels needs two LSEs for the special-purpose label itself, so you end up with three LSEs total to encode the flow identity 🧮.

So using special-purpose labels [RFC7274] as part of a method for encoding identity info has a bunch of undesirable effects on the data plane 😬. That's why, even though a solution may use special-purpose labels, methods that don't need them need to be carefully considered first 💜.

9. Control Plane 🎀

Any flow identity design should do two things: minimize the complexity of the control plane, and minimize the amount of label coordination needed among LSRs 🧚‍♀️. Less coordination, less drama, honestly iconic ✨.

10. Privacy Considerations 🔒

Here's the thing: putting originating and/or flow information in a packet gives out more identity information, which can potentially hurt the privacy of the communication 👀.

The IETF has been seriously concerned about pervasive monitoring [RFC7258], so there's a preference for a solution that does not degrade the privacy of user traffic below that of an MPLS network that isn't using the flow identification feature. Good news: using MPLS technology for this OAM solution has a privacy advantage. The label identifying a flow is limited to the scope of the MPLS domain, and it has no dependency on the identification of the user data. That keeps the flow characteristics much less observable 🫣🦋.

11. Security Considerations 🚨

Okay, this part is SERIOUS 😤. Any flow identification solution must not degrade the security of the MPLS network below that of an equivalent network that doesn't deploy the specified identity solution. To keep the current assumptions about MPLS privacy properties intact, propagating identification information outside the MPLS network that imposes it must be disabled by default 🛑. Any solution should let you restrict the identity information to only the parts of the network that need to know it. So it's desirable to limit knowledge of an endpoint's identity to only the LSRs that need to take part in the traffic flow 🕵️‍♀️. Using MPLS technology for this OAM solution, with MPLS encapsulation of user traffic, gives a key advantage over other data-plane solutions, because it provides controlled access and a trusted domain within a service provider's network 🏰👑.

If you want the full, comprehensive tea on MPLS security and attack mitigation techniques, check out "Security Framework for MPLS and GMPLS Networks" [RFC5920] 📚💅.

12. IANA Considerations 🌷

This document has no IANA considerations. Nothing to register here, bestie 🫶✨.

13. Informative References ✨

Okay so this is the reading list, bestie 📚🎀 These are the extra RFCs the document points at for more context. None of them are required to get the main idea, but they're super useful if you want to go deeper 🧚

Acknowledgments 🫶

The authors thank Nobo Akiya, Nagendra Kumar Nainar, George Swallow, and Deborah Brungard for their comments. Iconic of them to show up for the review squad 💜

Authors' Addresses 💌

Stewart Bryant Huawei

Email: stewart.bryant@gmail.com

Carlos Pignataro Cisco Systems, Inc.

Email: cpignata@cisco.com

Mach(Guoyi) Chen Huawei

Email: mach.chen@huawei.com

Zhenbin Li Huawei

Email: lizhenbin@huawei.com

Gregory Mirsky ZTE Corp.

Email: gregimirsky@gmail.com

That's the whole document, bestie. The authors did the hard part, and now you know the vibe 🎀✨
📦 where this page comes from