RFC 6372
MPLS Transport Profile (MPLS-TP) Survivability Framework
Part 1 of 13 🎀
MPLS Transport Profile (MPLS-TP) Survivability Framework ✨
RFC 6372 is an Informational doc from the IETF, and it's a joint effort with ITU-T. 💖
- Authors: N. Sprecher, Ed. (Nokia Siemens Networks) and A. Farrel, Ed. (Juniper Networks)
- Category: Informational
- Date: September 2011
- ISSN: 2070-1721
- Obsoletes/Updates: The title block doesn't list anything it obsoletes or updates. 🧚
Abstract 🌸
Okay so network survivability is a network's ability to keep traffic flowing after something breaks or gets degraded. It's so important for guaranteed services, especially ones with strict Service Level Agreements (SLAs) that cap how long a service can be degraded or down. 😭
The Transport Profile of Multiprotocol Label Switching (MPLS-TP) is a packet-based transport technology. It's built on the MPLS data plane and reuses lots of MPLS management and control plane stuff. 🦋
This document is a framework for survivability in an MPLS-TP network. It covers recovery elements, types, methods, and topological considerations. To enable data-plane recovery, survivability can be supported by the control plane, the management plane, and Operations, Administration, and Maintenance (OAM) functions. It describes mechanisms for recovering MPLS-TP Label Switched Paths (LSPs). Pseudowire recovery in MPLS-TP networks is a whole other story and is beyond the scope of this doc. 🙈
This document is also a product of a joint Internet Engineering Task Force (IETF) / International Telecommunication Union Telecommunication Standardization Sector (ITU-T) effort. The goal is to bring an MPLS Transport Profile into the IETF MPLS and Pseudowire Emulation Edge-to-Edge (PWE3) architectures, so they can support the capabilities and functionalities of a packet-based transport network as defined by the ITU-T. 👑
Status of This Memo 📜
This document is not an Internet Standards Track specification. It's published for informational purposes. 💅
It's a product of the Internet Engineering Task Force (IETF) and represents the consensus of the IETF community. It got public review and was approved for publication by the Internet Engineering Steering Group (IESG). Not every IESG-approved document is a candidate for any level of Internet Standard, so check Section 2 of RFC 5741 for that. 🫶
For the current status, any errata, and how to give feedback, go to http://www.rfc-editor.org/info/rfc6372. ✨
Copyright Notice ©️
Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved. 🌈
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as in effect on the date of publication. Please read these carefully, because they spell out your rights and restrictions. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they're provided without warranty as described in the Simplified BSD License. 💜
Table of Contents 📚
1. Introduction ....................................................4
1.1. Recovery Schemes ...........................................4
1.2. Recovery Action Initiation .................................5
1.3. Recovery Context ...........................................6
1.4. Scope of This Framework ....................................7
2. Terminology and References ......................................8
3. Requirements for Survivability .................................10
4. Functional Architecture ........................................10
4.1. Elements of Control .......................................10
4.1.1. Operator Control ...................................11
4.1.2. Defect-Triggered Actions ...........................12
4.1.3. OAM Signaling ......................................12
4.1.4. Control-Plane Signaling ............................12
4.2. Recovery Scope ............................................13
4.2.1. Span Recovery ......................................13
4.2.2. Segment Recovery ...................................13
4.2.3. End-to-End Recovery ................................14
4.3. Grades of Recovery ........................................15
4.3.1. Dedicated Protection ...............................15
4.3.2. Shared Protection ..................................16
4.3.3. Extra Traffic ......................................17
4.3.4. Restoration ........................................19
4.3.5. Reversion ..........................................20
4.4. Mechanisms for Protection .................................20
4.4.1. Link-Level Protection ..............................20
4.4.2. Alternate Paths and Segments .......................21
4.4.3. Protection Tunnels .................................22
4.5. Recovery Domains ..........................................23
4.6. Protection in Different Topologies ........................24
4.7. Mesh Networks .............................................25
4.7.1. 1:n Linear Protection ..............................26
4.7.2. 1+1 Linear Protection ..............................28
4.7.3. P2MP Linear Protection .............................29
4.7.4. Triggers for the Linear Protection
Switching Action ...................................30
4.7.5. Applicability of Linear Protection for LSP
Segments ...........................................31
4.7.6. Shared Mesh Protection .............................32
4.8. Ring Networks .............................................33
4.9. Recovery in Layered Networks ..............................34
4.9.1. Inherited Link-Level Protection ....................35
4.9.2. Shared Risk Groups .................................35
4.9.3. Fault Correlation ..................................36
5. Applicability and Scope of Survivability in MPLS-TP ............37
6. Mechanisms for Providing Survivability for MPLS-TP LSPs ........39
6.1. Management Plane ..........................................39
6.1.1. Configuration of Protection Operation ..............40
6.1.2. External Manual Commands ...........................41
6.2. Fault Detection ...........................................41
6.3. Fault Localization ........................................42
6.4. OAM Signaling .............................................43
6.4.1. Fault Detection ....................................44
6.4.2. Testing for Faults .................................44
6.4.3. Fault Localization .................................45
6.4.4. Fault Reporting ....................................45
6.4.5. Coordination of Recovery Actions ...................46
6.5. Control Plane .............................................46
6.5.1. Fault Detection ....................................47
6.5.2. Testing for Faults .................................47
6.5.3. Fault Localization .................................48
6.5.4. Fault Status Reporting .............................48
6.5.5. Coordination of Recovery Actions ...................49
6.5.6. Establishment of Protection and Restoration LSPs ...49
7. Pseudowire Recovery Considerations .............................50
7.1. Utilization of Underlying MPLS-TP Recovery ................50
7.2. Recovery in the Pseudowire Layer ..........................51
8. Manageability Considerations ...................................51
9. Security Considerations ........................................52
10. Acknowledgments ...............................................52
11. References ....................................................53
11.1. Normative References .....................................53
11.2. Informative References ...................................541. Introduction 🦋
Network survivability is the network's ability to recover traffic delivery after traffic delivery has failed or degraded. The cause can be a network fault or a denial-of-service attack on the network. 🔥
Survivability is a big deal for reliable services in transport networks. Guaranteed services, like the ones covered by Service Level Agreements (SLAs), need a resilient network. That network has to detect facility or node degradation or failures really quickly, and it has to start recovering right away according to the terms of the SLA. Main character energy, honestly. 👑
The MPLS Transport Profile (MPLS-TP) is described in [RFC5921]. It's designed to fit with existing transport network operations and management models, and it also provides survivability mechanisms like protection and restoration. The aim is for the functionality to be similar to or better than what you find in established transport networks, which set a super high bar for reliability. So operators get functions they already know from other transport networks. That doesn't rule out extra techniques, though. 🌸
This document gives a framework for MPLS-TP-based survivability that meets the recovery requirements in [RFC5654]. It uses the recovery terminology defined in [RFC4427], which draws heavily on [G.808.1], and it refers to the requirements in [RFC5654]. 💅
This document is also a product of a joint Internet Engineering Task Force (IETF) / International Telecommunication Union Telecommunication Standardization Sector (ITU-T) effort. It brings an MPLS Transport Profile into the IETF MPLS and PWE3 architectures, so they can support the capabilities and functionalities of a packet-based transport network, as defined by the ITU-T. 🫶
1.1. Recovery Schemes ✨
okay so the recovery schemes for protection and restoration were already defined and analyzed in [RFC4427] and [RFC4428] 💖 and guess what, they can also be used in MPLS-TP networks 🦋 to get end-to-end traffic delivery back on track according to the agreed service parameters 💅. They can also trigger recovery when a transport entity is "failed" or "degraded" 💀. For this doc, transport entities are nodes, links, transport path segments, concatenated transport path segments, and entire transport paths 🧚. Recovery actions start when a defect gets detected, or when someone external asks for them, like an operator who wants manual control of protection switching 🫶.
[RFC4427] makes a distinction between protection switching and restoration mechanisms 🌸.
- Protection switching uses pre-assigned capacity between nodes 🎀. The simplest version has one dedicated protection entity for each working entity, and the most complex version has m protection entities shared between n working entities (m:n) 👑.
- Restoration uses any capacity that's available between nodes and usually involves rerouting 🔥. The resources can be pre-planned, which means predetermined but not yet allocated to the recovery path 🍓. Recovery priority can also be used as a differentiation mechanism to decide which services get recovered and which don't 😭.
Both protection switching and restoration can be either unidirectional or bidirectional 🌈. Unidirectional means protection switching happens independently for each direction of a bidirectional transport path ✨. Bidirectional means both directions get switched at the same time using proper coordination 💜, even when the fault only hits one direction 🙈.
Both protection and restoration mechanisms can be either revertive or non-revertive, as described in Section 4.11 of [RFC4427] 🦋.
Preemption priority can decide which services get sacrificed so other services can recover 🔥. Restoration can also be unidirectional or bidirectional 💅. Usually, protection actions finish in tens of milliseconds ⚡, while automated restoration actions usually finish somewhere from hundreds of milliseconds up to a few seconds, max 💖. Restoration isn't guaranteed, though 💀. For example, the network resources might not be available at the moment of the defect 😭.
1.2. Recovery Action Initiation 🎀
so the recovery schemes in [RFC4427] and the ones evaluated in [RFC4428] are presented with control-plane-driven actions in mind 🧠, like configuring the protection entities and functions and stuff 🫶. A distributed control plane in an MPLS-TP network is optional 🌸. But if there's no control plane, the network still works fine, and you can still use MPLS-TP forwarding, Operations, Administration, and Maintenance (OAM), and survivability capabilities 💅. Basically, the concepts in [RFC4427] and [RFC4428] are about recovery actions in the data plane 🦋, so they apply to MPLS-TP either way, with or without a control plane ✨.
So some MPLS-TP recovery mechanisms don't need a control plane at all 👑. They use MPLS-TP OAM mechanisms or management actions to trigger recovery instead 🧚.
The principles of MPLS-TP protection switching are pretty similar to what [RFC4427] describes 💖, because protection works by detecting certain defects in the transport entities inside the recovery domain 🔥. The protection-switching controller doesn't care which initiation method is used 💅, as long as it can be told the status of the transport entities in the recovery domain (for example, OK, signal failure, signal degradation, etc.) 🫶.
In MPLS-TP it's absolutely essential that switchovers can happen no matter how the network is configured and managed 😭 (like whether a control-plane, management-plane, or OAM initiation mechanism is used) 🌈.
All MPLS and GMPLS protection mechanisms [RFC4428] apply to MPLS-TP environments 💜. You can also provision and manage the related protection entities and functions defined in MPLS and GMPLS using the management plane [RFC5654] 🎀. No matter which initiation mechanism is used (OAM, management, or control plane), protection switching is a data-plane operation 👑.
In some recovery schemes, like bidirectional protection switching ✨, the protection state has to be coordinated between the edges of the recovery domain so recovery can start for both directions 🦋. An MPLS-TP protocol can be used as an in-band (data-plane based) control protocol to do this coordination 🧚. When the MPLS-TP control plane is in use, a control-plane-based mechanism can also coordinate the protection states between the edges of the protection domain 🌸.
1.3. Recovery Context 🦋
an MPLS-TP Label Switched Path (LSP) might be covered by some or all of MPLS-TP link recovery, path-segment recovery, or end-to-end recovery 🌈. Here's what each one means 💅:
- MPLS-TP link recovery is about recovering one individual link between two MPLS-TP nodes, and that link's LSPs (all of them or just a subset) come along for the ride 🔥. For example, link recovery can be provided by server-layer recovery 🫶.
- Segment recovery is about recovering an LSP segment (a segment and a concatenated segment, in the language of [RFC5654]) between two nodes 🧚. It's used to recover from the failure of one or more links or nodes 💖.
- End-to-end recovery is about recovering an entire LSP, from its ingress node all the way to its egress node 👑.
For extra resiliency 💜, more than one of these recovery techniques can be set up at the same time on a single path 🎀.
Co-routed bidirectional MPLS-TP LSPs are defined so that both directions of the LSP follow the same route through the network 🌸. In this case, operators often want the directions to fate-share 🙈, which means if one direction fails, both directions should stop working 😭.
Associated bidirectional MPLS-TP LSPs are the ones where the two directions take different paths through the network 🦋. An operator can also ask for fate-sharing on associated bidirectional LSPs 💅.
Fate-sharing creates a direct link between the recovery processes for the two directions of an LSP 🔥, so both directions get recovered at the same time ✨. This mode of recovery is called bidirectional recovery, and it's basically a consequence of fate-sharing 🌈.
The data-plane recovery scheme can work in a multi-domain environment (using the broader sense of "domain" in [RFC4726]) 👑. It can also protect against a failure of a boundary node when you're doing inter-domain operation 🧚. MPLS-TP recovery schemes are meant to protect client services as they travel across the MPLS-TP network 💖.
1.4. Scope of This Framework 💅
this framework lays out the architecture of the MPLS-TP recovery domain 🏗️ and describes the MPLS-TP recovery schemes (based on the recovery types defined in [RFC4427]) 🎀. It also covers the principles of operation, recovery states, recovery triggers, and the information exchanges between the different elements that support the reference model 🫶.
The framework also describes the qualitative grades of survivability functions that can be provided, like dedicated recovery, shared protection, restoration, etc. ✨ When a network failure happens, the grade of recovery directly affects the service grade the end-user gets 👑.
The general description of the functional architecture applies to both LSPs and pseudowires (PWs) 🦋. However, PW recovery is only introduced in Section 7, and the relevant details are beyond the scope of this document and are for further study 🙈.
This framework applies to general recovery schemes, and also to mechanisms that are optimized for specific topologies and tailored to handle protection switching efficiently 🧚.
This document covers the need to coordinate protection switching across multiple layers and at sub-layers (for clarity, "layer" is used to mean both layers and sub-layers) 🌈. This lets an operator prevent race conditions 💀, and it lets the protection-switching mechanism of one layer recover from a failure before switching gets invoked at another layer 🔥.
This framework also specifies the functions that must be supported by MPLS-TP to provide the recovery mechanisms 💜. MPLS-TP brings a tool kit to enable recovery in MPLS-TP-based networks 🧰, so affected services get recovered when a failure happens 🫶.
Generally, network operators want the fastest, most stable, and best protection mechanism at a reasonable cost, based on what customers need 🌸. The greater the grade of protection required, the more resources get used up 😭. So it's expected that operators will offer a wide spectrum of service grades 🌈. MPLS-TP-based recovery gives you the flexibility to pick a recovery mechanism, decide the granularity at which traffic delivery is protected, and choose which traffic types get protected 💅. With MPLS-TP-based recovery, it should be possible to give different grades of protection to different traffic classes on the same path, depending on the service requirements 👑✨.
2. Terminology and References ✨
okay so bestie, this section is where the whole document sets up its vocab 🎀 the terms here are consistent with [RFC4427], and that one is consistent with [G.808.1]. ngl that's a lot of hand-holding between specs, but it's giving "everyone agrees on the same words" 💅
BUT lowkey there's a catch: ring protection and some other protection concepts aren't covered in [RFC4427], so for those the terms come from [G.841] instead 🦋 and the reader should go check those documents for the official, normative definitions. this doc just gives quick summaries so you're not lost, it does NOT redefine anything 🫶
Okay the spicy part 🌶️ the three big words from [RFC4427] that you need to know:
Protection: re-establishing end-to-end traffic delivery using pre-allocated resources. 🛡️
Restoration: re-establishing end-to-end traffic delivery using resources allocated at the time of need. Sometimes people call this "repair" of a service, LSP, or the traffic 🔧
Recovery: the umbrella term that covers both Protection AND Restoration 👑
Also, "survivability" is what [RFC5654] uses to cover the functional bits of protection and restoration, which all together get called "recovery" 🌈
If you want background on survivability, the bestie reading list is [RFC3386], [RFC3469], [RFC4426], [RFC4427], and [RFC4428] 📚🧚
Now the extra terms this doc adds on top:
- "Fault Management", as defined in [RFC5950]. 💜
- "defect" and "failure" are used interchangeably. They mean any defect or failure as defined in [G.806], and they also include any signal degradation event as defined in [G.806]. 🙈
- A "fault" is a fault or fault cause as defined in [G.806]. 🍓
- "Trigger" means any event that may kick off a recovery action. See Section 4.1 for the deeper dive on triggers ✨
- "OAM" stands for Operations, Administration, and Maintenance, consistent with [RFC6291]. 🌸
- A "Transport Entity" is a node, link, transport path segment, concatenated transport path segment, or entire transport path. 🦋
- A "Working Entity" is a transport entity that carries traffic during normal network operation. 💖
- A "Protection Entity" is a transport entity that's pre-allocated and used to protect and transport traffic when the working entity fails. 🫶
- A "Recovery Entity" is a transport entity that's used to recover and transport traffic when the working entity fails. 🔥
- "Survivability Actions" are the steps network nodes may take to communicate faults and switch traffic from faulted or degraded paths onto other paths. This can include sending messages and setting up new paths 💅
General MPLS-TP terminology lives in [RFC5921] and [ROSETTA] 🌈 and background on MPLS-TP requirements is in [RFC5654] 🎀
3. Requirements for Survivability 🌸
okay so MPLS-TP requirements are laid out in [RFC5654], and they're the normative references for ALL the MPLS-TP functionality, survivability included 👑 survivability gets described in [RFC5654] as playing a critical role in delivering reliable services, and its requirements are written using the recovery terminology from [RFC4427] 💖
4. Functional Architecture 🧚
this section gives you the overview of the survivability architecture inside an MPLS-TP network 🦋 the pieces are presented separately so you can see how they combine to give the different grades of recovery needed to meet the requirements in the previous section 💅
4.1. Elements of Control 🎀
recovery happens when specific actions get taken ✨ those actions aim to repair network resources or redirect traffic onto paths that dodge failures 🔧 they can be triggered automatically by MPLS-TP nodes when they detect a network defect, OR an operator can trigger them 👑 automated actions can be boosted by in-band (aka data-plane-based) OAM mechanisms, or by in-band or out-of-band control-plane signaling 🌈
4.1.1. Operator Control 💜
the survivability behavior of the whole network, and how each transport path reacts when a fault gets reported, can be controlled by the operator 🫶 this control splits into two buckets: policies and actions done when the transport path is set up, and commands used to control or force recovery actions on transport paths that already exist 🦋
the operator can set network-wide or local policies that decide what actions get taken when various defects are reported affecting different transport paths 🌸 also, when a service request causes one or more transport paths to be set up, the operator (or the requesting application) can define a particular grade of service. That grade gets mapped to specific survivability actions taken before and during path setup, after a failure of network resources is discovered, and when those resources recover 🔥
ngl it's kinda unusual to show a user or customer options directly about recovery actions 🙈 instead, the user/customer signs an SLA with the network provider, and the operator maps the SLA terms (like guaranteed delivery, availability, or reliability) to recovery schemes inside the network 💅
the operator can also issue commands to control recovery actions and events. For example, the operator may:
- Enable or disable the survivability function. 🍓
- Invoke the simulation of a network fault. 🧚
- Force a switchover from a working path to a recovery path, or vice versa. 💖
forced switchover can be done for network optimization purposes with minimal service interruption, like when modifying protected or unprotected services or replacing MPLS-TP network nodes 🌈 sometimes a fault gets reported to the operator, and then the operator can pick and start the right recovery action 👑 the different operator commands are described in Section 4.12 of [RFC4427] 🎀
4.1.2. Defect-Triggered Actions 🦋
survivability actions can be triggered directly by network defects 🔥 this means the device that detects the defect (for example, an issue reported from equipment in a lower layer, failing to receive an OAM Continuity message, or receiving an OAM message reporting a failure condition) can immediately perform a survivability action ✨
this action is triggered directly by events in the data plane 💜 heads up though: coordinating recovery actions between the edges of the recovery domain may need message exchanges for some recovery functions, or for performing a bidirectional recovery action 🫶
4.1.3. OAM Signaling 🌸
OAM signaling is data-plane OAM message exchange 🧚 these messages can detect and localize faults, or indicate a degradation in network operation. In this context though, the messages are used to control or trigger survivability actions 💅 the mechanisms for doing that are discussed in [RFC6371] 🦋
OAM signaling can also be used to coordinate recovery actions within the protection domain 💖
4.1.4. Control-Plane Signaling 👑
control-plane signaling handles setup, maintenance, and teardown of transport paths that do NOT fall under management-plane control 🌈 the control plane can also help coordinate detection, localization, and reaction to network defects for peer relationships (neighbor-to-neighbor or end-to-end). So control-plane signaling may initiate and coordinate survivability actions 🎀
the control plane can also distribute topology and resource availability info 💜 that means "graceful shutdown" [RFC5817] of resources can happen by withdrawing them, and this can invoke a survivability action the same way reporting or discovering a fault does, as described in the previous sections 🔥
the use of a control plane for MPLS-TP is discussed in [RFC6373] ✨
4.2. Recovery Scope 🦋
this section covers the elements of recovery: the quantitative bits, meaning the parts of the network where recovery can be provided 💅
the terminology here is consistent with [RFC4427] 🌸 where the terms differ from those in [RFC5654], a mapping is provided 🫶
4.2.1. Span Recovery 🧚
a span is a single hop between neighboring MPLS-TP nodes in the same network layer 💖 spans are sometimes called links, which can be confusing because a "link" might mean a data link OR a traffic engineering (TE) link 🙈 LSPs traverse TE links between neighboring MPLS-TP nodes in the MPLS-TP network layer 🌈 a TE link can be provided by any of these:
- A single data link. 🍓
- A series of data links in a lower layer, established as an LSP and presented to the upper layer as a single TE link. 🔥
- A set of parallel data links in the same layer, presented either as a bundle of TE links, or as a collection of data links that together provide a data-link-layer protection scheme. 👑
so span recovery can be provided by any of these:
- Selecting a different TE link from a bundle. 💜
- Moving the TE link so it's supported by a different data link between the same pair of neighbors. 🦋
- Rerouting the LSP in the lower layer. 🎀
moving the protected LSP to another TE link between the same pair of neighbors is a form of segment recovery, NOT span recovery 💀 segment recovery is described in Section 4.2.2 ✨
4.2.2. Segment Recovery 🌸
an LSP segment is one or more continuous hops on the path of the LSP 🧚 [RFC5654] defines two terms: a "segment" is a single hop along the path of an LSP, while a "concatenated segment" is more than one hop along it. In this doc, a segment covers BOTH of those concepts 💅
a PW segment refers to a Single-Segment PW (SS-PW), or to a single segment of a Multi-Segment PW (MS-PW) set up between two PE devices. Those PEs can be Terminating PEs (T-PEs) or Switching PEs (S-PEs), so the full set of possibilities is T-PE to S-PE, S-PE to S-PE, S-PE to T-PE, or T-PE to T-PE (for the SS-PW case) 🦋 as Section 1 said, recovery of PWs and PW segments is beyond the scope of this document, but check out Section 7 🫶
segment recovery means redirecting or copying traffic at the source end of a segment onto an alternate path that leads to the other end of the segment 🔥 depending on the required grade of recovery (described in Section 4.3), traffic may be redirected to a pre-established segment, through rerouting the protected segment, or tunneled to the far end of the protected segment through a "bypass" LSP 👑 for details on recovery mechanisms, see Section 4.4 💖
note that protecting a transport path against node failure requires segment recovery or end-to-end recovery, while a link failure can be protected using span, segment, or end-to-end recovery 🌈 🎀
4.2.3. End-to-End Recovery ✨
okay so end-to-end recovery is basically the special case of segment recovery where the protected segment is the WHOLE transport path 🫶 the recovery can be link-diverse or node-diverse, which just means the recovery path shares no links (or no nodes) with the working path. 🦋
ngl node-diverse paths are automatically link-diverse too, and if you want recovery to be guaranteed you need full, end-to-end node-diversity. no shortcuts bestie 💅
Two things to keep in mind about end-to-end recovery 👑
- First, node-diverse end-to-end paths don't always guarantee recovery. The ingress and egress nodes are themselves single points of failure, and there can be shared risks between diverse nodes (like geographic collocation or shared resources) as described in Section 4.9.2. 😭
- Second, you CAN use end-to-end recovery techniques even when there isn't full diversity, meaning the working and protection paths share some links or nodes. 🌸
4.3. Grades of Recovery 🌈
okay this section is all about the qualitative grades of survivability you can get 💖 when a network failure happens, the grade of recovery directly affects the service grade the end-user experiences. they'll notice it as how much data got lost during the fault and how long it takes to get connectivity back. 🙈
generally speaking, the recovery service grade (how fast you recover and how little data you lose) is correlated with how many network resources get used. better grades need resources pre-allocated to the recovery paths, and those resources can't be used for anything else if you want high-quality recovery. 💜 an operator has to think about how offering different grades may mean network resources get provisioned and set aside for the exclusive use of recovery paths, so they can't support other customer services. 🍓
Sections 6 and 7 of [RFC4427] have the full breakdown of the protection and recovery schemes ✨ this section just gives you the quick summary of the qualitative grades. 🧚
Also, when we're talking about recovery, there's a useful discussion of the term "resource" and how it's interpreted in both the IETF and ITU-T contexts in Section 3.2 of [RFC4397]. 💅
Picking the recovery grade and schemes to satisfy the service grades for an LSP (using whatever network resources are available) is subject to network and local policy. It can be pre-designated through network planning, or the network can determine it dynamically. 👑
4.3.1. Dedicated Protection 💎
in dedicated protection, the resources for the recovery entity are pre-assigned for the sole use of the protected transport path. 🔥 this is clearly the case with 1+1 protection, and it can also be the case with 1:1 protection when extra traffic (see Section 4.3.3) isn't supported. 🫶
when you're using protection tunnels (see Section 4.4.3), resources can also be dedicated to protecting a specific transport path. 🌸 in some cases (1:1 protection), the entire bypass tunnel can be dedicated to recovery for one specific transport path. in other cases (like facility backup), only a subset of the resources tied to the bypass tunnel gets pre-assigned for recovering one specific service. 🦋
BUT (and this is the tea 🍵), as Section 4.4.3 explains, the bypass tunnel method can ALSO be used for shared protection (Section 4.3.2). it can carry extra traffic (Section 4.3.3) or achieve best-effort recovery without any resource reservation. 🙌
4.3.2. Shared Protection 🤝
in shared protection, the resources for the recovery entities of several services are shared. 🌈 these can be shared as 1:n or m:n, and they're shared on individual links. Link-by-link resource sharing can be managed and operated along LSP segments, on PW segments, or on end-to-end transport paths (LSP or PW). 💜 side note: there's NO requirement for m:n recovery in the list of MPLS-TP requirements documented in [RFC5654]. 😭 shared protection can be applied in different topologies (mesh, ring, etc.) and can use different protection mechanisms (linear, ring, etc.). 🧚
end-to-end shared protection shares resources between a number of paths that have common end points. so a number of paths (n paths) are all protected by one or more protection paths (m paths, where m may equal 1). ✨ when there have been m failures, there are no more available protection paths, and the n paths are no longer protected. 💀 so in 1:n protection, one fault can be protected against before all the n paths are unprotected. 🍓
the fact that the paths have become unprotected needs to be conveyed to the path end points, since they may need to report the change in service grade or take further action to increase their protection. 🔥 in end-to-end shared protection this communication is simple, because the end points are common. 🫶
in shared mesh protection (see Section 4.7.6), the paths that share protection resources don't necessarily have the same end points. 🦋 this gives you a more flexible resource-sharing scheme, but the network planning and the coordination of protection state after a recovery action are more complex. 🌸
when a bypass tunnel is used (Section 4.4.3), the tunnel might not have enough resources to protect all of the paths it offers protection for at the same time. 🙈 if every path were hit by network defects and failures at once, not all of them would get recovered. 💔 policy decides how this gets handled: some paths might be protected while others simply fail, and traffic on some paths is guaranteed while traffic on other paths is treated as best-effort with the risk of dropped packets. 🍵 or, it's possible that protection wouldn't be attempted at all, according to local policy at the nodes doing the recovery actions. 💅
shared protection is a trade-off! 🌈 you're assigning network resources to protection (which isn't needed most of the time) and risking unrecoverable services if multiple network defects or failures happen. rapid recovery is achievable with dedicated protection, but it gets delayed by message exchanges in the management, control, or data planes for shared protection. so there's also a trade-off between rapid recovery and resource sharing. 💖 in some cases shared protection might not meet the speed required for protection, but it may still be faster than restoration. ✨
these trade-offs can be somewhat mitigated by the following 🎀
- Adjusting the value of n in 1:n protection.
- Using m:n protection for a value of m > 1.
- Establishing new protection paths as each available protection path is put into use.
in an MPLS-TP network, how much a resource is shared between LSPs is a policy issue 👑 this policy can be applied to the resource or to the LSPs, and it can be pre-configured, configured per LSP and installed during LSP establishment, or dynamically configured. 🧚
4.3.3. Extra Traffic 🦋
okay so Section 2.5.1.1 of [RFC5654] says: "Support for extra traffic (as defined in [RFC4427]) is not required in MPLS-TP and MAY be omitted from the MPLS-TP specifications". 😭 this document observes that extra traffic facilities may therefore be provided as part of the MPLS-TP survivability toolkit, depending on the development of suitable solution specifications. 🌸 the rest of this section explains the concepts of extra traffic without prejudging whether such solutions get specified or not. 💜
network resources allocated for protection are idle capacity while recovery isn't actually needed, and they can be used by carrying other traffic, called "extra traffic". 🍓 extra traffic doesn't need to start or terminate at the ends of the entity (e.g., LSP) that it uses. 🫶
when a network resource carrying extra traffic is needed to recover protected traffic from the failed working path, the extra traffic gets disrupted. 💀 this disruption can take one of two forms 🔥
- In "hard preemption", the extra traffic is excluded from the protection resource. the disruption is total, so the service supported by the extra traffic must be dropped, or some form of rerouting or restoration must be applied to the extra traffic LSP to recover the service. 😭 hard preemption is achieved by "setting a switch" on the path of the extra traffic so it no longer flows. this situation may be detected by OAM and reported as a fault, or it may be proactively reported through OAM or control-plane signaling. 🌈
- In "soft preemption", the extra traffic is NOT explicitly excluded from the protection resource, but it gets lower priority than the protected traffic. in a packet network (like MPLS-TP), this can lead to oversubscription of the protection resource, so the extra traffic receives "best-effort" delivery. ✨ depending on the volume of protection and extra traffic, and the level of oversubscription, the extra traffic may be slightly or heavily impacted. 🙈
the event of soft preemption may be detected by OAM and reported as a degradation of traffic delivery or as a fault. it may also be proactively reported through OAM or control-plane signaling. 🧚
note that both hard and soft preemption may use additional message exchanges in the management, control, or data planes. those messages don't necessarily mean recovery is delayed, but they can make the protection system more complex. 💅 so the benefits of carrying extra traffic must be weighed against the disadvantages of delayed recovery, additional network overhead, and the impact on the services that support the extra traffic, according to the details of the solutions selected. 🍵
note that extra traffic is not protected by definition, but it may be restored. 💔
extra traffic isn't supported on dedicated protection resources (which, by definition, are used for 1+1 protection, Section 4.3.1). it CAN be supported in other protection schemes, including shared protection (Section 4.3.2) and tunnel protection (Section 4.4.3). 🌟
best-effort traffic shouldn't be confused with extra traffic 🎀 for best-effort traffic, the network doesn't guarantee data delivery, and the user doesn't get guaranteed quality of service (e.g., in terms of jitter, packet loss, delay, etc.). best-effort traffic depends on the current traffic load. however, for extra traffic, quality can only be guaranteed until resources are needed for recovery. at that point, the extra traffic may be completely displaced, may be treated as best effort, or may itself be recovered (for example, by restoration techniques). 💖
4.3.4. Restoration ✨
okay bestie, this part is about LSP restoration 🦋 (PW restoration is out of scope for this doc, but go peek at Section 7 if you're curious).
restoration is the most efficient use of network resources ever 💅 because nothing gets reserved for recovery. the catch is that you have to compute a brand new path and activate a new LSP, either through the management plane or the control plane. doing all that can take way longer than just using protection techniques, ngl 😭
and there's no guarantee it works. all the good resources could already be taken by other LSPs, so no new path can be found 💀 one thing that helps a little is LSP setup priorities, so recovery LSPs can preempt existing LSPs that have lower priorities. it's giving "bump the less important people" 🫶
here's another messy bit. when a network defect happens, one event can knock out a bunch of LSPs at once. those LSPs might have been set up by different Network Management Stations (NMSes), or signaled by different head-end MPLS-TP nodes. so several points in the network try to compute and set up recovery LSPs at the same time 🙈 that can run the resources dry, cause recovery failures, and some recovery actions have to be retried, which makes things even slower for some services.
there are two flavors of restoration. with hard LSP restoration, the working LSP's resources are released before the recovery LSP is fully established (that's break-before-make 🍓). with soft LSP restoration, the working LSP's resources are released only after an alternate LSP is fully established (that's make-before-break 🌸). quick note: when reversion happens (Section 4.3.5), the resources tied to the working LSP are not released.
the restoration resources can be pre-calculated and even pre-signaled before the restoration action starts, but they are not pre-allocated. this is called pre-planned LSP restoration 🧚 the full establishment/activation of the restoration LSP only happens once the restoration action actually kicks off. pre-planning can run periodically and gives the most accurate picture of what's available in the network.
4.3.5. Reversion
after a service has been recovered and traffic is flowing on the recovery LSP, the broken network resource might get fixed or replaced 🎀 at that point you can send traffic back onto the original working LSP (this is called "reversion"), or leave it chilling on the recovery LSP (that's "non-revertive" behavior).
it should be possible to specify the reversion behavior for each service, and this could even be set up per recovery instance ✨
in non-revertive mode there's an extra operational option where the protection roles get switched. the recovery LSP becomes the working LSP, and the previous working path (or the resources it was using) gets reserved for recovery in case there's another fault 👑 main character energy, literally.
in revertive mode, you really want to stop the working and recovery paths from swapping back and forth too much when a defect is intermittent 💀 the fix is a reversion delay timer, called the Wait-To-Restore timer 🧘♀️ it controls how long to wait before reverting after the original working path has been repaired. an operator should be able to configure this timer per LSP, and a default value should be defined.
4.4. Mechanisms for Protection
this section gives general, MPLS-TP non-specific descriptions of the mechanisms you can use for protection 💜 like we said above, the functional architecture covers both LSPs and PWs, but the recovery mechanism in this document is only about LSPs and LSP segments. recovery mechanisms for pseudowires and pseudowire segments are still being studied and will get their own separate document (see Section 7 too 🌈).
4.4.1. Link-Level Protection
link-level protection comes in two paradigms: (1) protection provided in a lower network layer, and (2) protection provided by the MPLS-TP link layer 🦋
quick heads up: link-level protection doesn't protect the nodes at each end of the protected entity (like a link or span) 😭 so if you want to guard against the edge nodes failing, you need end-to-end or segment protection working alongside the link-level kind.
link-level protection offers these grades:
- Full protection, where a dedicated protection entity (like a link or span) is pre-established to protect a working entity. when the working entity fails, the protected traffic gets switched over to the protecting entity 🔥 every LSP carried over the working entity gets recovered in one protection operation. [RFC4427] calls this "bulk recovery" 💅
- Partial protection, where only some of the LSPs or traffic on a chosen entity get recovered when there's a failure. which ones get recovered is up to local policy 🎀
when the working entity is fine, the protection entity is allowed to carry extra traffic, which can be preempted when protection switching happens 🌸
if link-level protection is available, it can be worth trying it before other recovery mechanisms for the affected transport paths, because it's often faster and kinder to network resources 🧚 you can make that happen by limiting how far fault condition notifications spread and by delaying the other recovery actions. this idea can be compared with the discussion of recovery domains (Section 4.5) and recovery in multi-layer networks (Section 4.9) 🫶
a protection mechanism can also live at the MPLS-TP link layer (the layer connecting two MPLS-TP nodes) 🍓 that kind of mechanism can use the procedures in [RFC5586] to set up in-band communication channels at the MPLS-TP Section level. those channels can monitor the health of the MPLS-TP link and coordinate protection states between the two ends of it ✨
4.4.2. Alternate Paths and Segments
alternate paths and segments are all about protection happening in the same network layer where the protected LSP lives 🦋 this applies either to the whole end-to-end LSP or to just a segment of it. in this case, hierarchical LSPs are not used (compare with Section 4.4.3, which is the opposite vibe 💅).
there are different grades of protection:
- Dedicated protection, where a dedicated entity (like an LSP or LSP segment) is (fully) pre-established to protect a working entity (an LSP or LSP segment too). when a failure condition hits the working entity, traffic gets switched onto the protection entity 🔥 dedicated protection can use 1:1 or 1+1 linear protection schemes. once the failure condition clears, the traffic may revert to the working entity, but that depends on local configuration ⭐
- Shared protection, where one or more protection entities are pre-established to protect against a failure of one or more working entities (that's 1:n or m:n 👑).
once the fault on the working entity is gone, traffic should revert back to it. the reason is so the shared protection resource can go protect other related working entities 🌈
4.4.3. Protection Tunnels
a protection tunnel is pre-provisioned to protect against a failure condition along a sequence of spans in the network 🧚 this can be done using LSP hierarchy. we call that sequence a network segment. a failure on a network segment can affect one or more LSPs that transit it 💀
when a failure condition shows up in the network segment (spotted either by OAM on the network segment, or by OAM on a concatenated segment of one of the LSPs passing through it), one or more of the protected LSPs get switched over at the ingress point of the network segment and sent over the protection tunnel 🦋 this happens through label stacking. label mapping can be an option too 🎀
different grades of protection:
- Dedicated protection, where the protection tunnel reserves enough resources to protect all the protected LSPs without causing service degradation 💖
- Partial protection, where the protection tunnel has enough resources to protect some of the protected LSPs, but not all of them at the same time 🙈 policy decides what happens then. it's possible that some LSPs get protected while others just fail. it's possible that some LSPs get guaranteed traffic while others are treated as best effort, with the risk of packets being dropped. or it's possible that protection just wouldn't be attempted at all 😭
4.5. Recovery Domains ✨
okay so protection and restoration always happen inside a recovery domain 🦋 Think of it as a bubble with two or more recovery reference end points sitting right on its edges, touching the element that recovery can be provided for (more on that in Section 4.2). That element can be an end-to-end path, a segment, or a span 💅
An end-to-end path is basically just a special segment case, where the ingress and egress Label Edge Routers (LERs) act as the recovery reference end points 🌸
In the simple point-to-point (P2P) protected setup, there are two end points on the boundary of the protection domain 💖 An LSP can come in through one reference end point and leave the recovery domain through the other one 🎀
For unidirectional point-to-multipoint (P2MP), it gets a bit more crowded 👑 There are three or more end points on the boundary. One is called the source/root, and the rest are called sinks/leaves. An LSP can enter the recovery domain through the root and exit through the leaves 🧚
Here's the vibe: the recovery mechanism should restore traffic that got interrupted by a facility fault (a link or node going down) inside the recovery domain 🔥 A single link can be part of several recovery domains, which is totally normal. If two recovery domains share some links, then one of them has to be fully contained in the other one. That's called nested recovery domains 🪆 Their boundaries are allowed to line up, but recovery domains must not overlap 🙅♀️
Now the tea ☕ The edges of a recovery domain are not protected. Unless the whole domain sits inside another recovery domain, those edges are a single point of failure. Sad but true 💀
A recovery group lives inside a recovery domain. It has a working (primary) entity and one or more recovery (backup) entities, and all of them sit between the end points of the recovery domain 🌈 To guarantee protection in every situation, a dedicated recovery entity should be pre-provisioned on disjoint resources inside the domain, so it can cover a failure of the working entity 💜 You'll also need ways to detect faults and trigger the protection switch. Obviously ✨
How the health of the recovery element gets monitored is out of scope for this document 🙈 Whatever it is, the end points doing the recovery action have to receive info about its condition. That condition may be 'OK', 'failed', or 'degraded' 🍓
If OAM mechanisms are going to trigger the recovery, then an OAM Maintenance Entity Group must be defined for each of the working and protection entities 🫶
The recovery entities and functions in a recovery domain can be set up using either a management plane or a control plane 🎀 With the management plane, you configure the domain by setting the reference points, the working and recovery entities, and the recovery type (for example, 1:1 bidirectional linear protection, ring protection, and so on). You can also configure extra parameters for the recovery process. Full details are in Section 6.1 📝
With a control plane, the ingress LERs can talk to the recovery reference points and request that protection or restoration be configured across the recovery domain. Details are in Section 6.5 🦋
Now, multiple interconnections between separate recovery domains create a hierarchy 🪜 A single top-level recovery domain forms when you concatenate two recovery domains with multiple interconnections. In that setup, recovery actions can happen in the individual lower-level domains to protect any LSP segment crossing each one, and also in the higher-level domain to protect the longer LSP segment that travels through it 🌸
The MPLS-TP recovery mechanism can be arranged so the domains coordinate with each other 🤝 With interconnected rings, for example, it may be preferable to let the upstream ring recover before the downstream ring. That way, recovery happens in the ring where the defect actually occurred 💅 Coordination like this matters most in nested domains, and it's covered more in Section 4.9 ✨
4.6. Protection in Different Topologies 🌍
As the requirements in Section 3 and [RFC5654] lay out, the chosen recovery techniques can be optimized for different network topologies, but only if the optimized mechanisms perform significantly better than the generic ones in the same topology 💖
These mechanisms are required (R91 of [RFC5654]) to interoperate with the mechanisms defined for arbitrary topologies 🫶 The point is to allow end-to-end protection and make sure consistent protection techniques get used across the whole network. Here, 'interoperate' means that using one technique must not block another technique in an adjacent part of the network, for use on the same end-to-end transport path. It also must not prohibit the use of end-to-end protection mechanisms 🔥
The next sections (4.7 and 4.8) describe two different topologies and explain how recovery can look totally different in each scenario 👑 They also build out the recovery domain concept and show how end-to-end survivability can come from concatenating recovery domains, where each one gives some grade of recovery in part of the network 🧚
4.7. Mesh Networks 🕸️
A mesh network is any network with arbitrary interconnectivity between its nodes 🦋 People usually contrast mesh with more specific topologies like hub-and-spoke or ring (see Section 4.8), even though those are technically examples of mesh networks too 🙈 This section only covers protection techniques in the context of mesh networks. It does not include optimizations for specific topologies 💅
Linear protection is a protection mechanism that gives rapid and simple protection switching ⚡ In a mesh network, it's a really good fit because it can operate between any pair of points in the network. It can protect against a defect in a node, a span, a transport path segment, or an end-to-end transport path 💖 Linear protection also gives a clear indication of the protection status 🌈
Linear protection works inside a protection domain. A protection domain is a special type of recovery domain (see Section 4.5) tied to the protection function 🎀 It's made up of these architectural elements:
- A set of end points that reside at the boundary of the protection domain. In the simple case of 1:n or 1+1 P2P protection, two end points sit on the boundary. In each transmission direction, one end point is called the source and the other is called the sink. For unidirectional P2MP protection, three or more end points sit on the boundary. One is the source/root, and the others are the sinks/leaves 👑
- A Protection Group consists of one or more working (primary) paths and one or more protection (backup) paths running between the end points belonging to the protection domain. To guarantee protection in all scenarios, a dedicated protection path should be pre-provisioned to protect against a defect in a working path (that's the 1:1 or 1+1 protection schemes) 🧚 On top of that, the working and protection paths should be disjoint. The physical routes of the two should be physically diverse in every respect 💜
Heads up 🚨 If the resources of the protection path are less than those of the working path, the protection path may not have enough resources to protect the working path's traffic 😭
As mentioned in Section 4.3.2, the resources of the protection path may be shared as 1:n. In that case, the protection path won't have enough resources to protect all the working paths at a specific time 🍓
For bidirectional P2P paths, both unidirectional and bidirectional protection switching are supported 🫶 If a defect happens when bidirectional protection switching is defined, the protection actions are performed in both directions, even if the defect itself is unidirectional 🔥 The protection state is required to operate with a level of coordination between the end points of the protection domain 🤝
With unidirectional protection switching, protection actions are only performed in the affected direction 🦋
Revertive and non-revertive operations are both provided as options for the network operator 🎀
Linear protection supports the protection schemes described in the following sub-sections 🌸
4.7.1. 1:n Linear Protection ✨
okay so in the 1:1 scheme, a protection path gets allocated to guard against a defect, failure, or degradation in a working path 🛡️. like we said above, to actually guarantee protection, the protection entity should support the full capacity and bandwidth 💅. it can be configured to offer a degraded service compared to the working entity, though (say, because network resources are limited 🥺).
Figure 1 shows the 1:1 protection architecture 🎀. in normal conditions, data traffic goes over the working entity while the protection entity just chills in the idle state 😴. (OAM may run on the protection entity to check its state.) normal conditions are when there's no defect, failure, or degradation on the working entity, and no administrative configuration or request is making traffic flow over the protection entity.
|-----------------Protection Domain---------------|
==============================
/**********Working path***********\
+--------+ ============================== +--------+
| Node /| |\ Node |
| A {< | | >} B |
| | | |
+--------+ ============================== +--------+
Protection path
==============================
Figure 1: 1:1 Protection Architectureif there's a defect on the working entity or a specific administrative request, traffic gets switched over to the protection entity 🔄.
note that when you're operating with non-revertive behavior (see Section 4.3.5), once the conditions that caused the switchover have cleared, the traffic keeps flowing on the protection path 🌊. but the working and protection roles don't get swapped back 🙃.
in each transmission direction, the protection domain source bridges traffic onto the appropriate entity, while the sink selects traffic from the appropriate entity 🎯. the source and the sink need to coordinate the protection states so that bridging and selection happen to and from the same entity 🫶. for this reason, a signaling coordination protocol is required (either a data-plane in-band signaling protocol or a control-plane-based signaling protocol) 📡.
in bidirectional protection switching, both ends of the protection domain get switched to the protection entity, even when the fault is unidirectional 👀. this requires a protocol to coordinate the protection state between the two end points of the protection domain 💜.
when there's no defect, the bandwidth resources of the idle entity may be used for lower-priority traffic 🌈. when protection switching happens, that lower-priority traffic may be preempted by the protected traffic. this can happen by tearing down the lower-priority LSP, reporting a fault on the lower-priority LSP, or by treating the lower-priority traffic as best effort and discarding it when there's congestion 🗑️.
in the general case of 1:n linear protection, one protection entity is allocated to protect n working entities 🧚. the protection entity might not have enough resources to protect all the working entities that could be hit by faults at the same time 😬. in this case, to guarantee protection, the protection entity should support enough capacity and bandwidth to protect any of the n working entities 💪.
when defects or failures show up along multiple working entities, the entity to be protected should be prioritized 📋. the protection states between the edges of the protection domain should be fully coordinated so the behavior stays consistent 🤝. as explained in Section 4.3.5, revertive behavior is recommended when 1:n is supported 🔁.
4.7.2. 1+1 Linear Protection 💖
in the 1+1 protection scheme, a fully dedicated protection entity is allocated 👑.
as Figure 2 shows, data traffic is copied and fed at the source to both the working and the protection entities 👯. the traffic on the working and protection entities is transmitted simultaneously to the sink of the protection domain, where selection between the two entities happens (based on some predetermined criteria) 🎀.
|---------------Protection Domain---------------|
==============================
/**********Working path************\
+--------+ ============================== +--------+
| Node /| |\ Node |
| A {< | | >} Z |
| \| |/ |
+--------+ ============================== +--------+
\**********Protection path*********/
==============================
Figure 2: 1+1 Protection Architecturenote that control traffic between the edges of the protection domain (like OAM or a control protocol that coordinates the protection state, etc.) may be transmitted on an entity that's different from the one used for the protected traffic 📬. these packets should not be discarded by the sink 🚫.
in 1+1 unidirectional protection switching, there's no need to coordinate the protection state between the protection controllers at both ends of the protection domain 🙌. in 1+1 bidirectional protection switching, a protocol is required to coordinate the protection state between the edges of the protection domain 🤓.
in both protection schemes, traffic flows end-to-end on the working entity once the conditions that caused the switchover have cleared ✨. data selection may go back to selecting traffic from the working entity if reversion is enabled, and that will require coordinating the protection state between the edges of the protection domain 🔁. to avoid frequent switching caused by intermittent defects or failures when the network isn't stable, traffic is not selected from the working entity before the Wait-To-Restore (WTR) timer has expired ⏳.
4.7.3. P2MP Linear Protection 🌸
linear protection may be applied to protect unidirectional P2MP entities using the 1+1 protection architecture 🦋. the source/root MPLS-TP node bridges the user traffic to both the working and protection entities 🌊. each sink/leaf MPLS-TP node selects the traffic from one entity according to some predetermined criteria. note that when there's a fault condition on one of the branches of the P2MP path, some leaf MPLS-TP nodes may select the working entity while other leaf MPLS-TP nodes select traffic from the protection entity 🌿. (chaos, but make it make sense 💀)
in a 1:1 P2MP protection scheme, the source/root MPLS-TP node needs to identify whether there's a fault condition on any of the branches of the network 🔍. this means the sink/leaf MPLS-TP nodes need to notify the source/root MPLS-TP node of any fault condition 📣. this also necessitates a return path from the sinks/leaves back to the source/root MPLS-TP node 🛤️. when protection switching is triggered, the source/root MPLS-TP node selects the protection transport path for traffic transfer 🔀.
a form of "segment recovery for P2MP LSPs" could be constructed 🧩. given a P2MP LSP, you can protect any possible point of failure (link or node) using N backup P2MP LSPs. each backup P2MP LSP originates from the upstream node with respect to a different possible failure point, and terminates at all of the destinations downstream of that potential failure point 🎯. in case of a failure, traffic is redirected to the backup P2MP path 🏃♀️.
note that such mechanisms do not yet exist, and their exact behavior is for further study 🔮.
a 1:n protection scheme for P2MP transport paths is also required by [RFC5654]. such a mechanism is for future study 🕰️.
4.7.4. Triggers for the Linear Protection Switching Action 🚨
protection switching may be performed when:
- A defect condition is detected on the working entity, and the protection entity has "no" or an inferior condition. Proactive in-band OAM Continuity Check and Connectivity Verification (CC-V) monitoring of both the working and the protection entities may be used to enable the rapid detection of a fault condition. For protection switching, it is common to run a CC-V every 3.33 ms. In the absence of three consecutive CC-V messages, a fault condition is declared. In order to monitor the working and the protection entities, an OAM Maintenance Entity Group should be defined for each entity. OAM indications associated with fault conditions should be provided at the edges of the protection domain that is responsible for the protection-switching operation. Input from OAM performance monitoring that indicates degradation in the working entity may also be used as a trigger for protection switching. In the case of degradation, switching to the protection entity is needed only if the protection entity can exhibit better operating conditions. 🔍
- An indication is received from a lower-layer server that there is a defect in the lower layer. ⬇️
- An external operator command is received (e.g., 'Forced Switch', 'Manual Switch'). For details, see Section 6.1.2. 🕹️
- A request to switch over is received from the far end. The far end may initiate this request, for example, on receipt of an administrative request to switch over, or when bidirectional 1:1 protection switching is supported and a defect occurred that could only be detected by the far end, etc. 📩
as described above, the protection state should be coordinated between the end points of the protection domain 🤝. control messages should be exchanged between the edges of the protection domain to coordinate the protection state of the edge nodes 💌. control messages can be delivered using an in-band, data-plane-driven control protocol or a control-plane-based protocol 📡.
for 50-ms protection switching, it is recommended that an in-band, data-plane-driven signaling protocol be used to coordinate the protection states ⚡. an in-band, data-plane protocol for use in MPLS-TP networks is documented in [MPLS-TP-LP] for linear protection (ring protection is discussed in Section 4.8 of this document) 🌀. this protocol is also used to detect mismatches between the configurations provisioned at the ends of the protection domain 🕵️♀️.
as described in Section 6.5, the GMPLS control plane already includes procedures and message elements to coordinate the protection states between the edges of the protection domain 🧠. these procedures and protocol messages are specified in [RFC4426], [RFC4872], and [RFC4873]. however, these messages lack the capability to coordinate the revertive/non-revertive behavior and the consistency of configured timers at the edges of the protection domain (timers such as WTR, hold-off timer, etc.) 😤.
4.7.5. Applicability of Linear Protection for LSP Segments 🎀
Okay so, to do data-plane-based linear protection on LSP segments, we use the Sub-Path Maintenance Element (SPME) 🦋, which is an MPLS-TP architectural element defined in [RFC5921]. Maintenance operations (like monitoring, protection, or management) plug into message transmission (OAM, Protection Path Coordination, etc.) inside the domain being maintained. You can read more about the OAM and SPME architecture in [RFC5921] and [RFC6371]. An SPME is basically an LSP that exists only for OAM monitoring, protection, or management of LSP segments, and it uses the hierarchical, nested LSP construct from [RFC3031]. It's giving russian doll 🪆✨
For linear protection, SPMEs should be defined over the working and protection entities between the edges of a protection domain. OAM messages and messages that coordinate protection state can be started at the edge of the SPME and sent to the peer edge of the SPME. Note that these messages travel over the Generic Associated Channel (G-ACh) within the SPME, and they use a two-label stack: the SPME label and, at the bottom of the stack, the G-ACh label (GAL) [RFC5586]. Ngl, the bottom-of-stack label is doing a lot here 💜
The end-to-end traffic of the LSP is the whole package: data traffic plus control traffic (messages for OAM, management, signaling, and coordinating protection state). All of it gets tunneled inside the SPMEs using label stacking, as defined in [RFC3031] 🧚
Mapping between an LSP and an SPME can be 1:1. This is similar to the ITU-T Tandem Connection element, which defines a sub-layer corresponding to a segment of a path. It can also be 1:n, which lets you protect a whole set of LSP segments crossing the part of the network where a protection domain is defined, and that's how you keep it scalable 📈. Each of these LSPs can start or end at different endpoints in the network, but they all traverse the protection domain and share similar constraints (like requirements for quality of service (QoS), terms of protection, etc.) 💅
Also, in the context of segment protection, the SPMEs serve as the working and protection entities 🫶
4.7.6. Shared Mesh Protection 🌸
For shared mesh protection, the protection resources are used to protect multiple LSPs that do not all share the same end points. For example, in Figure 3 there are two paths, ABCDE and VWXYZ. These paths don't share end points, so they can't use 1:n linear protection, even though they have no common points of failure 😭
ABCDE may be protected by the path APQRE, while VWXYZ can be protected by the path VPQRZ. In both cases, 1:1 or 1+1 protection may be used. Here's the tea though: if 1:1 protection is used for both paths, the PQR network segment doesn't carry traffic when no failures affect either of the two working paths. And if only one failure happens, the PQR segment carries traffic from just one of the working paths 👀
So the network resources on the PQR segment can be shared by the two recovery paths. That's the magic of mesh protection: it can substantially reduce the number of network resources that have to be reserved in order to provide 1:n protection 🔥
A----B----C----D----E
\ /
\ /
\ /
P-----Q-----R
/ \
/ \
/ \
V----W----X----Y----Z
Figure 3: A Shared Mesh Protection TopologyAs the network gets more complex and the number of LSPs goes up, the potential for shared mesh protection goes up too. However, this can quickly become unmanageable because of the added complexity 😵💫 That's why shared mesh protection is normally pre-planned and configured by the operator, although an automated system cannot be ruled out 🤖
Note that shared mesh protection operates as 1:n linear protection (see Section 4.7.1). However, the protection state needs to be coordinated between a larger number of nodes: the end points of the shared concatenated protection segment (nodes P and R in the example), as well as the end points of the protected LSPs (nodes A, E, V, and Z in the example) 🧩
Additionally, the shared-protection resources could be used to carry extra traffic! For example, in Figure 4, an LSP JPQRK could be a preemptable LSP that is extra traffic over the PQR hops. It would be displaced in the event of a protection event 🙈 In this case, the protection state must also be coordinated with the ends of the extra-traffic LSPs, so don't forget them bestie 💖
A----B----C----D----E
\ /
\ /
\ /
J-----P-----Q-----R-----K
/ \
/ \
/ \
V----W----X----Y----Z
Figure 4: Shared Mesh Protection with Extra Traffic4.8. Ring Networks 💍
Several service providers have expressed great interest in running MPLS-TP in ring topologies. They demand a high degree of survivability functionality in these topologies, and honestly I respect the energy 👑
There are various criteria for optimization that get considered in ring topologies, such as:
- Simplification in ring operation in terms of the number of OAM Maintenance Entities that are needed to trigger the recovery actions, the number of recovery elements, the number of management-plane transactions during maintenance operations, etc.
- Optimization of resource consumption around the ring, such as the number of labels needed for the protection paths that traverse the network, the total bandwidth required in the ring to ensure path protection, etc. (see R91 of [RFC5654]).
[RFC5654] introduces a list of requirements for ring protection. These cover the recovery mechanisms needed to protect traffic in a single ring, as well as traffic that traverses more than one ring. Note that the configuration and operation of the recovery mechanisms in a ring must scale well with the number of transport paths, the number of nodes, and the number of ring interconnects 🌈
The requirements for ring protection are fully compatible with the generic requirements for recovery. Slay 💅
The architecture and the mechanisms for ring protection are specified in separate documents. These mechanisms need to be evaluated against the requirements specified in [RFC5654], which includes guidance on the principles for developing new mechanisms 📚
4.9. Recovery in Layered Networks 🍰
In multi-layer or multi-regional networking [RFC5212], recovery may be performed at multiple layers or across nested recovery domains. Layers within layers, it's a whole vibe 🪺
The MPLS-TP recovery mechanism must ensure that the timing of recovery is coordinated in order to avoid race scenarios. This also lets the recovery mechanism of the server layer fix the problem before recovery happens in the MPLS-TP layer, or lets the MPLS-TP layer perform recovery before a client network does ⏳
A hold-off timer is required to coordinate recovery timing in multiple layers or across nested recovery domains. Setting this configurable timer is a trade-off between rapid recovery and the creation of a race condition where multiple layers respond to the same fault, potentially allocating resources in an inefficient manner 🥴 So, the detection of a defect condition in the MPLS-TP layer should not immediately trigger the recovery process if the hold-off timer is configured as a value other than zero. Instead, the hold-off timer should be started when the defect is detected. On expiry, the recovery element should be checked to determine whether the defect condition still exists. If it does exist, the defect triggers the recovery operation 🎯
The hold-off timer should be configurable ⭐
In other configurations, where the lower layer does not have a restoration capability, or where it is not expected to provide protection, the lower layer needs to trigger the higher layer to immediately perform recovery. Although this can be forced by configuring the hold-off timer as zero, because of layer independence the higher layer may not know whether the lower layer will perform restoration. In this case, the higher layer will configure a non-zero hold-off timer and rely on receiving a specific notification from the lower layer if the lower layer cannot perform restoration. Since layer boundaries are always within nodes, this coordination is implementation-specific and doesn't need to be covered here 🙌
Reference should be made to [RFC3386], which discusses the interaction between layers in survivable networks 📖
4.9.1. Inherited Link-Level Protection 🦋
Where a link in the MPLS-TP network is formed through connectivity (i.e., a packet or non-packet LSP) in a lower-layer network, that connectivity may itself be protected. For example, the LSP in the lower-layer network may be provisioned with 1+1 protection. In this case, the link in the MPLS-TP network has an inherited grade of protection 🌷
An LSP in the MPLS-TP network may be provisioned with protection in the MPLS-TP network, as already described, or it may be provisioned to utilize only those links that have inherited protection.
By classifying the links in the MPLS-TP network according to the grade of protection they inherited from the server network, you can compute an end-to-end path in the MPLS-TP network that uses only those links with a specific or superior grade of inherited protection. This means the end-to-end MPLS-TP LSP can be protected at the grade needed to conform to the SLA without providing any additional protection in the MPLS-TP layer. That reduces complexity, saves network resources, and eliminates protection-switching coordination problems. Big brain energy 🧠✨
When the requisite grade of inherited protection is not available on all segments along the path in the MPLS-TP network, segment protection may be used to achieve the desired protection grade.
It should be noted, however, that inherited protection only applies to links. Nodes cannot be protected in this way. An operator will need to perform an analysis of the relative likelihood and consequences of node failure if this approach is taken without providing protection in the MPLS-TP LSP or PW layer to handle node failure. Not me going "wait, what about the nodes?" 😳 Do the analysis, babe 🫶
4.9.2. Shared Risk Groups 🔥
okay bestie, so when you set up an MPLS-TP protection scheme, the working and protection paths absolutely cannot share resources in the network 😭 If they do, one single defect can take out BOTH paths at once, and then the traffic just can't get delivered, because it was never actually protected 💀 not me...
Note that this restriction does not apply to restoration 🦋 Restoration happens after the fault already occurred, so the point of failure can be dodged if an available path exists ✨
When you're planning a recovery scheme, you can look at a topology map of the MPLS-TP layer to pick paths that use diverse links and nodes 🗺️ BUT that doesn't guarantee the paths are truly diverse, ngl. For example, two separate links in an MPLS-TP network might be two lambdas riding in the same optical fiber, or two fibers crossing the same bridge 🌉 And two totally separate MPLS-TP nodes could be chilling in the same building on a shared power supply 🔌 yikes.
So, for proper recovery planning, the MPLS-TP network has to understand the groups of lower-layer resources that share a common risk of failure 🧠 From that, MPLS-TP shared risk groups can be built to show which MPLS-TP resources share a common risk. Then diversity of working and protection paths can be planned, not just with nodes and links, but also by refraining from using resources from the same shared risk groups 👑
4.9.3. Fault Correlation 🧩
In a layered network, a low-layer fault can be detected and reported by multiple layers, and it can sometimes lead to multiple fault reports from the same layer 😵💫 For example, a failure of a data link might get reported by the line cards in an MPLS-TP node, but it could also be detected and reported by the MPLS-TP OAM.
Section 4.6 explains why it matters to coordinate the survivability actions configured and operated in a multi-layer network 🎀 The goal is to avoid over-equipping the survivability resources, while making sure recovery actions happen in only one layer at a time ✨
Fault correlation is about figuring out which single event generated a set of fault reports, so recovery actions can be coordinated and the fault logging system doesn't get overloaded 📉 It depends on understanding resource use at lower layers, shared risk groups, and a wider view of how the layers are interrelated 🌈
Fault correlation is easiest at the point of fault detection 💅 For example, an MPLS-TP node that gets a fault notification from the lower layer, and also detects a fault on an LSP in the MPLS-TP layer, can easily correlate those two events. And if the same node detects multiple faults on LSPs that share the same faulty data link, it can correlate those too 🫶 That node can use correlation to perform group-based recovery actions, and it can cut down the number of alarm events it sends to its management station 🔔
Fault correlation can also happen at a management station that receives fault reports from different layers and different nodes in the network 👑 This lets the management station coordinate management-originated recovery actions and present consolidated fault information to the user and automated management systems 💜
It's also necessary to correlate fault information detected and reported through OAM 🔍 This function lets a fault detected at a lower layer, and reported at a transit node of an MPLS-TP LSP, be correlated with an MPLS-TP-layer fault detected at a Maintenance End Point (MEP), for example the egress of the MPLS-TP LSP 🧚 That correlation allows coordination of recovery actions performed at the MEP. It also requires that the lower-layer fault information gets propagated to the MEP, which is most easily achieved using a control plane, management plane, or OAM message 📨
5. Applicability and Scope of Survivability in MPLS-TP 🌸
The MPLS-TP network can be viewed as two layers: the MPLS LSP layer and the PW layer 🥞 It operates over data-link connections and data-link networks, where the MPLS-TP links are provided by individual data links or by connections in a lower-layer network. The MPLS LSP layer is a mandatory part of the MPLS-TP network, while the PW layer is an optional add-on for supporting specific services 💖
MPLS-TP survivability provides recovery from failure of the links and nodes in the MPLS-TP network 🛟 Link defects and failures are typically caused by defects or failures in the underlying data-link connections and networks, but this section only cares about recovery actions performed in the MPLS-TP network. That network has to recover from the manifestation of any problem as a defect failure in the MPLS-TP network 🙈
This section lists the recovery elements (see Section 1) supported in each of the two layers that can recover from defects or failures of nodes or links in the MPLS-TP network 📋 Here's the breakdown:
+--------------+---------------------+------------------------------+
| Recovery | MPLS LSP Layer | PW Layer |
| Element | | |
+--------------+---------------------+------------------------------+
| Link | MPLS LSP recovery | The PW layer is not aware of |
| Recovery | can be used to | the underlying network. |
| | survive the failure | This function is not |
| | of an MPLS-TP link. | supported. |
+--------------+---------------------+------------------------------+
| Segment/Span | An individual LSP | For an SS-PW, segment |
| Recovery | segment can be | recovery is the same as |
| | recovered to | end-to-end recovery. |
| | survive the failure | Segment recovery for an MS-PW|
| | of an MPLS-TP link. | is for future study, and |
| | | this function is now |
| | | provided using end-to-end |
| | | recovery. |
+--------------+---------------------+------------------------------+
| Concatenated | A concatenated LSP | Concatenated segment |
| Segment | segment can be | recovery (in an MS-PW) is for|
| Recovery | recovered to | future study, and this |
| | survive the failure | function is now provided |
| | of an MPLS-TP link | using end-to-end recovery. |
| | or node. | |
+--------------+---------------------+------------------------------+
| End-to-End | An end-to-end LSP | End-to-end PW recovery can |
| Recovery | can be recovered to | be applied to survive any |
| | survive any node or | node (including S-PE) or |
| | link failure, | link failure, except for |
| | except for the | failure of the ingress or |
| | failure of the | egress T-PE. |
| | ingress or egress | |
| | node. | |
+--------------+---------------------+------------------------------+
| Service | The MPLS LSP layer | PW-layer service recovery |
| Recovery | is service- | requires surviving faults in |
| | agnostic. This | T-PEs or on Attachment |
| | function is not | Circuits (ACs). This is |
| | supported. | currently out of scope for |
| | | MPLS-TP. |
+--------------+---------------------+------------------------------+
Table 1: Recovery Elements Supported
by the MPLS LSP Layer and PW LayerSection 6 provides a description of mechanisms for MPLS-TP-LSP survivability 🔧 Section 7 gives a brief overview of mechanisms for MPLS-TP-PW survivability 🎀
6. Mechanisms for Providing Survivability for MPLS-TP LSPs 🛡️
This section describes the existing mechanisms that provide LSP protection within MPLS-TP networks, and it highlights areas where new work is required 👀
6.1. Management Plane 🎛️
As described above, a fundamental requirement of MPLS-TP is that recovery mechanisms should be capable of functioning in the absence of a control plane 🙌 Recovery may be triggered by MPLS-TP OAM fault management functions or by external requests (e.g., an operator's request for manual control of protection switching). Recovery LSPs (and in particular Restoration LSPs) may be provisioned through the management plane 🧑💻
The management plane may be used to configure the recovery domain by setting the reference end-point points (which control the recovery actions), the working and the recovery entities, and the recovery type (e.g., 1:1 bidirectional linear protection, ring protection, etc.) 🧩
Additional parameters associated with the recovery process (such as WTR and hold-off timers, revertive/non-revertive operation, etc.) may also be configured ⏲️
In addition, the management plane may initiate manual control of the recovery function 🕹️ A priority should be set for the fault conditions and the operator's requests 📊
Since provisioning the recovery domain involves picking a number of options, mismatches may occur at the different reference points 😬 The MPLS-TP protocol to coordinate protection state, which is specified in [MPLS-TP-LP], may be used as an in-band (i.e., data-plane-based) control protocol. It can coordinate the protection states between the end points of the recovery domain, and check the consistency of configured parameters (such as timers, revertive/non-revertive behavior, etc.). Any discovered inconsistencies are reported to the operator 🔍
It should also be possible for the management plane to track the recovery status, either by receiving reports or by issuing polls 📡
6.1.1. Configuration of Protection Operation ✨
okay so before any protection switching can happen, a bunch of stuff has to be set up 🎀 here's the checklist:
- The end points of a recovery domain. like we said above, these end points border on the element of recovery that recovery is applied to 💅
- The protection group 🫶 depending on the protection scheme you need, it's made of a recovery entity plus one or more working entities. for 1:1 or 1+1 P2P protection, the paths of the working entity and the recovery entities must be physically diverse in every respect (so no sharing any resources or physical locations) or protection isn't actually guaranteed 😭
- the SPME must be supported if you want data-plane-based LSP segment recovery 🧚 bc related control messages (like OAM, Protection Path Coordination, etc.) can only be started and ended at the edges of a path where push and pop operations are enabled. the SPME is an end-to-end LSP that, in this context, corresponds to the recovery entities (working and protection) and uses the MPLS construct of hierarchical nested LSP, as defined in [RFC3031]. OAM messages and messages that coordinate protection state can be initiated at the edge of the SPME and sent over G-ACH to the peer edge of the SPME. you have to configure the related SPMEs and map between the LSP segments being protected and the SPME 🌸 the mapping can be 1:1 or 1:N, which lets you protect a whole set of LSP segments that go through the part of the network where a protection domain is defined, and that's way more scalable 🔥
ngl the vibe here is: each of these LSPs can start or end at different end points in the network, but they all go through the protection domain and share similar constraints (like QoS requirements, terms of protection, etc.) 💜
- The protection type that should be defined (e.g., unidirectional 1:1, bidirectional 1+1, etc.) 🌈
- Revertive/non-revertive behavior should be configured. 🦋
- Timers (such as WTR, hold-off timer, etc.) should be set. ⭐
6.1.2. External Manual Commands 🎀
these are the external, manual commands you can use to control the protection switching by hand 🫶 they apply to a protection group, and they're listed in descending order of priority (so the first one wins, periodt 👑):
- Blocked protection action - a manual command that stops data traffic from switching to the recovery entity. this one actually disables the protection group 🙈
- Force protection action - a manual command that forces normal data traffic to switch to the recovery entity. 💅
- Manual protection action - a manual command that forces data traffic to switch to the recovery entity only when there's no defect in the recovery entity. 🦋
- Clear switching command - the operator can ask for a previous administrative switch command (manual or force switch) to be cleared. 🧚
6.2. Fault Detection 💖
fault detection is a fundamental part of recovery and survivability, no cap 😭 in every scheme (except some types of 1+1 protection), the actions needed to get traffic delivery back depend on finding some kind of fault first. in 1+1 protection, the selector at the receiving end may just be configured to pick the better signal. so it doesn't detect a fault or degradation in itself, it just figures out which path is better for delivery 🌸
faults can be detected in a bunch of ways depending on the traffic pattern and the hardware underneath. end-to-end faults can be reported by the application, or by knowing the application's data pattern, but that's a pretty unusual approach 👀 there are two more common mechanisms for detecting faults in the MPLS-TP layer:
- Faults reported by the lower layers. 🌈
- Faults detected by protocols within the MPLS-TP layer. 🔥
in an IP/MPLS network, that second mechanism can use control-plane protocols (like the routing protocols) to detect a failure of adjacency between neighboring nodes. in an MPLS-TP network, there might be no control plane at all 💀 and even if there is one, it'll be a GMPLS control plane [RFC3945], which logically separates control channels from data channels. so you can't conclude anything about the health of a data channel just because its associated control channel failed 🙅♀️ so MPLS-TP-layer faults are only detected through OAM protocols, as described in Section 6.4.1. 💜
faults can still come from a lower layer, though 🫶 these usually show up as interface failures or data-link failures (sometimes called connectivity failures) inside the MPLS-TP network. like, an underlying optical link might detect loss of light and report that the MPLS-TP link using it has failed. or an interface card failure might get reported up to the MPLS-TP layer ⚡
here's the catch: faults reported by lower layers are only visible at specific nodes in the MPLS-TP network (the adjacent end points of the MPLS-TP link). that only lets recovery happen locally. so, for recovery to be done by nodes that aren't right next to the fault, the fault has to be reported (see Sections 6.4.3 and 6.5.4) 🎀
6.3. Fault Localization 🔍
if an MPLS-TP node detects a fault in an LSP (not a network fault reported from a lower layer, but a fault found by examining the LSP itself), it can immediately do a recovery action 🧚 but unless the fault's location is known, the only practical options are:
- Perform end-to-end recovery. 💅
- Perform some other recovery as a speculative act. 🙈
speculative acts aren't guaranteed to work, and they can burn resources for no reason. and end-to-end recovery can need a LOT of network resources 😭 so being able to localize the fault is a big deal 👑
fault localization can be done by dividing the network into protection domains. end-to-end protection then works on LSP segments, depending on which domain the fault was found in. this means the LSP has to be monitored at the domain edges 🌸
or you can go with a proactive way of localizing faults, through OAM (Section 6.4.3) or through the control plane (Section 6.5.3). one of those is required 💖
fault localization matters a lot for restoration, because a new path has to be picked that avoids the fault. it might not be practical or desirable to avoid the entire failed working path, so isolating exactly where the fault is becomes necessary 🦋
6.4. OAM Signaling 🌈
MPLS-TP has a comprehensive set of OAM tools for fault management and performance monitoring at different nested levels (end-to-end, a portion of a path (LSP or PW), and at the link level) [RFC6371] 🔥
these tools support proactive and on-demand fault management (for fault detection and fault localization), plus performance monitoring (to measure signal quality and catch degradation) ✨
to support fast recovery, it's useful to use some of the proactive tools to detect fault conditions (like link/node failure or degradation) and to trigger the recovery action 🫶
the MPLS-TP OAM messages run in-band with the traffic, and they support unidirectional and bidirectional P2P paths as well as P2MP paths 🧚
as [RFC6371] explains, MPLS-TP OAM works in the context of a Maintenance Entity, which borders on the OAM responsibilities and represents the portion of a path between two points that's monitored and maintained, and along which OAM messages get exchanged 👀 [RFC6371] also talks about a Maintenance Entity Group (MEG), which is a collection of one or more Maintenance Entities (MEs) that belong to the same transport path (like a P2MP transport path) and get maintained and monitored as a group 💅
an ME includes two MEPs (Maintenance Entity Group End Points), which live at the boundaries of an ME, plus a set of zero or more MIPs (Maintenance Entity Group Intermediate Points) that sit inside the Maintenance Entity along the path. a MEP can start and end OAM messages, so it can only be located at the edges of a path where push and pop operations are supported. to define an ME over a portion of a path, you need to support SPMEs 🌸
the SPME is an end-to-end LSP that, in this context, corresponds to the ME. it uses the MPLS construct of hierarchical nested LSPs, which is defined in [RFC3031] 💜 OAM messages can be initiated at the edge of the SPME and sent over G-ACH to the peer edge of the SPME 🎀
the related SPMEs must be configured, and the mapping has to happen between the LSP segments being monitored and the SPME. the mapping can be 1:1 or 1:N to allow scalable operation. again, each of these LSPs can start or end at different end points in the network, and they can share similar constraints (like QoS requirements, terms of protection, etc.) 🦋
when it comes to recovery, and where MPLS-TP OAM is supported, an OAM Maintenance Entity Group is defined for each of the working and protection entities 👑
6.4.1. Fault Detection 🚨
MPLS-TP OAM tools may be used proactively to detect the following fault conditions between MEPs:
- Loss of continuity and misconnectivity - the proactive Continuity Check (CC) function is used to detect loss of continuity between two MEPs in an MEG. the proactive Connectivity Verification (CV) lets a sink MEP detect a misconnectivity defect (like a mismerge or misconnection) with its peer source MEP when the received packet carries an incorrect ME identifier. for protection switching, it's common to run a CC-V (Continuity Check and Connectivity Verification) message every 3.33 ms 🔥 if three consecutive CC-V messages are missing, loss of continuity is declared and gets reported locally to the edge of the recovery domain to trigger a recovery action 😭 in some cases, when a slower recovery time is okay, you can also lengthen the transmission rate 🐢
- Signal degradation - a notification from OAM performance monitoring that the working entity is degrading can also be used as a trigger for protection switching. when degradation happens, switching to the recovery entity is only necessary if the recovery entity can guarantee better conditions 💅 degradation can be measured by proactively turning on MPLS-TP OAM packet loss measurement or delay measurement 🌈
- A MEP can receive a Remote Defect Indication from its sink MEP and locally notify the end point of the recovery domain about the fault condition, so the recovery action gets triggered 🫶
6.4.2. Testing for Faults 🧪
okay so the management plane can kick off testing on links, LSP segments, or entire LSPs 🔥 it's literally the "go check it" button 💅
MPLS-TP has OAM tools you can invoke manually, on-demand, for a limited period, to troubleshoot links, LSP segments, or entire LSPs (think diagnostics, connectivity verification, packet loss measurements, etc.) ✨ On-demand monitoring is a combo of "in-service" and "out-of-service" monitoring functions. Out-of-service testing is handled by the OAM on-demand lock operation 🔒. The lock temporarily disables the transport entity (LSP, LSP segment, or link) and stops all types of traffic from going through, except test traffic and OAM dedicated to the locked entity. bestie that's closed-for-maintenance energy 🚧
[RFC6371] describes how the OAM functions you can start on-demand work, and it shares some considerations 📖
MPLS-TP also does in-service and out-of-service testing for the recovery mechanism (protection and restoration) 🛡️, the integrity of the protection/recovery transport paths 🛤️, and the coordination protocol between the end points of the recovery domain 🤝. The testing operation emulates a protection-switching request but doesn't actually do the switching action 🎭 it's a rehearsal, not the real show 💜
6.4.3. Fault Localization 🔍
MPLS-TP gives you OAM tools to locate a fault and pin down its precise location 📍 Fault detection often only happens at key points (like LSP end points or MEPs), so a fault could be anywhere within a segment of the relevant LSP 😵 Finer information granularity is needed to do optimal recovery actions or diagnose the fault 🩺 On-demand tools like trace-route, loopback, and on-demand CC-V can be used to localize it 🎯
The information may be notified locally to the end point of the recovery domain so it can carry out an optimal recovery action 🧠 This can be useful for recalculating a recovery path 🛣️
The information should also be reported to network management for diagnostic purposes 📊
6.4.4. Fault Reporting 📣
The end points of a recovery domain should be able to detect fault conditions in the recovery domain and notify the management plane 🙋
On top of that, a node inside a recovery domain that detects a fault condition should also be able to report it to network management 📨 Network management should be capable of correlating the fault reports and identifying the source of the fault 🕵️♀️
MPLS-TP OAM tools support a function where an intermediate node along a path can send an alarm report message to the MEP, indicating a fault condition in the server layer that connects it to its adjacent node 🚨 This lets a MEP suppress alarms that may be generated because of a failure condition in the server layer 🤫 iykyk
6.4.5. Coordination of Recovery Actions 🤝
As we covered, in some cases (like bidirectional protection switching, etc.) the protection states need to be coordinated between the edges of the recovery domain 🔄 [MPLS-TP-LP] defines procedures, protocol messages, and elements for exactly this 📚
The protocol is also used to signal administrative requests (e.g., manual switch, etc.), but only when these are provisioned at the edge of the recovery domain 🎀
The protocol also lets you detect mismatches between the configurations at the ends of the protection domain (like timers, revertive/non-revertive behavior) 🙈 and those mismatches can then be reported to the management plane 📣
Without suitable coordination (owing to failures in the delivery or processing of the coordination protocol messages), protection switching will fail 💀 So the protocol that coordinates the protection state is a fundamental part of protection switching 👑
6.5. Control Plane 🧠
The GMPLS control plane has been proposed as the control plane for MPLS-TP [RFC5317] Since GMPLS was designed for transport networks, and has been implemented and deployed in many networks, it's not surprising it has so many features that support a high degree of survivability 💪
The signaling elements of the GMPLS control plane use extensions to the Resource Reservation Protocol (RSVP) (as described in a series of documents commencing with [RFC3471] and [RFC3473]), even though it's based on [RFC3209] and [RFC2205] 📜 The architecture for GMPLS is in [RFC3945], and [RFC4426] gives a functional description of the protocol extensions needed for GMPLS-based recovery (i.e., protection and restoration) 🧩
A further control-plane protocol, the Link Management Protocol (LMP) [RFC4204], is part of the GMPLS protocol family and can be used to coordinate fault localization and reporting 🛰️
Clearly, the control-plane techniques here only apply where an MPLS-TP control plane is deployed and operated 🙃 All mandatory MPLS-TP survivability features must be enabled, even in the absence of the control plane. However, when it's present, the control plane may be used to provide alternative mechanisms, since they offer simple automation or a richer feature set ✨
6.5.1. Fault Detection 🔎
The control plane can't detect data-plane faults 🙅♀️ But it does provide mechanisms that detect control-plane faults, and these can be used to recognize data-plane faults when it's evident that the control and data planes are fate-sharing 🪢 Although [RFC5654] specifies that MPLS-TP must support an out-of-band control channel, it doesn't insist that it be used exclusively 🙌 This means there may be deployments where an in-band (or at least an in-fiber) control channel is used. In this scenario, failure of the control channel can be used to infer that there's a failure of the data channel or, at least, it can be used to trigger an investigation of the health of the data channel 🕵️
Both RSVP and LMP provide a control channel "keep-alive" mechanism (called the Hello message in both cases) ❤️ Failure to receive a message in the configured/negotiated time period indicates a control-plane failure ⏰ GMPLS routing protocols ([RFC4203] and [RFC5307]) also include keep-alive mechanisms designed to detect routing adjacency failures 🛤️ Although these keep-alives tend to operate at a relatively low frequency (on the order of seconds), it's still possible that the first indication of a control-plane fault will come through the routing protocol 🐢
Note, however, that you have to make sure a specific failure isn't caused by a problem in the control-plane software or in a processor component at the far end of a link ⚠️ (very important bestie)
Because of these various issues, it is not recommended that the control plane be used as the primary mechanism for fault detection in an MPLS-TP network 🚫
6.5.2. Testing for Faults 🧪
The control plane may be used to initiate and coordinate the testing of links, LSP segments, or entire LSPs 🎬 This is important in some technologies where data transmission has to halt while testing, but it may also be useful where testing needs to be specifically enabled or configured ⚙️
LMP provides a control-plane mechanism to test the continuity and connectivity (and naming) of individual links 🔗 A single management operation is required to initiate the test at one end of the link, while LMP handles the coordination with the other end 🤙 The test mechanism for an MPLS packet link relies on the LMP Test message inserted into the data stream at one end of the link and extracted at the other end 💉 This mechanism need not disrupt data flowing over the link 😌
Note that a link in LMP may, in fact, be an LSP tunnel used to form a link in the MPLS-TP network 🚇
GMPLS signaling (RSVP) offers two mechanisms that may also assist with fault testing 🧰 The first mechanism [RFC3473] defines the Admin_Status object that allows an LSP to be set into "testing mode" 🔧 The interpretation of this mode is implementation-specific and could be documented more precisely for MPLS-TP 🤔 The mode sets the whole LSP into a state where it can be tested, and this need not be disruptive to data traffic 🦋
The second mechanism provided by GMPLS to support testing is described in [GMPLS-OAM] This protocol extension supports the configuration (including enabling and disabling) of OAM mechanisms for a specific LSP 🎛️
6.5.3. Fault Localization 📍
Fault localization is the process whereby the exact location of a fault is determined 🕵️♀️ Fault detection often only takes place at key points in the network (such as at LSP end points or at MEPs), so a fault may be located anywhere within a segment of the relevant LSP 🙃
If segment or end-to-end protection is in use, this level of information is often sufficient to repair the LSP 🛠️ However, if finer information granularity is required (either to implement optimal recovery actions or to diagnose a fault), it's necessary to localize the specific fault 🎯
LMP provides a cascaded test-and-propagate mechanism that is designed specifically for this purpose 🪜
6.5.4. Fault Status Reporting 📡
GMPLS signaling uses the Notify message to report fault status [RFC3473] 📬 The Notify message can apply to a single LSP or can carry fault information for a set of LSPs, in order to improve the scalability of fault notification 📈
Since the Notify message is targeted at a specific node, it can be delivered rapidly without requiring hop-by-hop processing 🚀 It can be targeted at LSP end points or at segment end points (such as MEPs) The target points for Notify messages can be manually configured within the network, or they may be signaled when the LSP is set up 🧷
This enables the process to be made consistent with segment protection as well as with the concept of Maintenance Entities 🧩
GMPLS signaling also provides a slower, hop-by-hop mechanism for reporting individual LSP faults on a hop-by-hop basis using PathErr and ResvErr messages 🐌
[RFC4783] provides a mechanism to coordinate alarms and other event or fault information through GMPLS signaling 🚨 This mechanism is useful for understanding the status of the resources used by an LSP and for providing information as to why an LSP is not functioning 💡 however, it is not intended to replace other fault-reporting mechanisms 🙅
GMPLS routing protocols [RFC4203] and [RFC5307] are used to advertise link availability and capabilities within a GMPLS-enabled network 📢 Thus, the routing protocols can also provide indirect information about network faults: the protocol may stop advertising or may withdraw the advertisement for a failed link, or it may advertise that the link is about to be shut down gracefully [RFC5817] 👋 This mechanism is, however, not normally considered to be fast enough for use as a trigger for protection switching 🐢
6.5.5. Coordination of Recovery Actions 🤝
Fault coordination is an important feature for certain protection mechanisms (such as bidirectional 1:1 protection) 🔄 The use of the GMPLS Notify message for this purpose is described in [RFC4426] however, specific message field values have not yet been defined for this operation 🧩
Further work is needed in GMPLS for control and configuration of reversion behavior for end-to-end and segment protection, and for the coordination of timer values ⏳
6.5.6. Establishment of Protection and Restoration LSPs ✨
okay so the management plane can be used to set up protection and recovery LSPs 🎀 but if the control plane is there, it may be used too 💅
a few protocol extensions make this way less painful:
- [RFC4872] gives you features that support end-to-end protection switching 🦋
- [RFC4873] describes setting up a single, segment-protected LSP 🌸 ngl end-to-end protection is just a special case of segment protection, and [RFC4872] can also be used to provide end-to-end protection 💖
- [RFC4874] lets an LSP get signaled with a request that its path exclude specified resources like links, nodes, and shared risk link groups (SRLGs) 🔥 this means you can request a disjoint protection path, or set up a recovery path that dodges failed resources 🙈
- Lastly, it's worth noting that [RFC5298] gives an overview of the GMPLS techniques available to achieve protection in multi-domain environments 🌈
7. Pseudowire Recovery Considerations 🧚
okay pseudowires are giving end-to-end connectivity over the MPLS-TP network 💜 and they can be a single pseudowire segment, or multiple segments "stitched" together to get that end-to-end connection ✨
the pseudowire might need its own protection to hit the service-level guarantees in its SLA 😭 that protection could come from the MPLS-TP LSPs carrying it, or it could be a feature of the pseudowire layer itself 🫶
as mentioned above, the functional architecture in this document applies to both LSPs and pseudowires 💅 BUT the recovery mechanisms for pseudowires are still for further study, and they'll be defined in a separate document by the PWE3 working group 🍓 so stay tuned bestie 👑
7.1. Utilization of Underlying MPLS-TP Recovery 🦋
MPLS-TP PWs get carried across the network inside MPLS-TP LSPs 🌸 so an obvious way to protect a PW is to protect the LSP that carries it 💖 that protection can take any of the forms described in this document 🔥 the choice of recovery scheme depends on how fast recovery needs to be and how much traffic loss the SLA that the PW is providing can handle 💅
if the PW is a Multi-Segment PW, then LSP recovery can only protect the PW in individual segments 😭 which means one LSP recovery action can't protect against a failure of a PW switching point (an S-PE), and it also can't protect more than one segment at a time, since the LSP tunnel is terminated at each S-PE 🙈 in this way, LSP protection of a PW is very similar to the link-level protection an underlying network layer offers the MPLS-TP LSP layer (see Section 4.9) 🌈
7.2. Recovery in the Pseudowire Layer 🧚
recovery in the PW layer can be done by simply running separate PWs end-to-end ✨ other PW-layer recovery mechanisms, like segment or concatenated segment recovery, or service-level recovery involving survivability of T-PE or AC faults, will be described in a separate document 💜
as with any recovery mechanism, coordinating between layers is important 🫶 this coordination makes sure actions tied to recovery mechanisms only get performed in one layer at a time (so the recovery of an underlying LSP needs to be coordinated with the recovery of the PW itself) 👑 it also makes sure the working and protection PWs don't both use the same MPLS resources in the network (for example, by running over the same LSP tunnel; see also Section 4.9) 💅
8. Manageability Considerations 🎀
manageability of MPLS-TP networks and their functions is discussed in [RFC5950] 🌸 and OAM features are discussed in [RFC6371] 🦋
survivability has some key interactions with management, as described in this document 💖 specifically:
- recovery domains may be configured in a way that prevents a one-to-one correspondence between the MPLS-TP network and the recovery domains 🔥
- survivability policies may be configured per network, per recovery domain, or per LSP 🌈
- configuring OAM may involve selecting MEPs, enabling OAM on network segments, spans, and links, and operating OAM on LSPs, concatenated LSP segments, and LSP segments 💅
- manual commands may be used to control recovery functions, including forcing recovery and locking recovery actions 😭
see also the security considerations for management and OAM in Section 9 of this document 🫶
9. Security Considerations 💀
this framework doesn't introduce any new security considerations 🙈 general MPLS security issues can be found in [RFC5920] 💜
BUT several points about MPLS-TP survivability are worth noting here 🔥
- if an attacker can force a protection switch-over, it may cause a small perturbation to user traffic and could result in extra traffic getting preempted or displaced from the protection resources 😭 in the case of 1:n protection or shared mesh protection, this may leave other traffic unprotected 💀 so it's important that OAM protocols for detecting or notifying faults use adequate security to stop them from being abused (through inserting bogus messages or capturing legitimate messages) to falsely trigger a recovery event 🚨
- if manual commands get modified, captured, or simulated (including replay), an attacker might be able to perform forced recovery actions or impose lock-out 💀 these actions could hurt the ability to provide the recovery function and could also mess with the normal operation of the network for other traffic 😭 so management protocols used to perform manual commands must allow the operator to use appropriate security mechanisms 🫶 this includes verifying that the user performing the commands has appropriate authorization 👑
- if the control plane is used to configure or operate recovery mechanisms, the control-plane protocols must also be capable of providing adequate security 🛡️✨
10. Acknowledgments 🫶
thanks to the following people for useful comments and discussions: Italo Busi, David McWalter, Lou Berger, Yaacov Weingarten, Stewart Bryant, Dan Frost, Lievren Levrau, Xuehui Dai, Liu Guoman, Xiao Min, Daniele Ceccarelli, Scott Bradner, Francesco Fondelli, Curtis Villamizar, Maarten Vissers, and Greg Mirsky 💖
the Editors would also like to thank the participants in ITU-T Study Group 15 for their detailed review 🌸
some figures and text on shared mesh protection were borrowed from [MPLS-TP-MESH], with thanks to Tae-sik Cheung and Jeong-dong Ryoo 🦋
11. References 📚
11.1. Normative References 💜
- [G.806] ITU-T, "Characteristics of transport equipment - Description methodology and generic functionality", Recommendation G.806, January 2009.
- [G.808.1] ITU-T, "Generic Protection Switching - Linear trail and subnetwork protection", Recommendation G.808.1, December 2003.
- [G.841] ITU-T, "Types and Characteristics of SDH Network Protection Architectures", Recommendation G.841, October 1998.
- [RFC2205] Braden, R., Ed., Zhang, L., Berson, S., Herzog, S., and S. Jamin, "Resource ReSerVation Protocol (RSVP) -- Version 1 Functional Specification", RFC 2205, September 1997.
- [RFC3209] Awduche, D., Berger, L., Gan, D., Li, T., Srinivasan, V., and G. Swallow, "RSVP-TE: Extensions to RSVP for LSP Tunnels", RFC 3209, December 2001.
- [RFC3471] Berger, L., Ed., "Generalized Multi-Protocol Label Switching (GMPLS) Signaling Functional Description", RFC 3471, January 2003.
- [RFC3473] Berger, L., Ed., "Generalized Multi-Protocol Label Switching (GMPLS) Signaling Resource ReserVation Protocol-Traffic Engineering (RSVP-TE) Extensions", RFC 3473, January 2003.
- [RFC3945] Mannie, E., Ed., "Generalized Multi-Protocol Label Switching (GMPLS) Architecture", RFC 3945, October 2004.
- [RFC4203] Kompella, K., Ed., and Y. Rekhter, Ed., "OSPF Extensions in Support of Generalized Multi-Protocol Label Switching (GMPLS)", RFC 4203, October 2005.
- [RFC4204] Lang, J., Ed., "Link Management Protocol (LMP)", RFC 4204, October 2005.
- [RFC4427] Mannie, E., Ed., and D. Papadimitriou, Ed., "Recovery (Protection and Restoration) Terminology for Generalized Multi-Protocol Label Switching (GMPLS)", RFC 4427, March 2006.
- [RFC4428] Papadimitriou, D., Ed., and E. Mannie, Ed., "Analysis of Generalized Multi-Protocol Label Switching (GMPLS)-based Recovery Mechanisms (including Protection and Restoration)", RFC 4428, March 2006.
- [RFC4873] Berger, L., Bryskin, I., Papadimitriou, D., and A. Farrel, "GMPLS Segment Recovery", RFC 4873, May 2007.
- [RFC5307] Kompella, K., Ed., and Y. Rekhter, Ed., "IS-IS Extensions in Support of Generalized Multi-Protocol Label Switching (GMPLS)", RFC 5307, October 2008.
- [RFC5317] Bryant, S., Ed., and L. Andersson, Ed., "Joint Working Team (JWT) Report on MPLS Architectural Considerations for a Transport Profile", RFC 5317, February 2009.
- [RFC5586] Bocci, M., Ed., Vigoureux, M., Ed., and S. Bryant, Ed., "MPLS Generic Associated Channel", RFC 5586, June 2009.
- [RFC5654] Niven-Jenkins, B., Ed., Brungard, D., Ed., Betts, M., Ed., Sprecher, N., and S. Ueno, "Requirements of an MPLS Transport Profile", RFC 5654, September 2009.
- [RFC5921] Bocci, M., Ed., Bryant, S., Ed., Frost, D., Ed., Levrau, L., and L. Berger, "A Framework for MPLS in Transport Networks", RFC 5921, July 2010.
- [RFC5950] Mansfield, S., Ed., Gray, E., Ed., and K. Lam, Ed., "Network Management Framework for MPLS-based Transport Networks", RFC 5950, September 2010.
- [RFC6371] Buci, I., Ed. and B. Niven-Jenkins, Ed., "A Framework for MPLS in Transport Networks", RFC 6371, September 2011.
11.2. Informative References 📚✨
okay so these are the "good to know" refs, the ones that give context without being required reading 🫶 they're all still part of the story though, so here they are in full bestie 💅
[GMPLS-OAM] Takacs, A., Fedyk, D., and J. He, "GMPLS RSVP-TE
extensions for OAM Configuration", Work in Progress,
July 2011.
[MPLS-TP-LP] Weingarten, Y., Osborne, E., Sprecher, N., Fulignoli,
A., Ed., and Y. Weingarten, Ed., "MPLS-TP Linear
Protection", Work in Progress, August 2011.
[MPLS-TP-MESH] Cheung, T. and J. Ryoo, "MPLS-TP Shared Mesh
Protection", Work in Progress, April 2011.
[RFC3031] Rosen, E., Viswanathan, A., and R. Callon,
"Multiprotocol Label Switching Architecture", RFC
3031, January 2001.
[RFC3386] Lai, W., Ed., and D. McDysan, Ed., "Network Hierarchy
and Multilayer Survivability", RFC 3386, November
2002.
[RFC3469] Sharma, V., Ed., and F. Hellstrand, Ed., "Framework
for Multi-Protocol Label Switching (MPLS)-based
Recovery", RFC 3469, February 2003.
[RFC4397] Bryskin, I. and A. Farrel, "A Lexicography for the
Interpretation of Generalized Multiprotocol Label
Switching (GMPLS) Terminology within the Context of
the ITU-T's Automatically Switched Optical Network
(ASON) Architecture", RFC 4397, February 2006.
[RFC4426] Lang, J., Ed., Rajagopalan, B., Ed., and D.
Papadimitriou, Ed., "Generalized Multi-Protocol Label
Switching (GMPLS) Recovery Functional Specification",
RFC 4426, March 2006.
[RFC4726] Farrel, A., Vasseur, J.-P., and A. Ayyangar, "A
Framework for Inter-Domain Multiprotocol Label
Switching Traffic Engineering", RFC 4726, November
2006.
[RFC4783] Berger, L., Ed., "GMPLS - Communication of Alarm
Information", RFC 4783, December 2006.
[RFC4872] Lang, J., Ed., Rekhter, Y., Ed., and D. Papadimitriou,
Ed., "RSVP-TE Extensions in Support of End-to-End
Generalized Multi-Protocol Label Switching (GMPLS)
Recovery", RFC 4872, May 2007.
[RFC4874] Lee, CY., Farrel, A., and S. De Cnodder, "Exclude
Routes - Extension to Resource ReserVation Protocol-
Traffic Engineering (RSVP-TE)", RFC 4874, April 2007.
[RFC5212] Shiomoto, K., Papadimitriou, D., Le Roux, JL.,
Vigoureux, M., and D. Brungard, "Requirements for
GMPLS-Based Multi-Region and Multi-Layer Networks
(MRN/MLN)", RFC 5212, July 2008.
[RFC5298] Takeda, T., Ed., Farrel, A., Ed., Ikejiri, Y., and JP.
Vasseur, "Analysis of Inter-Domain Label Switched Path
(LSP) Recovery", RFC 5298, August 2008.
[RFC5817] Ali, Z., Vasseur, JP., Zamfir, A., and J. Newton,
"Graceful Shutdown in MPLS and Generalized MPLS
Traffic Engineering Networks", RFC 5817, April 2010.
[RFC5920] Fang, L., Ed., "Security Framework for MPLS and GMPLS
Networks", RFC 5920, July 2010.
[RFC6373] Andersson, L., Ed., Berger, L., Ed., Fang, L., Ed.,
and Bitar, N., Ed, and E. Gray, Ed., "MPLS-TP Control
Plane Framework", RFC 6373, September 2011.
[RFC6291] Andersson, L., van Helvoort, H., Bonica, R.,
Romascanu, D., and S. Mansfield, "Guidelines for the
Use of the "OAM" Acronym in the IETF", BCP 161, RFC
6291, June 2011.
[ROSETTA] Van Helvoort, H., Ed., Andersson, L., Ed., and N.
Sprecher, Ed., "A Thesaurus for the Terminology used
in Multiprotocol Label Switching Transport Profile
(MPLS-TP) drafts/RFCs and ITU-T's Transport Network
Recommendations", Work in Progress, June 2011.Authors' Addresses 🎀🧚
and that's the whole crew! here's where you can find the editors, nurit and adrian 🦋💖
Nurit Sprecher (editor)
Nokia Siemens Networks
3 Hanagar St.
Neve Ne'eman B Hod
Hasharon, 45241 Israel
EMail: nurit.sprecher@nsn.com
Adrian Farrel (editor)
Juniper Networks
EMail: adrian@olddog.co.uk📦 where this page comes from
- ietf: rfc6372 2011-09
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms