RFC 9026
Multicast VPN Fast Upstream Failover
Multicast VPN Fast Upstream Failover 🎀
Okay bestie, the title block is giving main character energy 💅 RFC 9026 is a Standards Track document from April 2021 (ISSN 2070-1721), so it's a real Internet standard, not just an opinion piece. The editors are Tony Morin from Orange, Robert Kebler from Juniper Networks, and Gregory Mirsky from ZTE Corp. Nothing in the title block says it obsoletes or updates any other RFC, so it stands on its own 🦋
Abstract ✨
Okay so this doc adds Multicast VPN (VPN) extensions and procedures for fast failover when something upstream breaks 😭 The big idea is that downstream Provider Edges (PEs) look at the status of Provider-Tunnels (P-tunnels) when they pick the Upstream PE for a VPN multicast flow. The fast failover works by using "Bidirectional Forwarding Detection (BFD) for Multipoint Networks" (RFC 8562) plus a brand new BGP Attribute called BFD Discriminator 🔥 It also introduces a new BGP Community called Standby PE, which lets a C-multicast route get advertised toward a Standby Upstream PE. Iconic 🧚
Status of This Memo
This is an Internet Standards Track document 💜
It's a product of the Internet Engineering Task Force (IETF), which means it reflects the consensus of the IETF community. It got public review and was approved for publication by the Internet Engineering Steering Group (IESG). If you want more on Internet Standards, check Section 2 of RFC 7841 🌸
Info about the current status of this document, any errata, and how to send feedback lives at https://www.rfc-editor.org/info/rfc9026 ✨
Copyright Notice
Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved 🎀
This doc is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they were in effect when it was published. Please actually read those, bestie, because they explain your rights and restrictions for this document 🫶 Code Components pulled out of this document must include the Simplified BSD License text described in Section 4.e of the Trust Legal Provisions, and they come without warranty, as described in the Simplified BSD License 💀
Table of Contents
1. Introduction
2. Conventions Used in This Document
2.1. Requirements Language
2.2. Terminology
2.3. Abbreviations
3. UMH Selection Based on Tunnel Status
3.1. Determining the Status of a Tunnel
3.1.1. MVPN Tunnel Root Tracking
3.1.2. PE-P Upstream Link Status
3.1.3. P2MP RSVP-TE Tunnels
3.1.4. Leaf-Initiated P-Tunnels
3.1.5. (C-S,C-G) Counter Information
3.1.6. BFD Discriminator Attribute
3.1.7. BFD Discriminator per PE-CE Link
3.1.8. Operational Considerations for Monitoring a P-Tunnel's
Status
4. Standby C-Multicast Route
4.1. Downstream PE Behavior
4.2. Upstream PE Behavior
4.3. Reachability Determination
4.4. Inter-AS
4.4.1. Inter-AS Procedures for Downstream PEs, ASBR Fast
Failover
4.4.2. Inter-AS Procedures for ASBRs
5. Hot Root Standby
6. Duplicate Packets
7. IANA Considerations
7.1. Standby PE Community
7.2. BFD Discriminator
7.3. BFD Discriminator Optional TLV Type
8. Security Considerations
9. References
9.1. Normative References
9.2. Informative References
Acknowledgments
Contributors
Authors' Addresses1. Introduction ✨
Okay so this doc assumes you already know how multicast MPLS/BGP IP VPNs work, as described in [RFC6513] and [RFC6514] 🧠
In the context of multicast in BGP/MPLS VPNs [RFC6513], we really want mechanisms that let connectivity recover fast when different kinds of failures happen 😭 This document is about failures of elements in the provider network that sit upstream of PEs connected to VPN sites with receivers.
Section 3 describes local procedures that let an egress PE (a PE connected to a receiver site) factor in the status of P-tunnels when it picks the Upstream Multicast Hop (UMH) for a given (C-S,C-G) 🦋 One of the optional methods uses [RFC8562] and the new BGP Attribute, BFD Discriminator. None of these methods is a "fast failover" solution all by itself, but they can be combined with the mechanism in Section 4 to get one.
Section 4 describes an optional BGP extension, the new Standby PE Community, which can speed up failover by not needing any Multicast VPN (MVPN) routing message exchange at recovery time 🚀
Section 5 describes a "hot root standby" mechanism that can improve failover time in MVPN 💖 It combines the mechanisms from Sections 3 and 4, and it has similarities with the solution in [RFC7431], which improves failover times when PIM routing is used in a network with certain topology and metric constraints.
All the procedures in this document are optional, and they let an operator protect multicast services in BGP/MPLS IP VPNs 🛡️ To turn them on, an operator uses a method from Section 3 together with the redundancy of a standby PE connected to the multicast flow source. PEs that support these mechanisms converge faster, which makes the multicast service more stable 🌈 And if a BGP implementation doesn't recognize these extensions, or is configured not to support them, it keeps providing the multicast service as described in [RFC6513] 🫶
2. Conventions Used in This Document 🎀
2.1. Requirements Language
Okay this part is important, so pay attention 🤓 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174], but only when they appear in all capitals, exactly as shown here 💅
2.2. Terminology
The terminology in this document is the terminology defined in [RFC6513] and [RFC6514] ✨
The term "upstream" (lowercase) throughout this document refers to links and nodes that are upstream of a PE connected to VPN sites with receivers of a multicast flow 🦋
The term "Upstream" (capitalized) throughout this document refers to a PE or an Autonomous System Border Router (ASBR) where (S,G) or (*,G) data packets enter the VPN backbone, or the local AS, while traveling through the VPN backbone 🧚
2.3. Abbreviations
Here's the abbreviation list, copied exactly so nothing gets lost in translation 🍓
PMSI: P-Multicast Service Interface
I-PMSI: Inclusive PMSI
S-PMSI: Selective PMSI
x-PMSI: Either an I-PMSI or an S-PMSI
P-tunnel: Provider-Tunnel
UMH: Upstream Multicast Hop
VPN: Virtual Private Network
MVPN: Multicast VPN
RD: Route Distinguisher
RP: Rendezvous Point
NLRI: Network Layer Reachability Information
VRF: VPN Routing and Forwarding Table
MED: Multi-Exit Discriminator
P2MP: Point-to-Multipoint3. UMH Selection Based on Tunnel Status 💜
Okay so Section 5.1 of [RFC6513] has the procedures a downstream MVPN PE uses to figure out the Upstream Multicast Hop (UMH) for a given (C-S,C-G) 🧠
For a given downstream PE and a given VRF, the P-tunnel for a given Upstream PE and a given (C-S,C-G) state is the S-PMSI tunnel that Upstream PE advertised for that (C-S,C-G) and imported into that VRF. If there isn't any such S-PMSI, it's the I-PMSI tunnel that PE advertised and imported into that VRF 🌸
This procedure is optional. It works by having a downstream PE check the status of P-tunnels rooted at each possible Upstream PE, and then decide whether to include each PE in the list of candidate UMHs for a given (C-S,C-G) state 🎀 If it isn't possible to tell whether a P-tunnel's current status is Up, the state shall be considered "not known to be Down", and it may be treated as if it is Up, so that attempts to use the tunnel are acceptable ✨ So if a P-tunnel is Down (see Section 3.1), the PE at the root of that P-tunnel won't be considered for UMH selection 😭 That makes the downstream PE fail over to the next Upstream PE in the candidate list.
Heads up, bestie: different downstream PEs might reach different conclusions about a tunnel's state, because the failure might only affect a subset of branches 👀 That's why the procedures of Section 9.1.1 of [RFC6513] apply when you're using I-PMSI P-tunnels. That document is the foundation for this one, and all of its processes apply here too 🫶
There are three options in Section 5.1 of [RFC6513] for a downstream PE to pick an Upstream PE 💅
- The first two options pick the Upstream PE from a candidate PE set, based either on an IP address or on a hashing algorithm. When you also use the optional procedure of considering the P-tunnel status (from this document), a candidate Upstream PE gets included in the set if it either:
- a. advertises an x-PMSI bound to a tunnel where the tunnel's state is not known to be Down, or,
- b. does not advertise any x-PMSI applicable to the given (C-S,C-G) but has a VRF Route Import BGP Extended Community associated with the unicast VPN route for S. This is needed to avoid wrongly invalidating a UMH PE that uses a policy where no I-PMSI is advertised for a given VRF and only S-PMSIs are used. The S-PMSI can only be advertised after the Upstream PE receives a C-multicast route for (C-S,C-G) / (C-*,C-G) to be carried over the advertised S-PMSI 🦋
- If the resulting candidate set is empty, the procedure is repeated without considering the P-tunnel status 💀
- The third option uses the installed UMH Route (the "best" route toward the C-root) as the Selected UMH Route, and its originating PE is the selected Upstream PE. When you also use the optional procedure of considering P-tunnel status (from this document), the Selected UMH Route is the best one among those whose originating PE's P-tunnel is not "down". If none of those exists, the installed UMH Route is selected regardless of P-tunnel status 🔥
3.1. Determining the Status of a Tunnel 💖
okay so there are a bunch of different factors you can look at to figure out the "status" of a P-tunnel 👀, and we go through them in the subsections below ✨. The optional procedures in this section also cover the case where the downstream PEs don't all use the same rules to decide what a P-tunnel's status is (see Section 6 🧚), and some of them can give a different answer for different downstream PEs. So the "status" here isn't some fixed thing about the tunnel itself, it's the tunnel status as seen from a particular downstream PE 🫶. Some of these methods are also really about whether a downstream PE can receive traffic on the P-tunnel, not about the P-tunnel itself. That could be called "P-tunnel reception status" 💅, but for simplicity we're just gonna use P-tunnel "status" for all of them.
Depending on the criteria, there might be an interaction with another resiliency mechanism used for the P-tunnel itself, and the UMH update may happen immediately or may need to be delayed ⏳. Each case is covered in its own subsection below 🌸.
An implementation can support any combination of the methods in this section and give a network operator control over which one to use in their deployment 👑.
3.1.1. MVPN Tunnel Root Tracking 🗺️
When figuring out if a P-tunnel's status is Up, one thing to check is whether the root of the tunnel (the one in the x-PMSI Tunnel attribute) is reachable through unicast routing tables 🧭. If it is, the downstream PE can update its UMH immediately when that reachability condition changes ⚡.
This is kinda like BGP next-hop tracking for VPN routes, except the address we care about isn't the BGP next-hop, it's the root address in the x-PMSI Tunnel attribute 💖. BGP next-hop tracking watches for BGP next-hop address changes in the routing table. Generally, when it spots a change, it does a next-hop scan to see if any next hops in the BGP table got affected, and then updates them accordingly 🔥.
Now, if BGP next-hop tracking is running for VPN routes and the root address of a given tunnel happens to be the same as the next-hop address in the BGP A-D Route advertising that tunnel, then checking unicast routing tables for the tunnel root is unnecessary duplication 🙄. So it won't bring any specific benefit ✨.
3.1.2. PE-P Upstream Link Status 🦋
When determining if a P-tunnel is Up, a thing to consider is whether the last-hop link of the P-tunnel is Up. And the flip side: if that last-hop link is Down, that can be taken as a sign the P-tunnel is Down 📉.
Using this when a fast restoration mechanism (such as MPLS Fast Reroute (FRR) [RFC4090]) is in place for the link is kinda tricky 😵💫, so it needs careful consideration and coordination of the defect detection intervals for the link and the tunnel 🧠. If you're doing multi-layer protection, you especially have to think about how defect detections at different network layers interact 🫠. It is recommended to use longer detection intervals at the higher layers. Some recommendations suggest using a multiplier of 3 or larger, e.g., 10 msec detection for the link failure detection and at least 100 msec for the tunnel failure detection 💀. In many cases it's not practical to use both protection methods at the same time, because uncorrelated timers might cause unnecessary switchovers and destabilize the network 🙈.
3.1.3. P2MP RSVP-TE Tunnels 🎀
For P-tunnels of type P2MP MPLS-TE, the status of the P-tunnel is considered Up if the sub-LSP to this downstream PE is in the Up state 🚀. Figuring out whether a P2MP RSVP-TE Label Switched Path (LSP) is in the Up state requires Path and Resv state for the LSP, and it's based on the procedures specified in [RFC4875]. So the downstream PE can update its UMH immediately when the reachability condition changes 🌈.
With this method, if the signaling state for a P2MP TE LSP is removed (for example, the ingress of the P2MP TE LSP sends a PathTear message), or the P2MP TE LSP changes state from Up to Down as determined by the procedures in [RFC4875], then the status of the corresponding P-tunnel MUST be re-evaluated 🔍. If the P-tunnel transitions from Up to Down state, the Upstream PE that is the ingress of the P-tunnel MUST NOT be considered to be a valid candidate UMH 🚫.
3.1.4. Leaf-Initiated P-Tunnels 🌷
An Upstream PE MUST be removed from the UMH candidate list for a given (C-S,C-G) if the P-tunnel (I-PMSI or S-PMSI) for this (S,G) is leaf triggered (PIM, mLDP), but for some reason, internal to the protocol, the upstream one-hop branch of the tunnel from P to PE cannot be built 🙅♀️. As a result, the downstream PE can update its UMH immediately when the reachability condition changes 💫.
3.1.5. (C-S,C-G) Counter Information 📊
In cases where the downstream node can be configured so that the maximum inter-packet time is known for all the multicast flows mapped on a P-tunnel, the local traffic counter information per (C-S,C-G) for traffic received on this P-tunnel can be used to determine the status of the P-tunnel 🧮.
When such a procedure is used, in the context where fast restoration mechanisms are used for the P-tunnels, a configurable timer MUST be set on the downstream PE to wait before updating the UMH, so the P-tunnel restoration mechanism gets to do its actions 🛠️. Deciding that a tunnel is probably down by waiting for enough packets to fail to arrive as expected is a heuristic and an operational matter that depends on the maximum inter-packet time 🤔. A timeout of three seconds is a generally suitable default for making sure the tunnel is down 🕒, though other values would be needed for atypical conditions ⏱️.
In cases where this mechanism is used together with the method described in Section 5, no prior knowledge of the rate or maximum inter-packet time on the multicast streams is required 🙌. Downstream PEs can periodically compare actual packet reception statistics on the two P-tunnels to determine when one of them is down 👯. The detailed specification of this mechanism is outside the scope of this document 📝.
3.1.6. BFD Discriminator Attribute 🧚♀️
The P-tunnel status may be derived from the status of a multipoint BFD session [RFC8562] whose discriminator is advertised along with an x-PMSI A-D Route 💜. A P2MP BFD session can be instantiated using a mechanism other than the BFD Discriminator attribute, e.g., MPLS LSP Ping ([MPLS-P2MP-BFD]). The description of these methods is outside the scope of this document 🌸.
This document defines the format and ways of using a new BGP attribute called the "BFD Discriminator" (38). It is an optional transitive BGP attribute. So it's expected that an implementation that does not recognize this attribute, or is configured not to support it, treats it as if the attribute was unrecognized, and follows the procedures defined for optional transitive path attributes in Section 5 of [RFC4271] 🎁. See Section 7.2 for more information. The format of this attribute is shown in Figure 1 🍓.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+
| BFD Mode |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| BFD Discriminator |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~ Optional TLVs ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 1: Format of the BFD Discriminator AttributeWhere:
- BFD Mode field is 1 octet long. This specification defines P2MP BFD Session as value 1 (Section 7.2).
- BFD Discriminator field is 4 octets long.
- Optional TLVs is the optional variable-length field that MAY be used in the BFD Discriminator attribute for future extensions. TLVs MAY be included in a sequential or nested manner. To allow for TLV nesting, it is advised to define a new TLV as a variable-length object. Figure 2 presents the Optional TLV format TLV that consists of:
- Type: a 1-octet-long field that characterizes the interpretation of the Value field (Section 7.3)
- Length: a 1-octet-long field equal to the length of the Value field in octets
- Value: a variable-length field
- All multibyte fields in TLVs defined in this specification are in network byte order.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type | Length | Value ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 2: Format of the Optional TLVAn optional Source IP Address TLV is defined in this document 🫶. The Source IP Address TLV MUST be used when the value of the BFD Mode field's value is P2MP BFD Session. The BFD Discriminator attribute that does not include the Source IP Address TLV MUST be handled according to the "attribute discard" approach, as defined in [RFC7606]. For the Source IP Address TLV, fields are set as follows:
- The Type field is set to 1 (Section 7.3).
- The Length field is 4 for the IPv4 address family and 16 for the IPv6 address family. The TLV is considered malformed if the field is set to any other value.
- The Value field contains the address associated with the MultipointHead of the P2MP BFD session.
The BFD Discriminator attribute MUST be considered malformed if its length is smaller than 11 octets or if Optional TLVs are present but not well formed 🚨. If the attribute is deemed to be malformed, the UPDATE message SHALL be handled using the approach of Attribute Discard per [RFC7606] 💅.
3.1.6.1. Upstream PE Procedures ✨
Okay so, to let downstream PEs track the P-tunnel with a point-to-multipoint (P2MP) BFD session, the Upstream PE:
- MUST initiate the BFD session and set
bfd.SessionType=MultipointHeadas described in [RFC8562] 💖 - when transmitting BFD Control packets MUST set the IP destination address of the inner IP header to the internal loopback address
127.0.0.1/32for IPv4 [RFC1122]. For IPv6, it MUST use the loopback address::1/128[RFC4291] 🦋 - MUST use the IP address included in the Source IP Address TLV of the BFD Discriminator attribute as the source IP address when transmitting BFD Control packets 🌸
- MUST include the BFD Discriminator attribute in the x-PMSI A-D Route with the value set to the My Discriminator value 🎀
- MUST periodically transmit BFD Control packets over the x-PMSI P-tunnel after the P-tunnel is considered established. Note that the methods to declare that a P-tunnel has been established are outside the scope of this specification. 🧚
If tracking the P-tunnel with a P2MP BFD session gets turned on after the x-PMSI A-D Route has already been advertised, the x-PMSI A-D Route MUST be resent, and the only change between the old advertisement and the new one is the inclusion of the BFD Discriminator attribute. Simple, but it matters 💅
Now, if the x-PMSI A-D Route is advertised with P-tunnel status tracked by a P2MP BFD session, and you want to stop tracking with BFD, then:
- the x-PMSI A-D Route MUST be resent, and the only change between the previous advertisement and the new advertisement is the exclusion of the BFD Discriminator attribute 🫶
- the P2MP BFD session MUST be deleted. The session MAY be deleted after some configurable delay, which should have a reasonable default. 🌈
3.1.6.2. Downstream PE Procedures 💜
Okay bestie, when the downstream PE gets the BFD Discriminator attribute in the x-PMSI A-D Route, it:
- MUST associate the received BFD Discriminator value with the P-tunnel originating from the Upstream PE and the IP address of the Upstream PE 👑
- MUST create a P2MP BFD session and set
bfd.SessionType=MultipointTailas described in [RFC8562] ✨ - to properly demultiplex the BFD session, MUST use:
- the IP address in the Source IP Address TLV included the BFD Discriminator attribute in the x-PMSI A-D Route;
- the value of the BFD Discriminator field in the BFD Discriminator attribute;
- the x-PMSI Tunnel Identifier [RFC6514] the BFD Control packet was received on. 🔥
Once the P2MP BFD session is up, i.e., bfd.SessionState == Up, that session state is what gets used to track the health of the P-tunnel. Main character energy 🦋
According to [RFC8562], if the downstream PE receives Down or AdminDown in the State field of the BFD Control packet, or if the Detection Timer associated with the BFD session expires, the BFD session is down, i.e., bfd.SessionState == Down. When the BFD session state is Down, the P-tunnel associated with that session MUST be considered down. 💀
If the site that contains C-S is connected to two or more PEs, a downstream PE will select one as its Primary Upstream PE, and the others are considered to be Standby Upstream PEs. In that setup, when the P-tunnel is considered down, the downstream PE MAY initiate a switchover of traffic from the Primary Upstream PE to the Standby Upstream PE, but only if the Standby Upstream PE is deemed to be in the Up state. That MAY be determined from the state of a P2MP BFD session with the Standby Upstream PE as the MultipointHead. 🌸
If the downstream PE's P-tunnel is already established when it receives the new x-PMSI A-D Route with the BFD Discriminator attribute, the downstream PE MUST associate the value of the BFD Discriminator field with the P-tunnel, and follow the procedures listed above in this section, if and only if the x-PMSI A-D Route was properly processed as per [RFC6514] and the BFD Discriminator attribute was validated. 🎀
Now the tricky one. If the downstream PE's P-tunnel is already established, its state is being monitored by the P2MP BFD session set up using the BFD Discriminator attribute, and the downstream PE receives the new x-PMSI A-D Route without the BFD Discriminator attribute, and that x-PMSI A-D Route was processed without any error as per the relevant specifications, then:
- The downstream PE MUST stop processing BFD Control packets for this P2MP BFD session. 🙈
- The P2MP BFD session associated with the P-tunnel MUST be deleted. The session MAY be deleted after some configurable delay, which should have a reasonable default. 🧚
- The downstream PE MUST NOT switch the traffic to the Standby Upstream PE. 🫶
3.1.7. BFD Discriminator per PE-CE Link 🌈
This approach is defined for when the Upstream PE detects a PE-CE link failure. Even though the provider tunnel is still up, the downstream PEs should switch to a backup Upstream PE. So, if the Upstream PE detects that its PE-CE link has failed, it MUST set the bfd.LocalDiag of the P2MP BFD session to Concatenated Path Down or Reverse Concatenated Path Down (per Section 6.8.17 of [RFC5880]), unless it switches to a new PE-CE link within the time of bfd.DesiredMinTxInterval for the P2MP BFD session. (In that case, the Upstream PE will start tracking the status of the new PE-CE link.) When a downstream PE receives that bfd.LocalDiag code, it treats it as if the tunnel itself failed and tries to switch to a backup PE. 😭
3.1.8. Operational Considerations for Monitoring a P-Tunnel's Status 🔍
Several methods to monitor the status of a P-tunnel are described in Section 3.1. Let's break down the vibes of each one 💅
Tracking the root of an MVPN (Section 3.1.1) reveals the status of a P-tunnel based on control plane information. The MPLS data plane isn't generally fate sharing with the control plane, so this method might produce false-positive or false-negative alarms. For example, you could end up with tunnels that look Up but can't reach the root, or ones that get declared down too early. On the flip side, BGP next-hop tracking is broadly supported and deployed, so this method might be the easiest to deploy. 🌸
The method in Section 3.1.2 monitors the state of the data plane, but only for an egress P-PE link of a P-tunnel. So network failures affecting upstream links might not get detected this way, and MVPN convergence would be determined by the convergence of the BGP control plane. 🥲
Using a state change of a P2MP RSVP-TE LSP as the trigger to re-evaluate the status of the P-tunnel (Section 3.1.3) relies on whatever mechanism is used to monitor the state of the P2MP LSP. 🧚
The method in Section 3.1.4 is simple and safe from causing false alarms, e.g., considering a tunnel operationally Up even though its data path has a defect, or declaring a tunnel failed when it's unaffected. The catch is that it only applies to a subset of MVPNs, namely those that use the leaf-triggered x-PMSI tunnels. 🎀
Some MVPNs might be used to provide a multicast service with predictable inter-packet intervals (Section 3.1.5), but the number of such cases seems limited. 🍓
Monitoring the status of a P-tunnel using a P2MP BFD session (Section 3.1.6) may produce the most accurate and expedient failure notification of all the monitoring methods discussed. On the other hand, it requires careful consideration of the additional load that BFD sessions put on network and PE nodes. Operators should consider the rate of BFD Control packets transmitted by root PEs, combined with the number of such PEs in the network. Also, the number of P2MP BFD sessions per PE determines how much state information a PE maintains. 💖
4. Standby C-Multicast Route 👑
The procedures below are limited to the case where the site that contains C-S is connected to two or more PEs. To keep things simple, the dual homing case is what gets described. If more than two PEs are connected to the C-S site, the Standby PE can be selected using one of the methods of selecting a UMH. Details of that selection are outside the scope of this document. The procedures require all the PEs of that MVPN to follow the same UMH selection procedure, as specified in [RFC6513], whether the PE is selected based on its IP address, the hashing algorithm described in Section 5.1.3 of [RFC6513], or the Installed UMH Route. The management plane is expected to ensure that all PEs use a consistent UMH selection method. The procedures assume that if a site of a given MVPN that contains C-S is dual homed to two PEs, then all the other sites of that MVPN would have two unicast VPN routes (VPN-IPv4 or VPN-IPv6) to C-S, each with its own RD. 🦋
As long as C-S is reachable via both PEs, a given downstream PE will select one of the PEs connected to C-S as its Upstream PE for C-S. We'll call the other PE connected to C-S the "Standby Upstream PE". If connectivity to C-S through the Primary Upstream PE becomes unavailable, the PE selects the Standby Upstream PE as its Upstream PE for C-S. When the Primary PE comes back, the PE selects the Primary Upstream PE again as its Upstream PE. This is called "revertive" behavior, and it MUST be supported. 🔁
Non-revertive behavior refers to continuing to select the backup PE as the UMH even after the Primary has come back up. This non-revertive behavior MAY also be supported by an implementation, and it would be enabled through some configuration. Choosing revertive or non-revertive is an operational issue, but it MUST be consistent on all PEs in the given MVPN. Revertive is considered the default behavior, but there might be cases where switching over to the standby tunnel doesn't affect other services and still provides the required quality of service. In that case, an operator might use non-revertive behavior to avoid unnecessary switchovers and minimize disruption to the multicast service. 🌈
To keep things readable, the following subsections describe the procedures for BGP C-multicast Source Tree Join routes, but they apply equally to BGP C-multicast Shared Tree Join routes when the customer RP is dual homed (substitute "C-RP" for "C-S"). 🍓
4.1. Downstream PE Behavior ✨
okay so, when a (downstream) PE hooked up to some site of an MVPN needs to send a C-multicast route (C-S,C-G) 💖, it follows the procedures in Section 11.1 of [RFC6514] and sends that route with an RT that identifies the Upstream PE the PE picked. As long as C-S is reachable via the Primary Upstream PE, the Upstream PE is the Primary Upstream PE 🎀. If C-S is reachable only via the Standby Upstream PE, then the Upstream PE is the Standby Upstream PE 🦋.
If C-S is reachable via both the Primary and the Standby Upstream PE, then the downstream PE sends the C-multicast route with an RT for the Primary Upstream PE, and it ALSO originates and sends a second C-multicast route with an RT for the Standby Upstream PE ✨. That second one has the "standby" vibe, so it's called a "Standby BGP C-multicast route", and it gets built like this:
- The NLRI is built like the C-multicast route with an RT for the Primary Upstream PE, except the RD is the same as if the C-multicast route had been built using the Standby Upstream PE as the UMH (so it carries the RD tied to the unicast VPN route the Standby Upstream PE advertised for S, plus a Route Target derived from the Standby Upstream PE's UMH route's VRF RT Import EC) 💅.
- It MUST carry the "Standby PE" BGP Community (
0xFFFF0009); see Section 7.1 🫶.
The Local Preference attribute of both the normal and the standby C-multicast route needs some adjusting 👑. If a BGP peer gets two C-multicast routes with the same NLRI, one carrying the "Standby PE" community and the other one not carrying it, the peer prefers the one WITHOUT the "Standby PE" community 😭. This can happen when, for instance, transient unicast routing inconsistencies or lack of support for the Standby PE community make two different downstream PEs think different Upstream PEs are the primary one 💀. In that case, without any precaution, both Upstream PEs would process a standby C-multicast route and possibly stop forwarding at the same time 🙈. So routes that carry the Standby PE BGP Community must have the LOCAL_PREF attribute set to a value lower than the LOCAL_PREF value for the route that does NOT carry it. The value of zero is RECOMMENDED 🍓.
Note that when a PE advertises such a Standby C-multicast join for a (C-S,C-G), it MUST join the corresponding P-tunnel 🌈.
Later on, if the PE figures out that C-S is no longer reachable through the Primary Upstream PE, the Standby Upstream PE becomes the Upstream PE. The PE then resends the C-multicast route with the RT for the Standby Upstream PE, except now the route does NOT carry the Standby PE BGP Community. That replaces the old route with a new one, and the only difference between them is the missing Standby PE BGP Community 🦋. The new Upstream PE must set the LOCAL_PREF attribute for that C-multicast route to the same value it had when the Standby PE BGP Community was included in the advertisement 🌸.
4.2. Upstream PE Behavior
When a PE supporting this specification receives a C-multicast route for a particular (C-S,C-G) where ALL of the following are true 🔥:
- the RT carried in the route results in importing the route into a particular VRF on the PE;
- the route carries the Standby PE BGP Community; and
- the PE determines (via a method of failure detection that is outside the scope of this document) that C-S is not reachable through some other PE (more details are in Section 4.3),
then the PE MAY install VRF PIM state corresponding to this Standby BGP C-multicast route (so a PIM Join message gets sent to the CE towards C-S, and the PE receives (C-S,C-G) traffic 🧚), and the PE MAY forward (C-S,C-G) traffic it receives to other PEs through a P-tunnel rooted at the PE 💜.
Furthermore, no matter whether C-S carried in that route is reachable through some other PE:
a. based on local policy, as soon as the PE receives this Standby BGP C-multicast route, the PE MAY install VRF PIM state corresponding to this BGP Source Tree Join route (so Join messages get sent to the CE toward C-S, and the PE receives (C-S,C-G) traffic) ✨; and
b. based on local policy, as soon as the PE receives this Standby BGP C-multicast route, the PE MAY forward (C-S,C-G) traffic to other PEs through a P-tunnel, independently of whether C-S is reachable through some other PE. (note that this implies also doing step a.) 🦋
Doing neither step a nor step b for a given (C-S,C-G) is called "cold root standby" 🧊.
Doing step a but not step b for a given (C-S,C-G) is called "warm root standby" 🌤️.
Doing step b (which implies also doing step a) for a given (C-S,C-G) is called "hot root standby" 🔥.
Note that if an Upstream PE uses an S-PMSI-only policy, it shall advertise an S-PMSI for a (C-S,C-G) as soon as it receives a C-multicast route for (C-S,C-G), normal or Standby. In other words, it shall not wait for a non-Standby C-multicast route before advertising the corresponding S-PMSI 💅.
Section 9.3.2 of [RFC6513] describes the procedures for sending a Source-Active A-D Route as a result of receiving the C-multicast route. These procedures MUST be followed for both the normal and Standby C-multicast routes 🫶.
4.3. Reachability Determination
The Standby Upstream PE can use the following info to figure out whether or not C-S can be reached through the Primary Upstream PE 🔍:
- presence/absence of a unicast VPN route toward C-S
- supposing that the Standby Upstream PE is the egress of the tunnel rooted at the Primary Upstream PE, the Standby Upstream PE can determine the reachability of C-S through the Primary Upstream PE based on the status of this tunnel, determined using the same criteria as the ones described in Section 3.1 (without using the UMH selection procedures of Section 3) 👑
- other mechanisms
4.4. Inter-AS
If the non-segmented inter-AS approach is used, the procedures described in Section 4.1 through Section 4.3 can be applied 🌍.
When MVPNs are used in an inter-AS context with the segmented inter-AS approach described in Section 9.2 of [RFC6514], the procedures in this section can be applied ✨.
Prerequisites for the procedures described below to be applied for a source of a given MVPN are:
- that any PE of this MVPN receives two or more Inter-AS I-PMSI A-D Routes advertised by the AS of the source
- that these Inter-AS I-PMSI A-D Routes have distinct Route Distinguishers (as described in item "(2)" of Section 9.2 of [RFC6514]) 🎀
As an example, these conditions will be satisfied when the source is dual homed to an AS that connects to the receiver AS through two ASBRs using autoconfigured RDs 🧚.
4.4.1. Inter-AS Procedures for Downstream PEs, ASBR Fast Failover
The following procedure is applied by downstream PEs of an AS, for a source S in a remote AS 🦋.
In addition to choosing an Inter-AS I-PMSI A-D Route advertised from the AS of the source to construct a C-multicast route, as described in Section 11.1.3 of [RFC6514], a downstream PE will choose a second Inter-AS I-PMSI A-D Route advertised from the AS of the source. It uses this route to construct and advertise a Standby C-multicast route (a C-multicast route carrying the Standby extended community), as described in Section 4.1 💖.
4.4.2. Inter-AS Procedures for ASBRs
When an Upstream ASBR receives a C-multicast route, and at least one of the RTs of the route matches one of the ASBR Import RTs, the ASBR that supports this specification must try to locate an Inter-AS I-PMSI A-D Route whose RD and Source AS respectively match the RD and Source AS carried in the C-multicast route. If the match is found, and the C-multicast route carries the Standby PE BGP Community, then the ASBR implementation that supports this specification MUST be configurable to perform as follows 🔧:
- If the route was received over iBGP and its LOCAL_PREF attribute is set to zero, then it MUST be re-advertised in eBGP with a MED attribute (MULTI_EXIT_DISC) set to the highest possible value (
0xffff). - If the route was received over eBGP and its MED attribute is set to
0xffff, then it MUST be re-advertised in iBGP with a LOCAL_PREF attribute set to zero.
Other ASBR procedures are applied without modification and, when applied, MAY modify the above-listed behavior 🌸.
5. Hot Root Standby
The mechanisms defined in Sections 3 and 4 can be used together as follows 🔥.
The principle is that, for a given VRF (or possibly only for a given (C-S,C-G)):
- Downstream PEs advertise a Standby BGP C-multicast route (based on Section 4).
- Upstream PEs use the "hot standby" optional behavior and will thus start forwarding traffic for a given multicast state after they have a (primary) BGP C-multicast route or a Standby BGP C-multicast route for that state (or both) ✨.
- A policy controls from which tunnel downstream PEs accept traffic. For example, the policy could be based on the status of the tunnel or tunnel-monitoring method (Section 3.1.5) 🎯.
Other combinations of the mechanisms proposed in Sections 3 and 4 are for further study 🔭.
Note that the same level of protection would be achievable with a simple C-multicast Source Tree Join route advertised to both the primary and secondary Upstream PEs (carrying, as Route Target extended communities, the values of the VRF Route Import Extended Community of each VPN route from each Upstream PE). The advantage of using the Standby semantic is that, supposing downstream PEs always advertise a Standby C-multicast route to the secondary Upstream PE, it allows you to choose the protection level through a change of configuration on the secondary Upstream PE without requiring any reconfiguration of all the downstream PEs 💅🌈.
6. Duplicate Packets
Multicast VPN specifications [RFC6513] impose that a PE only forwards to CEs the packets coming from the expected Upstream PE (Section 9.1 of [RFC6513]) 🚫.
We want to draw your attention to the fact that respecting this part of the MVPN specifications is especially important when two distinct Upstream PEs are susceptible to forwarding the same traffic on P-tunnels at the same time in the steady state. That will be the case when "hot root standby" mode is used (Section 5), and it can also be the case if the procedures of Section 3 are used. Likewise, it can also be the case when a) the rules determining the status of a tree are not the same on two distinct downstream PEs, or b) the rule determining the status of a tree depends on conditions local to a PE (e.g., the PE-P upstream link being Up) 😱💜.
7. IANA Considerations
7.1. Standby PE Community
IANA has allocated the BGP "Standby PE" community value 0xFFFF0009 from the "Border Gateway Protocol (BGP) Well-known Communities" registry using the First Come First Served registration policy 🎀✨.
7.2. BFD Discriminator ✨
okay so this section makes a brand new BGP optional transitive attribute called "BFD Discriminator" 💖 IANA gave it codepoint 38 in the "BGP Path Attributes" registry 🎀
IANA also made a new "BFD Mode" subregistry inside the "Border Gateway Protocol (BGP) Parameters" registry 🦋 The registration policies for it, per [RFC8126], follow Table 1:
+===========+=========================+
| Value | Policy |
+===========+=========================+
| 0- 175 | IETF Review |
+-----------+-------------------------+
| 176 - 249 | First Come First Served |
+-----------+-------------------------+
| 250 - 254 | Experimental Use |
+-----------+-------------------------+
| 255 | IETF Review |
+-----------+-------------------------+
Table 1: "BFD Mode" Subregistry
Registration PoliciesIANA made the initial assignments in Table 2 💅
+===========+==================+===============+
| Value | Description | Reference |
+===========+==================+===============+
| 0 | Reserved | This document |
+-----------+------------------+---------------+
| 1 | P2MP BFD Session | This document |
+-----------+------------------+---------------+
| 2- 175 | Unassigned | |
+-----------+------------------+---------------+
| 176 - 249 | Unassigned | |
+-----------+------------------+---------------+
| 250 - 254 | Experimental Use | This document |
+-----------+------------------+---------------+
| 255 | Reserved | This document |
+-----------+------------------+---------------+
Table 2: "BFD Mode" Subregistry7.3. BFD Discriminator Optional TLV Type 🌸
IANA also created a new "BFD Discriminator Optional TLV Type" subregistry in the "Border Gateway Protocol (BGP) Parameters" registry 🧚 The registration policies for this one, per [RFC8126], are in Table 3:
+===========+=========================+
| Value | Policy |
+===========+=========================+
| 0- 175 | IETF Review |
+-----------+-------------------------+
| 176 - 249 | First Come First Served |
+-----------+-------------------------+
| 250 - 254 | Experimental Use |
+-----------+-------------------------+
| 255 | IETF Review |
+-----------+-------------------------+
Table 3: "BFD Discriminator
Optional TLV Type" Subregistry
Registration PoliciesAnd the initial assignments are in Table 4 🍓
+===========+===================+===============+
| Value | Description | Reference |
+===========+===================+===============+
| 0 | Reserved | This document |
+-----------+-------------------+---------------+
| 1 | Source IP Address | This document |
+-----------+-------------------+---------------+
| 2- 175 | Unassigned | |
+-----------+-------------------+---------------+
| 176 - 249 | Unassigned | |
+-----------+-------------------+---------------+
| 250 - 254 | Experimental Use | This document |
+-----------+-------------------+---------------+
| 255 | Reserved | This document |
+-----------+-------------------+---------------+
Table 4: "BFD Discriminator Optional TLV
Type" Subregistry8. Security Considerations 🔥
okay this part is serious bestie 🙈 This document describes procedures based on [RFC6513] and [RFC6514], so it shares the security considerations that those specs cover 🫶
It also uses P2MP BFD, as defined in [RFC8562], which is itself based on [RFC5880]. Security considerations for each protocol are covered in their own specs. An implementation that supports this specification MUST provide a mechanism to limit the overall amount of capacity used by the BFD traffic (as the combination of the number of active P2MP BFD sessions and the rate of BFD Control packets to process). no cap, this one is non-negotiable 💀
The methods described in Section 3.1 may produce false-negative state changes. Those can trigger an unnecessary convergence in the control plane, which can negatively impact the multicast service provided by the VPN. An operator is expected to consider the network environment and use the available controls of the mechanism used to determine the status of a P-tunnel. 😭
9. References 📚
9.1. Normative References 💜
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC4271] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A
Border Gateway Protocol 4 (BGP-4)", RFC 4271,
DOI 10.17487/RFC4271, January 2006,
<https://www.rfc-editor.org/info/rfc4271>.
[RFC4875] Aggarwal, R., Ed., Papadimitriou, D., Ed., and S.
Yasukawa, Ed., "Extensions to Resource Reservation
Protocol - Traffic Engineering (RSVP-TE) for Point-to-
Multipoint TE Label Switched Paths (LSPs)", RFC 4875,
DOI 10.17487/RFC4875, May 2007,
<https://www.rfc-editor.org/info/rfc4875>.
[RFC5880] Katz, D. and D. Ward, "Bidirectional Forwarding Detection
(BFD)", RFC 5880, DOI 10.17487/RFC5880, June 2010,
<https://www.rfc-editor.org/info/rfc5880>.
[RFC6513] Rosen, E., Ed. and R. Aggarwal, Ed., "Multicast in MPLS/
BGP IP VPNs", RFC 6513, DOI 10.17487/RFC6513, February
2012, <https://www.rfc-editor.org/info/rfc6513>.
[RFC6514] Aggarwal, R., Rosen, E., Morin, T., and Y. Rekhter, "BGP
Encodings and Procedures for Multicast in MPLS/BGP IP
VPNs", RFC 6514, DOI 10.17487/RFC6514, February 2012,
<https://www.rfc-editor.org/info/rfc6514>.
[RFC7606] Chen, E., Ed., Scudder, J., Ed., Mohapatra, P., and K.
Patel, "Revised Error Handling for BGP UPDATE Messages",
RFC 7606, DOI 10.17487/RFC7606, August 2015,
<https://www.rfc-editor.org/info/rfc7606>.
[RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for
Writing an IANA Considerations Section in RFCs", BCP 26,
RFC 8126, DOI 10.17487/RFC8126, June 2017,
<https://www.rfc-editor.org/info/rfc8126>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[RFC8562] Katz, D., Ward, D., Pallagatti, S., Ed., and G. Mirsky,
Ed., "Bidirectional Forwarding Detection (BFD) for
Multipoint Networks", RFC 8562, DOI 10.17487/RFC8562,
April 2019, <https://www.rfc-editor.org/info/rfc8562>.9.2. Informative References 🦋
[MPLS-P2MP-BFD]
Mirsky, G., Mishra, G., and D. Eastlake 3rd, "BFD for
Multipoint Networks over Point-to-Multi-Point MPLS LSP",
Work in Progress, Internet-Draft, draft-mirsky-mpls-p2mp-
bfd-14, March 2021, <https://tools.ietf.org/html/draft-
mirsky-mpls-p2mp-bfd-14>.
[RFC1122] Braden, R., Ed., "Requirements for Internet Hosts -
Communication Layers", STD 3, RFC 1122,
DOI 10.17487/RFC1122, October 1989,
<https://www.rfc-editor.org/info/rfc1122>.
[RFC4090] Pan, P., Ed., Swallow, G., Ed., and A. Atlas, Ed., "Fast
Reroute Extensions to RSVP-TE for LSP Tunnels", RFC 4090,
DOI 10.17487/RFC4090, May 2005,
<https://www.rfc-editor.org/info/rfc4090>.
[RFC4291] Hinden, R. and S. Deering, "IP Version 6 Addressing
Architecture", RFC 4291, DOI 10.17487/RFC4291, February
2006, <https://www.rfc-editor.org/info/rfc4291>.
[RFC7431] Karan, A., Filsfils, C., Wijnands, IJ., Ed., and B.
Decraene, "Multicast-Only Fast Reroute", RFC 7431,
DOI 10.17487/RFC7431, August 2015,
<https://www.rfc-editor.org/info/rfc7431>.Acknowledgments 🎀
The authors want to thank Greg Reaume, Eric Rosen, Jeffrey Zhang, Martin Vigoureux, Adrian Farrel, and Zheng (Sandy) Zhang for their reviews, useful comments, and helpful suggestions. thank you queens 👑
Contributors 🌈
Below is a list of the other contributing authors, in alphabetical order:
Rahul Aggarwal
Arktan
Email: raggarwa_1@yahoo.com
Nehal Bhau
Cisco
Email: NBhau@cisco.com
Clayton Hassen
Bell Canada
2955 Virtual Way
Vancouver
Canada
Email: Clayton.Hassen@bell.ca
Wim Henderickx
Nokia
Copernicuslaan 50
2018 Antwerp
Belgium
Email: wim.henderickx@nokia.com
Pradeep Jain
Nokia
701 E Middlefield Rd
Mountain View, CA 94043
United States of America
Email: pradeep.jain@nokia.com
Jayant Kotalwar
Nokia
701 E Middlefield Rd
Mountain View, CA 94043
United States of America
Email: Jayant.Kotalwar@nokia.com
Praveen Muley
Nokia
701 East Middlefield Rd
Mountain View, CA 94043
United States of America
Email: praveen.muley@nokia.com
Ray (Lei) Qiu
Juniper Networks
1194 North Mathilda Ave.
Sunnyvale, CA 94089
United States of America
Email: rqiu@juniper.net
Yakov Rekhter
Juniper Networks
1194 North Mathilda Ave.
Sunnyvale, CA 94089
United States of America
Email: yakov@juniper.net
Kanwar Singh
Nokia
701 E Middlefield Rd
Mountain View, CA 94043
United States of America
Email: kanwar.singh@nokia.comAuthors' Addresses 🧚
Thomas Morin (editor)
Orange
2, avenue Pierre Marzin
22307 Lannion
France
Email: thomas.morin@orange.com
Robert Kebler (editor)
Juniper Networks
1194 North Mathilda Avenue
Sunnyvale, CA 94089
United States of America
Email: rkebler@juniper.net
Greg Mirsky (editor)
ZTE Corp.
Email: gregimirsky@gmail.com📦 where this page comes from
- ietf: rfc9026 2021-04
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms