RFC 6513

Multicast in MPLS/BGP IP VPNs

📅 published February 2012, from ietf

RFC 6513: Multicast in MPLS/BGP IP VPNs ✨

okay bestie, welcome to RFC 6513 🎀 It's an Internet Standards Track doc from the IETF, published in February 2012 and tagged with ISSN 2070-1721. The editors are E. Rosen (Ed.) from Cisco Systems, Inc. and R. Aggarwal (Ed.) from Juniper Networks. It's giving official, and it's very much a proper standards document. 💅

The title block I was given doesn't list any "Obsoletes" or "Updates" entries, so I'm not going to invent any. 🙈

Abstract

so the big idea 🦋 IP multicast traffic inside a BGP/MPLS IP VPN (Virtual Private Network) can't just hop from one VPN site to another by magic. The VPN Service Provider has to set up special protocols and procedures for that to work, and this document is where those get spelled out. ✨

Status of This Memo

this part is the paperwork, but it matters 📜 This is an Internet Standards Track document. It's an IETF product and it reflects the consensus of the IETF community. It went through public review and got approved by the Internet Engineering Steering Group (IESG). If you want more on Internet Standards, check Section 2 of RFC 5741.

For the current status, any errata, and how to send feedback, go to http://www.rfc-editor.org/info/rfc6513. 🌸

Copyright Notice

This document falls under BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as they stood on the publication date. Read those carefully because they spell out your rights and restrictions. Any Code Components pulled out of this document must include Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come with no warranty, as the Simplified BSD License describes. 🎀

Some material in this document came from IETF Documents or IETF Contributions that were published or made publicly available before November 10, 2008. The people who hold the copyright on some of that material may not have given the IETF Trust permission to allow changes outside the IETF Standards Process. So unless you get an adequate license from those copyright holders, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process. The exceptions are formatting it for publication as an RFC or translating it into languages other than English. 🫶

1. Introduction ✨

okay so bestie, [RFC4364] is the spec that lays out the procedures a Service Provider (SP) must implement to offer a particular kind of VPN service ("BGP/MPLS IP VPN") to its customers 💼. That service lets IP unicast packets travel from one customer site to another, but it doesn't give IP multicast traffic a way to make the same trip 🙈. not okay!!

This document is the glow-up 🦋. It extends that service so it can also handle IP multicast traffic. Making that happen takes a bunch of protocols working together as a team 🫶. This document gives the big-picture framework for how all those protocols fit together, and it also gives a detailed spec for some of them. The detailed spec for some of the others already lives in earlier documents or companion documents 📚.

A BGP/MPLS IP VPN service that supports multicast is called a "Multicast VPN" or "MVPN" 💜. iconic name, no notes.

Both this document and its companion document [MVPN-BGP] talk about the BGP messages and procedures used to support MVPN. We made every effort to keep the two documents consistent with each other, but discrepancies may have slipped in 😭. If there's any conflict or other discrepancy about how BGP supports MVPN service, [MVPN-BGP] is the authoritative document 👑, so that's the one to trust.

Throughout this document, we use the term "VPN-IP route" to mean a route that is in either the VPN-IPv4 address family [RFC4364] or the VPN-IPv6 address family [RFC4659] ✨.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119] 🎀. yes, these are the official requirement words, so read them as the spec means them, periodt 💅.

2. Overview 🌸

2.1. Optimality vs. Scalability ✨

Okay so in a "BGP/MPLS IP VPN" [RFC4364] 💅, unicast routing of VPN packets works without keeping any per-VPN state in the core of the SP's network (the "P routers") 🦋. Routing info from a particular VPN is kept only by the Provider Edge routers (the "PE routers", or "PEs") that attach directly to sites of that VPN. Customer data travels through the P routers in tunnels from one PE to another (usually MPLS Label Switched Paths, LSPs), so to support the VPN service, the P routers only need routes to the PE routers. The PE-to-PE routing is optimal, but the state in the P routers depends only on the number of PEs, not on the number of VPNs 🧚

However, to get optimal multicast routing for a particular multicast flow, the P routers that flow travels through have to hold state that is specific to that flow 😭. A multicast flow is identified by the (source, group) tuple, where the source is the IP address of the sender and the group is the IP multicast group address of the destination. Scalability would be poor if the state in the P routers were proportional to the number of multicast flows in the VPNs 💀. So when supporting multicast service for a BGP/MPLS IP VPN, the optimality of the multicast routing must be traded off against the scalability of the P routers. We explain this below in more detail 🎀.

If a particular VPN is transmitting "native" multicast traffic over the backbone, we call it an "MVPN" 🌸. By "native" multicast traffic, we mean packets that a Customer Edge router (a "CE router" or "CE") sends to a PE, such that the IP destination address of the packets is a multicast group address, the packets are multicast control packets addressed to the PE router itself, or the packets are IP multicast data packets encapsulated in MPLS.

We say that the backbone multicast routing for a particular multicast group in a particular VPN is "optimal" if and only if all of the following conditions hold 👑:

When a PE router receives a multicast data packet of that group from a CE router, it transmits the packet in such a way that the packet is received by every other PE router that is on the path to a receiver of that group;
The packet is not received by any other PEs;
While in the backbone, no more than one copy of the packet ever traverses any link.
While in the backbone, if bandwidth usage is to be optimized, the packet traverses minimum cost trees rather than shortest path trees.

Optimal routing for a particular multicast group requires that the backbone maintain one or more source trees that are specific to that flow. Each such tree requires that state be maintained in all the P routers that are in the tree 🌈.

Potentially, this would require an unbounded amount of state in the P routers, since the SP has no control over the number of multicast groups in the VPNs that it supports. The SP also doesn't have any control over the number of transmitters in each group, nor over the distribution of the receivers 🙈.

The procedures defined in this document let an SP provide multicast VPN service without requiring the state maintained by the P routers to be proportional to the number of multicast data flows in the VPNs 🫶. The amount of state is traded off against the optimality of the multicast routing. There's enough flexibility so that a given SP can make its own trade-offs between scalability and optimality. An SP can even let some multicast groups in some VPNs get optimal routing while others don't. Of course, the cost of this flexibility is an increase in the number of options provided by the protocols 🍓.

The basic technique for providing scalability is to aggregate a number of customer multicast flows onto a single multicast distribution tree through the P routers. A number of aggregation methods are supported 🦋.

The procedures defined in this document also accommodate the SP that does not want to build multicast distribution trees in its backbone at all. The ingress PE can replicate each multicast data packet and then unicast each replica through a tunnel to each egress PE that needs to receive the data 💖.

2.1.1. Multicast Distribution Trees 🌳

This document supports the use of a single multicast distribution tree in the backbone to carry all the multicast traffic from a specified set of one or more MVPNs. Such a tree is referred to as an "Inclusive Tree". An Inclusive Tree that carries the traffic of more than one MVPN is an "Aggregate Inclusive Tree". An Inclusive Tree contains, as its members, all the PEs that attach to any of the MVPNs using the tree 🎀.

With this option, even if each tree supports only one MVPN, the upper bound on the state maintained by the P routers is proportional to the number of VPNs supported, rather than to the number of multicast flows in those VPNs 🔥. If the trees are unidirectional, it would be more accurate to say that the state is proportional to the product of the number of VPNs and the average number of PEs per VPN. The state maintained by the P routers can be reduced even further by aggregating more MVPNs onto a single tree. If each such tree supports a set of MVPNs (call it an "MVPN aggregation set"), the state maintained by the P routers is proportional to the product of the number of MVPN aggregation sets and the average number of PEs per MVPN. Thus, the state does not grow linearly with the number of MVPNs 🧠.

However, as data from many multicast groups is aggregated together onto a single Inclusive Tree, it is likely that some PEs will receive multicast data they have no need for. In other words, some degree of optimality has been sacrificed 😬.

This document also provides procedures that enable a single multicast distribution tree in the backbone to be used to carry traffic belonging only to a specified set of one or more multicast groups, from one or more MVPNs. Such a tree is referred to as a "Selective Tree", and more specifically as an "Aggregate Selective Tree" when the multicast groups belong to different MVPNs. By default, traffic from most multicast groups could be carried by an Inclusive Tree, while traffic from, e.g., high bandwidth groups could be carried in one of the Selective Trees. When setting up the Selective Trees, one should include only those PEs that need to receive multicast data from one or more of the groups assigned to the tree. This gives more optimal routing than you can get with only Inclusive Trees, though it requires additional state in the P routers 💜.

2.1.2. Ingress Replication through Unicast Tunnels 🛤️

This document also provides procedures for carrying MVPN data traffic through unicast tunnels from the ingress PE to each of the egress PEs. The ingress PE replicates the multicast data packet received from a CE and sends it to each of the egress PEs using the unicast tunnels. This requires no multicast routing state in the P routers at all, but it puts the entire replication load on the ingress PE router and makes no attempt to optimize the multicast routing 🥹.

2.2. Multicast Routing Adjacencies 🤝

In BGP/MPLS IP VPNs [RFC4364], each CE (Customer Edge) router is a unicast routing adjacency of a PE router, but CE routers at different sites do not become unicast routing adjacencies of each other. This important characteristic is kept for multicast routing. A CE router becomes a multicast routing adjacency of a PE router, but CE routers at different sites do not become multicast routing adjacencies of each other 🫧.

We'll use the term "C-tree" to refer to a multicast distribution tree whose nodes include CE routers. (See Section 3.1 for further explication of this terminology.) The multicast routing protocol on the PE-CE link is presumed to be PIM (Protocol Independent Multicast) [PIM-SM]. Both the ASM (Any-Source Multicast) and the SSM (Source-Specific Multicast) service models are supported, so both shared C-trees and source-specific C-trees are supported. Shared C-trees may be unidirectional or bidirectional. In the latter case, the multicast routing protocol is presumed to be the BIDIR-PIM [BIDIR-PIM] "variant" of PIM-SM. A CE router exchanges "ordinary" PIM control messages with the PE router it's attached to 💅.

Support for PIM-DM (Dense Mode) is outside the scope of this document 🙅‍♀️.

The PEs attaching to a particular MVPN then have to exchange the multicast routing information with each other. Two basic methods for doing this are defined: (1) PE-PE PIM and (2) BGP. In the former case, the PEs need to be multicast routing adjacencies of each other. In the latter case, they do not. For example, each PE may be a BGP adjacency of a route reflector (RR) and not of any other PEs 🔁.

In order to support the "Carrier's Carrier" model of [RFC4364], mLDP (Label Distribution Protocol Extensions for Multipoint Label Switched Paths) [MLDP] may also be supported on the PE-CE interface. The use of mLDP on the PE-CE interface is described in [MVPN-BGP]. The use of BGP on the PE-CE interface is not within the scope of this document 🍰.

2.3. MVPN Definition 📖

An MVPN is defined by two sets of sites: the Sender Sites set and the Receiver Sites set, with the following properties 🌷:

Hosts within the Sender Sites set could originate multicast traffic for receivers in the Receiver Sites set.
Receivers not in the Receiver Sites set should not be able to receive this traffic.
Hosts within the Receiver Sites set could receive multicast traffic originated by any host in the Sender Sites set.
Hosts within the Receiver Sites set should not be able to receive multicast traffic originated by any host that is not in the Sender Sites set.

A site could be both in the Sender Sites set and the Receiver Sites set, which means hosts within that site could both originate and receive multicast traffic. An extreme case is when the Sender Sites set is the same as the Receiver Sites set, in which case all sites could originate and receive multicast traffic from each other 🤯.

Sites within a given MVPN may be either within the same organization or in different organizations, which means an MVPN can be either an Intranet or an Extranet 🏢.

A given site may be in more than one MVPN, which means MVPNs may overlap 🫂.

Not all sites of a given MVPN have to be connected to the same service provider, which means an MVPN can span multiple service providers 🌍.

Another way to look at an MVPN is to say that it is defined by a set of administrative policies. Such policies determine both the Sender Sites set and the Receiver Sites set. The policies are established by MVPN customers, but implemented and realized by MVPN Service Providers using the existing BGP/MPLS VPN mechanisms, such as Route Targets (RTs), with extensions as necessary. Honestly, the policy-first view is so iconic 👑✨.

2.4. Auto-Discovery

Okay so the PEs attached to a given MVPN need to swap control info with each other, which means each one has to find every other PE on that same MVPN first 🔍✨. Strictly speaking (and this part is kinda fun), a PE in the Receiver Sites set only needs to discover the other PEs in the Sender Sites set, and a PE in the Sender Sites set only needs to find the PEs in the Receiver Sites set. The RFC calls this "MVPN Auto-Discovery" 💅.

This doc talks about two ways to do it 🎀:

BGP can handle discovering and keeping track of MVPN membership 💜. PEs tell each other about their MVPN membership using BGP. A PE counts as a "member" of a particular MVPN if it has a VRF (Virtual Routing and Forwarding table, see [RFC4364]) that's configured to hold that MVPN's multicast routing info. This option makes no assumptions about how MVPN multicast data packets get moved through the backbone, so it's super flexible 🌈.
If we know the PE-PE multicast control packets (aka PIM packets) of a particular MVPN are going through a non-aggregated Inclusive Tree that supports the ASM service model (like a tree made by non-SSM PIM-SM or by BIDIR-PIM) 🦋, and the PEs on that MVPN are configured with the group address for that tree, then the PEs can auto-discover each other just by joining the tree and multicasting PIM Hellos over it. No extra fuss, just vibes 🧚.

2.5. PE-PE Multicast Routing Information

Okay, so the BGP/MPLS IP VPN spec [RFC4364] says a PE keeps, at most, one BGP peering with every other PE in the network 🫶. That peering is used to exchange VPN routing info. Route reflectors cut down the number of BGP adjacencies a PE has to keep for that job even more 💖.

This doc describes different options for trading MVPN control info between PE routers, using either PIM or BGP. Each option has its own overhead when it comes to how many routing adjacencies a PE has to maintain with other PEs, so it's giving "choose your fighter" energy 👑. Some of the options let you keep the unicast BGP/MPLS VPN model, where a PE keeps, at most, one BGP routing adjacency with other PE routers to exchange MVPN control info. Iconic ✨. BGP also brings reliable transport and incremental updates 🔥.

Another option is the already-existing "soft state" PIM standard [PIM-SM], which sends periodic complete updates instead 🌸.

2.6. PE-PE Multicast Data Transmission

Like [RFC4364], this doc splits up the procedures for exchanging routing info from the procedures for transmitting data traffic 🎀. So a bunch of different transport technologies can be used in the backbone, which is lowkey amazing 🍓. For Inclusive Trees, those include:

unicast PE-PE tunnels, using encapsulation in MPLS, IP, or GRE (Generic Routing Encapsulation)
multicast distribution trees created by PIM (unidirectional in the SSM or ASM service models, or bidirectional) using IP/GRE encapsulation
point-to-multipoint LSPs created by RSVP - Traffic Engineering (RSVP-TE) or mLDP
multipoint-to-multipoint LSPs created by mLDP

Now, if you want to pack traffic from multiple MVPNs onto a single multicast distribution tree, you need a way for the tree's egresses to demultiplex the traffic and figure out which MVPN each packet belongs to 🧩. This doc specifies a mechanism where upstream label assignment [MPLS-UPSTREAM-LABEL] is used by the root of the tree to assign a label to each flow. The receivers use that label to do the demultiplexing 💅. The doc also describes BGP-based procedures that the root of an Aggregate Tree uses to advertise the Inclusive and/or Selective binding and the demultiplexing info to the leaves of the tree 🦋.

It also covers the data plane encapsulations for supporting the various SP multicast transport options 💜.

Things that are not in scope (sorry bestie 🙈): aggregating traffic from multiple MVPNs onto a single multipoint-to-multipoint LSP or onto a single bidirectional multicast distribution tree. Using multicast distribution trees that support the ASM service model as Selective Trees is also out of scope, and so is using multipoint-to-multipoint LSPs as Selective Trees.

And here's an assumption to keep in mind 🫶: when SP multicast trees are used, traffic for a particular multicast group is sent by a particular PE on only one SP multicast tree. Using multiple SP multicast trees for traffic belonging to one multicast group is outside the scope of this document.

2.7. Inter-AS MVPNs

[RFC4364] describes different options for supporting BGP/MPLS IP unicast VPNs whose provider backbones contain more than one Autonomous System (AS) 🌍. These are known as "inter-AS VPNs". The ASes can belong to the same provider or to different ones. This doc describes how inter-AS MVPNs can be supported for each of the unicast BGP/MPLS VPN inter-AS options 💖.

It also specifies a model where inter-AS MVPN service can be offered without needing a single SP multicast tree to span multiple ASes 🧚. In this model, an inter-AS multicast tree is made up of a number of "segments", one per AS, that get stitched together at AS boundary points. These are called "segmented inter-AS trees". Each segment of a segmented inter-AS tree can use a different multicast transport technology, which is giving mix-and-match energy ✨.

It's also possible to support inter-AS MVPNs with non-segmented source trees that extend across AS boundaries 🌈.

2.8. Optionally Eliminating Shared Tree State

This doc also discusses some options and protocol extensions that can be used to get rid of the need for PE routers to distribute to each other the (,G) and (,G,rpt) states 🙈. Those states show up when the VPNs are creating unidirectional C-trees to support the ASM service model. Eliminating them is optional, so it's a whole "if you want it" situation 💅.

3. Concepts and Framework

3.1. PE-CE Multicast Routing

Okay, so support for multicast in BGP/MPLS IP VPNs is modeled super closely after the support for unicast in BGP/MPLS IP VPNs 💜. A multicast routing protocol runs on the PE-CE interfaces, so the PE and the CE become multicast routing adjacencies on that interface. CEs at different sites do not become multicast routing adjacencies of each other, which is a total boundary moment 🙅‍♀️🎀.

If a PE attaches to n VPNs for which multicast support is provided (aka n "MVPNs"), the PE runs n independent instances of a multicast routing protocol 🧚. We call these "VPN-specific multicast routing instances", or more briefly "multicast C-instances" 🫶. The "VRF" (VPN Routing and Forwarding Table), defined in [RFC4364], is extended to include multicast routing entries as well as unicast routing entries. So each multicast routing entry is associated with a particular VRF 🦋.

Whether a particular VRF belongs to an MVPN or not is determined by configuration ⚙️✨.

This doc doesn't try to support every possible multicast routing protocol that could run on the PE-CE link 🙃. Instead, it only considers multicast C-instances for these protocols:

PIM Sparse Mode (PIM-SM), supporting the ASM service model
PIM Sparse Mode, supporting the SSM service model
PIM Bidirectional Mode (BIDIR-PIM), which uses bidirectional C-trees to support the ASM service model

For the "Carrier's Carrier" model of [RFC4364], mLDP may also be supported on the PE-CE interface 🌸. The use of mLDP on the PE-CE interface is described in [MVPN-BGP]. The use of BGP on the PE-CE interface is not within the scope of this document 🙈.

Since the only multicast C-instances discussed by this doc are PIM-based C-instances, we'll generally just say "PIM C-instances" when we mean the multicast C-instances 💅.

A PE router may also be running a "provider-wide" instance of PIM (a "PIM P-instance") 🌍. In that case it has a PIM adjacency with, e.g., each of its IGP neighbors (i.e., with P routers), but NOT with any CE routers, and not with other PE routers (unless another PE router happens to be an IGP adjacency). P routers would also run the P-instance of PIM but NOT a C-instance 💜. If there is a PIM P-instance, it may or may not have a role to play in supporting VPN multicast, which gets discussed in later sections 🎀. But in no case will the PIM P-instance contain VPN-specific multicast routing information 🔒.

To keep it clear when we're talking about the PIM P-instance versus a PIM C-instance, we add the prefixes "P-" and "C-" to control messages, addresses, and so on 🍓. So a P-Join is a PIM Join that gets processed by the PIM P-instance, and a C-Join is a PIM Join that gets processed by a C-instance 🧚. A P-group address is a group address in the SP's address space, and a C-group address is a group address in a VPN's address space 🌈. A C-tree is a multicast distribution tree that gets constructed and maintained by the PIM C-instances ✨. A C-flow is a stream of multicast packets with a common C-source address and a common C-group address 💖. We use the notation "(C-S,C-G)" to identify specific C-flows 🦋. If a particular C-tree is a shared tree (whether unidirectional or bidirectional) rather than a source-specific tree, we'll sometimes talk about the entire set of flows traveling that tree, identifying the set as "(C-*,C-G)" 🫶💅

3.2. P-Multicast Service Interfaces (PMSIs) 💅

okay so, a PE must have the ability to forward multicast data packets it gets from a CE over to one or more of the other PEs in the same MVPN, so those packets can reach one or more other CEs. 🦋

we're going to define the notion of a "P-Multicast Service Interface" (PMSI) 💖. if a particular MVPN is supported by a particular set of PE routers, then there will be one or more PMSIs connecting those PE routers and/or subsets of them ✨. a PMSI is basically a conceptual "overlay" on the P-network with this super important property: a PE in a given MVPN can hand a packet to the PMSI, and the packet gets delivered to some or all of the other PEs in the MVPN. any PE that receives the packet can tell which MVPN it belongs to, which is so iconic 🙌.

a PMSI can be built out of a bunch of different transport mechanisms, depending on what the MVPN needs and what the SP needs 🧚. we're calling those transport mechanisms "P-tunnels" 🎀.

for each MVPN there are one or more PMSIs used to send the MVPN's multicast data from one PE to the others 🌈. we use the term "PMSI" so that a single PMSI belongs to a single MVPN. BUT the transport mechanism that builds a PMSI is allowed to let one P-tunnel carry the data of multiple PMSIs, which is giving efficiency 😭.

in this doc we are very intentional about separating the multicast service (the PMSI) from how it's actually instantiated 🫶. this way we can talk about different services without getting tangled up in the different ways each one can be built. the term "P-tunnel" refers to the transport mechanism that instantiates a service 💜.

PMSIs carry C-multicast data traffic 🍓. that C-multicast traffic travels along a C-tree, but inside the SP backbone every C-tree gets tunneled through P-tunnels. so we'll sometimes say a P-tunnel is carrying one or more C-trees 🌸.

some of the options for passing multicast control traffic among the PEs send that control traffic through a PMSI, and other options don't 👑.

3.2.1. Inclusive and Selective PMSIs 🌷

there are three different kinds of PMSIs we need to know about 💅:

"Multidirectional Inclusive" PMSI (MI-PMSI) 🦋

A Multidirectional Inclusive PMSI is one that lets ANY PE attached to a particular MVPN send a message so that it gets received by EVERY other PE attached to that MVPN 🔥.

There is, at most, one MI-PMSI per MVPN. (The P-tunnel or P-tunnels that build an MI-PMSI might actually carry the data of more than one PMSI 🧚.)

you can think of an MI-PMSI as an overlay broadcast network connecting the set of PEs that support a particular MVPN 💖.

"Unidirectional Inclusive" PMSI (UI-PMSI) 🌈

A Unidirectional Inclusive PMSI is one that lets a particular PE, attached to a particular MVPN, send a message so that it gets received by all the other PEs attached to that MVPN ✨. There is, at most, one UI-PMSI per PE per MVPN, though the P-tunnel that builds a UI-PMSI may actually carry the data of more than one PMSI 🎀.

"Selective" PMSI (S-PMSI). 🫶

A Selective PMSI is one that gives a particular PE in an MVPN a way to multicast messages so that they get received by a subset of the other PEs of that MVPN 💜. there can be an arbitrary number of S-PMSIs per PE per MVPN 😭. the P-tunnel that builds a given S-PMSI may carry data from multiple S-PMSIs 🍓.

later sections talk about the role each of these PMSI kinds plays 🌸. we'll use "I-PMSI" when we aren't distinguishing between "MI-PMSIs" and "UI-PMSIs" 👑.

3.2.2. P-Tunnels Instantiating PMSIs 🧚

the P-tunnels that are used to instantiate PMSIs get called "P-tunnels" 🎀. a bunch of different tunnel setup techniques can be used to create them. here are some of the big ones 🌈:

A PMSI can be instantiated as (a set of) Multicast Distribution trees created by the PIM P-instance ("P-trees") ✨.

The multicast distribution trees that instantiate I-PMSIs may be either shared trees or source-specific trees 💖.

This document (along with [MVPN-BGP]) specifies procedures for identifying a particular (C-S,C-G) flow and assigning it to a particular S-PMSI. Such an S-PMSI is most naturally instantiated as a source-specific tree 🦋.

The use of shared trees (including bidirectional trees) to instantiate S-PMSIs is outside the scope of this document 🙈.

The use of PIM-DM to create P-tunnels is not supported 💀.

P-tunnels may be shared by multiple MVPNs (i.e., a given P-tunnel may be the instantiation of multiple PMSIs), as long as the tunnel encapsulation provides some means of demultiplexing the data traffic by MVPN 🫶.

mLDP

mLDP Point-to-Multipoint (P2MP) LSPs or Multipoint-to-Multipoint (MP2MP) LSPs can be used to instantiate I-PMSIs 🌸.

An S-PMSI or a UI-PMSI could be instantiated as a single mLDP P2MP LSP, whereas an MI-PMSI would have to be instantiated as a set of such LSPs (each PE in the MVPN being the root of one such LSP) or as a single MP2MP LSP 🍓.

Procedures for sharing MP2MP LSPs across multiple MVPNs are outside the scope of this document 🙈.

The use of MP2MP LSPs to instantiate S-PMSIs is outside the scope of this document 💅.

Section 11.2.3 discusses a way of using a partial mesh of MP2MP LSPs to instantiate a PMSI. However, a full specification of the necessary procedures is outside the scope of this document 👑.

RSVP-TE

A PMSI may be instantiated as one or more RSVP-TE Point-to- Multipoint (P2MP) LSPs. An S-PMSI or a UI-PMSI would be instantiated as a single RSVP-TE P2MP LSP, whereas a Multidirectional Inclusive PMSI would be instantiated as a set of such LSPs, one for each PE in the MVPN. RSVP-TE P2MP LSPs can be shared across multiple MVPNs 🔥.

A Mesh of Unicast P-Tunnels.

If a PMSI is implemented as a mesh of unicast P-tunnels, a PE wishing to transmit a packet through the PMSI would replicate the packet and send a copy to each of the other PEs 🌈.

An MI-PMSI for a given MVPN can be instantiated as a full mesh of unicast P-tunnels among that MVPN's PEs. A UI-PMSI or an S-PMSI can be instantiated as a partial mesh ✨.

it's clear that each method of building PMSIs has its own area where it shines 🧚. so this spec lets you use any of these methods 💖. at first glance it might seem like an overabundance of options, but the history of multicast development and deployment makes it obvious that no single option is always acceptable 🦋. using segmented inter-AS trees lets each SP pick whatever option it finds most applicable in its own environment, without making any other SP pick that same option 🎀.

SPECIFYING THE CONDITIONS UNDER WHICH A PARTICULAR TREE-BUILDING METHOD IS APPLICABLE IS OUTSIDE THE SCOPE OF THIS DOCUMENT 🙈.

the choice of tunnel technique belongs to the sender router and is a local policy decision of that router 👑. the procedures in this document don't force the same tunnel technique to be used for all P-tunnels going through a given provider backbone 🌸. however, any tunnel technique a PE can use for a particular MVPN is expected to also be supported by all the other PEs that have VRFs for that MVPN 🫶. and if any PE uses ingress replication for an MVPN, then all other PEs MUST use ingress replication for this MVPN 🔥.

3.3. Use of PMSIs for Carrying Multicast Data 🎀

Okay bestie, every PE supporting a particular MVPN must have a way of discovering the following info 💅:

The set of other PEs in its AS that are attached to sites of that MVPN, and the set of other ASes that have PEs attached to sites of that MVPN. However, if non-segmented inter-AS trees are used (see Section 8.1), then each PE needs to know the entire set of PEs attached to sites of that MVPN.
If segmented inter-AS trees are to be used, the set of border routers in its AS that support inter-AS connectivity for that MVPN.
If the MVPN is configured to use an MI-PMSI, the information needed to set up and to use the P-tunnels instantiating the MI-PMSI.
For each other PE, whether the PE supports Aggregate Trees for the MVPN, and if so, the demultiplexing information that must be provided so that the other PE can determine whether a packet that it received on an Aggregate Tree belongs to this MVPN.

In some cases, that info comes from the BGP-based auto-discovery procedures in Section 4 of this document and in Section 9 of [MVPN-BGP] ✨. In other cases, it gets handed over after discovery is done, through the procedures in Section 7.4. Either way, the info has to be enough to bind the PMSI to the identified P-tunnel, create the P-tunnel if it doesn't already exist 🦋, and properly demultiplex the different PMSIs that might be riding the same P-tunnel. No cap, that's a lot of responsibility 😭.

If an MVPN uses an MI-PMSI, the PEs attached to the MVPN need to know the info identifying the P-tunnels that instantiate the MI-PMSI before any data can be sent on the MI-PMSI 🌸. That info is either statically configured or auto-discovered (see Section 4). Building the P-tunnels (with PIM, RSVP-TE, or mLDP, for example) SHOULD happen as soon as that info is known 🔥.

When MI-PMSIs are used, they can serve as the default way to carry C-multicast data traffic 💜. When we say an MI-PMSI is the "default" for a particular MVPN, we mean you don't need any special control procedures to bind a C-flow to it. Any C-flows that haven't been bound to other PMSIs just get assumed to travel through the MI-PMSI, lowkey so easy 🧚.

There's no requirement to use MI-PMSIs as the default for carrying C-flows 🙈. You can totally go with a policy where all C-flows ride UI-PMSIs or S-PMSIs. In that case, if an MI-PMSI isn't being used to carry routing info either, it's not needed at all.

Even when an MI-PMSI is the default for an MVPN's C-flows, a particular C-flow with certain characteristics might be better off migrating from the MI-PMSI to an S-PMSI 👑. Those characteristics, plus the procedures for doing the migration, are covered in Section 7.

Sometimes a bunch of C-flows are traveling the same shared C-tree (unidirectional or bidirectional), and it might make sense to move the whole set as one unit to an S-PMSI 🍓. The procedures for that are outside the scope of this specification, so we're leaving them alone 🫶.

Some procedures for sending C-multicast routing info between PEs require that info to go over an MI-PMSI. Other procedures don't use an MI-PMSI for that at all 💖.

For a given MVPN, whether an MI-PMSI carries C-multicast routing info is independent of whether an MI-PMSI is the default for carrying the C-multicast data traffic ✨. Those are two separate decisions, bestie.

As mentioned before, you can send all C-flows on a set of S-PMSIs and skip I-PMSIs entirely 🦋. This stops PEs from receiving data they don't need, but the tradeoff is needing more P-tunnels and more signaling to bind the C-flows to them. Making S-PMSIs cost-effective will probably require Aggregate P-trees, and that means the transmitting PE has to know which PEs need which multicast streams. This is called "explicit tracking" 🎀, and the procedures that enable it can cost something themselves. More on that in Section 7.4.1.2.

3.4. PE-PE Transmission of C-Multicast Routing 💅

As a PE attached to a given MVPN gets C-Join/Prune messages from its CEs in that MVPN, it has to pass the info in those messages along to the other PEs attached to the same MVPN 🌈.

There are a few different ways to do this 🧚. These methods aren't interoperable with each other, so the method used for a particular MVPN must be either configured or discovered as part of the auto-discovery process 👑.

3.4.1. PIM Peering

3.4.1.1. Full per-MVPN PIM Peering across an MI-PMSI

If the PEs attached to a given MVPN are connected via an MI-PMSI, they can form "normal" PIM adjacencies with each other 💜. Since the MI-PMSI works like a broadcast network, the standard PIM procedures for forming and maintaining adjacencies over a LAN can be applied 🔥.

So the C-Join/Prune messages a PE gets from a CE can be multicast to all the other PEs in the MVPN ✨. PIM "Join suppression" can be enabled, and the PEs can send Asserts as needed 🦋.

This procedure is fully specified in Section 5.2 💖.

3.4.1.2. Lightweight PIM Peering across an MI-PMSI

The procedure in the previous Section has these disadvantages 😭:

Periodic Hello messages must be sent by all PEs. Standard PIM procedures require that each PE in a particular MVPN periodically multicast a Hello to all the other PEs in that MVPN. If the number of MVPNs gets really big, sending and receiving all these Hellos can turn into a serious overhead for the PE routers 🙈.
Periodic retransmission of C-Join/Prune messages. PIM is a "soft-state" protocol, where reliability comes from frequent retransmissions (refreshes) of control messages. This can also start piling a big overhead onto the PE routers as the number of MVPNs grows 💀.

The first disadvantage is honestly easy to fix 🫶. The reason for the periodic PIM Hellos is to make sure each PIM speaker on a LAN knows who all the other PIM speakers are. But in the MVPN context, PEs in a given MVPN can learn the identities of all the other PEs through the BGP-based auto-discovery procedure of Section 4 🌸. Once that's happening, the periodic Hellos wouldn't serve any function and could just be eliminated. (Of course, this does imply a change to the standard PIM procedures 👀.)

When Hellos are suppressed, we can call it "lightweight PIM peering" ✨.

The periodic refresh of C-Join/Prune messages isn't as simple to get rid of 🥺. If and when "refresh reduction" procedures get specified for PIM, it could be useful to bring them in, so the lightweight PIM peering procedures become even more lightweight 🍓.

Lightweight PIM peering is not specified in this document 💅.

3.4.1.3. Unicasting of PIM C-Join/Prune Messages

PIM doesn't require that the C-Join/Prune messages a PE gets from a CE be multicast to all the other PEs. It also allows them to be unicast to a single PE: the one that's upstream on the path to the root of the multicast tree named in the Join/Prune message 🦋. Note that when C-Join/Prune messages are unicast, there's no such thing as "Join suppression" 🙈. So PIM Refresh Reduction may be considered a prerequisite for the procedure of unicasting C-Join/Prune messages 💜.

When C-Join/Prune messages are unicast, they aren't sent on a PMSI at all 🧚. Also, unicasting C-Join/Prune messages is different from sending them on an MI-PMSI that's instantiated as a mesh of unicast P-tunnels, so don't mix those up, bestie 😭.

If there are multiple PEs that can be used to reach a given C-source, the procedures in Sections 5.1 and 9 MUST be used to make sure duplicate packets don't get delivered 🚨. That one's non-negotiable 🫶.

Procedures for unicasting PIM control messages aren't further specified in this document 🌈.

3.4.2. Using BGP to Carry C-Multicast Routing

It's possible to use BGP to carry C-multicast routing info from PE to PE 👑, which means you can skip sending C-Join/Prune messages from PE to PE entirely 💖. This is discussed in Section 5.3 and fully specified in [MVPN-BGP] 🎀.

4. BGP-Based Auto-Discovery of MVPN Membership ✨

okay so BGP-based auto-discovery runs on a brand new address family, the MCAST-VPN address family 💅. It has other uses too, which we'll get to later 🦋. Any PE that attaches to an MVPN must send a BGP Update message with an NLRI ("Network Layer Reachability Information" element) in this address family, plus a specific set of attributes. This doc only spells out the information that must be inside those BGP Updates so auto-discovery works. The encoding details and the full procedures live in a separate document, [MVPN-BGP] 📚.

This section covers the intra-AS BGP-based auto-discovery procedures. When segmented inter-AS trees are used, extra procedures are needed, as specified in [MVPN-BGP]. (When segmented inter-AS trees are not used, the inter-AS procedures are almost identical to the intra-AS ones 🙈.)

BGP-based auto-discovery uses a special kind of MCAST-VPN route called an "auto-discovery route", or "A-D route" 🌸. Specifically, it uses two kinds of "A-D routes": the "Intra-AS I-PMSI A-D route" and the "Inter-AS I-PMSI A-D route". (There are other kinds too, like the Source Active A-D routes, which do more than auto-discovery. Those get covered in later sections 🌈.)

The Inter-AS I-PMSI A-D route is only used when segmented inter-AS P-tunnels are used, as specified in [MVPN-BGP].

The "Intra-AS I-PMSI A-D route" is originated by the PEs that are (directly) connected to the site(s) of an MVPN. It gets distributed to the other PEs that attach to sites of the MVPN. If segmented inter-AS P-tunnels are used, then the Intra-AS I-PMSI A-D routes are not distributed outside the AS where they originate. If segmented inter-AS P-tunnels are not used, then the Intra-AS I-PMSI A-D routes are, despite their name, distributed to all PEs attached to the VPN, no matter what AS the PEs are in 🧚.

The NLRI of an Intra-AS I-PMSI A-D route must contain the following information:

The route type (i.e., Intra-AS I-PMSI A-D route).
The IP address of the originating PE.
An RD ("Route Distinguisher", [RFC4364]) configured locally for the MVPN. This is an RD that can be prepended to that IP address to form a globally unique VPN-IP address of the PE 💜.

Intra-AS I-PMSI A-D routes carry the following attributes:

Route Target Extended Communities attribute. One or more of these MUST be carried by each Intra-AS I-PMSI A-D route. If any other PE has one of these Route Targets configured for import into a VRF, it treats the advertising PE as a member in the MVPN that the VRF belongs to. This is how each PE finds the PEs that belong to a given MVPN 🔍. More specifically, a PE in the Receiver Sites set can discover the PEs in the Sender Sites set of the MVPN, and the PEs in the Sender Sites set can discover the PEs in the Receiver Sites set. The PEs in the Receiver Sites set would be configured to import the Route Targets advertised in the BGP A-D routes by PEs in the Sender Sites set. The PEs in the Sender Sites set would be configured to import the Route Targets advertised in the BGP A-D routes by PEs in the Receiver Sites set 🎀.
PMSI Tunnel attribute. This attribute is present whenever the MVPN uses an MI-PMSI, or when it uses a UI-PMSI rooted at the originating router. It contains the following information:
- Tunnel technology, which may be one of the following:
  - Bidirectional multicast tree created by BIDIR-PIM,
  - Source-specific multicast tree created by PIM-SM, supporting the SSM service model,
  - Set of trees (one shared tree and a set of source trees) created by PIM-SM using the ASM service model,
  - Point-to-multipoint LSP created by RSVP-TE,
  - Point-to-multipoint LSP created by mLDP,
  - multipoint-to-multipoint LSP created by mLDP,
  - unicast tunnel.
- P-tunnel identifier. Before a P-tunnel can be constructed to instantiate the I-PMSI, the PE must be able to create a unique identifier for the tunnel. The syntax of this identifier depends on the tunnel technology used 🪄. Each PE attaching to a given MVPN must be configured with information specifying the allowable encapsulations to use for that MVPN, as well as the particular one of those encapsulations that the PE is to identify in the PMSI Tunnel attribute of the Intra-AS I-PMSI A-D routes that it originates.
- Multi-VPN aggregation capability and demultiplexor value. This specifies whether the P-tunnel is capable of aggregating I-PMSIs from multiple MVPNs. This will affect the encapsulation used. If aggregation is to be used, a demultiplexor value to be carried by packets for this particular MVPN must also be specified. The demultiplexing mechanism and signaling procedures are described in Section 6 💅.
PE Distinguisher Labels Attribute. Sometimes one PE needs to advertise an upstream-assigned MPLS label that identifies another PE. Under certain circumstances to be discussed later, a PE that is the root of a multicast P-tunnel will bind an MPLS label value to one or more of the PEs that belong to the P-tunnel, and it will distribute these label bindings using Intra-AS I-PMSI A-D routes. Specification of when this must be done is provided in Sections 6.4.4 and 11.2.2. We refer to these as "PE Distinguisher Labels". 🙌

Note that, as specified in [MPLS-UPSTREAM-LABEL], PE Distinguisher Label values are unique only in the context of the IP address identifying the root of the P-tunnel. They are not necessarily unique per tunnel ⚠️.

5. PE-PE Transmission of C-Multicast Routing 💖

okay this part is the big one 😭. As a PE attached to a given MVPN receives C-Join/Prune messages from its CEs in that MVPN, it must pass the info in those messages along to the other PEs attached to the same MVPN. This is called the "PE-PE transmission of C-multicast routing information" 📨.

This section specifies the procedures used for PE-PE transmission of C-multicast routing information. Not every procedure mentioned in Section 3.4 is specified here. This section focuses on two particular procedures:

Full PIM Peering. This procedure is fully specified herein.
Use of BGP to distribute C-multicast routing. This procedure is described herein, but the full specification appears in [MVPN-BGP].

Those aspects of the procedures that apply to both of the above are also specified fully herein. Specification of other procedures is outside the scope of this document 🫶.

5.1. Selecting the Upstream Multicast Hop (UMH) 🦋

When a PE receives a C-Join/Prune message from a CE, the message identifies a particular multicast flow as belonging either to a source-specific tree (S,G) or to a shared tree (,G). Throughout this section, we use the term "C-root" to refer to S, in the case of a source-specific tree, or to the Rendezvous Point (RP) for G, in the case of (,G). If the route to the C-root is across the VPN backbone, then the PE needs to find the "Upstream Multicast Hop" (UMH) for the (S,G) or (,G) flow. The UMH is either the PE at which (S,G) or (,G) data packets enter the VPN backbone, or the Autonomous System Border Router (ASBR) at which those data packets enter the local AS when traveling through the VPN backbone. The process of finding the upstream multicast hop for a given C-root is known as "upstream multicast hop selection" 🔥.

5.1.1. Eligible Routes for UMH Selection

In the simplest case, the PE does the upstream hop selection by looking up the C-root in the unicast VRF associated with the PE-CE interface over which the C-Join/Prune message was received. The route that matches the C-root will contain the information needed to select the UMH ✨.

However, in some cases, the CEs may be distributing to the PEs a special set of routes that are to be used exclusively for the purpose of upstream multicast hop selection, and not used for unicast routing at all. For example, when BGP is the CE-PE unicast routing protocol, the CEs may be using Subsequent Address Family Identifier 2 (SAFI 2) to distribute a special set of routes that are to be used for, and only for, upstream multicast hop selection. When OSPF [OSPF] is the CE-PE routing protocol, the CE may use an MT-ID (Multi-Topology Identifier) [OSPF-MT] of 1 to distribute a special set of routes that are to be used for, and only for, upstream multicast hop selection. When a CE uses one of these mechanisms to distribute to a PE a special set of routes to be used exclusively for upstream multicast hop selection, these routes are distributed among the PEs using SAFI 129, as described in [MVPN-BGP] 🌈.

Whether the routes used for upstream multicast hop selection are (a) the "ordinary" unicast routes or (b) a special set of routes that are used exclusively for upstream multicast hop selection is a matter of policy. How that policy is chosen, deployed, or implemented is outside the scope of this document. In the following, we will simply refer to the set of routes that are used for upstream multicast hop selection as the "Eligible UMH routes", with no presumptions about the policy by which this set of routes was chosen 🧐.

5.1.2. Information Carried by Eligible UMH Routes

Every route that is eligible for UMH selection SHOULD carry a VRF Route Import Extended Community [MVPN-BGP]. However, if BGP is used to distribute C-multicast routing information, or if the route is from a VRF that belongs to a multi-AS VPN as described in option b of Section 10 of [RFC4364], then the route MUST carry a VRF Route Import Extended Community. This attribute identifies the PE that originated the route 🎀.

If BGP is used for carrying C-multicast routes, OR if "Segmented inter-AS Tunnels" are used, then every UMH route MUST also carry a Source AS Extended Community [MVPN-BGP].

These two attributes are used in the upstream multicast hop selection procedures described below 💜.

5.1.3. Selecting the Upstream PE ✨

okay so step one for picking the upstream multicast hop for a C-root is choosing the Upstream PE for that C-root 💖

The PE that got the C-Join from a CE peeks into the VRF tied to the interfaces the C-Join came in on. It finds the Eligible UMH route that's the best match for the C-root in that C-Join. Call that the "Installed UMH Route" 🦋

Quick tea: the outgoing interface of the Installed UMH Route might be one of the VRF's interfaces. If so, the upstream multicast hop is a CE, and the route to the C-root does not go across the VPN backbone 🙈

Next, look at all the VPN-IP routes that (a) can be imported into the VRF based on their Route Targets, (b) are eligible for UMH selection, and (c) have exactly the same IP prefix as the installed UMH route. The RD doesn't have to match, tho 💅

For each route in that set, figure out its Upstream PE and Upstream RD. If the route has a VRF Route Import Extended Community, the Upstream PE comes from that. If it doesn't, the Upstream PE comes from the route's BGP Next Hop. Either way, the Upstream RD comes from the route's NLRI 🎀

This gives you a set of triples: <route, Upstream PE, Upstream RD> ✨

Call this the "UMH Route Candidate Set." Then the PE MUST select a single route from the set to be the "Selected UMH Route." The matching Upstream PE is the "Selected Upstream PE," and the matching Upstream RD is the "Selected Upstream RD" 👑

There are several ways a PE can pick one route from the candidate set 🧚

The default procedure is the main one, and it MUST be implemented. It picks the route whose Upstream PE address is numerically highest, where a 32-bit IP address is treated as a 32-bit unsigned integer. This is the "default Upstream PE selection." For a given C-root, as long as the routing information used to build the candidate set is stable, all PEs will land on the same default Upstream PE selection. (Different default selections can happen during a routing transient, tho 🌸)

There's also an alternative procedure that MUST be implemented, but it's disabled by default. This one makes sure that, except during a routing transient, every PE picks the same Upstream PE for a given combo of C-root and C-G 🍓

The PEs in the candidate set are numbered from lowest to highest IP address, starting from 0.
The following hash is performed:
- A bytewise exclusive-or of all the bytes in the C-root address and the C-G address is performed.
- The result is taken modulo n, where n is the number of PEs in the candidate set. Call this result N.

The Selected Upstream PE is then the one that sits in position N in the list from step 1 💖

Other hashing algorithms are allowed as well, but not required.

This alternative procedure gives you a form of "equal cost load balancing" 💅 Say egress PEs PE3 and PE4 can reach source C-S at equal cost through ingress PE PE1 or ingress PE PE2. With this load balancing, PE1 can be the ingress PE for (C-S,C-G1) data traffic while PE2 is the ingress PE for (C-S,C-G2) data traffic. Iconic, honestly 🦋

Another procedure, which SHOULD be implemented, is to use the Installed UMH Route as the Selected UMH Route. With this one, a given PE will probably choose the Upstream PE that's closest to it, based on the routing in the SP backbone. So for a given C-root, different PEs may pick different Upstream PEs. That's useful if the C-root is an anycast address, and it can also help if the C-root is in a multihomed site (aka a site attached to multiple PEs) 🌈 BUT, this procedure is more likely to cause steady state duplication of traffic, unless (a) PEs discard data traffic that arrives from the "wrong" Upstream PE or (b) data traffic is carried only in non-aggregated S-PMSIs. This whole issue gets the deep dive in Section 9 🔥

General policy-based procedures for selecting the UMH route are allowed but not required. They aren't discussed further in this specification 🫶

5.1.4. Selecting the Upstream Multicast Hop 🌸

Okay, in certain cases the Selected Upstream Multicast Hop is the same as the Selected Upstream PE. In other cases, the Selected Upstream Multicast Hop is the ASBR that's the BGP Next Hop of the Selected UMH Route 👑

If the Selected Upstream PE is in the local AS, then the Selected Upstream PE is also the Selected Upstream Multicast Hop. This happens if any of the following conditions holds:

The Selected UMH Route has a Source AS Extended Community, and the Source AS is the same as the local AS.
The Selected UMH Route does not have a Source AS Extended Community, but the route's BGP Next Hop is the same as the Upstream PE.

Otherwise, the Selected Upstream Multicast Hop is an ASBR. Figuring out exactly which ASBR it is depends on the inter-AS signaling method being used (PIM or BGP) and on whether segmented or non-segmented inter-AS tunnels are used. These details show up in later sections 🧚

5.2. Details of Per-MVPN Full PIM Peering over MI-PMSI 💜

When an MVPN uses an MI-PMSI, the C-instances of that MVPN can treat the MI-PMSI as a LAN interface and form full PIM adjacencies with each other over that LAN interface 🫶

Using PIM when an MI-PMSI is not in use is outside the scope of this document 🙈

To form full PIM adjacencies, the PEs run the standard PIM procedures on the LAN interface. That includes generating and processing PIM Hello, Join/Prune, Assert, DF (Designated Forwarder) election, and other PIM control messages. All of this runs independently for each C-instance. PIM "Join suppression" SHOULD be enabled 💅

5.2.1. PIM C-Instance Control Packets 🦋

All IPv4 PIM C-instance control packets of a particular MVPN are addressed to the ALL-PIM-ROUTERS (224.0.0.13) IP destination address and sent over the MI-PMSI of that MVPN. While they're in transit through the P-network, the packets are encapsulated however the particular kind of P-tunnel instantiating the MI-PMSI requires. So the C-instance control packets never get processed by the P routers, and MVPN-specific PIM routes can stretch from site to site without ever showing up in the P routers 🌸

The handling of IPv6 PIM C-instance control packets will be specified in a follow-on document 🔥

As specified in Section 5.1.2, when PIM is used to distribute C-multicast routing information, any PE distributing VPN-IP routes that are eligible for use as UMH routes SHOULD include a VRF Route Import Extended Community with each route. For a given VRF, the Global Administrator field of the VRF Route Import Extended Community MUST be set to the same IP address the PE puts in the IP source address field of the PE-PE PIM control messages it sends from that VRF 🎀

Also, BSR (Bootstrap Router Mechanism for PIM) [BSR] messages get treated the same as PIM C-instance control packets, and BSR processing counts as an integral part of PIM C-instance processing ✨

5.2.2. PIM C-Instance Reverse Path Forwarding (RPF) Determination 🧚

Even though PIM treats the MI-PMSI as a LAN interface, unicast routing is not run over it, and there are no unicast routing adjacencies over it. So we need special procedures for figuring out when the MI-PMSI should count as the "RPF Interface" for a particular C-address 😭

The PE follows the procedures of Section 5.1 to determine the Selected UMH Route. If that route is not a VPN-IP route learned from BGP as described in [RFC4364], or if the route's outgoing interface is one of the interfaces associated with the VRF, then ordinary PIM procedures for determining the RPF interface apply 💖

But if the Selected UMH Route is a VPN-IP route whose outgoing interface is not one of the interfaces associated with the VRF, then PIM will consider the RPF interface to be the MI-PMSI associated with the VPN-specific PIM instance 🌈

Once PIM has decided that the RPF interface for a particular C-root is the MI-PMSI, PIM still needs to figure out the "RPF neighbor" for that C-root. This will be one of the other PEs that's a PIM adjacency over the MI-PMSI. Specifically, it'll be the "Selected Upstream PE," as defined in Section 5.1 👑

5.3. Use of BGP for Carrying C-Multicast Routing 🍓

It's possible to use BGP to carry C-multicast routing information from PE to PE, skipping the sending of C-Join/Prune messages from PE to PE entirely 💅 This section describes the procedures for carrying intra-AS multicast routing information. Inter-AS procedures are described in Section 8. The complete specification of both sets of procedures, plus the encodings, can be found in [MVPN-BGP] 🫶

5.3.1. Sending BGP Updates 🦋

The MCAST-VPN address family is used for this. MCAST-VPN routes that carry C-multicast routing information are told apart from the ones carrying auto-discovery information by a "route type" field encoded into the NLRI. To advertise the MVPN routing information in BGP, you need the following. The NLRI contains:

The type of C-multicast route. There are two types:
- source tree join
- shared tree join
The C-group address
The C-source address (In the case of a shared tree join, this is the address of the C-RP.)
The Selected Upstream RD corresponding to the C-root address (determined by the procedures of Section 5.1).

Whenever a C-multicast route is sent, it must also carry the Selected Upstream Multicast Hop corresponding to the C-root address (determined by the procedures of Section 5.1). The Selected Upstream Multicast Hop must be encoded as part of a Route Target Extended Community, so that filters can optionally stop the update from being distributed to BGP speakers other than the Upstream Multicast Hop. See Section 10.1.3 of [MVPN-BGP] for the details 🌸

There's no C-multicast route for the PIM function of pruning a source off the shared tree when a PE switches from a (C-*,C-G) tree to a (C-S,C-G) tree. Section 9 of this document specifies a mandatory procedure that makes sure that if any PE joins a (C-S,C-G) source tree, all other PEs that have joined or will join the (C-*,C-G) shared tree will also join the (C-S,C-G) source tree 🔥

That means we don't need a C-multicast route that prunes C-S off the (C-*,C-G) shared tree when switching from (C-*,C-G) to (C-S,C-G) tree. Literally so clean 🧚✨

5.3.2. Explicit Tracking ✨

okay bestie, listen 👂 the upstream multicast hop is not part of the NLRI in the C-multicast BGP routes 🫶. So if several PEs join the same C-tree, the BGP routes they send look comparable to BGP, and only one gets installed 😭. If a route reflector is involved, the PE that reaches the C-source only knows that one or more of the other PEs have joined the tree, but it has no idea which ones 💀. So this BGP update mechanism does not provide "explicit tracking" 🙈. Explicit tracking isn't on by default because it piles on more state and that hurts scalability 📉. Plus, building the C-PIM messages to send "upstream" for a tree doesn't depend on knowing every downstream PE, so the C-multicast route updates have no real reason to provide explicit tracking 💅.

That said, there are times when explicit tracking is needed so PEs can set up certain kinds of P-trees 🏗️. There are also times when it's just nice to have, for figuring out the best way to aggregate multicast flows onto a given aggregate tree 🧠. Since all of this is about building infrastructure in the P-network (rather than spreading C-multicast routing info around), any explicit tracking that's needed gets handled by a special kind of A-D route called "Leaf A-D routes" 🦋.

Whenever a PE sends an A-D route with a PMSI Tunnel attribute, it can set a bit in that attribute that says "Leaf Information Required" 🎀. A PE that installs such an A-D route MUST respond by generating a Leaf A-D route, saying it needs to join (or be joined to) the specified PMSI Tunnel 🫶. The details are in [MVPN-BGP] 📖.

5.3.3. Withdrawing BGP Updates 🧹

A PE removes itself from a C-multicast tree (shared or source) by withdrawing the matching BGP Update 💖.

If a PE has pruned a C-source from a shared C-multicast tree and now wants to "unprune" that source, it does that by withdrawing the route that pruned the source from the tree 🌸.

5.3.4. BSR 📣

BGP doesn't have a way to carry the control info of BSR packets that a PE gets from a CE 🙈. So BSR support works by having one PE in an MVPN transmit the BSR control messages to all the other PEs in that MVPN 💜.

When a PE needs to send a BSR message for a particular MVPN to other PEs, it must put its own IP address into the BSR message as the IP source address 🌈. As Section 5.1.2 explains, when a PE distributes VPN-IP routes that are eligible for use as UMH routes, the PE MUST include a VRF Route Import Extended Community with each route 💅. For a given MVPN, a single such IP address MUST be used, and that same IP address MUST be used as the source address in all BSR packets that the PE transmits to other PEs 🔥.

The BSR message can go over any PMSI that will deliver it to all the other PEs in the MVPN 🦋. If no such PMSI exists yet, then an appropriate P-tunnel must be advertised, and the C-flow whose C-source address is the PE's own address and whose multicast group is ALL-PIM-ROUTERS (224.0.0.13) must be bound to it ✨. You can do this using the procedures in Sections 7.3 and 7.4 🎀. Heads up though: this is not meant to imply that the other PIM control packets from the PIM C-instance get sent to the other PEs 🙅‍♀️.

When a PE gets a BSR message for a particular MVPN from some other PE, it only accepts the message if the IP source address in that message is the Selected Upstream PE (see Section 5.1.3) for the IP address of the Bootstrap router 👑. Otherwise, it just discards the packet 🗑️. If it does accept the packet, it runs normal BSR processing on it, and it may forward a BSR message to one or more CEs as a result 🫧.

6. PMSI Instantiation 🌷

This section is all about the procedures for using P-tunnels to instantiate a PMSI 🧚. It covers setting up and maintaining the P-tunnels, plus sending and receiving C-data and/or C-control messages over them 💖. Procedures for binding particular C-flows to particular P-tunnels live in Section 7 🔗.

PMSIs can be instantiated either by P-multicast trees or by PE-PE unicast tunnels 🌈. In the second case, the PMSI is said to be instantiated by "ingress replication" 🍓.

This spec supports a bunch of different ways to set up P-multicast trees, and they're detailed below 📋. A P-tunnel can support a single VPN (a non-aggregated P-multicast tree) or multiple VPNs (an aggregated P-multicast tree) 🫶.

6.1. Use of the Intra-AS I-PMSI A-D Route 📡

6.1.1. Sending Intra-AS I-PMSI A-D Routes 📤

When a PE is provisioned with one or more VRFs that provide MVPN support, it announces its MVPN membership info using Intra-AS I-PMSI A-D routes, as discussed in Section 4 and detailed in Section 9.1.1 of [MVPN-BGP] 💅. (Under certain conditions, which [MVPN-BGP] spells out, the Intra-AS I-PMSI A-D route may be omitted 🙈.)

Usually, the Intra-AS I-PMSI A-D route has a PMSI Tunnel attribute that identifies a P-tunnel being used to instantiate the I-PMSI 🚇. Section 9.1.1 of [MVPN-BGP] describes certain conditions where the PMSI Tunnel attribute can be left out (or where a PMSI Tunnel attribute with the "no tunnel information present" bit can be sent) 🫧.

As a special case, when (a) C-PIM control messages are to be sent through an MI-PMSI and (b) the MI-PMSI is instantiated by a P-tunnel technique where each PE only needs to know a single P-tunnel identifier per VPN, then the Intra-AS I-PMSI A-D routes MAY be omitted, and the tunnel identifier can be configured statically instead 🧸. However, this is not recommended for long-term use 😬, and in all other cases, the Intra-AS I-PMSI A-D routes MUST be used 🔥.

The PMSI Tunnel attribute MAY contain an upstream-assigned MPLS label, assigned by the PE that originated the Intra-AS I-PMSI A-D route 🦋. If that label is there, the P-tunnel can carry data from several MVPNs 🌸. The label goes on the data packets moving through the tunnel to identify which MVPN they belong to 🏷️. (The label identifies the packet as belonging to the MVPN identified by the RTs of the Intra-AS I-PMSI A-D route 👑.)

Check Section 12.2 for how to put the label in the packet's label stack 📚.

The Intra-AS I-PMSI A-D route may contain a "PE Distinguisher Labels" attribute 💜. This holds a set of bindings between upstream-assigned labels and PE addresses 🔗. The originating PE can use it to bind an upstream-assigned label to one or more of the other PEs in the same MVPN 🫶. How PE Distinguisher Labels get used is covered in Sections 6.4.1, 6.4.3, 11.2.2, and 12.3 ✨. Any other uses of the PE Distinguisher Labels attribute are outside this document's scope 🙅‍♀️.

6.1.2. Receiving Intra-AS I-PMSI A-D Routes 📥

What a PE does when it receives an Intra-AS I-PMSI A-D route for a particular MVPN depends on the P-tunnel technology that MVPN is using 🧩. If that P-tunnel technology needs tunnels built with receiver-initiated joins, the PE SHOULD join the tunnel immediately ⚡.

6.2. When C-flows Are Specifically Bound to P-Tunnels 🎯

This situation is discussed in Section 7 📖.

6.3. Aggregating Multiple MVPNs on a Single P-Tunnel 🌈

When a P-multicast tree is shared across multiple MVPNs, it's called an "Aggregate Tree" 🌳. The procedures in this document let a single SP multicast tree be shared across multiple MVPNs 🫶. Unless specified otherwise, P-multicast tree technology supports aggregation 💅.

All the procedures specific to multi-MVPN aggregation are OPTIONAL, and they're explicitly pointed out 🎀.

Aggregate Trees let one P-multicast tree serve multiple MVPNs, so state in the SP core grows per set of MVPNs instead of per MVPN 🧠. Depending on how congruent the aggregated MVPNs are, this can mean trading off some optimality of multicast routing ⚖️.

An Aggregate Tree can be used by a PE to provide a UI-PMSI or MI-PMSI service for more than one MVPN 🍓. When that's the case, the Aggregate Tree is said to have an inclusive mapping 🦋.

6.3.1. Aggregate Tree Leaf Discovery 🔍

BGP MVPN membership discovery (Section 4) lets a PE figure out which Aggregate Trees it should create and which MVPNs should map onto each one 🗺️. The leaves of an Aggregate Tree are determined by the PEs that support aggregation and belong to all the MVPNs mapped onto that tree 🫶.

If an Aggregate Tree is used to instantiate one or more S-PMSIs, it can be useful for the root PE to know which PEs (in its MVPN) are receivers on that tree 👀. That helps the PE decide when to aggregate two S-PMSIs based on congruence (as the next section covers) 🧠. So explicit tracking may be needed 💡. Since the procedures for spreading C-multicast routes don't provide explicit tracking, a type of A-D route called a "Leaf A-D route" gets used 🦋. A PE that wants to assign a particular C-multicast flow to a particular Aggregate Tree can send an A-D route, which triggers Leaf A-D routes from the PEs that need to receive that C-multicast flow 📬. That gives the explicit tracking info needed for the aggregation method in the next section ✨. For more on Leaf A-D routes, check out [MVPN-BGP] 📖💜.

6.3.2. Aggregation Methodology ✨

okay so this section is basically "the spec doesn't make you pick one rule for deciding which MVPNs share an Aggregate Tree 🌳". Your team can use implementation-specific heuristics, configuration, or even offline tools. the spec doesn't care, bestie 💅

BUT the big promise here is that the control procedures will always get all the PEs of an MVPN to agree on which PMSIs get used and which tunnels instantiate them. no drama allowed 🫶

Now the vibe check on congruence 🔍. Congruence is how much the leaves of the customer trees that get aggregated onto one SP tree overlap. For Aggregate Trees with an inclusive mapping, it depends on how much the MVPN memberships overlap. If every MVPN has exactly the same sites, aggregation is perfectly congruent 👑. As the overlap shrinks (fewer sites are shared across all the MVPNs), congruence goes down too 📉

If aggregation isn't perfectly congruent, a PE might get traffic for MVPNs it doesn't even belong to 😭. The more multicast traffic in those unwanted MVPNs, the less optimal the aggregation gets for delivered traffic. So it's a classic trade-off: less state vs. delivering unwanted traffic. Not me making the networking world this dramatic 💀

Your implementation should give you knobs to control congruence 🎛️. These are implementation dependent. One example is configuring the percentage of sites that MVPNs must have in common before they can be aggregated 🍓. That lets an SP deploy aggregation based on MVPN membership and traffic profiles in its network. And if different PEs or servers are setting up Aggregate Trees, it also lets the SP engineer the maximum amount of unwanted MVPNs a particular PE might receive traffic for. Main character SP energy 👑✨

6.3.3. Demultiplexing C-Multicast Traffic 🧩

okay this part is about figuring out whose packet is whose 🕵️‍♀️

If a P-multicast tree belongs to only one MVPN, then just knowing which P-multicast tree the packet came in on is enough to know its MVPN 🎀. The egress PE only needs to know which MVPN that P-multicast tree is associated with. Easy peasy 🌸

But when multiple MVPNs are aggregated onto one P-multicast tree, the tree alone is NOT enough 😬. The packet has to carry some demultiplexing info so the egress PEs can figure out its MVPN. And since the packet got multicast through the P-network, any given demultiplexing value has to mean the same thing to all the egress PEs 🫶

Here's the move: the demultiplexing value is an MPLS label that matches the multicast VRF the packet belongs to 💜. The ingress PE puts this label right beneath the P-multicast tree header. Every egress PE has to be able to link this label to the same MVPN ✨.

Why not downstream-assigned labels? Those would make every egress PE in the MVPN agree on one common label, which is a lot of coordination 💀. So instead, the MPLS label is upstream-assigned [MPLS-UPSTREAM-LABEL] 🙌. The label bindings get advertised via BGP Updates that the ingress PEs originate 📢.

This procedure means each egress PE needs a separate label space for every other PE 🧠. The egress PEs make a forwarding entry for the upstream-assigned MPLS label (allocated by the ingress PE) in that label space. So when an egress PE gets a packet over an Aggregate Tree, it first figures out which tree the packet came over 🌳. That tree identifier decides which label space the upstream-assigned label lookup happens in 🔎.

You can use the same label space for every P-multicast tree rooted at the same ingress PE, OR your implementation can go with a separate label space for each P-multicast tree 🎨. Your call, bestie.

Full procedures for aggregation on shared trees or on MP2MP LSPs are outside the scope of this document 🙅‍♀️. Not this one, anyway!

The encapsulation can be either MPLS or MPLS-in-something (for example, MPLS-in-GRE [MPLS-IP]) 🎁. With plain MPLS, this label shows up right below the label that identifies the P-multicast tree. With MPLS-in-GRE, this label is the top MPLS label you see once the GRE header is stripped off 🧚.

When IP encapsulation is used for the P-multicast tree, whatever that encapsulation format uses to identify a particular tunnel is what decides the label space the MPLS label gets looked up in 🦋.

If the P-multicast tree uses MPLS encapsulation, the tree itself is identified by an MPLS label 🏷️. The egress PE MUST NOT advertise IMPLICIT NULL or EXPLICIT NULL for that tree 🙅‍♀️. Once the tree's label is popped off the stack, the next label is the demultiplexing info that tells you which MVPN the packet belongs to 💖.

Here's the requirement: to support this kind of aggregation, there has to be at least one upstream-assigned label per MVPN 🎯. It doesn't require that there be only one, though 🌈. For example, an ingress PE could give each (C-S,C-G) its own unique label. (This could be done with the same technique used to assign a particular (C-S,C-G) to an S-PMSI, see Section 7.4.) 💅

When an egress PE gets a C-multicast data packet over a P-multicast tree, it needs to forward that packet to the CEs with receivers in the packet's C-multicast group 📦. To do that, the egress PE has to figure out which P-tunnel the packet came in on 🧭. Then it can determine the MVPN the packet belongs to and, if needed, do whatever further lookups are required to forward the packet 🚀.

6.4. Considerations for Specific Tunnel Technologies 🛠️

Here's the deal: the architecture in this document doesn't put limits on the protocols used to set up and maintain P-tunnels, as far as we believe 🤞. But the only protocols that have been explicitly considered are PIM-SM (both the SSM and ASM service models are covered, plus bidirectional trees), RSVP-TE, mLDP, and BGP 🌟. (BGP's job in setting up and maintaining P-tunnels is to "stitch" together the intra-AS segments of a segmented inter-AS P-tunnel 🪡. Iconic stitching behavior 🧵.)

6.4.1. RSVP-TE P2MP LSPs 🌊

If an I-PMSI is going to be instantiated as one or more non-segmented P-tunnels, and those P-tunnels are RSVP-TE P2MP LSPs, then ONLY the PEs at the head ends of those LSPs will ever include the PMSI Tunnel attribute in their Intra-AS I-PMSI A-D routes 👑. (These are the PEs in the "Sender Sites set".)

If an I-PMSI is going to be instantiated as one or more segmented P-tunnels, where some of the intra-AS segments are RSVP-TE P2MP LSPs, then only a PE or ASBR at the head end of one of those LSPs will ever include the PMSI Tunnel attribute in its Inter-AS I-PMSI A-D route 🫧.

Other PEs send Intra-AS I-PMSI A-D routes WITHOUT PMSI Tunnel attributes 🙈. (These are the PEs in the "Receiver Sites set" that are NOT in the "Sender Sites set".) Each "Sender Site" PE that receives an Intra-AS I-PMSI A-D route from a PE in the Receiver Sites set adds the PE that originated that route to the set of receiving PEs for the P2MP LSP 📥. Then the PE at the head end MUST use RSVP-TE [RSVP-P2MP] signaling to add the receiver PEs to the P-tunnel 📣.

When RSVP-TE P2MP LSPs are used to instantiate S-PMSIs, and a particular C-flow is supposed to be bound to the LSP, you need explicit tracking 🕵️. That way the head end of the LSP knows which PEs need to receive data from the specified C-flow. If the binding is done using S-PMSI A-D routes (see Section 7.4.1), the "Leaf Information Required" bit MUST be set in the PMSI Tunnel attribute 🚩.

RSVP-TE P2MP LSPs can optionally support aggregation of multiple MVPNs 🍭.

If an RSVP-TE P2MP LSP Tunnel is used for only a single MVPN, the mapping between the LSP and the MVPN can be configured or deduced from the procedures used to announce the LSP (for example, from the RTs in the A-D route that announced the LSP) 🎯. If the LSP is used for multiple MVPNs, the set of MVPNs using it (and the matching MPLS labels) is inferred from the PMSI Tunnel attributes that specify the LSP 🧠.

If an RSVP-TE P2MP LSP is carrying a set of C-flows traveling along a bidirectional C-tree, using the procedures of Section 11.2, the head end MUST include the PE Distinguisher Labels attribute in its Intra-AS I-PMSI A-D route or S-PMSI A-D route 🏷️. It also MUST provide an upstream-assigned label for each PE it has selected as the Upstream PE for the C-tree's RPA (Rendezvous Point Address) 💜. See Section 11.2 for the details, babe 🫶.

A PMSI Tunnel attribute that specifies an RSVP-TE P2MP LSP contains the following info 📋:

The type of the tunnel is set to RSVP-TE P2MP Tunnel
The RSVP-TE P2MP Tunnel's SESSION Object.
Optionally, the RSVP-TE P2MP LSP's SENDER_TEMPLATE Object. This object is included when you want to identify a particular P2MP TE LSP 🔖.

Demultiplexing C-multicast data packets at the egress PE follows the procedures in Section 6.3.3 🧩. As specified there, an egress PE MUST NOT advertise IMPLICIT NULL or EXPLICIT NULL for an RSVP-TE P2MP LSP that is carrying traffic for one or more MVPNs 🙅‍♀️.

If (and ONLY if) a particular RSVP-TE P2MP LSP is possibly carrying data from multiple MVPNs, these special procedures apply 🌈:

A packet in a particular MVPN, when transmitted into the LSP, must carry the MPLS label specified in the PMSI Tunnel attribute that announced that LSP as a P-tunnel for that for that MVPN.
Demultiplexing the C-multicast data packets at the egress PE is done by means of the MPLS label that rises to the top of the stack after the label corresponding to the P2MP LSP is popped off 🪄.

It's possible that when a PE learns, via an A-D route with a PMSI Tunnel attribute, that it needs to receive traffic on a particular RSVP-TE P2MP LSP, the signaling to set up that LSP hasn't finished yet ⏳. In that case, the PE has to wait for the RSVP-TE signaling to happen before it can change its forwarding tables as the A-D route directs 🕰️.

It's also possible that the signaling to set up an RSVP-TE P2MP LSP finishes BEFORE a given PE learns, via a PMSI Tunnel attribute, what that LSP is going to be used for 😳. The PE MUST discard any traffic received on that LSP until then 🗑️.

For the egress PE to be able to discard that traffic, it needs to know that the LSP is associated with an MVPN, and that the A-D route binding the LSP to an MVPN or to a particular C-flow hasn't been received yet 🔐. This is provided by extending [RSVP-P2MP] with [RSVP-OOB] 💖.

6.4.2. PIM Trees 🌸

okay so when the P-tunnels are PIM trees, the PMSI Tunnel attribute carries enough info that every other PE in the same MVPN can use P-PIM signaling to join the P-tunnel 🫶

if an I-PMSI is going to be set up as one or more PIM trees, the PE sitting at the root of a given PIM tree MUST send an Intra-AS I-PMSI A-D route with a PMSI Tunnel attribute holding everything the other PEs need to join that tree 💅

if PIM trees are being used for an MI-PMSI, each PE in the MVPN MUST send an Intra-AS I-PMSI A-D route with a PMSI Tunnel attribute like that 🎀

if a PMSI is going to be done with a shared tree, the PMSI Tunnel attribute names the P-group address. it does NOT say which RP or RPA goes with that group, and that RP MUST be known to all the PEs, obvi 👑 the PEs are presumed to learn the RP-to-group mappings automatically (like with the Bootstrap Router Protocol [BSR]), or else the mapping gets configured by hand ✨

if a PMSI is going to be done with a source-specific tree, the PMSI Tunnel attribute names the PE router that's the root of the tree AND a P-group address 🦋 it always says whether the PIM tree is a unidirectional shared tree, a bidirectional shared tree, or a source-specific tree, no guessing 🙈

if PIM trees are being used for S-PMSIs, the procedures above assume every PE has a set of group P-addresses it can use to set up the PIM trees, and each PE MUST be configured with that set 🧚 if the P-tunnels are source-specific trees, the PEs MAY have overlapping sets of group P-addresses. but if the trees are NOT source-specific, each PE MUST have a unique set (no overlap with any other PE's set) 😭 managing those addresses is way simpler with source-specific trees, so using them is strongly RECOMMENDED whenever you want unidirectional trees 💖

full procedures for using bidirectional PIM trees to set up S-PMSIs are out of scope for this doc 🌈

the details for building the PMSI Tunnel attribute that identifies a PIM tree are in [MVPN-BGP] 💜

6.4.3. mLDP P2MP LSPs 🦋

when the P-tunnels are mLDP P2MP trees, each Intra-AS I-PMSI A-D route has a PMSI Tunnel attribute with enough info for every other PE in the MVPN to use mLDP signaling to join the P-tunnel 🫶 the tunnel identifier is a P2MP Forwarding Equivalence Class (FEC) Element [mLDP] ✨

an mLDP P2MP LSP MAY carry the traffic of multiple VPNs, as long as the PMSI Tunnel attribute that specifies it has a non-zero MPLS label 🌸

if an mLDP P2MP LSP is carrying the flows along a particular bidirectional C-tree, using the procedures of Section 11.2, the root of the LSP MUST include the PE Distinguisher Labels attribute in its Intra-AS I-PMSI A-D route or S-PMSI A-D route, AND it MUST provide an upstream-assigned label for the PE it picked to be the Upstream PE for the C-tree's RPA 👑 see Section 11.2 for the deets 💅

6.4.4. mLDP MP2MP LSPs 💀

how to assign C-flows to mLDP MP2MP LSPs that act as P-tunnels is out of scope for this doc, so no spoilers here 🙈

6.4.5. Ingress Replication 🍓

like we talked about in Section 3, a PMSI can be set up with Unicast Tunnels between the PEs in the MVPN 🎀 here the ingress PE replicates a C-multicast data packet that belongs to a particular MVPN and sends a copy to all or some of the PEs in that MVPN. each copy gets tunneled to a remote PE over a Unicast Tunnel to that remote PE 💖 IP/GRE Tunnels or MPLS LSPs are examples of unicast tunnels that can be used 🌈 and the same Unicast Tunnel CAN carry packets for different MVPNs ✨

so for a PE to use Unicast P-tunnels to send a C-multicast data packet for a particular MVPN to a set of remote PEs, those remote PEs MUST be able to correctly decapsulate such packets and assign each one to the right MVPN 🫶 that means the encapsulation used on the P-tunnel MUST have demultiplexing info the receiver can tie to a specific MVPN 💅

if ingress replication is being used to set up the PMSIs for an MVPN, the PEs announce that as part of the BGP-based MVPN membership auto-discovery process (see Section 4) 🦋 the PMSI Tunnel attribute says ingress replication AND carries a downstream-assigned MPLS label 🌸 that label identifies which MVPN a packet belongs to (as inferred from its RTs) 👑 so if PE1 specifies a particular label value for a particular MVPN, then any other PE sending PE1 a packet for that MVPN through a unicast P-tunnel MUST put that label on the packet's label stack 😭 PE1 then treats that label as the demultiplexor value identifying that MVPN 🧚

ingress replication MAY be used to set up any kind of PMSI 💖 when ingress replication is done, it is RECOMMENDED (except in the one particular case in the next paragraph) that explicit tracking gets done, so a particular C-flow's data packets only go to the PEs that actually need to see that C-flow 🌈 there is never any need to use the procedures of Section 7.4 for binding particular C-flows to particular P-tunnels ✨

the one particular case where explicit tracking is NOT needed is when ingress replication is used to create a one-hop ASBR-ASBR inter-AS segment of a segmented inter-AS P-tunnel 💅

Section 9.1 lays out three different methods for preventing duplication of multicast data packets 🎀 any given deployment MUST use at least one of them 🫶 heads up that the method in Section 9.1.1 ("Discarding Packets from Wrong PE") assumes that the egress PE of a P-tunnel can, on receiving a packet from the P-tunnel, figure out which PE sent the packet into the P-tunnel 👀 SPs that use ingress replication to set up their PMSIs are cautioned AGAINST using it for this purpose with unicast P-tunnel technologies that don't let the egress PE identify the ingress PE (for example, MP2P LSPs where penultimate-hop-popping is done) 💀 deploying ingress replication with that kind of P-tunnel technology MUST NOT happen unless it's known that the deployment relies entirely on the procedures of Sections 9.1.2 or 9.1.3 for duplicate prevention 🚫💅

Binding Specific C-Flows to Specific P-Tunnels 🦋

okay so as we covered earlier, Intra-AS I-PMSI A-D routes MAY (or MAY NOT) have PMSI Tunnel attributes, naming P-tunnels that can serve as the default P-tunnels for carrying C-multicast traffic, aka C-multicast traffic that hasn't been specifically bound to another P-tunnel 🌸

if none of the Intra-AS I-PMSI A-D routes a particular PE originates for a particular MVPN carry PMSI Tunnel attributes at all (or the only ones they carry have type "No tunnel information present"), then that PE has no default P-tunnels to use when sending C-multicast traffic in that MVPN to other PEs 😭 in that case, ALL such C-flows MUST be assigned to specific P-tunnels using one of the mechanisms in Section 7.4 💅 so all of them end up on P-tunnels that instantiate S-PMSIs 🧚

there are other cases where it might be necessary or just desirable to use the Section 7.4 mechanisms to identify specific C-flows and bind them to, or unbind them from, specific P-tunnels 👑 some possible cases:

the policy for a particular MVPN is to send ALL C-data on S-PMSIs, even if the Intra-AS I-PMSI A-D routes carry PMSI Tunnel attributes 🎀 (this is another case where all C-data rides on S-PMSIs; presumably the I-PMSIs are for control info 🫶)
it's desired to optimize the routing of a particular C-flow, which may already be traveling on an I-PMSI, by sending it on an S-PMSI instead 💖
if a particular C-flow is traveling on an S-PMSI, it may be considered desirable to move it to an I-PMSI (aka optimizing the routing for that flow may no longer be considered desirable 🙈)
it's desired to change the encapsulation used to carry the C-flow, e.g. because someone wants to aggregate it on a P-tunnel with flows from other MVPNs 🌈

note that if Full PIM Peering over an MI-PMSI (Section 5.2) is being used, then from the PIM state machine's point of view, the "interface" connecting the PEs to each other is the MI-PMSI 🦋 even if some or all of the C-flows are being sent on S-PMSIs ✨ so from the C-PIM state machine's perspective, when a C-flow is sent or received on an S-PMSI, the output or input interface (respectively) is considered to be the MI-PMSI 💅

Section 7.1 covers general considerations that apply whenever a specified C-flow is bound to a specified P-tunnel using the Section 7.4 mechanisms 🫶 this includes the case where the C-flow moves from one P-tunnel to another, AND the case where the C-flow is first bound to an S-PMSI P-tunnel 🌸

Section 7.2 covers the specific case of using the Section 7.4 mechanisms to optimize multicast routing by switching specific flows from one P-tunnel to another 🎀

Section 7.3 covers the case where the Section 7.4 mechanisms are used to announce the presence of "unsolicited flooded data" and assign that data to a particular P-tunnel 👑

Section 7.4 specifies the protocols for assigning specific C-flows to specific P-tunnels 💖 these protocols can assign a C-flow to a P-tunnel initially, or switch a flow from one P-tunnel to another 🌈

procedures for binding to a specified P-tunnel the set of C-flows traveling along a specified C-tree (or binding a set of C-flows that share some relevant characteristic, without identifying each flow individually) are out of scope for this doc ✨

7.1. General Considerations 🧚

7.1.1. At the PE Transmitting the C-Flow on the P-Tunnel ✨

Okay bestie, the call to bind a particular C-flow (designated (C-S,C-G)) to a particular P-tunnel, or to switch that C-flow over to a different P-tunnel, is always made by the PE that's going to transmit the C-flow onto the P-tunnel 💅

Whenever a PE moves a C-flow from one P-tunnel, say P1, to another, say P2, you have to be SUPER careful about steady state duplication of traffic 😭 At any given moment the PE sends the C-flow either on P1 or on P2, but never on both. Main character energy is not sending the same packet twice 🙈

So when a particular PE, say PE1, decides to bind a particular C-flow to a particular P-tunnel, say P2, the following procedures MUST be applied:

PE1 must issue the required control plane information to signal that the specified C-flow is now bound to P-tunnel P2 (see Section 7.4).
If P-tunnel P2 needs to be constructed from the root downwards, PE1 must initiate the signaling to construct P2. This is only required if P2 is an RSVP-TE P2MP LSP.
If the specified C-flow is currently bound to a different P-tunnel, say P1, then:
- PE1 MUST wait for a "switch-over" delay before sending traffic of the C-flow on P-tunnel P2. It is RECOMMENDED to allow this delay to be configurable.
- Once the "switch-over" delay has elapsed, PE1 MUST send traffic for the C-flow on P2 and MUST NOT send it on P1. In no case is any C-flow packet sent on both P-tunnels.

When a C-flow is switched from one P-tunnel to another, the whole point of the switch-over timer is to minimize packet loss without introducing packet duplication. Heads up though: jitter may creep in because the transit delays differ between the old and new P-tunnels 🥺

For best effect, set the switch-over timer to a value that's "just long enough" (a) to let all the PEs learn about the new binding of the C-flow to the P-tunnel, and (b) to let the PEs construct the P-tunnel, if it doesn't already exist 🍓

If, after such a switch, the "old" P-tunnel P1 is no longer needed, it SHOULD be torn down and the resources supporting it freed 🧹 How "tearing down" works is specific to the P-tunnel technology.

Procedures for binding sets of C-flows traveling along specified C-trees (or sets of C-flows sharing any other characteristic) to a specified P-tunnel (or for moving them from one P-tunnel to another) are outside the scope of this document 🙅‍♀️

7.1.2. At the PE Receiving the C-flow from the P-Tunnel 🎧

Say a particular PE, PE1, learns (through the procedures of Section 7.4) that some other PE, PE2, has bound a particular C-flow, designated (C-S,C-G), to a particular P-tunnel, P2. Then PE1 needs to figure out whether it needs to receive (C-S,C-G) traffic from PE2 🤔

If BGP is being used to distribute C-multicast routing information from PE to PE, the conditions under which PE1 needs to receive (C-S,C-G) traffic from PE2 are specified in Section 12.3 of [MVPN-BGP].

If PIM over an MI-PMSI is being used to distribute C-multicast routing from PE to PE, PE1 needs to receive (C-S,C-G) traffic from PE2 if one or more of the following conditions holds:

PE1 has (C-S,C-G) state such that PE2 is PE1's Upstream PE for (C-S,C-G), and PE1 has downstream neighbors ("non-null olist") for the (C-S,C-G) state.
PE1 has (C-*,C-G) state with an Upstream PE (not necessarily PE2) and with downstream neighbors ("non-null olist"), but PE1 does not have (C-S,C-G) state.
Native PIM methods are being used to prevent steady-state packet duplication, and PE1 has either (C-*,C-G) or (C-S,C-G) state such that the MI-PMSI is one of the downstream interfaces. Note that this includes the case where PE1 is itself sending (C-S,C-G) traffic on an S-PMSI. (In this case, PE1 needs to receive the (C-S,C-G) traffic from PE2 in order to allow the PIM Assert mechanism to function properly.) 🙈

Irrespective of whether BGP or PIM is being used to distribute C-multicast routing information, once PE1 works out that it needs to receive (C-S,C-G) traffic from PE2, the following procedures MUST be applied:

PE1 MUST take all necessary steps to be able to receive the (C-S,C-G) traffic on P2.
- If P2 is a PIM tunnel or an mLDP LSP, PE1 will need to use PIM or mLDP (respectively) to join P2 (unless it is already joined to P2).
- PE1 may need to modify the forwarding state for (C-S,C-G) to indicate that (C-S,C-G) traffic is to be accepted on P2. If P2 is an Aggregate Tree, this also implies setting up the demultiplexing forwarding entries based on the inner label, as described in Section 6.3.3.
If PE1 was previously receiving the (C-S,C-G) C-flow on another P-tunnel, say P1, then:
- PE1 MAY run a switch-over timer, and until it expires, SHOULD accept traffic for the given C-flow on both P1 and P2;
- If, after such a switch, the "old" P-tunnel P1 is no longer needed, it SHOULD be torn down and the resources supporting it freed. The procedures for "tearing down" a P-tunnel are specific to the P-tunnel technology.
If PE1 later determines that it no longer needs to receive any of the C-multicast data being sent on a particular P-tunnel, it may initiate signaling (specific to the P-tunnel technology) to remove itself from that tunnel.

7.2. Optimizing Multicast Distribution via S-PMSIs 🚀

Whenever a particular multicast stream is sent on an I-PMSI, the data is likely going to PEs that don't need it 😬 If a stream has a significant amount of traffic, it may be worth moving it to an S-PMSI that includes only those PEs that are transmitters and/or receivers (or at least includes fewer PEs that are neither).

If explicit tracking is being done, S-PMSI creation can also be triggered by other criteria. For instance, there could be a "pseudo-wasted bandwidth" criterion: switching to an S-PMSI would happen if the bandwidth multiplied by the number of uninterested PEs (PEs that are receiving the stream but have no receivers) is above a specified threshold. The motivation is that (a) the total bandwidth wasted by many sparsely subscribed low-bandwidth groups may be large, and (b) there's no point moving a high-bandwidth group to an S-PMSI if all the PEs have receivers for it 💭

Switching a (C-S,C-G) stream to an S-PMSI may require the root of the S-PMSI to determine the egress PEs that need to receive the (C-S,C-G) traffic. This is true in the following cases:

If the P-tunnel is a source-initiated tree, such as an RSVP-TE P2MP Tunnel, the PE needs to know the leaves of the tree before it can instantiate the S-PMSI.
If a PE instantiates multiple S-PMSIs, belonging to different MVPNs, using one P-multicast tree, such a tree is termed an Aggregate Tree with a selective mapping. Setting up such an Aggregate Tree requires the ingress PE to know all the other PEs that have receivers for multicast groups mapped onto the tree.

The above two cases require that explicit tracking be done for the (C-S,C-G) stream. The root of the S-PMSI MAY decide to do explicit tracking of this stream only after it has decided to move the stream to an S-PMSI, or it MAY have been doing explicit tracking all along 👀

If the S-PMSI is instantiated by a P-multicast tree, the PE at the root of the tree must signal the leaves of the tree that the (C-S,C-G) stream is now bound to the S-PMSI. Note that the PE could create the identity of the P-multicast tree prior to the actual instantiation of the P-tunnel.

If the S-PMSI is instantiated by a source-initiated P-multicast tree (e.g., an RSVP-TE P2MP tunnel), the PE at the root of the tree must establish the source-initiated P-multicast tree to the leaves. This tree MAY have been established before the leaves receive the S-PMSI binding, or it MAY be established after the leaves receive the binding. The leaves MUST NOT switch to the S-PMSI until they receive both the binding and the tree signaling message 🛑

7.3. Announcing the Presence of Unsolicited Flooded Data 📢

A PE may receive "unsolicited" data from a CE, where the data is intended to be flooded to the other PEs of the same MVPN and then on to other CEs. By "unsolicited", we mean that the data is to be delivered to all the other PEs of the MVPN, even though those PEs may not have sent any control information indicating that they need to receive that data 🫣

For example, if the BSR [BSR] is being used within the MVPN, BSR control messages may be received by a PE from a CE. These need to be forwarded to other PEs, even though no PE ever issues any kind of explicit signal saying that it wants to receive BSR messages.

If a PE receives a BSR message from a CE, and if the CE's MVPN has an MI-PMSI, then the PE can just send BSR messages on the appropriate P-tunnel. Otherwise, the PE MUST announce the binding of a particular C-flow to a particular P-tunnel, using the procedures of Section 7.4. The particular C-flow in this case would be (C-IPaddress_of_PE, ALL-PIM-ROUTERS). The P-tunnel identified by the procedures of Section 7.4 may or may not be one that was previously identified in the PMSI Tunnel attribute of an I-PMSI A-D route. Further procedures for handling BSR may be found in Sections 5.2.1 and 5.3.4 🧚

Analogous procedures may be used for announcing the presence of other sorts of unsolicited flooded data, e.g., dense mode data or data from proprietary protocols that presume messages can be flooded. However, a full specification of the procedures for traffic other than BSR traffic is outside the scope of this document 💅

7.4. Protocols for Binding C-Flows to P-Tunnels 🔗

We describe two protocols for binding C-flows to P-tunnels.

These protocols can be used for moving C-flows from I-PMSIs to S-PMSIs, as long as the S-PMSI is instantiated by a P-multicast tree. (If the S-PMSI is instantiated by means of ingress replication, the procedures of Section 6.4.5 suffice.)

These protocols can also be used for other cases in which it is necessary to bind specific C-flows to specific P-tunnels.

7.4.1. Using BGP S-PMSI A-D Routes 🌈

Even though it's called "S-PMSI A-D routes", the mechanism specified in this section may be used any time it is necessary to advertise a binding of a C-flow to a particular P-tunnel ✨ that's the whole vibe ✨ 🦋

7.4.1.1. Advertising C-Flow Binding to P-Tunnel 📣

Okay so the ingress PE has to let every PE on the path to receivers of the (C-S,C-G) know that it's binding a P-tunnel to that (C-S,C-G) 💖. The announcement goes out as a BGP update in the MCAST-VPN address family, and it uses an S-PMSI A-D route that carries the following:

The IP address of the originating PE. 🦋
The RD configured locally for the MVPN. This is required to uniquely identify the (C-S,C-G), because the addresses could overlap between different MVPNs 👀. It's the same RD value used in the auto-discovery process.
The C-S address.
The C-G address.
A PE MAY use a single P-tunnel to aggregate two or more S-PMSIs 🫶. If the PE already advertised unaggregated S-PMSI A-D routes for these S-PMSIs, then deciding to aggregate them means the PE has to re-advertise those routes. The re-advertised routes MUST be the same as the originals, except for the PMSI Tunnel attribute. If the PE hasn't previously advertised S-PMSI A-D routes for these S-PMSIs, then aggregating means the PE has to advertise (new) S-PMSI A-D routes for them instead. The PMSI Tunnel attribute in the newly advertised or re-advertised routes MUST carry the identity of the P-tunnel that aggregates the S-PMSIs ✨.

If all of these aggregated S-PMSIs belong to the same MVPN, and that MVPN uses PIM as its C-multicast routing protocol, then the corresponding S-PMSI A-D routes MAY carry an MPLS upstream-assigned label [MPLS-UPSTREAM-LABEL]. In this case the labels MUST be distinct on a per-MVPN basis, and they MAY be distinct on a per-route basis 🌸.

If all of these aggregated S-PMSIs belong to MVPN(s) that use mLDP as their C-multicast routing protocol, then the corresponding S-PMSI A-D routes MUST carry an MPLS upstream-assigned label [MPLS-UPSTREAM-LABEL], and these labels MUST be distinct on a per-route (per-mLDP-FEC) basis. That's true whether the aggregated S-PMSIs belong to the same MVPN or to different ones 🔥.

When a PE distributes this info via BGP, it must include the following:

An identifier for the particular P-tunnel that the stream is going to be bound to 🎀. This identifier is a structured field with these pieces:
- The type of tunnel
- An identifier for the tunnel. The form of the identifier depends on the tunnel type. The combo of tunnel identifier and tunnel type should contain enough info for all the PEs to "join" the tunnel and receive messages from it 🧚.
Route Target Extended Communities attribute. This gets used as described in Section 4 💜.

7.4.1.2. Explicit Tracking 🕵️‍♀️

If the PE wants explicit tracking for the specified flow, it indicates that in the A-D route it uses to bind the flow to a particular P-tunnel too. Then any PE that receives that A-D route responds with a "Leaf A-D route" where it identifies itself as a receiver of the specified flow. The Leaf A-D route gets withdrawn once the PE is no longer a receiver for the flow 🙈.

If the PE needs explicit tracking for a flow without binding that flow to a specific P-tunnel at the same time, it can do that by sending an S-PMSI A-D route. The NLRI in that route identifies the flow, and its PMSI Tunnel attribute has the tunnel type value set to "no tunnel information present" and the "leaf information required" bit set to 1 🍓. This gets the Leaf A-D routes going. It's super useful when the PE needs to know who the receivers are before it picks a P-tunnel 👑.

7.4.2. UDP-Based Protocol ✨

This procedure carries its control messages in UDP, and it requires that the MVPN have an MI-PMSI that can be used to carry those control messages 💅.

7.4.2.1. Advertising C-Flow Binding to P-Tunnel 📮

For a given PE to move a particular C-flow onto a particular P-tunnel, an "S-PMSI Join message" gets sent periodically on the MI-PMSI. (Name aside, this mechanism may be used to bind a flow to any P-tunnel 🌈.) The S-PMSI Join message is a UDP-encapsulated message. Its destination address is ALL-PIM-ROUTERS (224.0.0.13) and its destination port is 3232 🦋.

The S-PMSI Join message contains the following info:

An identifier for the particular multicast stream that is to be bound to the P-tunnel. This can be represented as an (S,G) pair.
An identifier for the particular P-tunnel that the stream is to be bound to. This identifier is a structured field with these pieces:
- The type of tunnel used to instantiate the S-PMSI.
- An identifier for the tunnel. The form of the identifier depends on the tunnel type. The combo of tunnel identifier and tunnel type should contain enough info to enable all the PEs to "join" the tunnel and receive messages from it.
- If (and only if) the identified P-tunnel is aggregating several S-PMSIs, any demultiplexing info needed by the tunnel encapsulation protocol to identify a particular S-PMSI 🫧.

If the policy for the MVPN is that traffic is sent or received by default over an MI-PMSI, then traffic for a particular C-flow can be switched back to the MI-PMSI simply by ceasing to send S-PMSI Joins for that C-flow 🍓.

Note that an S-PMSI Join that is not received over a PMSI (e.g., one that is received directly from a CE) is an illegal packet that MUST be discarded 🚫.

7.4.2.2. Packet Formats and Constants 🧮

The S-PMSI Join message is encapsulated within UDP and has the following type/length/value (TLV) encoding 💖:

        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |     Type      |            Length           |     Value       |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                               .                               |
       |                               .                               |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Type (8 bits)

Length (16 bits): the total number of octets in the Type, Length, and Value fields combined

Value (variable length)

Only one type of S-PMSI Join is defined in this spec 🎀. A Type 1 S-PMSI Join is used when the S-PMSI tunnel is a PIM tunnel that carries a single multicast stream, where the packets of that stream have IPv4 source and destination IP addresses.

The S-PMSI Join format to use when the C-source and C-group are IPv6 addresses will be defined in a follow-on document 👀.

        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |     Type      |           Length            |    Reserved     |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                           C-source                            |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                           C-group                             |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                           P-group                             |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Type (8 bits): 1

Length (16 bits): 16

Reserved (8 bits): This field SHOULD be zero when transmitted, and MUST be ignored when received 🧊.

C-source (32 bits): the IPv4 address of the traffic source in the VPN.

C-group (32 bits): the IPv4 address of the multicast traffic destination address in the VPN.

P-group (32 bits): the IPv4 group address that the PE router is going to use to encapsulate the flow (C-source, C-group).

The P-group identifies the S-PMSI P-tunnel, and the (C-S,C-G) identifies the multicast flow carried in the P-tunnel 🦄.

The protocol uses the following constants 🧁.

[S-PMSI_DELAY]:

Once an S-PMSI Join message has been sent, the PE router that is to transmit onto the S-PMSI waits this amount of time before it starts using the S-PMSI. The default value is 3 seconds ⏳.

[S-PMSI_TIMEOUT]:

If a PE (other than the transmitter) doesn't receive any packets over the S-PMSI P-tunnel for this amount of time, that PE prunes itself from the S-PMSI P-tunnel and expects (C-S,C-G) packets to arrive on an I-PMSI 🙅‍♀️. The default value is 3 minutes. This value must be consistent among PE routers.

[S-PMSI_HOLDOWN]:

If the PE that transmits onto the S-PMSI doesn't see any (C-S,C-G) packets for this amount of time, it resumes sending (C-S,C-G) packets on an I-PMSI 🔄. This is used to avoid oscillation when traffic is bursty. The default value is 1 minute.

[S-PMSI_INTERVAL]:

The interval the transmitting PE router uses to periodically send the S-PMSI Join message 🔁. The default value is 60 seconds.

7.4.3. Aggregation 🫧

S-PMSIs can be aggregated on a P-multicast tree 🌳. The S-PMSI to (C-S,C-G) binding advertisement supports aggregation. On top of that, the aggregation procedures of Section 6.3 apply. It's also possible to aggregate both S-PMSIs and I-PMSIs on the same P-multicast tree 💜.

8. Inter-AS Procedures ✨

okay bestie, so if an MVPN has sites in more than one AS, it needs one or more PMSIs instantiated by inter-AS P-tunnels 🌸 this doc describes two types, and they're giving very different energy:

"Segmented inter-AS P-tunnels"

A segmented inter-AS P-tunnel is made of a bunch of independent segments that get stitched together at the ASBRs 🧩 there are two kinds of segment: inter-AS segments and intra-AS segments. So the segmented inter-AS P-tunnel alternates between intra-AS and inter-AS segments, which is very pattern-y of it.

Inter-AS segments connect adjacent ASBRs of different ASes. These "one-hop" segments are instantiated as unicast P-tunnels 💜

Intra-AS segments connect ASBRs and PEs that are in the same AS. An intra-AS segment can use whatever technology the SP that administers that AS wants 🫶 different intra-AS segments can even use different technologies, and that's totally allowed.

Note that the intra-AS segments of inter-AS P-tunnels form a category of P-tunnels that is distinct from simple intra-AS P-tunnels. we'll rely on this distinction later (see Section 9) 👀

A segmented inter-AS P-tunnel can be thought of as a tree rooted at a particular AS, with the other ASes that need to receive multicast data from the root AS as its leaves 🌳

"Non-segmented Inter-AS P-tunnels"

A non-segmented inter-AS P-tunnel is a single P-tunnel that spans AS boundaries. The tunnel technology can't change from one point in the tunnel to the next, so every AS the P-tunnel passes through has to support that technology 🚫 basically, AS boundaries don't matter at all to a non-segmented inter-AS P-tunnel. no cap, it's very chill about borders 🦋

Section 10 of [RFC4364] describes three different options for supporting unicast inter-AS BGP/MPLS IP VPNs, known as options A, B, and C. We describe below how both segmented and non-segmented inter-AS trees can be supported when options B or C are used. (Option A does not pass any routing information through an ASBR at all, so no special inter-AS procedures are needed.) 💅

8.1. Non-Segmented Inter-AS P-Tunnels 🎀

In this model, the previously described discovery and tunnel setup mechanisms are used, even though the PEs belonging to a given MVPN may be in different ASes. the PEs are in different ASes and it still just works 🧚

8.1.1. Inter-AS MVPN Auto-Discovery

The previously described BGP-based auto-discovery mechanisms work "as is" when an MVPN contains PEs that are in different Autonomous Systems. HOWEVER, please note that, if non-segmented inter-AS P-tunnels are to be used, then the Intra-AS I-PMSI A-D routes MUST be distributed across AS boundaries! 🚨

8.1.2. Inter-AS MVPN Routing Information Exchange

When non-segmented inter-AS P-tunnels are used, MVPN C-multicast routing information may be exchanged by means of PIM peering across an MI-PMSI or by means of BGP carrying C-multicast routes. two options, pick your fighter 🥊

When PIM peering is used to distribute the C-multicast routing information, a PE that sends C-PIM Join/Prune messages for a particular (C-S,C-G) must be able to identify the PE that is its PIM adjacency on the path to S. This is the "Selected Upstream PE" described in Section 5.1.3 🫧

If BGP (rather than PIM) is used to distribute the C-multicast routing information, and if option b of Section 10 of [RFC4364] is in use, then the C-multicast routes will be installed in the ASBRs along the path from each multicast source in the MVPN to each multicast receiver in the MVPN. If option b is not in use, the C-multicast routes are not installed in the ASBRs. The handling of the C-multicast routes in either case is thus exactly analogous to the handling of unicast VPN-IP routes in the corresponding case. it's giving consistency, which is iconic 💖

8.1.3. Inter-AS P-Tunnels

The procedures described earlier in this document can be used to instantiate either an I-PMSI or an S-PMSI with inter-AS P-tunnels. Specific tunneling techniques need some explanation, so let's go 🌈

If ingress replication is used, the inter-AS PE-PE P-tunnels will use the inter-AS tunneling procedures for the tunneling technology used.

Procedures in [RSVP-P2MP] are used for inter-AS RSVP-TE P2MP P-tunnels.

Procedures for using PIM to set up the P-tunnels are discussed in the next section. ⏭️

8.1.3.1. PIM-Based Inter-AS P-Multicast Trees

When PIM is used to set up a non-segmented inter-AS P-multicast tree, the PIM Join/Prune messages used to join the tree contain the IP address of the Upstream PE. However, there are two special considerations that must be taken into account 👀

It is possible that the P routers within one or more of the ASes will not have routes to the Upstream PE. For example, if an AS has a "BGP-free core", the P routers in an AS will not have routes to addresses outside the AS.
If the PIM Join/Prune message must travel through several ASes, it is possible that the ASBRs will not have routes to the PE routers. For example, in an inter-AS VPN constructed according to "option b" of Section 10 of [RFC4364], the ASBRs do not necessarily have routes to the PE routers.

In either case, "ordinary" PIM Join/Prune messages cannot be routed to the Upstream PE. Therefore, in that case, the PIM Join/Prune messages MUST contain the "PIM MVPN Join attribute". This allows the multicast distribution tree to be properly constructed, even if routes to PEs in other ASes do not exist in the given AS's IGP and even if the routes to those PEs do not exist in BGP. The use of a PIM MVPN Join attribute in the PIM messages allows the inter-AS trees to be built 🌲✨

The PIM MVPN Join attribute adds the following information to the PIM Join/Prune messages: a "proxy address", which contains the address of the next ASBR on the path to the Upstream PE. When the PIM Join/Prune arrives at the ASBR that is identified by the "proxy address", that ASBR must change the proxy address to identify the next hop ASBR.

This information allows the PIM Join/Prune to be routed through an AS, even if the P routers of that AS do not have routes to the Upstream PE. However, this information is not sufficient to enable the ASBRs to route the Join/Prune if the ASBRs themselves do not have routes to the Upstream PE.

However, even if the ASBRs do not have routes to the Upstream PE, the procedures of this document ensure that they will have Intra-AS I-PMSI A-D routes that lead to the Upstream PE. (Recall that if non-segmented inter-AS P-tunnels are being used, the ASBRs and PEs will have Intra-AS I-PMSI A-D routes that have been distributed inter-AS.) so the ASBRs are not left out in the cold 🥹

So, rather than having the PIM Join/Prune messages routed by the ASBRs along a route to the Upstream PE, the PIM Join/Prune messages MUST be routed along the path determined by the Intra-AS I-PMSI A-D routes. 🛣️

The basic format of a PIM Join attribute is specified in [PIM-ATTRIB]. The details of the PIM MVPN Join attribute are specified in the next section.

8.1.3.2. The PIM MVPN Join Attribute

8.1.3.2.1. Definition

In [PIM-ATTRIB], the notion of a "join attribute" is defined, and a format for included join attributes in PIM Join/Prune messages is specified. We now define a new join attribute, which we call the "MVPN Join attribute". 🎉

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |F|E| Attr_Type | Length        |     Proxy IP address
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                                    |      RD
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-.......

The Attr_Type field of the MVPN Join attribute is set to 1.

The F bit is set to 0.

Two information fields are carried in the MVPN Join attribute:

Proxy IP address: The IP address of the node towards which the PIM Join/Prune message is to be forwarded. This will be either an IPv4 or an IPv6 address, depending on whether the PIM Join/Prune message itself is IPv4 or IPv6.
RD: An eight-byte RD. This immediately follows the proxy IP address.

The PIM message also carries the address of the Upstream PE.

In the case of an intra-AS MVPN, the proxy and the Upstream PE are the same. In the case of an inter-AS MVPN, the proxy will be the ASBR that is the exit point from the local AS on the path to the Upstream PE. 🚪

8.1.3.2.2. Usage

When a PE router originates a PIM Join/Prune message in order to set up an inter-AS PMSI, it does so as a result of having received a particular Intra-AS I-PMSI A-D route or S-PMSI A-D route. It includes an MVPN Join attribute whose fields are set as follows:

If the Upstream PE is in the same AS as the local PE, then the proxy field contains the address of the Upstream PE. Otherwise, it contains the address of the BGP Next Hop of the route to the Upstream PE.
The RD field contains the RD from the NLRI of the Intra-AS A-D route.
The Upstream PE field contains the address of the PE that originated the Intra-AS I-PMSI A-D route or S-PMSI A-D route (obtained from the NLRI of that route).

When a PIM router processes a PIM Join/Prune message with an MVPN Join attribute, it first checks to see if the proxy field contains one of its own addresses.

If not, the router uses the proxy IP address in order to determine the RPF interface and neighbor. The MVPN Join attribute must be passed upstream unchanged.

If the proxy address is one of the router's own IP addresses, then the router looks in its BGP routing table for an Intra-AS A-D route whose NLRI consists of the Upstream PE address prepended with the RD from the Join attribute. If there is no match, the PIM message is discarded. If there is a match, the IP address from the BGP next hop field of the matching route is used in order to determine the RPF interface and neighbor. When the PIM Join/Prune is forwarded upstream, the proxy field is replaced with the address of the BGP next hop, and the RD and Upstream PE fields are left unchanged. this part is a lot, but you've got this 💪

The use of non-segmented inter-AS trees constructed via BIDIR-PIM is outside the scope of this document. 🙅‍♀️

8.2. Segmented Inter-AS P-Tunnels 🧶

The procedures for setting up and maintaining segmented inter-AS Inclusive and Selective P-tunnels may be found in [MVPN-BGP]. 📖

9. Preventing Duplication of Multicast Data Packets ✨

okay bestie, picture an egress PE getting packets for one C-flow, (C-S,C-G), over a non-aggregated S-PMSI. The stuff we've already covered will never make that PE get duplicate copies of anything in that stream 🙅‍♀️. The (C-S,C-G) stream can show up in more than one S-PMSI, though, and that happens when the site with C-S is multihomed to more than one PE 🌸. But a PE that needs (C-S,C-G) only joins one of those S-PMSIs, so it only gets one copy of each packet 💅.

Here's the plot twist 😭: if the data packets of (C-S,C-G) ride an I-PMSI or an aggregated S-PMSI, the procedures so far let an egress PE get more than one copy of each data packet. So we need extra procedures to either make that impossible or make sure the egress PE doesn't pass duplicates on to the CE routers 🫶.

This section only covers the situation where the C-trees are unidirectional, in either the ASM or SSM service models. The bidirectional C-tree case gets its own deep dive in Section 11 🦋.

There are two cases where the procedures so far make it possible for an egress PE to receive duplicate copies of a multicast data packet 👀:

The first case happens when both of these conditions hold:
- a. an MVPN site that contains C-S or C-RP is multihomed to more than one PE, and
- b. either an I-PMSI or an aggregated S-PMSI is used for carrying the packets originated by C-S.

In this case, an egress PE may receive one copy of the packet from each PE to which the site is homed. We'll dig into this more in Section 9.2 🔍.

The second case happens when all of these conditions hold:
- a. the IP destination address of the customer packet, C-G, identifies a multicast group that is operating in ASM mode and whose C-multicast tree is set up using PIM-SM,
- b. an MI-PMSI is used for carrying the data packets, and
- c. a router or a CE in a site connected to the egress PE switches from the C-RP tree to the C-S tree.

In this case, it's possible to get one copy of a given packet from the ingress PE attached to the C-RP's site and one from the ingress PE attached to the C-S's site. This one gets its own breakdown in Section 9.3 🧚.

So yeah, extra procedures are needed to make sure no MVPN customer sees steady state multicast data packet duplication 💖. There are three procedures that may be used:

Discarding data packets received from the "wrong" PE
Single Forwarder Selection
Native PIM methods

These methods are described in Section 9.1 📚. Their applicability to the two duplication scenarios gets discussed in Sections 9.2 and 9.3, so stay tuned 🎀.

9.1. Methods for Ensuring Non-Duplication

Every MVPN MUST use at least one of the three methods for ensuring non-duplication. Non-negotiable, bestie 👑.

9.1.1. Discarding Packets from Wrong PE

Per Section 5.1.3, an egress PE (let's call her PE1) picks a specific Upstream PE for a given (C-S,C-G). When PE1 gets a (C-S,C-G) packet from a PMSI, she may be able to figure out which PE transmitted the packet onto that PMSI 🕵️‍♀️. If that transmitter is anyone other than the Upstream PE PE1 selected, then PE1 can drop the packet. So the PE will still see a duplicate, but the duplicate won't get forwarded. Main character energy, literally 💅.

How an egress PE figures out the ingress PE for a packet depends on the P-tunnel tech that instantiates the PMSI 🧠. If the P-tunnel is a P2MP LSP, a PIM-SM or PIM-SSM tree, or a unicast P-tunnel that uses IP encapsulation, then the tunnel encapsulation has info that can be used (maybe along with other state in the PE) to determine the ingress PE. This works as long as the P-tunnel is instantiating an intra-AS PMSI, or an inter-AS PMSI that's supported by a non-segmented inter-AS tunnel 🦋.

Even when inter-AS segmented P-tunnels are used, if an aggregated S-PMSI is carrying the packets, the tunnel encapsulation must have some info that can be used to identify the PMSI. That in turn implicitly identifies the ingress PE 🫶.

Now consider an I-PMSI that spans multiple ASes and is instantiated by segmented inter-AS P-tunnels 🌈. Say it's carrying data traveling along a particular C-tree. Also say the C-root of that C-tree is multihomed to two or more PEs, and each of those PEs is in a different AS. Then, if there's any duplicate traffic, the duplicates arrive on a different P-tunnel. Specifically, if the PE was expecting the traffic on a particular inter-AS P-tunnel, the duplicate traffic arrives either on an intra-AS P-tunnel (not an intra-AS segment of an inter-AS P-tunnel) or on some other inter-AS P-tunnel 😳. To catch duplicates, the PE has to keep track of which inter-AS A-D route it uses for sending MVPN multicast routing info towards the C-S/C-RP. The PE MUST process received (multicast) traffic originated by C-S/C-RP only from the inter-AS P-tunnel that was carried in the best Inter-AS A-D route for the MVPN, and that was originated by the AS containing C-S/C-RP (where "the best" is determined by the PE). The PE MUST discard, as duplicates, all other multicast traffic originated by the C-S/C-RP but received on any other P-tunnel 🚫.

If, for a given MVPN, (a) an MI-PMSI carries multicast data packets, (b) the MI-PMSI is instantiated by a segmented inter-AS P-tunnel, (c) the C-S or C-RP is multihomed to different PEs, and (d) at least two of those PEs are in the same AS, then it may not always be possible for the egress PE to determine the Upstream PE. It depends on the tunneling tech used to instantiate the MI-PMSI 🥺. In that case, the procedure of Sections 9.1.2 or 9.1.3 must be used.

NB: Section 10 describes an exception case where PE1 has to accept a packet even if it isn't from the Selected Upstream PE. Plot twist, but it's documented ✨.

9.1.2. Single Forwarder Selection

Section 5.1 specifies a procedure for choosing a "default Upstream PE selection," so that (except during routing transients) all PEs choose the same default Upstream PE 🤝. To make sure duplicate packets don't get sent through the backbone (except during routing transients), an ingress PE does not forward to the backbone any (C-S,C-G) multicast data packet it receives from a CE, unless that PE is the default Upstream PE selection 🙅‍♀️.

One difference between this procedure and the one in Section 9.1.1: this one sends only one copy of each packet to each egress PE, instead of sending multiple copies and forcing the egress PE to discard all but one. Efficient and iconic 💜.

9.1.3. Native PIM Methods

If PE-PE multicast routing info for a given MVPN is being disseminated by running PIM over an MI-PMSI, then native PIM methods prevent steady state data packet duplication 🌸. The PIM Assert mechanism prevents steady state duplication in the Section 9.2 scenario, even if Single Forwarder Selection isn't done. The PIM Prune(S,G,rpt) mechanism handles the Section 9.3 scenario 🦋.

9.2. Multihomed C-S or C-RP

Any of the three methods in Section 9.1 will prevent steady state duplicates when the C-S or C-RP is multihomed 💅. Easy win.

9.3. Switching from the C-RP Tree to the C-S Tree

9.3.1. How Duplicates Can Occur

If some PEs are on the C-S tree and some are on the C-RP tree, then a PE may also receive duplicate data traffic after a (C-*,C-G) to (C-S,C-G) switch 😭.

If PIM is used on an MI-PMSI to disseminate multicast routing info, native PIM methods (in particular, the use of the Prune(S,G,rpt) message) prevent steady state data duplication in this case 💖.

If BGP C-multicast routing is being used, then the procedure of Section 9.1.1 can prevent duplication, if it applies. But if that procedure doesn't apply, the procedure of Section 9.1.2 is not enough to prevent steady state data duplication in all scenarios 🙈.

Now the spicy one 🌶️. In the scenario where (a) BGP C-multicast routing is being used, (b) there are inter-site shared C-trees, and (c) there are inter-site source C-trees, extra procedures are needed. Picture this topology:

                    CE1---C-RP
                     |
                     |
              CE2---PE1-- ... --PE2---CE5---C-S
                          ...
       C-R1---CE3---PE3-- ... --PE4---CE4---C-R2

Say C-R1 and C-R2 use PIM to join the (C-*,C-G) tree, where C-RP is the RP for C-G. So CE3 and CE4 send PIM Join(*,G) messages to PE3 and PE4, respectively 🧚. That makes PE3 and PE4 originate C-multicast Shared Tree Join Routes specifying (C-*,C-G). These routes identify PE1 as the Upstream PE.

Now say C-S is a transmitter for multicast group C-G and sends its multicast data packets to C-RP in PIM Register messages. Then PE1 receives (C-S,C-G) data packets from CE1 and forwards them over an I-PMSI to PE3 and PE4, who forward them on to CE3 and CE4 🌈.

When C-R1 receives (C-S,C-G) data packets, it may decide to join the (C-S,C-G) source tree by sending a PIM Join(S,G) to CE3 🎯. That makes CE3 send a PIM Join(S,G) to PE3, which makes PE3 originate a C-multicast Source Tree Join Route specifying (C-S,C-G) and identifying PE2 as the Upstream PE. So when PE2 receives (C-S,C-G) data packets from CE5, it forwards them on a PMSI to PE3 🫶.

At this point, the following situation exists:

If PE1 receives (C-S,C-G) packets from CE1, PE1 must forward them on the I-PMSI, because PE4 is still expecting to receive the (C-S,C-G) packets from PE1.
PE3 must continue to receive packets from the I-PMSI, since there may be other sources transmitting C-G traffic and PE3 currently has no other way to receive that traffic.
PE3 must also receive (C-S,C-G) traffic from PE2.

So PE3 may receive two copies of each (C-S,C-G) packet 😱. The Section 9.1.2 procedure (Single Forwarder Selection) doesn't stop PE3 from getting two copies, because it doesn't stop one PE from forwarding (C-S,C-G) traffic along the shared C-tree while another forwards (C-S,C-G) traffic along a source-specific C-tree.

So if PE3 can't apply the method of Section 9.1.1 (Discarding Packets from Wrong PE), maybe because the tunneling tech doesn't let the egress PE identify the ingress PE, then additional procedures are needed 🥹✨.

9.3.2. Solution Using Source Active A-D Routes 🦋

okay so the problem from Section 9.3.1 gets fixed with Source Active A-D routes 💅 I'll walk through an example and then give you the informal vibe of the procedures.

A full and precise spec for all of this lives in Section 13 of [MVPN-BGP] 📜 If anything I say here conflicts with or doesn't match the description in [MVPN-BGP], [MVPN-BGP] is the authoritative doc, no debate 🫶

Heads up: this section only applies when inter-site shared trees are being used 🌸

Whenever a PE creates a (C-S,C-G) state because it got a C-multicast route for (C-S,C-G) from some other PE, and the C-G group is an ASM group, the PE that creates the state MUST originate a Source Active A-D route (see [MVPN-BGP], Section 4.5). ✨ The NLRI of the route includes C-S and C-G. By default, the route carries the same set of Route Targets as the Intra-AS I-PMSI A-D route of the MVPN originated by the PE. Using the normal BGP procedures, the route gets propagated to all the PEs of the MVPN. For more details, see Section 13.1 ("Source within a Site - Source Active Advertisement") of [MVPN-BGP] 💖

When, as a result of receiving a new Source Active A-D route, a PE updates its VRF with the route, the PE MUST check if the newly received route matches any (C-*,C-G) entries. If (a) there is a matching entry, (b) the PE does not have (C-S,C-G) state in its MVPN Tree Information Base (MVPN-TIB) for (C-S,C-G) carried in the route, and (c) the received route is selected as the best (using the BGP route selection procedures), then the PE takes the following action:

If the PE's (C-*,C-G) state has a PMSI as a downstream interface, the PE acts as if all the other PEs had pruned C-S off the (C-*,C-G) tree. That is:
- If the PE receives (C-S,C-G) traffic from a CE, it does not transmit it to other PEs.
- Depending on the PIM state of the PE's PE-CE interfaces, the PE may or may not need to invoke PIM procedures to prune C-S off the (C-*,C-G) tree by sending a PIM Prune(S,G,rpt) to one or more of the CEs. This is determined by ordinary PIM procedures. If this does need to be done, the PE SHOULD delay sending the Prune until it first runs a timer; this helps ensure that the source is not pruned from the shared tree until all PEs have had time to receive the Source Active A-D route. ⏰
If the PE's (C-*,C-G) state does not have a PMSI as a downstream interface, the PE sets up its forwarding path to receive (C-S,C-G) traffic from the originator of the selected Source Active A-D route. 🧚

Whenever a PE deletes the (C-S,C-G) state that was previously created because it got a C-multicast route for (C-S,C-G) from some other PE, the PE that deletes the state also withdraws the Source Active A-D route (if there is one) that was advertised when the state was created. 🧹

In the example topology of Section 9.3.1, this procedure causes PE2 to generate a Source Active A-D route for (C-S,C-G). When that route is received, PE4 sets up its forwarding state to expect (C-S,C-G) packets from PE2. PE1 changes its forwarding state so that (C-S,C-G) packets it receives from CE1 are not forwarded to any other PEs. (Note that PE1 may still forward (C-S,C-G) packets received from CE1 to CE2, if CE2 has receivers for C-G and those receivers did not switch from the (C-*,C-G) tree to the (C-S,C-G) tree.) As a result, PE3 and PE4 do not receive duplicate packets of the (C-S,C-G) C-flow. 🎉

With this procedure in place, there is no need to have any kind of C-multicast route that has the semantics of a PIM Prune(S,G,rpt) message. 🙌

It is worth noting that if, as a result of this procedure, a PE sets up its forwarding state to receive (C-S,C-G) traffic from the source tree, the UMH is not necessarily the same as it would be if the PE had joined the source tree as a result of receiving a PIM Join for the same source tree from a directly attached CE. 👀

Note that the mechanism described in Section 7.4.1 can be leveraged to advertise an S-PMSI binding along with the source active messages. This is accomplished by using the same BGP Update message to carry both the NLRI of the S-PMSI A-D route and the NLRI of the Source Active A-D route. (Though an implementation processing the received routes cannot assume that this will always be the case.) 🫧

10. Eliminating PE-PE Distribution of `(C-*,C-G)` State ✨

In the ASM service model, a node that wants to become a receiver for a particular multicast group G first joins a shared tree, rooted at a rendezvous point. 🌳 When the receiver detects traffic from a particular source, it has the option of joining a source tree, rooted at that source. If it does so, it has to prune that source from the shared tree, so it receives packets from that source on only one tree.

Maintaining the shared tree can require considerable state 😵‍💫 It's necessary not only to know who the upstream and downstream nodes are, but also to know which sources have been pruned off which branches of the shared tree.

The BGP-based signaling procedures defined in this document and in [MVPN-BGP] eliminate the need for PEs to distribute to each other any state having to do with which sources have been pruned off a shared C-tree. ✂️ Those procedures do still allow multicast data traffic to travel on a shared C-tree, but they do not allow a situation in which some CEs receive (S,G) traffic on a shared tree and some on a source tree. This results in a considerable simplification of the PE-PE procedures with minimal change to the multicast service seen within the VPN. However, shared C-trees are still supported across the VPN backbone. That is, (C-*,C-G) state is distributed PE-PE, but (C-*,C-G,rpt) state is not. 💜

In this section, we specify a number of optional procedures that go further and completely eliminate the support for shared C-trees across the VPN backbone. 🧨 In these procedures, the PEs keep track of the active sources for each C-G. As soon as a CE tries to join the (*,G) tree, the PEs instead join the (S,G) trees for all the active sources. Thus, all distribution of (C-*,C-G) state is eliminated. These procedures are optional because they require some additional support on the part of the VPN customer and because they are not always appropriate. (For example, a VPN customer may have their own policy of always using shared trees for certain multicast groups.) There are several different options, described in the following sub-sections. 🍓

10.1. Co-Locating C-RPs on a PE 🌈

[MVPN-REQ] describes C-RP engineering as an issue when PIM-SM (or BIDIR-PIM) is used in Any-Source Multicast (ASM) mode [RFC4607] on the VPN customer site. To quote from [MVPN-REQ]:

In the case of PIM-SM, when a source starts to emit traffic toward a group (in ASM mode), if sources and receivers are located in VPN sites that are different than that of the RP, then traffic may transiently flow twice through the SP network and the CE-PE link of the RP (from source to RP, and then from RP to receivers). This traffic peak, even short, may not be convenient depending on the traffic and link bandwidth. Thus, a VPN solution MAY provide features that solve or help mitigate this potential issue.

One of the C-RP deployment models is for the customer to outsource the RP to the provider. In this case, the provider may co-locate the RP on the PE that is connected to the customer site [MVPN-REQ]. This section describes how "anycast-RP" can be used to achieve this. This is described below. 🦋

10.1.1. Initial Configuration 🎀

For a particular MVPN, at least one or more PEs that have sites in that MVPN act as an RP for the sites of that MVPN connected to these PEs. Within each MVPN, all of these RPs use the same (anycast) address. All of these RPs use the Anycast RP technique. 📡

10.1.2. Anycast RP Based on Propagating Active Sources 💅

This mechanism is based on propagating active sources between RPs. 🔁

10.1.2.1. Receiver(s) within a Site 👂

The PE that receives a C-Join message for (*,G) does not send the information that it has receiver(s) for G until it receives information about active sources for G from an Upstream PE. 🙈

On receiving this (described in the next section), the downstream PE responds with a Join message for (C-S,C-G). Sending this information could be done using any of the procedures described in Section 5. Only the Upstream PE processes this information. 🫶

10.1.2.2. Source within a Site 📣

When a PE receives a PIM Register message from a site that belongs to a given VPN, the PE follows the normal PIM anycast RP procedures. It then advertises the source and group of the multicast data packet carried in the PIM Register message to other PEs in BGP using the following information elements:

Active source address
Active group address
Route target of the MVPN

This advertisement goes to all the PEs that belong to that MVPN. When a PE receives this advertisement, it checks whether there are any receivers in the sites attached to the PE for the group carried in the source active advertisement. If there are, then it generates an advertisement for (C-S,C-G) as specified in the previous section. 🎯

10.1.2.3. Receiver Switching from Shared to Source Tree 🌸

No additional procedures are required when multicast receivers in customer's site shift from shared tree to source tree. ✨

10.2. Using MSDP between a PE and a Local C-RP ✨

Okay so, remember 10.1, where each PE is a C-RP? 🎀 That setup lets the PEs know the active multicast sources for each MVPN, and then they use BGP to share that info with each other. The PEs don't have to join any shared C-trees, which makes PE operation so much simpler. Iconic. 💅

Now here's another deployment scenario. The PEs are not C-RPs themselves, but they use Multicast Source Discovery Protocol (MSDP) [RFC3618] to talk to the C-RPs 🦋. Specifically, a PE that attaches to a site containing a C-RP becomes an MSDP peer of that C-RP. That PE then uses BGP to distribute the information about the active sources to the other PEs. When the PE learns through MSDP that a particular source is no longer active, it withdraws the corresponding BGP Update. Again, the PEs don't have to join any shared C-trees, and they don't have to be C-RPs either. Main character energy, no cap 👑

MSDP lets a Source Active (SA) message carry an encapsulated data packet 🧚. This is the trick that lets an MSDP speaker receive the first (or first several) packet(s) of an (S,G) flow, even though it hasn't joined the (S,G) tree yet. (Presumably it will join that tree once it gets the SA message carrying the encapsulated data packet.) If this capability isn't used, the first several data packets of an (S,G) stream may get lost 😭.

A PE that's talking MSDP to an RP might receive one of these encapsulated data packets from the RP. The data packet should be decapsulated and transmitted to the other PEs in the MVPN. If the packet belongs to a particular (S,G) flow, and the PE is a transmitter for some S-PMSI to which (S,G) has already been bound, the decapsulated data packet should be transmitted on that S-PMSI. Otherwise, if an I-PMSI exists for that MVPN, the decapsulated data packet should be transmitted on it. (If an MI-PMSI exists, this would typically be the one used.) If neither of these conditions holds, the decapsulated data packet is not transmitted to the other PEs in the MVPN. And the decision about whether and how to transmit the decapsulated data packet doesn't affect how the SA control message itself gets processed 🌸.

Here's the tricky part. Suppose PE1 transmits a multicast data packet on a PMSI, and that packet is part of an (S,G) flow, and PE2 receives it from that PMSI. According to Section 9, if PE1 isn't the PE that PE2 expects to be transmitting (S,G) packets, then PE2 must discard the packet. But if an MSDP-encapsulated data packet gets transmitted on a PMSI as described above, that Section 9 rule would likely cause the packet to be discarded 🙈. So, when MSDP-encapsulated data packets are decapsulated and transmitted on a PMSI, we need to modify the rules of Section 9 like this:

If the receiving PE, PE2, has already joined the (S,G) tree, and has chosen PE1 as the Upstream PE for the (S,G) tree, but this packet does not come from PE1, PE2 must discard the packet.
If the receiving PE, PE2, has not already joined the (S,G) tree, but is a PIM adjacency to a CE that is downstream on the (*,G) tree, the packet should be forwarded to the CE.

11. Support for PIM-BIDIR C-Groups 🌈

In BIDIR-PIM, each multicast group is associated with a Rendezvous Point Address (RPA). The Rendezvous Point Link (RPL) is the link that attaches to the RPA. Usually it's a LAN where the RPA is in the IP subnet assigned to the LAN. The root node of a BIDIR-PIM tree is a node that has an interface on the RPL. 💜

On any LAN (other than the RPL) that is a link in a BIDIR-PIM tree, there must be a single node chosen to be the DF. (More precisely, for each RPA there is a single node that is the DF for that RPA.) A node that receives traffic from an upstream interface may forward it on a particular downstream interface only if the node is the DF for that downstream interface. A node that receives traffic from a downstream interface may forward it on an upstream interface only if that node is the DF for the downstream interface.

If, for any period of time, there is a link on which each of two different nodes believes itself to be the DF, data forwarding loops can form. Loops in a bidirectional multicast tree can be very harmful 💀. However, any election procedure will have a convergence period. The BIDIR-PIM DF election procedure is very complicated, because it goes to great pains to ensure that if convergence is not extremely fast, then there is no forwarding at all until convergence has taken place.

Other variants of PIM also have a DF election procedure for LANs. However, as long as the multicast tree is unidirectional, disagreement about who the DF is can result only in duplication of packets, not in loops. Therefore, the time taken to converge on a single DF is of much less concern for unidirectional trees than it is for bidirectional trees.

In the MVPN environment, if PIM signaling is used among the PEs, then the standard LAN-based DF election procedure can be used. However, election procedures optimized for a LAN may not work as well in the MVPN environment. So, an alternative to DF election would be desirable 🍓.

If BGP signaling is used among the PEs, an alternative to DF election is necessary. You might think the "Single Forwarder Selection" procedures described in Sections 5 and 9 could be used to choose a single PE "DF" for the backbone (for a given RPA in a given MVPN). However, that would still likely leave a convergence period of at least several seconds during which loops could form, and there could be a much longer convergence period if anything disrupts the smooth flow of BGP Updates. So a simple procedure like that is not sufficient 🙅‍♀️.

The rest of this section describes two different methods that can be used to support BIDIR-PIM while eliminating the DF election ✨.

11.1. The VPN Backbone Becomes the RPL

On a per-MVPN basis, this method treats the whole service provider(s) infrastructure as a single RPL. We refer to such an RPL as an "MVPN-RPL". This eliminates the need for the PEs to engage in any "DF election" procedure, because BIDIR-PIM does not have a DF on the RPL 🦋.

However, this method can only be used if the customer is "outsourcing" the RPL/RPA functionality to the SP.

An MVPN-RPL could be realized via an I-PMSI (this I-PMSI is on a per-MVPN basis and spans all the PEs that have sites of a given MVPN), via a collection of S-PMSIs, or even via a combination of an I-PMSI and one or more S-PMSIs. Mix and match, bestie 🎀.

11.1.1. Control Plane

Associated with each MVPN-RPL is an address prefix that is unambiguous within the context of the MVPN associated with the MVPN-RPL.

For a given MVPN, each VRF connected to an MVPN-RPL of that MVPN is configured to advertise to all of its connected CEs the address prefix of the MVPN-RPL.

Since BIDIR-PIM has no Designated Forwarder on an RPL, in the context of MVPN-RPL there is no need to perform the Designated Forwarder election among the PEs. (Note it is still necessary to perform the Designated Forwarder election between a PE and its directly attached CEs, but that is done using plain BIDIR-PIM procedures.) 🧚

For a given MVPN, a PE connected to an MVPN-RPL of that MVPN should send multicast data (C-S,C-G) on the MVPN-RPL only if at least one other PE connected to the MVPN-RPL has a downstream multicast state for C-G. In the context of MVPN, this is accomplished by requiring a PE that has a downstream state for a particular C-G of a particular VRF present on the PE to originate a C-multicast route for (C-*,C-G). The RD of this route should be the same as the RD associated with the VRF. The RTs carried by the route should be such as to ensure that the route gets distributed to all the PEs of the MVPN 💖.

11.1.2. Data Plane

A PE that receives (C-S,C-G) multicast data from a CE should forward this data on the MVPN-RPL of the MVPN the CE belongs to only if the PE receives at least one C-multicast route for (C-*, C-G). Otherwise, the PE should not forward the data on the RPL/I-PMSI 🙈.

When a PE receives a multicast packet with (C-S,C-G) on an MVPN-RPL associated with a given MVPN, the PE forwards this packet to every directly connected CE of that MVPN, provided that the CE sends Join (C-,C-G) to the PE (provided that the PE has the downstream (C-,C-G) state). The PE does not forward this packet back on the MVPN-RPL. If a PE has no downstream (C-*,C-G) state, the PE does not forward the packet 🌸.

11.2. Partitioned Sets of PEs

This method does not require the use of the MVPN-RPL, and it does not require the customer to outsource the RPA/RPL functionality to the SP 🍓.

11.2.1. Partitions

Consider a particular C-RPA, call it C-R, in a particular MVPN. Consider the set of PEs that attach to sites that have senders or receivers for a BIDIR-PIM group C-G, where C-R is the RPA for C-G. (As always, we use the "C-" prefix to indicate that we are referring to an address in the VPN's address space rather than in the provider's address space 🌈.)

Following the procedures of Section 5.1, each PE in the set independently chooses some other PE in the set to be its "Upstream PE" for those BIDIR-PIM groups with RPA C-R. Optionally, they can all choose the "default selection" (described in Section 5.1) to ensure that each PE chooses the same Upstream PE. Note that if a PE has a route to C-R via a VRF interface, then the PE may choose itself as the Upstream PE 🦋.

The set of PEs can now be partitioned into a number of subsets. We'll say that PE1 and PE2 are in the same partition if and only if there is some PE3 such that PE1 and PE2 have each chosen PE3 as the Upstream PE for C-R. Note that each partition has exactly one Upstream PE. So it is possible to identify the partition by identifying its Upstream PE 🧠.

Consider packet P, and let PE1 be its ingress PE. PE1 will send the packet on a PMSI so that it reaches the other PEs that need to receive it. This is done by encapsulating the packet and sending it on a P-tunnel. If the original packet is part of a PIM-BIDIR group (its ingress PE determines this from the packet's destination address C-G), and if the VPN backbone is not the RPL, then the encapsulation MUST carry information that can be used to identify the partition to which the ingress PE belongs 🔥.

When PE2 receives a packet from the PMSI, PE2 must determine, by examining the encapsulation, whether the packet's ingress PE belongs to the same partition (relative to the C-RPA of the packet's C-G) to which PE2 itself belongs. If not, PE2 discards the packet. Otherwise, PE2 performs the normal BIDIR-PIM data packet processing. With this rule in place, harmful loops cannot be introduced by the PEs into the customer's bidirectional tree 👑.

Note that if there is more than one partition, the VPN backbone will not carry a packet from one partition to another. The only way for a packet to get from one partition to another is for it to go up towards the RPA and then down another path to the backbone. If this is not considered desirable, then all PEs should choose the same Upstream PE for a given C-RPA. Then, multiple partitions will only exist during routing transients 💅.

11.2.2. Using PE Distinguisher Labels ✨

okay so bestie, if a given P-tunnel is carrying packets along a bidirectional C-tree, then except for the case in Sections 11.1 and 11.2.3, every packet on that P-tunnel MUST carry a PE Distinguisher Label (defined in Section 4) 💅, using the encapsulation from Section 12.3. no skipping it!

when a PE sends a packet of a bidirectional C-group into the P-tunnel, that packet carries the PE Distinguisher Label for the partition (for the C-group's C-RPA) that the transmitting PE belongs to 🫶. this is the label bound to the Upstream PE of that partition, and it is NOT necessarily the label bound to the transmitting PE itself. the difference is giving main character energy 👑

quick reminder: PE Distinguisher Labels are upstream-assigned labels, so they're assigned and advertised by the node at the root of the P-tunnel 🌸. the info about them rides along with Intra-AS I-PMSI A-D routes and/or S-PMSI A-D routes, encoded into the PE Distinguisher Labels attribute carried by those routes ✨

if a PE gets a packet with a PE label that does NOT identify its own partition, the receiving PE discards the packet. no exceptions, periodt 🙅‍♀️

also, this procedure does not necessarily make the root of a P-tunnel assign a PE Distinguisher Label for every PE in the tunnel 🦋. if the root is the only PE that can transmit to the P-tunnel, then the root only needs to assign PE Distinguisher Labels for the PEs it has picked as the UMHs for the particular C-RPAs it knows about. less work, same slay ✨

11.2.3. Partial Mesh of MP2MP P-Tunnels 🧚

there's ONE case where supporting BIDIR-PIM C-groups doesn't need a PE Distinguisher Label, and it's honestly so clean 😭. for each C-RPA, suppose a distinct MP2MP LSP is used as the P-tunnel serving that C-RPA's partition. then, for a given packet, a PE receiving it from a P-tunnel can infer the partition from the tunnel itself. so PE Distinguisher Labels are not needed in this case. the tunnel does the job for you 💖

12. Encapsulations 🎀

okay the BGP-based auto-discovery procedures make sure that the PEs in a single MVPN only use tunnels they can ALL support 🫶. and for a given kind of tunnel, they only use encapsulations that they can all support too. everyone's on the same page, no drama 🌈

12.1. Encapsulations for Single PMSI per P-Tunnel 🌸

12.1.1. Encapsulation in GRE 💜

GRE encapsulation can be used for any PMSI instantiated by a mesh of unicast P-tunnels 🦋, and also for any PMSI instantiated by one or more PIM P-tunnels of any sort. so it's super flexible, bestie ✨

   Packets received        Packets in transit      Packets forwarded
   at the ingress PE       in the service          by the egress PEs
                           provider network

                           +---------------+
                           |  P-IP Header  |
                           +---------------+
                           |      GRE      |
   ++=============++       ++=============++       ++=============++
   || C-IP Header ||       || C-IP Header ||       || C-IP Header ||
   ++=============++ >>>>> ++=============++ >>>>> ++=============++
   || C-Payload   ||       || C-Payload   ||       || C-Payload   ||
   ++=============++       ++=============++       ++=============++

the IP Protocol Number field in the P-IP header MUST be set to 47 🔥. the Protocol Type field of the GRE header is set to either 0x800 or 0x86dd, depending on whether the C-IP header is IPv4 or IPv6, respectively. so it's basically matching the vibe of the inner packet 💅

when a particular PE transmits an encapsulated packet, the source IP address in the P-IP header must be the same address that the PE uses to identify itself in the VRF Route Import Extended Communities it attaches to any of the VPN-IP routes eligible for UMH determination that it advertises via BGP (see Section 5.1). it has to match, no cap 🧚

if the PMSI is instantiated by a PIM tree, the destination IP address in the P-IP header is the group P-address associated with that tree. the GRE key field value is omitted in this case ✨

if the PMSI is instantiated by unicast P-tunnels, the destination IP address is the address of the destination PE, and the optional GRE key field is used to identify a particular MVPN 🌈. in this case, each PE would have to advertise a key field value for each MVPN, and each PE would assign the key field value that it expects to receive. it's a whole little exchange, lowkey iconic 🫶

[RFC2784] specifies an optional GRE checksum 🍓, and [RFC2890] specifies an optional GRE sequence number field 💀.

the GRE sequence number field is not needed because the transport layer services for the original application will be provided by the C-IP header 🙈. the C-IP header already has them covered, so GRE doesn't need to!

the use of the GRE checksum field must follow [RFC2784] 🎀.

to make things go fast on the hardware side 🔥, this document recommends that the ingress PE routers encapsulate VPN packets without setting the checksum or sequence fields. speed is the goal, bestie 💨

12.1.2. Encapsulation in IP 🦋

IP-in-IP [RFC2003] is also a viable option 💖. the diagram below shows how the packet progresses as it enters and leaves the service provider network 🌸.

   Packets received        Packets in transit      Packets forwarded
   at the ingress PE       in the service          by the egress PEs
                           provider network

                           +---------------+
                           |  P-IP Header  |
   ++=============++       ++=============++       ++=============++
   || C-IP Header ||       || C-IP Header ||       || C-IP Header ||
   ++=============++ >>>>> ++=============++ >>>>> ++=============++
   || C-Payload   ||       || C-Payload   ||       || C-Payload   ||
   ++=============++       ++=============++       ++=============++

when the P-IP header is an IPv4 header, its Protocol Number field is set to either 4 or 41, depending on whether the C-IP header is an IPv4 header or an IPv6 header, respectively ✨.

when the P-IP header is an IPv6 header, its Next Header field is set to either 4 or 41, depending on whether the C-IP header is an IPv4 header or an IPv6 header, respectively 🧚.

same rule as before: when an encapsulated packet is transmitted by a particular PE, the source IP address in the P-IP header must be the same address that the PE uses to identify itself in the VRF Route Import Extended Communities it attaches to any of the VPN-IP routes eligible for UMH determination that it advertises via BGP (see Section 5.1). consistency is everything 💅

12.1.3. Encapsulation in MPLS 🌈

if the PMSI is instantiated as a P2MP MPLS LSP or an MP2MP LSP, MPLS encapsulation is used 💜. penultimate-hop-popping MUST be disabled for the LSP. don't skip this one, it's a big deal 🙅‍♀️

if other methods of assigning MPLS labels to multicast distribution trees are in use, those multicast distribution trees may be used as appropriate to instantiate PMSIs, and appropriate additional MPLS encapsulation procedures may be used 🦋.

   Packets received        Packets in transit      Packets forwarded
   at the ingress PE       in the service          by the egress PEs
                           provider network

                           +---------------+
                           | P-MPLS Header |
   ++=============++       ++=============++       ++=============++
   || C-IP Header ||       || C-IP Header ||       || C-IP Header ||
   ++=============++ >>>>> ++=============++ >>>>> ++=============++
   || C-Payload   ||       || C-Payload   ||       || C-Payload   ||
   ++=============++       ++=============++       ++=============++

12.2. Encapsulations for Multiple PMSIs per P-Tunnel 🍓

the encapsulations for transmitting multicast data messages when there are multiple PMSIs per P-tunnel are based on the single-PMSI encapsulation, but with an MPLS label used for demultiplexing 🫶. it's the same idea with an extra sorting step.

the label is upstream-assigned and distributed via BGP as specified in Section 4 ✨. the label must enable the receiver to select the proper VRF, and it may also enable the receiver to select a particular multicast routing entry within that VRF 💖.

12.2.1. Encapsulation in GRE 🌸

rather than the IP-in-GRE encapsulation from Section 12.1.1, we use the MPLS-in-GRE encapsulation 🧚. this is specified in [MPLS-IP]. the GRE protocol type MUST be set to 0x8847 💅. (the reason for using the unicast rather than the multicast value is specified in [MPLS-MCAST-ENCAPS] 🙈.)

12.2.2. Encapsulation in IP 🦋

rather than the IP-in-IP encapsulation from Section 12.1.2, we use the MPLS-in-IP encapsulation 🌈. this is specified in [MPLS-IP]. the IP protocol number field MUST be set to the value identifying the payload as an MPLS unicast packet 🔥. (there is no "MPLS multicast packet" protocol number, so unicast it is 😭.)

12.3. Encapsulations Identifying a Distinguished PE 👑

12.3.1. For MP2MP LSP P-Tunnels 💜

as discussed in Section 9, if a multicast data packet is traveling on a unidirectional C-tree, it is highly desirable for the PE that receives the packet from a PMSI to be able to figure out which PE transmitted it onto the PMSI 🕵️‍♀️. the encapsulations we've covered so far all give you that info, except in one case: if a PMSI is instantiated by an MP2MP LSP, then those encapsulations don't tell you which PE sent the packet onto the PMSI. sus, right? 🙈

so, when a packet traveling on a unidirectional C-tree is traveling on an MP2MP LSP P-tunnel, it MUST carry, as its second label, a label that has been bound to the packet's ingress PE 🫶. this label is an upstream-assigned label that the LSP's root node has bound to the ingress PE and has distributed via the PE Distinguisher Labels attribute of a PMSI A-D route (see Section 4) ✨. this label will appear immediately beneath the labels discussed in Sections 12.1.3 and 12.2 🌈.

a full specification of the procedures for advertising and for using the PE Distinguisher Labels attribute in this case is outside the scope of this document 💅. so that's a different story for another day!

12.3.2. For Support of PIM-BIDIR C-Groups 🦋

as we talked about in Section 11, when a packet belongs to a PIM-BIDIR multicast group, the set of PEs in that packet's VPN can be partitioned into a number of subsets, where exactly one PE in each partition is the Upstream PE for that partition 🍓. when such packets are transmitted on a PMSI, unless the procedures of Section 11.2.3 are being used, the packet needs to carry info identifying a particular partition 🧚. this is done by having the packet carry the PE Distinguisher Label corresponding to the Upstream PE of one partition. for a particular P-tunnel, this label will have been advertised by the node that is the root of that P-tunnel. (a full specification of the procedures for advertising PE Distinguisher Labels is out of the scope of this document 🙃.)

this label needs to be used whenever a packet belongs to a PIM-BIDIR C-group, no matter what encapsulation the P-tunnel uses 🔥. hence, the encapsulations of Section 12.2 MUST be used. if the P-tunnel contains only one PMSI, the PE label replaces the label discussed in Section 12.2. if the P-tunnel contains multiple PMSIs, the PE label follows the label discussed in Section 12.2 💖.

in general, PE Distinguisher Labels can be carried if the encapsulation is MPLS, MPLS-in-IP, or MPLS-in-GRE ✨. however, procedures for advertising and using PE Distinguisher Labels when the encapsulation is LDP-based MP2P MPLS are outside the scope of this specification 🌸.

12.4. General Considerations for IP and GRE Encapsulations 🌈

these apply also to the MPLS-in-IP and MPLS-in-GRE encapsulations 💜.

12.4.1. MTU (Maximum Transmission Unit) 🎀

okay bestie, so it's the originator of a C-packet who has to make sure the packet is small enough to reach all of its destinations, even once it's wrapped up in IP or GRE 💅

When a packet gets encapsulated in IP or GRE, the router doing the encapsulating MUST set the DF bit in the outer header ✨ This way the decapsulating router won't need to reassemble the encapsulating packets before it decapsulates. iconic, right?

Sometimes the encapsulating router already knows that a particular C-packet is too big to reach its destinations 🙈 How it knows that is outside the scope of this document, but if it does know, then:

If the DF bit is set in the IP header of a C-packet that is known to be too large, the router will discard the C-packet as being "too large" and follow normal IP procedures (which may require the return of an ICMP message to the source).
If the DF bit is not set in the IP header of a C-packet that is known to be too large, the router MAY fragment the packet before encapsulating it and then encapsulate each fragment separately. Alternatively, the router MAY discard the packet.

If the router discards a packet as too large, it should maintain Operations, Administration, and Maintenance (OAM) information related to this behavior, allowing the operator to properly troubleshoot the issue. this is the "help future me debug this" move 🧚

Note that if the entire path of the P-tunnel doesn't support an MTU that's large enough to carry a particular encapsulated C-packet, and if the encapsulating router does not do fragmentation, then the customer will not receive the expected connectivity 😭

12.4.2. TTL (Time to Live) 🦋

The ingress PE should not copy the TTL field from the payload IP header received from a CE router to the delivery IP or MPLS header. The setting of the TTL of the delivery header is determined by the local policy of the ingress PE router. the TTL drama is so real, don't copy it over, periodt 💅

12.4.3. Avoiding Conflict with Internet Multicast 🌸

If the SP is providing Internet multicast, distinct from its VPN multicast services, and using PIM based P-multicast trees, it must ensure that the group P-addresses that it used in support of MVPN services are distinct from any of the group addresses of the Internet multicasts it supports. This is best done by using administratively scoped addresses [ADMIN-ADDR]. 👑

The group C-addresses need not be distinct from either the group P-addresses or the Internet multicast addresses.

12.5. Differentiated Services 💜

The setting of the DS (Differentiated Services) field in the delivery IP header should follow the guidelines outlined in [RFC2983]. Setting the Traffic Class field [RFC5462] in the delivery MPLS header should follow the guidelines in [RFC3270]. An SP may also choose to deploy any of additional Differentiated Services mechanisms that the PE routers support for the encapsulation in use. Note that the type of encapsulation determines the set of Differentiated Services mechanisms that may be deployed. the encapsulation you pick basically decides your options, so plan ahead 🍓

13. Security Considerations 🔒💀

This document describes an extension to the procedures of [RFC4364], and hence shares the security considerations described in [RFC4364] and [RFC4365].

When GRE encapsulation is used, the security considerations of [MPLS-IP] are also relevant. Additionally, the security considerations of [RFC4797] are relevant as it discusses implications on packet spoofing in the context of BGP/MPLS IP VPNs.

The security considerations of [MPLS-HDR] apply when MPLS encapsulation is used.

This document makes use of a number of control protocols: PIM [PIM-SM], BGP [MVPN-BGP], mLDP [MLDP], and RSVP-TE [RSVP-P2MP]. Security considerations relevant to each protocol are discussed in the respective protocol specifications.

If one uses the UDP-based protocol for switching to S-PMSI (as specified in Section 7.4.2), then an S-PMSI Join message (i.e., a UDP packet with destination port 3232 and destination address ALL-PIM-ROUTERS) that is not received over a PMSI (e.g., one received directly from a CE router) is an illegal packet and MUST be dropped. absolutely do not let a CE router's S-PMSI Join sneak in, that's an illegal packet, bestie 🚫

The various procedures for P-tunnel construction have security issues that are specific to the way that the P-tunnels are used in this document. When P-tunnels are constructed via such techniques as PIM, mLDP, or RSVP-TE, each P or PE router receiving a control message MUST ensure that the control message comes from another P or PE router, not from a CE router. (Interpreting an mLDP or PIM or RSVP-TE control message from a CE router as referring to a P-tunnel would be a bug.) 🐛

A PE MUST NOT accept BGP routes of the MCAST-VPN address family from a CE.

If BGP is used as a CE-PE routing protocol, then when a PE receives an IP route from a CE, if this route carries the VRF Route Import Extended Community, the PE MUST remove this Community from the route before turning it into a VPN-IP route. Routes that a PE advertises to a CE MUST NOT carry the VRF Route Import Extended Community. 🫶

An ASBR may receive, from one SP's domain, an mLDP, PIM, or RSVP-TE control message that attempts to extend a P-tunnel from one SP's domain into another SP's domain. This is perfectly valid if there is an agreement between the SPs to jointly provide an MVPN service. In the absence of such an agreement, however, this could be an illegitimate attempt to intercept data packets. By default, an ASBR MUST NOT allow P-tunnels to extend beyond AS boundaries. However, it MUST be possible to configure an ASBR to allow this on a specified set of interfaces.

Big warning, bestie: P-tunnels crossing AS boundaries are only okay with an actual agreement between the SPs. Otherwise it could be someone trying to intercept data packets. 👀🚨

Many of the procedures in this document cause the SP network to create and maintain an amount of state that is proportional to customer multicast activity. If the amount of customer multicast activity exceeds expectations, this can potentially cause P and PE routers to maintain an unexpectedly large amount of state, which may cause control and/or data plane overload. To protect against this situation, an implementation should provide ways for the SP to bound the amount of state it devotes to the handling of customer multicast activity.

In particular, an implementation SHOULD provide mechanisms that allow an SP to place limitations on the following:

total number of (C-*,C-G) and/or (C-S,C-G) states per VRF
total number of P-tunnels per VRF used for S-PMSIs
total number of P-tunnels traversing a given P router

A PE implementation MAY also provide mechanisms that allow an SP to limit the rate of change of various MVPN-related states on PEs, as well as the rate at which MVPN-related control messages may be received by a PE from the CEs and/or sent from the PE to other PEs. 🌈

An implementation that provides the procedures specified in Sections 10.1 or 10.2 MUST provide the capability to impose an upper bound on the number of Source Active A-D routes generated and on how frequently they may be originated. This MUST be provided on a per-PE, per-MVPN granularity. per-PE AND per-MVPN, no cap, this is non-negotiable 💅

Lack of the mechanisms that allow an SP to limit the rate of change of various MVPN-related states on PEs, as well as the rate at which MVPN-related control messages may be received by a PE from the CEs and/or sent from the PE to other PEs may result in the control plane overload on the PE, which in turn would adversely impact all the customers connected to that PE, as well as to other PEs. 😱

See also the Security Considerations section of [MVPN-BGP].

14. IANA Considerations 📋✨

Section 7.4.2 defines the "S-PMSI Join message", which is carried in a UDP datagram whose port number is 3232. This port number had already been assigned by IANA to "MDT port". The reference has been updated to this document.

IANA has created a registry for the "S-PMSI Join message Type Field". Assignments are to be made according to the policy "IETF Review" as defined in [RFC5226]. The value 1 has been registered with a reference to this document. The description reads "PIM IPv4 S-PMSI (unaggregated)".

[PIM-ATTRIB] establishes a registry for "PIM Join attribute Types". IANA has assigned the value 1 to the "MVPN Join Attribute" with a reference to this document.

IANA has assigned SAFI 129 to "Multicast for BGP/MPLS IP Virtual Private Networks (VPNs)" with a reference to this document and [MVPN-BGP]. 🦋

15. Acknowledgments 🫶🌟

Significant contributions were made Arjen Boers, Toerless Eckert, Adrian Farrel, Luyuan Fang, Dino Farinacci, Lenny Giuliano, Shankar Karuna, Anil Lohiya, Tom Pusateri, Ted Qian, Robert Raszuk, Tony Speakman, Dan Tappan. the legends who made this happen 🌈👑

16. References 📚

16.1. Normative References 📚✨

okay bestie, these are the docs this RFC leans on SO hard that you literally can't implement it without reading them first 🫶

[MLDP] Wijnands, IJ., Ed., Minei, I., Ed., Kompella, K., and B. Thomas, "Label Distribution Protocol Extensions for Point-to-Multipoint and Multipoint-to-Multipoint Label Switched Paths", RFC 6388, November 2011.
[MPLS-HDR] Rosen, E., Tappan, D., Fedorkow, G., Rekhter, Y., Farinacci, D., Li, T., and A. Conta, "MPLS Label Stack Encoding", RFC 3032, January 2001.
[MPLS-IP] Worster, T., Rekhter, Y., and E. Rosen, Ed., "Encapsulating MPLS in IP or Generic Routing Encapsulation (GRE)", RFC 4023, March 2005.
[MPLS-MCAST-ENCAPS] Eckert, T., Rosen, E., Ed., Aggarwal, R., and Y. Rekhter, "MPLS Multicast Encapsulations", RFC 5332, August 2008.
[MPLS-UPSTREAM-LABEL] Aggarwal, R., Rekhter, Y., and E. Rosen, "MPLS Upstream Label Assignment and Context-Specific Label Space", RFC 5331, August 2008.
[MVPN-BGP] Aggarwal, R., Rosen, E., Morin, T., and Y. Rekhter, "BGP Encodings and Procedures for Multicast in MPLS/BGP IP VPNs", RFC 6514, February 2012.
[OSPF] Moy, J., "OSPF Version 2", STD 54, RFC 2328, April 1998.
[OSPF-MT] Psenak, P., Mirtorabi, S., Roy, A., Nguyen, L., and P. Pillay-Esnault, "Multi-Topology (MT) Routing in OSPF", RFC 4915, June 2007.
[PIM-ATTRIB] Boers, A., Wijnands, I., and E. Rosen, "The Protocol Independent Multicast (PIM) Join Attribute Format", RFC 5384, November 2008.
[PIM-SM] Fenner, B., Handley, M., Holbrook, H., and I. Kouvelas, "Protocol Independent Multicast - Sparse Mode (PIM-SM): Protocol Specification (Revised)", RFC 4601, August 2006.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC4364] Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private Networks (VPNs)", RFC 4364, February 2006.
[RFC4659] De Clercq, J., Ooms, D., Carugi, M., and F. Le Faucheur, "BGP-MPLS IP Virtual Private Network (VPN) Extension for IPv6 VPN", RFC 4659, September 2006.
[RFC5462] Andersson, L. and R. Asati, "Multiprotocol Label Switching (MPLS) Label Stack Entry: "EXP" Field Renamed to "Traffic Class" Field", RFC 5462, February 2009.
[RSVP-OOB] Ali, Z., Swallow, G., and R. Aggarwal, "Non-Penultimate Hop Popping Behavior and Out-of-Band Mapping for RSVP-TE Label Switched Paths", RFC 6511, February 2012.
[RSVP-P2MP] Aggarwal, R., Ed., Papadimitriou, D., Ed., and S. Yasukawa, Ed., "Extensions to Resource Reservation Protocol - Traffic Engineering (RSVP-TE) for Point-to-Multipoint TE Label Switched Paths (LSPs)", RFC 4875, May 2007.

16.2. Informative References 🔍

these ones are the "good to know" crew 💖 not required reading, but they give you the full picture

[ADMIN-ADDR] Meyer, D., "Administratively Scoped IP Multicast", BCP 23, RFC 2365, July 1998.
[BIDIR-PIM] Handley, M., Kouvelas, I., Speakman, T., and L. Vicisano, "Bidirectional Protocol Independent Multicast (BIDIR-PIM)", RFC 5015, October 2007.
[BSR] Bhaskar, N., Gall, A., Lingard, J., and S. Venaas, "Bootstrap Router (BSR) Mechanism for Protocol Independent Multicast (PIM)", RFC 5059, January 2008.
[MVPN-REQ] Morin, T., Ed., "Requirements for Multicast in Layer 3 Provider-Provisioned Virtual Private Networks (PPVPNs)", RFC 4834, April 2007.
[RFC2003] Perkins, C., "IP Encapsulation within IP", RFC 2003, October 1996.
[RFC2784] Farinacci, D., Li, T., Hanks, S., Meyer, D., and P. Traina, "Generic Routing Encapsulation (GRE)", RFC 2784, March 2000.
[RFC2890] Dommety, G., "Key and Sequence Number Extensions to GRE", RFC 2890, September 2000.
[RFC2983] Black, D., "Differentiated Services and Tunnels", RFC 2983, October 2000.
[RFC3270] Le Faucheur, F., Wu, L., Davie, B., Davari, S., Vaananen, P., Krishnan, R., Cheval, P., and J. Heinanen, "Multi-Protocol Label Switching (MPLS) Support of Differentiated Services", RFC 3270, May 2002.
[RFC3618] Fenner, B., Ed., and D. Meyer, Ed., "Multicast Source Discovery Protocol (MSDP)", RFC 3618, October 2003.
[RFC4365] Rosen, E., "Applicability Statement for BGP/MPLS IP Virtual Private Networks (VPNs)", RFC 4365, February 2006.
[RFC4607] Holbrook, H. and B. Cain, "Source-Specific Multicast for IP", RFC 4607, August 2006.
[RFC4797] Rekhter, Y., Bonica, R., and E. Rosen, "Use of Provider Edge to Provider Edge (PE-PE) Generic Routing Encapsulation (GRE) or IP in BGP/MPLS IP Virtual Private Networks", RFC 4797, January 2007.
[RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 5226, May 2008.

Contributing Authors 👯‍♀️💜

these are the babes who helped build this one, iykyk 🌸

Sarveshwar Bandi
Motorola
Vanenburg IT park, Madhapur,
Hyderabad, India
EMail: sarvesh@motorola.com

Yiqun Cai
Cisco Systems, Inc.
170 Tasman Drive
San Jose, CA, 95134
EMail: ycai@cisco.com

Thomas Morin
France Telecom R & D
2, avenue Pierre-Marzin
22307 Lannion Cedex
France
EMail: thomas.morin@francetelecom.com

Yakov Rekhter
Juniper Networks
1194 North Mathilda Ave.
Sunnyvale, CA 94089
EMail: yakov@juniper.net

IJsbrand Wijnands
Cisco Systems, Inc.
170 Tasman Drive
San Jose, CA, 95134
EMail: ice@cisco.com

Seisho Yasukawa
NTT Corporation
9-11, Midori-Cho 3-Chome
Musashino-Shi, Tokyo 180-8585,
Japan
Phone: +81 422 59 4769
EMail: yasukawa.seisho@lab.ntt.co.jp

Editors' Addresses 💅🎀

and these are the editors who kept everything in order 🫶✨

Eric C. Rosen
Cisco Systems, Inc.
1414 Massachusetts Avenue
Boxborough, MA, 01719
EMail: erosen@cisco.com

Rahul Aggarwal
Juniper Networks
1194 North Mathilda Ave.
Sunnyvale, CA 94089
EMail: raggarwa_1@yahoo.com

📦 where this page comes from

ietf: rfc6513 2012-02
💖 girlified by claude-haiku-5-5 on 2026-10-17: a language model's rewrite of the original, under its terms