RFC 6037

Cisco Systems' Solution for Multicast in BGP/MPLS IP VPNs

📅 published October 2010, from ietf

RFC 6037: Cisco Systems' Solution for Multicast in BGP/MPLS IP VPNs 🎀

okay bestie, the title block is giving very specific vibes 💅 This is an Independent Submission, so it isn't from an IETF working group. It's in the Historic category, meaning it's kept for the record 📜 The ISSN is 2070-1721, and it was published in October 2010 🍓

The authors are E. Rosen (Ed.), Y. Cai (Ed.), and I. Wijnands, all from Cisco Systems, Inc. 🧚 The header doesn't have an "Obsoletes" or "Updates" line, so it doesn't claim to replace or change any other RFC ✨

Abstract

okay so this doc describes the MVPN (Multicast in BGP/MPLS IP VPNs) solution that Cisco Systems designed and deployed 🦋 The procedures in here are mostly a subset of the generalized MVPN framework that the IETF standardized more recently 💜 But here's the tea: Cisco deployed these procedures before the IETF standards came out (in some cases by over five years!), so a version built on them is a little different from a fully standards-compliant one 😭 The doc points out those differences as it goes 🫶

Status of This Memo

this one's not an Internet Standards Track spec 🙈 It's published for the historical record 📜

This document defines a Historic Document for the Internet community. It's a contribution to the RFC Series, independent of any other RFC stream. The RFC Editor chose to publish it at their discretion and says nothing about how valuable it is to implement or deploy 💅 Documents the RFC Editor approves for publication aren't candidates for any level of Internet Standard. See Section 2 of RFC 5741 for more on that 🌸

For the current status, any errata, and how to give feedback, check http://www.rfc-editor.org/info/rfc6037 🔥

Copyright (c) 2010 IETF Trust and the persons identified as the document authors. All rights reserved. 🌈

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as they're in effect on the publication date. Please read these carefully, because they spell out your rights and restrictions for this document 🫶

Table of Contents

1. Introduction ....................................................3
   1.1. Specification of Requirements ..............................3
   1.2. Scaling Multicast State Information in the Network Core ....3
   1.3. Overview ...................................................4
2. Multicast VRFs ..................................................6
3. Multicast Domains ...............................................7
   3.1. Model of Operation .........................................7
4. Multicast Tunnels ...............................................7
   4.1. Ingress PEs ................................................8
   4.2. Egress PEs .................................................8
   4.3. Tunnel Destination Address(es) .............................8
   4.4. Auto-Discovery .............................................8
        4.4.1. MDT-SAFI ...........................................10
   4.5. Which PIM Variant to Use ..................................10
   4.6. Inter-AS MDT Construction .................................11
        4.6.1. The PIM MVPN Join Attribute ........................11
               4.6.1.1. Definition ................................11
               4.6.1.2. Usage .....................................12
   4.7. Encapsulation in GRE ......................................13
   4.8. MTU .......................................................14
   4.9. TTL .......................................................14
   4.10. Differentiated Services ..................................14
   4.11. Avoiding Conflict with Internet Multicast ................14
5. The PIM C-Instance and the MT ..................................15
   5.1. PIM C-Instance Control Packets ............................15
   5.2. PIM C-Instance RPF Determination ..........................15
        5.2.1. Connector Attribute ................................16
6. Data MDT: Optimizing Flooding ..................................17
   6.1. Limitation of Multicast Domain ............................17
   6.2. Signaling Data MDTs .......................................17
   6.3. Use of SSM for Data MDTs ..................................19
7. Packet Formats and Constants ...................................20
   7.1. MDT TLV ...................................................20
   7.2. MDT Join TLV for IPv4 Streams .............................20
   7.3. MDT Join TLV for IPv6 Streams .............................21
   7.4. Multiple MDT Join TLVs per Datagram .......................22
   7.5. Constants .................................................22
8. IANA Considerations ............................................23
9. Security Considerations ........................................23
10. Acknowledgments ...............................................23
11. References ....................................................24
   11.1. Normative References .....................................24
   11.2. Informative References ...................................24

1. Introduction ✨

okay so this doc is the MVPN (Multicast in BGP/MPLS IP VPNs) solution Cisco designed and deployed 🎀 It's being made available for the record, and as a reference for interoperating with deployed implementations 📜 It's a technical spec, so please don't use it to guess Cisco's current or future plans 🙈

Like the Abstract said, the procedures here are mostly a subset of the generalized MVPN framework in [MVPN]. This implementation came out several years before [MVPN] was standardized, so it does differ in a few ways from a fully standards-compliant one 💅 Those differences get called out where they happen 🫶

The base spec for BGP/MPLS IP VPNs, [RFC4364], doesn't give IP multicast data or control traffic a way to travel from one VPN site to another 😭 This doc extends that spec by defining the protocols and procedures needed to support IP multicast 🔥

This spec assumes that:

  1. Protocol Independent Multicast (PIM) [PIM-SM], running over either IPv4 or IPv6, is the multicast routing protocol used within the VPN,
  2. PIM, running over IPv4, is the multicast routing protocol used within the service-provider (SP) network, and
  3. the SP network supports native IPv4 multicast forwarding.

Knowing the terminology and procedures of [RFC4364] is assumed. Knowing [PIM-SM] is also assumed 🧚

1.1. Specification of Requirements 💜

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. 🫶

1.2. Scaling Multicast State Information in the Network Core 🌈

okay this is the big scaling problem bestie 😭 The BGP/MPLS IP VPN service in [RFC4364] gives a VPN "optimal" unicast routing across the SP backbone, meaning a packet follows the "shortest path" the backbone's own routing algorithm picks 💅 That optimal routing works without making the "P routers" (the backbone routers that aren't "provider edge" or "PE" routers) keep any routing info specific to a VPN. Honestly, the P routers don't keep any per-VPN state at all 🍓

Multicast is where it gets messy 🙈 Optimal multicast routing would need one or more multicast distribution trees built in the backbone for every multicast group in use. If a multicast group inside a VPN uses source-based distribution trees, optimal routing needs one tree per transmitter of that group. If shared trees are used, you still need one tree per group. Each tree needs state in some set of the P routers, and that state grows with the number of multicast transmitters. The reason there's at least one tree per group is that each group might have different receivers, and multicast routing algorithms work really hard to make sure a multicast packet never goes to a node that isn't on the path to a receiver 🦋

Here's the problem: an SP usually supports lots of VPNs, each VPN can have lots of multicast groups, and each group can have lots of transmitters. So one or more trees per group just doesn't scale 😭 The SP has zero control over how many multicast groups and transmitters show up in the VPNs, and it's hard to put any limit on those numbers 💀

So for a scalable multicast solution in BGP/MPLS IP VPNs, the state the P routers keep has to scale with something the SP does control 👑 This spec describes exactly that kind of solution. In it, the state in the P routers scales only with the number of VPNs running over the backbone. The P router state is NOT sensitive to the number of multicast groups or the number of multicast transmitters inside the VPNs ✨ The catch is that multicast routes become less optimal. A PE that isn't on the path to any receiver of a particular multicast group may still get multicast packets for that group, and if that happens, it has to discard them 🥺 But the SP still gets to pick the tradeoff between optimal routing and scalability, which is honestly iconic 💅🔥

1.3. Overview 💖

okay so, an SP decides whether a VPN is multicast-enabled 🌸. If it is, that VPN is a "Multicast Domain", and any PE that plugs into that VPN belongs to that Multicast Domain 🦋. Each Multicast Domain gets a default multicast distribution tree ("MDT") running through the backbone, and it connects ALL of the PEs in that domain ✨. A PE can be in as many Multicast Domains as it has VPNs attached, but every domain gets its own MDT, no sharing 🫶. The MDTs are built by running PIM in the backbone, and an MDT usually includes P routers on the paths between the PE routers 🧚.

The Default MDT for a Multicast Domain is built automatically as the PEs in that domain come up, which is a totally different vibe from normal multicast tree setup 👑. It does not wait for any multicast traffic to show up. It just exists before any multicast traffic is seen 🌈. Default MDTs match the Multidirectional Inclusive P-Multicast Service Interfaces ("MI-PMSIs") from [MVPN] 💜.

In BGP/MPLS IP VPNs, each CE ("Customer Edge", see [RFC4364]) router becomes a unicast routing adjacency of a PE router, but CE routers at different sites do NOT become unicast routing adjacencies of each other 🙈. This is a really important rule, and it stays the same for multicast routing. A CE router becomes a PIM adjacency of a PE router, but CE routers at different sites do NOT become PIM adjacencies of each other 💅. Multicast packets from inside a VPN get received from a CE router by an ingress PE router ✨. That ingress PE encapsulates the multicast packets and (at first) forwards them along the Default MDT to all the PE routers connected to sites of that VPN 🎀. So every PE attached to a site of the VPN gets all the multicast packets from inside that VPN 😭. If a particular PE router isn't on the path to any receiver of that multicast group, the PE just discards that packet 💀.

Now, if a ton of traffic is going to a multicast group, but that group doesn't have receivers at every VPN site, sending that group's traffic along the Default MDT is super wasteful 😤. So we also specify a way to build individual MDTs for specific multicast groups 🔥. These are called "Data MDTs" 🍓. A Data MDT delivers VPN data traffic for one particular multicast group only to the PE routers that are on the path to receivers of that group 💜. Using a Data MDT cuts down the multicast traffic on the backbone and also lightens the load on some of the PEs 🌸. The tradeoff is that P routers have to keep track of more state ⭐. The SP gets to decide how that tradeoff works 👑. Data MDTs match the Selective PMSI ("S-PMSIs") from [MVPN] 🦋.

This solution asks the SP to deploy the right protocols and procedures, but the customers never see it, so it's totally transparent to them 🫶. An enterprise running PIM-based multicasting can move from a private network to a BGP/MPLS IP VPN service and keep using whatever multicast router configurations it already had 🎀. No changes are needed on CE routers or on other routers at customer sites ✨. For example, any dynamic Rendezvous Point ("RP")-discovery procedures already in use can just stay right where they are 💖.

2. Multicast VRFs 🎀

The VPN Routing and Forwarding table ("VRF") idea, defined in [RFC4364], gets an upgrade here: it now includes multicast routing entries too, not just unicast ones 🌈.

Each VRF has its own multicast routing table 💅. When a multicast data or control packet arrives from a particular CE device, multicast routing happens inside the associated VRF ✨.

Each PE router runs a bunch of instances of PIM - Sparse Mode (PIM-SM), up to one per VRF 🧚. In each PIM-SM instance, the PE keeps a PIM adjacency with each PIM-capable CE router linked to that VRF 🫶. The multicast routing table made by each instance is specific to its VRF 🌸. We'll call these "VPN-specific PIM instances", or "PIM C-instances" 💜.

Each PE router also runs a "provider-wide" PIM-SM instance (a "PIM P-instance") 👑. In that one, it has a PIM adjacency with each of its IGP neighbors (in other words, with P routers), but NOT with any CE routers 🙈. It also doesn't form adjacencies with other PE routers, unless they just happen to be adjacent in the SP's network 🦋. P routers run the P-instance of PIM too, but they do NOT run a C-instance 💀.

To keep things clear about whether we're talking about the PIM P-instance or a PIM C-instance, we add the prefixes "P-" and "C-" to control messages, addresses, and so on 🍓. So a P-Join is a PIM Join handled by the PIM P-instance, and a C-Join is a PIM Join handled by a C-instance ✨. A P-group address is a group address in the SP's address space, and a C-group address is a group address in a VPN's address space 🔥.

3. Multicast Domains 🦋

3.1. Model of Operation ✨

A Multicast Domain ("MD") is basically a set of VRFs tied to interfaces that can send multicast traffic to each other 💖. From the PIM C-instance's point of view, a Multicast Domain works just like a multi-access interface 🌸. The PE routers in a given MD become PIM adjacencies of each other in the PIM C-instance 🫶.

Each multicast VRF gets assigned to one MD 🎀. Each MD is set up with its own distinct multicast P-group address, called the "Default MDT group address" 💜. That address is what gets used to build the Default MDT for the MD 👑.

When a PE router needs to send PIM C-instance control traffic to the other PE routers in the MD, it wraps that control traffic up 🌈. The outer packet uses the PE's own IPv4 address as the source IP address and the Default MDT group address as the destination IP address 💅. Here's the fun part: the Default MDT is part of the PIM P-instance, but the PEs that talk over the Default MDT are PIM adjacencies in a C-instance 🧚. Inside that C-instance, the Default MDT looks like a multi-access network that all the PEs are plugged into ✨. We dig into this more in Section 4 🔥.

The Default MDT doesn't only carry the PIM control traffic of the MD's PIM C-instance 🍓. By default, it also carries the multicast data traffic of the C-instance 🌸. Sometimes, though, multicast data traffic in a particular MD gets sent on a Data MDT instead of the Default MDT ⭐. You can read about how Data MDTs work in Section 6 💖.

Now, if an MDT (Default or Data) is set up using the ASM ("Any-Source Multicast") Service Model, the MDT (Default or Data) must have a P-group address that is "globally unique" 🙈. To be more precise, it has to be unique across the set of SP networks carrying the multicast traffic of the corresponding MD 😭. If the MDT is set up using the SSM ("Source-Specific Multicast") model, the P-group address of an MDT only needs to be unique relative to the source of the MDT 🫶 (but check out Section 4.4 for more on that 👀). Even so, some implementations require the same SSM group address to be assigned to all the PEs 💜. If you want to interoperate with those implementations, you have to follow this restriction 🎀.

4. Multicast Tunnels 🌈

An MD can be thought of as a set of PE routers connected by a multicast tunnel ("MT") 🧚. From the point of view of a VPN-specific PIM instance, an MT is one single multi-access interface 💅. In the SP network, a single MT is made from a Default MDT combined with zero or more Data MDTs 👑.

4.1. Ingress PEs 🦋

An ingress PE is a PE router that is either directly connected to the multicast sender in the VPN, or connected through a CE router 💖. When the multicast sender starts transmitting, and there are receivers (or a PIM RP) behind other PE routers in the common MD, the ingress PE becomes the transmitter of either the Default MDT group or a Data MDT group in the SP network ✨.

4.2. Egress PEs 🌸

A PE router with a VRF configured in an MD becomes a receiver of the Default MDT group for that MD 🫶. A PE router may also join a Data MDT group if it has a VPN-specific PIM instance that is forwarding traffic for a particular C-group to one of its attached sites, and that C-group has been associated with that particular Data MDT 🎀. When a PE router joins any P-group used for encapsulating VPN multicast traffic, it becomes one of the endpoints of the corresponding MT 💜.

When a packet comes in from an MT, the receiving PE works out the MD from the destination address of the received packet, which is a P-group address 🌈. Then the packet gets passed to the matching multicast VRF and VPN-specific PIM instance for further processing 💅.

4.3. Tunnel Destination Address(es) 🧚

An MT is an IP tunnel whose destination address is a P-group address 🔥. But an MT isn't stuck with just one P-group address for encapsulation ⭐. Based on the payload VPN multicast traffic, it can pick the Default MDT group address or one of the Data MDT group addresses (as described in Section 6 of this document) 🍓. That lets the MT reach a different set of PE routers in the common MD 👑.

4.4. Auto-Discovery 💖

Okay so, any PIM variant can be used to set up the Default MDT: PIM-SM, Bidirectional PIM [BIDIR], or PIM-Source-Specific Multicast (PIM-SSM) [SSM]. Except for PIM-SSM, the PEs only need to know the right P-group address to start setting up the Default MDTs 🧚. After that they find each other's addresses just by receiving PIM control traffic, like PIM Hellos, that they source (and encapsulate) for one another ✨.

BUT, with PIM-SSM, the necessary MDTs for an MD can't get set up until each PE in the MD knows the source address of every other PE in that same MD. So that info has to be auto-discovered 😭.

Here's the plot twist: a new BGP address family, the MDT-Subsequent Address Family Identifier ("MDT-SAFI"), gets defined 💅. Its Network Layer Reachability Information (NLRI) is made of a Route Distinguisher (RD), an IPv4 unicast address, and a multicast group address. A given PE router in a given MD builds an NLRI in this family from:

  • Its own IPv4 address. If it has several, it uses the one it will be placing in the IP Source Address field of multicast packets that it sends over the MDT.
  • An RD that has been assigned to the MD.
  • The P-group address, an IPv4 multicast address that is used as the IP Destination Address field of multicast packets sent over the MDT.

When a PE distributes this NLRI via BGP, it may include a Route Target (RT) Extended Communities attribute. This RT MUST be an "Import RT" [RFC4364] of each VRF in the MD. The ordinary BGP distribution procedures used by [RFC4364] then make sure each PE learns the MDT-SAFI "address" of every other PE in the MD, and that the learned MDT-SAFI addresses get associated with the right VRFs 🎀.

If a PE receives an MDT-SAFI NLRI that doesn't have an RT attribute, the P-group address from the NLRI has to be used to associate it with a particular VRF. In this case, each Multicast Domain MUST be associated with a unique P-address, even if PIM-SSM is used. However, finding a unique P-address for a multi-provider multicast group may be difficult 🙈.

So, to make multi-provider Multicast Domains easier to deploy, this specification REQUIRES the use of the MDT-SAFI NLRI (even if PIM-SSM is not used to set up the Default MDT). This specification also REQUIRES that an implementation be capable of using PIM-SSM to set up the Default MDT 👑.

In [MVPN], the MDT-SAFI gets replaced by the Intra-Autonomous-System Inclusive-PMSI auto-discovery ("Intra-AS I-PMSI A-D") route. That one is a generalized version of the MDT-SAFI, which lets the "Default MDTs" and "Data MDTs" be implemented as MPLS P2MP LSPs ("Point-to-Multipoint Label Switched Paths") or MP2MP LSPs ("Multipoint-to-Multipoint Label Switched Paths"), as well as by PIM-created multicast distribution trees. When it's the PIM-created kind, the Intra-AS A-D routes carry the same information the MDT-SAFI does, just with a different encoding 🦋.

The Intra-AS A-D routes also carry Route Targets, so they can be distributed the same way unicast routes are, including being distributed inter-AS. (Despite the name, inter-AS distribution of Intra-AS I-PMSI A-D routes is sometimes necessary in [MVPN]. Iykyk 💀.)

The encoding of the MDT-SAFI is specified in the next subsection.

4.4.1. MDT-SAFI

BGP messages where AFI=1 and SAFI=66 are "MDT-SAFI" messages 🍓.

The NLRI format is the 8-byte-RD:IPv4-address followed by the MDT group address. In other words, the MP_REACH attribute for this SAFI contains one or more tuples of the following form:

       +-------------------------------+
       |                               |
       |  RD:IPv4-address (12 octets)  |
       |                               |
       +-------------------------------+
       |    Group Address (4 octets)   |
       +-------------------------------+

The IPv4 address identifies the PE that originated this route, and the RD identifies a VRF in that PE. The group address MUST be an IPv4 multicast group address and is used to build the P-tunnels. All PEs attached to a given MVPN MUST specify the same group address, even if the group is an SSM group. MDT-SAFI routes don't carry RTs, so the group address is what associates a received MDT-SAFI route with a VRF 🌸.

4.5. Which PIM Variant to Use 🌈

To keep the amount of multicast routing state the P routers maintain as low as possible, the Default MDTs should be realized as shared trees, such as PIM bidirectional trees. However, the operational procedures for assigning P-group addresses may be greatly simplified, especially in the case of multi-provider MDs, if PIM-SSM is used 🧠.

Data MDTs are best realized as source trees, constructed via PIM-SSM ✨.

4.6. Inter-AS MDT Construction 🦋

Standard PIM techniques for constructing source trees presuppose that every router has a route to the source of the tree. But if the source of the tree is in a different AS than a particular P router, that P router might not have a route to the source. For example, the remote AS may be using BGP to distribute a route to the source, but a particular P router could be part of a "BGP-free core", where the P routers aren't aware of BGP-distributed routes 😳.

What's needed here is a way for a PE to tell PIM to construct the tree through a particular BGP speaker, the "BGP Next Hop" for the tree source. This can be done with a PIM extension 🔥.

If the PE has selected the source of the tree from the MDT SAFI address family, it may be desirable to build the tree along the route to the MDT SAFI address, rather than along the route to the corresponding IPv4 address. That way the inter-AS portion of the tree can follow a path specifically chosen for multicast (in other words, the inter-AS multicast topology can be "non-congruent" to the inter-AS unicast topology). This too needs a PIM extension 💅.

The necessary PIM extension is the PIM MVPN Join Attribute, described in the next subsection.

4.6.1. The PIM MVPN Join Attribute

4.6.1.1. Definition

In [PIM-ATTRIB], the notion of a "Join Attribute" is defined, and a format for including Join Attributes in PIM Join/Prune messages is specified. Now we define a new Join Attribute, which we call the "MVPN Join Attribute" 🎀.

   0                   1                   2                   3
   0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |F|E|   Type    | Length        |     Proxy IP address
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                                   |      RD
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-.......

The 6-bit Type field of the MVPN Join Attribute is set to 1 ✨.

The F-bit is set to 0, which means the attribute is non-transitive 🙅‍♀️.

The rules for setting the E-bit are given in [PIM-ATTRIB].

Two information fields are carried in the MVPN Join Attribute:

  • Proxy IP address: The IP address of the node towards which the PIM Join/Prune message is to be forwarded. This will either be an IPv4 or an IPv6 address, depending on whether the PIM Join/Prune message itself is IPv4 or IPv6.
  • RD: An eight-byte RD. This immediately follows the proxy IP address.

The PIM message also carries the address of the upstream PE.

For an intra-AS MVPN, the proxy and the upstream PE are the same. For an inter-AS MVPN, the proxy is the AS Border Router (ASBR) that is the exit point from the local AS on the path to the upstream PE 🌍.

4.6.1.2. Usage

When a PE router creates a PIM Join/Prune message to set up an inter-AS Default MDT, it does so because it received a particular MDT-SAFI route. It includes an MVPN Join Attribute whose fields are set as follows 📝:

  • If the upstream PE is in the same AS as the local PE, the Proxy field contains the address of the upstream PE. Otherwise, it contains the address of the BGP Next Hop on the route to the upstream PE.
  • The RD field contains the RD from the NLRI of the MDT-SAFI route.
  • The Upstream PE field contains the address of the PE that originated the MDT-SAFI route (obtained from the NLRI of that route).

When a PIM router processes a PIM Join/Prune message with an MVPN Join Attribute, it first checks whether the Proxy field contains one of its own addresses 👀.

If not, the router uses the proxy IP address to determine the Reverse Path Forwarding (RPF) interface and neighbor. The MVPN Join Attribute MUST be passed upstream, unchanged 🫶.

If the proxy address is one of the router's own IP addresses, the router looks in its BGP routing table for an MDT-SAFI route whose NLRI consists of the upstream PE address prepended with the RD from the Join Attribute. If there's no match, the PIM message is discarded 🥲. If there is a match, the IP address from the BGP Next Hop field of the matching route is used to determine the RPF interface and neighbor. When the PIM Join/Prune is forwarded upstream, the Proxy field is replaced with the address of the BGP Next Hop, and the RD and Upstream PE fields are left unchanged ✨.

4.7. Encapsulation in GRE 🧚

Generic Routing Encapsulation (GRE) [GRE1701] is used when sending multicast traffic through an MDT. The diagram below shows how the packet progresses as it enters and leaves the service-provider network 🌸.

   Packets received        Packets in transit      Packets forwarded
   at ingress PE           in the service-         by egress PEs
                           provider network

                           +---------------+
                           |  P-IP Header  |
                           +---------------+
                           |      GRE      |
   ++=============++       ++=============++       ++=============++
   || C-IP Header ||       || C-IP Header ||       || C-IP Header ||
   ++=============++ >>>>> ++=============++ >>>>> ++=============++
   || C-Payload   ||       || C-Payload   ||       || C-Payload   ||
   ++=============++       ++=============++       ++=============++

The IPv4 Protocol Number field in the P-IP Header MUST be set to 47. The Protocol Type field of the GRE Header MUST be set to 0x0800 if the C-IP header is an IPv4 header; it MUST be set to 0x86dd if the C-IP header is an IPv6 header 💜.

[GRE2784] specifies an optional GRE checksum, and [GRE2890] specifies optional GRE Key and Sequence Number fields 🍓.

The GRE Key field isn't needed, because the P-group address in the delivery IP header already identifies the MD, and so associates the VRF context, for the payload packet to be processed further 🔑.

The GRE Sequence Number field isn't needed either, because the transport layer services for the original application are provided by the C-IP Header 🙌.

The use of the GRE Checksum field MUST follow [GRE2784] 💅.

To make high-speed implementation easier, this document recommends that the ingress PE routers encapsulate VPN packets without setting the Checksum, Key, or Sequence Number field 🚀.

4.8. MTU 📏✨

okay so bestie, multicast group addresses get used as tunnel destination addresses, which means the usual Path MTU discovery stuff literally can't be used 💀. so there are two rules:

  1. The ingress PE router (the one doing the encapsulation) MUST NOT set the DF ("Don't Fragment") bit in the outer header, and
  2. If the "DF" bit is cleared in the IP header of the C-Packet, fragment the C-Packet before encapsulation if appropriate. this one is SUPER important in practice bc the reassembly function is way slower than decapsulating and forwarding packets on today's router implementations 😭🫶

4.9. TTL ⏳

The ingress PE should not copy the Time to Live (TTL) field from the payload IP header received from a CE router to the delivery IP header. Setting the TTL of the delivery IP header is determined by the local policy of the ingress PE router. so the TTL is basically the ingress PE's call, not the customer's 💅

4.10. Differentiated Services 🎀

By default, setting of the DS ("Differentiated Services") field in the delivery IP header should follow the guidelines outlined in [DIFF2983]. an SP may also choose to deploy any of the additional mechanisms the PE routers support 🌈

4.11. Avoiding Conflict with Internet Multicast 🙅‍♀️

If the SP is providing Internet multicast, distinct from its VPN multicast services, it must ensure that the P-group addresses that correspond to its MDs are distinct from any of the group addresses of the Internet multicasts it supports. this is best done by using administratively scoped addresses [ADMIN-ADDR] ✨

The C-group addresses need not be distinct from either the P-group addresses or the Internet multicast addresses. so no stress there 🫶

5. The PIM C-Instance and the MT 🦋

If a particular VRF is in a particular MD, the corresponding MT is treated by that VRF's VPN-specific PIM instances as a LAN interface. as a result, the PEs that are adjacent on the MT will generate and process PIM control packets, such as Hello, Join/Prune, and Assert. Designated Forwarder election occurs just as it would on an actual LAN interface. it's giving real LAN vibes 🧚

5.1. PIM C-Instance Control Packets 📦

The PIM protocol packets are sent to ALL-PIM-ROUTERS (224.0.0.13 for IPv4 or ff02::d for IPv6) in the context of that VRF, but when in transit in the provider network, they are encapsulated using the Default MDT group configured for that MD. this is what lets VPN-specific PIM routes get extended from site to site without showing up in the P routers 👑

If a PIM C-Instance control packet is an IPv6 packet, its source address is the IPv4-mapped IPv6 address corresponding to the IPv4 address of the PE router sending the packet. 💜

5.2. PIM C-Instance RPF Determination 🔍

Although the MT is treated as a PIM-enabled interface, unicast routing is NOT run over it, and there are no unicast routing adjacencies over it. it is therefore necessary to specify special procedures for determining when the MT is to be regarded as the "RPF Interface" for a particular C-address. main character energy for the MT, honestly 🙈

When a PE needs to determine the RPF interface of a particular C-address, it looks up the C-address in the VRF. If the route matching it is not a VPN-IP route learned from MP-BGP as described in [RFC4364], or if that route's outgoing interface is one of the interfaces associated with the VRF, then ordinary PIM procedures for determining the RPF interface apply.

However, if the route matching the C-address is a VPN-IP route whose outgoing interface is not one of the interfaces associated with the VRF, then PIM will consider the outgoing interface to be the MT associated with the VPN-specific PIM instance.

Once PIM has determined that the RPF interface for a particular C-address is the MT, it is necessary for PIM to determine the RPF neighbor for that C-address. This will be one of the other PEs that is a PIM adjacency over the MT.

The BGP "Connector" Attribute is defined. Whenever a PE router distributes a VPN-IP address from a VRF that is part of an MD, it SHOULD distribute a Connector Attribute along with it. The Connector Attribute specifies the MDT address family, and its value is the IP address that the PE router is using as its source IP address for the multicast packets that are encapsulated and sent over the MT. When a PE has determined that the RPF interface for a particular C-address is the MT, it looks up the Connector Attribute that was distributed along with the VPN-IP address corresponding to that C-address. The value of this Connector Attribute is considered to be the RPF adjacency for the C-address. 🔗✨

There are older implementations in which the Connector Attribute is not present. In this case, as long as the "BGP Next Hop" for the C-address is one of the PEs that is a PIM adjacency, then that PE is treated as the RPF adjacency for that C-address. okay, the older gear gets a pass here 🥹

However, if the MD spans multiple Autonomous Systems, and an "option b" interconnect ([RFC4364], Section 10) is used, the BGP Next Hop might not be a PIM adjacency, and the RPF check will not succeed unless the Connector Attribute is used. so for multi-AS setups it's lowkey essential 🚨

In [MVPN], the Connector Attribute is replaced by the "VRF Route Import Extended Community" attribute. The latter is a generalized version, but carries the same information as the Connector Attribute does; the encoding, however, is different. same tea, different cup ☕💖

The Connector Attribute is defined in the following subsection.

5.2.1. Connector Attribute 🔌

The Connector Attribute is an optional transitive attribute. Its value field is formatted as follows:

           0                   1
           0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
          |0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1|
          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
          |                               |
          |  IPv4 Address of PE           |
          |                               |
          +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

6. Data MDT: Optimizing Flooding 🚀

6.1. Limitation of Multicast Domain 🧱

While the procedure specified in the previous section requires the P routers to maintain multicast state, the amount of state is bounded by the number of supported VPNs. The P routers do NOT run any VPN-specific PIM instances.

In particular, the use of a single bidirectional tree per VPN scales well as the number of transmitters and receivers increases, but not so well as the amount of multicast traffic per VPN increases.

The multicast routing provided by this scheme is not optimal, in that a packet of a particular multicast group may be forwarded to PE routers that have no downstream receivers for that group, and which hence may need to discard the packet. ngl this is the main tradeoff, so keep it in mind 👀

In the simplest configuration model, only the Default MDT group is configured for each MD. The result of the configuration is that all VPN multicast traffic, whether control or data, will be encapsulated and forwarded to all PE routers that are part of the MD. While this limits the number of multicast routing states the provider network has to maintain, it also requires PE routers to discard multicast C-packets if there are no receivers for those packets in the corresponding sites. In some cases, especially when the content involves high bandwidth but only a limited set of receivers, it is desirable that certain C-packets only travel to PE routers that do have receivers in the VPN to save bandwidth in the network and reduce load on the PE routers. 🍓

6.2. Signaling Data MDTs 📡

A simple protocol is proposed to signal additional P-group addresses to encapsulate VPN traffic. These P-group addresses are called Data MDT groups. The ingress PE router advertises a different P-group address (as opposed to always using the Default MDT group) to encapsulate VPN multicast traffic. Only the PE routers on the path to eventual receivers join the P-group, and therefore form an optimal multicast distribution tree in the service-provider network for the VPN multicast traffic. These multicast distribution trees are called Data MDTs because they do not carry PIM control packets exchanged by PE routers. 🌸

The following text documents the procedures of the initiation and teardown of the Data MDTs. The definition of the constants and timers can be found in Section 7.

  • The PE router connected to the source of the content initially uses the Default MDT group when forwarding the content to the MD.
  • When one or more pre-configured conditions are met, it starts to periodically announce the MDT Join TLV at the interval of [MDT_INTERVAL]. The MDT Join TLV is forwarded to all the PE routers in the MD.
    • A commonly used condition is the bandwidth. When the VPN traffic exceeds a certain threshold, it is more desirable to deliver the flow to the PE routers connected to receivers in order to optimize the performance of PE routers and the resources of the provider network. However, other conditions can also be devised, and they are purely implementation specific. 🧠
  • The MDT Join TLV is encapsulated in UDP.
    • UDP over IPv4 is used if the multicast stream being assigned to a Data MDT is an IPv4 stream. In this case, the UDP datagram is addressed to ALL-PIM-ROUTERS (224.0.0.13).
    • UDP over IPv6 is used if the multicast stream being assigned to a Data MDT is an IPv6 stream. In this case, the UDP datagram is addressed to ALL-PIM-ROUTERS (ff02::d).
    • The destination UDP port is 3232.
    • The UDP datagram is sent on the Default MDT. This allows all PE routers to receive the information. Any MDT Join that is not received over a Default MDT MUST be dropped. 🚫
  • Upon receiving an MDT Join TLV, PE routers connected to receivers will join the Data MDT group announced by the MDT Join TLV in the global table. When the Data MDT group is in PIM-SM or bidirectional PIM mode, the PE routers build a shared tree toward the RP. When the Data MDT group is set up using PIM-SSM, the PE routers build a source tree toward the PE router that is advertising the MDT Join TLV. The IP address of that PE router is learned from the IP Source Address field of the UDP packet that contains the MDT Join TLV.
    • PE routers that are not connected to receivers may wish to cache the states in order to reduce the delay when a receiver comes up in the future. 💭
  • After [MDT_DATA_DELAY], the PE router connected to the source starts encapsulating traffic using the Data MDT group. ⏰
  • When the pre-configured conditions are no longer met, e.g., the traffic stops, the PE router connected to the source stops announcing the MDT Join TLV. 🛑
  • If the MDT Join TLV is not received for an interval longer than [MDT_DATA_TIMEOUT], PE routers connected to the receivers just leave the Data MDT group in the global instance. 🧹

6.3. Use of SSM for Data MDTs 🦋

Okay so, Data MDTs need a set of multicast P-addresses that gets pre-allocated and dedicated just for them. These are the destination addresses the Data MDTs use, so they can't be shared with anything else 🎀

If you set up Data MDTs with SSM, each MD needs its own set of these multicast P-addresses. Every VRF in that MD has to be configured with the same set. If the set has n addresses, then each PE in the MD can be the source of n Data MDTs in that MD ✨

If you're not using SSM to set up Data MDTs, it gets messy 🙈 Each VRF needs its own unique set of multicast P-addresses, and two VRFs in the same MD can't have the same set. That means you need way more multicast P-addresses pre-allocated, and giving every VRF a different set is a huge management headache. So SSM for Data MDTs is very strongly recommended 💅

7. Packet Formats and Constants ✨

7.1. MDT TLV 🌸

The MDT TLV looks like this 👇

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |     Type      |            Length           |     Value       |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                               .                               |
   |                               .                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Type (8 bits): this is the type of the MDT TLV. In this spec, types 1 and 4 are defined 🧚

Length (16 bits): the total number of octets in the TLV for this type, including both the Type and Length fields 📏

Value (variable length): the content of the TLV 💜

7.2. MDT Join TLV for IPv4 Streams 💖

The MDT Join TLV for IPv4 streams looks like this 🍓

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |     Type      |           Length            |    Reserved     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           C-source                            |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           C-group                             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           P-group                             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Type (8 bits): Must be set to 1. 🔢

Length (16 bits): Must be set to 16. 🔢

Reserved (8 bits): for future use 🔮

C-source (32 bits): the IPv4 address of the traffic source in the VPN 🌈

C-group (32 bits): the IPv4 address of the multicast traffic destination address in the VPN 🎯

P-group (32 bits): the IPv4 group address that the PE router is going to use to encapsulate the flow (C-source, C-group) 🦋

7.3. MDT Join TLV for IPv6 Streams 🌟

The MDT Join TLV for IPv6 streams looks like this 💅

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |     Type      |           Length            |    Reserved     |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   |                           C-source                            |
   |                                                               |
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                                                               |
   |                           C-group                             |
   |                                                               |
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |                           P-group                             |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Type (8 bits): Must be set to 4. 🔢

Length (16 bits): Must be set to 40. 🔢

Reserved (8 bits): for future use 🔮

C-source (128 bits): the IPv6 address of the traffic source in the VPN 🌊

C-group (128 bits): the IPv6 address of the multicast traffic destination address in the VPN 🎀

P-group (32 bits): the IPv4 group address that the PE router is going to use to encapsulate the flow (C-source, C-group). Yes, it's still IPv4 here 👀

7.4. Multiple MDT Join TLVs per Datagram 🧺

A single UDP datagram MAY carry multiple MDT Join TLVs, as many as can fit entirely within it ✨ If there are multiple MDT Join TLVs in a UDP datagram, they MUST be of the same type. The end of the last MDT Join TLV (as determined by the MDT Join TLV Length field) MUST coincide with the end of the UDP datagram, as determined by the UDP Length field. When processing a received UDP datagram that contains one or more MDT Join TLVs, a router MUST be able to process all the MDT Join TLVs that fit into the datagram 🫶

7.5. Constants 🔥

[MDT_DATA_DELAY]: the interval before the PE router connected to the source will switch to the Data MDT group. The default value is 3 seconds ⏱️

[MDT_DATA_TIMEOUT]: the interval before which the PE router connected to the receivers will time out and leave the Data MDT group if no MDT_JOIN_TLV message has been received. The default value is 3 minutes. This value must be consistent among PE routers 🥺

[MDT_DATA_HOLDDOWN]: the interval before which the PE router will switch back to the Default MDT after it started encapsulating packets using the Data MDT group. This is used to avoid oscillation when traffic is bursty. The default value is 1 minute 🌀

[MDT_INTERVAL]: the interval the source PE router uses to periodically send MDT_JOIN_TLV messages. The default value is 60 seconds ⏰

8. IANA Considerations 📋

The codepoint for the Connector Attribute is defined in IANA's registry of BGP attributes. The reference has been updated to refer to this document. On the IANA web page, the codepoint is denoted as "deprecated". This document does not change that status. However, note that there are a large number of deployments using this codepoint, and this is likely to be the case for a number of years 😭

The codepoint for MDT-SAFI is defined in IANA's registry of BGP SAFI assignments. The reference has been updated to refer to this document ✅

9. Security Considerations 🚨

Okay this section is serious, bestie. Read it carefully 💀

[RFC4364] discusses in general the security considerations that pertain to when the RFC 4364 type of VPN is deployed.

[PIM-SM] discusses the security considerations that pertain to the use of PIM.

The security considerations of [RFC4023] and [RFC4797] apply whenever VPN traffic is carried through IP or GRE tunnels. No skipping this part, periodt 🔐

10. Acknowledgments 💜

Major contributions to this work have been made by Dan Tappan and Tony Speakman 👑

The authors also wish to thank Arjen Boers, Robert Raszuk, Toerless Eckert, and Ted Qian for their help and their ideas 🫶

11. References 📚

11.1. Normative References 💎

[GRE2784]      Farinacci, D., Li, T., Hanks, S., Meyer, D., and P.
               Traina, "Generic Routing Encapsulation (GRE)",
               RFC 2784, March 2000.

[PIM-SM]       Fenner, B., Handley, M., Holbrook, H., and I.
               Kouvelas, "Protocol Independent Multicast - Sparse
               Mode (PIM-SM): Protocol Specification (Revised)",
               RFC 4601, August 2006.

[PIM-ATTRIB]   Boers, A., Wijnands, I., and E. Rosen, "The Protocol
               Independent Multicast (PIM) Join Attribute Format",
               RFC 5384, November 2008.

[RFC2119]      Bradner, S., "Key words for use in RFCs to Indicate
               Requirement Levels", BCP 14, RFC 2119, March 1997.

[RFC4364]      Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private
               Networks (VPNs)", RFC 4364, February 2006.

11.2. Informative References 🌷

[ADMIN-ADDR]   Meyer, D., "Administratively Scoped IP Multicast",
               BCP 23, RFC 2365, July 1998.

[BIDIR]        Handley, M., Kouvelas, I., Speakman, T., and L.
               Vicisano, "Bidirectional Protocol Independent
               Multicast (BIDIR-PIM)", RFC 5015, October 2007.

[DIFF2983]     Black, D., "Differentiated Services and Tunnels",
               RFC 2983, October 2000.

[GRE1701]      Hanks, S., Li, T., Farinacci, D., and P. Traina,
               "Generic Routing Encapsulation (GRE)", RFC 1701,
               October 1994.

[GRE2890]      Dommety, G., "Key and Sequence Number Extensions to
               GRE", RFC 2890, September 2000.

[MVPN]         Rosen, E., Ed., and R. Aggarwal, Ed., "Multicast in
               MPLS/BGP IP VPNs", Work in Progress, January 2010.

[SSM]          Holbrook, H. and B. Cain, "Source-Specific Multicast
               for IP", RFC 4607, August 2006.

[RFC4023]      Worster, T., Rekhter, Y., and E. Rosen, Ed.,
               "Encapsulating MPLS in IP or Generic Routing
               Encapsulation (GRE)", RFC 4023, March 2005.

[RFC4797]      Rekhter, Y., Bonica, R., and E. Rosen, "Use of
               Provider Edge to Provider Edge (PE-PE) Generic Routing
               Encapsulation (GRE) or IP in BGP/MPLS IP Virtual
               Private Networks", RFC 4797, January 2007.

Authors' Addresses 🏡

Eric C. Rosen (editor)
Cisco Systems, Inc.
1414 Massachusetts Avenue
Boxborough, MA  01719
EMail: erosen@cisco.com


Yiqun Cai (editor)
Cisco Systems, Inc.
170 Tasman Drive
San Jose, CA  95134
EMail: ycai@cisco.com


IJsbrand Wijnands
Cisco Systems, Inc.
170 Tasman Drive
San Jose, CA  95134
EMail: ice@cisco.com

That's the whole RFC, bestie! 🌈✨ Data MDTs, PIM C-instances, MTU, TTL, all of it, served 💅💖

📦 where this page comes from