RFC 10039
Interconnecting EVPN and IPVPN Domains
RFC 10039: Interconnecting EVPN and IPVPN Domains 🎀
Okay bestie, here's the title-block tea ☕
- Number: RFC 10039, Standards Track, ISSN 2070-1721 📜
- Date: September 2026 🗓️
- Editors: J. Rabadan (Nokia) and A. Sajassi (Cisco) 💜
- Authors: J. Drake (Independent), W. Lin (HPE), J. Uttaro (Independent), and A. Simpson (Nokia) 🫶
- Updates: [RFC4271]. Section 1 says this document updates the BGP best-path selection procedures specified there, but only for the IPVPN and EVPN families when D-PATH is used for inter-domain connectivity 🔥
- Obsoletes: Nothing is listed in the title block 🙅♀️
Abstract 💖
Ethernet Virtual Private Network (EVPN) gives you one unified BGP control plane for both intra- and inter-subnet forwarding inside tenant networks, and it's iconic for that. But when a tenant network spans multiple domains (any combo of EVPN and IPVPN domains), you need rules for how those BGP domains (EVPN and IPVPN) talk to each other so the tenant stays connected end to end. That's exactly what this document defines ✨
It also introduces a brand-new BGP Path Attribute called Domain Path (D-PATH). D-PATH gives gateway nodes loop prevention by guarding against control plane loops 🛡️. Adding it changes how BGP picks best paths for Multiprotocol BGP inter-subnet forwarding (ISF) routes in Subsequent Address Family Identifiers (SAFIs) 128 (IPVPN) and 70 (EVPN). Main character energy 👑
Status of This Memo 🌸
So this is an Internet Standards Track document. It's a product of the Internet Engineering Task Force (IETF), it represents the consensus of the IETF community, it got public review, and the Internet Engineering Steering Group (IESG) approved it for publication 💅
For more on Internet Standards, see Section 2 of RFC 7841. For the current status, any errata, and how to give feedback, head to https://www.rfc-editor.org/info/rfc10039 🦋
Copyright Notice 🎀
Copyright (c) 2026 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License. Read the fine print, bestie, it matters 🙈
Table of Contents 🧚
1. Introduction and Problem Statement
2. Conventions Used in This Document
3. Terminology and Interworking PE Components
4. Domain Path (D-PATH) Attribute
5. BGP Path Attribute Propagation Across Domains
5.1. No Propagation Mode
5.2. Uniform Propagation Mode
5.3. Aggregation of Routes and Path Attribute Propagation
6. Route Selection Process for ISF Routes
6.1. Tie-Breaking and Selection Rules
6.2. Examples
7. Composite PE Procedures
8. Gateway PE Procedures
8.1. Export Conditions
8.2. Advertisement Behavior
9. Interworking Use Cases
10. BGP Error Handling on Interworking PEs
11. Security Considerations
12. IANA Considerations
13. References
13.1. Normative References
13.2. Informative References
Acknowledgments
Authors' Addresses1. Introduction and Problem Statement ✨
EVPN is the unified BGP control plane that handles both intra- and inter-subnet forwarding for tenant networks. Now picture a tenant network spread across several domains. Some run EVPN, and others rely on the BGP VPN-IPv4/VPN-IPv6 address families for inter-subnet forwarding (ISF). Those domains need interworking procedures so the tenant still gets seamless end-to-end connectivity across all these different setups. It's giving multi-domain drama, honestly 💅
This document lays out how EVPN and other BGP address families, including VPN-IPv4 and VPN-IPv6, interwork for ISF. It also covers interconnecting domains that use EVPN, IPVPN, or a combo of both. So that's two EVPN domains, two IPVPN domains, or an EVPN domain plugged into an IPVPN domain 🌈
Now for the loop problem 👀. When redundant gateway Provider Edges (PEs) interconnect distinct domains, control plane loops can happen if routes get redistributed between domains without proper safeguards. For example, say gateway PE1 imports an IPVPN route for a given prefix and redistributes it as an EVPN IP Prefix route into the EVPN domain. If a second gateway, PE2, receives that EVPN route and re-advertises it back into the IPVPN domain, a loop can form 🙈. That's why this specification introduces the Domain Path (D-PATH) Path Attribute, which provides domain-level loop detection and avoidance.
D-PATH also alters the BGP best-path selection logic for Multiprotocol BGP routes of SAFI 128 (VPN-IPv4/IPv6) and for EVPN IP Prefix routes. So this document updates the BGP best-path selection procedures specified in [RFC4271], but only for the IPVPN and EVPN families when D-PATH is used for inter-domain connectivity 🔥
EVPN can advertise IPv4 or IPv6 prefixes through two route types:
- Route Type 2 - EVPN MAC / IP Advertisement route, as defined in [RFC9135], supporting host routes (i.e., /32 or /128).
- Route Type 5 - EVPN IP Prefix route, as defined in [RFC9136].
When interworking for ISF, the IP prefixes carried in these EVPN route types get re-originated into corresponding address families (e.g., IPVPN), and vice versa. That re-origination needs clarified procedures for route selection, loop prevention, and BGP Path Attribute handling across AFI/SAFI boundaries 🧩
This document defines the Interworking PE (Section 3), which is responsible for interconnecting different domains. An Interworking PE imports routes from one domain (along with the domain-specific encapsulation parameters), installs them in an IP Virtual Routing and Forwarding (IP-VRF) table [RFC9135], and re-originates the routes with the encapsulation attributes suitable for the adjacent domain before advertisement. That re-origination lets the solution work independently of the transport encapsulation used within each domain, and it acts as a service interworking function 🦋
These procedures keep tenant inter-subnet connectivity alive across a mix of EVPN and non-EVPN domains, while preventing routing loops and keeping protocol consistency across BGP address families 💜
As a summary, the following key procedures are specified by this document:
- A route selection algorithm that enables a PE to deterministically select the best path among candidates learned via EVPN and other ISF SAFIs.
- A new BGP Path Attribute, referred to as the D-PATH Path Attribute, which provides loop prevention capabilities and conveys domain traversal information for a given route.
- The rules governing BGP Path Attribute propagation across domains to maintain semantic consistency and enable cross-domain route processing.
- The operational procedures required for Interworking PEs that function as composite PEs, gateway PEs, or devices supporting both roles.
Put it all together and operators can deploy scalable tenant networks spanning multiple administrative or routing domains. They can use different ISF SAFIs for IP prefix dissemination while keeping forwarding deterministic and routing loops protected 🫶
2. Conventions Used in This Document 💅
Quick vibe check on the rules-speak 🎀 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.
3. Terminology and Interworking PE Components 🦋
This section summarizes the terminology related to the "Interworking PE" concept that will be used throughout the rest of the document. Let's look at the picture first 👀
+-------------------------------------------------------------+
| |
| +------------------+ Interworking PE |
| Attachment | +------------------+ |
| Circuit (AC1)| | +----------+ | MPLS/NVO/SRv6 tnl
----------------------*Bridge | | +------
MPLS/NVO | | |Table(BT1)| | +-----------+ / \ \
/SRv6 tnl +-------->| *---------* |<--> | Eth |
-------+ | | | |Eth-Tag x | |IRB1| | \ / /
/ Eth / \<-+ | | +----------+ | | | +------
| | | | | ... | | IP-VRF1 |MPLS/NVO/SRv6
\ \ /<-+ | | +----------+ | | RD2/RT2 | tnl
-------+ | | | |Bridge | | | | +------
| +-------->|Table(BT2)| |IRB2| | / \ \
| | | | *---------* |<--> | IP |
----------------------*Eth-Tag y | | +-----*-----+ \ / /
| AC2 | | +----------+ | AC3| +------
| | | MAC-VRF1 | | |
| +-+ RD1/RT1 | | |
| +------------------+ | SAFIs |
| | IPVPN +---+ |
-------------------------------------------------+ EVPN |BGP| |
| IP +---+ |
| |
+-------------------------------------------------------------+Figure 1: EVPN-IPVPN Interworking PE
Note: "tnl" refers to "tunnel".
- AC: An Attachment Circuit or logical interface associated to a given Bridge Table (BT) or IP-VRF. To determine the AC on which a packet arrived, the PE will examine the combination of a physical port and VLAN tags (where the VLAN tags can be individual VLAN tags, Q-in-Q tags, or ranges of both).
- Example: In Figure 1, AC1 is associated to BT1, AC2 to BT2, and AC3 to IP-VRF1. ✨
BT: Okay so a Bridge Table (defined in [RFC7432]) is basically how a PE brings a broadcast domain to life 🌸 If an EVI has just one broadcast domain, the MAC-VRF on each PE holds exactly one BT. If a MAC-VRF is hosting several broadcast domains, every BT gets its own distinct Ethernet Tag. Any EVPN route about a particular BT carries that BT's Ethernet Tag, so everyone knows which broadcast domain it's talking about 💜
Example: In Figure 1, MAC-VRF1 has two BTs: BT1 and BT2. Ethernet Tag x is defined in BT1 and Ethernet Tag y in BT2.
CE: A Customer Edge device 🍓
Composite Domain: A domain where more than one control plane ISF SAFI is in play, i.e., IPVPN and/or EVPN. It's made up of regular PEs and composite PEs (more on those below) 🦋
Composite PE: An Interworking PE that's connected to at least one composite domain and can advertise a given prefix to several kinds of peers using the right route types ✨ To be specific, a composite PE advertises the prefix to an IPVPN peer using an IPVPN ISF route, to an EVPN peer using an EVPN ISF route, and to a Route Reflector (RR) [RFC4456] using both IPVPN and EVPN ISF routes (assuming the same RR is used for IPVPN and EVPN). A composite PE implements the procedures defined in Section 7.
Example: Figure 2 shows PE1 as a composite PE, since PE1 has both EVPN and IPVPN enabled toward the same RR. PE1 advertises a given IP prefix (IPn/x) twice: once using EVPN and once using IPVPN. PE2 and PE3 are not composite PEs.
+---+
|PE2|
+---+
^
Interworking |EVPN
PE EVPN v
+---+ IPVPN +--+ +---+
|PE1| <----> |RR| <---> |PE3|
+---+ +--+ IPVPN +---+
CompositeFigure 2: Interworking Composite PE Example
Composite/Gateway PE: An Interworking PE doing the jobs of a composite PE and a gateway PE at the same time 👑 It's connected to two or more domains: one or more regular domains and one or more composite domains. Here's how it works:
- Re-originates an ISF route received from the regular domain into the composite domain. Within the composite domain, it performs the behavior of a composite PE.
- Re-originates an ISF route received from the composite domain into the regular domain. In the regular domain, the route is advertised using the ISF SAFI applicable to that domain.
This is super useful when a tenant network stretches across domains using different ISF SAFIs (like IPVPN and EVPN) and any-to-any tenant connectivity is needed 🫶 In those setups, keeping consistent end-to-end control plane behavior across domains is desirable when feasible.
Example: Figure 3 shows PE1 as a composite/gateway PE.
+---+
|PE2|
+---+
^
Interworking |EVPN
PE EVPN v
+---+ EVPN +---+ IPVPN +--+ +---+
|PE4| <----> |PE1| <----> |RR| <---> |PE3|
+---+ +---+ +--+ IPVPN +---+
Composite/GatewayFigure 3: Interworking Composite/Gateway PE Example
Domain: Two PEs are in the same domain if they're attached to the same tenant and the packets exchanged between them don't need a data-path IP lookup (in the tenant space) at any transit router 🧚 A gateway PE interconnects multiple DOMAIN-IDs. Domain boundaries aren't restricted to an Autonomous System or an IGP instance. The PEs in a domain may sit in the same or in different Autonomous Systems, and a single Autonomous System may also contain multiple domains.
Example 1: Figure 4 shows Tenant Systems TS1 and TS2 belonging to the same tenant, but sitting in different data centers that are connected by gateway PEs (see the "Gateway PE" definition below). These gateway PEs use IPVPN in the WAN. When TS1 sends traffic to TS2, the intermediate routers between PE1 and PE2 require a tenant IP lookup in their IP-VRFs so that the packets can be forwarded. In this example, there are three different domains. The gateway PEs connect the EVPN domains to the IPVPN domain.
GW1------------GW3
+------+ +------+
+-------------|IP-VRF| |IP-VRF|-------------+
PE1 +------+ +------+ PE2
+------+ DC1 | WAN | DC2 +------+
TS1-|IP-VRF| EVPN | IPVPN | EVPN |IP-VRF|-TS2
+------+ GW2 GW4 +---+--+
| +------+ +------+ |
+-------------|IP-VRF| |IP-VRF|-------------+
+------+ +------+
+--------------+
DOMAIN 1 DOMAIN 2 DOMAIN 3
<---------------> <------------> <---------------->Figure 4: Multiple Domain Data Center Interconnects (DCIs) Example
Example 2: Figure 5 shows a similar scenario, but PE1 and PE2 are now connected by a BGP Labeled Unicast (BGP-LU) tunnel, and they have a BGP peer relationship for EVPN. Unlike Example 1, there's no need for tenant IP lookups on the intermediate routers in order to forward packets between PE1 and PE2. So there is only one domain in the network, and PE1/PE2 belong to it 🌈
EVPN
<------------------------------------------------->
BGP-LU
<------------------------------------------------->
ASBR------------ASBR
+------+ +------+
+-------------| | | |-------------+
PE1 +------+ +--+---+ PE2
+------+ DC1 | WAN | DC2 +------+
TS1-|IP-VRF| EVPN | | EVPN |IP-VRF|-TS2
+------+ ASBR ASBR +---+--+
| +------+ +------+ |
+-------------| | | |-------------+
+------+ +------+
+--------------+
<--------------------DOMAIN-1--------------------->Figure 5: Single Domain DCI Example
Ethernet Tag: Used to represent a broadcast domain [RFC7432] 🎀
EVI: An EVPN Instance spanning the PE devices participating in that EVPN [RFC7432] 🌸
Gateway PE: An Interworking PE that connects two or more distinct domains, where each domain may be either a regular domain or a composite domain. A gateway PE may establish either Internal BGP (IBGP) [RFC4271] or External BGP (EBGP) [RFC4271] sessions with peers in the connected domains. Depending on its configuration, the gateway PE performs one of the following functions:
- Re-originates ISF routes using the same ISF SAFI between the connected domains.
- Translates an ISF route received via one ISF SAFI and re-originates it into a domain that uses a different ISF SAFI.
A gateway PE follows the procedures defined in Section 8. It interconnects multiple domains. If the gateway PE is configured to use D-PATH, each domain is identified by a DOMAIN-ID. These DOMAIN-IDs are encoded in the D-PATH and included in ISF SAFI route advertisements. The structure and behavior of the D-PATH Path Attribute are described in Section 4.
Example: Figure 6 shows PE1 as a gateway PE, since the EVPN and IPVPN SAFIs are enabled on different BGP peers and a given local IP prefix IPn/x is sent to both BGP peers for the same tenant. PE2 and PE1 are in one domain, and PE3 and PE1 are in another domain.
Interworking PE
+---+ EVPN +---+ IPVPN +---+
|PE2| <----> |PE1| <----> |PE3|
+---+ +---+ +---+
GatewayFigure 6: Interworking Gateway PE Example
Interworking PE: A PE that can advertise a given IP prefix using one or more of these route types: an EVPN ISF route (either an EVPN MAC/IP Advertisement route or an EVPN IP Prefix route) or an IPVPN ISF route ⭐ An Interworking PE keeps a single IP-VRF per tenant and zero, one, or more MAC-VRFs per tenant. Each MAC-VRF may include one or more BTs, and each BT may be associated with the tenant's IP-VRF via an Integrated Routing and Bridging (IRB) interface. There are two types of Interworking PEs:
- Composite PE
- Gateway PE
These two functions may be implemented independently on a per-tenant basis, and they may also coexist for the same tenant on a single PE 💖
Example: Figure 1 shows an Interworking PE where ISF SAFIs are enabled. IP-VRF1 and MAC-VRF1 are instantiated on the PE and together provide ISF for the tenant.
IP-VRF: An IP Virtual Routing and Forwarding table, as defined in [RFC4364] and [RFC9135]. Route Distinguisher and Route Target(s) are required properties of an IP-VRF. An IP-VRF is programmed with ISF routes 💜
IRB: An Integrated Routing and Bridging interface [RFC9135]. It's the logical interface that connects a BT to an IP-VRF, and it allows the forwarding of packets with the destination in a different subnet 🦋
ISF route: An ISF route for a given prefix, whose ISF SAFI may change as it transits different domains. IPVPN routes as in [RFC4364] and [RFC4659], EVPN IP Prefix routes as in [RFC9136], and EVPN MAC/IP Advertisement routes when they are programmed within an IP-VRF [RFC9135] are all considered ISF routes in this document 🔥
ISF SAFI: The ISF Subsequent Address Family Identifier defines a Multiprotocol BGP (MP-BGP) [RFC4760] Sub-Address Family used to advertise IP prefix reachability for ISF within a tenant network. The SAFIs used for ISF include 1 (applicable only to IPv4 and IPv6 AFIs), 128 (applicable only to IPv4 and IPv6 AFIs), and 70 (EVPN, applicable only to AFI 25). The procedures defined in this document apply only to SAFI 128 and SAFI 70. So, for the purposes of this document, the term "ISF SAFI" refers exclusively to SAFI 128 or SAFI 70. The routes for these ISF SAFIs are referred to as IPVPN and EVPN routes. Note that "ISF SAFI" doesn't define a new SAFI. It's used solely as a collective reference to SAFI 128 and SAFI 70 🌈
MAC-VRF: okay so the MAC-VRF is defined in [RFC7432] 💖 It's basically the instance of an EVI on a PE 🧚 Every MAC-VRF gets its own unique Route Distinguisher (RD) plus one or more Route Targets (RTs), and those are required attributes for it to work 🎀 These RD and RT values are typically different from the ones used by any associated IP-VRF, when that IP-VRF is linked to the MAC-VRF through a BT via an IRB interface [RFC9135] ✨
MPLS/NVO/SRv6 tunnel: this one is a tunnel that can be built on Multiprotocol Label Switching (MPLS), a Network Virtualization Overlay (NVO) technology [RFC8365], or Segment Routing over IPv6 (SRv6) [RFC9252] 🦋 Both MAC-VRFs and IP-VRFs use these tunnels. No matter what tunneling tech is underneath, the tunnel can carry either Ethernet or IP payloads 🌸 MAC-VRFs are restricted to tunnels that carry Ethernet payloads, like Ethernet NVO tunnels [RFC9136] and SRv6 tunnels with Ethernet payload, and those are usually set up through EVPN signaling 💅 IP-VRFs are more flexible. They can use tunnels carrying Ethernet payloads (signaled via EVPN) or IP payloads (signaled via EVPN or IPVPN mechanisms) 🫶 IPVPN-only PEs support IP-VRFs, but they can't send or receive traffic over tunnels carrying Ethernet payloads 😭
Example: Figure 1 shows an MPLS, NVO-based, or SRv6 tunnel carrying Ethernet frames that belong to MAC-VRF1 🍓 The PE finds the right MAC-VRF and BT using the EVPN label, which is an MPLS label, a Virtual Network Identifier (VNI), or an SRv6 Segment ID, depending on the encapsulation type ✨ Figure 1 also shows two separate MPLS/NVO/SRv6 tunnels used by IP-VRF1. One carries Ethernet frames and the other carries IP packets. So yes, IP-VRFs can run multiple tunnel types at once, depending on the payload and the signaling mechanism (EVPN or IPVPN) 👑
NVE: a Network Virtualization Edge router [RFC8365] 🌈
PE: a Provider Edge device 💜
Regular Domain: a domain where only one control plane ISF SAFI is used, which means either IPVPN or EVPN 🎀 A regular domain is made up of regular PEs (see the definition below). In Figures 4 and 5, every domain is a regular domain 🧚
Regular PE: a PE that is attached to a domain, either regular or composite, and that uses one of the control plane ISF SAFIs (IPVPN or EVPN) running in that domain 🦋
RT-2: Route Type 2, also known as a MAC/IP route, as per [RFC7432] 🌸
RT-5: Route Type 5, also known as an IP Prefix route, as per [RFC9136] 💖
4. Domain Path (D-PATH) Attribute ✨
Okay, the BGP D-PATH Path Attribute is an optional and transitive BGP Path Attribute 🦋
Just like AS_PATH, D-PATH is made of a sequence of domain segments. Each domain segment is made of <domain segment length, domain segment value>, where the domain segment value is a sequence of one or more domains, as shown in Figure 7 🧚 Each domain is written as <DOMAIN-ID:ISF_SAFI_TYPE> 💅
Octets
0 1 8 n
+---------------+----------------//--+----//-------------------+
|Domain Segment | Last Domain | Domain of Origin |
| Length | | |
+---------------+----------------//--+----//-------------------+
\__________________/
|
Octets v
0 6 7
+------------------//-----+----------------+
| DOMAIN-ID | ISF_SAFI_TYPE |
+------------------//-----+----------------+
\________________________/
|
Octets v
0 1 2 3 4 5 6
+-----------------------+-----------+
| Global | Local |
| Admin | Admin |
+-----------------------+-----------+Figure 7: D-PATH Domain Segment
Domain Segment Length: a 1-octet length field. It holds the number of domains in the segment 🎀
Last Domain: the most recently added domain 💜
Domain of Origin: the first domain added by the gateway PE that initialized the D-PATH for the ISF route. There can be multiple domains between the Last Domain and the Domain of Origin 🌈
DOMAIN-ID: a 6-octet field that represents a domain. It has a 4-octet Global Administrator sub-field and a 2-octet Local Administrator sub-field 🫶 The Global Administrator sub-field MAY be filled with a public or private Autonomous System Number (ASN), an IPv4 address, or any value 🌸 The combined Global and Local Administrator values can be anything that guarantees the DOMAIN-ID is unique (when the tenant network is connected to multiple Operators) and that helps with troubleshooting and debugging D-PATH in ISF routes ✨ A gateway PE that interconnects two domains is associated with two distinct DOMAIN-IDs, one per domain 🦋 All gateway PEs attached to the same domain MUST use the same DOMAIN-ID value to represent that domain 👑 Showing the Global and Local Administrator values as opaque unsigned integers in user interfaces and reporting (for example, the command-line interface (CLI) and YANG) is RECOMMENDED 💅
ISF_SAFI_TYPE: a 1-octet field that shows which inter-subnet forwarding SAFI type a route was received in by the gateway PE, before the gateway PE re-exports it into a different domain 🧚 This field is informational only and has no impact on loop detection or BGP path selection procedures 🍓 Encoding it is still useful, since operators can check that the intended interworking is in place and that the route went through the expected domains using the intended ISF SAFIs in each one 💖 The non-zero ISF_SAFI_TYPE values come from the IANA "SAFI Values" registry [IANA-SAFI]. These are the values allowed by this document:
+=======+==================================+
| Value | ISF_SAFI_TYPE |
+=======+==================================+
| 0 | Gateway PE local ISF route |
+-------+----------------------------------+
| 70 | EVPN (BGP EVPNs) |
+-------+----------------------------------+
| 128 | IPVPN (MPLS-labeled VPN address) |
+-------+----------------------------------+Table 1
The BGP D-PATH Path Attribute is supported on ISF routes of types IPVPN and EVPN, and it MUST NOT be advertised along with routes that are different from IPVPN and EVPN routes 🙈 By default, the BGP D-PATH Path Attribute is not advertised, and it MUST be explicitly enabled by configuration on the gateway PEs 🔥 The rest of this section lays out the procedures related to D-PATH 🫶
- D-PATH identifies the sequence of domains, each one written as a
<DOMAIN-ID:ISF_SAFI_TYPE>, that a given ISF route of type IPVPN or EVPN has passed through 🌟- This attribute list MAY contain one or more segments. Each segment's Domain Segment Length MUST be equal to or greater than one 💜
- The first entry in the list (the leftmost one) is the
<DOMAIN-ID:ISF_SAFI_TYPE>that a gateway PE is re-originating an ISF IPVPN or EVPN route from. The last entry in the list (the rightmost one) is the<DOMAIN-ID:ISF_SAFI_TYPE>from which a gateway PE received an ISF IPVPN or EVPN route that had no D-PATH Path Attribute (the Domain of Origin). Entries in the middle are domains the ISF IPVPN or EVPN route has transited 🦋 - As an example, an ISF IPVPN or EVPN route received with a D-PATH Path Attribute containing the domain segment
{length=2, <6500:2:IPVPN>,<6500:1:EVPN>}means the route was originated in EVPN domain 6500:1 and re-originated into IPVPN domain 6500:2 🎀 - To keep the number of segments in the D-PATH Path Attribute down, the local gateway PE MUST prepend its own domain as the last element of the domain segment. If prepending a new domain overflows the domain segment (that is, more than 255 domains), the local gateway PE MUST prepend a new segment and prepend its own domain to this new segment 🧚
- D-PATH is added or modified by a gateway PE when it re-originates an update to a different domain (one that runs the same or a different ISF SAFI), as long as D-PATH is configured as follows 💖
- The IP-VRF of a gateway PE that interconnects two domains is associated with two distinct DOMAIN-IDs, one per domain. These DOMAIN-IDs MUST be different. Each domain MUST be identified by a unique DOMAIN-ID. All gateway PEs attached to the same domain MUST use the same DOMAIN-ID value to represent that domain 👑
- Whenever a prefix arrives at a gateway PE in a particular ISF SAFI route, and the gateway PE needs to export that prefix to a BGP peer, the gateway PE MUST prepend a
<DOMAIN-ID:ISF_SAFI_TYPE>to the list of domains in the D-PATH of the received route. This applies as long as the gateway PE works in Uniform Propagation Mode (as explained in Section 5.2) and D-PATH is configured as described at the beginning of this section 🫶 - For instance, take an IP-VRF configured with DOMAIN-IDs 6500:1 for EVPN and 6500:2 for IPVPN. If an EVPN route for prefix P is received and P is installed in the IP-VRF, then the matching IPVPN route for P, when exported to an IPVPN peer, will include the domain identifier
<6500:1:EVPN>prepended to the existing D-PATH Path Attribute, assuming D-PATH is configured as described at the beginning of this section. In the same way, prefixes received in the IP-VRF from an IPVPN peer get exported to EVPN peers with the domain identifier<6500:2:IPVPN>appended to the D-PATH Path Attribute, again assuming D-PATH is configured 🌈 - In the example above, if the EVPN route is received without D-PATH, the gateway PE adds the D-PATH Path Attribute with one segment,
{length=1, <6500:1:EVPN>}, when it re-advertises to domain 6500:2 ✨ - Within the Domain of Origin, the update does not contain a D-PATH Path Attribute, because the update has not passed through a gateway PE yet 🧚
- For a local ISF route, meaning a configured static route or a route learned from a local Attachment Circuit, a gateway PE following this specification has three choices 🎀
- The gateway PE advertises that ISF route without a D-PATH Path Attribute into one or more of its configured domains. In that case, the D-PATH Path Attribute will be added by the other gateway PEs in each of those domains 💅
okay so the gateway PE can push that ISF route out with a D-PATH Path Attribute into one or more of its configured domains 💖 (only if D-PATH is actually configured, bestie). When it does, every copy of the ISF route gets its D-PATH Path Attribute initialized with an ISF_SAFI_TYPE of 0 and the DOMAIN-ID of the domain that ISF route is associated with ✨
- The gateway PE advertises the ISF route with a D-PATH Path Attribute (assuming D-PATH is configured 🎀) that contains a locally configured domain identifier tied to its local ISF routes, into one or more of its configured domains. In this case, every copy of the ISF route gets its D-PATH Path Attribute initialized with an
ISF_SAFI_TYPEvalue of0and theDOMAIN-IDrepresenting the local ISF domain. TheDOMAIN-IDMUST be globally unique and MAY be shared across multiple gateway PEs 🔥
Okay, so although all three options give you ways to catch control plane loops 🙈, this third option is RECOMMENDED 👑 because it tells you way more about where the route came from. Specifically, the receiving PE can recognize the route as coming from a local gateway just by looking at the combo of the DOMAIN-ID and the ISF_SAFI_TYPE value 🧚
- An ISF route of type IPVPN or EVPN received by a gateway PE that includes a D-PATH Path Attribute containing one or more
DOMAIN-IDvalues locally associated with the corresponding IP-VRF MUST be considered a looped ISF route for the purposes of re-advertisement into adjacent domains. In such cases:- The ISF route MUST be flagged as "looped".
- The route MUST NOT be re-exported to any other domain.
- The route is installed in the IP-VRF only if it is selected as the best path according to the procedures defined in Section 6.
Lowkey the loop check doesn't care about the ISF_SAFI_TYPE value attached to a DOMAIN-ID in the D-PATH Path Attribute, it's giving "irrelevant" 💅 So a route counts as looped if it has at least one DOMAIN-ID that matches any local DOMAIN-ID on the gateway PE, no matter what the ISF_SAFI_TYPE is 🌈
Example: In the scenario illustrated in Figure 4, gateway GW1 receives two ISF routes for the same prefix associated with TS1:
- An EVPN IP Prefix route with a next hop of PE1 and no D-PATH Path Attribute.
- An IPVPN route with a next hop of GW2 and a D-PATH Path Attribute containing a single segment:
{length=1, <6500:1:EVPN>}, where6500:1is assumed to be theDOMAIN-IDfor domain 1, which is local to GW1.
Okay the IPVPN route hits GW1, GW1 sees 6500:1 is one of its own local DOMAIN-IDs, and boom, it gets flagged as "looped" 😭 So GW1 doesn't install it in the tenant IP-VRF, because the route selection process prefers the EVPN IP Prefix route (its D-PATH Path Attribute is shorter, as specified in Section 6) 💜 Loop detection still kicks in even if the ISF_SAFI_TYPE value in that D-PATH Path Attribute is unknown to GW1 or doesn't match any SAFI defined in this specification 🙈
- A
DOMAIN-IDconfigured on a gateway PE MAY be either assigned at the domain interconnection level or scoped individually per tenant IP-VRF.- When the
DOMAIN-IDis allocated at the peering domain level, it SHALL apply to all tenant IP-VRFs associated with that domain. - When the
DOMAIN-IDis allocated for a specific tenant IP-VRF, the processing of received D-PATH Path Attributes and their subsequent propagation SHALL be performed in the context of that IP-VRF'sDOMAIN-ID.
- When the
A per-tenant IP-VRF DOMAIN-ID assignment is especially cute for route leaking 🦋 Say PE1 and PE2 are each tied to different tenant IP-VRFs, called IP-VRF-1 and IP-VRF-2. If PE1 advertises ISF SAFI routes for IP-VRF-1 with a DOMAIN-ID of 6500:1, and PE2 receives those routes and leaks them from IP-VRF-1 into IP-VRF-2, then when PE2 re-advertises them back to PE1 in the context of IP-VRF-2, PE1 won't treat them as looped. That's because PE1 processes the route in the context of IP-VRF-2, where DOMAIN-ID 6500:1 isn't locally configured 💅
- The number of domains encoded in the D-PATH Path Attribute reflects the number of gateway PEs that the corresponding ISF route update has traversed. If a transit gateway PE performs route leaking between two local tenant IP-VRFs, it MAY prepend a domain to the D-PATH Path Attribute with an
ISF_SAFI_TYPEvalue of0when exporting the leaked route into an ISF SAFI. In such cases, the total number of domain entries in the D-PATH Path Attribute reflects not only the number of gateway PEs through which the ISF route has been re-originated but also the number of tenant IP-VRF instances across those gateway PEs 🌸 - Okay, here are the error-handling rules for the D-PATH Path Attribute 🫶
- A received D-PATH Path Attribute MUST be considered malformed if it contains a malformed domain segment or if the total length of the D-PATH Path Attribute is less than 8 octets.
- A domain segment MUST be considered malformed under any of the following conditions:
- The length of the domain segment is zero.
- The length of the domain segment exceeds the remaining length of the enclosing D-PATH Path Attribute.
- Less than 8 octets remain after the last successfully parsed domain segment.
- Each domain segment consists of a 1-octet length field indicating the number of domains in the segment, with each domain encoded in 7 octets. If the total length of the domain segment (i.e.,
1 + 7 x number of domains) exceeds the remaining length of the D-PATH Path Attribute, the domain segment is considered malformed.
- A BGP speaker receiving an UPDATE message containing a malformed D-PATH Path Attribute SHALL apply the "treat-as-withdraw" procedure, as specified in [RFC7606].
- Domains within the D-PATH Path Attribute that contain unrecognized
ISF_SAFI_TYPEvalues MAY be accepted and MUST NOT be considered an error. - The D-PATH Path Attribute MUST NOT appear more than once in the Path Attributes of a given BGP UPDATE message. If multiple instances of the D-PATH Path Attribute are present, all instances other than the first MUST be discarded, and the UPDATE message MUST continue to be processed. This behavior follows [RFC7606], including the associated logging considerations.
- The D-PATH Path Attribute MAY be included only in UPDATE messages that carry IPVPN or EVPN routes. It MUST NOT be included with any other AFI/SAFI combinations. If a D-PATH Path Attribute is received in an UPDATE message associated with an unsupported AFI/SAFI, the "treat-as-withdraw" procedure MUST be applied in accordance with [RFC7606].
5. BGP Path Attribute Propagation Across Domains 🎀
A gateway PE, depending on its local configuration, is required to re-originate an ISF route between two domains that utilize either the same or different ISF SAFIs. This requires defining how a gateway PE handles the BGP Path Attributes associated with the ISF route during such re-origination.
This section specifies the BGP Path Attribute propagation behaviors that a gateway PE MAY apply when it receives an ISF route with ISF SAFI x, installs the route into the relevant IP-VRF, and subsequently re-advertises the route as an ISF route using ISF SAFI y. The values of ISF SAFI x and SAFI y MAY be the same or different ✨
5.1. No Propagation Mode 💅
The No Propagation Mode is the default operational mode for Gateway PEs when re-exporting ISF routes from one domain into another. In this mode, the gateway PE re-initializes the BGP Path Attributes during the re-origination of an ISF route, treating it in the same manner as a directly connected or locally originated IP prefix.
This mode is suitable for deployment scenarios where the source domain -- for example, an EVPN domain -- is "abstracted" and treated as a virtual Customer Edge (CE), and where remote IPVPN or IP-based PEs do not rely on the BGP Path Attributes of the source EVPN domain for best-path selection or the application of routing policy 🧚
It is important to note that, in No Propagation Mode, the D-PATH Path Attribute is not propagated. As a result, redundant gateway PEs may be susceptible to routing loops 💀 While such loops may be mitigated using routing policies or additional attributes, such as the Route Origin extended community [RFC4360], this approach does not guarantee detection or prevention of all potential loop scenarios.
5.2. Uniform Propagation Mode 🫧
okay so Uniform Propagation Mode is when the gateway PE keeps and copies a consistent set of the commonly used BGP Path Attributes when it re-originates an ISF route between domains 💅. This mode is typically used in deployments where IP prefixes get seamlessly distributed using EVPN and/or IPVPN SAFIs ✨. The spec allows a limited set of common attributes to get propagated, but it's giving "no indiscriminate copying and re-advertising" 🙅, mainly for security reasons 🔐. security is literally the whole vibe here.
Here's the normative stuff a gateway PE running in Uniform Propagation Mode MUST follow, no exceptions bestie 🫶:
- So when an ISF route shows up, and there are no validation errors and local policy says yes 🙌, the gateway PE imports it into the associated IP-VRF and keeps the original BGP Path Attributes. When it re-advertises the route into a different domain, it SHOULD, by default, only propagate this set of attributes. Everything else SHOULD NOT be propagated unless local import/export policies explicitly permit it:
AS_PATHD-PATH(only when advertising IPVPN or EVPN routes)- IBGP-only attributes (when advertising to IBGP peers):
LOCAL_PREF,ORIGINATOR_ID, andCLUSTER_ID MULTI_EXIT_DISC(MED)- Accumulated IGP (AIGP) [RFC7311]
COMMUNITY,EXTENDED_COMMUNITY, andLARGE_COMMUNITY, except where explicitly excluded in Item 4 below.
- When re-advertising an ISF route to an IBGP peer, the gateway PE SHOULD keep the
AS_PATHof the original ISF route exactly as it is, no edits. When re-advertising to an EBGP peer, the gateway PE SHOULD prepend the IP-VRF's ASN to the preservedAS_PATH. 🧁 - When re-originating an ISF route to IBGP peers, the gateway PE SHOULD retain IBGP-only attributes (e.g.,
LOCAL_PREF,ORIGINATOR_ID, andCLUSTER_ID) from the original ISF route. As the route gets re-originated, the gateway PE is not required to perform the RR function described in [RFC4456]. - As stated in Item 1, the gateway PE SHOULD preserve the
COMMUNITY,EXTENDED_COMMUNITY, andLARGE_COMMUNITYattributes from the original ISF route. However, the following exceptions apply:- a. BGP Encapsulation Extended Communities, as defined in [RFC9012], SHOULD NOT be propagated.
- b. Route Target Extended Communities SHOULD NOT be propagated and SHOULD be re-initialized when re-advertising the ISF route into a different domain. The re-initialized Route Target value MAY match the value used in the original route.
- c. All EVPN-specific Extended Communities SHOULD NOT be propagated.
- d. Gateway PEs SHOULD support import/export policies capable of matching
COMMUNITY,EXTENDED_COMMUNITY, andLARGE_COMMUNITYvalues, so they can permit or deny propagation between domains when the default propagation behavior needs to be overridden.
The gateway PE SHOULD NOT copy the above Extended Community types in "a", "b", and "c" from the original ISF route into the re-advertised ISF route 🚫. Certain Extended Communities can influence how the receiving PE processes the route, so propagating them into another domain could lead to unintended behavior 😬. For example, if the BGP Encapsulation Extended Community is propagated into a destination domain that uses a different encapsulation, a receiving PE in that domain might interpret the label field of the EVPN ISF route according to an encapsulation context that doesn't apply locally [RFC8365]. That could result in the route being discarded or programmed with incorrect encapsulation parameters 💀.
- For a given ISF route, only the BGP Path Attributes associated with the best path MAY be propagated when re-advertising the route into a different domain. If multiple paths are received for the same prefix within the same ISF SAFI, the standard BGP best-path selection procedure MUST be applied to determine the active path and its associated attributes. Even when Equal-Cost Multipath (ECMP) is enabled for the IP-VRF, only the Path Attributes of the selected best path SHOULD be propagated 👑.
5.3. Aggregation of Routes and Path Attribute Propagation ✨
instead of re-originating a huge pile of (host) ISF routes between domains, a gateway PE that receives multiple ISF routes from a domain MAY choose to re-originate a single ISF aggregate route into a different domain 🎀. In this document, aggregation means combining the characteristics of multiple ISF routes so that one aggregate ISF route can be re-originated to the destination domain. Aggregating multiple ISF routes of one ISF SAFI into an aggregate ISF route is only done by a gateway PE.
Aggregation on gateway PEs may use either the No Propagation Mode or the Uniform Propagation Mode explained in Sections 5.1 and 5.2, respectively.
When using Uniform Propagation Mode, Path Attributes of the same type code MAY be aggregated according to the following rules:
AS_PATHis aggregated based on the rules in [RFC4271]. Gateway PEs aren't expected to receiveAS_PATHattributes with path segments of type AS_SET [RFC9774]. Routes received withAS_PATHattributes that include AS_SET path segments MUST NOT be aggregated 🙅.- An ISF aggregate route SHOULD NOT be advertised unless all the contributing ISF routes have the same
D-PATHDOMAIN-ID members, regardless of their order. If at least one contributing ISF route has a differentD-PATHDOMAIN-ID, the gateway PE SHOULD advertise each contributing ISF route with its ownD-PATH(prepended with the gateway's domain). An implementation MAY, by local policy, override this behavior and advertise an ISF aggregate route without theD-PATHPath Attribute when the contributing routes don't share identicalD-PATHDOMAIN-ID members. In such cases, redundant gateway PEs SHOULD apply a consistent policy so they don't advertise aggregate routes with inconsistentD-PATHusage into the destination domain 🧐. - The
Community,Extended Community, andLarge Communityattributes of an aggregated ISF route SHOULD include the union of the corresponding attributes from all the constituent ISF routes that were aggregated. The exceptions are the Extended Community types explicitly excluded from propagation as specified in Section 5.2, or those where the applicable specifications define different handling 🌈. - For other attributes, the rules described in [RFC4271] or the attribute-applicable specifications are followed.
If the conditions for route aggregation, as specified above, are satisfied, operators SHOULD consider enabling aggregation in environments with large-scale tenant networks where a significant number of host routes are present. This is especially useful for deployments like large-scale data centers 🏢.
6. Route Selection Process for ISF Routes ✨
A PE router may receive the same IP prefix via ISF routes with different ISF SAFIs, and from either the same or different BGP peers 🧩. On top of that, the same IP prefix (e.g., a host route) may be received in both an EVPN MAC/IP Advertisement route and an EVPN IP Prefix route. For consistent, deterministic forwarding behavior, a route selection procedure across all ISF SAFIs is required 💜.
The objectives of this route selection process are as follows:
- To ensure that all composite and gateway PEs have a consistent and deterministic view of the preferred path to reach a given IP prefix.
- To enable meaningful comparison of routes advertised in EVPN and non-EVPN ISF SAFIs based on commonly used path attributes.
- To support ECMP forwarding across EVPN and non-EVPN ISF SAFI routes, where applicable.
For a given prefix received via one or more non-EVPN ISF routes, the standard BGP best-path selection procedure, as defined in [RFC4271], is applied to determine the "non-EVPN best paths". Similarly, for a given prefix received via one or more EVPN ISF routes, the same procedure is applied to determine the "EVPN best paths" 🙌.
When both EVPN and non-EVPN ISF routes are present for the same prefix within a single IP-VRF, the PE MUST perform a tie-breaking selection procedure on the union of these best-path sets. The process treats all candidate ISF routes as equally preferable at first, then iteratively removes routes until a single best path (or a valid ECMP set) remains 🔍.
6.1. Tie-Breaking and Selection Rules 💅
The selection procedure MUST follow the standard route selection rules defined in [RFC4271], with the following additional rules and exceptions applied in the specified order:
- Immediately after applying the Local Preference comparison step from [RFC4271], the PE MUST remove from consideration any routes that do not have the shortest
D-PATHPath Attribute. Routes with noD-PATHPath Attribute are considered to have aD-PATHlength of zero. This rule MUST NOT be applied to ISF routes that are not imported into an IP-VRF. - After applying Rule 1, the standard selection steps in [RFC4271] MUST continue in order.
- If, after the previous steps, one or more candidate routes remain and at least one of them is an EVPN MAC/IP Advertisement route (EVPN Route Type 2), then all EVPN IP Prefix routes (EVPN Route Type 5) MUST be removed from consideration.
- If ECMP is enabled by policy and the remaining candidate routes after Steps 1 through 3 include both EVPN and non-EVPN paths, then both paths MUST be retained. If ECMP is not enabled and such a case arises, the EVPN path MUST be selected, and the non-EVPN path MUST be removed from consideration.
This procedure extends the standard BGP best-path selection behavior from [RFC4271] for IPVPN and EVPN IP Prefix routes, by adding D-PATH-based tie-breaking that prefers routes traversing the fewest Gateway PEs or domains 🛣️. These rules MUST NOT be applied to routes received under AFI/SAFI combinations other than IPVPN or EVPN. Routes that are different from IPVPN or EVPN get treat-as-withdraw procedures if they are received with a D-PATH Path Attribute, as described in Section 4 ⚠️.
6.2. Examples 🎀
Example 1: PE1 receives three candidate routes for prefix IP1/32, all eligible for import into IP-VRF-1:
{SAFI=EVPN, RT-2, Local-Pref=100, AS_PATH=(65536,65537)}
{SAFI=EVPN, RT-5, Local-Pref=100, AS_PATH=(65536,65537)}
{SAFI=128, Local-Pref=100, AS_PATH=(65536,65537)}Selected route:
{SAFI=EVPN, RT-2, Local-Pref=100, AS_PATH=(65536,65537)}This outcome is due to Step 3 above, which gives preference to Route Type 2 when both Type 2 and Type 5 EVPN routes exist 💖.
Example 2: PE1 receives two candidate routes for prefix IP2/24, both eligible for import into IP-VRF-1:
{SAFI=EVPN, RT-5, D-PATH=(6500:3:IPVPN), AS_PATH=(65536,65537), MED=10}
{SAFI=128, D-PATH=(6500:1:EVPN,6500:2:IPVPN), AS_PATH=(65537), MED=200}Selected route:
{SAFI=EVPN, RT-5, D-PATH=(6500:3:IPVPN), AS_PATH=(65536,65537), MED=10}This result is due to Step 1 above, which prefers the route with the shortest D-PATH 🧚.
7. Composite PE Procedures ✨
Okay so, remember Section 3? Composite PEs are the ones that show up in tenant networks where EVPN and IPVPN are both running to provide ISF inside the same composite domain. Main character energy 👑
Figure 8 shows a composite domain where PE1/PE2/PE4 are composite PEs (they support EVPN and IPVPN ISF SAFIs on their peering to the RR), and PE3 is a regular IPVPN PE 🦋
+-----------------------------------+
| |
| MPLS/SRv6 IPVPN PE3
| Network +----------+ IP3/24
| IPVPN |+------+ | +---+
| +----->||IP-VRF|------|CE3|
Composite PE1 | |+------+ | +---+
+---------------+ | +----------+
| +------+ | EVPN v |
| |IP-VRF| | IPVPN +--+ |
| +----| | | <------> |RR| |
+---+ | | +------+ | +--+ Composite PE4
|CE2|----|MAC-VRF| | ^ ^ +---------+ IP4/24
+---+ | +-------+ | EVPN | | EVPN |+------+ | +---+
+---|-----------+ IPVPN | | IPVPN ||IP-VRF|-----|CE4|
| | +----+ +-------->|+------+ | +---+
IP1/24 | | v +---------+
+---+ | | +---------------+ |
|CE1|--+ +----| +------+ +--------------+
+---+ | |IP-VRF| |
| | +----| | |
| | | +------+ |
+--------------|MAC-VRF| |
| +-------+ |
+---------------+
Composite PE2
Figure 8: Composite PE ExampleSo in a composite domain that has both composite and regular PEs, these behaviors apply 🫶
Prefix Advertisement Consistency: Composite PEs MUST advertise the same IP prefixes using each ISF SAFI to the RR, assuming the same RR is used for both ISF SAFIs. For example, as shown in Figure 8, the prefix IP1/24 is advertised by PE1 and PE2 to the RR in two separate Network Layer Reachability Information (NLRI) entries: one for AFI/SAFI 1/128 (IPVPN) and another for EVPN. If both routes are advertised with the same set of BGP Path Attributes, the receiving composite PE will select the EVPN route over the IPVPN route, following the route selection procedures defined in Section 6. Prioritizing the advertisement of the EVPN route before the IPVPN route is an OPTIONAL optimization. This ensures that the EVPN route is more likely to be selected first, avoiding unnecessary replacement if the IPVPN route arrives later. 💅
Route Reflector SAFI-Specific Forwarding Behavior: The RR does not forward EVPN routes to peers for which the EVPN SAFI is not enabled, and likewise does not forward IPVPN routes to peers lacking IPVPN SAFI support. For instance, in Figure 8, the RR does not forward EVPN routes to PE3 if the EVPN SAFI is not enabled on its BGP session with PE3. However, the IPVPN routes are forwarded to all PEs since they all have IPVPN SAFI enabled. 🙈
IPVPN PE Route Processing: Regular IPVPN PEs process and import IPVPN routes as specified in [RFC4364] and [RFC9252]. For example, PE3 receives only the IPVPN route for prefix IP1/24 and resolves the BGP next hop to an MPLS/SRv6 tunnel (with IP payload) toward PE1 and/or PE2. 🌸
Composite PE Route Selection: Composite PEs MUST perform route selection for prefixes received via multiple ISF SAFIs, applying the procedures described in Section 6:
- For example, PE4 receives prefix IP1/24 via both an EVPN route and a non-EVPN ISF route (e.g., an IPVPN route). Route selection is performed as specified in Section 6.
- If the EVPN route is selected, PE4 resolves the BGP next hop to a tunnel (which may carry either Ethernet or IP payloads) to PE1 and/or PE2. As described in Section 3, the tunnel type used between EVPN PEs depends on the supported model per [RFC9136].
- Other composite PEs (e.g., PE1 and PE2) receiving the same prefix via both EVPN and IPVPN SAFIs must also apply the route selection process defined in Section 6.
Forwarding Behavior Based on Selected Route: Once a route has been selected for a given IP prefix, packet forwarding MUST follow the forwarding rules associated with the AFI/SAFI of the selected route. 🧚
Applicability of EVPN Forwarding Enhancements: In composite domains such as the one depicted in Figure 8, the advanced forwarding features provided by EVPN are available only to composite and EVPN-capable PEs that select an EVPN IP Prefix route as the best path. These enhancements are not available to IPVPN-only PEs. For example, if PE1 advertises IP1/24 using both EVPN and IPVPN routes, and the EVPN route is selected as the best path, only composite PEs such as PE2 and PE4 can leverage EVPN-specific recursive resolution and forwarding mechanisms [RFC9136]. IPVPN PEs, such as PE3, cannot utilize these capabilities. Consequently, the benefits of EVPN-based indirection and route resolution in large-scale deployments may not be available uniformly across all PEs in the network. 😭
8. Gateway PE Procedures
Okay, so as defined in Section 3, a gateway PE is an Interworking PE that connects two or more domains and facilitates the re-origination of ISF routes between those domains. Typical examples include data center gateway devices that interconnect domains using different ISF SAFIs, such as EVPN and IPVPN, for the same tenant network. 🍓
The gateway PE procedures in this document define the mechanisms required to support ISF route interconnection across such domains. These procedures stretch the idea of a gateway PE beyond what Section 3 covers, which is Layer 2 interconnection. Here we get an analogous interconnection model for ISF route exchange at Layer 3 ✨
The procedures in this section apply to both of the following scenarios:
- Interconnection between domains utilizing different ISF SAFIs (e.g., EVPN to IPVPN).
- Interconnection between domains utilizing the same ISF SAFI (e.g., EVPN to EVPN).
Figure 9 gives an illustrative example of this model, where PE1 and PE2 (as well as PE3 and PE4) operate as gateway PEs interconnecting different domains associated with the same tenant 🌈
<----EVPN----> <----------IPVPN---------> <----EVPN---->
6500:1:EVPN 6500:2:IPVPN 6500:3:EVPN
<DOMAIN-ID:ISF_SAFI_TYPE>
+-----------------------+
Gateway PE1 Gateway PE3
+----------+ +----------+
+-----------|+------+ | MPLS tnls |+------+ |------------+
| ||IP-VRF| | SRv6 ||IP-VRF| | |
PE5 |+------+ | |+------+ | PE6
+------+ +----------+ +----------+ +------+
|IP-VRF| NVO tnls | | | | NVO tnls |IP-VRF|
| | | | | | | |
+------+ +----------+ +----------+ +------+
IP1/24--> |+------+ | |+------+ | |
| ||IP-VRF| | ||IP-VRF| | |
+-----------|+------+ | |+------+ |------------+
+----------+ +----------+
Gateway PE2 +------+ Gateway PE4
+-------|IP-VRF|---------+
| |
+------+
PE7
Figure 9: Gateway PE ExampleNote: "tnls" refers to "tunnels".
A gateway PE that is enabled for two ISF SAFIs on the same IP-VRF, referred to here as SAFI x and SAFI y, MUST follow the procedures described below for re-originating routes between domains. 💖
8.1. Export Conditions
- A gateway PE that imports an ISF SAFI x route for prefix P into an IP-VRF MUST export P using ISF SAFI y if all of the following conditions are met: a. The route for P is installed in the IP-VRF, indicating that the SAFI x route is well-formed, valid, and selected as the best route. b. The PE has an active BGP session with a peer supporting SAFI y, enabled for the same IP-VRF. c. Export policy permits the advertisement of the route. d. SAFI x and SAFI y are valid ISF SAFIs as defined in Section 3. SAFI x and SAFI y MAY be the same.
Example: In Figure 9, Gateway PEs PE1 and PE2 receive an EVPN IP Prefix route for prefix IP1/24, install the route in their respective IP-VRFs, and re-advertise it using IPVPN. 🦋
- A gateway PE that receives an ISF SAFI x route for prefix P into an IP-VRF MUST NOT export P using SAFI y under any of the following conditions: a. The SAFI x route is not well-formed or valid. Criteria for route validity are defined in the corresponding ISF SAFI specification. For example, an EVPN IP Prefix route that contains both a non-zero Ethernet Segment Identifier (ESI) and a Gateway IP address is invalid, as specified in [RFC9136], Section 3.2. b. The D-PATH Path Attribute of the SAFI x route includes one or more DOMAIN-ID values locally configured on the gateway PE for the associated IP-VRF. In this case, the route is considered a looped ISF route, as described in Section 4, and MUST NOT be exported using SAFI y. 🚨
8.2. Advertisement Behavior ✨
Okay so once the export conditions from 8.1 are satisfied, the gateway PE MUST advertise prefix P using ISF SAFI y, and it follows these procedures 💅
- bestie, if Uniform Propagation Mode (see Section 5.2) is enabled, the gateway PE MUST follow the procedures defined in Section 5.2, and the gateway PE MUST include the D-PATH Path Attribute when SAFI y is either IPVPN or EVPN. This enables loop detection at downstream gateway PEs 🔁
When re-originating an ISF route, the gateway PE MUST prepend a <DOMAIN-ID:ISF_SAFI_TYPE> element to the received D-PATH Path Attribute. The DOMAIN-ID reflects the domain from which the route was received, and the ISF_SAFI_TYPE reflects the SAFI of the received route 🫶
If the received route does not include a D-PATH Path Attribute, the gateway PE MUST create and attach a new D-PATH Path Attribute containing a single segment: the <DOMAIN-ID:ISF_SAFI_TYPE> corresponding to the received route ✨
Example: In Figure 9, gateway PEs PE1 and PE2 receive an EVPN IP Prefix route from PE5 that does not include a D-PATH Path Attribute. PE1 and PE2 add domain <6500:1:EVPN> to form the new D-PATH. Gateway PEs PE3 and PE4, upon re-advertising the route, prepend <6500:2:IPVPN>, resulting in PE6 receiving the route with D-PATH {<6500:2:IPVPN>, <6500:1:EVPN>}. This information is then used by PE6 in BGP path selection 🦋
- The gateway PE uses the RD of the IP-VRF when re-advertising prefix P via ISF SAFI y 🎀
- The encapsulation-specific context (e.g., label) allocation is a local matter. The gateway PE MAY use per-VRF, per-prefix, or other label allocation models 🌸
- The gateway PE MUST support the use of distinct RT sets per domain on the same IP-VRF. If multiple domains associated with a tenant use different RT sets, the gateway PE MUST be capable of importing and exporting routes according to each domain's RT configuration 👑
- Although Figure 9 illustrates a scenario with only two domains per gateway PE, gateway PEs may interconnect more than two domains 🌈
- There is no restriction on the number of gateway PEs that a given prefix P may traverse before reaching its destination 🧚
Informative Note: If prefix P is originated in an EVPN domain and subsequently traverses one or more non-EVPN ISF SAFI domains, it will lose EVPN-specific attributes used for advanced EVPN procedures. For example, if PE1 advertises prefix IP1/24 along with a non-zero ESI (for recursive resolution to that ESI), the ESI value will be reset to zero by the time the route reaches PE6, as it passed through an ISF SAFI domain that is not EVPN-capable. Consequently, certain EVPN-specific functionalities may not be preserved end to end. 😭
9. Interworking Use Cases 🌈
Okay so, network deployments with Interworking PEs may line up with the scenarios in Sections 7 and 8, but sometimes you need gateway PE and composite PE functionality together. Figure 10 shows a setup where gateway PEs also act as composite PEs. Here the devices don't just re-originate ISF routes between domains (EVPN and IPVPN SAFIs, or multiple EVPN domains), they also have to interoperate with IPVPN-only PEs inside domains that mix composite and IPVPN-only PEs. Main character energy 👑
+-----------------------------------+
| |
| MPLS IPVPN PE3
| Network +---------+
| IPVPN |+------+ |
| +----->||IP-VRF|---TS3
(GW+composite) PE1 | |+------+ |
+---------------+ | +---------+
| +------+ | EVPN v |
| |IP+VRF| | IPVPN +--+ |
| +----| | | <------>|RR| |
+--------| | +------+ | +--+ Composite PE4
| | |MAC+VRF| | ^ ^ +---------+
| | +-------+ | EVPN | | EVPN |+------+ |
+----+ +---------------+ IPVPN | | IPVPN ||IP-VRF|---TS4
TS1-|NVE1| | +----+ +------->|+------+ |
+----+ | v +---------+
| EVPN DC | +---------------+ |
| NVO tnls +----| +------+ |-------------+
| | |IP+VRF| |
| | +----| | |
| | | +------+ |
| +----+ | |MAC+VRF| |
+-----|NVE2|---------| +-------+ |
+----+ +---------------+
| (GW+composite) PE2
TS2
Figure 10: Gateway and Composite Combined Functions ExampleNote: "tnls" refers to "tunnels" 🍓
In Figure 10, PE1 and PE2 follow the procedures defined in Sections 7 and 8. Unlike the scenario described in Section 8, PE1 and PE2 are additionally required to re-originate ISF routes between EVPN domains (i.e., EVPN-to-EVPN), in addition to EVPN-to-IPVPN re-origination. It is important to note that PE1 and PE2 will receive the IP prefix associated with TS4 via both IPVPN and EVPN IP Prefix routes. When re-advertising the selected route to NVE1 and NVE2, PE1 and PE2 MUST apply the D-PATH handling rules and related attribute processing as described in Section 6 ("Route Selection Process for ISF Routes") 💅
10. BGP Error Handling on Interworking PEs 🚨
BGP speakers following this specification MUST adhere to the following error-handling procedures when processing ISF routes 🫶
- Any BGP UPDATE message for an ISF route that includes a D-PATH Path Attribute MUST be handled in accordance with the error-handling rules defined in Section 4 of this document.
- All received BGP UPDATE messages for ISF routes MUST conform to the general error-handling procedures specified in [RFC7606].
- This specification introduces no new error-handling behaviors for BGP UPDATE messages that contain NLRI and BGP Path Attributes defined in other specifications. Implementations SHOULD apply the relevant error-handling rules specified for each supported route type.
If a gateway PE is configured to propagate BGP Path Attributes for ISF routes between domains, the procedures specified in Section 5.2 are intended to ensure that receiving BGP speakers do not encounter UPDATE messages containing well-formed but semantically inappropriate BGP Path Attributes. However, if a gateway PE incorrectly propagates such attributes in violation of the procedures in Section 5.2, receiving PEs MUST apply the error-handling rules defined in the applicable specifications for the relevant route type and attribute 🙈
The following are examples of such scenarios and their handling:
- If a gateway PE erroneously propagates the BGP Encapsulation Extended Community or the equivalent Encapsulation TLV in the Tunnel Encapsulation attribute [RFC9012] from one EVPN domain to another, the receiving PE MAY receive two encapsulation indications with different values. In such a case, the PE MUST follow the procedures in [RFC8365], which permit signaling multiple encapsulation types. As specified in [RFC9012], encapsulations carried via the Tunnel Encapsulation attribute MUST be treated as equivalent to those conveyed via the Encapsulation Extended Community 🧚
- If a gateway PE propagates an EVPN Extended Community from an EVPN domain into an IPVPN domain, the receiving IPVPN PE MUST ignore such communities, as their semantics are not applicable to the IPVPN SAFI 💀
- If a gateway PE erroneously propagates a BGP Prefix-SID attribute containing SRv6 Service TLVs [RFC9252] for an ISF route between domains, and the receiving PE receives multiple SRv6 TLV instances, it MUST apply the procedures specified in [RFC9252] for resolving multiple TLVs 🔥
11. Security Considerations 🔐
Okay bestie, this section is SERIOUS 😭 and it deserves its full dramatic energy. The security considerations in [RFC9136] and [RFC8365] for ISF EVPN routes and [RFC4364] for ISF IPVPN routes still apply here. On top of that, the security considerations sections in [RFC9252] and [RFC7606] are relevant, and so is the entire text of [RFC4272]. Main character homework, fr fr 📖
Next up is the D-PATH Path Attribute (Section 4) 🦋. It's a control plane loop prevention mechanism for when ISF IPVPN and EVPN routes get re-originated across multiple domains through gateway PEs. When it's configured and supported correctly, D-PATH helps prevent both control plane and data plane loops ✨. BUT (and this is a big but 💀) wrong DOMAIN-ID values or inconsistent D-PATH support across gateway PEs can cause false-positive loop detection, traffic discarding, or suboptimal and inconsistent routing behavior. Plus, D-PATH is a transitive BGP attribute, so a malicious actor could try to inject incorrect domain information that spreads across multiple administrative boundaries 😱
To keep things safe, D-PATH use is explicitly restricted to IPVPN and EVPN routes within "walled garden" Virtual Private Networks, as specified in Section 4 🌸. A PE that conforms to this specification MUST remove the D-PATH Path Attribute before advertising a prefix to a CE router in a SAFI 1 (NLRI used for unicast forwarding) route. Now here's the scary part 🙈. If a non-upgraded PE that doesn't support D-PATH gets one of these routes and is connected to a CE with Internet access, it may erroneously propagate the D-PATH Path Attribute in a SAFI 1 UPDATE to the CE. If the CE keeps passing the route along, the D-PATH Path Attribute could inadvertently escape into the public Internet 🚨
Good news though 🫶. The presence of the D-PATH Path Attribute in SAFI 1 routes MUST NOT impact BGP best-path selection for those routes, so it cannot introduce routing loops or instability in the Internet. Also, BGP speakers beyond the "walled garden" that support D-PATH and receive the attribute in SAFI 1 routes MUST apply the "treat-as-withdraw" behavior, as described in Section 4 and consistent with [RFC7606] 🧹
As an extra safeguard, implementations SHOULD enforce local policy on upgraded PEs to discard any ISF EVPN or IPVPN routes received from non-upgraded peers if those routes include a D-PATH Path Attribute, so nothing gets propagated by accident 🛡️ The mechanism an implementation uses to figure out that routes came from non-upgraded peers is outside the scope of this document, so that's a problem for a different day 💅
Now for the attribute drama 👀. Section 5.2 introduces Uniform Propagation Mode, which lets gateway PEs propagate a consistent set of BGP Path Attributes across domain boundaries. This gives you great operational visibility because attributes stay preserved end to end along the route path ✨. However, it also opens the door for an attacker to inject attributes that are malformed or semantically inappropriate, but still syntactically correct, and those could influence BGP path selection in remote domains 😤
To mitigate that risk, an operator MAY choose to deploy No Propagation Mode (Section 5.1), where BGP Path Attributes get re-initialized upon domain transition. This limits attribute-based attack vectors, but it also takes away the ability of downstream PEs to inspect the original set of BGP Path Attributes as the route originator intended 🥺
Operators SHOULD carefully weigh the trade-offs between visibility and control when picking a propagation mode, and make sure policies are in place to validate attribute contents at domain boundaries 💖
12. IANA Considerations 🧾
This document defines a new BGP Path Attribute known as the BGP D-PATH Path Attribute 🦋
IANA has assigned a new attribute code type from the "BGP Path Attributes" registry in the "Border Gateway Protocol (BGP) Parameters" registry group, like so:
+=======+==========================+===========+
| Value | Code | Reference |
+=======+==========================+===========+
| 36 | BGP Domain Path (D-PATH) | RFC 10039 |
+-------+--------------------------+-----------+
Table 213. References 📚
13.1. Normative References
These are the must-reads, the ones you literally can't skip 🤓
[RFC7432] Sajassi, A., Ed., Aggarwal, R., Bitar, N., Isaac, A.,
Uttaro, J., Drake, J., and W. Henderickx, "BGP MPLS-Based
Ethernet VPN", RFC 7432, DOI 10.17487/RFC7432, February
2015, <https://www.rfc-editor.org/info/rfc7432>.
[RFC8365] Sajassi, A., Ed., Drake, J., Ed., Bitar, N., Shekhar, R.,
Uttaro, J., and W. Henderickx, "A Network Virtualization
Overlay Solution Using Ethernet VPN (EVPN)", RFC 8365,
DOI 10.17487/RFC8365, March 2018,
<https://www.rfc-editor.org/info/rfc8365>.
[RFC4271] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A
Border Gateway Protocol 4 (BGP-4)", RFC 4271,
DOI 10.17487/RFC4271, January 2006,
<https://www.rfc-editor.org/info/rfc4271>.
[RFC4364] Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private
Networks (VPNs)", RFC 4364, DOI 10.17487/RFC4364, February
2006, <https://www.rfc-editor.org/info/rfc4364>.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[RFC7606] Chen, E., Ed., Scudder, J., Ed., Mohapatra, P., and K.
Patel, "Revised Error Handling for BGP UPDATE Messages",
RFC 7606, DOI 10.17487/RFC7606, August 2015,
<https://www.rfc-editor.org/info/rfc7606>.
[RFC4760] Bates, T., Chandra, R., Katz, D., and Y. Rekhter,
"Multiprotocol Extensions for BGP-4", RFC 4760,
DOI 10.17487/RFC4760, January 2007,
<https://www.rfc-editor.org/info/rfc4760>.
[RFC9136] Rabadan, J., Ed., Henderickx, W., Drake, J., Lin, W., and
A. Sajassi, "IP Prefix Advertisement in Ethernet VPN
(EVPN)", RFC 9136, DOI 10.17487/RFC9136, October 2021,
<https://www.rfc-editor.org/info/rfc9136>.
[RFC9135] Sajassi, A., Salam, S., Thoria, S., Drake, J., and J.
Rabadan, "Integrated Routing and Bridging in Ethernet VPN
(EVPN)", RFC 9135, DOI 10.17487/RFC9135, October 2021,
<https://www.rfc-editor.org/info/rfc9135>.
[RFC9252] Dawra, G., Ed., Talaulikar, K., Ed., Raszuk, R., Decraene,
B., Zhuang, S., and J. Rabadan, "BGP Overlay Services
Based on Segment Routing over IPv6 (SRv6)", RFC 9252,
DOI 10.17487/RFC9252, July 2022,
<https://www.rfc-editor.org/info/rfc9252>.
[RFC4659] De Clercq, J., Ooms, D., Carugi, M., and F. Le Faucheur,
"BGP-MPLS IP Virtual Private Network (VPN) Extension for
IPv6 VPN", RFC 4659, DOI 10.17487/RFC4659, September 2006,
<https://www.rfc-editor.org/info/rfc4659>.
[RFC7311] Mohapatra, P., Fernando, R., Rosen, E., and J. Uttaro,
"The Accumulated IGP Metric Attribute for BGP", RFC 7311,
DOI 10.17487/RFC7311, August 2014,
<https://www.rfc-editor.org/info/rfc7311>.13.2. Informative References
Nice-to-reads for extra context 🌸
[RFC4360] Sangli, S., Tappan, D., and Y. Rekhter, "BGP Extended
Communities Attribute", RFC 4360, DOI 10.17487/RFC4360,
February 2006, <https://www.rfc-editor.org/info/rfc4360>.
[RFC9012] Patel, K., Van de Velde, G., Sangli, S., and J. Scudder,
"The BGP Tunnel Encapsulation Attribute", RFC 9012,
DOI 10.17487/RFC9012, April 2021,
<https://www.rfc-editor.org/info/rfc9012>.
[RFC9774] Kumari, W., Sriram, K., Hannachi, L., and J. Haas,
"Deprecation of AS_SET and AS_CONFED_SET in BGP",
RFC 9774, DOI 10.17487/RFC9774, May 2025,
<https://www.rfc-editor.org/info/rfc9774>.
[RFC4456] Bates, T., Chen, E., and R. Chandra, "BGP Route
Reflection: An Alternative to Full Mesh Internal BGP
(IBGP)", RFC 4456, DOI 10.17487/RFC4456, April 2006,
<https://www.rfc-editor.org/info/rfc4456>.
[RFC4272] Murphy, S., "BGP Security Vulnerabilities Analysis",
RFC 4272, DOI 10.17487/RFC4272, January 2006,
<https://www.rfc-editor.org/info/rfc4272>.
[IANA-SAFI]
IANA, "SAFI Values",
<https://www.iana.org/assignments/safi-namespace>.Acknowledgments 🫶
The authors want to give a huge shoutout to Russell Kelly, Dhananjaya Rao, Suresh Basavarajappa, Mallika Gautam, Senthil Sathappan, Arul Mohan Jovel, Naveen Tubugere, Mathanraj Petchimuthu, Eduard Vasilenko, Amit Kumar, Mohit Kumar, Lukas Krattiger, Gyan Mishra, and Stephane Litkowski for their reviews and suggestions 💜 Thanks also to Sue Hares and Jeff Haas for their detailed reviews that clarified the procedures of the D-PATH Path Attribute 🌈 And the authors especially want to thank Eric Rosen, Gunter Van de Velde, and Ketan Talaulikar for the thorough reviews that helped raise the quality of the document significantly 👑
Authors' Addresses 💌
Jorge Rabadan (editor)
Nokia
520 Almanor Avenue
Sunnyvale, CA 94085
United States of America
Email: jorge.rabadan@nokia.com
Ali Sajassi (editor)
Cisco
225 West Tasman Drive
San Jose, CA 95134
United States of America
Email: sajassi@cisco.com
John Drake
Independent
Email: je_drake@yahoo.com
Wen Lin
HPE
Email: wen.lin@hpe.com
James Uttaro
Independent
Email: jimmy.uttaro@gmail.com
Adam Simpson
Nokia
Email: adam.1.simpson@nokia.com📦 where this page comes from
- ietf: rfc10039 2026-09
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms