RFC 9871
BGP Color-Aware Routing (CAR)
BGP Color-Aware Routing (CAR) 🦋
okay bestie, this one is RFC 9871, titled BGP Color-Aware Routing (CAR) 🌈 It's an IETF document, and it's Experimental 🧪 so it's for testing and trying things out, not a Standards Track spec. The authors are D. Rao, Ed. and S. Agrawal, Ed., both from Cisco Systems. It came out in November 2025 and has ISSN 2070-1721. The header doesn't list anything it obsoletes or updates, so it's standing on its own 💅
Abstract ✨
okay so this doc describes a BGP-based way to set up end-to-end, intent-aware paths across a multi-domain transport network 🛣️ That network can cross multiple service provider and customer domains. The intent-aware paths can then steer traffic for service routes that need a specific intent. The whole solution is called BGP Color-Aware Routing (BGP CAR) 💖
The doc lays out the routing framework and the BGP extensions that make intent-aware routing work. It defines two new BGP SAFIs, BGP CAR SAFI and BGP VPN CAR SAFI, for IPv4 and IPv6 🎀 It also has an extensible NLRI model for both SAFIs, so you can define more NLRI types for different use cases. Each NLRI type has key fields plus TLV-based non-key fields, which keeps per-prefix info compact 🧚 This spec defines two NLRI types: the Color-Aware Route NLRI and the IP Prefix NLRI. It also defines non-key TLV types for the MPLS label stack, the SR-MPLS label index, and Segment Routing over IPv6 (SRv6) Segment Identifiers (SIDs) 🍓 And finally, it defines a new Local Color Mapping (LCM) Extended Community ✨
Status of This Memo 📜
this one is not an Internet Standards Track spec. It's published so people can look at it, implement it experimentally, and evaluate it 🧪
It defines an Experimental Protocol for the Internet community. It's an IETF product and represents the consensus of the IETF community. It got public review and was approved for publication by the Internet Engineering Steering Group (IESG). Not every IESG-approved doc can be a candidate for any level of Internet Standard, so see Section 2 of RFC 7841 for that 🫶
For the current status, any errata, and how to send feedback, go to https://www.rfc-editor.org/info/rfc9871 🔗
Copyright Notice ©️
Copyright (c) 2025 IETF Trust and the persons identified as the document authors. All rights reserved 💜
This doc is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents, which are at https://trustee.ietf.org/license-info and apply on the publication date. Please actually read those, because they lay out your rights and restrictions for this doc 📖 Code Components pulled out of this doc must include Revised BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come without warranty as described in the Revised BSD License 🌸
Table of Contents 🗂️
1. Introduction
1.1. Terminology
1.2. Illustration
1.3. Requirements Language
2. BGP CAR SAFI
2.1. Data Model
2.2. Extensible Encoding
2.3. BGP CAR Route Origination
2.4. BGP CAR Route Validation
2.5. BGP CAR Route Resolution
2.6. AIGP Metric Computation
2.7. Inherent Multipath Capability
2.8. BGP CAR Signaling Through Different Color Domains
2.9. Format and Encoding
2.9.1. BGP CAR SAFI NLRI Format
2.9.2. Type-Specific Non-Key TLV Format
2.9.3. Color-Aware Route (E, C) NLRI Type
2.9.4. IP Prefix (E) NLRI Type
2.9.5. Local Color Mapping (LCM) Extended Community
2.10. LCM-EC and BGP Color-EC Usage
2.11. Error Handling
3. Service Route Automated Steering on Color-Aware Paths
4. Filtering
5. Scaling
5.1. Ultra-Scale Reference Topology
5.2. Deployment Model
5.2.1. Flat
5.2.2. Hierarchical Design with Next-Hop-Self at Ingress
Domain BR
5.2.3. Hierarchical Design with Next-Hop-Unchanged at Ingress
Domain BR
5.3. Scale Analysis
5.4. Anycast SID
5.4.1. Anycast SID for Transit Inter-Domain Nodes
5.4.2. Anycast SID for Transport Color Endpoints
6. Routing Convergence
7. CAR SRv6
7.1. Overview
7.1.1. Routed Service SID
7.1.2. Non-Routed Service SID
7.2. Deployment Options for CAR SRv6 Locator Reachability
Distribution and Forwarding
7.2.1. Hop-by-Hop IPv6 Forwarding for BGP SRv6 Prefixes
7.2.2. Encapsulation Between BRs for BGP SRv6 Prefixes
7.3. Operational Benefits of Using CAR SAFI for SRv6 Locator
Prefix Distribution
8. CAR IP Prefix Route
9. VPN CAR
9.1. Format and Encoding
9.1.1. VPN CAR (E, C) NLRI Type
9.1.2. VPN CAR IP Prefix NLRI Type
10. IANA Considerations
10.1. BGP CAR SAFIs
10.2. "BGP CAR NLRI Types" Registry
10.3. "BGP CAR NLRI TLV" Registry
10.4. Guidance for Designated Experts
10.4.1. Additional Evaluation Criteria for the "BGP CAR NLRI
Types" Registry
10.4.2. Additional Evaluation Criteria for the "BGP CAR NLRI
TLV" Registry
10.5. "Border Gateway Protocol (BGP) Extended Communities"
Registry
11. Manageability and Operational Considerations
12. Security Considerations
13. References
13.1. Normative References
13.2. Informative References
Appendix A. Illustrations of Service Steering
A.1. E2E BGP Transport CAR Intent Realized Using IGP Flexible
Algorithm
A.2. E2E BGP Transport CAR Intent Realized Using SR Policy
A.3. BGP Transport CAR Intent Realized in a Section of the
Network
A.3.1. Provide Intent for Service Flows Only in Core Domain
Running IS-IS Flexible Algorithm
A.3.2. Provide Intent for Service Flows Only in Core Domain
over TE Tunnel Mesh
A.4. Transit Network Domains That Do Not Support CAR
A.5. Resource Avoidance Using BGP CAR and IGP Flexible Algorithm
A.6. Per-Flow Steering over CAR Routes
A.7. Advertising BGP CAR Routes for Shared IP Addresses
Appendix B. Color Mapping Illustrations
B.1. Single Color Domain Containing Network Domains with N:N
Color Distribution
B.2. Single Color Domain Containing Network Domains with N:M
Color Distribution
B.3. Multiple Color Domains
Appendix C. CAR SRv6 Illustrations
C.1. BGP CAR SRv6 Locator Reachability Hop-by-Hop Distribution
C.2. BGP CAR SRv6 Locator Reachability Distribution with
Encapsulation
C.3. BGP CAR (E, C) Route Distribution for Steering Non-Routed
Service SID
Appendix D. CAR SAFI NLRI Update Packing Efficiency Calculation
Acknowledgements
Contributors
Authors' Addresses1. Introduction 💖
BGP Color-Aware Routing (CAR) is a BGP-based routing solution for setting up end-to-end intent-aware paths across a multi-domain service provider transport network 🚀 It hands out different routes to one destination network endpoint, like a Provider Edge (PE) router, one route per intent or color. A color is a non-zero 32-bit integer tied to a network intent (things like low cost, low delay, avoiding some resources, a 5G network slice, and so on), as defined in Section 2.1 of [RFC9256] 🌈
BGP CAR fulfills the transport and VPN problem statement and the requirements described in [INTENT-AWARE] ✅
So this doc specifies two new BGP SAFIs, called BGP CAR SAFI (83) and VPN CAR SAFI (84), which carry infrastructure routes to set up transport paths 🎀 Both of them apply to IPv4 Unicast and IPv6 Unicast AFIs (AFI 1 and AFI 2). Using these SAFIs with other AFIs is outside the scope of this doc 🙈
BGP CAR SAFI can be turned on for transport devices in a provider network (the underlay) to set up color-aware transport and infrastructure paths across provider networks 🧚 A multi-domain transport network might have several BGP Autonomous Systems (ASes), and it can also have several IGP domains inside one BGP AS. BGP CAR SAFI can also run inside a VPN Routing and Forwarding (VRF) on a PE router facing a peering Customer Edge (CE) router, and on devices inside a customer network 🏠 VPN CAR SAFI is the one that distributes intent-aware routes from different customers, received on a PE, across the provider networks. It keeps the customer address spaces separate even when they overlap, which is honestly iconic 👑 So the BGP CAR solution lets intent-aware transport paths span a multi-domain network that covers both customer and provider domains 🫶
This doc also defines two BGP CAR route types for this purpose 💅
The BGP CAR Type-1 NLRI (E, C) lets you generate and distribute several color-aware routes to the same destination IP prefix, one per color 🦋 This comes up when a transport node like a PE has one common IP address (like a loopback) to advertise for multiple intents. The operator wants to use that common address as both the BGP next hop for service routes and the transport endpoint for the data plane path. Multiple routes for that same address or prefix are how you set up a unique path for each intent. One example is setting up multiple Label Switched Paths (LSPs) for MPLS or Segment Routing over MPLS (SR-MPLS) to an egress PE, one per intent ✨
The BGP CAR Type-2 NLRI (IP Prefix or E) distributes multiple color-aware routes to a transport node for the case where the operator gives each intent its own unique network IP address block. The transport node then gets a unique IP prefix or address for each intent 🌸 An example use case is Segment Routing over IPv6 (SRv6) per-intent locators 💜
These BGP CAR intent-aware paths are then used by an ingress node (like a PE) to steer traffic flows for service routes that need specific intents 🔥 Steering can point toward a destination for all traffic flows or only for specific ones.
BGP CAR sticks to the flat routing model of BGP for IP routing (BGP-IP) [RFC4271] or BGP Labeled Unicast (BGP-LU) (SAFI 4 in [RFC8277]), and extends it to support intent awareness 🌈 That means you get a consistent operational experience with those widely deployed transport routing technologies, which is very much giving familiar energy 🍓
1.1. Terminology 💖
Okay bestie, let's get the vocab down first, because this section is basically the glossary for the whole doc ✨
- Intent (in routing): any behaviours that influence routing or path selection, including any combination of these:
- a. Topology path selection (e.g., minimize metric or avoid resource) 🛣️
- b. Network Function Virtualization (NFV) service insertion (e.g., service chain steering) 🧩
- c. Per-hop behavior (e.g., a 5G slice) 📶
This one is way more specific when it comes to routing beyond best-effort, compared with intent as a declarative abstraction in [RFC9315]. 🎀
- Color: a non-zero 32-bit integer value tied to an intent (e.g., low cost, low delay, or avoid some resources), as defined in Section 2.1 of [RFC9256]. The operator is the one who manages color assignment. 🌈
- Colored service route: an egress PE (e.g., E2) colors its BGP service (e.g., VPN) route (e.g., V/v) to say what intent it wants for the traffic bound to V/v. The color gets encoded as a BGP Color Extended Community [RFC9012], used as per [RFC9256], or shows up as the locator part of an SRv6 Service SID [RFC9252]. 💅
- Color-aware path to (E2, C): a path for forwarding packets toward E2 that satisfies the intent tied to color C. A few technologies can provide this, like SR Policy [RFC9256], IGP Flexible Algorithm [RFC9350], and BGP CAR (as specified in this document). 🛤️
- Color-aware route (E2, C): a distributed or signaled route that builds a color-aware path to E2 for color C. 🪄
- Service route automated steering on color-aware path: an ingress PE (or ASBR) E1 automatically steers traffic for a C-colored service route V/v from E2 onto an (E2, C) color-aware path. If several of these paths exist, a preference scheme picks the best one (for example, IGP Flexible Algorithm is preferred over SR Policy, and SR Policy is preferred over BGP CAR). 👑
- Color domain: a set of nodes that share the same color-to-intent mapping, usually under single administration. This set can be organized into one or multiple network domains (IGP areas/instances within a single BGP AS, or multiple BGP ASes). Color-to-intent mapping on nodes is set by configuration. Color re-mapping and filtering may happen at color domain boundaries. Refer to [INTENT-AWARE]. 🗺️
- Resolution of a BGP CAR route (E, C): an inter-domain BGP CAR route (E, C) via N gets resolved on an intra-domain color-aware path (N, C), where N is the next hop of the BGP CAR route. 🔁
- Resolution versus steering: matching the terminology in the SR Policy document (Section 8 of [RFC9256]), this document uses steering for mapping the traffic of a service route onto a BGP CAR path. The word resolution is kept for mapping an inter-domain BGP CAR route onto an intra-domain color-aware path. ⚖️
- Service steering: the service route maps traffic to a BGP CAR path (or another color-aware path, e.g., SR Policy). If a color-aware path isn't available, local policy may map it to a color-unaware routing/TE path (e.g., BGP-LU, RSVP-TE, IGP/LDP). The service steering concept doesn't care which transport technology is used. Section 3 describes the specific service steering mechanisms used for MPLS, SR-MPLS, and SRv6. 🚦
- Intra-domain resolution: a BGP CAR route maps to an intra-domain color-aware path (e.g., SR Policy, IGP Flexible Algorithm, BGP CAR) or to a color-unaware routing/TE path (e.g., RSVP-TE, IGP/LDP, BGP-LU). 🧭
- Transport network: a network made up of multiple cooperating domains managed by one or more operators, using routing technologies such as IP, MPLS, and SR to forward packets for connectivity and other services. In an SR deployment, the transport network sits within a trusted domain as per [RFC8402]. 🚇
- Transport layer: the underlay network layer (e.g., MPLS LSPs between PEs) that an overlay or service layer (e.g., MPLS VPNs) gets to use. 🪨
- Transport RR: a BGP Route Reflector (RR) used to distribute transport/underlay routes within a domain or across domains. 📣
- Service RR: a BGP Route Reflector (RR) used to distribute service/overlay routes within a domain or across domains. 📮
- Abbreviations:
- ABR: Area Border Router
- AFI: Address Family Identifier
- AIGP: Accumulated IGP Metric Attribute [RFC7311]
- ASBR: Autonomous System Border Router
- BGP-LU: BGP Labeled Unicast SAFI (SAFI value 4 as per [RFC8277])
- BGP-IP: BGP IPv4/IPv6 Unicast SAFI (SAFI value 1 as per [RFC4760] and [RFC4271])
- BR: Border Router (either for an IGP area (an ABR) or a BGP autonomous system (an ASBR))
- Color-EC: BGP Color Extended Community [RFC9012]
- E: Generic representation of a transport endpoint such as a PE, ABR, or ASBR
- LCM-EC: BGP Local Color Mapping Extended Community
- NLRI: Network Layer Reachability Information [RFC4271]
- P node: An intra-domain transport router
- RD: Route Distinguisher
- RR: Route Reflector
- T-RR: Transport Route Reflector
- S-RR: Service Route Reflector
- SAFI: Subsequent Address Family Identifier
- TEA: Tunnel Encapsulation Attribute [RFC9012]
- V/v, W/w: Generic representations of a service route (indicating prefix/mask length), regardless of AFI/SAFI or actual NLRI encoding
1.2. Illustration 🦋
Here's a quick picture of the big-deal properties of the BGP CAR solution 📸
+-------------+ +-------------+ +-------------+
| | | | | | V/v with C1
|----+ |------| |------| +----|/
| E1 | | | | | | E2 |\
|----+ | | | | +----| W/w with C2
| |------| |------| |
| Domain 1 | | Domain 2 | | Domain 3 |
+-------------+ +-------------+ +-------------+
Figure 1: BGP CAR Solution IllustrationEverything here is part of one inter-domain network under a single authority, with a consistent color-to-intent mapping 🫶
- C1 is mapped to "low delay" ⏱️
- Flex-Algo 1 is mapped to "low delay", and hence to C1
- C2 is mapped to "low delay and avoid resource R" 🚧
- Flex-Algo 2 is mapped to "low delay and avoid resource R", and hence to C2
E1 receives two service routes from E2:
- V/v with BGP Color-EC C1
- W/w with BGP Color-EC C2
E1 has the following color-aware paths:
- (E2, C1) provided by BGP CAR with the following per-domain support:
- Domain 1: over IGP FA1
- Domain 2: over SR Policy bound to color C1
- Domain 3: over IGP FA1
- (E2, C2) provided by SR Policy
E1 automatically steers traffic for the received service routes like this:
- V/v via (E2, C1) provided by BGP CAR
- W/w via (E2, C2) provided by SR Policy
Illustrated properties (the highlights reel) 🎬
- Leverage of the BGP Color-EC
- The service routes are colored with the widely used BGP Color Extended Community [RFC9012] to request intent 💖
- (E, C) automated steering
- V/v and W/w are automatically steered on the appropriate color-aware path 🤖
- Seamless coexistence of BGP CAR and SR Policy
- V/v is steered on a color-aware path provided by BGP CAR
- W/w is steered on a color-aware path provided by SR Policy
- Seamless interworking of BGP CAR and SR Policy
- V/v is steered on a BGP CAR path that is itself resolved within domain 2 onto an SR Policy bound to the color of V/v 🪡
Other properties 🌸
- MPLS data plane: with 300k PEs and 5 colors, the BGP CAR solution ensures that no single node needs to support a data plane scaling in the order of
Remote PE * C(Section 5). Otherwise that would blow past the MPLS data plane 💀 - Control plane: a node should not install a (E, C) path if it's not participating in that color-aware path. 🙅♀️
- Incongruent color-intent mapping: the solution supports signaling a BGP CAR route across different color domains (Section 2.8). 🌍
The key benefits of this model are 👑
- Leverage of the BGP Color-EC [RFC9012] to color service routes
- The definition of the automated service steering: a C-colored service route V/v from E2 is steered onto a color-aware path (E2, C)
- The definition of the data model of a BGP CAR path: (E, C)
- Natural extension of BGP-IP/BGP-LU data model (E)
- Consistent with SR Policy data model
- The definition of the recursive resolution of a BGP CAR route: a BGP CAR (E2, C) route via N is resolved onto the color-aware path (N, C), which may itself be provided by BGP CAR or via another color-aware routing solution (e.g., SR Policy, IGP Flexible Algorithm)
- Explicit definitions for multiple transport encapsulations (e.g., MPLS, SR, SRv6, IP) 🧁
1.3. Requirements Language 🔤
Heads up, these are the official magic words. "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document get read as described in BCP 14 [RFC2119] [RFC8174], but only when they show up in all capitals, exactly like that. ✨
2. BGP CAR SAFI 🧚
2.1. Data Model 🍓
Here's the BGP CAR data model in a nutshell:
- NLRI key: falls into two categories, so it can cover the use cases from the intro 🗂️
- Type-1: the key is IP Prefix and Color (E, C). Color in the NLRI key tells apart a color-aware route for a common IP prefix, one per intent. Color also says which intent the route is for.
- Type-2: the key is IP Prefix (E). The unique IP prefix assigned for an intent (i.e, IP Prefix == intent) is what tells the color-aware route apart. Color isn't needed in the NLRI key as a distinguisher.
- NLRI non-key encapsulation data: data such as the MPLS label stack, SR-MPLS label index, and SRv6 SID list tied to the NLRI. It's carried in TLVs, as described in Section 2.9.2. 📦
- BGP next hop: the next hop address associated with a particular NLRI key [RFC4760]. 📍
- AIGP metric [RFC7311]: accumulates a metric value specific to color/intent for a CAR route across multiple BGP hops. 🧮
- Local Color Mapping Extended Community (LCM-EC): an optional non-zero 32-bit color value used to represent the intent of a CAR route 🎨
- when the CAR route propagates between different color domains.
- when the CAR route has a unique IP prefix for an intent.
- BGP Color Extended Community (Color-EC) [RFC9012]: an optional non-zero 32-bit color value used to represent the intent of the BGP CAR next hop. It's used as per [RFC9256] for automated route resolution, when the intent/color used for the next hop is different from the CAR route's intent/color. 🌈
The sections below dig into the data model in detail. The sections describing protocol processing for the CAR SAFI generally apply the same way to both route types (for instance, any operation based on color). The examples use (E, C) to keep things simple 💜
2.2. Extensible Encoding 🧩
okay so extensibility is literally built into this thing, bestie 🌈 it comes from a few different places:
- NLRI Type field 💜: this is what lets new NLRI formats show up for new route types.
NLRI (route) types other than Type-1 (E, C) and Type-2 (E) are totally out of scope for this doc 🙅♀️
- Key Length field 📏: this one specifies the key length. It lets new NLRI types get handled opaquely, which is what makes transitivity of new route types through BGP speakers like Route Reflectors (RRs) possible 🦋
- TLV-based encoding of non-key part of NLRI 🎀: this allows extra non-key fields for a prefix, so different types of transport can run at the same time with efficient BGP update packing (Section 2.9) ✨
- AIGP attribute 👑: this gives extensibility via TLVs, so you can define extra metric semantics for a color as an intent needs them 💅
2.3. BGP CAR Route Origination 🚀
a BGP CAR route can be originated locally (think loopback 🍓) or redistributed from an (E, C) color-aware path that comes from another routing solution (like SR Policy, IGP Flexible Algorithm, RSVP-TE, or BGP-LU [RFC8277]) 🧚
2.4. BGP CAR Route Validation ✅
so a BGP CAR path (E, C) via next hop N with encapsulation T is valid if the color-aware path (N, C) exists with encapsulation T available in the data plane. simple, right? 🥰
a local policy can customize the validation process though:
- the color constraint in the first check may be relaxed 🫶 if N is reachable via alternate color(s) or in the default routing table, the route may be considered valid
- the data plane availability constraint of T may be relaxed to use an alternate encapsulation 🌸
- a performance-measurement verification may be added to make sure the intent associated with C is met (e.g., delay < bound) ⏱️
a path that is not valid MUST NOT be considered for BGP best path selection. no exceptions, bestie 🙅♀️💀
2.5. BGP CAR Route Resolution 🌊
a BGP color-aware route (E2, C1) with next hop N gets automatically resolved over a color-aware route (N, C1) by default. that (N, C1) route comes from color-aware mechanisms like IGP Flexible Algorithm [RFC9350], SR Policy (Section 2.2 of [RFC9256]), or recursively from BGP CAR itself. when there are multiple producers of (N, C1), the default preference is: IGP Flexible Algorithm, then SR Policy, then BGP CAR 👑
local policy SHOULD provide additional control 🎛️
- a BGP color-aware route (E2, C1) with next hop N may be resolved over a color-aware route (N, C2). basically the local policy maps the resolution of C1 over a different color, C2. examples where this can happen:
- in a domain where resource R is known to not be present, the inter-domain intent C1="low delay and avoid R" may be resolved over an intra-domain path of intent C2="low delay" 🦋
- in a domain where, if no (N, C1) path is available, resolution may fall back to a C2 path if the user has permitted it 🙈
- route resolution may be driven by an egress node. in an SRv6 domain, an egress node selects and advertises an SRv6 SID from its locator for intent C1, with a BGP CAR route. then the ingress node resolves the received SRv6 SID over an IPv6 route for the intent-aware locator of the egress node for C1, or over a summary route that covers the locator. that summary route can come from SRv6 Flexible Algorithm or from the BGP CAR IP Prefix route itself (e.g., Appendix C.2) 🧚♀️
- local policy may map the CAR route to mechanisms that are unaware of color or that provide best-effort, such as RSVP-TE, IGP/LDP, or BGP-LU/BGP-IP (e.g., Appendix A.3.2) for brownfield scenarios 🏚️➡️🏡
route resolution via a different color C2 can be automated by attaching BGP Color-EC C2 to the CAR route (E2, C1), which leverages automated steering as described in Section 8.4 of "Segment Routing Policy Architecture" [RFC9256] for BGP CAR routes ⚙️ this mechanism is illustrated in Appendix B.2. this mechanism SHOULD be supported 🫶
for CAR route resolution, if a Color-EC color is present with the route, it takes precedence over the route's intent color. the route's intent color is the LCM-EC color if present (see Section 2.9.5), or else it's the NLRI color 🎨
local policy takes precedence over the color-based automated resolution specified above 👑
the color-aware route (N, C1) may be provided by BGP CAR itself in a hierarchical transport routing design. in those cases, based on the procedures above, recursive resolution may happen over the same or a different CAR route type. Section 7.1.2 describes a scenario where a CAR (E, C) route resolves over a CAR IP Prefix route 💖
a CAR IP Prefix route is allowed to be without color for best-effort. in that case, resolution is based on the BGP next hop N or, when present, a best-effort SRv6 SID advertised by node N 🌷
a BGP CAR route may recursively resolve over a BGP route that carries a TEA and follows Section 6 of [RFC9012] for validation. in this case, the procedures of Section 8 of [RFC9012] apply to BGP CAR routes, using the color precedence specified above for resolution 🧁
the procedures of [RFC9012], Section 6, also apply to BGP CAR routes (AFI/SAFI = 1/83 or 2/83). for instance, a BGP CAR BR may advertise a BGP CAR route to an ingress BR or PE with a specific BGP next hop per color, with a TEA or Tunnel Encapsulation EC, as per Section 6 of [RFC9012] 📮
BGP CAR resolution in one network domain is independent of resolution in another domain 🏰
2.6. AIGP Metric Computation 📊
the Accumulated IGP (AIGP) Metric Attribute [RFC7311] gets updated as the BGP CAR route propagates across the network 🛣️
the value that is set (or appropriately incremented) in the AIGP TLV corresponds to the metric associated with the underlying intent of the color. for example, when the color is tied to a low latency path, the metric value is set based on the delay metric ⚡
information about the metric type used by the underlying intra-domain mechanism can also be used to set the metric value 🧠
if BGP CAR routes traverse a discontinuity in the transport path for a given intent, a penalty is added in the AIGP metric (the value is set by user policy). this could happen when the color C1 path isn't available and the route resolves via the color C2 path (see Appendix A.3 for an example) 😬
AIGP metric computation is recursive 🔁
to avoid continuous IGP metric changes causing end-to-end BGP CAR route churn, an implementation should provide thresholds to trigger AIGP updates 🌀
additional AIGP extensions may be defined to signal state for specific use cases, such as Maximum SID Depth (MSD) along the BGP CAR route advertisement and minimum MTU along the BGP CAR route advertisement. this is out of scope for this document 🙅♀️
2.7. Inherent Multipath Capability 🛤️
the (E, C) route definition inherently gives you redundant paths at every BGP hop, same as BGP-LU or BGP-IP. so like, BGP CAR routes originated by two or more egress ABRs in a domain are advertised as multiple paths to the ingress ABRs in that domain, where they become equal-cost or primary-backup paths 💅 if an egress ABR fails, the ingress ABRs detect and handle it locally within the domain for faster convergence, with no need to propagate the event upstream just to restore traffic 🏃♀️💨
BGP ADD-PATH [RFC7911] SHOULD be enabled for BGP CAR to signal multiple next hops through a transport RR (T-RR) 📣
2.8. BGP CAR Signaling Through Different Color Domains 🌍
[Color Domain 1 A]-----[B Color Domain 2 E2]
[C1=low delay ] [C2=low delay ]okay picture this 🖼️ let's assume a BGP CAR route (E2, C2) is signaled from B to A, which are two BRs of Domain 2 and Domain 1 respectively. let's also assume these two domains don't share the same color-to-intent mapping (aka they belong to different color domains). low delay in Domain 2 is color C2, while it's C1 in Domain 1 (C1 <> C2) 🎨
it's not expected to be a typical scenario for an underlay transport path (e.g., an MPLS LSP) to extend across different color domains. however, the BGP CAR solution seamlessly supports this rare scenario while keeping the separation and independence of administrative authority in each color domain 🫶
the solution works like this 🧩
- within Domain 2, the BGP CAR route is (E2, C2) via E2.
- B signals to A the BGP CAR route as (E2, C2) via B, with Local Color Mapping Extended Community (LCM-EC) of color C2.
- A is aware of the intent-to-color mapping within Domain 2 ("low delay" in Domain 2 is C2), as per the typical coordination that exists between operators of peering domains 🤝
- A maps C2 in the LCM-EC to C1 and signals within Domain 1 the received BGP CAR route as (E2, C2) via A, with LCM-EC (C1).
- the nodes within the receiving Domain 1 use the local color encoded in the LCM-EC for next-hop resolution and service steering 🚦
the following procedures apply at a color domain boundary for BGP CAR routes, performed by route policy at the sending and receiving peer 🚧
- use local policy to control which routes are advertised to or accepted from a peer in a different color domain.
- attach LCM-EC if it isn't present with the route. if LCM-EC is present, update the value to re-map the color as needed.
- this function may be done by the advertising BGP speaker or the receiving BGP speaker, as determined by the operator peering agreement and indicated by local policy on the BGP peers 📜
these procedures apply to both CAR route types, in addition to all procedures specified in earlier sections. LCM-EC is described in Section 2.9.5 💜
salient properties 💎
- the NLRI never changes, even though the color-to-intent mapping changes.
- E is globally unique, which makes E-C in that order unique 🔑
- in typical expected cases, the color of the NLRI is used for resolution and steering.
- in the rare case of color incongruence, the local color encoded in LCM-EC takes precedence.
operational considerations are in Section 11. further illustrations are provided in Appendix B 📚
2.9. Format and Encoding 🎀
BGP CAR leverages BGP multiprotocol extensions [RFC4760] and uses the MP_REACH_NLRI and MP_UNREACH_NLRI attributes for route updates within SAFI value 83, along with AFI 1 for IPv4 prefixes and AFI 2 for IPv6 prefixes 🌈
BGP speakers MUST use the BGP Capabilities Advertisement to ensure support for processing BGP CAR updates. this is done as specified in [RFC4760], by using capability code 1 (multiprotocol BGP), with AFI 1 and 2 (as required) and SAFI 83 📢
the Next Hop network address field in the MP_REACH_NLRI may be either an IPv4 address or an IPv6 address, independent of AFI. if the next hop length is 4, then the next hop is an IPv4 address. the next hop length may be 16 or 32 for an IPv6 next hop address, set as per Section 3 of [RFC2545]. processing of the Next Hop field is governed by standard BGP procedures, as described in Section 3 of [RFC4760] 🧠✨
the sub-sections below specify the generic encoding of the BGP CAR NLRI and non-key TLV fields, followed by the encoding for the specific NLRI types introduced in this document 💖
2.9.1. BGP CAR SAFI NLRI Format ✨
okay bestie, here's the generic layout of the BGP CAR SAFI NLRI 💖
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| NLRI Length | Key Length | NLRI Type | //
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ //
| Type-Specific Key Fields //
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type-Specific Non-Key TLV Fields (if applicable) //
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+where:
NLRI Length: a 1-octet field that tells you the length in octets of the NLRI, not counting the NLRI Length field itself. 🧁
Key Length: another 1-octet field, this one giving the length in octets of the NLRI Type-Specific Key Fields. Key Length MUST be at least 2 less than the NLRI Length. no cap, that's a hard rule 🔥
NLRI Type: a 1-octet field that says what kind of BGP CAR NLRI you're looking at.
Type-Specific Key Fields: what these fields actually are depends on the NLRI Type. Their length is whatever the Key Length says.
Type-Specific Non-Key TLV Fields: these are optional and can hold one or more non-key TLVs (of different types), depending on the NLRI Type. This is how the NLRI can carry non-key info about the route, so updates can get packed more efficiently 📦
The non-key TLVs portion of the NLRI MUST be omitted while carrying it within the MP_UNREACH_NLRI when withdrawing the route advertisement. so when you're withdrawing a route, the TLVs get left off 🙈
If something goes wrong with CAR SAFI NLRI or non-key TLVs, the error handling lives in Section 2.11. 🫶
The benefits of CAR NLRI design are super iconic 💅
- The key length lets BGP speakers figure out where the key portion sits. They can then treat it, along with the NLRI Type field, as an opaque blob when they meet NLRI types they don't understand or don't support. That's what helps deployed Route Reflectors (RR) pass along NLRI types that get introduced later, totally transparently 🦋
- Key length also makes error handling more resilient and less disruptive. You can read how it's used in error handling in Section 2.11. 🛡️
- Being able to carry more than one non-key TLV (of different types) in a single route is a huge win. For example, you can signal several encapsulations for the same route at once, each with its own value (label/SID, etc.). That makes migrations simpler and cheaper 🌈
- no more duplicate routes just to signal different encapsulations
- no more separate control planes for distribution
- update packing stays intact (see Appendix D)
2.9.2. Type-Specific Non-Key TLV Format 🎀
Here's the generic shape of a Non-Key TLV 🧚
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type | Length | Value (variable) //
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+where:
Type: 1 octet that holds the type code and the flags. It's laid out like this:
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
|R|T| Type code |
+-+-+-+-+-+-+-+-+where:
R: this bit is reserved. It MUST be set to 0 and ignored on receive. ✨
T: the Transitive bit. It only matters to speakers that change the BGP CAR next hop. 🦋
- Setting T means the TLV is transitive. An unrecognized transitive TLV MUST be propagated by a speaker that changes the next hop.
- Unsetting T means the TLV is non-transitive. An unrecognized non-transitive TLV MUST NOT be propagated by a speaker that changes the next hop. 🙅♀️
A speaker that does not change the next hop SHOULD propagate all received TLVs. 💜
Type code: the remaining 6 bits hold the TLV's type.
Length: a 1-octet field giving the length of the Non-Key TLV's value portion, in octets.
Value: a variable-length field. Its size comes from the Length field, and you read it according to the Type field.
The sub-sections that follow spell out the non-key TLVs. Each NLRI Type MUST list the TLVs that can be associated with it. 📝
2.9.2.1. Label TLV 🏷️
The Label TLV is what gets CAR routes advertised along with their MPLS labels. It uses the format from Section 2.9.2:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|R|T| Type | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+It's followed by one (or more) labels, encoded like this:
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Label |Rsrv |S|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+where:
Type: the type code is 1. The T bit MUST be unset. 🌸
Length: the length is in octets. It's variable, and MUST be a multiple of 3. 🔢
Label Information: multiples of 3-octet fields that carry the MPLS label(s) tied to the advertised CAR route. You can use it to encode one label or a whole stack of them, as described in [RFC8277]. The number of labels comes from the Length field. The 3-bit Rsrv field and the 1-bit S field SHOULD be set to zero on transmission and MUST be ignored on reception. 🧼
If a BGP transport CAR speaker sets itself as the next hop while propagating a CAR route, it allocates a local label for the type-specific key and updates the value in this TLV. It also MUST program a label cross-connect that results in the label swap operation for the incoming label it advertises, using the label received from its best-path router(s). the label swap is non-negotiable, bestie 🔁
2.9.2.2. Label-Index TLV 🌈
The Label-Index TLV carries Segment Routing over MPLS (SR-MPLS) Segment Identifier (SID) [RFC8402] info for the labeled CAR routes. It has this format, as per Section 2.9.2:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|R|T| Type | Length | Reserved | Flags ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~ | Label Index ~
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
~ |
+-+-+-+-+-+-+-+-+where:
Type: the type code is 2. The T bit MUST be set. 🍓
Length: the length is in octets and is 7. 🎯
Reserved: a 1-octet field that MUST be set to 0 and ignored on receipt. 🤐
Flags: a 2-octet field, defined the same way as the Flags field of the Label-Index TLV in the BGP Prefix-SID attribute (Section 3.1 of [RFC8669]). 🚩
Label Index: a 4-octet field, defined the same way as the Label Index field of the Label-Index TLV in the BGP Prefix-SID attribute (Section 3.1 of [RFC8669]). 🔖
This TLV gives you the same functionality as the Label-Index TLV from [RFC8669], just for Transport CAR routes in SR-MPLS deployments. When a speaker allocates a local label for a received CAR route as per Section 2.9.2.1, it SHOULD use the received Label Index as a hint, following the procedures in [RFC8669], Section 4. ✨
The Label-Index TLV packs really efficiently, because it carries the Label Index in the NLRI instead of in the BGP Prefix-SID attribute (Appendix D). it's giving packing efficiency 📦💅
The Label-Index TLV MUST NOT be carried in the Prefix-SID attribute for BGP CAR routes. If a speaker receives a CAR route with the Label-Index TLV in the Prefix-SID attribute, it SHOULD ignore it. The BGP Prefix-SID attribute SHOULD NOT be sent with the labeled CAR routes if the attribute is only being used to convey the Label-Index TLV. 🚫
2.9.2.3. SRv6 SID TLV 🦋
BGP Transport CAR can also set up end-to-end color-aware connectivity using Segment Routing over IPv6 (SRv6) [RFC8402]. [RFC8986] specifies the SRv6 Endpoint behaviors (e.g., End Penultimate Segment Pop (PSP)), and BGP CAR with SRv6 can take advantage of them. The SRv6 SID TLV is what advertises CAR routes along with their SRv6 SIDs. It has this format, as per Section 2.9.2:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|R|T| Type | Length | SRv6 SID Info (variable) //
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+where:
Type: the type code is 3. The T bit MUST be unset. 🌷
Length: the length is in octets. It's variable, and MUST be either less than or equal to 16, or be a multiple of 16. 📏
SRv6 SID Information: a field whose size comes from the length. It carries the SRv6 SID(s) for the advertised CAR route in one of these forms:
- A single 128-bit SRv6 SID, or an ordered list of 128-bit SRv6 SIDs.
- A transposed portion (refer to [RFC9252]) of the SRv6 SID. It MUST be sized in multiples of one octet and be less than 16.
BGP CAR SRv6 SID TLV definitions bring these benefits:
- Explicitly encoding SIDs avoids the robustness problems that come from overloading MPLS label fields. 🛡️
- Including the entire 128 bits for unique SIDs (non-transposition) keeps BGP update prefix packing intact. 📦
- The highly efficient transposition scheme (12-14 bytes saved per NLRI) also keeps BGP update prefix packing intact. 🤩
The BGP CAR route update for SRv6 encapsulation MUST include the BGP Prefix-SID attribute, along with the SRv6 L3 Service TLV carrying the SRv6 SID information as specified in [RFC9252]. When you use the transposition scheme of encoding for packing efficiency of BGP updates [RFC9252], the transposed part of the SID rides in the SRv6 SID TLV and isn't limited by the MPLS label field size. 🧩
If a BGP transport CAR speaker sets itself as the next hop while propagating a CAR route and allocates an SRv6 SID that maps to the received SRv6 SID, it updates the value in this TLV. 🔄
Received MPLS information can map to SRv6, and vice versa. 💫
| [SRv6-INTERWORK] describes MPLS and SRv6 interworking
| procedures and their applicability to BGP CAR routes.2.9.3. Color-Aware Route (E, C) NLRI Type ✨
okay so this is the Color-Aware Route NLRI type, and it's giving main character energy 💅 It's how we advertise BGP CAR color-aware routes (E, C) 🎀 and the format looks like this:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| NLRI Length | Key Length | NLRI Type |Prefix Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IP Prefix (variable) //
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Color (4 octets) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+after the key stuff, you can add optional Non-Key TLVs, encoded just like in Section 2.9.2 🦋
Where:
NLRI Length: variable, bestie 🌸
Key Length: variable too 💖 It counts up the Prefix Length field, the IP Prefix field, and the Color field together. For IPv4 (AFI=1), the minimum is 5 and the maximum is 9. For IPv6 (AFI=2), the minimum is 5 and the maximum is 21. 🧚
NLRI Type: 1 🍓
Type-Specific Key Fields: here's the tea 🫖
- Prefix Length: a 1-octet field holding the prefix length in bits. It MUST be less than or equal to 32 for IPv4 (AFI=1), and less than or equal to 128 for IPv6 (AFI=2). 💅
- IP Prefix: an IPv4 or IPv6 prefix, depending on the AFI 🌈 It's a variable-size field holding the most significant octets of the prefix. For IPv4, the size works like this:
- 0 octets for prefix length 0
- 1 octet for prefix length 1 to 8
- 2 octets for prefix length 9 to 16
- 3 octets for prefix length 17 up to 24
- 4 octets for prefix length 25 up to 32
IPv6 follows the same pattern for prefix lengths of 1-128 (octets 1-16) ✨
The last octet has enough trailing bits to make the field end on an octet boundary. The trailing bits MUST be set to zero 🙈 The field size MUST be less than or equal to 4 for IPv4 (AFI=1), and less than or equal to 16 for IPv6 (AFI=2). 😭
- Color: 4 octets holding a non-zero color value that's attached to the prefix. 🎀
Type-Specific Non-Key TLVs: the Label TLV, Label-Index TLV, and SRv6 SID TLV (Section 2.9.2) MAY be associated with this NLRI type 🦋 (wait, the original says "may", so I'm keeping it as "may" in lowercase, iykyk)
ngl this one's the Color-Aware route, so the prefix is unique across the administrative domains where BGP transport CAR is deployed. It's possible the same prefix gets originated by multiple BGP CAR speakers with anycast addressing or multihoming 🫶
The Color is what lets us send multiple route advertisements for the same prefix. The color is tied to an intent (e.g., low latency) in the originator color domain 💜
2.9.4. IP Prefix (E) NLRI Type ✨
this one is the IP Prefix route NLRI type, used for advertising BGP CAR IP Prefix routes (E) 🌸 The format:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| NLRI Length | Key Length | NLRI Type |Prefix Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IP Prefix (variable) //
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+it's followed by optional Non-Key TLVs encoded as per Section 2.9.2 🧚
Where:
NLRI Length: variable 🌈
Key Length: variable 💖 It counts the Prefix Length field and the IP Prefix field together. For IPv4 (AFI=1), the minimum is 1 and the maximum is 5. For IPv6 (AFI=2), the minimum is 1 and the maximum is 17. 👑
NLRI Type: 2 🍓
Type-Specific Key Fields: 🫖
- Prefix Length: 1-octet field with the prefix length in bits. It MUST be less than or equal to 32 for IPv4 (AFI=1), and less than or equal to 128 for IPv6 (AFI=2). 💅
- IP Prefix: an IPv4 or IPv6 prefix, based on the AFI. It's a variable-size field holding the most significant octets. For IPv4:
- 0 octet for prefix length 0
- 1 octet for prefix length 1 to 8
- 2 octets for prefix length 9 to 16
- 3 octets for prefix length 17 up to 24
- 4 octets for prefix length 25 up to 32
IPv6 uses the same pattern for prefix lengths of 1-128 (octets 1-16) 🦋
The trailing bits in the last octet MUST be set to zero, and the field size MUST be less than or equal to 4 for IPv4 (AFI=1), and less than or equal to 16 for IPv6 (AFI=2). 🙈
Type-Specific Non-Key TLVs: the Label TLV, Label-Index TLV, and SRv6 SID TLV (Section 2.9.2) MAY be associated with the IP Prefix NLRI Type 🫶
2.9.5. Local Color Mapping (LCM) Extended Community ✨
this doc defines a brand new BGP Extended Community called "LCM" 👑 It's a Transitive Opaque Extended Community with this encoding:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type=0x3 | Sub-Type=0x1b | Reserved |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Color |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+where:
- Type: 0x3 🎀
- Sub-Type: 0x1b 🌸
- Reserved: 2-octet reserved field that MUST be set to zero on transmission and ignored on reception. 🙈
- Color: 4-octet field carrying the non-zero 32-bit color value. 💜
When a CAR route crosses the originator's color domain boundary, LCM-EC gets added or updated, as specified in Section 2.8 🦋 LCM-EC conveys the local color mapping for the intent (e.g., low latency) in other (transit or destination) color domains. 🌈
For CAR IP Prefix routes, LCM-EC MAY also be added in the originator color domain to show the color tied to the IP prefix ✨
An implementation SHOULD NOT send more than one instance of the LCM-EC. However, if more than one instance is received, an implementation MUST disregard all instances other than the one with the numerically highest value. 💅
If a node gets multiple BGP CAR routes (paths) for a given destination endpoint and color that have different LCM values, that's a misconfiguration in color re-mapping for one of the routes. 😭
In this case, the LCM from the selected BGP best path SHOULD be chosen to be installed into the routing table. 👑
A warning message SHOULD also be logged for further operator intervention. 🫶
If present, LCM-EC contains the intent of a BGP CAR route. LCM-EC Color is used instead of the Color in CAR route NLRI for the procedures in earlier sections, like route validation (Section 2.4), route resolution (Section 2.5), AIGP calculation (Section 2.6) and steering (Section 3). 🧚
The LCM-EC MAY be used for filtering of BGP CAR routes and/or for applying routing policies on the intent, when present. 💖
2.10. LCM-EC and BGP Color-EC Usage ✨
there are 2 distinct requirements to be supported, as stated in [INTENT-AWARE] 🔥
- Domains with different intent granularity (Section 6.3.1.9 of [INTENT-AWARE])
- Network domains under different administration (i.e., color domains; see Section 6.3.1.10 of [INTENT-AWARE])
Requirement 1 is the case where, within the same administrative or color domain, BGP CAR routes for N end-to-end intents may need to traverse an intermediate transit domain where only M intents are available, N >= M. For example, a multi-domain network designed as Access-Core-Access: the core may have the most granular N intents, whereas the access only has fewer M intents. 🍓 So the BGP next-hop resolution for a CAR route in the access domain MUST be via a color-aware path for one of these M intents. As the procedures in Section 2.5 describe, and the example in Appendix B.2 illustrates, BGP Color-EC is used to automate CAR route resolution in this case. 🌸
For requirement 2, where CAR routes traverse different color domains, LCM-EC carries the local color mapping for the NLRI color in other color domains. The related procedures are in Section 2.8, and there's an example in Appendix B.3 🦋
Both LCM-EC and BGP Color-EC MAY be present at the same time with a BGP CAR route. For example, a BGP CAR route (E, C1) from color domain D1, with LCM-EC C2 in color domain D2, MAY also carry Color-EC C3 and next hop N in a transit network domain within D2, where C2 is being resolved via an available intra-domain intent C3 (see the detailed example in the combination of Appendices B.2 and B.3). 🧚
In this case, as described in Section 2.5, the default order of processing for resolution in the presence of LCM-EC is local policy, then BGP Color-EC color, and finally LCM-EC color. 👑
2.11. Error Handling 💅
Okay so the error handling rules in [RFC7606] totally apply to BGP update messages for BGP CAR SAFI 🫶. The whole goal is to minimize the disruption a session reset or 'AFI/SAFI disable' causes, to the extent possible, because nobody wants a whole session tanking over one bad NLRI 😭.
If the error lets the router skip the malformed NLRI(s) and keep processing the rest of the update message, then it MUST handle those malformed NLRIs as 'treat-as-withdraw' ✨. In other cases, where the NLRI encoding error means the router can't process the BGP update message at all, the router SHOULD handle those malformed NLRIs as 'AFI/SAFI disable' when other AFI/SAFIs besides BGP CAR are being advertised over the same session 🙈. Alternately, the router MUST perform 'session reset' when the session is only being used for BGP CAR SAFI 💀.
The CAR NLRI definition encodes NLRI length and key length explicitly 🎀. The NLRI length MUST be relied upon to find where the next NLRI field starts. Key length MUST be relied upon to pull out the key and do 'treat-as-withdraw' for malformed information 🧚.
A sender MUST make sure the NLRI and key lengths are the number of actual bytes encoded in the NLRI and key fields, respectively, no matter what content is being encoded 💜.
Given that the NLRI length and Key length MUST be valid, failures in the following checks result in 'AFI/SAFI disable' or 'session reset' 👑:
- The minimum
NLRI LengthMUST be at least 2, asKey LengthandNLRI Typeare required fields. - The
Key LengthMUST be at least 2 less thanNLRI Length.
NLRI type-specific error handling: 🔥
- By default, a speaker SHOULD discard an unrecognized or unsupported NLRI type and move to the next NLRI.
- Key length and key errors of a known NLRI type SHOULD result in the discard of NLRI similar to an unrecognized NLRI type. (This MUST be logged for trouble shooting.)
Transparent propagation of unrecognized NLRI type: 🦋
- Key length allows unrecognized route types to transit through the RR transparently without a software upgrade. The RR receiving unrecognized route types does not need to interpret the key portion of an NLRI and handles the NLRI as an opaque value of a specific length. An implementation SHOULD provide configuration that controls the RR unrecognized route type propagation behavior, possibly at the granularity of route type values allowed. This configuration option gives the operator the ability to allow specific route types to be transparently passed through RRs based on client speaker support.
- In such a case, the RRs may reflect NLRIs with NLRI type-specific key length and field errors. Clients of such an RR that consume the route for installation will perform the key error handling of known NLRI type or discard the unrecognized type. This prevents propagation of routes with NLRI errors any further in network.
Type-specific Non-Key TLV handling: 🌈
- Either the length of a TLV would cause the NLRI length to be exceeded when parsing the TLV, or fewer than 2 bytes remain when beginning to parse the TLV. In either of these cases, an error condition exists, and the
'treat-as-withdraw'approach MUST be used. - Type-specific length constraints should be verified. The TLV MUST be discarded if there is an error. When discarded, an error may be logged for further analysis.
- If multiple instances of same type are encountered, all but the first instance MUST be discarded. When discarded, an error may be logged for further analysis.
- If a speaker that performs encapsulation to the BGP next hop does not receive at least one recognized forwarding information TLV with the
Tbit unset (such as label or SRv6 SID), such NLRI is considered invalid and not eligible for best path selection.'Treat-as-withdraw'may be used, though it is recommended to keep the NLRI for debugging purposes 🍓.
3. Service Route Automated Steering on Color-Aware Paths ✨
Okay so an ingress PE (or ASBR) E1 automatically steers a C-colored service route V/v from E2 onto an (E2, C) color-aware path, as illustrated in Section 1.2 🦋. If several of these paths exist, a preference scheme picks the best one. The default preference scheme goes IGP Flexible Algorithm first, then SR Policy, followed by BGP CAR. A configuration option may be used to adjust the default preference scheme 💅.
An egress PE may express its intent that traffic should be steered a certain way through the transport layer by including the BGP Color-EC [RFC9012] with the relevant service routes 💌. An ingress PE steers service traffic over a CAR (E, C) route using the service route's next hop and BGP Color-EC 🫶.
This is consistent with the automated service route steering on SR Policy (a routing solution providing color-aware paths) defined in [RFC9256] 🌸. All the steering variations described in [RFC9256] are applicable to BGP CAR paths: on-demand steering, per-destination steering, per-flow steering, and color-only steering. For brevity, please refer to Section 8 of [RFC9256] 🙌.
Appendix A provides illustrations of service route automated steering over BGP CAR (E, C) routes 📎.
An egress PE may express its intent that traffic should be steered a certain way through the transport layer by allocating the SRv6 Service SID from a routed intent-aware locator prefix (Section 3.3 of [RFC8986]). Steering at an ingress PE is via resolution of the Service SID over a CAR Type-2 IP Prefix route. Service steering over BGP CAR SRv6 transport is described in Section 7 🧚.
Service steering via BGP CAR routes is applicable to any BGP SAFI, including SAFIs for IPv4/IPv6 (SAFI 1), L3VPN (SAFI 128), pseudowire (PW), EVPN (SAFI 70), FlowSpec, and BGP-LU (SAFI 4) 👑.
4. Filtering 🧹
PEs and BRs may support filtering of CAR routes. For instance, the filtering may only accept routes of locally configured colors 🎀.
Techniques such as RT Constrain [RFC4684] may also be applied to the CAR SAFI, where Route Target (RT) Extended Communities [RFC4360] can be used to constrain distribution and automate filtering of CAR routes. RT assignment may be via user policy; for example, an RT value can be assigned to all routes of a specific color 💜.
A PE may support on-demand installation of a CAR route based on the presence of a service route whose next hop resolves via the CAR route ✨.
Similarly, a PE may dynamically subscribe to receive individual CAR routes from upstream routers or Route Reflectors (RRs) to limit the routes that it needs to learn. On-demand subscription and automated filtering procedures for individual CAR routes are outside the scope of this document 🙈.
5. Scaling 📈
This section analyzes the key scale requirement of [INTENT-AWARE], specifically:
- No intermediate node data plane should need to scale to (Colors * PEs).
- No node should learn and install a BGP CAR route to (E, C) if it does not install a colored service route to E.
While the requirements and design principles generally apply to any transport, the logical analysis based on the network design in this section focuses on MPLS/SR-MPLS transport since the scaling constraints are specifically relevant to these technologies. BGP CAR SAFI is used here, but the considerations can apply to [RFC8277] or [RFC8669] used with MPLS/SR-MPLS 🌈.
Two key principles used to address the scaling requirements are a hierarchical network and routing design, and on-demand route subscription and filtering 🔥.
Figure 2 in Section 5.1 provides an ultra-scale reference topology. Section 5.1 describes this topology. Section 5.2 presents three design models to deploy BGP CAR in the reference topology, including hierarchical options. Section 5.3 analyzes the logical scaling properties of each model 🦋.
Filtering techniques described in the previous section allow a PE to limit the CAR routes that it needs to learn or install. Scaling benefits of on-demand BGP subscription and filtering will be described in a separate document 💅.
5.1. Ultra-Scale Reference Topology 🌸
RD:V/v via E2
+-----+ +-----+ vpn label:30030 +-----+
....... |S-RR1| <........... |S-RR2| <...............|S-RR3| <......
: +-----+ +-----+ Color C1 +-----+ :
: :
: :
: :
+:------------+--------------+--------------+--------------+--------:-+
|: | | | | : |
|: | | | | : |
|: +---+ +---+ +---+ +---+ : |
|: |121| |231| |341| |451| : |
|: +---+ +---+ +---+ +---+ : |
|---+ | | | | +---|
| E1| | | | | | E2|
|---+ | | | | +---|
| +---+ +---+ +---+ +---+ |
| |122| |232| |342| |452| |
| +---+ +---+ +---+ +---+ |
| Access | Metro | Core | Metro | Access |
| domain 1 | domain 2 | domain 3 | domain 4 | domain 5 |
+-------------+--------------+--------------+--------------+----------+
iPE iBRM iBRC eBRC eBRM ePE
Figure 2: Ultra-Scale Reference TopologyThe following description applies to the reference topology above 🗺️:
- Independent IS-IS/OSPF SR instance in each domain.
- Each domain has Flex-Algo 128. Prefix-SID for a node is Segment Routing Global Block (SRGB) 168000 plus node number.
- A BGP CAR route (E2, C1) is advertised by egress BRM node 451. The route is sourced locally from redistribution from IGP Flex-Algo 128.
- Not shown for simplicity, node 452 will also advertise (E2, C1).
- When a transport RR (T-RR) is used within the domain or across domains, ADD-PATH is enabled to advertise paths from both egress BRs to its clients.
- Egress PE E2 advertises a VPN route
RD:V/vwith BGP Color-EC C1 that propagates via service RRs to ingress PE E1. - E1 steers V/v prefix via color-aware path (E2, C1) and VPN label 30030.
5.2. Deployment Model 🦋
5.2.1. Flat 🦋
okay bestie, this is the flat design, and it's giving simplicity 💅 no hierarchy, just one big path from E2 all the way to E1 ✨
RD:V/v via E2
+-----+ +-----+ vpn label:30030 +-----+
....... |S-RR1| <........... |S-RR2| <...............|S-RR3| <......
: +-----+ +-----+ Color C1 +-----+ :
: :
: :
: :
+:------------+--------------+--------------+--------------+--------:-+
|: | | | | : |
|: | (E2,C1) | (E2,C1) | (E2,C1) | : |
|: +---+ via 231 +---+ via 341 +---+ via 451 +---+ : |
|:(E2,C1) |121|<---------|231|<---------|341|<---------|451| : |
|: via 121 /+---+ L=168002 +---+ L=168002 +---+ L=168002 +---+ : |
|---+ / | | | | +---|
| E1| <--/ | | | | | E2|
|---+ L=168002| | | | +---|
| +---+ +---+ +---+ +---+ |
| |122| |232| |342| |452| |
| +---+ +---+ +---+ +---+ |
| Access | Metro | Core | Metro | Access |
| domain 1 | domain 2 | domain 3 | domain 4 | domain 5 |
+-------------+--------------+--------------+--------------+----------+
iPE iBRM iBRC eBRC eBRM ePE
168121 168231 168341 168451
168002 168002 168002 168002 168002
30030 30030 30030 30030 30030 30030Figure 3: Flat Transport Network Design
- Node 451 advertises the BGP CAR route (E2, C1) to 341, which passes it to 231, then to 121, and finally to E1. 🎀
- Each BGP hop allocates a local label and programs a swap entry in forwarding for (E2, C1). 🔥
- E1 receives the BGP CAR route (E2, C1) via 121 with label
168002. 💜- Let's assume E1 selects that path. 🙈
- E1 resolves the BGP CAR route (E2, C1) via 121 on the color-aware path (121, C1). ✨
- The color-aware path (121, C1) is the Flex-Algo 128 path to 121 (label
168121). 🌸
- The color-aware path (121, C1) is the Flex-Algo 128 path to 121 (label
- E1's imposition color-aware label stack for V/v is thus: 🧚
30030<=> V/v168002<=> (E2, C1)168121<=> (121, C1)
- Each BGP hop performs a swap operation on
168002bound to the color-aware path (E2, C1). 👑
5.2.2. Hierarchical Design with Next-Hop-Self at Ingress Domain BR 🌈
this one adds a hierarchy, so the ingress domain border router handles things with next-hop-self, and it's lowkey iconic 💖
(E2,C1)
+-----+ via 451 +-----+
|T-RR1| <-------------- |T-RR2|
/ +-----+ L=168002 +-----+\
/ \
+-------------+---/----------+--------------+-----------\--+----------+
| | / | | \ | |
| (E2,C1) | / (451,C1) | (451,C1) | \| |
| via 121 +---+ via 231 +---+ via 341 +---+ +---+ |
| L=168002 |121| <======= |231| <========|341| <======= |451| |
| / +---+ L=168451 +---+ L=168451 +---+ +---+ |
|---+ / | | | | +---|
| E1|<--/ | | | | | E2|
|---+ | | | | +---|
| +---+ +---+ +---+ +---+ |
| |122| |232| |342| |452| |
| +---+ +---+ +---+ +---+ |
| Access | Metro | Core | Metro | Access |
| domain 1 | domain 2 | domain 3 | domain 4 | domain 5 |
+-------------+--------------+--------------+--------------+----------+
iPE iBRM iBRC eBRC eBRM ePE
168231 168341
168121 168451 168451 168451
168002 168002 168002 168002 168002
30030 30030 30030 30030 30030 30030Figure 4: Hierarchical BGP Transport CAR, Next-Hop-Self (NHS) at iBR
- Node 451 advertises the BGP CAR route (451, C1) to 341, which passes it to 231, and finally to 121. 🦋
- Each BGP hop allocates a local label and programs a swap entry in forwarding for (451, C1). 💅
- 121 resolves the received BGP CAR route (451, C1) via 231 (label
168451) on the color-aware path (231, C1).- The color-aware path (231, C1) is the Flex-Algo 128 path to 231 (label
168231). ✨
- The color-aware path (231, C1) is the Flex-Algo 128 path to 231 (label
- Node 451 advertises the BGP CAR route (E2, C1) via 451 to T-RR2, which reflects it to T-RR1, which reflects it to 121. 🔥
- 121 receives the BGP CAR route (E2, C1) via 451 with label
168002. 💜- Let's assume 121 selects that path. 🙈
- 121 resolves the BGP CAR route (E2, C1) via 451 on the color-aware path (451, C1). 🌸
- The color-aware path (451, C1) is a BGP CAR path to 451 (label
168451). 🧚
- The color-aware path (451, C1) is a BGP CAR path to 451 (label
- 121's imposition of the color-aware label stack for (E2, C1) is thus: 👑
168002<=> (E2, C1)168451<=> (451, C1)168231<=> (231, C1)
- 121 advertises (E2, C1) to E1 with next-hop-self (121) and label
168002. 🎀 - E1 constructs the same imposition color-aware label stack for V/v via (E2, C1) as in the flat model: 🌈
30030<=> V/v168002<=> (E2, C1)168121<=> (121, C1)
- 121 performs a swap operation on
168002with the hierarchical color-aware label stack for (E2, C1) via 451 from step 7. 💖 - Nodes 231 and 341 perform a swap operation on
168451bound to the color-aware path (451, C1). ⭐ - Node 451 performs a swap operation on
168002bound to the color-aware path (E2, C1). 💀
Note: E1 does not need the BGP CAR route (451, C1) in this design. 🙈
5.2.3. Hierarchical Design with Next-Hop-Unchanged at Ingress Domain BR 💜
same hierarchy, but now the ingress domain border router keeps the next hop unchanged, and the plot twist is E1 ends up on the 451 path 😭
(E2,C1)
+-----+ via 451 +-----+
|T-RR1| <-------------- |T-RR2|
/ +-----+ L=168002 +-----+\
/ \
+-------------+---/----------+--------------+-----------\--+----------+
| | / | | \ | |
| (E2,C1) | / (451,C1) | (451,C1) | \| |
| via 451 +---+ via 231 +---+ via 341 +---+ +---+ |
| L=168002/|121| <======= |231| <========|341| <======= |451| |
| / +---+ L=168451 +---+ L=168451 +---+ +---+ |
|---+ <--/ //| | | | +---|
| E1| // | | | | | E2|
|---+ <===// | | | | +---|
| (451,C1) +---+ +---+ +---+ +---+ |
| via 121 |122| |232| |342| |452| |
| L=168451 +---+ +---+ +---+ +---+ |
| | | | | |
| Access | Metro | Core | Metro | Access |
| domain 1 | domain 2 | domain 3 | domain 4 | domain 5 |
+-------------+--------------+--------------+--------------+----------+
iPE iBRM iBRC eBRC eBRM ePE
168121 168231 168341
168451 168451 168451 168451
168002 168002 168002 168002 168002
30030 30030 30030 30030 30030 30030Figure 5: Hierarchical BGP Transport CAR, Next-Hop-Unchanged (NHU) at iBR
- Nodes 341, 231, and 121 receive and resolve the BGP CAR route (451, C1) the same as in the previous model. 🌸
- Node 121 allocates a local label and programs a swap entry in forwarding for (451, C1). 🧚
- 451 advertises the BGP CAR route (E2, C1) to T-RR2, which reflects it to T-RR1, which reflects it to 121. 👑
- Node 121 advertises (E2, C1) to E1 with the next hop as 451 (i.e., next-hop-unchanged). 🎀
- 121 also advertises (451, C1) to E1 with next-hop-self (121) and label
168451. 🌈 - E1 resolves the BGP CAR route (451, C1) via 121 on the color-aware path (121, C1). 💖
- The color-aware path (121, C1) is the Flex-Algo 128 path to 121 (label
168121). ⭐
- The color-aware path (121, C1) is the Flex-Algo 128 path to 121 (label
- E1 receives the BGP CAR route (E2, C1) via 451 with label
168002. 💀- Let's assume E1 selects that path. 🙈
- E1 resolves the BGP CAR route (E2, C1) via 451 on the color-aware path (451, C1). 😭
- The color-aware path (451, C1) is a BGP CAR path to 451 (label
168451). 🔥
- The color-aware path (451, C1) is a BGP CAR path to 451 (label
- E1's imposition color-aware label stack for V/v is thus: 💜
30030<=> V/v168002<=> (E2, C1)168451<=> (451, C1)168121<=> (121, C1)
- Nodes 121, 231, and 341 perform a swap operation on
168451bound to (451, C1). 🦋 - 451 performs a swap operation on
168002bound to the color-aware path (E2, C1). 💅
5.3. Scale Analysis ✨
Okay bestie, these two tables give you the logically analyzed scaling for the control plane and the data plane across the previous three models 💅 Table 1 is the control plane view and Table 2 is the data plane view 🦋
+=======+=====================+=====================+=============+
| | E1 | 121 | 231 |
+=======+=====================+=====================+=============+
| FLAT | (E2,C) via (121,C) | (E2,C) via (231,C) | (E2,C) via |
| | | | (341,C) |
+=======+---------------------+---------------------+-------------+
| H.NHS | (E2,C) via (121,C) | (E2,C) via (451,C) | (451,C) via |
| | | (451,C) via (231,C) | (341,C) |
+=======+---------------------+---------------------+-------------+
| H.NHU | (E2,C) via (451,C) | (451,C) via (231,C) | (451,C) via |
| | (451,C) via (121,C) | | (341,C) |
+=======+---------------------+---------------------+-------------+
Table 1
+=======+============+==================+==================+
| | E1 | 121 | 231 |
+=======+============+==================+==================+
| FLAT | V -> 30030 | 168002 -> 168002 | 168002 -> 168002 |
| | 168002 | 168231 | 168341 |
| | 168121 | | |
+=======+------------+------------------+------------------+
| H.NHS | V -> 30030 | 168002 -> 168002 | 168451 -> 168451 |
| | 168002 | 168451 | 168341 |
| | 168121 | 168231 | |
+=======+------------+------------------+------------------+
| H.NHU | V -> 30030 | 168451 -> 168451 | 168451 -> 168451 |
| | 168002 | 168231 | 168341 |
| | 168451 | | |
| | 168121 | | |
+=======+------------+------------------+------------------+
Table 2- The flat model is the simplest design 🎀 with a single BGP transport level. It results in the minimum label/SID stack at each BGP hop. However, it significantly increases the scale impact on the core BRs (e.g., 341), whose FIB capacity and even MPLS label space may be exceeded. 😭
- 341's data plane scales with (E2, C) where there may be 300k Es and 5 Cs, hence 1.5M entries > 1M MPLS data plane. 💀
- The hierarchical models avoid the need for core BRs to learn routes and install label forwarding entries for (E, C) routes. 🧚
- Whether next hop is set to self or left unchanged at 121, 341's data plane scales with (451, C) where there may be thousands of 451s and 5 Cs. Therefore, this scaling is well under the 1 million MPLS labels data plane limit. 🌈
- They also aid faster convergence by allowing the PE routes to be distributed via out-of-band RRs that can be scaled independent of the transport BRs. ⭐
- The next-hop-self option at ingress BRM (e.g., 121) hides the hierarchical design from the ingress PE, keeping its outgoing label programming as simple as the flat model. However, the ingress BRM requires an additional BGP transport level recursion, which coupled with load-balancing adds data plane complexity. It needs to support a swap and push operation. It also needs to install label forwarding entries for the egress PEs that are of interest to its local ingress PEs. 🍓
- With the next-hop-unchanged option at ingress BRM (e.g., 121), only an ingress PE needs to learn and install output label entries for egress (E, C) routes. The ingress BRM only installs label forwarding entries for the egress ABR (e.g., 451). However, the ingress PE needs an additional BGP transport level recursion and pushes a BGP VPN label and two BGP transport labels. It may also need to handle load-balancing for the egress ABRs. This is the most complex data plane option for the ingress PE. 🔥
5.4. Anycast SID 🦋
This section is all about how Anycast SID complements and improves the scaling designs above, and honestly it's a glow-up 💖
5.4.1. Anycast SID for Transit Inter-Domain Nodes
- Redundant BRs (e.g., two egress BRMs, 451 and 452) advertise BGP CAR routes for a local PE (e.g., E2) with the same SID (based on label index). Such egress BRMs may be assigned a common Anycast SID, so that the BGP next hops for these routes will also resolve via a color-aware path to the Anycast SID. 👑
- The use of Anycast SID naturally provides fast local convergence upon failure of an egress BRM node. In addition, it decreases the recursive resolution and load-balancing complexity at an ingress BRM or PE in the hierarchical designs above. 🫶
5.4.2. Anycast SID for Transport Color Endpoints
The common Anycast SID technique may also be used for a redundant pair of PEs that share an identical set of service (VPN) attachments. Iconic redundancy moment 💅
- For example, assume a node E2' is paired with E2 above (e.g., Figure 5). Both PEs should be configured with the same static label/SID for the services (e.g., per-VRF VPN label/SID), and will advertise associated service routes with the Anycast IP as BGP next hop. 🎀
- This design provides a convergence and recursive resolution benefit on an ingress PE or ABR similar to the egress ABR case in Section 5.4.1. However, its applicability is limited to cases where the above constraints can be met. 🙈
6. Routing Convergence ⚡
BGP CAR leverages existing well-known design techniques to provide fast convergence, and it's giving main character energy 🌸
Section 2.7 describes how BGP CAR provides localized convergence within a domain for BR failures, including originating BRs, without propagating failure churn into other domains. Contained chaos, no spillover 🧚
Anycast SID techniques described in Section 5.4 can provide further convergence optimizations for BR and PE failures deployed in redundant designs. Periodt 💜
7. CAR SRv6 🦋
7.1. Overview
Steering services over SRv6-based intent-aware multi-domain transport paths may be categorized into two distinct cases that are described in Section 5 of [RFC9252]. Both cases are supported by BGP CAR, as described below. Two cases, both covered, no cap 💖
7.1.1. Routed Service SID
The SRv6 Service SID that is advertised with a service route is allocated by an egress PE from a routed intent-aware locator prefix (Section 3.3 of [RFC8986]). Service steering at an ingress PE is via resolution of the Service SID signaled with the service route as described in [RFC9252].
The intent-aware transport path to the SRv6 locator of the egress PE is provided by underlay IP routing. Underlay IP routing can include IGP Flexible Algorithm [RFC9350] within a domain, and BGP CAR (as defined in this document) across multiple IGP domains or BGP ASes. 🌈
An SRv6 locator prefix is assigned for a given intent or color. The SRv6 locator may be shared with an IGP Flexible Algorithm, or it may be assigned specific to BGP CAR for a given intent. 🍓
Distribution of SRv6 locators in BGP CAR SAFI:
- In a multi-domain network, the SRv6 locator prefix is distributed using BGP CAR SAFI to ingress PEs and ASBRs in a remote domain. The SRv6 locator prefix may be advertised in the BGP CAR SAFI from an egress PE, or redistributed into BGP CAR from an IGP Flex-Algo at a BR. The locator prefix may also be summarized on a border node along the path and a summary route distributed to ingress PEs. 🫶
- An IP Prefix CAR route (Type-2) is defined to distribute SRv6 locator prefixes and described in Sections 2.9.4 and 8. ✨
- A BGP CAR advertised SRv6 locator prefix may also be used for resolution of the SRv6 Service SID advertised for best-effort connectivity. 💅
Appendices C.1 and C.2 illustrate the control, and forwarding behaviors for routed SRv6 Service SIDs. Go check those out 🎀
Section 7.2 describes the deployment options.
Section 7.3 describes operational considerations of using BGP CAR SAFI versus BGP IPv6 SAFI for inter-domain route distribution of SRv6 locators. 🌸
7.1.2. Non-Routed Service SID
The SRv6 Service SID allocated by an egress PE is not routed. The service route carrying the non-routed SRv6 Service SID is advertised by the egress PE with a Color-EC C ([RFC9252], Section 5). An ingress PE in a remote domain steers traffic for the received service route with Color-EC C and this SRv6 Service SID as described below. 🧚
BGP CAR distribution of (E, C) underlay route:
- The intent-aware path to the egress PE within the egress domain is provided by an SR-TE or similar policy (E, C) [RFC9256]. This (E, C) policy is distributed into the multi-domain network from egress BRs using a BGP CAR (E, C) route towards ingress PEs in other domains. This signaling is the same as for SR-MPLS as described in earlier sections. 💜
- The (E, C) BGP CAR Type-1 route is advertised from a BR with an SRv6 transport SID allocated from an SRv6 locator assigned for the intent C. An SR-PCE or local configuration may ensure multiple BRs in the egress domain that originate the (E, C) route advertise the same SRv6 transport SID. 🔥
BGP CAR distribution of SRv6 locator underlay route:
- BGP CAR MAY also provide the underlay intent-aware inter-domain route to resolve the intent-aware SRv6 transport SID advertised with the (E, C) BGP CAR route as follows:
- An egress domain BR has an SRv6 locator prefix that covers the SRv6 transport SID allocated by the egress BR for the (E, C) BGP CAR route.
- The egress domain BR advertises an IP Prefix Type-2 CAR route for the SRv6 locator prefix, and the route is distributed across BGP hops in the underlay towards ingress PEs. This distribution is the same as the previous case in Section 7.1.1. The route may also be summarized in another CAR Type-2 route prefix. 🦋
Service traffic steering and SRv6 transport SID resolution at ingress PE:
- An ingress PE in a remote domain resolves the received service route with color C via the (E, C) BGP CAR route above, as described in Section 3. 🌈
- Additionally, the ingress PE resolves the SRv6 transport SID received in the BGP CAR (E, C) route via the BGP CAR IP Prefix route, similar to the SRv6 Routed Service SID resolution in Section 7.1.1.
- Multiple (E, C) routes may resolve via a single IP Prefix CAR route.
- Resolution of (E, C) routes over IP Prefix CAR routes is the typical resolution order as the IP Prefix route provides intent-aware reachability to the BRs that advertise the (E, C) specific routes for each egress PE. However, there can be use cases where an IP Prefix CAR route may resolve via a (E, C) route. 🙈
- Multiple (E, C) routes may resolve via a single IP Prefix CAR route.
- The ingress PE via the recursive resolution above builds the packet encapsulation that contains the SRv6 Service SID and the received (E, C) route's SRv6 transport SID in the SID list. 💖
Appendix C.3 contains an example that illustrates the control plane distribution, recursive resolution and forwarding behaviors described above. 🎀
Note: An SR Policy may also be defined for multi-domain end to end [RFC9256], independent of BGP CAR. In that case, both BGP CAR and SR-TE inter-domain paths may be available at an ingress PE for an (E, C) route (Section 1.2). 🧚
7.2. Deployment Options for CAR SRv6 Locator Reachability Distribution and Forwarding
okay so bestie, an SRv6 locator (or summary) is just an IPv6 prefix 💖 which means it gets installed into the IPv6 forwarding table on a BGP router (like an ABR or ASBR) so packets can actually get forwarded 🦋 and tbh this is the best part: with IPv6 locator prefixes there is no need to allocate and install per-PE SIDs on each BGP hop just to forward packets 🧚✨
a few options for forwarding packets for BGP SRv6 prefixes from [SRv6-INTERWORK] also apply to BGP CAR 🎀 they're covered in Sections 7.2.1 and 7.2.2 below 💅
7.2.1. Hop-by-Hop IPv6 Forwarding for BGP SRv6 Prefixes
this option is giving classic hop-by-hop vibes 🌈 it uses hop-by-hop IPv6 lookup and forwarding on both BRs and P nodes in a domain, along the path that BGP CAR routes travel 🛤️ the procedures look like this:
- In addition to BRs, P nodes within a domain also learn BGP CAR IP Prefix routes (for SRv6) and install them into the forwarding table.
- BGP routing is enabled on all internal nodes (iBGP) using full-mesh or an RR.
- BRs distribute external BGP SRv6 routes to internal peers including P routers, with the following conditions:
- the external BGP next hop is advertised unchanged to the internal peers;
- internal nodes use recursive resolution via IGP at each hop to forward IPv6 packets towards the external BGP next hop; and
- resolution is per intent/color (e.g., via IGP IPv6 Flex-Algo).
this design is illustrated with an example in Appendix C.1 📖
the benefits of this scheme are so good 😭
- A simpler design, as no tunnel encapsulation is required between BRs in a domain.
- No per-PE SID allocation and installation on any BGP hop.
- This design is similar to the well-known Internet / BGP hop-by-hop IP routing model and can support large-scale route distribution.
- In addition, since SRv6 locator prefixes can be summarized, this minimizes the number of routes, and hence the scale requirements on P routers.
7.2.2. Encapsulation Between BRs for BGP SRv6 Prefixes
okay plot twist 🍓 in this design, IPv6 lookup and forwarding for BGP SRv6 prefixes only happen on BGP BRs. the procedures include:
- These nodes use SRv6 (or other) encapsulations to reach the BGP SRv6 next hop.
- SRv6 outer encapsulation may be
H.Encaps.Red. - Encapsulation is not needed for directly connected next hops, such as with eBGP single-hop sessions.
- SRv6 outer encapsulation may be
- BGP route distribution is enabled between BRs via RRs, or directly if single-hop BGP.
- An egress BR sets itself as BGP next hop, and selects and advertises an appropriate encapsulation towards itself.
- If SRv6 encapsulation, then the SRv6 SID advertised from egress BR is from an SRv6 locator for the specific intent within the domain. Multiple BGP SRv6 prefixes may share a common SID, avoiding per-PE SID allocation and installation on any BGP hop.
- If MPLS/SR-MPLS transport, the route will carry the label/Prefix-SID allocated by the next hop. The label/SID may be shared among multiple routes.
- An ingress BR encapsulates SRv6 egress PE destined packets with encapsulation to BGP next hop (i.e., the egress BR).
the benefits of this scheme:
- P nodes do not need to learn or install BGP SRv6 prefixes in this (BGP-free core) design.
- No per-PE SID allocation and installation on any BGP hop.
this design is illustrated in Appendix C.2 🦋
7.3. Operational Benefits of Using CAR SAFI for SRv6 Locator Prefix Distribution
so when reachability to an SRv6 SID is provided by distributing a locator prefix via underlay routing, the BGP IPv6 SAFI (AFI/SAFI=2/1) can also be used for inter-domain distribution of these IPv6 prefixes, as described in Section 7.1.2 of [SRv6-INTERWORK] or [RFC9723] 💅
using the BGP CAR SAFI gives you some seriously good operational perks 👑
- CAR SAFI is a separate BGP SAFI used for underlay transport intent-aware routing. It avoids overloading BGP IPv6 SAFI, which also carries Internet (service) prefixes. Using CAR SAFI provides:
- Automatic separation of SRv6 locator (transport) routes from Internet (service) routes,
- preventing inadvertent leaking of routes, and
- avoiding the need to configure specific route filters for locator routes.
- Priority handling of infrastructure routes over service (Internet) routes.
- Automatic separation of SRv6 locator (transport) routes from Internet (service) routes,
- CAR SAFI also supports inter-domain distribution of (E, C) routes sourced from SR Policy, in addition to SRv6 locator IPv6 prefixes.
- CAR SAFI may also be used for best-effort routes in addition to intent-aware routes as described in the next section.
Note: If infrastructure routes such as SRv6 locator routes are carried in both BGP-IP [RFC4271] / BGP-LU [RFC8277] [RFC4798], and BGP CAR, Section 8 describes the path selection preference between them. 🙈
8. CAR IP Prefix Route
an IP Prefix CAR route is a route type (Type-2) that carries a routable IP prefix, and its processing follows the semantics of [RFC4271] and [RFC2545] 🌸 IP Prefix CAR routes are installed in the default routing and forwarding table and provide longest-prefix-match forwarding. this is unlike Type-1 (E, C) routes, where it's the signaled forwarding data (like labels/SIDs) that gets installed in the forwarding table to create end-to-end paths 🛣️
IP Prefix CAR routes may be originated into BGP CAR SAFI either from an egress PE or from a BR in a domain. Type-2 routes carry infrastructure routes for both IPv4 and IPv6 ✨
as described in Section 2.1, it's used for cases where a unique routable IP prefix is assigned for a given intent or color. it may also be used for routes providing best-effort connectivity 🦋
a few applicable example use cases:
- SRv6 locator prefix with color for specific intents.
- SRv6 locator prefix without color for best-effort.
- Best-effort transport reachability to a PE/BR without color.
for specific intents, color may be signaled with the IP Prefix CAR route for purposes such as intent-aware SRv6 SID or BGP next hop selection at each transit BR, color-based routing policies and filtering, and intent-aware next-hop resolution (Section 2.5). these purposes are the same as with (E, C) routes. for such purposes, color associated with the CAR IP Prefix route is signaled using LCM-EC 💜
reminder bestie: LCM-EC conveys end-to-end intent/color associated with route/NLRI 🫶 when traversing network domain(s) where a different intent/color is used for next-hop resolution, BGP Color-EC may additionally be used as in Section 2.10.
a special case of intent is best-effort, which may be represented by a color and follow the above procedures. however, to be compatible with existing operational usage, the CAR IP Prefix route is allowed to be without color for best-effort. in this case, the routes will not carry an LCM-EC. resolution is described in Section 2.5 🌈
as described in Section 7.3, infrastructure prefixes are intended to be carried in CAR SAFI instead of SAFIs that also carry service routes, such as BGP-IP (SAFI 1, [RFC4271]) and BGP-LU (SAFI 4, [RFC4798]). however, if such infrastructure routes are also distributed in these SAFIs, a router may receive both BGP CAR SAFI paths and IP/LU SAFI paths. by default, the CAR SAFI transport path is preferred over the BGP-IP or BGP-LU SAFI path 👑
a BGP transport CAR speaker that supports packet forwarding lookup based on the IPv6 prefix route (such as a BR) will set itself as next hop while advertising the route to peers. it will also install the IPv6 route into forwarding with the received next hop and/or encapsulation. if such a transit router does not support this route type, it will not install this route and will not set itself as next hop; hence, it will not propagate the route any further 💀
9. VPN CAR
this section extends BGP CAR to address the VPN intent-aware routing requirement stated in Section 6.1.2 of [INTENT-AWARE] 🧚 the examples use MPLS, but other transport types can also be used (e.g., SRv6) 🎀
CE1 -------------- PE1 -------------------- PE2 -------------- CE2 - V- BGP CAR SAFI is enabled on CE1-PE1 and PE2-CE2 sessions.
- BGP VPN CAR SAFI is enabled between PE1 and PE2.
- Provider publishes to customer that intent "low delay" is mapped to color CP ("Provider Color") on its inbound peering links.
- Within its infrastructure, provider maps intent "low delay" to color CPT ("Provider Transport Color").
- On CE1 and CE2, intent "low delay" is mapped to CC ("Customer Color").
(V, CC) is a color-aware route originated by CE2 💅 here's how the story plays out:
- CE2 sends to PE2:
[(V, CC), label L1] via CE2 with LCM-EC (CP) as per peering
agreement.- PE2 installs in VRF A:
[(V, CC), L1] via CE2, which resolves on (CE2, CP) or
connected Outgoing Interface (OIF).- PE2 allocates VPN label L2 and programs swap entry for (V, CC).
- PE2 sends to PE1:
[(RD, V, CC), L2] via PE2, LCM-EC (CP) with regular Color-EC
(CPT).- PE1 installs in VRF A:
[(V, CC), L2] via (PE2, CPT) steered on (PE2, CPT).- PE1 allocates label L3 and programs swap entry for (V, CC).
- PE1 sends to CE1:
[(V, CC), L3] via PE1 after removing LCM-EC through route
policy.- CE1 installs:
[(V, CC), L3] via PE1, which resolves on (PE1, CC) or
connected OIF.- Label L3 is installed as the imposition label for (V, CC). 🔥
VPN CAR distribution for (RD, V, CC) requires a new SAFI that follows the same VPN semantics as defined in [RFC4364] and also supports the distribution of routes with the CAR NLRI and associated non-key TLVs defined in Section 2.9 of this document 💖
procedures defined in [RFC4364] and [RFC4659] apply to VPN CAR SAFI. further, all CAR SAFI procedures described in Section 2 above apply to CAR SAFI enabled within a VRF. since CE and PE are typically in different administrative domains, LCM-EC is attached to CAR routes 🫶
VPN CAR SAFI routes follow color-based steering techniques as described in Section 3 and illustrated in the example above ✨
VPN CAR SAFI routes may also be advertised with a specific BGP next hop per color, with a TEA or Tunnel Encapsulation EC, and follow the procedures of Section 6 of [RFC9012] 🌸
CAR routes distributed in VPN CAR SAFI are infrastructure routes advertised by CEs in different customer VRFs on a PE. example use cases are intent-aware L3VPN Carriers' Carriers (Section 9 of [RFC4364]) and SRv6 over a provider network 🧚 the VPN RD distinguishes CAR routes of different customers being advertised by the PE 🌈
9.1. Format and Encoding 🎀
okay so BGP VPN CAR SAFI runs on the BGP multiprotocol extensions [RFC4760] 💅 and uses the MP_REACH_NLRI and MP_UNREACH_NLRI attributes for route updates within SAFI value 84. It pairs that with AFI 1 for IPv4 VPN CAR prefixes and AFI 2 for IPv6 VPN CAR prefixes ✨
BGP speakers MUST use the BGP Capabilities Advertisement so everyone knows they can handle BGP VPN CAR updates 🫶 You do this the way [RFC4760] says, with capability code 1 (multiprotocol BGP), AFI 1 and 2 (as needed), and SAFI 84.
The Next Hop network address field in the MP_REACH_NLRI may hold either a VPN-IPv4 or a VPN-IPv6 address with an 8-octet RD set to zero, no matter the AFI 🌸 If the next hop length is 12, the next hop is a VPN-IPv4 address with an RD of 0, built as per [RFC4364]. If the next hop length is 24 or 48, the next hop is a VPN-IPv6 address built as per Section 3.2.1.1 of [RFC4659] 🦋
9.1.1. VPN CAR (E, C) NLRI Type 🌈
okay, here's the layout for the VPN CAR Type-1 (E, C) NLRI with RD ✨
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| NLRI Length | Key Length | NLRI Type |Prefix Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Route Distinguisher |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Route Distinguisher |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IP Prefix (variable) //
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Color (4 octets) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+After that, you can add optional Non-Key TLVs, encoded as per Section 2.9.2 💖
where:
all fields are encoded as per Section 2.9.3, with these tweaks 🎀
Key Length: this one counts up the whole chunk made of the RD, the Prefix Length field, the IP Prefix field, and the Color field 💅Route Distinguisher: an 8-octet field encoded according to [RFC4364] 🌸- Type-Specific Non-Key TLVs: the Label TLV, Label-Index TLV, and SRv6 SID TLV (Section 2.9.2) may be associated with the VPN CAR (E, C) NLRI Type ✨
9.1.2. VPN CAR IP Prefix NLRI Type 🦋
here's the VPN CAR IP Prefix NLRI layout 🧚
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| NLRI Length | Key Length | NLRI Type |Prefix Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Route Distinguisher |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Route Distinguisher |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IP Prefix (variable) //
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Optional Non-Key TLVs come after it, encoded as per Section 2.9.2 🫧
where:
all fields are encoded as per Section 2.9.4, with these tweaks 💜
Key Length: this counts up the total length of the RD, the Prefix Length field, and the IP Prefix field 🌷Route Distinguisher: an 8-octet field encoded according to [RFC4364] 💅- Type-Specific Non-Key TLVs: the Label TLV, Label-Index TLV, and SRv6 SID TLV (Section 2.9.2) may be associated with the VPN CAR IP Prefix NLRI Type ✨
the error handling in Section 2.11 also applies to VPN CAR SAFI, so don't skip it 🙈
10. IANA Considerations 📋
10.1. BGP CAR SAFIs 🎀
IANA has assigned SAFI value 83 (BGP CAR) and SAFI value 84 (BGP VPN CAR) from the "SAFI Values" registry in the "Subsequent Address Family Identifiers (SAFI) Parameters" registry group, with this document as a reference 🌈
10.2. "BGP CAR NLRI Types" Registry 🦋
IANA has created a "BGP CAR NLRI Types" registry in the "Border Gateway Protocol (BGP) Parameters" registry group, with this document as a reference. The registry holds the 1-octet code points for BGP CAR NLRI types, and it's populated with the values below 💖
+=======+===================+===========+
| Type | NLRI Type | Reference |
+=======+===================+===========+
| 0 | Reserved | RFC 9871 |
+-------+-------------------+-----------+
| 1 | Color-Aware Route | RFC 9871 |
+-------+-------------------+-----------+
| 2 | IP Prefix | RFC 9871 |
+-------+-------------------+-----------+
| 3-255 | Unassigned |
+-------+-------------------------------+
Table 3Allocations in this registry are made with the "Specification Required" policy from [RFC8126] and Section 10.4 ✨
10.3. "BGP CAR NLRI TLV" Registry 🌸
IANA has created a "BGP CAR NLRI TLV Types" registry in the "Border Gateway Protocol (BGP) Parameters" registry group, with this document as a reference. It's for the 6-bit code points for BGP CAR NLRI non-key TLV types, and it's populated with the values below 🧚
+======+===============+===========+
| Type | NLRI TLV Type | Reference |
+======+===============+===========+
| 0 | Reserved | RFC 9871 |
+------+---------------+-----------+
| 1 | Label | RFC 9871 |
+------+---------------+-----------+
| 2 | Label-Index | RFC 9871 |
+------+---------------+-----------+
| 3 | SRv6 SID | RFC 9871 |
+------+---------------+-----------+
| 4-64 | Unassigned |
+------+---------------------------+
Table 4Allocations in this registry are made with the "Specification Required" policy from [RFC8126] and Section 10.4 💅
For a new TLV to be used with existing NLRI types, the documentation of those NLRI types must be updated 🫶
10.4. Guidance for Designated Experts 👑
okay, so the Designated Expert (DE) gets to check a few things here 🔍 In every review described in this document, the DE is expected to confirm that suitable documentation (a specification) exists, as described in [RFC8126], for both the "BGP CAR NLRI Types" registry and the "BGP CAR NLRI TLV" registry ✨
The DE is also expected to check how clear the purpose and use of the requested code points are. On top of that, the DE must verify that any request for one of these code points has been made available for review and comment within the IETF. The DE will post the request to the IDR Working Group mailing list (or a successor mailing list designated by the IESG). The DE must also make sure that a code point request doesn't conflict with work that's active or already published within the IETF 💜
The DE is expected to confirm that the specification meets the requirements for the "Specification Required" policy (Section 4.6 of [RFC8126]). As a reminder, the specification is required to be "permanent and readily available" 🌷 The DE may assume that any document in the Internet-Draft or RFC repository meets the requirement for permanence and availability. In other cases, and especially for any document not hosted by another standards development organization, the burden of proof for permanence falls on the applicant 🙈
10.4.1. Additional Evaluation Criteria for the "BGP CAR NLRI Types" Registry 🦋
- Check the interoperability between the new NLRI type and the current NLRI types specified in this document for BGP CAR SAFIs (BGP CAR SAFI and VPN CAR SAFI), and any updates to this document.
- Check whether the specification says which non-key TLVs are applicable for the new NLRI type.
10.4.2. Additional Evaluation Criteria for the "BGP CAR NLRI TLV" Registry 🌈
- Check the applicability of the new TLV for the BGP CAR NLRI types defined.
- Check the T bit setting for the new TLV.
10.5. "Border Gateway Protocol (BGP) Extended Communities" Registry 💖
IANA has assigned the sub-type 0x1b for "Local Color Mapping (LCM)" in the "Transitive Opaque Extended Community Sub-Types" registry, in the "Border Gateway Protocol (BGP) Extended Communities" registry group ✨
11. Manageability and Operational Considerations 🌸
Color assignments in a multi-domain network under common or cooperating administrative control (i.e., a color domain) should be managed much like transport layer IP addresses. They should also ensure a unique, non-conflicting color allocation across the different network domains in that color domain 🧚 This is a logical best practice within a single color or administrative domain, which is the most typical deployment scenario 💅
When color-aware routes cross a color domain boundary, the color assignments typically don't need to be identical in both color domains, since the IP prefix is unique in the inter-domain transport network 🦋 That unique IP prefix gives the color in an (E, C) route a unique, non-conflicting scope 💜 Coordination between the operators of the color domains is only needed to re-map the color into a local color (carried in the LCM-EC) assigned for the same intent in the receiving color domain 🫶
However, if networks under different administrative control set up a shared transport service between them, where the same transport service IP address is coordinated and shared among two (or more) color domain networks, then the color assignments tied to that shared IP address should also be coordinated to avoid conflicts in either network (Appendix A.7) 👑
It's worth noting that color assignment coordination is only necessary for routes specific to the shared service IP. Colors used for intra-domain intents, or for inter-domain intents tied to unique IP addresses, don't need any coordination 🌈
Extended communities (LCM-EC/Color-EC) carried in BGP CAR and service routes MUST NOT be filtered, or the desired intent won't be achieved 🚨✨
12. Security Considerations 🔒✨
okay bestie, this doc doesn't change the underlying security stuff already baked into BGP, like what's in [RFC4271] and [RFC4272]. Those are the OG security concerns and they still apply 💅
This document defines a new BGP SAFI and related extensions to carry color-aware routes and their attributes 🎨. The separate SAFI is expected to be explicitly configured by an operator. It's also expected that the right BGP route policy filtering gets configured on this new SAFI, so routing info coming from the routers in this network gets filtered at the right points, both inside the network and at its edges 🚧. Filtering is everything, babe.
Also, this SAFI and its mechanisms can only be turned on through config on routers inside an operator's network, so standard security measures should be taken to lock down access to the management interface(s) of routers that implement them 🔐. Don't leave the front door open, periodt.
Additionally, BGP sessions SHOULD be protected using the TCP Authentication Option [RFC5925] and the Generalized TTL Security Mechanism [RFC5082] 🛡️. BGP origin validation [RFC6811] and BGPsec [RFC8205] could also be used with this SAFI 🧚.
Since CAR SAFI is its own separate BGP SAFI carrying transport or infrastructure routes for the routers in the operator network, it automatically separates infrastructure routes from the service routes that ride in existing BGP SAFIs like BGP IPv4/IPv6 (SAFI=1) and BGP-LU (SAFI=4) (e.g., 6PE [RFC4798]) 🌸. Using CAR SAFI gives you better security, like protection against route leaking, than you'd get by sending infrastructure routes in existing SAFIs that also carry service routes. It's giving separation of concerns, iconic 👑.
BGP CAR distributes label binding similar to [RFC8277], so its security considerations apply too 🏷️.
In SR deployments, BGP CAR distributes infrastructure prefixes along with their SID information for both SR-MPLS and SRv6 🦋. These deployments are within an SR domain [RFC8402], and the security considerations of [RFC8402] apply. On top of that, the security considerations for SRv6 deployments discussed in Section 9.3 of [RFC9252] also apply 🫶.
As [RFC4272] discusses, BGP is vulnerable to traffic-diversion attacks 😱. This SAFI route adds a new way an attacker could divert traffic from its normal path. The potential consequences are scary, ngl: "hijacking" of traffic (inserting an undesired node into the path, which lets someone inspect or modify traffic, or dodge security controls), or denial of service (steering traffic toward a node that doesn't want to receive it) 💀.
The restriction of this SAFI's applicability to its intended, well-defined scope, plus the techniques described above, limits the likelihood of traffic diversions 🌈. Scope it tight, lock it down, and it stays safe, fr fr.
13. References 📚
13.1. Normative References 💜
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119,
DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>.
[RFC2545] Marques, P. and F. Dupont, "Use of BGP-4 Multiprotocol
Extensions for IPv6 Inter-Domain Routing", RFC 2545,
DOI 10.17487/RFC2545, March 1999,
<https://www.rfc-editor.org/info/rfc2545>.
[RFC4360] Sangli, S., Tappan, D., and Y. Rekhter, "BGP Extended
Communities Attribute", RFC 4360, DOI 10.17487/RFC4360,
February 2006, <https://www.rfc-editor.org/info/rfc4360>.
[RFC4684] Marques, P., Bonica, R., Fang, L., Martini, L., Raszuk,
R., Patel, K., and J. Guichard, "Constrained Route
Distribution for Border Gateway Protocol/MultiProtocol
Label Switching (BGP/MPLS) Internet Protocol (IP) Virtual
Private Networks (VPNs)", RFC 4684, DOI 10.17487/RFC4684,
November 2006, <https://www.rfc-editor.org/info/rfc4684>.
[RFC4760] Bates, T., Chandra, R., Katz, D., and Y. Rekhter,
"Multiprotocol Extensions for BGP-4", RFC 4760,
DOI 10.17487/RFC4760, January 2007,
<https://www.rfc-editor.org/info/rfc4760>.
[RFC7311] Mohapatra, P., Fernando, R., Rosen, E., and J. Uttaro,
"The Accumulated IGP Metric Attribute for BGP", RFC 7311,
DOI 10.17487/RFC7311, August 2014,
<https://www.rfc-editor.org/info/rfc7311>.
[RFC7606] Chen, E., Ed., Scudder, J., Ed., Mohapatra, P., and K.
Patel, "Revised Error Handling for BGP UPDATE Messages",
RFC 7606, DOI 10.17487/RFC7606, August 2015,
<https://www.rfc-editor.org/info/rfc7606>.
[RFC8126] Cotton, M., Leiba, B., and T. Narten, "Guidelines for
Writing an IANA Considerations Section in RFCs", BCP 26,
RFC 8126, DOI 10.17487/RFC8126, June 2017,
<https://www.rfc-editor.org/info/rfc8126>.
[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
May 2017, <https://www.rfc-editor.org/info/rfc8174>.
[RFC8277] Rosen, E., "Using BGP to Bind MPLS Labels to Address
Prefixes", RFC 8277, DOI 10.17487/RFC8277, October 2017,
<https://www.rfc-editor.org/info/rfc8277>.
[RFC8402] Filsfils, C., Ed., Previdi, S., Ed., Ginsberg, L.,
Decraene, B., Litkowski, S., and R. Shakir, "Segment
Routing Architecture", RFC 8402, DOI 10.17487/RFC8402,
July 2018, <https://www.rfc-editor.org/info/rfc8402>.
[RFC8669] Previdi, S., Filsfils, C., Lindem, A., Ed., Sreekantiah,
A., and H. Gredler, "Segment Routing Prefix Segment
Identifier Extensions for BGP", RFC 8669,
DOI 10.17487/RFC8669, December 2019,
<https://www.rfc-editor.org/info/rfc8669>.
[RFC8986] Filsfils, C., Ed., Camarillo, P., Ed., Leddy, J., Voyer,
D., Matsushima, S., and Z. Li, "Segment Routing over IPv6
(SRv6) Network Programming", RFC 8986,
DOI 10.17487/RFC8986, February 2021,
<https://www.rfc-editor.org/info/rfc8986>.
[RFC9012] Patel, K., Van de Velde, G., Sangli, S., and J. Scudder,
"The BGP Tunnel Encapsulation Attribute", RFC 9012,
DOI 10.17487/RFC9012, April 2021,
<https://www.rfc-editor.org/info/rfc9012>.
[RFC9252] Dawra, G., Ed., Talaulikar, K., Ed., Raszuk, R., Decraene,
B., Zhuang, S., and J. Rabadan, "BGP Overlay Services
Based on Segment Routing over IPv6 (SRv6)", RFC 9252,
DOI 10.17487/RFC9252, July 2022,
<https://www.rfc-editor.org/info/rfc9252>.
[RFC9256] Filsfils, C., Talaulikar, K., Ed., Voyer, D., Bogdanov,
A., and P. Mattes, "Segment Routing Policy Architecture",
RFC 9256, DOI 10.17487/RFC9256, July 2022,
<https://www.rfc-editor.org/info/rfc9256>.
[RFC9350] Psenak, P., Ed., Hegde, S., Filsfils, C., Talaulikar, K.,
and A. Gulko, "IGP Flexible Algorithm", RFC 9350,
DOI 10.17487/RFC9350, February 2023,
<https://www.rfc-editor.org/info/rfc9350>.13.2. Informative References 🌷
[INTENT-AWARE]
Hegde, S., Rao, D., Uttaro, J., Bogdanov, A., and L.
Jalil, "Problem statement for Inter-domain Intent-aware
Routing using Color", Work in Progress, Internet-Draft,
draft-hr-spring-intentaware-routing-using-color-04, 31
January 2025, <https://datatracker.ietf.org/doc/html/
draft-hr-spring-intentaware-routing-using-color-04>.
[RFC4271] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A
Border Gateway Protocol 4 (BGP-4)", RFC 4271,
DOI 10.17487/RFC4271, January 2006,
<https://www.rfc-editor.org/info/rfc4271>.
[RFC4272] Murphy, S., "BGP Security Vulnerabilities Analysis",
RFC 4272, DOI 10.17487/RFC4272, January 2006,
<https://www.rfc-editor.org/info/rfc4272>.
[RFC4364] Rosen, E. and Y. Rekhter, "BGP/MPLS IP Virtual Private
Networks (VPNs)", RFC 4364, DOI 10.17487/RFC4364, February
2006, <https://www.rfc-editor.org/info/rfc4364>.
[RFC4659] De Clercq, J., Ooms, D., Carugi, M., and F. Le Faucheur,
"BGP-MPLS IP Virtual Private Network (VPN) Extension for
IPv6 VPN", RFC 4659, DOI 10.17487/RFC4659, September 2006,
<https://www.rfc-editor.org/info/rfc4659>.
[RFC4798] De Clercq, J., Ooms, D., Prevost, S., and F. Le Faucheur,
"Connecting IPv6 Islands over IPv4 MPLS Using IPv6
Provider Edge Routers (6PE)", RFC 4798,
DOI 10.17487/RFC4798, February 2007,
<https://www.rfc-editor.org/info/rfc4798>.
[RFC5082] Gill, V., Heasley, J., Meyer, D., Savola, P., Ed., and C.
Pignataro, "The Generalized TTL Security Mechanism
(GTSM)", RFC 5082, DOI 10.17487/RFC5082, October 2007,
<https://www.rfc-editor.org/info/rfc5082>.
[RFC5462] Andersson, L. and R. Asati, "Multiprotocol Label Switching
(MPLS) Label Stack Entry: "EXP" Field Renamed to "Traffic
Class" Field", RFC 5462, DOI 10.17487/RFC5462, February
2009, <https://www.rfc-editor.org/info/rfc5462>.
[RFC5925] Touch, J., Mankin, A., and R. Bonica, "The TCP
Authentication Option", RFC 5925, DOI 10.17487/RFC5925,
June 2010, <https://www.rfc-editor.org/info/rfc5925>.
[RFC6811] Mohapatra, P., Scudder, J., Ward, D., Bush, R., and R.
Austein, "BGP Prefix Origin Validation", RFC 6811,
DOI 10.17487/RFC6811, January 2013,
<https://www.rfc-editor.org/info/rfc6811>.
[RFC7911] Walton, D., Retana, A., Chen, E., and J. Scudder,
"Advertisement of Multiple Paths in BGP", RFC 7911,
DOI 10.17487/RFC7911, July 2016,
<https://www.rfc-editor.org/info/rfc7911>.
[RFC8205] Lepinski, M., Ed. and K. Sriram, Ed., "BGPsec Protocol
Specification", RFC 8205, DOI 10.17487/RFC8205, September
2017, <https://www.rfc-editor.org/info/rfc8205>.
[RFC9315] Clemm, A., Ciavaglia, L., Granville, L. Z., and J.
Tantsura, "Intent-Based Networking - Concepts and
Definitions", RFC 9315, DOI 10.17487/RFC9315, October
2022, <https://www.rfc-editor.org/info/rfc9315>.
[RFC9723] Wang, H., Dong, J., Talaulikar, K., Han, T., and R. Chen,
"BGP Colored Prefix Routing (CPR) for Services Based on
Segment Routing over IPv6 (SRv6)", RFC 9723,
DOI 10.17487/RFC9723, May 2025,
<https://www.rfc-editor.org/info/rfc9723>.
[SRv6-INTERWORK]
Agrawal, S., Filsfils, C., Voyer, D., Dawra, G., Li, Z.,
and S. Hegde, "SRv6 and MPLS interworking", Work in
Progress, Internet-Draft, draft-ietf-spring-srv6-mpls-
interworking-01, 7 July 2025,
<https://datatracker.ietf.org/doc/html/draft-ietf-spring-
srv6-mpls-interworking-01>.Appendix A. Illustrations of Service Steering 🧚♀️
okay so this appendix walks through example scenarios of colored service route steering over end-to-end (E2E) BGP CAR paths 🛤️, which resolve over different intra-domain mechanisms. It's giving "show me the vibes in action," and I'm obsessed 😭
The examples in this section use MPLS/SR for the transport data plane 💜. The SRv6 encapsulation scenarios get their own section below, so stay tuned, bestie ✨
A.1. E2E BGP Transport CAR Intent Realized Using IGP Flexible Algorithm ✨
okay bestie, let's start with the first picture 🧚
RD:V/v via E2
+-----+ vpn label: 30030 +-----+
...... |S-RR1| <..................................|S-RR2| <.......
: +-----+ Color C1 +-----+ :
: :
: :
: :
+-:-----------------------+----------------------+------------------:--+
| : | | : |
| : | | : |
| : (E2,C1) via 121 | (E2,C1) via 231 | (E2,C1)via E2 : |
| : L=168002,AIGP=110 +---+ L=168002,AIGP=10 +---+ L=0x3,LI=8002 : |
| : |-------------------|121|<-----------------|231|<-------------| : |
| : V LI=8002 +---+ LI=8002 +---+ | : |
|----+ | | +-----|
| E1 | | | | E2 |
|----+(E2,C1) via 122 | (E2,C1) via 232 | (E2,C1)via E2+-----|
| ^ L=168002,AIGP=210 +---+ L=168002,AIGP=20 +---+ L=0x3 | |
| |---------------- |122|<-----------------|232|<-------------| |
| LI=8002 +---+ LI=8002 +---+ LI=8002 |
| | | |
| IS-IS SR | IS-IS SR | IS-IS SR |
| FA128 | FA128 | FA128 |
+-------------------------+----------------------+---------------------+
iPE iABR eABR ePE
---------direction of traffic-------->
+------+ +------+
|168121| |168231|
+------+ +------+
+------+ +------+ +------+
|168002| |168002| |168002|
+------+ +------+ +------+
+------+ +------+ +------+
|30030 | |30030 | |30030 |
+------+ +------+ +------+Figure 6: BGP Flexible Algorithm Aware Transport CAR Path 🌸
Use case: Provide end-to-end intent for service flows. 💖
- Here's how the reference topology above works:
- IGP Flex-Algo 128 runs in each domain and is mapped to color C1 🎀
- Egress PE E2 advertises a VPN route RD:V/v colored with Color-EC C1 to steer traffic into BGP transport CAR (E2, C1). That VPN route travels through the service RRs to ingress PE E1 🦋
- The BGP CAR route (E2, C1), with its next hop, label index, and label as shown above, gets advertised through the BRs in each domain. If an RR is used in the domain, ADD-PATH is enabled so it can advertise multiple available paths ✨
- On each BGP hop, the (E2, C1) route's next hop gets resolved over IGP Flex-Algo 128 of that domain. The AIGP attribute affects the BGP CAR route best path decision, as per [RFC7311] 👑 The BGP CAR label swap entry is installed and goes over the Flex-Algo 128 LSP to the next hop that provides intent in each IGP domain. The AIGP metric should be updated to reflect the Flex-Algo 128 metric to the next hop 🫶
- Ingress PE E1 learns the CAR route (E2, C1) and steers the colored VPN route RD:V/v into (E2, C1) 💜
- Important:
- The IGP Flex-Algo 128 top label provides intent within each domain 🔥
- The BGP CAR label (e.g., 168002) carries end-to-end intent. Thus, it stitches intent over intra-domain Flex-Algo 128 💅
A.2. E2E BGP Transport CAR Intent Realized Using SR Policy
next up, same vibe but with SR Policy instead 🌈
RD:1/8 via E2
+-----+ vpn label: 30030 +-----+
...... |S-RR1| <..................................|S-RR2| <......
: +-----+ Color C1 +-----+ :
: :
: :
: :
+-:-----------------------+----------------------+------------------:-+
| : | | : |
| : | | : |
| : <-(E2,C1) via 121 | <-(E2,C1) via 231 | <-(E2,C1)via E2 : |
| : +---+ +---+ : |
| : ------------------>|121|----------------->|231|--------------| : |
| : | SR Policy(C1,121) +---+ SR Policy(C1,231)+---+ SR Policy v : |
|----+ | | (C1,E2) +---|
| E1 | | | |E2 |
|----+ <-(E2,C1) via 122 | (E2,C1) via 232 | <-(E2,C1)via E2+---|
| | +---+ +---+ ^ |
| ------------------>|122|----------------->|232|---------------| |
| SR Policy(C1,122) +---+ SR Policy(C1,232)+---+ SR Policy(C1,E2) |
| | | |
| | | |
| IS-IS SR | IS-IS SR | IS-IS SR |
+-------------------------+----------------------+--------------------+
iPE iABR eABR ePE
---------direction of traffic-------->
+------+ +------+
| S1 | | S2 |
+------+ +------+
+------+ +------+ +------+
|160121| |160231| | S3 |
+------+ +------+ +------+
+------+ +------+ +------+
|168002| |168002| |168002|
+------+ +------+ +------+
+------+ +------+ +------+
|30030 | |30030 | |30030 |
+------+ +------+ +------+Figure 7: BGP SR Policy Aware Transport CAR Path 🍓
Use case: Provide end-to-end intent for service flows. 🦋
- Here's how the reference topology above works:
- An SR Policy provides intra-domain intent 🧚 The example SID lists realized from the SR policies in each domain, matching the label stack in Figure 7, are:
- SR Policy (C1, 121) segments
<S1, 121>, - SR Policy (C1, 231) segments
<S2, 231>, and - SR Policy (C1, E2) segments
<S3, E2>.
- SR Policy (C1, 121) segments
- Egress PE E2 advertises a VPN route RD:V/v colored with Color-EC C1 to steer traffic into BGP transport CAR (E2, C1). That VPN route travels through the service RRs to ingress PE E1 🎀
- The BGP CAR route (E2, C1), with its next hop, label index, and label as shown above, gets advertised through the BRs in each domain. If an RR is used in the domain, ADD-PATH is enabled so it can advertise multiple available paths ✨
- On each BGP hop, the CAR route (E2, C1) next hop gets resolved over an SR Policy (C1, next hop). The BGP CAR label swap entry is installed and goes over the SR Policy segment list 👑
- Ingress PE E1 learns the CAR route (E2, C1) and steers the colored VPN route RD:V/v into (E2, C1) 💜
- An SR Policy provides intra-domain intent 🧚 The example SID lists realized from the SR policies in each domain, matching the label stack in Figure 7, are:
- Important:
- The SR Policy provides intent within each domain 🔥
- The BGP CAR label (e.g., 168002) carries end-to-end intent. Thus, it stitches intent over intra-domain SR policies 💅
A.3. BGP Transport CAR Intent Realized in a Section of the Network
this one zooms in on just part of the network 🔍
A.3.1. Provide Intent for Service Flows Only in Core Domain Running IS-IS Flexible Algorithm
RD:1/8 via E2
+-----+ vpn label: 30030 +-----+
...... |S-RR1| <..................................|S-RR2| <.......
: +-----+ Color C1 +-----+ :
: :
: :
: :
+-:-----------------------+----------------------+------------------:--+
| : | | : |
| : | | : |
| : (E2,C1) via 121 | (E2,C1) via 231 | (E2,C1) via E2 : |
| : L=168002,AIGP=1110+---+L=168002,AIGP=1010+---+ L=0x3 : |
| : |-------------------|121|<-----------------|231|<-------------| : |
| : V LI=8002 +---+ LI=8002 +---+ | : |
|----+ | | +-----|
| E1 | | | | E2 |
|----+(E2,C1) via 122 | (E2,C1) via 232 | (E2,C1) via E2+-----|
| ^ L=168002,AIGP=1210+---+L=168002,AIGP=1020+---+ L=0x3 | |
| |---------------- |122|<-----------------|232|<-------------| |
| LI=8002 +---+ LI=8002 +---+ |
| | | |
| IS-IS SR | IS-IS SR | IS-IS SR |
| Algo 0 | Flex-Algo 128 | Algo 0 |
| Access | Core | Access |+-------------------------+----------------------+---------------------+
iPE iABR eABR ePE
---------direction of traffic-------->
+------+ +------+
|160121| |168231|
+------+ +------+
+------+ +------+ +------+
|168002| |168002| |160002|
+------+ +------+ +------+
+------+ +------+ +------+
|30030 | |30030 | |30030 |
+------+ +------+ +------+Figure 8: BGP Hybrid Flexible Algorithm Aware Transport CAR Path
- Okay so here's the scoop on the reference topology above 💅
IGP Flex-Algo 128is only enabled in the core (think WAN network) and it's mapped toC1. The access network domain only runsBase Algo 0🌸- Egress PE
E2advertises a VPN routeRD:V/vcolored withColor-EC C1to steer traffic via BGP transport CAR(E2, C1). That VPN route propagates through the service RRs over to ingress PEE1✨ - The BGP CAR route
(E2, C1), with its next hop, label index, and label as shown above, gets advertised through the BRs in each domain. If an RR is used in the domain,ADD-PATHis enabled so multiple available paths can be advertised 🦋 - Local policy on
231and232maps intentC1to resolve the CAR route next hop overIGP Base Algo 0in the right access domain. The BGP CAR label swap entry gets installed so it goes over theBase Algo 0LSP to the next hop. The AIGP metric is updated to reflect theBase Algo 0metric to the next hop, plus an additional penalty of(+1000)💖 - On
121and122, the CAR route(E2, C1)next hop learnt from the Core domain is resolved overIGP Flex-Algo 128. The BGP CAR label swap entry is installed to go over theFlex-Algo 128LSP to the next hop, which provides the intent in the Core IGP domain 👑 - Ingress PE
E1learns the CAR route(E2, C1). It maps intentC1to resolve the CAR route next hop overIGP Base Algo 0, and then it steers the colored VPN routeRD:V/vvia(E2, C1)🎀
- Important:
- The
IGP Flex-Algo 128top label provides intent in the Core domain 🔥 - The BGP CAR label (e.g.,
168002) carries intent from the PEs, and that intent is realized in the Core domain 🫶
- The
A.3.2. Provide Intent for Service Flows Only in Core Domain over TE Tunnel Mesh ✨
RD:1/8 via E2
+-----+ vpn label: 30030 +-----+
...... |S-RR1| <..................................|S-RR2| <.......
: +-----+ Color C1 +-----+ :
: :
: :
: :
+-:-----------------------+----------------------+-----------------:-+
| : | | : |
| : | | : |
| : (E2,C1) via 121 | (E2,C1) via 231 | (E2,C1) via E2 : |
| : L=242003,AIGP=1110+---+L=242002,AIGP=1010+---+ L=0x3 : |
| : |-------------------|121|<-----------------|231|<-------------|: |
| : V +---+ TE tunnel(231) +---+ |: |
|----+ | | +---|
| E1 | | | |E2 |
|----+(E2,C1) via 122 | (E2,C1) via 232 | (E2,C1) via E2+---|
| ^ L=242004,AIGP=1210+---+L=242001,AIGP=1020+---+ L=0x3 | |
| |---------------- |122|<-----------------|232|<-------------| |
| +---+ TE tunnel(232) +---+ |
| | | |
| | | |
| IS-IS/LDP | IS-IS/RSVP-TE | IS-IS/LDP |
| Access 0 | Core | Access 1 |
+-------------------------+----------------------+-------------------+
iPE iABR eABR ePE
---------direction of traffic-------->
+------+ +------+
|240121| |241231|
+------+ +------+
+------+ +------+ +------+
|242003| |242002| |240002|
+------+ +------+ +------+
+------+ +------+ +------+
|30030 | |30030 | |30030 |
+------+ +------+ +------+Figure 9: BGP CAR over TE Tunnel Mesh in Core Network
- Here's the lowdown on this reference topology 🧚
- An RSVP-TE MPLS tunnel mesh is configured only in the core (think WAN network). The access domains only have IS-IS/LDP. (The figure doesn't show every TE tunnel, so don't be shocked 🙈)
- Egress PE
E2advertises a VPN routeRD:V/vcolored withColor-EC C1to steer traffic via BGP transport CAR(E2, C1). That VPN route propagates through the service RRs to ingress PEE1💜 - The BGP CAR route
(E2, C1), with the next hops and labels shown above, is advertised through the BRs in each domain. If an RR is used in the domain,ADD-PATHis enabled to advertise multiple available paths 🌈 - Local policy on
231and232maps intentC1to resolve the CAR route next hop over a best-effort LDP LSP in access domain 1. The BGP CAR label swap entry is installed to go over the LDP LSP to the next hop. The AIGP metric is updated to reflect the best-effort metric to the next hop, plus an additional penalty of(+1000)💅 - Local policy on
121and122maps intentC1to resolve the CAR route next hop in the Core domain over RSVP-TE tunnels. The BGP CAR label swap entry is installed to go over a TE tunnel to the next hop, which provides the intent in the Core domain. The AIGP metric is updated to reflect the TE tunnel metric ✨ - Ingress PE
E1learns the CAR route(E2, C1). It maps intentC1to resolve the CAR route's next hop over a best-effort LDP LSP in access domain 0, and then it steers the colored VPN routeRD:V/vvia(E2, C1)🎀
- Important:
- The RSVP-TE tunnel LSP provides intent in the Core domain 🔥
- The dynamic BGP CAR label carries intent from the PEs, and that intent is realized in the Core domain by resolution via the RSVP-TE tunnel 👑
A.4. Transit Network Domains That Do Not Support CAR 🌸
- In a brownfield deployment, color-aware paths between two PEs may need to go through a transit domain that doesn't support CAR. Examples of a brownfield network like this include an MPLS LDP network with IGP best-effort, or a multi-domain network based on BGP-LU. An MPLS LDP network with best-effort IGP can adopt the scheme from Appendix A.3. Here's the example scenario for BGP-LU 🦋
- Reference topology:
E1 --- BR1 --- BR2 ......... BR3 ---- BR4 --- E2
Ci <----LU----> CiFigure 10: BGP CAR Not Supported in Transit Domain
- The network between BR2 and BR3 is made up of multiple BGP-LU hops (over IGP-LDP domains) 🧚
E1,BR1,BR4, andE2are enabled for BGP CAR, withCicolors 💖BR1andBR2are directly connected, andBR3andBR4are directly connected 🫶
BR1andBR4form an over-the-top peering (via RRs as needed) to exchange BGP CAR routes 💜BR1andBR4also form direct BGP-LU sessions toBR2andBR3, respectively, to establish labeled paths between each other through the BGP-LU network. The sessions may be eBGP or iBGP 🌈BR1recursively resolves the BGP CAR next hop for CAR routes learnt fromBR4via the BGP-LU path toBR4💅BR1signals the transport discontinuity toE1via the AIGP TLV, so thatE1can prefer other paths if they're available ✨BR4does the same thing in the reverse direction 🎀- So the color awareness of the routes, and therefore the paths in the data plane, stay maintained between
E1andE2, even if the intent isn't available within the BGP-LU island 👑 - A similar design can be used for going over network islands of other types 🌸
A.5. Resource Avoidance Using BGP CAR and IGP Flexible Algorithm 💅
This example shows a case of resource avoidance within a domain for a multi-domain color-aware path ✨
+-------------+ +-------------+
| | | | V/v with C1
|----+ |------| +----|/
| E1 | | | | E2 |\
|----+ | | +----| W/w with C2
| |------| IGP FA128 |
| IGP FA128 | | IGP FA129 |
| Domain 1 | | Domain 2 |
+-------------+ +-------------+Figure 11: BGP CAR Resolution over IGP Flexible Algorithm for Resource Avoidance in a Domain
C1andC2are two unique intents in the multi-domain network:C1is mapped to "minimize IGP metric", andC2is mapped to "minimize IGP metric and avoid resource R".
- Resource R is the link(s) or node(s) to be avoided 🙈
Flex-Algo 128in Domain 2 is mapped to "minimize IGP metric", and so it's mapped toC1💖Flex-Algo 129in Domain 2 is mapped to "minimize IGP metric and avoid resource R", and so it's mapped toC2🔥Flex-Algo 128in Domain 1 is mapped to "minimize IGP metric" (there's no resource R to avoid in Domain 1, so bothC1andC2are mapped toFlex-Algo 128) 🌈E1receives the following two service routes fromE2:V/vwith BGP Color-ECC1, andW/wwith BGP Color-ECC2.
E1has the following color-aware paths:(E2, C1)provided by BGP CAR, with this per-domain resolution:- Domain 1: over
IGP Flex-Algo 128, and - Domain 2: over
IGP Flex-Algo 128.
- Domain 1: over
(E2, C2)provided by BGP CAR, with this per-domain resolution:- Domain 1: over
IGP Flex-Algo 128, and - Domain 2: over
IGP Flex-Algo 129(avoiding resource R).
- Domain 1: over
E1automatically steers the received service routes like this:V/vvia(E2, C1)provided by BGP CAR.W/wvia(E2, C2)provided by BGP CAR. 🎀
Observations:
C1andC2are realized over a common intra-domain intent (Flex-Algo 128) in one domain, and over distinct intents in another domain, as required 👑- The 32-bit Color space gives you a lot of flexibility for defining a large number of intents in a multi-domain network. They can be efficiently realized by mapping to a smaller number of intra-domain intents in different domains 🦋
A.6. Per-Flow Steering over CAR Routes ✨
okay so this section shows how ingress PE per-flow steering from Section 8.6 of [RFC9256] works when the paths are BGP CAR routes 💅. Picture the reference topology in Figure 6:
- Ingress PE
E1learns the best-effort BGP-LU routeE2. 🌸 - Ingress PE
E1learns the CAR route (E2,C1), andC1is mapped to "low delay". 🚀 - Ingress PE
E1learns the CAR route (E2,C2), andC2is mapped to "low delay and avoid resource R". 🦋 - Ingress PE
E1is configured to set up an array of paths toE2. Entry 0 is the BGP-LU path to the next hop, colorC1is the first entry, and colorC2is the second entry. The index into the array is called a Forwarding Class (FC). The index can have values 0 to 7, especially when it's derived from the MPLS TC bits [RFC5462]. 💖 E1is configured to match flows on its ingress interfaces (based on any field, like Ethernet destination/source/VLAN/TOS or IP destination/source/DSCP or transport ports, etc.) and color them with an internal per-packetFCvariable (0, 1, or 2 in this example). 🎀- This array shows up as a composite candidate path of SR Policy (
E2,C100). It acts as a container for grouping the constituent paths of different colors/best-effort. This setup gives you automated steering for services colored with Color-ECC100across paths of different colors, which is so clean 😭✨. Just note that Color-ECC100is only used as indirection to the composite policy configured on the ingress PE. - Egress PE
E2advertises a VPN routeRD:V/vwith Color-ECC100, so traffic gets steered via the composite SR Policy (E2,C100), meaning the FC array of paths. 👑
Now let's say E1 receives three packets, K, K1, and K2, on its incoming interface. All three match the VPN route that recurses on E2. E1 colors these three packets with forwarding class 0, 1, and 2, respectively. 🧚
So here's what happens, bestie:
E1forwardsKalong the best-effort path toE2. For the MPLS data plane, that means it pushes the best-effort label ofE2. 🍓E1forwardsK1along the (E2,C1) BGP CAR route. 💜E1forwardsK2along the (E2,C2) BGP CAR route. 🫶
A.7. Advertising BGP CAR Routes for Shared IP Addresses
This one is giving anycast energy 🙈. Check out Figure 12:
+-------------+ +--------------+
| | | +----|
| |------| | E2 |(IP1)
|----+ | | +----|
| E1 | | | Domain 2 |
|----+ | +--------------+
| | +--------------+
| | | +----|
| Domain 1 |------| | E3 |(IP1)
+-------------+ | +----|
| Domain 3 |
+--------------+Figure 12: BGP CAR Advertisements for Shared IP Addresses
Okay, so this is the case where the same transport IP address gets originated from multiple nodes in different network domains. 💅
One use case for this is an anycast transport service, where packet encapsulation (like LSP) can terminate on any one node in a set. All of those nodes can forward the inner payload, usually through an IP lookup in the global table for Internet routes. 🌈
There are a couple of variations on this use case, and we'll walk through them below. 🎀
Only one node is shown per domain, but in practice there'll be multiple nodes for redundancy. ✨
Example 1: Anycast with forwarding to nearest egress 🦋
- Both
E2(in egress domain 2) andE3(in egress domain 3) advertise the anycast (shared) IP (IP1,C1) with the same labelL1. 💖 - By default, an ingress PE
E1receives the best path(s) for (IP1,C1) propagated through BGP hops across the network. 🧚 - The paths to (
IP1,C1) fromE2andE3may merge at a common node along the path toE1, forming equal cost multipaths or active-backup paths at that node. 🌸 - Service route
V/vis advertised from egress domains D2 and D3 with colorC1and next hopIP1. 👑 - Traffic for
V/vsteered atE1via (IP1,C1) is forwarded to eitherE2orE3(or both), as decided by the routing along the network (the nodes in the path). 🍓
Example 2: Anycast with egress domain visibility at ingress PE 👀
E2advertises the (IP1,C1) CAR route andE3advertises the (IP1,C2) CAR route, both for the anycast IPIP1. ColorsC1andC2are assigned to tell apart the egress domains that originate the routes toIP1. 💜- An ingress PE
E1receives the best path(s) propagated through BGP hops across the network for both (IP1,C1) and (IP1,C2). 🫶 - The CAR routes (
IP1,C1) and (IP1,C2) don't get merged at any intermediate node. That givesE1control over path selection and load-balancing of traffic across these two routes. Each route may itself provide multipathing or anycast to a set of egress nodes. 🔥 - Service route
V/vis advertised from egress domains D2 and D3 with colorsC1andC2, respectively, but with the same next hopIP1. 🦋 E1will resolve and steer theV/vpath from D2 via (IP1,C1) and the path from D3 via (IP2,C2).E1will load-balance traffic toV/vacross the two paths, according to a local load-balancing policy. 🌈- Traffic for colored service routes steered at
E1is forwarded to eitherE2orE3(or load-balanced across both), as determined byE1. ✨
In the example above, D2 and D3 belonged to the same color or administrative domain. If D2 and D3 belong to different color domains, those domains will need to coordinate the assignment of colors with the shared IP IP1, so they don't cause conflicts. For instance, in Example 1:
- D2 and D3 may both use
C1for the same intent when they originate a CAR route forIP1.- In this case, neither D2 nor D3 will reuse
C1for some other intent. 🙅♀️
- In this case, neither D2 nor D3 will reuse
- Alternatively, D2 may use
C2and D3 may useC3for originating a CAR route forIP1for the same intent.- In this case, D2 will not use
C3for originating a CAR route forIP1for some other intent. Similarly, D3 will not useC2for originating a CAR route forIP1for some other intent. 💅
- In this case, D2 will not use
Appendix B. Color Mapping Illustrations 🌈
There are a bunch of deployment scenarios that pop up when different color mappings are used in an inter-domain environment. This appendix tries to list them out and make the usage of the color-related protocol constructs super clear 🫶.
B.1. Single Color Domain Containing Network Domains with N:N Color Distribution
- All network domains (ingress, egress, and all transit domains) are enabled for the same N colors.
- A color may of course be realized by different technologies in different domains, as described above. 🎀
- The N intents are signaled end-to-end via BGP CAR routes and are also realized in the data plane. 💖
- Appendix A.1 is an example of this case. 🦋
B.2. Single Color Domain Containing Network Domains with N:M Color Distribution
- Certain network domains may not be enabled for some of the colors used for end-to-end intents, but they may still be required to provide transit for routes of those colors. 🥺
- When a (E, C1) route traverses a domain where color C1 is not available, the operator may decide to use a different intent of color C2 that is available in that domain to resolve the next hop and set up a path through the domain. 🧚
- The next-hop resolution may happen via paths of any intra-domain protocol, or even via paths provided by BGP CAR. 🌸
- The next-hop resolution color C2 may be defined as a local policy at ingress or transit nodes of the domain. 👑
- It may also be signaled automatically from egress border nodes by attaching a Color-EC with value C2 to the BGP CAR routes. 🍓
- So, routes of N end-to-end colors may be resolved over paths from a smaller set of M colors in a transit domain, while the original color awareness end-to-end is preserved. 💜
- Any ingress PE that installs a service (VPN) route with a color C1 must have C1 enabled locally, so it can install IP routes to (E, C1) and resolve the service route's next hop. 🫶
- A degenerate variation of this scenario is where a transit domain doesn't support any color. Appendix A.3 describes an example of this case. 🙈
Illustration for N end-to-end intents over fewer M intra-domain intents:
RD:V/v via E2 Color-EC: 100
RD:W/w via E2 Color-EC: 200
+-----+ RD:X/x via E2 Color-EC: 300 +-----+
...... |S-RR1| <..................................|S-RR2| <........
: +-----+ RD:Y/y via E2 Color-EC: 400 +-----+ :
: :
: :
: :
+-:---------------------+---------------------+----------------------:-+
| : | | : |
| | | |
| (E2,100) via 121 | (E2,100) via 231 | (E2,100) via E2 |
| Color-EC: 1,10 | Color-EC: 1,10 | Color-EC: 1,10 |
| | | |
| (E2,200) via 121 | (E2,200) via 231 | (E2,200) via E2 |
| Color-EC: 1,20 | Color-EC: 1,20 | Color-EC: 1,20 |
| <--- <---- |
| (E2,300) via 121 | (E2,300) via 231 | (E2,300) via E2 |
| Color-EC: 2,30 | Color-EC: 2,30 | Color-EC: 2,30 |
| | | |
| (E2,400) via 121 | (E2,400) via 231 | (E2,400) via E2 |
| Color-EC: 2,40 | Color-EC: 2,40 | Color-EC: 2,40 |
| | | |
| +===+ +===+ |
|====+ | |-------C10-------| | +=====|
| |-------C1-------| |-------C20-------| |-------C1-------| |
| E1 | |121| |231| | E2 |
| |-------C2-------| |-------C30-------| |-------C2-------| |
|====+ | |-------C40-------| | +=====|
| +===+ +===+ |
| C1=FA132 | C10=FA128 | C1=FA132 |
| C2=FA133 | C20=FA129 | C2=FA133 |
| | C30=FA130 | |
| | C40=FA131 | |
| | | |
| IS-IS SR | IS-IS SR | IS-IS SR |
| ACCESS | CORE | ACCESS |+-----------------------+---------------------+------------------------+
iPE iABR eABR ePE
Figure 13: N:M IllustrationOkay, this is the N:M picture, and it's giving whole puzzle 🧩
- The following description applies to the reference topology above: 🗺️
- Core domain provides 4 intra-domain intents as described below:
- FA128 mapped to C10,
- FA129 mapped to C20,
- FA130 mapped to C30, and
- FA131 mapped to C40.
- Access domain provides the following 2 intra-domain intents:
- FA132 mapped to C1, and
- FA133 mapped to C2.
- Operator defines the following 4 BGP CAR end-to-end intents as below:
- CAR color C100 that resolves on C1 in access and C10 in Core domain,
- CAR color C200 that resolves on C1 in access and C20 in Core domain,
- CAR color C300 that resolves on C2 in access and C30 in Core domain, and
- CAR color C400 that resolves on C2 in access and C40 in Core domain.
- E2 may originate BGP CAR routes with multiple BGP Color-ECs as shown above. At each hop, CAR route's next hop is resolved over the available intra-domain color. For example
(E2, C100)with BGP Color-ECs C1, C10 resolves over C1 at ABR 231, C10 at ABR 121, and C1 at E1. 💅 - Egress PE E2 advertises a VPN route
RD:V/vcolored with BGP Color-EC C100 to steer traffic through Flex-Algo 132 in access and Flex-Algo 128 in core. It also advertises another VPN routeRD:W/wcolored with BGP Color-EC C200 to steer traffic through Flex-Algo 132 in access and Flex-Algo 129 in core. ✨
- Core domain provides 4 intra-domain intents as described below:
- Important:
- End-to-end (BGP CAR) colors can be decoupled from intra-domain transport colors. 🔥
- Each end-to-end BGP CAR color is a combination of various intra-domain colors or intents.
- Combination can be expressed by local policy at ABRs or by attaching multiple BGP Color-ECs at origination point of BGP CAR route.
- Service traffic is steered into suitable CAR color to use the most granular intent in a domain multiple hops away from ingress PE. 👑
- Consistent reuse of standard color-based resolution mechanism at both service and transport layers. 🫶
B.3. Multiple Color Domains
When routes are distributed between domains with different color-to-intent mapping schemes, both N:N and N:M cases are possible. Although an N:M mapping is more likely to occur. 🦋
Reference topology:
D1 ----- D2 ----- D3
C1 C2 C3
Figure 14: Multiple Color Domains- C1 in D1 maps to C2 in D2 and to C3 in D3.
- BGP CAR is enabled in all three color domains. 🌈
The reference topology above is used to elaborate on the design described in Section 2.8 🎀
When the route originates in color domain D1 and gets advertised to a different color domain D2, the following procedures apply:
- The NLRI of the BGP CAR route is preserved end to end (i.e., route is
(E, C1)). - A BR of D1 attaches LCM-EC with value C1 when advertising to a BR in D2.
- A BR in D2 receiving
(E, C1)maps C1 in received LCM-EC to local color, say C2.- A BR in D2 may receive
(E, C1)from multiple D1 BRs, which provide equal cost or primary/backup paths.
- A BR in D2 may receive
- Within D2, this LCM-EC value of C2 is used instead of the Color in CAR route NLRI
(E, C1). This applies to all procedures described in the earlier section for a single color domain, such as next-hop resolution and service steering. 🧚 - A colored service route
V/voriginated in color domain D1 with next hop E and Color-EC C1 will also have its Color-EC value re-mapped to C2, typically at a service RR. - On an ingress PE in D2,
V/vwill resolve via C2. 💖 - When a BR in D2 advertises the route to a BR in D3, the same process repeats.
Appendix C. CAR SRv6 Illustrations ✨
C.1. BGP CAR SRv6 Locator Reachability Hop-by-Hop Distribution
RD:V/v via E2
+-----+ SRv6SID=B:C11:2:DT4:: +-----+
...... |S-RR1| <..................................|S-RR2| <.....
: +-----+ +-----+ :
: :
: :
: AS2 AS1 :
+-:------------------------------------+ +--------------:--+
| : | | : |
| : B:C11::/32 via IP1 | | : |
| : +-----+ LCM=C1, AIGP=10 | | : |
| : |T-RR |<.............. | | : |
| : +-----+<.......... : | | : |
| : : B:C11::/32 : : | | : |
| : : via IP2 : : | | : |
| : : LCM=C1,AIGP=10: : | | : |
| : ......... : : : | B:C11::/32 | : |
| : : : : : | via 231 | +-----|
| : : : : : | LCM=C1 | | E2 |
: : +---+ : +---+ : : | AIGP=10 | +-----|
| : : |P11|<.:..>|P13| : +----+ +---+ : |
| : : +---+ : +---+ : | 121|-----IP1|231| : |
| V V : : +----+ eBGP +---+ : |
|----+ : : | | +-----|
| E1 | +---+ : +---+ : | | | En |
|----+ |P12|<.:..>|P14| : | | +-----|
| +---+ +---+ : +----+ eBGP +---+ |
| IPv6 FIB: ...| 122|-----IP2|232| |
| B:C11::/32 via IP1 +----+ +---+ |
| via IP2 | B:C11::/32 | |
| | via 232 | |
| | LCM=C1 | |
| | AIGP=10 | |
| IS-ISv6 | | IS-ISv6 |
| FA128 (B:C12::/32) | |FA128(B:C11::/32)|
| FA0 (B:02::/32) | |FA0 (B:01::/32) |
+--------------------------------------+ +-----------------+
iPE ASBR ASBR ePE
Figure 15The topology above is an example to illustrate the BGP CAR SRv6 locator prefix route-based design (Section 7.1.1) with hop-by-hop IPv6 routing within and between domains. 🌸
- Multi-AS network with eBGP CAR session between ASBRs.
- Transport RR (T-RR) peers with P, BR, and PE clients within an AS to propagate CAR prefixes. ADD-PATH is enabled to propagate multiple paths.
- IS-IS (IGP) Flex-Algo 128 for SRv6 is running in each AS (AS may consist of multiple IGP domains), where the following steps apply:
- Prefix B:C11::/32 summarizes Flex-Algo 128 block in AS1 for the given intent. Node locators in the egress domain are sub-allocated from the block for the given intent.
- Similarly, Prefix B:C12::/32 summarizes Flex-Algo 128 block in AS2.
- Per Flex-Algo external subnets for eBGP next hops IP1 and IP2 are distributed in IS-IS within AS2.
- BGP CAR prefix route B:C11::/32 with LCM C1 is originated by AS1 BRs 231 and 232 on eBGP sessions to AS2 BRs 121 and 122. 🔥
- ASBR 121 and 122 propagate the route in AS2 to all the P, ABRs, and PEs through T-RR.
- Every router in AS2 resolves BGP CAR prefix B:C11::/32 next hops IP1 and IP2 in IS-ISv6 Flex-Algo 128 and programs B:C11::/32 prefix in global IPv6 forwarding table.
- AIGP attribute influences BGP CAR route best path decision.
- Egress PE E2 advertises a VPN route
RD:V/vwith SRv6 Service SID B:C11:2:DT4::. Service SID is allocated by E2 from its locator of color C1 intent. 👑 - Ingress PE E1 learns (via service RRs S-RR1 and S-RR2) VPN route
RD:V/vwith SRv6 SID B:C11:2:DT4::. - Service traffic encapsulated with SRv6 Service SID B:C11:2:DT4:: is inherently steered hop by hop along IPv6 routed path to B:C11::/32 provided by BGP CAR in AS2.
- Encapsulated service traffic is inherently steered along IPv6 routed path to B:C11::/32 provided by IS-ISv6 Flex-Algo 128 in AS1.
- Design applies to multiple ASNs. BGP next hop is rewritten across an eBGP hop.
Important: 🚨
- No tunneling/encapsulation on ingress PE and BRs for BGP CAR provided transport.
- Uses longest prefix match of SRv6 Service SID to BGP CAR IP prefix. No mapping to labels/SIDs, instead use of simple IP-based forwarding. 💀
Packet forwarding:
@E1: IPv4 VRF V/v => H.Encaps.red <B:C11:2:DT4::> => Forward based
on B:C11::/32
@P*: IPv6 table: B:C11::/32 => Forward to interface, NH
@121: IPv6 Table: B:C11::/32 => Forward to interface, NH
@231: IPv6 table: B:C11:2::/48 :: => Forward via IS-ISv6 Flex-Algo
path to E2
@231: IPv6 Table B:C11:2::/48 => Forward via IS-ISv6 Flex-Algo
path to E2
@E2: My SID table B:C11:2:DT4:: => Pop the outer header and look up
the inner DA in the VRFC.2. BGP CAR SRv6 Locator Reachability Distribution with Encapsulation
RD:V/v via E2
+-----+ SRv6SID=B:C11:2:DT4:: +-----+
...... |S-RR1| <..................................|S-RR2| <.......
: +-----+ +-----+ :
: :
: :
: :
+-:-----------------------+----------------------+------------------:--+
| : | | : |
| : | | : |
| : B:C11::/32 via 121 | B:C11::/32 via 231 | : |
| : SID=B:C13:121:END:: | SID=B:C12:231:END:: | : |
| : LCM=C1,AIGP=110 +---+LCM=C1 AIGP=10 +---+ : || : |-------------------|121|<-----------------|231|<-------------| : |
| : V +---+ +---+ | : |
|----+ | | +-----|
| E1 | | | | E2 |
|----+ | | +-----|
| ^ | | : |
| | | | : |
| | | | +-----|
| | | | | En |
| | | | +-----|
| | +---+ +---+ | |
| |---------------- |122|<-----------------|232|<-------------| |
| +---+ +---+ |
| B:C11::/32 via 122 | B:C11::/32 via 232 | |
| SID=B:C13:122:END:: | SID=B:C12:232:END:: | |
| LCM=C1 AIGP=120 | LCM=C1 AIGP=20 | |
| | | |
| IS-ISv6 | IS-ISv6 | IS-ISv6 |
| FA128 (B:C13::/32) | FA128 (B:C12::/32) | FA128 (B:C11::/32) |
| FA0 (B:03::/32) | FA0 (B:02::/32) | FA0 (B:01::/32) |
+-------------------------+----------------------+---------------------+
iPE iABR eABR ePE
Figure 16Okay, bestie, this one is a whole diagram situation 🦋✨ The topology is an example showing the BGP CAR SRv6 locator prefix route-based design ([Section 7.1.1](#)) with intra-domain encapsulation. The example is iBGP, but it works the same way for eBGP (multi-AS) too 🌈
- IGP Flex-Algo 128 is running in each domain, and the prefixes work like this 💅
B:C11::/32summarizes the Flex-Algo 128 block in the egress domain for the given intent. Node locators in the egress domain are sub-allocated from that block 🎀B:C12::/32summarizes the Flex-Algo 128 block in the transit domain 🌸B:C13::/32summarizes the Flex-Algo 128 block in the ingress domain 🧚
- BGP CAR route
B:C11::/32is originated by ABRs 231 and 232 with LCM C1. Along the propagation path, BRs set next-hop-self and appropriately update the intra-domain encapsulation information for the C1 intent. For example, 231 and 121 signal an SRv6 SID of End behavior [RFC8986] allocated from their respective locators for the C1 intent. (Note: IGP Flexible Algorithm is shown for the intra-domain path, but an SR Policy may also provide the path as shown in Appendix C.3 💖) - The AIGP attribute influences the BGP CAR route best path decision 👑
- Egress PE E2 advertises a VPN route
RD:V/vwith SRv6 Service SIDB:C11:2:DT4::. That Service SID is allocated by E2 from its locator of color C1 intent 🫶 - Ingress PE E1 learns the CAR route
B:C11::/32and the VPN routeRD:V/vwith SRv6 SIDB:C11:2:DT4::👀 - Traffic encapsulated with SRv6 Service SID
B:C11:2:DT4::is steered along the IPv6 routed path provided by the BGP CAR IP Prefix route to locatorB:C11::/32🦋
Important 🚨
- It uses longest prefix match of the SRv6 Service SID to the BGP CAR prefix. There is no mapping of labels/SIDs; there is simple IP-based forwarding instead 🍓
- Originating domain PE locators of the given intent can be summarized on transit BGP hops, which means no per-PE state on BRs 💜
Packet forwarding 📦
@E1: IPv4 VRF V/v => H.Encaps.red <B:C13:121:END::, B:C11:2:DT4::>
@121: My SID table: B:C13:121:END:: => Update DA with B:C11:2:DT4::
@121: IPv6 Table: B:C11::/32 => H.Encaps.red <B:C12:231:END::>
@231: My SID table: B:C12:231:END:: => Remove IPv6 header;
Inner DA B:C11:2:DT4::
@231: IPv6 Table B:C11:2::/48 => Forward via IS-ISv6 Flex-Algo
path to E2
@E2: My SID table B:C11:2:DT4:: => Pop the outer header and look up
the inner DA in the VRFC.3. BGP CAR (E, C) Route Distribution for Steering Non-Routed Service SID
RD:V/v via E2
+-----+ SRv6SID: B:01:2:DT4:: +-----+
...... |S-RR1| <..................................|S-RR2| <.......
: +-----+ Color C2 +-----+ :
: :
: +-----+ (E2,C2) via 231 :
: -----------------|T-RR |-------------------| :
:| +-----+ SID=B:C21:2:B6:: | :
+-:|---------------------+---------------------|+------------------:--+
| :| | || : |
| :| | || : |
| :| B:C21::/32 via 121 | B:C21::/32 via 231 ||SR Policy(E2,C2) : |
| :| LCM=C2,AIGP=110 | LCM=C2 AIGP=10 ||BSID=B:C21:2:B6:: : |
| :| +---+ +---+ : |
| :|-------------------|121|<-----------------|231|<-------------| : |
| :V SR Policy(121,C2) +---+SR Policy(231,C2) +---+ | : |
|----+ | | +-----|
| E1 | | | | E2 |
|----+ | | +-----|
| ^ SR Policy(122,C2) +---+SR Policy(232,C2) +---+ | |
| |---------------- |122|<-----------------|232|<-------------| |
| B:C21::/32 via 121+---+B:C21::/32 via 232+---+ SR Policy(E2,C2) |
| LCM=C2,AIGP=120 | LCM=C2 AIGP=20 | BSID=B:C21:2:B6:: |
| | | |
| IS-ISv6 | IS-ISv6 | IS-ISv6 |
| FA0 (B:03::/32) | FA0 (B:02::/32) | FA0(B:01::/32) |
+------------------------+----------------------+---------------------+
iPE iABR eABR ePE
Figure 17This next one is the BGP CAR (E, C) route-based design ([Section 7.1.2](#)) in action 🎀 The example is iBGP, but the design also applies to eBGP (multi-AS) 🌈
- SR Policy (E2, C2) provides the given intent in the egress domain 🔥
- SR Policy (E2, C2) has the segments
<B:01:z:END::, B:01:2:END::>, wherezis the node id in the egress domain 🧚
- SR Policy (E2, C2) has the segments
- Egress ABRs 231 and 232 redistribute the SR Policy into BGP CAR Type-1 NLRI (E2, C2) to other domains, with an SRv6 SID of End.B6 behavior. This route gets propagated to ingress PEs through T-RR, or inline through BRs (121 and 122) with next-hop-unchanged 💅
- The ABRs also advertise a BGP CAR prefix route
B:C21::/32that summarizes the locator part of the SRv6 SIDs for SR policies of the given intent to different PEs in the egress domain. The BGP CAR prefix route propagates through BRs. At each BGP hop, the BGP CAR prefix next-hop resolution triggers an intra-domain transit SR Policy (C2, CAR next hop). For example 🌸- SR Policy (231, C2) with segments
<B:02:y:END::, B:02:231:END::>, and - SR Policy (121, C2) with segments
<B:03:x:END::, B:03:121:END::>, - where
xandyare node ids within the respective domains
- SR Policy (231, C2) with segments
- Egress PE E2 advertises a VPN route
RD:V/vwith Color-EC C2 💖 - Ingress PE E1 steers the VPN route from E2 onto the BGP CAR route (E2, C2), which results in H.Encaps.red of SRv6 transport SID
B:C21:2:B6::and the SRv6 Service SID as the last segment in the IPv6 header 🦋 - IPv6 destination
B:C21:2:B6::matches the CAR prefixB:C21::/32, which steers the packet into the intra-domain (intent-aware) SR Policy on ingress PE E1 and ABR 121 🍓 - An IPv6 packet destination
B:C21:2:B6::lookup in the mySID table on ABR 231 or 232 results in END.B6 behavior (i.e., a push of policy segments to E2) 🧚
Important 🚨
- Ingress PE steers services via the (E, C) CAR route as per [RFC9256] 💜
- In the data plane, (E, C) resolution results in the IPv6 header destination being the SRv6 SID of END.B6 behavior, whose locator is of the given intent on the originating ABRs ✨
- The CAR IP Prefix route along the transit path provides simple Longest Prefix Match (LPM) IPv6 forwarding along the transit BGP hops 🌈
- The CAR NLRI Type-2 prefix summarizes the binding SIDs of all SR policies on the originating ABR of a given intent to different PEs in the egress domain. This eliminates per-PE state on transit routers 👑
Packet forwarding 📦
@E1: IPv4 VRF V/v => H.Encaps.red <B:C21:2:B6::, B:0:E2:DT4::>
H.Encaps.red <SR Policy (C2,121) sid list>
@121: My SID table: B:03:121:END:: => Remove outer IPv6 header;
Inner DA B:C21:2:B6::
@121: IPv6 Table: B:C21::/32 => H.Encaps.red <SR Policy (C2,231) sid
list>
@231: My SID table: B:02:231:END:: => Remove outer IPv6 header;
Inner DA B:C21:2:B6::
@231: MySIDtable B:C21:2:B6:: => H.Encaps.red <SR Policy (C2,E2) sid
list>
@E2: IPv6 Table B:0:2:DT4:: => Pop the outer header and look up the
inner DA in the VRFAppendix D. CAR SAFI NLRI Update Packing Efficiency Calculation ✨
CAR SAFI NLRI encoding is built for update packing 📦💖. Per-route stuff like the label, the label index, and the SRv6 SID encapsulation data gets tucked into the non-key TLV part of the NLRI. So when the other attributes are shared (LCM-EC included, if it's there), you can squish multiple NLRIs into one update message. Iconic efficiency 💅
The table below is a theoretical analysis, but it's based on BGP update message sizes that were actually observed in operational networks 👀. It compares the total BGP data on the wire for CAR SAFI against encoding as specified in [RFC8277] in three cases: an MPLS label (CASE A), an SR extension with MPLS (a per-prefix label index in the Prefix-SID attribute; see [RFC8669]) (CASE B), and an SRv6 SID (CASE C). There are three packing scenarios:
- the ideal case, where the maximum number of routes gets packed up to the 4k-byte update message limit 🌸
- the practical case of average packing, where 5 routes share a set of BGP path attributes and so ride together in a single update message 🫶
- the worst case of no packing, where every route gets its own update message 😭
+=============+=========+===========+==============================+
| Encoding | BGP CAR | NLRI as | Result |
| | NLRI | per | |
| | | [RFC8277] | |
+=============+=========+===========+==============================+
| CASE A: Label |
+=============+=========+===========+==============================+
| (Ideal) | 27.5 MB | 26 MB | No degradation compared to |
| | | | encoding as specified in |
| | | | [RFC8277]. |
+-------------+---------+-----------+ |
| (Practical) | 86 MB | 84 MB | |
+-------------+---------+-----------+ |
| (Worst; no | 325 MB | 324 MB | |
| packing) | | | |
+=============+=========+===========+==============================+
| CASE B: Label & Label-Index |
+=============+=========+===========+==============================+
| (Ideal) | 42 MB | 339 MB | CAR SAFI encoding is more |
| | | Packing | efficient by 88% in the best |
| | | not | case and 71% in the average |
| | | possible | case over the encoding as |
| | | | specified in [RFC8277] |
| | | | (which precludes packing). |
+-------------+---------+-----------+ |
| (Practical) | 99 MB | 339 MB | |
| | | Packing | |
| | | not | |
| | | possible | |
+-------------+---------+-----------+ |
| (Worst; no | 339 MB | 339 MB | |
| packing) | | | |
+=============+=========+===========+==============================+
| CASE C: SRv6 SID |
+=============+=========+===========+==============================+
| (Ideal) | 49 MB | 378 MB | Results are similar to the |
| | | Packing | SR-MPLS case. Transposition |
| | | not | provides a further 20% |
| | | possible | reduction in BGP data. |
+-------------+---------+-----------+ |
| (Practical) | 115 MB | 378 MB | |
| | | Packing | |
| | | not | |
| | | possible | |
+-------------+---------+-----------+ |
| (Worst; no | 378 MB | 378 MB | |
| packing) | | | |
+-------------+---------+-----------+------------------------------+ Table 5: Summary of the Ideal, Practical, and Worst Cases of
Packing BGP DataThis analysis considers 1.5 million routes (5 colors across 300k endpoints) 🌈. Here's the math behind each case, exactly as the numbers were worked out 🧮:
CASE A: BGP data exchanged for MPLS (non-SR):
Consider 200 bytes of shared attributes
CAR SAFI signals label in non-key TLV part of NLRI
Each NLRI size for AFI 1 = 12(key) + 5(label) = 17 bytes
Ideal packing:
Number of NLRIs in 4k update size = 223 (4k-200/17)
Number of update messages of 4k size = 1.5 million/223 = 6726
Total BGP data on wire = 6726 * 4k = ~27.5 MB
Practical packing (5 routes in update message):
Size of update message = (17 * 5) + 200 = 285
Total BGP data on wire = 285 * 300k = ~86 MB
No-packing case (1 route per update message):
Size of update message = 17 + 200 = 217
Total BGP data on wire = 217 * 1.5 million = ~325 MB
SAFI 128 using encoding specified in RFC 8277 with label in NLRI
Each NLRI size for AFI 1 = 13(key) + 3(label) = 16 bytes
Ideal packing:
Number of NLRIs in 4k update size = 237 (4k-200/16)
Number of update messages of 4k size = 1.5 million/237 = ~6330
Total BGP data on wire = 6330 * 4k = ~25.9 MB
Practical packing (5 routes in update message):
Size of update message = (16 * 5) + 200 = 280
Total BGP data on wire = 280 * 300k = ~84 MB
No-packing case (1 route per update message):
Size of update message = 16 + 200 = 216
Total BGP data on wire = 216 * 1.5 million = ~324 MB
CASE B: BGP data exchanged for SR-MPLS label index:
Consider 200 bytes of shared attributes
CAR SAFI signals label index in non-key TLV part of NLRI
Each NLRI size for AFI 1
= 12(key) + 5(label) + 9(Index) = 26 bytes
Ideal packing:
Number of NLRIs in 4k update size = 146 (4k-200/26)
Number of update messages of 4k size = 1.5 million/146 = 6726
Total BGP data on wire = 10274 * 4k = ~42 MB
Practical packing (5 routes in update message)
Size of update message = (26 * 5) + 200 = 330
Total BGP data on wire = 330 * 300k = ~99 MB
No-packing case (1 route per update message)
Size of update message = 26 + 200 = 226
Total BGP data on wire = 226 * 1.5 million = ~339 MB
SAFI 128 using encoding specified in RFC 8277 with label in NLRI
Each NLRI size for AFI 1 = 13(key) + 3(label) = 16 bytes
Ideal packing:
Not supported as label index is encoded in Prefix-SID
attribute
Practical packing (5 routes in update message):
Not supported as label index is encoded in Prefix-SID
attribute
No-packing case (1 route per update message):
Size of update message = 16 + 210 = 226
Total BGP data on wire = 216 * 1.5 million = ~339 MB
CASE C: BGP data exchanged with 128 bit single SRv6 SID:
Consider 200 bytes of shared attributes
CAR SAFI signals SRv6 SID in non-key TLV part of NLRI
Each NLRI size for AFI 1 = 12(key) + 18(SRv6 SID) = 30 bytes
Ideal packing:
Number of NLRIs in 4k update size = 126 (4k-200/30)
Number of update messages of 4k size = 1.5 million/126 = ~12k
Total BGP data on wire = 12k * 4k = ~49 MB
Practical packing (5 routes in update message):
Size of update message
= (30 * 5) + 236 (including Prefix-SID) = 386
Total BGP data on wire = 386 * 300k = ~115 MB
No-packing case (1 route per update message):
Size of update message = 12 + 236 (SID in Prefix-SID) = 252
Total BGP data on wire = 252 * 1.5 million = ~378 MB
SAFI 128 using encoding specified in RFC 8277 with label in NLRI
(No transposition)
Each NLRI size for AFI 1 = 13(key) + 3(label) = 16 bytes
Ideal packing:
Not supported as SRv6 SID is encoded in Prefix-SID
attribute
Practical packing (5 routes in update message):
Not supported as SRv6 SID is encoded in Prefix-SID
attribute
No-packing case (1 route per update message):
Size of update message = 16 + 236 = 252
Total BGP data on wire = 252 * 1.5 million = ~378 MB
BGP data exchanged with transposition of 4 bytes from SRv6 SID into
SRv6 SID TLV:
Consider 200 bytes of shared attributes
CAR SAFI signals SRv6 SID in non-key TLV part of NLRI
Each NLRI size for AFI 1 = 12(key) + 6(SRv6 SID) = 18 bytes
Ideal packing:
Number of NLRIs in 4k update size = 211 (4k-200/18)
Number of update messages of 4k size = 1.5 million/211 = ~7110
Total BGP data on wire = 7110 * 4k = ~29 MB
Practical packing (5 routes in update message):
Size of update message
= (18 * 5) + 236 (including Prefix-SID) = 326
Total BGP data on wire = 326 * 300k = ~98 MB
No-packing case (1 route per update message):
Size of update message
= 12 + 236 (SID in Prefix-SID attribute) = 252
Total BGP data on wire = 252 * 1.5 million = ~378 MBAcknowledgements 🫶
The authors want to give a huge shoutout 📣💜 to the many collaborators who made the BGP CAR solution and this document happen. They brought use cases, brainstormed, joined the mailing list debates, and reviewed the solution and the draft revisions, all of which was seriously appreciated 🙌. Beyond the people in the Contributors section, the authors also thank Robert Raszuk, Bin Wen, Chaitanya Yadlapalli, Satoru Matsushima, Moses Nagarajah, Gyan Mishra, Jorge Rabadan, Daniel Voyer, Stephane Litkowski, Hannes Gredler, Jose Liste, Jakub Horn, Brent Foster, Dave Smith, Jiri Chaloupka, Miya Kohno, Kamran Raza, Zafar Ali, Xing Jiang, Oleksander Nestorov, Peter Psenak, Kaliraj Vairavakkalai, Natrajan Venkataraman, Srihari Sangli, Ran Chen, and Jingrong Xie 🌸✨.
The authors also appreciate the detailed reviews and sharp suggestions from Sue Hares (as document shepherd), Jeff Haas, Yingzhen Qu, and John Scudder 🎀. Their feedback made this document so much better 💅👑.
Contributors ✨
Okay bestie, these are the people who put serious work into this document 🫶 The original says they should be considered coauthors, because their contributions were substantial 💜
- Clarence Filsfils Cisco Systems Belgium Email: cfilsfil@cisco.com
- Bruno Decraene Orange France Email: bruno.decraene@orange.com
- Luay Jalil Verizon United States of America Email: luay.jalil@verizon.com
- Yuanchao Su Alibaba, Inc Email: yitai.syc@alibaba-inc.com
- Jim Uttaro Individual United States of America Email: juttaro@ieee.org
- Jim Guichard Futurewei United States of America Email: james.n.guichard@futurewei.com
- Ketan Talaulikar Cisco Systems India Email: ketant.ietf@gmail.com
- Keyur Patel Arrcus, Inc United States of America Email: keyur@arrcus.com
- Haibo Wang Huawei Technologies China Email: rainsword.wang@huawei.com
- Jie Dong Huawei Technologies China Email: jie.dong@huawei.com
Additional contributors: 🌸
- Dirk Steinberg Lapishills Consulting Limited Germany Email: dirk@lapishills.com
- Israel Means AT&T United States of America Email: im8327@att.com
- Reza Rokui Ciena United States of America Email: rrokui@ciena.com
Authors' Addresses 🦋
And these are the editors who pulled the whole thing together 👑 Two Cisco folks, both marked as editor, so if you have questions about the document, these are the people to ask 💅
- Dhananjaya Rao (editor) Cisco Systems United States of America Email: dhrao@cisco.com
- Swadesh Agrawal (editor) Cisco Systems United States of America Email: swaagraw@cisco.com
That's the whole RFC, bestie! We made it!! 🎀🌈✨
📦 where this page comes from
- ietf: rfc9871 2025-11
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms