RFC 9125

Gateway Auto-Discovery and Route Advertisement for Site Interconnection Using Segment Routing

📅 published August 2021, from ietf

RFC 9125: Gateway Auto-Discovery and Route Advertisement for Site Interconnection Using Segment Routing ✨

okay bestie, let's get you set up 🧚 this is RFC 9125 📜, a Standards Track doc from the IETF, published in August 2021 with ISSN 2070-1721. The authors are A. Farrel (Old Dog Consulting), J. Drake and E. Rosen (both at Juniper Networks), K. Patel (Arrcus, Inc.), and L. Jalil (Verizon). The title block doesn't list any RFCs that it obsoletes or updates 💅

Abstract

okay so data centers hook up to the Internet or a backbone network through gateway routers 🌐 A data center usually has more than one gateway, for commercial reasons, load balancing, and resiliency. Other sites, like access networks, also need gateways to connect across backbone networks 🫶

this doc defines a mechanism that uses the BGP Tunnel Encapsulation attribute ✨ It lets data center gateway routers advertise routes to the prefixes reachable in their site, and that includes advertising on behalf of other gateways at the same site. This is giving segment routing the power to identify multiple paths across the Internet or a backbone between different gateways. Those paths can then be picked for load balancing, resilience, and quality 💖

Status of This Memo

this one is an Internet Standards Track document 🎀

it's a product of the Internet Engineering Task Force (IETF) 🧚 It represents the consensus of the IETF community, got public review, and was approved for publication by the Internet Engineering Steering Group (IESG). For more on Internet Standards, see Section 2 of RFC 7841 📚

for the current status, any errata, and how to give feedback, go to https://www.rfc-editor.org/info/rfc9125 🦋

Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved. 🌸

this document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they're in effect on the date of publication. Please review those docs carefully, because they spell out your rights and restrictions for this document 💅 Code Components pulled from this document must include Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions. They're provided without warranty, as described in the Simplified BSD License 🫶

Table of Contents 📖

1.  Introduction
2.  Requirements Language
3.  Site Gateway Auto-Discovery
4.  Relationship to BGP - Link State and Egress Peer Engineering
5.  Advertising a Site Route Externally
6.  Encapsulation
7.  IANA Considerations
8.  Security Considerations
9.  Manageability Considerations
  9.1.  Relationship to Route Target Constraint
10. References
  10.1.  Normative References
  10.2.  Informative References
Acknowledgements
Authors' Addresses

1. Introduction ✨

okay so data centers (DCs) are a huge deal for network operators, since they're how services get delivered to customers 💖 DCs (sites) are connected by a backbone network made of any number of private networks and/or the Internet 🌐 They attach to that backbone through routers called gateways (GWs). One DC usually has more than one GW, for reasons like commercial preferences, load balancing, or resiliency against a connection or device failing 🍓

Segment Routing (SR) ([RFC8402]) is a protocol mechanism that works inside a DC and also steers traffic flowing between two DC sites 🧚 Here's the thing. A source site (also called an ingress site) that uses SR wants to load-balance the flows it sends to a destination site (also called an egress site). To do that, it needs the full set of entry nodes (aka GWs) for that egress DC, pulled from the backbone network that connects the two DCs 👑 It's assumed that the connected DC sites and the border nodes on the backbone paths between them are part of the same SR BGP - Link State (LS) instance (see [RFC7752] and [RFC9086]). That way, traffic engineering with SR can be used for these flows 🔥

other sites, like access networks, also need to be connected across backbone networks through gateways 🌸 For illustration, picture the ingress and egress sites in Figure 1 as separate Autonomous Systems (ASes). (The sites could also be part of the ASes they're attached to, or be separate ASes. Both work.) The ASes that provide connectivity between the ingress and egress sites can each be built differently and use different technologies, like IP; MPLS using global table routing information from BGP; MPLS IP VPN; SR-MPLS IP VPN; or SRv6 IP VPN. So the ingress and egress sites can be connected by tunnels across a whole variety of technologies 🦋 This document describes how SR Segment Identifiers (SIDs) are used to identify the paths between the ingress and egress sites 💜

the solution here doesn't care whether the transit ASes have SR capabilities or not 🫶 It uses SR to stitch together path segments between GWs and through the Autonomous System Border Routers (ASBRs). So there's a requirement that the GWs and ASBRs are SR capable 💅 If the ingress and egress sites are SR capable, the solution also supports the SR path extending into those sites ✨

the solution in this document can be seen in the bigger picture of site interconnection in [SR-INTERCONNECT] 🎀 That doc shows how other existing protocol elements can be combined with this solution to make a full system. It isn't a required read to understand this document, though 🌈

ok now the drama 😭 Suppose there are two gateways, GW1 and GW2, as shown in Figure 1, for a given egress site. They each advertise a route to prefix X, which lives inside the egress site, with each one setting itself as the next hop. You might think the GWs for X could be inferred from the routes' next-hop fields. But typically, not both routes get distributed across the backbone. Only the best route, as selected by BGP, gets distributed 💀 That means load balancing flows across both GWs is off the table 🙈

        -----------------                    ---------------------
       | Ingress         |                  | Egress     ------   |
       | Site            |                  | Site      |Prefix|  |
       |                 |                  |           |   X  |  |
       |                 |                  |            ------   |
       |       --        |                  |   ---          ---  |
       |      |GW|       |                  |  |GW1|        |GW2| |
        -------++--------                    ----+-----------+-+--
               | \                               |          /  |
               |  \                              |         /   |
               |  -+-------------        --------+--------+--  |
               | ||ASBR|     ----|      |----  |ASBR| |ASBR| | |
               | | ----     |ASBR+------+ASBR|  ----   ----  | |
               | |           ----|      |----                | |
               | |               |      |                    | |
               | |           ----|      |----                | |
               | | AS1      |ASBR+------+ASBR|           AS2 | |
               | |           ----|      |----                | |
               |  ---------------        --------------------  |
             --+-----------------------------------------------+--
            | |ASBR|                                       |ASBR| |
            |  ----               AS3                       ----  |
            |                                                     |
             -----------------------------------------------------

                Figure 1: Example Site Interconnection

the obvious fix for this is to use the BGP feature called Add-Paths ([RFC7911]), which advertises multiple paths in BGP. That way, every route to X gets advertised by BGP 🧚 But even with Add-Paths, the identity of the GWs gets lost as soon as the routes go through an ASBR, since that ASBR sets itself as the next hop 😭 And if there are multiple ASes in the backbone, the next hop changes several times, and Add-Paths runs into scaling issues too. So the Add-Paths approach is effectively limited to sites connected over a single AS 💔

this document defines a solution that gets past that limit and works just as well with a backbone built from one or more ASes, using the Tunnel Encapsulation attribute ([RFC9012]) 👑 Here's the core idea:

When a GW to a given site advertises a route to a prefix X within that site, it will include a Tunnel Encapsulation attribute that contains the union of the Tunnel Encapsulation attributes advertised by each of the GWs to that site, including itself.

so every route a GW advertises identifies all of the GWs to the same site (Section 3 covers how GWs discover each other) 🎀 Basically, the Tunnel Encapsulation attribute from each GW carries multiple Tunnel TLVs, one or more from each active GW. Each Tunnel TLV includes a Tunnel Egress Endpoint sub-TLV that identifies the GW for that Tunnel TLV 🌸 The payoff? Even if only one of the routes gets distributed to other ASes, it doesn't matter how many times the next hop changes, because the Tunnel Encapsulation attribute stays the same 🦋✨

let's run this back with Figure 1 🧚 GW1 and GW2 discover each other as gateways for the egress site. Both advertise themselves as having routes to prefix X. GW1 includes a Tunnel Encapsulation attribute that's the union of its own and GW2's. GW2 does the same thing, combining its own with GW1's 💅 So the gateway in the ingress site can see all the possible paths to X in the egress site, no matter which route got propagated to it. Then it can pick one or balance traffic flows however it sees fit 💖🔥

2. Requirements Language 🎀

the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here 🧚 so only the all-caps version counts as the official BCP 14 vibe 💅

part 2 is coming next, bestie 💖 stay tuned 🦋

3. Site Gateway Auto-Discovery ✨

Okay bestie, so the GWs at a site need to find each other and actually coordinate, right? 🫶 Here's how it goes down:

  • A route target ([RFC4360]) MUST be attached to each GW's auto-discovery route (defined below), and its value MUST be set to a value that indicates the site identifier. The rules for constructing a route target are detailed in [RFC4360]. It is RECOMMENDED that a Type x00 or x02 route target be used. 🎀
  • Site identifiers are set through configuration. The site identifiers MUST be the same across all GWs to the site (so every GW to the same site uses the same identifier) and MUST be unique across all sites that are connected (so across all GWs to all sites that are interconnected). 💅
  • Each GW MUST construct an import filtering rule to import any route that carries a route target with the same site identifier that the GW itself uses. This means only these GWs will import those routes, and all GWs to the same site will import each other's routes and learn (auto-discover) the current set of active GWs for the site. 🦋

The auto-discovery route that each GW advertises consists of the following:

  • IPv4 or IPv6 Network Layer Reachability Information (NLRI) ([RFC4760]) containing one of the GW's loopback addresses. That means an AFI/SAFI pair that is one of these: IPv4/NLRI used for unicast forwarding (1/1); IPv6/NLRI used for unicast forwarding (2/1); IPv4/NLRI with MPLS Labels (1/4); or IPv6/NLRI with MPLS Labels (2/4). 🌸
  • A Tunnel Encapsulation attribute ([RFC9012]) containing the GW's encapsulation information encoded in one or more Tunnel TLVs. 🧚

To avoid the side effect of applying the Tunnel Encapsulation attribute to any packet that is addressed to the GW itself, the address advertised for auto-discovery MUST be a different loopback address than the one advertised for packets directed to the gateway itself. 🙈

As described in Section 1, each GW will include a Tunnel Encapsulation attribute with the GW encapsulation information for each of the site's active GWs (including itself) in every route advertised externally to that site. As the current set of active GWs changes (because a new GW is added or an existing GW fails or is removed), each externally advertised route will be re-advertised with a new Tunnel Encapsulation attribute that reflects the current set of active GWs. 🔥

If a gateway becomes disconnected from the backbone network, or if the site operator decides to terminate the gateway's activity, it MUST withdraw the advertisements described above. This means remote gateways at other sites will stop seeing advertisements from or about this gateway. Note that if the routing within a site is broken (for example, there is a route from one GW to another but not in the reverse direction), then incoming traffic may be routed to the wrong GW to reach the destination prefix. In this degraded network situation, traffic may be dropped. 💀

Heads up: a GW that is (mis)configured with a different site identifier from the other GWs to the same site will not be auto-discovered by those GWs, and it will not auto-discover them either. This would result in a GW for another site receiving only the Tunnel Encapsulation attribute included in the BGP best route, i.e., the Tunnel Encapsulation attribute of the (mis)configured GW or that of the other GWs. 👀

When a remote GW receives a route to a prefix X, it uses the Tunnel Egress Endpoint sub-TLVs in the containing Tunnel Encapsulation attribute to identify the GWs through which X can be reached. It uses this information to compute SR Traffic Engineering (SR TE) paths across the backbone network. It looks at the information advertised to it in SR BGP - Link State (BGP-LS) ([RFC9085]) and correlated using the site identity. SR Egress Peer Engineering (EPE) ([RFC9086]) can be used to supplement the information advertised in BGP-LS. 🧭

5. Advertising a Site Route Externally 📣

When a packet destined for prefix X is sent on an SR TE path to a GW for the site containing X (that is, the packet is sent in the ingress site on an SR TE path that describes the whole path, including the parts within the egress site), it needs to carry the receiving GW's SID for X. That SID becomes the next SID due to be processed before the GW completes its processing of the packet. To achieve this, each Tunnel TLV in the Tunnel Encapsulation attribute contains a Prefix-SID sub-TLV ([RFC9012]) for X. 💖

As defined in [RFC9012], the Prefix-SID sub-TLV is only for IPv4/IPV6 Labeled Unicast routes, so the solution described in this document only applies to routes of those types. If the use of the Prefix-SID sub-TLV for routes of other types is defined in the future, further documents will be needed to describe their use for site interconnection consistent with this document. 🙌

Alternatively, if MPLS SR is in use, and if the GWs for a given egress site are configured to allow GWs at remote ingress sites to perform SR TE through that egress site for a prefix X, then each GW to the egress site computes an SR TE path through the egress site to X. Each path is placed in an MPLS Label Stack sub-TLV ([RFC9012]) in the SR Tunnel TLV for that GW. ✨

Please refer to Section 7 of [SR-INTERCONNECT] for worked examples of how the SID stack is constructed in this case and how the advertisements would work. 📚

6. Encapsulation 🍓

If a site is configured to allow remote GWs to send packets to the site in the site's native encapsulation, then each GW to the site will also include multiple instances of a Tunnel TLV for that native encapsulation in externally advertised routes: one for each GW. Each Tunnel TLV contains a Tunnel Egress Endpoint sub-TLV with the address of the GW that the Tunnel TLV identifies. A remote GW may then encapsulate a packet according to the rules defined via the sub-TLVs included in each of the Tunnel TLVs. 🎁

7. IANA Considerations 🗂️

IANA maintains the "BGP Tunnel Encapsulation Attribute Tunnel Types" registry in the "Border Gateway Protocol (BGP) Tunnel Encapsulation" registry.

IANA had previously assigned the value 17 from this subregistry for "SR Tunnel", referencing this document as an Internet-Draft. At that time, the assignment policy for this range of the registry was "First Come First Served" [RFC8126].

IANA has marked that assignment as deprecated. IANA may reclaim that codepoint at such a time that the registry is depleted. 🥀

8. Security Considerations 🔐

From a protocol point of view, the mechanisms described in this document can leverage the security mechanisms already defined for BGP. Further discussion of security considerations for BGP may be found in the BGP specification itself ([RFC4271]) and in the security analysis for BGP ([RFC4272]). The original discussion of the use of the TCP MD5 signature option to protect BGP sessions is found in [RFC5925], while [RFC6952] includes an analysis of BGP keying and authentication issues. 🛡️

The mechanisms described in this document involve sharing routing or reachability information between sites, which may mean disclosing information that is normally contained within a site. So it needs to be understood that normal security paradigms based on the boundaries of sites are weakened, and interception of BGP messages may result in information being disclosed to third parties. Discussion of these issues with respect to VPNs can be found in [RFC4364], while [RFC7926] describes many of the issues associated with the exchange of topology or TE information between sites. 😱

Particular exposures resulting from this work include:

  • Gateways to a site will know about all other gateways to the same site. This feature applies within a site, so it is not a substantial exposure. However, it does mean that if the BGP exchanges within a site can be snooped, or if a gateway can be subverted, then an attacker may learn the full set of gateways to a site. This would facilitate more effective attacks on that site. 🕵️
  • The existence of multiple gateways to a site becomes more visible across the backbone and even into remote sites. This means an attacker is able to prepare a more comprehensive attack than exists when only the locally attached backbone network (e.g., the AS that hosts the site) can see all of the gateways to a site. For example, a Denial-of-Service attack on a single GW is mitigated by the existence of other GWs, but if the attacker knows about all the gateways, then the whole set can be attacked at once. 🚨
  • A node in a site that does not have external BGP peering (i.e., is not really a site gateway and cannot speak BGP into the backbone network) may be able to get itself advertised as a gateway by letting other genuine gateways discover it (by speaking BGP to them within the site). It may then get those genuine gateways to advertise it as a gateway into the backbone network. This would allow the malicious node to attract traffic without having to have secure BGP peerings with out-of-site nodes. 👾
  • An external party intercepting BGP messages anywhere between sites may learn information about the functioning of the sites and the locations of endpoints. While this is not necessarily a significant security or privacy risk, it is possible that the disclosure of this information could be used by an attacker. 🫣
  • If it is possible to modify a BGP message within the backbone, it may be possible to spoof the existence of a gateway. This could cause traffic to be attracted to a specific node and might result in traffic not being delivered. 🥺

All of the issues in the list above could cause disruption to site interconnection. They are not new protocol vulnerabilities so much as new exposures of information that SHOULD be protected against using existing protocol mechanisms, such as securing the TCP sessions over which the BGP messages flow. Furthermore, it is a general observation that if these attacks are possible, then it is highly likely that far more significant attacks can be made on the routing system. It should be noted that BGP peerings are not discovered but always arise from explicit configuration. 🔑

Given that the gateways and ASBRs are connected by tunnels that may run across parts of the network that are not trusted, data center operators using the approach set out in this network MUST consider using gateway-to-gateway encryption to protect the data center traffic. Additionally, due consideration MUST be given to encrypting end-to-end traffic, as it would be for any traffic that uses a public or untrusted network for transport. 🔒💜

9. Manageability Considerations 🎀

okay bestie, the big config thing here is picking a site identifier 💖. Every GW in the same site MUST get the same identifier, and each site MUST have its own different one. So yes, it needs coordination, probably through some central management agent 💅.

It should be noted that BGP peerings are not discovered but always arise from explicit configuration. This is no different from any other BGP operation 🧚.

The site identifiers that are configured and carried in route targets (see Section 3) are a super important feature, because they make sure all the gateways to a site discover each other 🫶. So it's really important that this value isn't misconfigured, because that would mean the gateways don't discover each other and don't advertise each other 😭.

9.1. Relationship to Route Target Constraint

okay so, to limit the VPN routing info a route reflector has to keep track of, [RFC4364] suggests that route reflectors use "Cooperative Route Filtering". That got renamed "Outbound Route Filtering" and gets defined in [RFC5291] 🦋. [RFC4684] then extends that mechanism to support multiple autonomous systems and asymmetric VPN topologies like hub-and-spoke 👑. The mechanism in RFC 4684 is known as Route Target Constraint (RTC).

An operator would not normally configure RTC by default for any AFI/SAFI combination, and would only turn it on after careful consideration 🙈. When using the mechanisms in this document, the operator should carefully consider the effects of filtering routes. Sometimes that's desirable, and other times it could limit how well the procedures work 💀.

10. References

10.1. Normative References

10.2. Informative References

Acknowledgements

big thanks to Bruno Rijsman, Stephane Litkowski, Boris Hassanov, Linda Dunbar, Ravi Singh, and Daniel Migault for the review comments 🌸, and to Robert Raszuk for the useful discussions 💜. Gyan Mishra did a helpful GenArt review, and John Scudder and Benjamin Kaduk made helpful comments during IESG review 🫶.

Authors' Addresses

  • Adrian Farrel, Old Dog Consulting, Email: adrian@olddog.co.uk
  • John Drake, Juniper Networks, Email: jdrake@juniper.net
  • Eric Rosen, Juniper Networks, Email: erosen52@gmail.com
  • Keyur Patel, Arrcus, Inc., Email: keyur@arrcus.com
  • Luay Jalil, Verizon, Email: luay.jalil@verizon.com
📦 where this page comes from