RFC 7947
Internet Exchange BGP Route Server
Internet Exchange BGP Route Server 🦋
okay bestie, this is RFC 7947 and it's giving "how do 500 networks interconnect without everyone needing a session with everyone else" energy 💅✨ It's an IETF Standards Track document, published in September 2016 (ISSN 2070-1721). The authors are E. Jasinska (BigWave IT), N. Hilliard (INEX), R. Raszuk (Bloomberg LP), and N. Bakker (Akamai Technologies B.V.). It doesn't list anything it obsoletes or updates, so it's standing on its own 🌸
Abstract
okay so the abstract is the quick tour 🎀 This doc lays out a spec for multilateral interconnections at Internet Exchange Points (IXPs). Multilateral interconnection means swapping routing information among three or more External BGP (EBGP) speakers through one middleman called a route server. Route servers are typically used on shared access media networks, like IXPs, to make interconnecting lots of Internet routers way simpler 💖
Status of This Memo
This is an Internet Standards Track document.
This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 7841.
Info about the current status of this document, any errata, and how to send feedback lives at http://www.rfc-editor.org/info/rfc7947 🔗
Copyright Notice
Copyright (c) 2016 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.
Table of Contents
1. Introduction to Multilateral Interconnection . . . . . . . . 3
1.1. Notational Conventions . . . . . . . . . . . . . . . . . 3
2. Technical Considerations for Route Server Implementations . . 4
2.1. Client UPDATE Messages . . . . . . . . . . . . . . . . . 4
2.2. Attribute Transparency . . . . . . . . . . . . . . . . . 4
2.2.1. NEXT_HOP Attribute . . . . . . . . . . . . . . . . . 4
2.2.2. AS_PATH Attribute . . . . . . . . . . . . . . . . . . 5
2.2.2.1. Route Server AS_PATH Management . . . . . . . . . 5
2.2.2.2. Route Server client AS_PATH Management . . . . . 5
2.2.3. MULTI_EXIT_DISC Attribute . . . . . . . . . . . . . . 5
2.2.4. Communities Attributes . . . . . . . . . . . . . . . 5
2.3. Per-Client Policy Control in Multilateral Interconnection 6
2.3.1. Path Hiding on a Route Server . . . . . . . . . . . . 6
2.3.2. Mitigation of Path Hiding . . . . . . . . . . . . . . 7
2.3.2.1. Multiple Route Server RIBs . . . . . . . . . . . 7
2.3.2.2. Advertising Multiple Paths . . . . . . . . . . . 8
2.3.3. Implementation Suggestions . . . . . . . . . . . . . 9
3. Security Considerations . . . . . . . . . . . . . . . . . . . 9
4. References . . . . . . . . . . . . . . . . . . . . . . . . . 10
4.1. Normative References . . . . . . . . . . . . . . . . . . 10
4.2. Informative References . . . . . . . . . . . . . . . . . 10
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . 11
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 121. Introduction to Multilateral Interconnection
okay so IXPs give their participants IP data interconnection, usually over shared Layer 2 media like Ethernet 🧚 The Border Gateway Protocol (BGP) [RFC4271], an inter-Autonomous System (inter-AS) routing protocol, is the usual tool for swapping network reachability info over that kind of media.
Bilateral EBGP sessions between exchange participants used to be the most common way to do this. But the overhead of dense interconnection can cause seriously big operational scaling problems for participants at larger IXPs 😭
Multilateral interconnection is a way for BGP-speaking routers to interconnect through a third-party broker, usually called a route server and typically run by the IXP operator 💜 Each participant (usually called a "route server client") announces its network reachability info to the route server using EBGP. The route server then forwards that info to each route server client connected to it, according to its configuration. A route server uses BGP to trade reachability info with each of its clients, but it does not forward traffic itself, so it is not a router 🙈
A route server is kind of similar in function to a route reflector [RFC4456], except it operates using EBGP instead of Internal BGP (IBGP). Certain adaptions to [RFC4271] are required so an EBGP router can operate as a route server, and these are outlined in Section 2 of this document. Route server functionality is not mandatory in BGP implementations 🎀
The term "route server" gets used in a different context too, for a BGP node that takes BGP feeds from multiple clients for operational analysis and troubleshooting. That kind of system may also be known as a "route collector" or a "route-views server". This document uses the term "route server" exclusively for multilateral peering brokerage systems, so no mix-ups 💅
1.1. Notational Conventions
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119].
2. Technical Considerations for Route Server Implementations
so a route server uses BGP [RFC4271] to broker network reachability info amongst its clients 🦋 It behaves a bit differently from a BGP implementation that is strictly compliant with [RFC4271]. Those differences are described below, bestie 💖
2.1. Client UPDATE Messages
A route server MUST accept all UPDATE messages received from each of its clients for inclusion in its Adj-RIB-In. These UPDATE messages MAY be omitted from the route server's Loc-RIB or Loc-RIBs, due to filters configured for the purpose of implementing routing policy. The route server SHOULD perform one or more BGP Decision Processes to select routes for subsequent advertisement to its clients, taking into account possible configuration to provide multiple Network Layer Reachability Information (NLRI) paths to a particular client as described in Section 2.3.2.2 or multiple Loc-RIBs as described in Section 2.3.2.1. The route server SHOULD forward UPDATE messages from its Loc-RIB or Loc-RIBs to its clients as determined by local policy ✨
2.2. Attribute Transparency
okay this one is the main vibe 👑 A route server mostly does brokering, so if it changes attributes, route server clients might change their BGP Decision Process for the prefix reachability info they got. That would mess with the routing policies the exchange participants intended 😬 So, contrary to what Section 5 of [RFC4271] specifies, route servers SHOULD NOT by default (unless explicitly configured) update well-known BGP attributes received from route server clients before passing them on to their other route server clients. Optional recognized and unrecognized BGP attributes, whether transitive or non-transitive, SHOULD NOT be updated by the route server (unless enforced by local IXP operator configuration), and SHOULD be passed on to other route server clients 🫶
2.2.1. NEXT_HOP Attribute
The NEXT_HOP is a well-known mandatory BGP attribute that defines the IP address of the router used as the next hop to the destinations listed in the NLRI field of the UPDATE message. Since the route server doesn't take part in actually routing traffic, the NEXT_HOP attribute MUST be passed unmodified to the route server clients, similar to the "third-party" next-hop feature described in Section 5.1.3. of [RFC4271] 🌈
2.2.2. AS_PATH Attribute
AS_PATH is a well-known mandatory attribute that identifies the ASes through which routing information carried in the UPDATE message has passed 🍓
2.2.2.1. Route Server AS_PATH Management
A route server doesn't take part in forwarding data between client routers. Modifying the AS_PATH attribute could affect the route server client BGP Decision Process, so the route server SHOULD NOT prepend its own AS number to the AS_PATH segment, nor modify the AS_PATH segment in any other way 🙅♀️ This is different from Section 5.1.2 of [RFC4271], which requires that the BGP speaker prepend its own AS number as the last element of the AS_PATH segment. This is a recommendation rather than a requirement, purely to stay backwards compatible with legacy route server client implementations that don't yet support the requirements in Section 2.2.2.2 🎀
2.2.2.2. Route Server client AS_PATH Management
Contrary to what Section 6.3 of [RFC4271] recommends, route server clients need to be able to accept UPDATE messages where the leftmost AS in the AS_PATH attribute is not equal to the AS number of the route server that sent the UPDATE message 🌸 If the route server client BGP system has implemented a check for this, the BGP implementation MUST allow this check to be disabled, and SHOULD allow the check to be disabled on a per-peer basis 🔥
2.2.3. MULTI_EXIT_DISC Attribute
MULTI_EXIT_DISC is an optional non-transitive attribute meant for external (inter-AS) links, to tell apart multiple exit or entry points to the same neighboring AS 🦋 Contrary to Section 5.1.4 of [RFC4271], if it's applied to an NLRI UPDATE sent to a route server, this attribute SHOULD be propagated to other route server clients, and the route server SHOULD NOT modify its value 💖
2.2.4. Communities Attributes
The BGP Communities [RFC1997] and Extended Communities [RFC4360] attributes are meant for labeling info carried in BGP UPDATE messages 🧚 Transitive and non-transitive Communities attributes applied to an NLRI UPDATE sent to a route server SHOULD NOT be modified, processed, or removed, except as defined by local policy. If a Communities attribute is meant for processing by the route server itself, as determined by local policy, it MAY be modified or removed 💅
That's the first chunk of RFC 7947 done, bestie 🌈✨
2.3. Per-Client Policy Control in Multilateral Interconnection ✨
okay so bestie, IXP participants usually spin up a route server because they want to interconnect with literally everyone else on it 🌸 but sometimes you need finer control over who gets which paths, per client, so your interconnection policies actually hold up 💅
when you do that, a path can end up hidden from a route server client. we call this "path hiding" 🙈
Neither Section 2.3 nor its subsections form part of the normative specification of this document; they are included for information purposes only. (ngl this part is just for context, not the official rules 📖✨)
2.3.1. Path Hiding on a Route Server 🦋
___ ___
/ \ / \
..| AS1 |..| AS2 |..
: \___/ \___/ :
: \ / | :
: \ / | :
: IXP \/ | :
: /\ | :
: / \ | :
: ___/____\_|_ :
: / \ / \ :
..| AS3 |..| AS4 |..
\___/ \___/Figure 1: Per-Client Policy Controlled Interconnection at an IXP
okay look at Figure 1 bestie 👀 AS1 doesn't directly swap prefix info with AS2 or AS3 at the IXP. it only interconnects with AS4 🫶 the lines between AS1, AS2, AS3, and AS4 are interconnection relationships, whether they're bilateral or multilateral 🍓
so in the traditional bilateral model, per-client policy control toward a third-party participant happens either by not interconnecting bilaterally with them at all, or by doing outbound filtering on the BGP session toward them 💜 but in a multilateral setup, only the route server can do outbound filtering toward its route server clients. the clients literally have to rely on the route server to do their outbound filtering for them 😭
assuming the BGP Decision Process [RFC4271] is used, when the same prefix shows up at a route server from multiple route server clients, the route server picks a single path to send to all connected clients 💅 but if the route server has been set up to filter its calculated best path from reaching a particular client, that client gets no path for that prefix, even if the alternate paths the route server received would have been policy compliant for them 🥺 this is what we call "path hiding" 🙈
for example, in Figure 1, say the same prefix is sent to the route server via AS2 and AS4, and the route via AS2 wins the BGP Decision Process on the route server. but AS2's policy stops the route server from sending that path to AS1. so AS1 never gets a path to this prefix, even though the route server already got a valid alternative via AS4 💀 this happens because the BGP Decision Process only runs once on the route server for all clients 🎀
path hiding only happens on route servers that do per-client policy control! if an IXP operator runs a route server without a per-client routing policy control system, path hiding doesn't occur, because all paths are considered equally valid from the route server's point of view ✨
2.3.2. Mitigation of Path Hiding 🌈
there are several approaches you can take to mitigate path hiding 💖
2.3.2.1. Multiple Route Server RIBs 🧚
the most portable way to get per-client policy control without path hiding is to use a route server BGP implementation that runs the per-client best path calculation for each set of paths to a prefix, after the route server's client policies have been applied 👑 this can be done with per-client Loc-RIBs, with path filtering happening between the Adj-RIB-In and the per-client Loc-RIB 🫶 implementations can optimize this by keeping paths that aren't subject to filtering policies in a global Loc-RIB, with the per-client Loc-RIBs stored as deltas 💜
this one is super portable because it makes no assumptions about the feature capabilities of the route server clients 🦋
2.3.2.2. Advertising Multiple Paths 🎀
the path distribution model we just talked about assumes standard BGP session encoding, where the route server sends a single path to its client for any given prefix 🥺 that path is selected with the BGP path selection Decision Process described in [RFC4271] 💅 but if the route server could send more than one path to a route server client, the clients wouldn't have to depend on receiving a single path for a prefix anymore 🙌 so the path-hiding problem from Section 2.3.1 would just disappear ✨
we present two methods for how this extra path diversity could be implemented 🌸
2.3.2.2.1. Diverse BGP Path Approach 🍓
the diverse BGP path proposal defined in [RFC6774] is a simple way to send multiple prefix paths from a route server to a route server client, by using a separate BGP session from the route server to the client for each different path 💖
the number of paths you can send to a client is limited by how many BGP sessions the server and the client are willing to set up with each other 😭 the distributed paths can come from the global BGP Loc-RIB on the route server as well as any per-client Loc-RIB 🧚 and since there may be more potential paths to a given prefix than configured BGP sessions, this method is not guaranteed to eliminate path hiding in all situations 💀 plus, it may significantly increase the number of BGP sessions the route server has to handle, which can negatively impact its performance 🥲
2.3.2.2.2. BGP ADD-PATH Approach 🔥
[RFC7911] takes a different approach to multiple path propagation, letting a BGP speaker forward multiple paths for the same prefix over a single BGP session 🦋 but here's the catch: [RFC4271] says a BGP listener must perform an implicit withdraw when it receives an UPDATE message for a prefix that already exists in its Adj-RIB-In, so this approach needs explicit support for the feature on both the route server and its clients 👑
if the ADD-PATH capability is negotiated in both directions between the route server and a route server client, and the client sends multiple paths for the same prefix to the route server, then inactive, invalid, or suboptimal paths could end up getting propagated to the route server 😱 that could cause loss of reachability to other route server clients 💔 for this reason, ADD-PATH implementations on a route server should enforce a send-only mode with the route server clients, which results in negotiating a receive-only mode from the client to the route server 🛡️✨
2.3.3. Implementation Suggestions 💅
authors of route server implementations may wish to consider one of the methods in Section 2.3.2 to allow per-client route server policy control without path hiding 🫶
recommendations for route server operations are described separately in [RFC7948] 📖🎀
3. Security Considerations 🚨
okay this one is serious bestie 😭🔥 the path-hiding problem from Section 2.3.1 can be abused in certain circumstances to proactively block third-party path announcements from other route server clients 💀 route server operators need to be aware that security issues may come up unless they take steps to mitigate path hiding 🛡️
the AS_PATH check described in Section 2.2.2 is normally enabled to catch malformed AS paths 🔍 if you disable this check, the route server client loses the ability to check incoming UPDATE messages for certain categories of problems 😱 that could let corrupted BGP UPDATE messages get propagated when they wouldn't be if the check were on 💔 and regardless of malformed UPDATE messages, this check is also used to detect BGP loops, so removing it could cause routing loops to form 🌀
so this check SHOULD NOT be disabled by IXP participants unless it's needed to establish BGP sessions with a route server, and if possible it should only be disabled for peers that are route servers 🙏💜
route server operators should carefully consider the security practices discussed in "BGP Operations and Security" [RFC7454] 🔐✨
4. References 📚
4.1. Normative References 💜
[RFC1997] Chandra, R., Traina, P., and T. Li, "BGP Communities Attribute", RFC 1997, DOI 10.17487/RFC1997, August 1996, <http://www.rfc-editor.org/info/rfc1997>.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, <http://www.rfc-editor.org/info/rfc2119>.
[RFC4271] Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A Border Gateway Protocol 4 (BGP-4)", RFC 4271, DOI 10.17487/RFC4271, January 2006, <http://www.rfc-editor.org/info/rfc4271>.
[RFC4360] Sangli, S., Tappan, D., and Y. Rekhter, "BGP Extended Communities Attribute", RFC 4360, DOI 10.17487/RFC4360, February 2006, <http://www.rfc-editor.org/info/rfc4360>.
4.2. Informative References 🦋
[RFC1863] Haskin, D., "A BGP/IDRP Route Server alternative to a full mesh routing", RFC 1863, DOI 10.17487/RFC1863, October 1995, <http://www.rfc-editor.org/info/rfc1863>.
[RFC4223] Savola, P., "Reclassification of RFC 1863 to Historic", RFC 4223, DOI 10.17487/RFC4223, October 2005, <http://www.rfc-editor.org/info/rfc4223>.
[RFC4456] Bates, T., Chen, E., and R. Chandra, "BGP Route Reflection: An Alternative to Full Mesh Internal BGP (IBGP)", RFC 4456, DOI 10.17487/RFC4456, April 2006, <http://www.rfc-editor.org/info/rfc4456>.
[RFC6774] Raszuk, R., Ed., Fernando, R., Patel, K., McPherson, D., and K. Kumaki, "Distribution of Diverse BGP Paths", RFC 6774, DOI 10.17487/RFC6774, November 2012, <http://www.rfc-editor.org/info/rfc6774>.
[RFC7454] Durand, J., Pepelnjak, I., and G. Doering, "BGP Operations and Security", BCP 194, RFC 7454, DOI 10.17487/RFC7454, February 2015, <http://www.rfc-editor.org/info/rfc7454>.
[RFC7911] Walton, D., Retana, A., Chen, E., and J. Scudder, "Advertisement of Multiple Paths in BGP", RFC 7911, DOI 10.17487/RFC7911, July 2016, <http://www.rfc-editor.org/info/rfc7911>.
[RFC7948] Hilliard, N., Jasinska, E., Raszuk, R., and N. Bakker, "Internet Exchange BGP Route Server Operations", RFC 7948, DOI 10.17487/RFC7948, September 2016, <http://www.rfc-editor.org/info/rfc7948>.
Acknowledgments 🌸
okay bestie, the authors gave a shoutout to some legends 🫶 Huge thanks to Ryan Bickhart, Steven Bakker, Martin Pels, Chris Hall, Aleksi Suhonen, Bruno Decraene, Pierre Francois, and Eduardo Ascenco Reis for their valuable input 💖
They also acknowledged the developers of BIRD, OpenBGPD, Quagga, and IOS 👑 Their BGP implementations include route server capabilities that are compliant with this document, so they're literally the reason this stuff works in the real world ✨
Route server functionality was first described in 1995 in [RFC1863] 🦋 Modern route server implementations are based on ideas from the 1990s, when the Routing Arbiter Project and the Route Server Next Generation (RSNG) Project, managed by ISI and Merit, were cooking 🍓 The original RSNG code isn't used at any IXPs anymore, but the IXP community owes a debt of gratitude to everyone who helped build route servers back then 🙏 Also, pls note that [RFC1863] was made historical by [RFC4223] 🥀
Authors' Addresses 💅
Here's where you can find the authors, bestie 🎀
Elisa Jasinska
BigWave IT
ul. Skawinska 27/7
Krakow, MP 31-066
Poland
Email: elisa@bigwaveit.orgNick Hilliard
INEX
4027 Kingswood Road
Dublin 24
Ireland
Email: nick@inex.ieRobert Raszuk
Bloomberg LP
731 Lexington Ave
New York City, NY 10022
United States of America
Email: robert@raszuk.netNiels Bakker
Akamai Technologies B.V.
Kingsfordweg 151
Amsterdam 1043 GR
Netherlands
Email: nbakker@akamai.comThat's the whole RFC, babe! 🦋✨ Every one of these people put serious brainpower into route servers, so go send them a 💜 if you ever see them at an IXP meeting 🧚
📦 where this page comes from
- ietf: rfc7947 2016-09
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms