RFC 4451

BGP MULTI_EXIT_DISC (MED) Considerations

📅 published March 2006, from ietf

RFC 4451: BGP MULTI_EXIT_DISC (MED) Considerations 🦋

(Part 1 of 3 ✨ more is coming, bestie)

  • RFC number: 4451 💖
  • Stream: Network Working Group
  • Authors: D. McPherson (Arbor Networks, Inc.) and V. Gill (AOL) 🫶
  • Category: Informational 📚
  • Date: March 2006 🗓️
  • Obsoletes / Updates: the title block doesn't list anything this one obsoletes or updates 🤷‍♀️

Status of This Memo 🌸

Okay so this memo is just here to give info to the Internet community. It is NOT a standard of any kind, no cap. Distribution is totally unlimited, so share it with everyone 💅

Copyright (C) The Internet Society (2006).

Abstract 🔥

The MED attribute is how BGP speakers tell an adjacent AS which entry point into their local AS is the optimal one. Honestly, MEDs work just fine in a lot of setups, but they can get messy when your topology is dynamic or complicated 🙈 This doc talks about both the implementation side and the deployment side of MEDs, and it's basically a must-read for anyone building or running this stuff ✨

Table of Contents 📑

   1. Introduction ....................................................3
   2. Specification of Requirements ...................................3
      2.1. About the MULTI_EXIT_DISC (MED) Attribute ..................3
      2.2. MEDs and Potatoes ..........................................5
   3. Implementation and Protocol Considerations ......................6
      3.1. MULTI_EXIT_DISC Is an Optional Non-Transitive Attribute ....6
      3.2. MED Values and Preferences .................................6
      3.3. Comparing MEDs between Different Autonomous Systems ........7
      3.4. MEDs, Route Reflection, and AS Confederations for BGP ......7
      3.5. Route Flap Damping and MED Churn ...........................8
      3.6. Effects of MEDs on Update Packing Efficiency ...............9
      3.7. Temporal Route Selection ...................................9
   4. Deployment Considerations ......................................10
      4.1. Comparing MEDs between Different Autonomous Systems .......10
      4.2. Effects of Aggregation on MEDs ............................11
   5. Security Considerations ........................................11
   6. Acknowledgements ...............................................11
   7. References .....................................................12
      7.1. Normative References ......................................12
      7.2. Informative References ....................................12

1. Introduction 🧚

So MEDs let BGP speakers tell an adjacent AS which entry point into the local AS is the best one, and that's the whole vibe 💜 MEDs are fine in a bunch of scenarios, but things can get chaotic when the topology is dynamic or complex 😭

Heads up while you read: this doc covers both implementation and deployment considerations for BGP MEDs. It's written for implementors and network operators to be familiar with. Sometimes the advice for implementors and the advice for operators is different, and that's totally normal ✨

2. Specification of Requirements 📜

Okay, the important words first. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119].

2.1. About the MULTI_EXIT_DISC (MED) Attribute 💅

Fun fact: the MULTI_EXIT_DISC (MED) attribute used to be called INTER_AS_METRIC. It's defined right now in section 5.1.4 of [BGP4], and the core rules go like this:

MULTI_EXIT_DISC is an optional, non-transitive attribute meant for external (inter-AS) links, where it helps tell apart several exit or entry points into the same neighboring AS. Its value is a four-octet unsigned number called a metric. Everything else being equal, the exit point with the lower metric SHOULD be preferred 🔥 If a route comes in over External BGP (EBGP), the MULTI_EXIT_DISC MAY get passed along over Internal BGP (IBGP) to other BGP speakers in the same AS (see also 9.1.2.2). But a MULTI_EXIT_DISC received from a neighboring AS MUST NOT be passed on to other neighboring ASes 🙅‍♀️

Every BGP speaker MUST have a mechanism, driven by local configuration, that lets it strip the MULTI_EXIT_DISC off a route. If a speaker is set up to do that, the removal MUST happen before the route's degree of preference gets worked out and before route selection (Decision Process phases 1 and 2) runs 💖

An implementation MAY also, based on local configuration, change the value of a MULTI_EXIT_DISC received over EBGP. If it's configured to do that, the change MUST happen before the route's preference is worked out and before route selection (Decision Process phases 1 and 2). Section 9.1.2.2 has the necessary restrictions on this 🧐

Over in Section 9.1.2.2 (c) of [BGP4], the route selection criteria for MEDs go like this 👑 Drop any routes that have a less-preferred MULTI_EXIT_DISC. MULTI_EXIT_DISC can only be compared between routes learned from the same neighboring AS, and that neighboring AS is worked out from the AS_PATH attribute. A route with no MULTI_EXIT_DISC at all counts as having the lowest possible value.

The same idea is written up as a procedure too:

for m = all routes still under consideration
    for n = all routes still under consideration
        if (neighborAS(m) == neighborAS(n)) and (MED(n) < MED(m))
            remove route m from consideration

Quick decoder ring for that pseudo-code 🔍 MED(n) gives back the value of route n's MULTI_EXIT_DISC. If route n doesn't have one, you get the lowest possible value, which is 0 (i.e., 0).

neighborAS(n) gives back the neighbor AS the route came from. If the route came over IBGP and the other IBGP speaker didn't originate it, that's the neighbor AS the other speaker learned it from. If it came over IBGP and the other speaker either (a) originated it, or (b) built it by aggregation with an AS_PATH that's empty or starts with an AS_SET, then it's the local AS 🫧

Now the spicy part 🌶️ If you strip a MULTI_EXIT_DISC before re-advertising a route into IBGP, comparing against the EBGP-received MULTI_EXIT_DISC MAY still happen. If an implementation decides to strip it, the optional MULTI_EXIT_DISC comparison, when it happens, MUST be done only among EBGP-learned routes. The best EBGP-learned route can then be compared against IBGP-learned routes once the attribute is gone. If the MULTI_EXIT_DISC is stripped from only some EBGP-learned routes, and the chosen "best" EBGP route won't have had it stripped, then the MULTI_EXIT_DISC must be used when comparing with IBGP-learned routes. For IBGP-learned routes, the MULTI_EXIT_DISC MUST be used in any route comparison that reaches this step of the Decision Process. Bringing an EBGP route's MULTI_EXIT_DISC into a comparison with an IBGP route, then stripping it and advertising the route, has been proven to cause route loops 💀

2.2. MEDs and Potatoes 🥔

Picture this: traffic flows between a pair of hosts, and each host is connected to a different transit network. Those two transit networks are themselves interconnected at two or more locations 🌍 Each transit network gets to choose between two moves. It can send traffic to the closest peering point toward the adjacent transit network, or it can pass traffic to whichever interconnection advertises the least-cost path to the destination host.

The first move is called "hot potato routing" (or closest-exit) 🔥 Think about a hot potato in your bare hands: whoever's holding it wants to get rid of it fast. Hot potato routing happens when you don't pass the EBGP-learned MULTI_EXIT_DISC into IBGP, and it keeps transit traffic low for the provider doing the routing. The other move, "cold potato routing" (or best-exit), is way less common 🧊 Here the transit provider uses its own transit capacity to carry the traffic to the point the adjacent transit provider advertised as closest to the destination. Cold potato routing happens when you do pass the EBGP-learned MULTI_EXIT_DISC into IBGP.

If one transit provider does hot potato and the other does cold potato, traffic between the two tends to be more symmetric 🦋 But if both providers do cold potato, or both do hot potato, between their networks, you're likely to see a bigger amount of asymmetry.

It depends on the business relationships. If one provider has more capacity, or a much less congested backbone, that provider might go with cold potato 👑 A famous example of cold potato routing being used widely was the NSF-funded NSFNET backbone and the NSF-funded regional networks in the mid-1990s.

Sometimes a provider does hot potato for some destinations and cold potato for others, for a given peer AS 🤹‍♀️ The mid-1990s NSFNET is a great example here too, since it treated commercial and research traffic differently. These days, a lot of commercial networks exchange MEDs with their customers but not with their bilateral peers. Commercial use of MEDs really varies, from everywhere to nowhere at all.

Also, lots of MED deployments today are probably behaving differently than the operator intended (for example, giving sub-optimal routing) 😩 So you don't end up with hot or cold potatoes, you end up with mashed potatoes! 🥔💀 More on this unintended behavior shows up throughout the rest of the document.

3. Implementation and Protocol Considerations 🛠️

Heads up, there are a bunch of implementation and protocol quirks relating to MEDs that people have discovered, and they can affect how the network behaves 🙈 The next sections give you the details on these issues.

3.1. MULTI_EXIT_DISC Is an Optional Non-Transitive Attribute 🎀

MULTI_EXIT_DISC is a non-transitive optional attribute, and whether it gets advertised to IBGP and EBGP peers is up to the implementation. Because of that, some implementations send MEDs to IBGP peers by default, and others don't 🌈 This can lead to sub-optimal route selection inside an AS, which is not the vibe. On top of that, some implementations send MEDs to EBGP peers by default and some don't, and that can lead to sub-optimal inter-domain route selection too 😭

3.2. MED Values and Preferences 🎀

Okay so some implementations think an MED of zero is less preferable than having no MED at all 💀. That caused path selection drama inside an AS, which is so not it. The current BGP spec [BGP4] cleared up the ambiguity that existed in [RFC1771] by saying that if route n has no MULTI_EXIT_DISC attribute, the lowest possible MULTI_EXIT_DISC value (i.e., 0) should be assigned to the attribute ✨.

Honestly, everyone has been all over the map about missing-MED. Earlier versions of the spec wanted a missing MED to get the highest possible MED value (i.e., 2^32-1) 🙈.

It gets messier. Some implementations secretly use the maximum possible MED value (2^32-1) as an "infinity" metric 🚫, so they can tag routes as unfeasible. When they receive an update with an MED value of 2^32-1, they rewrite it to 2^32-2 😭. That new value then gets propagated and can cause routing inconsistencies or unintended path selections. Not a vibe.

Because of all these implementation inconsistencies and protocol revision variances, lots of network operators now explicitly reset all MED values on ingress 🧹. They set them to zero or some other 'fixed' value, and their internal routing policy requires that MED values of 0 and 2^32-1 not be used in configurations, whether the MEDs are computed directly or configured. That way they don't have to rely on every router having the same missing-MED behaviour 💅.

Here's the thing: implementations don't normally give you a way to disable MED comparisons in the decision algorithm. So "not using MEDs" usually means explicitly setting all MEDs to some fixed value on ingress to the routing domain 🫶. When you assign one fixed MED value consistently to all routes across the network, MEDs are effectively a non-issue in the decision algorithm. Main character energy for your routing policy 👑.

3.3. Comparing MEDs between Different Autonomous Systems 🌸

The MED was originally meant for external (inter-AS) links, to tell apart multiple exit or entry points to the same neighbouring AS 🦋. But a huge number of MED applications now use MEDs to decide route preference between like routes received from different autonomous systems. Bestie, that's a whole different plot 😳.

A lot of implementations let you turn on MED comparison between routes received from different neighbouring autonomous systems. This capability has shown some benefit (e.g., the one described in [RFC3345]) 🙌, but operators should be wary of the potential side effects of enabling it. The deployment section below gives some examples of why this might lead to undesirable behaviour ⚠️.

3.4. MEDs, Route Reflection, and AS Confederations for BGP 🧚

Okay this one is a lot. In particular configurations, the BGP scaling mechanisms defined in "BGP Route Reflection - An Alternative to Full Mesh IBGP" [RFC2796] and "Autonomous System Confederations for BGP" [RFC3065] can cause persistent BGP route oscillation [RFC3345] 🌀. The problem is baked into the way BGP works: there's a conflict between information hiding/hierarchy and the non-hierarchical selection process that comes from the lack of total ordering caused by the MED rules. Given current practices, the problem shows up most often with MED + route reflectors or confederations 💀.

One potential way to avoid this is to configure inter-Member-AS or inter-cluster IGP metrics higher than intra-Member-AS IGP metrics, and/or use other tie-breaking policies so BGP route selection isn't based on incomparable MEDs. Of course, IGP metric constraints might be unreasonably onerous for some applications 😮‍💨.

Not comparing MEDs between multiple paths for a prefix learned from different adjacent autonomous systems (as discussed in section 2.3), or not using MEDs at all, significantly decreases the probability of introducing potential route oscillation conditions into the network 🫧.

Also, although modifying MED attributes received on any type of IBGP session might be "legal" as far as current specifications are concerned, it is not recommended 🚫💖. That includes standard IBGP, EBGP sessions between Member-ASes of a BGP confederation, route reflection, and so on.

3.5. Route Flap Damping and MED Churn 🦋

MEDs are often derived dynamically from IGP metrics, or from additive costs associated with an IGP metric to a given BGP NEXT_HOP. This usually gives you an efficient model: the BGP MED advertised to peers (which represents the best path to a given destination within the network) stays aligned with the IGP within a given AS 🧠.

The catch is that with dynamically derived IGP-based MEDs, instability within an AS, or even on a single link within the AS, can cause widespread BGP instability or route advertisement churn that spreads across multiple domains 😵‍💫. In short: if your MED "flaps" every time your IGP metric flaps, your routes are likely going to get suppressed by BGP Route Flap Damping [RFC2439]. Ate and left no crumbs... except your reachability 💀.

Using MEDs can make the effects of BGP flap-dampening worse, because it may cause routes to be re-advertised just to reflect an internal topology change 😤.

Many implementations don't have a practical problem with IGP flapping, though. They either latch their IGP metric upon first advertisement, or use some internal suppression mechanism 🧊. Some implementations treat BGP attribute changes as less significant than route withdrawals and announcements, which helps soften the impact of this kind of event 🌈.

3.6. Effects of MEDs on Update Packing Efficiency 📦

Multiple unfeasible routes can be advertised in a single BGP Update message 📨. The BGP4 protocol also allows multiple prefixes that share a common set of path attributes to be advertised in a single update message. This is commonly called "update packing" 🎁. When possible, update packing is recommended, because it gives you more efficient behaviour in a bunch of areas, including:

  • Reduction in system overhead due to generating or receiving fewer Update messages 🙌.
  • Reduction in network overhead from fewer packets and lower bandwidth consumption 📉.
  • Less frequent processing of path attributes and fewer searches for matching sets in your AS_PATH database (if you have one). Consistent ordering of the path attributes makes matching in the database easier, because you don't end up with different representations of the same data 🗂️✨.

Update packing requires that all feasible routes within a single update message share a common attribute set, including a common MULTI_EXIT_DISC value 🫶. So potential wide-scale variance in MED values adds another variable and may cause a marked decrease in update packing efficiency 😬.

3.7. Temporal Route Selection ⏳

Some implementations had bugs that led to temporal behaviour in MED-based best path selection 🕰️. These usually involved methods to store the oldest route and to order routes for MED, which made it non-deterministic whether the oldest route would actually get selected 🎲.

The reasoning is that older paths are presumably more stable, and so more preferable. But temporal behaviour in route selection results in non-deterministic behaviour, and as such it is often undesirable 🙅‍♀️.

4. Deployment Considerations 🚀

It has been discussed that accepting MEDs from other autonomous systems can cause traffic flow churn in the network 🌪️. Some implementations only ratchet the MED down and never move it back up, to prevent excessive churn 🔽.

But if a session is reset, the MEDs being advertised might change 😳. If a network relies on received MEDs to route traffic properly, its traffic patterns could change dramatically, potentially causing congestion on the network 🚦. Essentially, accepting and routing traffic based on MEDs lets other people traffic engineer your network. Whether that's okay is up to you 🤷‍♀️.

As mentioned earlier, many network operators choose to reset MED values on ingress 🧹. On top of that, many operators explicitly do not use MED values of 0 or 2^32-1, to avoid inconsistencies with implementations and various revisions of the BGP specification 🍓.

4.1. Comparing MEDs between Different Autonomous Systems 💜

The MED was meant to be used only when comparing paths received from different external peers in the same AS. Even so, many implementations let you compare MEDs between different autonomous systems as well 🙃. AS operators often use LOCAL_PREF to select external preferences (primary and secondary upstreams, peers, customers, etc.). Using MED instead of LOCAL_PREF could lead to an inconsistent distribution of best routes, because MED is compared only after the AS_PATH length 😵.

Though this may seem like a fine idea for some configurations, you have to take care when comparing MEDs between different autonomous systems 🧐. BGP speakers often derive MED values by getting the IGP metric associated with reaching a given BGP NEXT_HOP within the local AS. That lets MEDs reasonably reflect IGP topologies when advertising routes to peers 🗺️. Comparing MEDs between multiple paths learned from a single AS is fine, but comparing MEDs between different autonomous systems can lead to potentially "weighted" decisions 🧁. This is most typical when the autonomous systems use different mechanisms to derive IGP metrics or BGP MEDs, or when they use different IGP protocols with very contrasting metric spaces (e.g., OSPF vs. traditional metric space in IS-IS) 🫠.

4.2. Effects of Aggregation on MEDs 🧩

Another MED deployment consideration is how aggregation of BGP routing information affects MEDs 🧺. Aggregates are often generated from multiple locations in an AS to support stability, redundancy, and other network design goals. When MEDs are derived from IGP metrics associated with said aggregates, the MED value advertised to peers can result in very suboptimal routing 😩.

5. Security Considerations 🔐

⚠️ Pay attention, bestie, this one matters. The MED was purposely designed to be a "weak" metric that would only be used late in the best-path decision process. The BGP working group was concerned that any metric specified by a remote operator would only affect routing in a local AS if no other preference was specified. A paramount goal of the MED's design was to ensure that peers could not "shed" or "absorb" traffic for networks that they advertise. So accepting MEDs from peers may, in some sense, increase a network's susceptibility to exploitation by peers 🚨.

6. Acknowledgements 🫶

Thanks to John Scudder for applying his usual keen eye and constructive insight 👑. Also, thanks to Curtis Villamizar, JR Mitchell, and Pekka Savola for their valuable feedback 💖.

7. References ✨

7.1. Normative References

Okay bestie, these are the docs you have to read first 📚💅

  • [RFC1771] Rekhter, Y. and T. Li, "A Border Gateway Protocol 4 (BGP-4)", RFC 1771, March 1995.
  • [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
  • [RFC2796] Bates, T., Chandra, R., and E. Chen, "BGP Route Reflection - An Alternative to Full Mesh IBGP", RFC 2796, April 2000.
  • [RFC3065] Traina, P., McPherson, D., and J. Scudder, "Autonomous System Confederations for BGP", RFC 3065, February 2001.
  • [BGP4] Rekhter, Y., Li, T., and S. Hares, "A Border Gateway Protocol 4 (BGP-4)", RFC 4271, January 2006.

7.2. Informative References

These ones aren't required, but they're so worth the read 🌸

  • [RFC2439] Villamizar, C., Chandra, R., and R. Govindan, "BGP Route Flap Damping", RFC 2439, November 1998.
  • [RFC3345] McPherson, D., Gill, V., Walton, D., and A. Retana, "Border Gateway Protocol (BGP) Persistent Route Oscillation Condition", RFC 3345, August 2002.

Authors' Addresses

Danny McPherson 🦋 Arbor Networks EMail: danny@arbor.net

Vijay Gill 💜 AOL EMail: VijayGill9@aol.com

Now for the legal bits, which are still important even though they're boring 🙈

Copyright (C) The Internet Society (2006).

This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights.

This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property

The IETF takes no position on the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document, or on the extent to which any license under such rights might or might not be available. It also doesn't claim to have made any independent effort to identify any such rights. Info on how rights work for RFC documents lives in BCP 78 and BCP 79 📖

Copies of IPR disclosures made to the IETF Secretariat, any assurances of licenses to be made available, or the result of an attempt to get a general license or permission for implementers or users to use these proprietary rights can be found in the IETF on-line IPR repository at http://www.ietf.org/ipr.

The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology needed to implement this standard. Send that info to the IETF at ietf-ipr@ietf.org 💌

Acknowledgement

Funding for the RFC Editor function is provided by the IETF Administrative Support Activity (IASA). 🫶

📦 where this page comes from