RFC 8327

Mitigating the Negative Impact of Maintenance through BGP Session Culling

📅 published March 2018, from ietf

RFC 8327: Mitigating the Negative Impact of Maintenance through BGP Session Culling ✨

Okay bestie, this is RFC 8327 🦋 and it's a Best Current Practice (so it's also BCP 214 🌸). It was published in March 2018 by the IETF, and its ISSN is 2070-1721. The authors are W. Hargrave from LONAP, M. Griswold from 20C, J. Snijders from NTT, and N. Hilliard from INEX 🫶. The title block doesn't list any RFCs that it obsoletes or updates, so it stands on its own 💅.

Abstract ✨

So this doc is all about stopping maintenance from wrecking networks 🧹. It gives guidance for IP networks and for Internet Exchange Points (IXPs). The big move is making sure BGP-4 sessions that'll be affected by maintenance get forcefully torn down before the maintenance even starts 💥.

Status of This Memo 🌈

This memo documents an Internet Best Current Practice.

This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on BCPs is available in Section 2 of RFC 7841.

Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at https://www.rfc-editor.org/info/rfc8327.

Copyright (c) 2018 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Simplified BSD License.

Table of Contents 🎀

1.  Introduction
2.  Requirements Language
3.  BGP Session Culling
  3.1.  Voluntary BGP Session Teardown Recommendations
    3.1.1.  Maintenance Considerations
  3.2.  Involuntary BGP Session Teardown Recommendations
    3.2.1.  Packet-Filter Considerations
    3.2.2.  Hardware Considerations
  3.3.  Procedural Considerations
4.  Security Considerations
5.  IANA Considerations
6.  References
  6.1.  Normative References
  6.2.  Informative References
Appendix A.  Example Packet Filters
  A.1.  Example Configuration for Cisco IOS, IOS XR, and Arista EOS
  A.2.  Example Configuration for Nokia SR OS
Acknowledgments
Authors' Addresses

1. Introduction 🧚

BGP Session Culling is the practice of making sure BGP sessions get forcefully torn down before maintenance on a lower-layer network kicks off -- maintenance that would otherwise mess with the flow of data between the BGP speakers 😭.

Okay so, here's the same idea again from a different angle 💖. BGP Session Culling means forcefully tearing down BGP sessions before maintenance starts on a lower-layer network, when that maintenance would otherwise affect the data flowing between the BGP speakers.

BGP Session Culling cuts down how much disruption lower-layer maintenance causes 🫧. It does this by letting BGP speakers move onto alternative paths early, while the lower-layer network's forwarding plane is still fully working. Main character energy 👑.

The grace period you need for BGP Session Culling to work is the total of two things: the time it takes to notice the BGP session is gone, plus the time the BGP speaker needs to converge onto alternative paths ⏳. The first part is usually set by the BGP Hold Timer (see Section 6.5 of [RFC4271]), which is commonly somewhere between 90 and 180 seconds. The second part depends on the implementation. It could be as long as 15 minutes when a router with a slow control plane is getting a full set of Internet routes 🙈.

Throughout this document, the "Caretaker" is the one in control of the lower-layer network, while "Operators" directly administrate the BGP speakers 🍓. Operators and Caretakers doing BGP Session Culling are encouraged to skip a fixed grace period. Instead, they should watch forwarding-plane activity while the culling is happening, and call it done once traffic levels have dropped to a minimum (Section 3.3) ✨.

2. Requirements Language 🔥

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here 💅.

3. BGP Session Culling 🦋

From the Operator's point of view, there are two flavours of BGP Session Culling 🍓:

  • Voluntary BGP Session Teardown: The Operator starts the teardown of the potentially affected BGP session by issuing an Administrative Shutdown.
  • Involuntary BGP Session Teardown: The Caretaker of the lower-layer network disrupts (higher-layer) BGP control-plane traffic. That makes the BGP Hold Timers of the affected BGP session expire, which then triggers rerouting of end-user traffic 🌸.

3.1. Voluntary BGP Session Teardown Recommendations 🌸

Before an Operator starts activities that could disrupt data flowing through the lower-layer network, they can cut down on lost traffic. To do that, they issue an administrative shutdown to all BGP sessions running across the lower-layer network, then wait a few minutes for data-plane traffic to calm down 🫧.

Some architectures help networks reconverge quickly, such as BGP Prefix Independent Convergence (PIC) [BGP_PIC]. But an Operator can't assume the remote side has those capabilities 🙈. So a grace period between the Administrative Shutdown and the impactful maintenance is warranted ⏳.

Once the maintenance is done, the Operator is expected to put the BGP sessions back to their original Administrative state ✨.

3.1.1. Maintenance Considerations 🎀

Initiators of the Administrative Shutdown MAY consider using Graceful Shutdown [RFC8326] to help traffic drain smoothly before the session is torn down 🌊. They might also use the Shutdown Communication [RFC8203] to tell the remote side what the nature and duration of the maintenance is 💜.

3.2. Involuntary BGP Session Teardown Recommendations 🌈

Things can work differently when BGP speakers interconnect with multiple peers through a switched Layer 2 fabric, which is what you typically see at Internet Exchange Points (IXPs) 🏝️.

Honestly, operational experience shows that a lot of Operators can't pull off the Voluntary BGP Session Teardown recommendations. Coordinating the two configuration changes costs a lot and carries risk 😩. That hurts Internet performance overall 💔.

So, if the lower layer doesn't send any notifications about the planned maintenance in a switched Layer 2 fabric (for example, an Ethernet link going down), the fabric's Caretaker could decide to cull BGP sessions on behalf of the Operators connected to the fabric 🧹.

This culling of control-plane traffic stops end-user traffic from getting dropped by causing the BGP Hold Timers to expire earlier than they would without the fabric's Caretaker stepping in 🛟.

In this setup, BGP Session Culling is done the way the next subsection describes, by applying a combined Layer 3 and Layer 4 (Layer 3/4) packet filter in the Caretaker's switched fabric 🔒.

3.2.1. Packet-Filter Considerations 🧩

The IXP's peering LAN prefixes make up the control plane, and these considerations apply to the packet-filter design 🗺️:

  • The packet filter MUST only affect BGP traffic specific to the Layer 2 fabric, i.e., traffic forming part of the control plane of the system described, rather than multihop BGP traffic that merely transits.
  • The packet filter MUST only affect BGP, i.e., TCP port 179.
  • The packet filter SHOULD make provision for the bidirectional nature of BGP, i.e., sessions may be established in either direction.
  • The packet filter MUST affect all Address Family Identifiers.

Appendix A has examples of correct packet filters for various platforms 📚.

3.2.2. Hardware Considerations 🛠️

Not every piece of hardware can deploy combined Layer 3/4 filters on Layer 2 ports 😬. Even platforms that say they support this might have limitations. Hardware resource allocation failures can also happen while filters are being deployed, and those can lead to unexpected results 💀. Problems can include:

  • The platform can't apply Layer 3/4 filters on ports that already have Layer 2 filters applied.
  • Layer 3/4 filters are supported for IPv4 but not for IPv6.
  • Layer 3/4 filters are supported on physical ports, but not on IEEE 802.1AX Link Aggregate ports [IEEE802.1AX].
  • The Caretaker fails to apply filters to all IEEE 802.1AX Link Aggregate ports [IEEE802.1AX].
  • Limitations in Access Control List (ACL) hardware mechanisms cause filters not to be applied.
  • Fragmentation of ACL lookup memory causes transient ACL application problems, which get resolved after the ACL is removed and reapplied.
  • There's temporary service loss while the hardware is being programmed.
  • Hardware ACL capacity goes down if the platform enables lossless ACL application.

It's a smart move for the Caretaker to know what their hardware can and can't do, and to thoroughly test all the complicated configurations ahead of time 🧪. That way, nothing goes sideways during production deployments 🙏.

3.3. Procedural Considerations 💅

The Caretaker of the lower-layer network can keep an eye on data-plane traffic (for example, interface counters). Once session culling is complete, they can do the maintenance without it affecting traffic 🫶.

It's recommended that the packet filters be deployed only for the duration of the maintenance, and removed right after the maintenance is finished 🧼. To avoid unnecessary troubleshooting, it is RECOMMENDED that Caretakers notify the affected Operators before the maintenance happens, and make it crystal clear that the Involuntary BGP Session Culling methodology will be applied 📣.

Part 1 of 2 is done bestie 🎀✨ Part 2 is next, with Security Considerations, IANA Considerations, the references, the appendices, and the acknowledgments 💜🌸.

4. Security Considerations ✨

okay bestie, this one is short and sweet 🎀 there are no security considerations in this doc 💅 nothing extra to panic about, but we still take the whole vibe seriously 🫶

5. IANA Considerations 🌈

This document has no actions for IANA 📮 nothing to file and no paperwork for the IANA crew this time 🌸

6. References 📚

6.1. Normative References

these are the must-read ones 💜 bookmark them, bestie 🔖

   [RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119,
              DOI 10.17487/RFC2119, March 1997,
              <https://www.rfc-editor.org/info/rfc2119>.

   [RFC4271]  Rekhter, Y., Ed., Li, T., Ed., and S. Hares, Ed., "A
              Border Gateway Protocol 4 (BGP-4)", RFC 4271,
              DOI 10.17487/RFC4271, January 2006,
              <https://www.rfc-editor.org/info/rfc4271>.

   [RFC8174]  Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC
              2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174,
              May 2017, <https://www.rfc-editor.org/info/rfc8174>.

6.2. Informative References

these are the extra-credit reads 🦋 good for the deep dives 🧚

   [BGP_PIC]  Bashandy, A., Ed., Filsfils, C., and P. Mohapatra, "BGP
              Prefix Independent Convergence", Work in Progress,
              draft-ietf-rtgwg-bgp-pic-06, November 2017.

   [IEEE802.1AX]
              IEEE, "IEEE Standard for Local and metropolitan area
              networks -- Link Aggregation", IEEE Std 802.1AX-2014,
              DOI 10.1109/IEEESTD.2014.7055197, December 2014,
              <http://ieeexplore.ieee.org/servlet/
              opac?punumber=6997981>.

   [RFC8203]  Snijders, J., Heitz, J., and J. Scudder, "BGP
              Administrative Shutdown Communication", RFC 8203,
              DOI 10.17487/RFC8203, July 2017,
              <https://www.rfc-editor.org/info/rfc8203>.

   [RFC8326]  Francois, P., Ed., Decraene, B., Ed., Pelsser, C., Patel,
              K., and C. Filsfils, "Graceful BGP Session Shutdown",
              RFC 8326, DOI 10.17487/8326, March 2018,
              <https://www.rfc-editor.org/info/rfc8326>.

Appendix A. Example Packet Filters 🧚✨

okay the examples are giving sweet little packet-filter fits 🎀 this section shows packet filters doing Involuntary BGP Session Teardown at an IXP 👑 the control plane is the peering LAN prefixes 192.0.2.0/24 and 2001:db8:2::/64 💖

there's a whole repo of config examples for loads of different platforms too 🌈 go peek at https://github.com/bgp/bgp-session-culling-config-examples 🔥

A.1. Example Configuration for Cisco IOS, IOS XR, and Arista EOS

   ipv6 access-list acl-ipv6-permit-all-except-bgp
      10 deny tcp 2001:db8:2::/64 eq bgp 2001:db8:2::/64
      20 deny tcp 2001:db8:2::/64 2001:db8:2::/64 eq bgp
      30 permit ipv6 any any
   !
   ip access-list acl-ipv4-permit-all-except-bgp
      10 deny tcp 192.0.2.0/24 eq bgp 192.0.2.0/24
      20 deny tcp 192.0.2.0/24 192.0.2.0/24 eq bgp
      30 permit ip any any
   !
   interface Ethernet33
      description IXP Participant Affected by Maintenance
      ip access-group acl-ipv4-permit-all-except-bgp in
      ipv6 access-group acl-ipv6-permit-all-except-bgp in
   !

A.2. Example Configuration for Nokia SR OS

   ip-filter 10 create
       filter-name "ACL IPv4 Permit All Except BGP"
       default-action forward
       entry 10 create
           match protocol tcp
               dst-ip 192.0.2.0/24
               src-ip 192.0.2.0/24
               port eq 179
           exit
           action
               drop
           exit
       exit
   exit

   ipv6-filter 10 create
       filter-name "ACL IPv6 Permit All Except BGP"
       default-action forward
       entry 10 create
           match next-header tcp
               dst-ip 2001:db8:2::/64
               src-ip 2001:db8:2::/64
               port eq 179
           exit
           action
               drop
           exit
       exit
   exit

   interface "port-1/1/1"
       description "IXP Participant Affected by Maintenance"
       ingress
           filter ip 10
           filter ipv6 10
       exit
   exit

Acknowledgments 🫶

the authors want to give a big shoutout to these legends for their contributions 👑 Saku Ytti, Greg Hankins, James Bensley, Wolfgang Tremmel, Daniel Roesen, Bruno Decraene, Tore Anderson, John Heasley, Warren Kumari, Stig Venaas, and Brian Carpenter 💜 you're all iconic ✨

Authors' Addresses 💌

here's who wrote this 🦋

   Will Hargrave
   LONAP Ltd
   5 Fleet Place
   London  EC4M 7RD
   United Kingdom

   Email: will@lonap.net
   Matt Griswold
   20C
   1658 Milwaukee Ave # 100-4506
   Chicago, IL  60647
   United States of America

   Email: grizz@20c.com
   Job Snijders
   NTT Communications
   Theodorus Majofskistraat 100
   Amsterdam  1065 SZ
   The Netherlands

   Email: job@ntt.net
   Nick Hilliard
   INEX
   4027 Kingswood Road
   Dublin  24
   Ireland

   Email: nick@inex.ie
📦 where this page comes from