RFC 8604

Interconnecting Millions of Endpoints with Segment Routing

📅 published June 2019, from ietf

Okay bestie, this is RFC 8604 🦋 titled Interconnecting Millions of Endpoints with Segment Routing. It's an Independent Submission with Informational category, dated June 2019, ISSN 2070-1721. The authors are C. Filsfils, Ed. (Cisco Systems, Inc.), S. Previdi (Huawei Technologies), G. Dawra, Ed. (LinkedIn), W. Henderickx (Nokia), and D. Cooper (CenturyLink) 💜. The header doesn't list anything it obsoletes or updates, so there's nothing to flag there 🌸

Abstract ✨

So this doc is about using Segment Routing to scale a network up to hundreds of thousands of network nodes and tens of millions of physical underlay endpoints 🤯 That's giving massive-scale Data Centers (DCs) and/or large aggregation networks. The forwarding tables of midpoint and leaf nodes only need a few tens of thousands of entries, and that's because Segment Routing is inherently scaleable and because of the design proposed in this document 💅

Status of This Memo

This document is not an Internet Standards Track specification, so it's published just for informational purposes 📚

This is a contribution to the RFC Series, independent of any other RFC stream. The RFC Editor picked to publish it at their discretion and makes no statement about its value for implementation or deployment. Documents the RFC Editor approves for publication aren't candidates for any level of Internet Standard. See Section 2 of RFC 7841 if you want the receipts 🧾

Info about the current status of this document, any errata, and how to give feedback on it lives at https://www.rfc-editor.org/info/rfc8604 🌈

Copyright (c) 2019 IETF Trust and the persons identified as the document authors. All rights reserved ©️

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication. Please read these carefully, because they spell out your rights and restrictions with respect to this document 📜

Table of Contents

   1. Introduction ....................................................3
   2. Terminology .....................................................3
   3. Reference Design ................................................3
   4. Control Plane ...................................................5
   5. Illustration of the Scale .......................................5
   6. Design Options ..................................................6
      6.1. Segment Routing Global Block (SRGB) Size ...................6
      6.2. Redistribution of Routes for Agg Nodes .....................7
      6.3. Sizing and Hierarchy .......................................7
      6.4. Local Segments to Hosts/Servers ............................7
      6.5. Compressed SRTE Policies ...................................7
   7. Deployment Model ................................................8
   8. Benefits ........................................................8
      8.1. Simplified Operations ......................................8
      8.2. Inter-domain SLAs ..........................................8
      8.3. Scale ......................................................9
      8.4. ECMP .......................................................9
   9. IANA Considerations .............................................9
   10. Manageability Considerations ...................................9
   11. Security Considerations ........................................9
   12. Informative References .........................................9
   Acknowledgements ..................................................10
   Contributors ......................................................10
   Authors' Addresses ................................................11

1. Introduction ✨

This doc explains how Segment Routing (SR) can be used to interconnect millions of endpoints 🫶

2. Terminology 🎀

Here are the terms and abbreviations we're using in this document 💖

      Term          Definition
      -------------------------------------------------------------
      Agg           Aggregation
      BGP           Border Gateway Protocol
      DC            Data Center
      DCI           Data Center Interconnect
      ECMP          Equal-Cost Multipath
      FIB           Forwarding Information Base
      LDP           Label Distribution Protocol
      LFIB          Label Forwarding Information Base
      MPLS          Multiprotocol Label Switching
      PCE           Path Computation Element
      PCEP          Path Computation Element Communication Protocol
      PW            Pseudowire
      SLA           Service Level Agreement
      SR            Segment Routing
      SRTE Policy   Segment Routing Traffic Engineering Policy
      TE            Traffic Engineering
      TI-LFA        Topology Independent Loop-Free Alternate

3. Reference Design 🧚

Here's the network diagram that shows the reference topology we use in this document 🗺️

           +-------+ +--------+ +--------+ +-------+ +-------+
           A       DCI1       Agg1       Agg3      DCI3      Z
           |  DC1  | |   M1   | |   C    | |   M2  | |  DC2  |
           |       DCI2       Agg2       Agg4      DCI4      |
           +-------+ +--------+ +--------+ +-------+ +-------+

Figure 1: Reference Topology

Here's what applies to that reference topology above 👑

  • Independent ISIS-OSPF/SR instance in core (C) region 💅
  • Independent ISIS-OSPF/SR instance in Metro1 (M1) region 💅
  • Independent ISIS-OSPF/SR instance in Metro2 (M2) region 💅
  • BGP/SR in DC1 🌸
  • BGP/SR in DC2 🌸
  • Agg routes (Agg1, Agg2, Agg3, Agg4) are redistributed from C to M (M1 and M2) and from M to DC domains 🦋
  • No other route is advertised or redistributed between regions 🙈
  • The same homogeneous Segment Routing Global Block (SRGB) is used throughout the domains (e.g., 16000-23999) ✨
  • Unique SRGB sub-ranges are allocated to each metro (M) and core (C) domain 🍓
    • The 16000-16999 range is allocated to the core (C) domain/region.
    • The 17000-17999 range is allocated to the M1 domain/region.
    • The 18000-18999 range is allocated to the M2 domain/region.
    • Specifically, the Agg1 router has Segment Identifier (SID) 16001 allocated, and the Agg2 router has SID 16002 allocated.
    • Specifically, the Agg3 router has SID 16003 allocated, and the anycast SID for Agg3 and Agg4 is 16006.
    • Specifically, the DCI3 router has SID 18003 allocated, and the anycast SID for DCI3 and DCI4 is 18006.
    • Specifically, at the Agg1 router, the binding SID 4001 leads to DCI pair (DCI3, DCI4) via a specific low-latency path {16002, 16003, 18006}.
  • The same SRGB sub-range is reused within each DC (DC1 and DC2) region for each DC (e.g., 20000-23999). Specifically, nodes A and Z both have SID 20001 allocated to them 🧁

6.3. Sizing and Hierarchy 🎀

okay so the operator is totally free to pick a few big leaf domains, lots of small leaf domains, or a mix of small and large core/leaf domains 🧚 you get to decide the vibe!

the operator is also free to go with a two-tier (Core/Metro) design or a three-tier (Core/Metro/DC) design 💅

6.4. Local Segments to Hosts/Servers 🌸

okay this one is lowkey so cute. local segments can be programmed at any leaf node (e.g., node Z) so it can identify hosts or VMs that are locally attached to it 🫶

for example, if node Z has bound a local segment 40001 to a local host ZH1, then node A uses this SRTE Policy to reach that host:

{16006, 18006, 20001, 40001}

the local segment could represent the NID (Network Interface Device) in a service provider access network, or a VM in a DC network. it's giving flexibility 🦋

6.5. Compressed SRTE Policies 💜

let's do an example, bestie! following Section 3, we assume that node A can reach node Z (e.g., with a low-latency SLA contract) through the SRTE policy that goes Agg1, Agg2, Agg3, DCI3/4(anycast), Z. the path is represented by the segment list {16001, 16002, 16003, 18006, 20001} 🔥

so it's clear that the control-plane solution can install an SRTE Policy {16002, 16003, 18006} at Agg1, collect the binding SID that Agg1 allocates to that policy (e.g., 4001), and then program node A with the compressed SRTE Policy {16001, 4001, 20001}. the way that works is so clean 😭

here's how the walk goes:

  • from node A, 16001 leads to Agg1.
  • once at Agg1, 4001 leads to the DCI pair (DCI3, DCI4) via a specific low-latency path {16002, 16003, 18006}.
  • once at that DCI pair, 20001 leads to Z.

binding SIDs allocated to "intermediate" SRTE Policies achieve the compression of end-to-end SRTE Policies. literally the whole point 👑

the segment list {16001, 4001, 20001} expresses the same path as {16001, 16002, 16003, 18006, 20001}, but with two fewer segments 🍓

the binding SID also gives you inherent churn protection. when the core topology changes, the control plane can update the low-latency SRTE Policy from Agg1 to the DCI pair to DC2 without updating the SRTE Policy from A to Z. no drama, no cap 🛡️

7. Deployment Model 🌈

this design is expected to be used in "green field" deployments, and also in interworking ("brown field") deployments with an MPLS design across multiple domains. it fits right in wherever you are 🦋

8. Benefits ✨

the design options in this document allow interconnections on a very large scale. millions of endpoints across different domains can be interconnected. iconic behavior, honestly 😭

8.1. Simplified Operations 💅

two control-plane protocols are not needed in this design: LDP and RSVP-TE. no new protocol has been introduced either! the design leverages the core IP protocols ISIS, OSPF, BGP, and PCEP with straightforward SR extensions. so easy, so chill 🎀

8.2. Inter-domain SLAs 💖

fast reroute and resiliency are provided by TI-LFA, with sub-50-ms fast reroute upon failure of a link, node, or Shared Risk Link Group (SRLG). TI-LFA is described in [SR-TI-LFA]. the TI-LFA fast reroute is the real MVP here 🔥

the use of anycast SIDs also provides improved availability and resiliency. ate and left no crumbs 💜

inter-domain SLAs can be delivered (e.g., latency vs. cost-optimized paths, disjointness from backbone planes, disjointness from other services, disjointness between primary and backup paths). it's giving customization 🧚

existing inter-domain solutions do not provide any support for SLA contracts. they just provide best-effort reachability across domains. not me being shocked 🙈

8.3. Scale 🌟

in addition to eliminating the need for LDP and RSVP-TE, per-service midpoint states have also been removed from the network. so much less state to babysit, iykyk 🍓

8.4. ECMP 🦋

each policy (intra-domain or inter-domain, with or without TE) is expressed as a list of segments. since each segment is optimized for ECMP, the entire policy is optimized for ECMP. slay the load balancing 💅

the benefit of an anycast prefix segment optimized for ECMP should also be considered. for example, 16001 load-shares across any gateway from the M1 leaf domain to the Core, and 16002 load-shares across any gateway from the Core to the M1 leaf domain. main character energy 👑

9. IANA Considerations 🌸

this document has no IANA actions. nothing to register, bestie ✨

10. Manageability Considerations 🛠️

this document describes an application of SR over the MPLS data plane. SR does not introduce any changes in the MPLS data plane. the manageability considerations described in [RFC8402] apply to the MPLS data plane when used with SR. read [RFC8402] and you're set 💜

11. Security Considerations 🔒

⚠️ this one is serious, so listen up! this document does not introduce additional security requirements and mechanisms other than those described in [RFC8402]. the security story is whatever [RFC8402] says it is. no sneaky extras here, but don't skip that reference 🫶

12. Informative References 📚

these are the references, all the details included, periodt 💅

[RFC4655]  Farrel, A., Vasseur, J.-P., and J. Ash, "A Path
           Computation Element (PCE)-Based Architecture", RFC 4655,
           DOI 10.17487/RFC4655, August 2006,
           <https://www.rfc-editor.org/info/rfc4655>.

[RFC5440]  Vasseur, JP., Ed. and JL. Le Roux, Ed., "Path Computation
           Element (PCE) Communication Protocol (PCEP)", RFC 5440,
           DOI 10.17487/RFC5440, March 2009,
           <https://www.rfc-editor.org/info/rfc5440>.

[RFC7752]  Gredler, H., Ed., Medved, J., Previdi, S., Farrel, A., and
           S. Ray, "North-Bound Distribution of Link-State and
           Traffic Engineering (TE) Information Using BGP", RFC 7752,
           DOI 10.17487/RFC7752, March 2016,
           <https://www.rfc-editor.org/info/rfc7752>.

[RFC8402]  Filsfils, C., Ed., Previdi, S., Ed., Ginsberg, L.,
           Decraene, B., Litkowski, S., and R. Shakir, "Segment
           Routing Architecture", RFC 8402, DOI 10.17487/RFC8402,
           July 2018, <https://www.rfc-editor.org/info/rfc8402>.

[SR-TI-LFA]
           Litkowski, S., Bashandy, A., Filsfils, C.,
           Decraene, B., Francois, P., Voyer, D., Clad, F., and
           P. Camarillo, "Topology Independent Fast Reroute
           using Segment Routing", Work in Progress,
           draft-ietf-rtgwg-segment-routing-ti-lfa-01, March 2019.

Acknowledgements 🫶

we would like to thank Giles Heron, Alexander Preusche, Steve Braaten, and Francis Ferguson for their contributions to the content of this document. huge thank you, you're all iconic 🎀

Contributors 🌈

the following people substantially contributed to the editing of this document 💖

Dennis Cai
Individual

Tim Laberge
Individual

Steven Lin
Google Inc.

Bruno Decraene
Orange

Luay Jalil
Verizon

Jeff Tantsura
Individual

Rob Shakir
Google Inc.

Authors' Addresses 💜

Clarence Filsfils (editor)
Cisco Systems, Inc.
Brussels
Belgium

Email: cfilsfil@cisco.com


Stefano Previdi
Huawei Technologies

Email: stefano@previdi.net


Gaurav Dawra (editor)
LinkedIn
United States of America

Email: gdawra.ietf@gmail.com


Wim Henderickx
Nokia
Copernicuslaan 50
Antwerp  2018
Belgium

Email: wim.henderickx@nokia.com


Dave Cooper
CenturyLink

Email: Dave.Cooper@centurylink.com
📦 where this page comes from