RFC 8735

Scenarios and Simulation Results of PCE in a Native IP Network

📅 published February 2020, from ietf

RFC 8735: Scenarios and Simulation Results of PCE in a Native IP Network 🎀

Bestie, this is the title block. The paper is from the IETF, and it's labelled Informational (not a Standards Track spec, so it's a "here's some cool stuff we found" doc 🦋). It was published in February 2020, and the ISSN is 2070-1721 💜

  • Title: Scenarios and Simulation Results of PCE in a Native IP Network
  • Authors: A. Wang (China Telecom), X. Huang (no affiliation listed), C. Kou (BUPT), Z. Li (China Mobile), P. Mi (Huawei Technologies)
  • Category: Informational
  • Date: February 2020
  • Obsoletes / Updates: the title block doesn't list anything this doc obsoletes or updates 🙈

Abstract ✨

Okay so service providers are starting to need End-to-End (E2E) performance assurance 🍓 There are a bunch of tech options out there, but none of them fully covers this for a native IP network. What they really need is one universal E2E solution that works for both intra-domain and inter-domain setups 💅

One promising route is adding central control to a native IP network. This doc walks through some pretty complex scenarios and shows simulation results for using the Path Computation Element (PCE) in a native IP network. The solution is called Centralized Control Dynamic Routing (CCDR), and it's giving best of both worlds: it takes the strengths of distributed protocols and mixes them with the power of centralized control. It handles traffic engineering for native IP networks, and it works the same way for intra-domain and inter-domain cases 👑

Status of This Memo 📜

This doc is not an Internet Standards Track spec. It's published purely for informational purposes 🌸

It's a product of the Internet Engineering Task Force (IETF), so it reflects the consensus of the IETF community. It went through public review and got approved for publication by the Internet Engineering Steering Group (IESG). Heads up though: not every IESG-approved doc is eligible for any level of Internet Standard. For the details, check Section 2 of RFC 7841 🤓

If you want the latest status, any errata, or a way to send feedback, head to https://www.rfc-editor.org/info/rfc8735 ✨

Copyright (c) 2020 IETF Trust and the persons identified as the document authors. All rights reserved. 🫶

This doc is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info), as they stand on the publication date. Please actually read these, because they spell out your rights and restrictions for this doc 🙈

One important bit: any Code Components pulled out of this document have to include Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions. They also come with no warranty, per the Simplified BSD License 💀

Table of Contents 🧚

   1.  Introduction
   2.  Terminology
   3.  CCDR Scenarios
     3.1.  QoS Assurance for Hybrid Cloud-Based Application
     3.2.  Link Utilization Maximization
     3.3.  Traffic Engineering for Multi-domain
     3.4.  Network Temporal Congestion Elimination
   4.  CCDR Simulation
     4.1.  Case Study for CCDR Algorithm
     4.2.  Topology Simulation
     4.3.  Traffic Matrix Simulation
     4.4.  CCDR End-to-End Path Optimization
     4.5.  Network Temporal Congestion Elimination
   5.  CCDR Deployment Consideration
   6.  Security Considerations
   7.  IANA Considerations
   8.  References
     8.1.  Normative References
     8.2.  Informative References
   Acknowledgements
   Contributors
   Authors' Addresses

1. Introduction 💖

Picture a service provider network: thousands of routers, all running distributed protocols to swap reachability info 🌐 For the most part, those distributed protocols pick the path to each destination network and keep it under control. They're sturdy enough for most apps, but they struggle with the fancier stuff traffic engineering needs, like E2E performance assurance or squeezing the most out of link utilization inside an IP network 😭

Multiprotocol Label Switching (MPLS) using Traffic-Engineering (TE) [RFC3209] is one option for TE networks. The catch is that it adds an MPLS network and its related tech on top of the IP network, basically an overlay. MPLS-TE is mostly used for Label Switched Path (LSP) protection and for setting up complicated paths inside a single domain. It hasn't really caught on for meeting dynamic E2E performance assurance needs (especially inter-domain) in an IP network, so it's not the main character here 🦋

Segment Routing [RFC8402] is another option, and it borrows some of the good parts of distributed protocols and central control. The downside is that it asks the underlying network, especially the provider edge router, to do deep label push and pop actions. That adds complexity when it has to live alongside non-segment routing networks. On top of that, it can only steer E2E paths for MPLS and IPv6 traffic, and it does that through different mechanisms 🙃

Deterministic Networking (DetNet) [RFC8578] is also a possible solution. It's mainly about giving a flow bounded latency, and it adds extra requirements on the domain edge router. The current DetNet scope is limited to one domain. The use cases in this doc don't need that extra deterministic complexity, so they're a different vibe from the DetNet use cases ✨

So this doc describes a few scenarios for a native IP network where a Centralized Control Dynamic Routing (CCDR) framework can make things way more efficient, and it doesn't need any change to the router's data-plane behaviour 🔥 Here's how it works: the central PCE takes what it knows about the Border Gateway Protocol (BGP) session-specific prefixes a router advertises, the network topology, and near-real-time link-utilization info from network management systems. From that, it computes an optimal path and tells the underlying routers which destination address to use to reach the BGP nexthop. That way, the distributed routing protocol just uses the computed path through its normal recursive lookup procedure 🧠 The doc also shares simulation results for path optimization, to show concrete cases where CCDR beats traditional distributed routing protocols by a lot 👑

This doc is the base document for two others. The universal solution doc, which works for both intra-domain and inter-domain TE, is [PCE-NATIVE-IP]. The related protocol extension contents are in [PCEP-NATIVE-IP-EXT] 💜

2. Terminology 🌈

In this doc, PCE means what [RFC5440] defines it to mean. Here are the other terms, so we're all on the same page 📖

   BRAS:   Broadband Remote Access Server

   CD:     Congestion Degree

   CR:     Core Router

   CCDR:   Centralized Control Dynamic Routing

   E2E:    End to End

   IDC:    Internet Data Center

   MAN:    Metro Area Network

   QoS:    Quality of Service

   SR:     Service Router

   TE:     Traffic Engineering

   UID:    Utilization Increment Degree

   WAN:    Wide Area Network

3. CCDR Scenarios 🦄

The next sections go through different deployment scenarios where applying the CCDR framework is expected to bring improvements. The expectation comes from the framework's big-picture properties and from each scenario itself 💅

3.1. QoS Assurance for Hybrid Cloud-Based Application ☁️

Cloud computing is blowing up, so enterprises are moving more and more services to public cloud environments while keeping their core business in private clouds 💖 The traffic between the private and public cloud sites crosses the WAN 🌍 The bandwidth needs between those sites change over time, and so does the background traffic between them. Enterprise apps need E2E QoS performance guaranteed on demand, even though the bandwidth keeps shifting 😭

CCDR fits this scenario perfectly, since it combines the perks of distributed protocols with the power of centralized control 🫶 Here's the possible solution framework:

                            +------------------------+
                            | Cloud-Based Application|
                            +------------------------+
                                        |
                                  +-----------+
                                  |    PCE    |
                                  +-----------+
                                        |
                                        |
                               //--------------\\
                          /////                  \\\\\
     Private Cloud Site ||       Distributed          |Public Cloud Site
                         |       Control Network      |
                          \\\\\                  /////
                               \\--------------//

               Figure 1: Hybrid Cloud Communication Scenario

As Figure 1 shows, the source of the "Cloud-Based Application" traffic is at the "Private Cloud Site", and its destination is the "Public Cloud Site" 🌸

By default, the traffic path between the private and public cloud sites is decided by the distributed control network. When an application needs E2E QoS assurance, it can send those requirements to the PCE. The PCE then computes one E2E path, based on the underlying network topology and real traffic info, so it can meet the application's QoS requirements 🎀 Section 4.4 of this document has the simulation results for this use case 🔥

3.2. Link Utilization Maximization

okay so 🌸 Metro Area Networks (MANs) are usually laid out in a star shape ✨ (see Figure 2 below), with different devices hooked up to different customer types 💅. The traffic from these customers follows a tidal pattern 🌊 — the links between the Core Router (CR) / Broadband Remote Access Server (BRAS) and CR/Service Router (SR) get congested at different times. Subscribers under the BRAS are usually online at night 🌙, while the leased line users under the SR are mostly on during the daytime ☀️. The link between BRAS/SR and CR has to be big enough to handle the maximum traffic volume between them, so these links end up being underutilized most of the time 🥺. Not ideal, bestie 💀.

+--------+

CR

+----|---+

+--|-+ +-|+ +--|-+ +-|+

BRASSRBRASSR

+----+ +--+ +----+ +--+

Figure 2: Star-Mode Network Topology within MAN

Now here's the glow-up 🦋. If we connect the BRAS/SR with a local link loop (which is usually lower cost) and let the CCDR framework run the overall MAN topology, we can exploit that tidal thing between the BRAS/CR and SR/CR links. That means we get to max out the utilization of these central trunk links (which usually cost more than the local loops) 🤑. Iconic, honestly 👑.

+-------+ ----- PCE |

+-------+

+----|---+

CR

+----|---+

+--|-+ +-|+ +--|-+ +-|+

BRAS-----SRBRAS-----SR

+----+ +--+ +----+ +--+

Figure 3: Link Utilization Maximization via CCDR

3.3. Traffic Engineering for Multi-domain

Service provider networks are often made up of different domains that connect to each other, forming a seriously complex topology 🕸️ (see Figure 4). Because of the traffic going to and from the MAN and IDC, the utilization of the links between them is often asymmetric 🙃. It's almost impossible to balance those links with a distributed protocol, but CCDR framework can fix that imbalance 🫶.

+---+ +---+

MAN----------------IDC

+---+ | +---+

-----Backbone-----

+---+ | +---+

IDC----------------MAN

+---+ +---+

Figure 4: Traffic Engineering for Complex Multi-domain Topology

A solution for this scenario needs to gather NetFlow information 📊, analyze the source/destination autonomous system (AS), and figure out what the main cause of the congested link(s) is 🔍. After that, the operator can use the external Border Gateway Protocol (eBGP) sessions to schedule traffic among the different domains, following the solution described in the CCDR framework 💜.

3.4. Network Temporal Congestion Elimination

In more general situations, there's often temporal congestion within the service provider's network 🕰️ — think daily or weekly periodic bursts, or big events that get scheduled way in advance 🎉. This kind of congestion shows up regularly, and if the service provider has ways to mitigate it, their network operation gets so much better and customers are happier too 🥰. CCDR is also a great fit for these scenarios, since the controller can schedule traffic off the congested links and lower their utilization during those times ✨. Section 4.5 describes the simulation results of this scenario.

  1. CCDR Simulation ✨

The following sections walk through a specific case study to show how the CCDR algorithm works with concrete paths/metrics 🧩, plus a procedure for generating topology and traffic matrices, and the results of simulations applying CCDR for E2E QoS (assured path and congestion elimination) over the generated topologies and traffic matrices 📈. In every case examined, the CCDR algorithm gives a qualitatively significant improvement over the reference (OSPF) algorithm 🔥, which suggests CCDR will have broad applicability 🌈.

The structure and scale of the simulated topology is similar to that of real networks 🌍. Multiple different traffic matrices were generated to simulate different congestion conditions in the network. Only one of them is illustrated since the others produce similar results 🙈.

4.1. Case Study for CCDR Algorithm

In this section, we look at a specific network topology for the case study: examining the path selected by OSPF and CCDR, and evaluating how and why the paths differ 🤔. Figure 5 shows the topology of the network in this case. There are eight forwarding devices in the network, and the original cost and utilization are marked on it as shown in the figure. For example, the original cost and utilization for the link (1, 2) are 3 and 50%, respectively. There are two flows: f1 and f2. Both of these flows go from node 1 to node 8. For simplicity, it is assumed that the bandwidth of the link in the network is 10 Mb/s. The flow rate of f1 is 1 Mb/s and the flow rate of f2 is 2 Mb/s. The threshold of the link in congestion is 90% 🚦.

If the OSPF protocol, which uses Dijkstra's algorithm (IS-IS is similar because it also uses Dijkstra's algorithm), is applied in the network, the two flows from node 1 to node 8 can only use the OSPF path (p1: 1->2->3->8). That's because Dijkstra's algorithm mainly looks at the original cost of the link 💅. CCDR considers cost and utilization at the same time, so it won't pick the same path as OSPF, since link (2, 3) is severely congested 🚨. In this case, f1 will select the path (p2: 1->5->6->7->8) because the new cost of this path beats the OSPF path. On top of that, p2 is also better than the path (p3: 1->2->4->7->8) for f1 💖. However, f2 won't take the same path, since that would cause new congestion on link (6, 7) 😬. So f2 ends up on the path (p3: 1->2->4->7->8) 🦋.

+----+ f1 +-------> +-----+ ----> +-----+

Edge-----------++--------3-------8
Node---------++-----> +-----+ ----> +-----+

+----+ | | 4/95%||| 6/50% |

5/60%
v

+----+ +-----+ -----> +-----+ +-----+ +-----+

Edge-------1--------2------4------7
Node-----> +-----+ -----> +-----+7/60% +-----+5/45% +-----+

+----+ f2 | 3/50% |

3/60% +-----+ 5/55%+-----+ 3/75%

+-----------| 5 |------| 6 |---------+ +-----+ +-----+ (a) Dijkstra's Algorithm (OSPF/IS-IS)

+----+ f1 +-----+ ----> +-----+

Edge-----------+ +--------3-------8
Node---------++-----+ ----> +-----+

+----+ | | 4/95% | 6/50% ^|^

5/60%
v

+----+ +-----+ -----> +-----+ ---> +-----+ ---> +-----+

Edge-------1--------2------4------7
Node-----> +-----+ +-----+7/60% +-----+5/45% +-----+

+----+ f2 || 3/50% |^

3/60% +-----+5/55% +-----+ 3/75%
+-----------5------6---------+

+----------> +-----+ ---> +-----+ ---------+ (b) CCDR Algorithm

Figure 5: Case Study for CCDR's Algorithm

4.2. Topology Simulation

Moving on from the specific case study, we now look at a class of networks that's more representative of real deployments 🏗️. It has a fully linked core network that connects edge nodes, and those edge nodes each connect to only a subset of the core. An example of such a topology is shown in Figure 6 for the case of 4 core nodes and 5 edge nodes. The CCDR simulations presented in this work use topologies involving 100 core nodes and 400 edge nodes. The resulting graph doesn't fit on this page, but a network of this scale is similar to what is deployed in production environments 🤯.

+----+ /|Edge|\

+----+

+----+ +----+ +----+

Edge----Core-----Core---------+

+----+ +----+ +----+ | / | \ / | | +----+ | \ / | |

EdgeX

+----+ | / \ | | \ | / \ | | +----+ +----+ +----+ |

Edge----Core-----Core

+----+ +----+ +----+ |

+------\ +----+
---Edge

+-----------------/ +----+

Figure 6: Topology of Simulation

For the simulations, the number of links connecting one edge node to the set of core nodes is randomly chosen between two and thirty, and the total number of links is more than 20,000 🔢. Each link has a congestion threshold, which can be set arbitrarily, for example to 90% of the nominal link capacity, without affecting the simulation results 🎯.

4.3. Traffic Matrix Simulation

For each topology, a traffic matrix is generated based on the link capacity of the topology. It can result in many kinds of situations, such as congestion, mild congestion, and non-congestion 🌤️.

In the CCDR simulation, the dimension of the traffic matrix is 500*500 (100 core nodes plus 400 edge nodes). About 20% of links are overloaded when the Open Shortest Path First (OSPF) protocol is used in the network 😭.

4.4. CCDR End-to-End Path Optimization ✨

okay so CCDR E2E path optimization is all about finding the best path 💅 which means the one with the lowest metric value, AND where utilization on every single link of that path stays way below the congestion threshold 🫶 lowest metric alone is not the vibe bestie. Based on the current state of the network, the PCE inside the CCDR framework mixes the shortest path algorithm with a penalty theory from classical optimization and graph theory 🧠✨

So picture this: there's a background traffic matrix that isn't scheduled 🙈 and then a batch of new flows shows up. The E2E path optimization figures out the optimal paths for those new flows, and the paths it picks cause the least congestion degree to the network 👑

The link Utilization Increment Degree (UID), once the new flows are added, is shown in Figure 7 🦋 The first graph is the UID with OSPF, and the second graph is the UID with CCDR E2E path optimization. The average UID on the first graph is more than 30%. After path optimization, the average UID drops to less than 5% 😭 which is an eye-catching decrease compared to the path chosen based on OSPF. Iconic 🔥

ngl real-world results never match simulations perfectly 🙃 for example, real-world topologies are likely to show correlation in how edge nodes attach to the core, and these results don't capture that. BUT the dramatic improvement in UID, plus the choice of a simulated topology that looks like real-world conditions, suggests that real deployments will also see significant UID improvement 🌈💖

          +-----------------------------------------------------------+
          |                *                               *    *    *|
        60|                *                             * * *  *    *|
          |*      *       **     * *         *   *   *  ** * *  * * **|
          |*   * ** *   * **   *** **  *   * **  * * *  ** * *  *** **|
          |* * * ** *  ** **   *** *** **  **** ** ***  **** ** *** **|
        40|* * * ***** ** ***  *** *** **  **** ** *** ***** ****** **|
    UID(%)|* * ******* ** ***  *** ******* **** ** *** ***** *********|
          |*** ******* ** **** *********** *********** ***************|
          |******************* *********** *********** ***************|
        20|******************* ***************************************|
          |******************* ***************************************|
          |***********************************************************|
          |***********************************************************|
         0+-----------------------------------------------------------+
         0    100   200   300   400   500   600   700   800   900  1000
          +-----------------------------------------------------------+
          |                                                           |
        60|                                                           |
          |                                                           |
          |                                                           |
          |                                                           |
        40|                                                           |
    UID(%)|                                                           |
          |                                                           |
          |                                                           |
        20|                                                           |
          |                                                          *|
          |                                     *                    *|
          |        *         *  *    *       *  **                 * *|
         0+-----------------------------------------------------------+
         0    100   200   300   400   500   600   700   800   900  1000
                               Flow Number

          Figure 7: Simulation Results with Congestion Elimination

4.5. Network Temporal Congestion Elimination ✨

during the simulations, different degrees of network congestion were in play 🌸 so to see how CCDR affects link congestion, we look at the Congestion Degree (CD) of a link, which is defined as the link utilization beyond its threshold 📊

The CCDR congestion elimination performance is shown in Figure 8 ✨ The first graph is the CD distribution before congestion elimination kicks in, and the average CD of all congested links is about 20% 😬 The second graph is the CD distribution after the congestion elimination process. Only twelve links out of the total 20,000 exceed the threshold, and every CD value is less than 3% 🎀 So once the traffic is scheduled away from the congested paths, network congestion is greatly eliminated and network utilization is in balance. Main character energy 👑

               Before congestion elimination
           +-----------------------------------------------------------+
           |                *                            ** *   ** ** *|
         20|                *                     *      **** * ** ** *|
           |*      *       **     * **       **  **** * ***** *********|
           |*   *  * *   * **** ****** *  ** *** **********************|
         15|* * * ** *  ** **** ********* *****************************|
           |* * ******  ******* ********* *****************************|
     CD(%) |* ********* ******* ***************************************|
         10|* ********* ***********************************************|
           |*********** ***********************************************|
           |***********************************************************|
          5|***********************************************************|
           |***********************************************************|
           |***********************************************************|
          0+-----------------------------------------------------------+
              0            0.5            1            1.5            2

                        After congestion elimination
          +-----------------------------------------------------------+
          |                                                           |
        20|                                                           |
          |                                                           |
          |                                                           |
        15|                                                           |
          |                                                           |
    CD(%) |                                                           |
        10|                                                           |
          |                                                           |
          |                                                           |
        5 |                                                           |
          |                                                           |
          |        *        **  * *  *  **   *  **                 *  |
        0 +-----------------------------------------------------------+
           0            0.5            1            1.5            2
                            Link Number(*10000)

          Figure 8: Simulation Results with Congestion Elimination

It is clear that by using an active path-computation mechanism that takes observed link traffic and congestion into account, congestion events can be greatly reduced 🦋 Only when a preponderance of links in the network are near their congestion threshold will the central controller be unable to find a clear path. Compare that to a static metric-based procedure, which will produce congested paths once a single bottleneck approaches its capacity 🙈 More detailed information about the algorithm can be found in [PTCS] 📚💜

5. CCDR Deployment Consideration ✨

okay so the scenarios and simulation results above show that one single general solution can cope with multiple complex situations 🧚 The specific situations considered aren't known to have any special properties, so the benefits shown are expected to have general applicability 🌈 That means using a centralized controller for the more complex optimal path computations in a native IP network should bring significant improvements without impacting the underlying network infrastructure 💖

For intra-domain or inter-domain native IP TE scenarios, deploying CCDR looks pretty much the same: the centralized controller computes paths, and no changes are needed to the underlying network infrastructure 🫶 This universal deployment characteristic is a lowkey game changer, since operators don't have to differentiate between the intra-domain and inter-domain TE cases. Generic traffic-engineering solution energy 👑

To deploy CCDR, the PCE should collect the underlying network topology dynamically, for example via Border Gateway Protocol - Link State (BGP-LS) [RFC7752] 🦋 It also needs to gather network traffic information periodically from the network management platform. The simulation results show that the PCE can compute the E2E optimal path within seconds, so it can cope with a change to the underlying network in a matter of minutes ⏱️✨ If you need more agile requirements, you'd need to increase the sample rate of the underlying network and decrease the detection and notification interval of the underlying network 🔥 The methods of gathering this information, and of decreasing its latency, are out of the scope of this document 🙈

6. Security Considerations 🔐

Okay, this part is serious bestie 😭 This document mainly looks at the integration of distributed protocols and the central control capability of a PCE. While centralized control can definitely simplify managing a network across the various traffic-engineering scenarios described here, it also introduces a new point that may be easily attacked 💀 Solutions for CCDR scenarios need to consider protection of the PCE and communication with the underlying devices, no skipping this one 🛡️

[RFC5440] and [RFC8253] provide additional information 📚

The control priority and interaction process should also be carefully designed for the combination of the distributed protocol and central control 🫶 Generally, the central control instructions should have higher priority than the forwarding actions determined by the distributed protocol. When communication between the PCE and the underlying devices is disrupted, the distributed protocol should take control of the underlying network 🌸 [PCE-NATIVE-IP] provides more considerations corresponding to the solution 💜

7. IANA Considerations 🎀

This document has no IANA actions 💅

8. References ✨

8.1. Normative References ✨

These are the must-read docs 💅 Here's the full list, exactly as written:

8.2. Informative References 🦋

Bestie, these are the extra reads for the curious 🤓

Acknowledgements 🙏

Huge thank-yous to Deborah Brungard, Adrian Farrel, Huaimo Chen, Vishnu Beeram, and Lou Berger for backing this doc and dropping comments 🫶

Also a big shoutout to Benjamin Kaduk for his careful review and super valuable suggestions ✨ And thanks to Roman Danyliw, Alvaro Retana, and Éric Vyncke for their reviews and comments 💜

Contributors 🌸

Lu Huang contributed to the content of this document 💖

Authors' Addresses 💅

Meet the squad 👑

Aijun Wang
China Telecom
Beiqijia Town, Changping District
Beijing
Beijing, 102209
China

Email: wangaj3@chinatelecom.cn
Xiaohong Huang
Beijing University of Posts and Telecommunications
No.10 Xitucheng Road, Haidian District
Beijing
China

Email: huangxh@bupt.edu.cn
Caixia Kou
Beijing University of Posts and Telecommunications
No.10 Xitucheng Road, Haidian District
Beijing
China

Email: koucx@lsec.cc.ac.cn
Zhenqiang Li
China Mobile
32 Xuanwumen West Ave, Xicheng District
Beijing
100053
China

Email: li_zhenqiang@hotmail.com
Penghui Mi
Huawei Technologies
Tower C of Bldg.2, Cloud Park, No.2013 of Xuegang Road
Shenzhen
Bantian,Longgang District, 518129
China

Email: mipenghui@huawei.com

That's the whole doc, bestie! 🌈✨ You made it through! 🎀

📦 where this page comes from