RFC 6413
Benchmarking Methodology for Link-State IGP Data-Plane Route Convergence
RFC 6413: Benchmarking Methodology for Link-State IGP Data-Plane Route Convergence ✨
okay bestie, here's the header tea 🫖 this is RFC 6413, a benchmarking methodology doc for link-state IGP data-plane route convergence 🦋
- Authors: S. Poretsky (Allot Communications), B. Imhoff (Juniper Networks), and K. Michielsen (Cisco Systems) 💜
- Category: Informational
- Date: November 2011 📅
- ISSN: 2070-1721
- Obsoletes / Updates: the header doesn't list anything this RFC obsoletes or updates 🙅♀️
Abstract
okay so the abstract is giving "here's what we're doing" 🎀 This document describes the methodology for benchmarking Link-State Interior Gateway Protocol (IGP) Route Convergence. The methodology is to be used for benchmarking IGP convergence time through externally observable (black-box) data-plane measurements. It can be applied to any link-state IGP, such as IS-IS and OSPF 🌈
Status of This Memo
not a Standards Track spec, just published for info purposes 📚 It's still a real product of the IETF, though, and it reflects what the IETF community agreed on 🫶 It got public review and was approved for publication by the Internet Engineering Steering Group (IESG) ✨ But not every IESG-approved document is a candidate for any level of Internet Standard, so check Section 2 of RFC 5741 if you're curious 🧐
For the current status, any errata, and how to send feedback, go to http://www.rfc-editor.org/info/rfc6413 🔗
Copyright Notice
Copyright (c) 2011 IETF Trust and the persons identified as the document authors. All rights reserved. 💅
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as in effect on the date of publication. Please read these carefully, because they spell out your rights and restrictions for this document 📜 Code Components pulled out of this document must include Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come without warranty, as described in the Simplified BSD License ⚖️
This document may contain material from IETF Documents or IETF Contributions that were published or made publicly available before November 10, 2008. The people controlling the copyright in some of that older material may not have given the IETF Trust permission to allow modifications outside the IETF Standards Process. So unless you get an adequate license from those copyright holders, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be made outside the IETF Standards Process 🙈 The one exceptions are formatting it for publication as an RFC or translating it into languages other than English 🌸
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.1. Motivation . . . . . . . . . . . . . . . . . . . . . . . . 4
1.2. Factors for IGP Route Convergence Time . . . . . . . . . . 4
1.3. Use of Data Plane for IGP Route Convergence
Benchmarking . . . . . . . . . . . . . . . . . . . . . . . 5
1.4. Applicability and Scope . . . . . . . . . . . . . . . . . 6
2. Existing Definitions . . . . . . . . . . . . . . . . . . . . . 6
3. Test Topologies . . . . . . . . . . . . . . . . . . . . . . . 7
3.1. Test Topology for Local Changes . . . . . . . . . . . . . 7
3.2. Test Topology for Remote Changes . . . . . . . . . . . . . 8
3.3. Test Topology for Local ECMP Changes . . . . . . . . . . . 10
3.4. Test Topology for Remote ECMP Changes . . . . . . . . . . 11
3.5. Test topology for Parallel Link Changes . . . . . . . . . 11
4. Convergence Time and Loss of Connectivity Period . . . . . . . 12
4.1. Convergence Events without Instant Traffic Loss . . . . . 13
4.2. Loss of Connectivity (LoC) . . . . . . . . . . . . . . . . 16
5. Test Considerations . . . . . . . . . . . . . . . . . . . . . 17
5.1. IGP Selection . . . . . . . . . . . . . . . . . . . . . . 17
5.2. Routing Protocol Configuration . . . . . . . . . . . . . . 17
5.3. IGP Topology . . . . . . . . . . . . . . . . . . . . . . . 17
5.4. Timers . . . . . . . . . . . . . . . . . . . . . . . . . . 18
5.5. Interface Types . . . . . . . . . . . . . . . . . . . . . 18
5.6. Offered Load . . . . . . . . . . . . . . . . . . . . . . . 18
5.7. Measurement Accuracy . . . . . . . . . . . . . . . . . . . 19
5.8. Measurement Statistics . . . . . . . . . . . . . . . . . . 20
5.9. Tester Capabilities . . . . . . . . . . . . . . . . . . . 20
6. Selection of Convergence Time Benchmark Metrics and Methods . 20
6.1. Loss-Derived Method . . . . . . . . . . . . . . . . . . . 21
6.1.1. Tester Capabilities . . . . . . . . . . . . . . . . . 21
6.1.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 21
6.1.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 21
6.2. Rate-Derived Method . . . . . . . . . . . . . . . . . . . 22
6.2.1. Tester Capabilities . . . . . . . . . . . . . . . . . 22
6.2.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 23
6.2.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 23
6.3. Route-Specific Loss-Derived Method . . . . . . . . . . . . 24
6.3.1. Tester Capabilities . . . . . . . . . . . . . . . . . 24
6.3.2. Benchmark Metrics . . . . . . . . . . . . . . . . . . 24
6.3.3. Measurement Accuracy . . . . . . . . . . . . . . . . . 24
7. Reporting Format . . . . . . . . . . . . . . . . . . . . . . . 25
8. Test Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 26
8.1. Interface Failure and Recovery . . . . . . . . . . . . . . 27
8.1.1. Convergence Due to Local Interface Failure and
Recovery . . . . . . . . . . . . . . . . . . . . . . . 27
8.1.2. Convergence Due to Remote Interface Failure and
Recovery . . . . . . . . . . . . . . . . . . . . . . . 28
8.1.3. Convergence Due to ECMP Member Local Interface
Failure and Recovery . . . . . . . . . . . . . . . . . 30
8.1.4. Convergence Due to ECMP Member Remote Interface
Failure and Recovery . . . . . . . . . . . . . . . . . 31
8.1.5. Convergence Due to Parallel Link Interface Failure
and Recovery . . . . . . . . . . . . . . . . . . . . . 32
8.2. Other Failures and Recoveries . . . . . . . . . . . . . . 33
8.2.1. Convergence Due to Layer 2 Session Loss and
Recovery . . . . . . . . . . . . . . . . . . . . . . . 33
8.2.2. Convergence Due to Loss and Recovery of IGP
Adjacency . . . . . . . . . . . . . . . . . . . . . . 34
8.2.3. Convergence Due to Route Withdrawal and
Re-Advertisement . . . . . . . . . . . . . . . . . . . 35
8.3. Administrative Changes . . . . . . . . . . . . . . . . . . 37
8.3.1. Convergence Due to Local Interface Administrative
Changes . . . . . . . . . . . . . . . . . . . . . . . 37
8.3.2. Convergence Due to Cost Change . . . . . . . . . . . . 38
9. Security Considerations . . . . . . . . . . . . . . . . . . . 39
10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 40
11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 40
11.1. Normative References . . . . . . . . . . . . . . . . . . . 40
11.2. Informative References . . . . . . . . . . . . . . . . . . 411. Introduction ✨
1.1. Motivation
convergence time is a seriously critical performance parameter 💅 Service Providers use IGP convergence time as a key metric of router design and architecture 🏗️ Fast network convergence can be optimally achieved through deployment of fast converging routers 🚀 Customers of Service Providers use packet loss due to Interior Gateway Protocol (IGP) convergence as a key metric of their network service quality 📉 IGP route convergence is a Direct Measure of Quality (DMOQ) when benchmarking the data plane 🎯
the fundamental basis network users and operators use to benchmark convergence is packet loss and other packet impairments 😭 these are externally observable events that directly hit their application performance, so they really matter 💀 that's why it's so important to develop a standard methodology for benchmarking link-state IGP convergence time through externally observable (black-box) data-plane measurements 🫶 all the factors that contribute to convergence time get accounted for when you measure on the data plane 🧚
1.2. Factors for IGP Route Convergence Time
there are four major categories of factors that contribute to the measured IGP convergence time 🌈 As discussed in [Vi02], [Ka02], [Fi02], [Al00], [Al02], and [Fr05], these categories are Event Detection, Shortest Path First (SPF) Processing, Link State Advertisement (LSA) / Link State Packet (LSP) Advertisement, and Forwarding Information Base (FIB) Update 💖 each one has a bunch of components that influence convergence time, including (but not limited to!) the list below:
- Event Detection
- Physical-Layer Failure/Recovery Indication Time
- Layer 2 Failure/Recovery Indication Time
- IGP Hello Dead Interval
- SPF Processing
- SPF Delay Time
- SPF Hold Time
- SPF Execution Time
- LSA/LSP Advertisement
- LSA/LSP Generation Time
- LSA/LSP Flood Packet Pacing
- LSA/LSP Retransmission Packet Pacing
- FIB Update
- Tree Build Time
- Hardware Update Time
- Increased Forwarding Delay due to Queueing
the contribution of each of those factors is going to vary with each router vendor's architecture and IGP implementation 🦋 routers might have a centralized forwarding architecture, where one forwarding table gets calculated and referenced for every arriving packet 🗂️ or they might have a distributed forwarding architecture, where the central forwarding table gets calculated and then pushed out to the interfaces for local look-up as packets arrive 🧩 those distributed forwarding tables are typically maintained (loaded and changed) in software 💻
this variation in router architecture and implementation means you need a convergence test that considers all of these components contributing to convergence time, and that stays independent of the Device Under Test (DUT) architecture and implementation 🎀 the win here is that it enables black-box testing, where you don't need to know anything about the routers' internal implementation 🙌 that makes it possible to use the convergence benchmarking metrics validly when you compare routers from different vendors 👑
convergence performance is tightly linked to how many tasks a router has to juggle 🤹♀️ since the most important tasks are mainly about the control plane and the data plane, the more you stress the DUT, the closer your measurement results will match what you'd see in a live production environment 🔥
1.3. Use of Data Plane for IGP Route Convergence Benchmarking ✨
okay so customers of Service Providers literally use packet loss and other packet impairments as metrics to calculate convergence time 😭 and honestly that makes sense, because those are externally observable events that directly hit their app performance 💅. so it's important to develop a standard router benchmarking methodology that is a Direct Measure of Quality (DMOQ) for measuring IGP convergence 🎀
an extra perk of using packet loss to calculate IGP Route Convergence time is that it lets you design black-box tests 🦋 here's the vibe: you offer data traffic to the Device Under Test (DUT), force an emulated network event to happen, and then measure the packet loss and other impaired packets from the outside to calculate the convergence time. you don't need to know the DUT architecture or the IGP implementation 🙈 you also don't have to rely on the DUT to produce the test results, and you don't need to build intrusive test harnesses for it. all the factors that contribute to convergence time get accounted for by measuring on the data plane 👑
other work from the Benchmarking Methodology Working Group (BMWG) focuses on characterizing single router control-plane convergence. see [Ma05], [Ma05t], and [Ma05c] 🌸
1.4. Applicability and Scope 🧚
the methodology in this document can be applied to IPv4 and IPv6 traffic and to link-state IGPs such as IS-IS [Ca90][Ho08], OSPF [Mo98][Co08], and others 💖 IGP adjacencies established over any kind of tunnel (such as Traffic Engineering tunnels) are outside the scope of this document 🙅♀️ convergence time benchmarking in topologies with IGP adjacencies that aren't point-to-point will be covered in a later document ⭐ convergence from Bidirectional Forwarding Detection (BFD) is also outside the scope, and so are Non-Stop Forwarding (NSF), Non-Stop Routing (NSR), Graceful Restart (GR), and any other High Availability mechanism 😭 fast reroute stuff like IP Fast-Reroute [Sh10i] or MPLS Fast-Reroute [Pa05] is outside the scope too 🫶
2. Existing Definitions 📚
okay the keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14, RFC 2119 [Br97] 🔥 RFC 2119 is the thing that defines how these keywords are used to make the intent of Standards Track documents as clear as possible. this document uses those keywords, but it is not a Standards Track document, so no cap 🧁
this document uses much of the terminology defined in [Po11t]. for any conflicting content, this document supersedes [Po11t] 💅 it also uses existing terminology from other BMWG documents. examples include, but aren't limited to, the following:
Throughput [Br91], Section 3.17
Offered Load [Ma98], Section 3.5.2
Forwarding Rate [Ma98], Section 3.6.1
Device Under Test (DUT) [Ma98], Section 3.1.1
System Under Test (SUT) [Ma98], Section 3.1.2
Out-of-Order Packet [Po06], Section 3.3.4
Duplicate Packet [Po06], Section 3.3.5
Stream [Po06], Section 3.3.2
Forwarding Delay [Po06], Section 3.2.4
IP Packet Delay Variation (IPDV) [De02], Section 1.2
Loss Period [Ko02], Section 43. Test Topologies 🗺️
3.1. Test Topology for Local Changes 🌈
okay so Figure 1 shows the test topology for measuring IGP convergence time due to local Convergence Events. that includes Local Interface failure and recovery (Section 8.1.1), Layer 2 session failure and recovery (Section 8.2.1), and IGP adjacency failure and recovery (Section 8.2.2) 💖 this same topology is also used to measure IGP convergence time due to route withdrawal and re-advertisement (Section 8.2.3), and due to route cost change (Section 8.3.2) Convergence Events ✨ IGP adjacencies MUST be established between Tester and DUT: one on the Ingress Interface, one on the Preferred Egress Interface, and one on the Next-Best Egress Interface 🎀 for this, the Tester emulates three routers (RTa, RTb, and RTc), and each one establishes one adjacency with the DUT 🫶
-------
| | Preferred .......
| |------------------. RTb .
....... Ingress | | Egress Interface .......
. RTa .------------| DUT |
....... Interface | | Next-Best .......
| |------------------. RTc .
| | Egress Interface .......
-------Figure 1: IGP convergence test topology for local changes
okay Figure 2 is the topology for measuring IGP convergence time due to local Convergence Events with a non-Equal Cost Multipath (ECMP) Preferred Egress Interface and ECMP Next-Best Egress Interfaces (Section 8.1.1) 🦋 here the DUT is configured with each Next-Best Egress Interface as a member of a single ECMP set, and the Preferred Egress Interface is not a member of an ECMP set 💅 the Tester emulates N+2 neighbor routers (N>0): one for the Ingress Interface (RTa), one for the Preferred Egress Interface (RTb), and N routers for the members of the ECMP set (RTc1...RTcN) 😭 IGP adjacencies MUST be established between Tester and DUT: one on the Ingress Interface, one on the Preferred Egress Interface, and one on each member of the ECMP set 🔥 when the test says to observe the Next-Best Egress Interface statistics, the combined statistics for all ECMP members should be observed 🍓
-------
| | Preferred .......
| |------------------. RTb .
| | Egress Interface .......
| |
| | ECMP Set ........
....... Ingress | |------------------. RTc1 .
. RTa .------------| DUT | Interface 1 ........
....... Interface | | .
| | .
| | .
| | ECMP Set ........
| |------------------. RTcN .
| | Interface N ........
-------Figure 2: IGP convergence test topology for local changes with non- ECMP to ECMP convergence
3.2. Test Topology for Remote Changes 🌸
okay Figure 3 shows the topology for measuring IGP convergence time due to Remote Interface failure and recovery (Section 8.1.2) 💜 in this one, the two routers DUT1 and DUT2 are considered the System Under Test (SUT), and they SHOULD be identically configured devices of the same model 🫶 IGP adjacencies MUST be established between Tester and SUT, one on the Ingress Interface, one on the Preferred Egress Interface, and one on the Next-Best Egress Interface 🧚 the Tester emulates three routers for this (RTa, RTb, and RTc) 😭 also, a packet forwarding loop (also known as a micro-loop, see [Sh10]) may occur transiently between DUT1 and DUT2 during convergence 🙈
--------
| | -------- Preferred .......
| |--| DUT2 |------------------. RTb .
....... Ingress | | -------- Egress Interface .......
. RTa .------------| DUT1 |
....... Interface | | Next-Best .......
| |----------------------------. RTc .
| | Egress Interface .......
--------Figure 3: IGP convergence test topology for remote changes
okay Figure 4 is for measuring IGP convergence time due to remote Convergence Events with a non-ECMP Preferred Egress Interface and ECMP Next-Best Egress Interfaces (Section 8.1.2) 🎀 the two routers DUT1 and DUT2 are the System Under Test (SUT), and they MUST be identically configured devices of the same model ✨ router DUT1 is configured with the Next-Best Egress Interface as an ECMP set of interfaces, and the Preferred Egress Interface of DUT1 is not a member of an ECMP set 💅 the Tester emulates N+2 neighbor routers (N>0): one for the Ingress Interface (RTa), one for DUT2 (RTb), and one for each member of the ECMP set (RTc1...RTcN) 👑 IGP adjacencies MUST be established between Tester and SUT, one on each interface of the SUT, and each of the N+2 routers emulated by the Tester establishes one adjacency with the SUT 🔥 there's a possibility of a packet-forwarding loop that may occur transiently between DUT1 and DUT2 during convergence (micro-loop, see [Sh10]) 😭 and when the test says to observe the Next-Best Egress Interface statistics, the combined statistics for all members of the ECMP set should be observed 🍓
--------
| | -------- Preferred .......
| |--| DUT2 |------------------. RTb .
| | -------- Egress Interface .......
| |
| | ECMP Set ........
....... Ingress | |----------------------------. RTc1 .
. RTa .------------| DUT1 | Interface 1 ........
....... Interface | | .
| | .
| | .
| | ECMP Set ........
| |----------------------------. RTcN .
| | Interface N ........
--------Figure 4: IGP convergence test topology for remote changes with non-ECMP to ECMP convergence
3.3. Test Topology for Local ECMP Changes ✨
okay so this one is about what happens when a local Convergence Event hits one member of an Equal Cost Multipath (ECMP) set 🦋 (see Section 8.1.3). The DUT has every egress interface in one single ECMP set, and the Tester pretends to be N+1 next-hop routers: one for the Ingress Interface (RTa) and one for each ECMP member (RTb1...RTbN). 💖 IGP adjacencies MUST be established between Tester and DUT, one on the Ingress Interface and one on each ECMP member. So each of those N+1 emulated routers gets its own adjacency with the DUT. 🫶 And if the test wants Next-Best Egress Interface statistics, the combined stats for every ECMP member except the one that got hit should be observed. ngl that's a very specific little rule, but it's giving precision 💅
-------
| | ECMP Set ........
| |-------------. RTb1 .
| | Interface 1 ........
....... Ingress | | .
. RTa .------------| DUT | .
....... Interface | | .
| | ECMP Set ........
| |-------------. RTbN .
| | Interface N ........
-------
Figure 5: IGP convergence test topology for local ECMP changes3.4. Test Topology for Remote ECMP Changes 🌸
this is the remote version 👑 (see Section 8.1.4). Now we've got two routers, DUT1 and DUT2, and together they're the System Under Test (SUT). They MUST be identically configured devices of the same model, no cap, because the whole point is comparing like with like. 🍓 DUT1 has each egress interface as a member of a single ECMP set, and the Tester emulates N+1 neighbor routers (N>0): one for the Ingress Interface (RTa) and one for each ECMP member (RTb1...RTbN). IGP adjacencies MUST be established between Tester and SUT, one on each interface of the SUT. Each of those N+1 emulated routers gets one adjacency with the SUT. Just so you can picture it: N-1 emulated routers sit next to DUT1's egress interfaces, one sits next to DUT1's Ingress Interface, and one sits next to DUT2. 🧚
The sneaky part 🙈 is that there's a possibility of a packet-forwarding loop that can happen transiently between DUT1 and DUT2 while things converge. This is called a micro-loop, see [Sh10]. Lowkey terrifying, honestly 💀 And again, if the test wants Next-Best Egress Interface statistics, the combined stats for all ECMP members except the affected one should be observed.
--------
| | ECMP Set -------- ........
| |-------------| DUT2 |---. RTb1 .
| | Interface 1 -------- ........
| |
| | ECMP Set ........
....... Ingress | |------------------------. RTb2 .
. RTa .------------| DUT1 | Interface 2 ........
....... Interface | | .
| | .
| | .
| | ECMP Set ........
| |------------------------. RTbN .
| | Interface N ........
--------
Figure 6: IGP convergence test topology for remote ECMP changes3.5. Test topology for Parallel Link Changes 🦋
last topology in this part! 🎀 This one is for local Convergence Events affecting members of a Parallel Link (see Section 8.1.5). The DUT has each egress interface as a member of a Parallel Link. The Tester pretends to be two neighbor routers: one for the Ingress Interface (RTa) and one for the Parallel Link members (RTb). IGP adjacencies MUST be established on the Ingress Interface and on all N members of the Parallel Link between Tester and DUT (N>0). So the emulated routers end up building N+1 adjacencies with the DUT, which is a lot of little handshakes but we love a multi-tasker 😭 Same deal as before for the stats: when the test wants Next-Best Egress Interface statistics, the combined stats for every Parallel Link member except the one affected by the Convergence Event should be observed.
------- .......
| | Parallel Link . .
| |----------------. .
| | Interface 1 . .
....... Ingress | | . . .
. RTa .------------| DUT | . . RTb .
....... Interface | | . . .
| | Parallel Link . .
| |----------------. .
| | Interface N . .
------- .......
Figure 7: IGP convergence test topology for Parallel Link changes4. Convergence Time and Loss of Connectivity Period 💜
two concepts to hold onto in this section, bestie: convergence time and loss of connectivity period 🧠✨ They sound similar but they are NOT the same, so pay attention 👀
Route Convergence [Po11t] time is the stretch between the Convergence Event Instant [Po11t] and the moment the DUT is ready to forward traffic for a specific route on its Next-Best Egress Interface. 🌈 And the DUT has to stay in that state for the whole Sustained Convergence Validation Time [Po11t], so no fake-outs allowed 💅 To measure it, you need to observe the Convergence Event Instant and the traffic received from the Next-Best Egress Interface.
The Route Loss of Connectivity Period [Po11t] is the time during which traffic to a specific route gets lost after a Convergence Event, until Full Convergence [Po11t] completes. This period can be made of one or more Loss Periods [Ko02]. For the test cases in this document, we expect a single Loss Period, so that's the vibe we're going for ✨ To measure it, you observe the traffic received from the Preferred Egress Interface and the traffic received from the Next-Best Egress Interface.
Quick heads-up 🚨 the Route Loss of Connectivity Period is the MOST important one, because it directly hits the network user's application performance. Users feel this one, bestie 🫶
Generally speaking, Route Convergence time is larger than or equal to the Route Loss of Connectivity Period. It depends on which Convergence Event happens and how it's applied, though. Traffic for a route can still go out the Preferred Egress Interface after the Convergence Event Instant and only then move over to the Next-Best Egress Interface. In that case, the Route Loss of Connectivity Period is shorter than the Route Convergence time. The plot twist, lol 🙈
Heads up: the two numbers only come out equal under one condition, and that condition is a real plot twist. The Convergence Event has to cause an instantaneous traffic loss for the measured route. A fiber cut on the Preferred Egress Interface is an example of exactly that kind of event ⚡
There's a second condition that applies to Route Convergence time measurements based on Connectivity Packet Loss [Po11t]: there is only a single Loss Period during Route Convergence. 🎀 For the test cases in this document, that second condition is expected to apply too, so we're in good shape 🌸
4.1. Convergence Events without Instant Traffic Loss ✨
okay so, when the Tester causes a Convergence Event itself (like an IGP cost change 💅), it MAY start to discard all traffic received from the Preferred Egress Interface right at the Convergence Event Instant, or it MAY separately watch the packets coming in from the Preferred Egress Interface before that moment 👀. Either way, these events get treated the same as the ones that cause instant traffic loss, which is so clean 🫶.
Now for convergence times without instant traffic loss at the Convergence Event Instant (whether it's real or induced by the Tester), like when a link failure gets reverted 🔄. Here the Tester SHALL only observe packet statistics on the Next-Best Egress Interface 👑. If you're using the Rate-Derived method, the Tester MUST collect a timestamp at the Convergence Event Instant ⏰. If you're using a loss-derived method, the Tester MUST measure the period in time between the Start Traffic Instant and the Convergence Event Instant. To measure that, it can collect timestamps at the Start Traffic Instant and the Convergence Event Instant 🧚.
The Convergence Event Instant plus the receive rate observations on the Next-Best Egress Interface let you derive the convergence time benchmarks using the Rate-Derived Method [Po11t]. The math is giving main character energy 💖.
Since you're only looking at packets on the Next-Best Egress Interface, the Impaired Packet count you see is the number of Impaired Packets between the Traffic Start Instant and the Convergence Recovery Instant 📦. For a loss-derived method, you need the Impaired Packet count between the Convergence Event Instant and the Convergence Recovery Instant. The time between the Traffic Start Instant and the Convergence Event Instant must be accounted for too 🙈. Let me show you what I mean 🦋.
Figure 8 shows a Convergence Event without instantaneous traffic loss for all routes. The top graph is the Forwarding Rate over all routes, and the bottom graph is the Forwarding Rate for one single route, Rta. Some time after the Convergence Event Instant, the Forwarding Rate on the Preferred Egress Interface starts to drop 📉. In this example, route Rta is the first one to lose packets, at time Ta. A bit later, the Forwarding Rate on the Next-Best Egress Interface starts to climb 📈. Rta is also the first route to finish converging, at time Ta' 🏁.
^
Fwd |
Rate |------------- ............
| \ .
| \ .
| \ .
| \ .
|.................-.-.-.-.-.-.----------------
+----+-------+---------------+----------------->
^ ^ ^ ^ time
T0 CEI Ta Ta'
^
Fwd |
Rate |------------- .................
Rta | | .
| | .
|.............-.-.-.-.-.-.-.-.----------------
+----+-------+---------------+----------------->
^ ^ ^ ^ time
T0 CEI Ta Ta'
Preferred Egress Interface: ---
Next-Best Egress Interface: ...
T0 : Start Traffic Instant
CEI : Convergence Event Instant
Ta : the time instant packet loss for route Rta starts
Ta' : the time instant packet impairment for route Rta ends
Figure 8If you only observe packets received on the Next-Best Egress Interface, you can work out the duration of the loss period for route Rta from the received packets, as in Equation 1 💅. Since the Convergence Event Instant is where the convergence time measurement starts, the time between T0 and CEI has to be subtracted from that result to get the convergence time, as in Equation 2 🧮.
Next-Best Egress Interface loss period
= (packets transmitted
- packets received from Next-Best Egress Interface) / tx rate
= Ta' - T0
Equation 1
convergence time
= Next-Best Egress Interface loss period - (CEI - T0)
= Ta' - CEI
Equation 24.2. Loss of Connectivity (LoC) 💔
The Route Loss of Connectivity Period SHOULD be measured using the Route-Specific Loss-Derived Method 🎯. Here's the thing: the start and end instants of the Route Loss of Connectivity Period can be different for each route, so you can't accurately pin them down by only looking at global statistics over all routes 🙅♀️. Let me give you an example 🫶.
After a Convergence Event, route Rta is the first route where packet impairment starts, so its Route Loss of Connectivity Period starts at time Ta. Route Rtb is the last route where packet impairment starts, so its Route Loss of Connectivity Period starts at time Tb, with Tb>Ta 😳.
^
Fwd |
Rate |-------- -----------
| \ /
| \ /
| \ /
| \ /
| ---------------
+------------------------------------------>
^ ^ ^ ^ time
Ta Tb Ta' Tb'
Tb'' Ta''
Figure 9: Example Route Loss Of Connectivity PeriodNow picture a DUT where route Rta is the first route whose traffic loss ends, at time Ta' (with Ta'>Tb), and route Rtb is the last route whose traffic loss ends, at time Tb' (with Tb'>Ta'). If you only look at global traffic statistics over all routes, the minimum Route Loss of Connectivity Period you'd measure is Ta'-Ta, and the maximum calculated one is Tb'-Ta. But the real minimum and maximum Route Loss of Connectivity Periods are Ta'-Ta and Tb'-Tb 🤯. Plug in Ta=0, Tb=1, Ta'=3, and Tb'=5, and the global stats give you a Loss of Connectivity Period between 3 and 5, while the real one is between 3 and 4 📊.
Next up, a DUT where route Rtb is the first route whose packet loss ends, at time Tb'', and route Rta is the last route whose packet impairment ends, at time Ta''. Looking only at global traffic statistics, the minimum and maximum Route Loss of Connectivity Periods would come out as Tb''-Ta and Ta''-Ta. The real ones are Tb''-Tb and Ta''-Ta. With Ta=0, Tb=1, Ta''=5, and Tb''=3, the global stats say between 3 and 5, but the real Loss of Connectivity Period is between 2 and 5 😭.
So, bestie, these two implementation variations give you the exact same derived minimum and maximum Route Loss of Connectivity Periods when you only watch global packet statistics, even though the real Route Loss of Connectivity Periods are totally different 💀. That's why per-route measurement matters ✨.
5. Test Considerations 🎀
5.1. IGP Selection 🧭
The test cases described in Section 8 can be used for link-state IGPs, like IS-IS or OSPF 🌐. The IGP convergence time test methodology is identical for both 💜.
5.2. Routing Protocol Configuration ⚙️
The results you get for IGP convergence time can vary if other routing protocols are enabled and the routes they learn get installed 🌀. IGP convergence times SHOULD be benchmarked without routes installed from other protocols 🙌. And any enabled IGP routing protocol extension (like extensions for Traffic Engineering) and any enabled IGP routing protocol security mechanism must be reported along with the results 📝.
5.3. IGP Topology 🕸️
The Tester emulates a single IGP topology. The DUT establishes IGP adjacencies with one or more of the emulated routers in that single topology 🤝. See the test topology details in Section 3 🗺️. The emulated topology SHOULD only be advertised on the DUT egress interfaces 📡.
The number of IGP routes, the number of nodes in the topology, and the type of topology all impact the measured IGP convergence time 📈. To get results that look like what you'd see in an operational network, it is RECOMMENDED that the number of installed routes and nodes closely approximate that of the network (for example, thousands of routes with tens or hundreds of nodes) 🏙️.
The number of areas (for OSPF) and levels (for IS-IS) can also change the benchmark results 🔢.
5.4. Timers ⏱️
There are timers that may impact the measured IGP convergence times. The benchmark metrics MAY be measured at any fixed values for these timers 🎲. To get results that match an operational network, it is RECOMMENDED to configure the timers with the values used in that operational network 🪞.
Examples of timers that may impact measured IGP convergence time include, but are not limited to:
- Interface failure indication
- IGP hello timer
- IGP dead-interval or hold-timer
- Link State Advertisement (LSA) or Link State Packet (LSP) generation delay
- LSA or LSP flood packet pacing
- Route calculation delay
5.5. Interface Types 🔌
All test cases in this methodology document can be run with any interface type 💖. The media type might decide which test cases you can run, though. Each interface type has its own unique mechanism for detecting link failures, and how fast that mechanism works will influence the measurement results ⚡. All interfaces MUST be the same media and Throughput [Br91][Br99] for each test case 🫧. All interfaces SHOULD be configured as point-to-point 🎀.
5.6. Offered Load 💖
okay so the Throughput of the device, as defined in [Br91] and benchmarked in [Br99] at a fixed packet size, needs to be figured out over the preferred path AND over the next-best path 🛣️✨ The Offered Load SHOULD be the minimum of the measured Throughput of the device over the primary path and over the backup path. The packet size is selectable and MUST be recorded 📏 Packet size is measured in bytes and includes the IP header and payload.
The destination addresses for the Offered Load MUST be spread out so that all routes, or a statistically representative subset of all routes, get matched, and every one of those routes gets an equal share of the Offered Load 🌈 It is RECOMMENDED to send traffic matching all routes, but a statistically representative subset of all routes can be used if you really need it to be.
Okay this part is so iconic 🔥 Splitting traffic across multiple paths (like with ECMP or Parallel Link sets) is usually done by hashing on different fields in the IP header or the headers inside it. The hashing is typically based on the IP source and destination addresses, the protocol ID, and higher-layer flow-dependent fields like TCP/UDP ports. Inside a network core, though, the hashing is mainly or only based on the IP source and destination addresses 🧚 Figuring out the hashing algorithm the DUT uses isn't always possible ahead of time, and it would break the black-box spirit of this document 🙈 So it is RECOMMENDED to use a randomly distributed range of source and destination IP addresses, protocol IDs, and higher-layer flow-dependent fields for the packets of the Offered Load (see also [Ne07]). The content of the Offered Load MUST remain the same during the test. It is RECOMMENDED to repeat a test multiple times with different random ranges of the header fields, so convergence time benchmarks get measured for different ways traffic is spread over the available paths 🦋
Heads up bestie: in the Remote Interface failure test cases using topologies 3, 4, and 6, a packet-forwarding loop can briefly happen between DUT1 and DUT2 during convergence (a micro-loop, see [Sh10]) 💀 The Time To Live (TTL) or Hop Limit value of the packets the Tester sends may influence the benchmark measurements, because it decides which device in the topology may send an ICMP Time Exceeded Message for looped packets.
The duration of the Offered Load MUST be greater than the convergence time plus the Sustained Convergence Validation Time ⏳ And offered load should send a packet to each destination before sending another packet to the same destination 🎀 It is RECOMMENDED that the packets be transmitted in a round-robin fashion with a uniform interpacket delay 💅
5.7. Measurement Accuracy 🎯
Since Impaired Packet count is what gets observed to measure the Route Convergence Time, the time between two successive packets offered to each individual route is the highest possible accuracy of any Impaired-Packet-based measurement. The higher the traffic rate offered to each route, the higher the possible measurement accuracy 📈
Also see Section 6 for method-specific measurement accuracy 🌸
5.8. Measurement Statistics 📊
The benchmark measurements may vary for each trial, because of the statistical nature of timer expirations, CPU scheduling, and so on 🎲 Evaluation of the test data must be done with an understanding of generally accepted testing practices regarding repeatability, variance, and statistical significance of a small number of trials 🧐
5.9. Tester Capabilities 🧰
It is RECOMMENDED that the Tester used to execute each test case have the following capabilities:
- Ability to establish IGP adjacencies and advertise a single IGP topology to one or more peers.
- Ability to measure Forwarding Delay, Duplicate Packets, and Out-of-Order Packets.
- An internal time clock to control timestamping, time measurements, and time calculations ⏰
- Ability to distinguish traffic load received on the Preferred and Next-Best Interfaces [Po11t].
- Ability to disable or tune specific Layer 2 and Layer 3 protocol functions on any interface(s).
The Tester MAY be capable of making non-data-plane convergence observations and using those observations for measurements. The Tester MAY be capable of sending and receiving multiple traffic Streams [Po06] ✨
Also see Section 6 for method-specific capabilities 💜
6. Selection of Convergence Time Benchmark Metrics and Methods 🦋
Different convergence time benchmark methods MAY be used to measure convergence time benchmark metrics. The Tester capabilities are important criteria for picking a specific convergence time benchmark method 🔍 The criteria to select a specific benchmark method include, but are not limited to:
Tester capabilities: Sampling Interval, number of
Stream statistics to collect
Measurement accuracy: Sampling Interval, Offered Load,
number of routes
Test specification: number of routes
DUT capabilities: Throughput, IP Packet Delay
Variation6.1. Loss-Derived Method 🍓
6.1.1. Tester Capabilities
To collect statistics of Out-of-Order Packets per flow (see [Th00], Section 3), the Offered Load SHOULD consist of multiple Streams [Po06], and each Stream SHOULD consist of a single flow. If the Tester sends multiple Streams, the measured traffic statistics for all Streams MUST be added together ➕
To verify Full Convergence completion and the Sustained Convergence Validation Time, the Tester MUST measure Forwarding Rate each Packet Sampling Interval 📉
The total number of Impaired Packets between the start of the traffic and the end of the Sustained Convergence Validation Time is what gets used to calculate the Loss-Derived Convergence Time 🧮
6.1.2. Benchmark Metrics
The Loss-Derived Method can be used to measure the Loss-Derived Convergence Time, which is the average convergence time over all routes, and the Loss-Derived Loss of Connectivity Period, which is the average Route Loss of Connectivity Period over all routes 💅
6.1.3. Measurement Accuracy
The actual value falls within the accuracy interval [-(number of destinations/Offered Load), +(number of destinations/Offered Load)] around the value as measured using the Loss-Derived Method 🎯
6.2. Rate-Derived Method 🌈
6.2.1. Tester Capabilities
To collect statistics of Out-of-Order Packets per flow (see [Th00], Section 3), the Offered Load SHOULD consist of multiple Streams [Po06], and each Stream SHOULD consist of a single flow. If the Tester sends multiple Streams, the measured traffic statistics for all Streams MUST be added together 🫶
The Tester measures Forwarding Rate each Sampling Interval. The Packet Sampling Interval affects how well the different convergence time instants can be seen. If the Packet Sampling Interval is large compared to the time between the convergence time instants, those instants may not be easy to pick out from the Forwarding Rate observation 😭 The presence of IP Packet Delay Variation (IPDV) [De02] may cause the Forwarding Rate observation to fluctuate, and that can stop the different convergence time instants from being observed correctly 🌀
The Packet Sampling Interval MUST be larger than or equal to the time between two consecutive packets to the same destination. For maximum accuracy, the Packet Sampling Interval SHOULD be as small as possible, but the presence of IPDV may require a larger Packet Sampling Interval. The Packet Sampling Interval MUST be reported 📝
IPDV makes the number of received packets fluctuate during each Packet Sampling Interval. To account for IPDV when deciding whether a convergence instant has been reached, Forwarding Delay SHOULD be observed during each Packet Sampling Interval ⏱️ The minimum and maximum number of packets expected in a Packet Sampling Interval in the presence of IPDV can be calculated with Equation 3 🧠
number of packets expected in a Packet Sampling Interval
in presence of IP Packet Delay Variation
= expected number of packets without IP Packet Delay Variation
+/-( (maxDelay - minDelay) * Offered Load)
where minDelay and maxDelay indicate (respectively) the minimum and
maximum Forwarding Delay of packets received during the Packet
Sampling Interval
Equation 3To decide whether a convergence instant has been reached, the number of packets received in a Packet Sampling Interval gets compared with the range of expected numbers of packets calculated in Equation 3 💖
6.2.2. Benchmark Metrics
The Rate-Derived Method SHOULD be used to measure First Route Convergence Time and Full Convergence Time. It SHOULD NOT be used to measure Loss of Connectivity Period (see Section 4) 🙅♀️💜
6.2.3. Measurement Accuracy ✨
Okay so, the measurement accuracy interval for the Rate-Derived Method depends on the metric being measured or calculated and on the characteristics of the related transition 🦋. IP Packet Delay Variation (IPDV) [De02] adds uncertainty to how many packets get received in a Packet Sampling Interval, and that uncertainty adds to the measurement error 💅. The effect of IPDV is not accounted for in the accuracy intervals below 🙈. IPDV matters most at the convergence instants where a variation in Forwarding Rate has to be observed. This applies to the Convergence Recovery Instant for all topologies, and for topologies with ECMP it also applies to the Convergence Event Instant and the First Route Convergence Instant. and for topologies with ECMP also Convergence Event Instant and First Route Convergence Instant).
If the Convergence Event Instant is observed on the data plane using the Rate-Derived Method, it needs to be instantaneous for all routes (see Section 4.1) ⏱️. The actual value of the Convergence Event Instant falls within the accuracy interval [-(Packet Sampling Interval + 1/Offered Load), +0] around the value measured with the Rate-Derived Method 🌸.
If the Convergence Recovery Transition is non-instantaneous for all routes, then the actual value of the First Route Convergence Instant falls within the accuracy interval [-(Packet Sampling Interval + time between two consecutive packets to the same destination), +0] around the value measured with the Rate-Derived Method. The actual value of the Convergence Recovery Instant falls within the accuracy interval [-(2 * Packet Sampling Interval), -(Packet Sampling Interval - time between two consecutive packets to the same destination)] around the value measured with the Rate-Derived Method 🧚.
The term "time between two consecutive packets to the same destination" gets added to the accuracy intervals above, and here's why 🤔. Packets are sent in a particular order to all destinations in a stream, and when some of the routes experience packet loss, it's unknown where in the transmit cycle the packets to those routes were sent. That uncertainty adds to the error 🍓.
The accuracy intervals for the derived metrics First Route Convergence Time and Rate-Derived Convergence Time are calculated from the convergence instants accuracy intervals above 💜. The actual value of First Route Convergence Time falls within the accuracy interval [-(Packet Sampling Interval + time between two consecutive packets to the same destination), +(Packet Sampling Interval + 1/Offered Load)] around the calculated value. The actual value of Rate-Derived Convergence Time falls within the accuracy interval [-(2 * Packet Sampling Interval), +(time between two consecutive packets to the same destination + 1/Offered Load)] around the calculated value 🎀.
6.3. Route-Specific Loss-Derived Method 🦋
6.3.1. Tester Capabilities
The Offered Load is made up of multiple Streams 🌈. The Tester MUST measure Impaired Packet count for each Stream separately 💅.
To verify Full Convergence completion and the Sustained Convergence Validation Time, the Tester MUST measure Forwarding Rate each Packet Sampling Interval 📈. This measurement at each Packet Sampling Interval MAY be per Stream 🫶.
Only the total number of Impaired Packets measured per Stream at the end of the Sustained Convergence Validation Time gets used to calculate the benchmark metrics with this method 🧮.
6.3.2. Benchmark Metrics
The Route-Specific Loss-Derived Method SHOULD be used to measure Route-Specific Convergence Times 💖. It is the RECOMMENDED method for measuring Route Loss of Connectivity Period 👑.
Under the conditions explained in Section 4, First Route Convergence Time and Full Convergence Time, as benchmarked using the Rate-Derived Method, may be equal to the minimum and maximum (respectively) of the Route-Specific Convergence Times ✨.
Heads up, this one's a sneaky detail: those "may" only apply under the Section 4 conditions, so don't assume they always line up 🙈.
6.3.3. Measurement Accuracy
The actual value falls within the accuracy interval [-(number of destinations/Offered Load), +(number of destinations/Offered Load)] around the value measured with the Route-Specific Loss-Derived Method 🌸.
7. Reporting Format 📋
For each test case, it is RECOMMENDED that the reporting tables below get completed 💕. All time values SHOULD be reported with a sufficiently high resolution (fractions of a second are enough to tell significant differences between measured values) ⏱️.
Parameter Units
------------------------------------- ---------------------------
Test Case test case number
Test Topology Test Topology Figure number
IGP (IS-IS, OSPF, other)
Interface Type (GigE, POS, ATM, other)
Packet Size offered to DUT bytes
Offered Load packets per second
IGP Routes Advertised to DUT number of IGP routes
Nodes in Emulated Network number of nodes
Number of Parallel or ECMP links number of links
Number of Routes Measured number of routes
Packet Sampling Interval on Tester seconds
Forwarding Delay Threshold seconds
Timer Values configured on DUT:
Interface Failure Indication Delay seconds
IGP Hello Timer seconds
IGP Dead-Interval or Hold-Time seconds
LSA/LSP Generation Delay seconds
LSA/LSP Flood Packet Pacing seconds
LSA/LSP Retransmission Packet Pacing seconds
Route Calculation Delay seconds
Test Details:
Describe the IGP extensions and IGP security mechanisms that are
configured on the DUT.
Describe how the various fields on the IP and contained headers
for the packets for the Offered Load are generated (Section 5.6).
If the Offered Load matches a subset of routes, describe how this
subset is selected.
Describe how the Convergence Event is applied; does it cause
instantaneous traffic loss or not?The table below should be completed for the initial Convergence Event and the reversion Convergence Event 🔁.
Parameter Units
------------------------------------------- ----------------------
Convergence Event (initial or reversion)
Traffic Forwarding Metrics:
Total number of packets offered to DUT number of packets
Total number of packets forwarded by DUT number of packets
Connectivity Packet Loss number of packets
Convergence Packet Loss number of packets
Out-of-Order Packets number of packets
Duplicate Packets number of packets
Excessive Forwarding Delay Packets number of packets
Convergence Benchmarks:
Rate-Derived Method:
First Route Convergence Time seconds
Full Convergence Time seconds
Loss-Derived Method:
Loss-Derived Convergence Time seconds
Route-Specific Loss-Derived Method:
Route-Specific Convergence Time[n] array of seconds
Minimum Route-Specific Convergence Time seconds
Maximum Route-Specific Convergence Time seconds
Median Route-Specific Convergence Time seconds
Average Route-Specific Convergence Time seconds
Loss of Connectivity Benchmarks:
Loss-Derived Method:
Loss-Derived Loss of Connectivity Period seconds
Route-Specific Loss-Derived Method:
Route Loss of Connectivity Period[n] array of seconds
Minimum Route Loss of Connectivity Period seconds
Maximum Route Loss of Connectivity Period seconds
Median Route Loss of Connectivity Period seconds
Average Route Loss of Connectivity Period seconds8. Test Cases 🧪
It is RECOMMENDED that all applicable test cases be performed for the best characterization of the DUT 💯. The test cases follow a generic procedure tailored to the specific DUT configuration and Convergence Event [Po11t]. This generic procedure goes like this:
- Establish DUT and Tester configurations and advertise an IGP topology from Tester to DUT.
- Send Offered Load from Tester to DUT on Ingress Interface.
- Verify traffic is routed correctly. Verify if traffic is forwarded without Impaired Packets [Po06].
- Introduce Convergence Event [Po11t].
- Measure First Route Convergence Time [Po11t].
- Measure Full Convergence Time [Po11t].
- Stop Offered Load.
- Measure Route-Specific Convergence Times, Loss-Derived Convergence Time, Route Loss of Connectivity Periods, and Loss-Derived Loss of Connectivity Period [Po11t]. At the same time, measure number of Impaired Packets [Po11t].
- Wait sufficient time for queues to drain. The duration of this time period MUST be larger than or equal to the Forwarding Delay Threshold ⏳.
- Restart Offered Load.
- Reverse Convergence Event.
- Measure First Route Convergence Time.
- Measure Full Convergence Time.
- Stop Offered Load.
- Measure Route-Specific Convergence Times, Loss-Derived Convergence Time, Route Loss of Connectivity Periods, and Loss-Derived Loss of Connectivity Period. At the same time, measure number of Impaired Packets [Po11t].
8.1. Interface Failure and Recovery 🔌
8.1.1. Convergence Due to Local Interface Failure and Recovery
Objective:
To obtain the IGP convergence measurements for Local Interface failure and recovery events 💫. The Next-Best Egress Interface can be a single interface (Figure 1) or an ECMP set (Figure 2). The test with ECMP topology (Figure 2) is OPTIONAL 🌷.
Procedure:
- Advertise an IGP topology from Tester to DUT using the topology shown in Figures 1 or 2.
- Send Offered Load from Tester to DUT on Ingress Interface.
- Verify traffic is forwarded over Preferred Egress Interface.
- Remove link on the Preferred Egress Interface of the DUT. This is the Convergence Event 💥.
- Measure First Route Convergence Time.
- Measure Full Convergence Time.
- Stop Offered Load.
- Measure Route-Specific Convergence Times and Loss-Derived Convergence Time. At the same time, measure number of Impaired Packets.
- Wait sufficient time for queues to drain.
- Restart Offered Load.
- Restore link on the Preferred Egress Interface of the DUT.
- Measure First Route Convergence Time.
- Measure Full Convergence Time.
- Stop Offered Load.
- Measure Route-Specific Convergence Times, Loss-Derived Convergence Time, Route Loss of Connectivity Periods, and Loss-Derived Loss of Connectivity Period. At the same time, measure number of Impaired Packets.
8.1.2. Convergence Due to Remote Interface Failure and Recovery
Objective:
okay bestie 🫶 we're checking IGP convergence when the Remote Interface fails and then comes back 🔌✨. The Next-Best Egress Interface can be a single interface (Figure 3) or an ECMP set (Figure 4). The ECMP topology test (Figure 4) is OPTIONAL, so don't stress if you skip it 💅.
Procedure:
- Advertise an IGP topology from Tester to SUT using the topology shown in Figures 3 or 4.
- Send Offered Load from Tester to SUT on Ingress Interface.
- Verify traffic is forwarded over Preferred Egress Interface.
- Remove link on the interface of the Tester connected to the Preferred Egress Interface of the SUT. This is the Convergence Event 💥.
- Measure First Route Convergence Time.
- Measure Full Convergence Time.
- Stop Offered Load.
- Measure Route-Specific Convergence Times and Loss-Derived Convergence Time. At the same time, measure number of Impaired Packets 📊.
- Wait sufficient time for queues to drain 🧼.
- Restart Offered Load.
- Restore link on the interface of the Tester connected to the Preferred Egress Interface of the SUT.
- Measure First Route Convergence Time.
- Measure Full Convergence Time.
- Stop Offered Load.
- Measure Route-Specific Convergence Times, Loss-Derived Convergence Time, Route Loss of Connectivity Periods, and Loss-Derived Loss of Connectivity Period. At the same time, measure number of Impaired Packets 📊🎀.
Discussion:
okay this one's a little spicy 🌶️ There's a chance a packet-forwarding loop pops up briefly between DUT1 and DUT2 while things are converging (that's a micro-loop, see [Sh10]) 🌀. That can make your convergence times and loss of connectivity periods look longer than they really are 🙈.
8.1.3. Convergence Due to ECMP Member Local Interface Failure and Recovery
Objective:
we're measuring IGP convergence for a Local Interface link failure and recovery on an ECMP Member 🧩.
Procedure:
- Advertise an IGP topology from Tester to DUT using the test setup shown in Figure 5.
- Send Offered Load from Tester to DUT on Ingress Interface.
- Verify traffic is forwarded over the ECMP member interface of the DUT that will be failed in the next step.
- Remove link on one of the ECMP member interfaces of the DUT. This is the Convergence Event 💥.
- Measure First Route Convergence Time.
- Measure Full Convergence Time.
- Stop Offered Load.
- Measure Route-Specific Convergence Times and Loss-Derived Convergence Time. At the same time, measure number of Impaired Packets 📊.
- Wait sufficient time for queues to drain 🧼.
- Restart Offered Load.
- Restore link on the ECMP member interface of the DUT.
- Measure First Route Convergence Time.
- Measure Full Convergence Time.
- Stop Offered Load.
- Measure Route-Specific Convergence Times, Loss-Derived Convergence Time, Route Loss of Connectivity Periods, and Loss-Derived Loss of Connectivity Period. At the same time, measure number of Impaired Packets 📊🎀.
8.1.4. Convergence Due to ECMP Member Remote Interface Failure and Recovery
Objective:
this one is the Remote Interface version for an ECMP Member 🦋. We want the IGP convergence numbers when the link fails and comes back.
Procedure:
- Advertise an IGP topology from Tester to DUT using the test setup shown in Figure 6.
- Send Offered Load from Tester to DUT on Ingress Interface.
- Verify traffic is forwarded over the ECMP member interface of the DUT that will be failed in the next step.
- Remove link on the interface of the Tester to R2. This is the Convergence Event Trigger 💥.
- Measure First Route Convergence Time.
- Measure Full Convergence Time.
- Stop Offered Load.
- Measure Route-Specific Convergence Times and Loss-Derived Convergence Time. At the same time, measure number of Impaired Packets 📊.
- Wait sufficient time for queues to drain 🧼.
- Restart Offered Load.
- Restore link on the interface of the Tester to R2.
- Measure First Route Convergence Time.
- Measure Full Convergence Time.
- Stop Offered Load.
- Measure Route-Specific Convergence Times, Loss-Derived Convergence Time, Route Loss of Connectivity Periods, and Loss-Derived Loss of Connectivity Period. At the same time, measure number of Impaired Packets 📊🎀.
Discussion:
same micro-loop drama as before 🌀. Between DUT1 and DUT2, a packet-forwarding loop might show up temporarily during convergence (micro-loop, see [Sh10]), and that can inflate the measured convergence times and loss of connectivity periods 😭.
8.1.5. Convergence Due to Parallel Link Interface Failure and Recovery
Objective:
we're checking IGP convergence for local link failure and recovery on a member of a parallel link 🪄. Fun fact: those links can be used for data load-balancing ⚖️.
Procedure:
- Advertise an IGP topology from Tester to DUT using the test setup shown in Figure 7.
- Send Offered Load from Tester to DUT on Ingress Interface.
- Verify traffic is forwarded over the parallel link member that will be failed in the next step.
- Remove link on one of the parallel link member interfaces of the DUT. This is the Convergence Event 💥.
- Measure First Route Convergence Time.
- Measure Full Convergence Time.
- Stop Offered Load.
- Measure Route-Specific Convergence Times and Loss-Derived Convergence Time. At the same time, measure number of Impaired Packets 📊.
- Wait sufficient time for queues to drain 🧼.
- Restart Offered Load.
- Restore link on the Parallel Link member interface of the DUT.
- Measure First Route Convergence Time.
- Measure Full Convergence Time.
- Stop Offered Load.
- Measure Route-Specific Convergence Times, Loss-Derived Convergence Time, Route Loss of Connectivity Periods, and Loss-Derived Loss of Connectivity Period. At the same time, measure number of Impaired Packets 📊🎀.
8.2. Other Failures and Recoveries
8.2.1. Convergence Due to Layer 2 Session Loss and Recovery
Objective:
we're measuring IGP convergence for a local Layer 2 loss and recovery 🌸.
Procedure:
- Advertise an IGP topology from Tester to DUT using the topology shown in Figure 1.
- Send Offered Load from Tester to DUT on Ingress Interface.
- Verify traffic is routed over Preferred Egress Interface.
- Remove Layer 2 session from Preferred Egress Interface of the DUT. This is the Convergence Event 💥.
- Measure First Route Convergence Time.
- Measure Full Convergence Time.
- Stop Offered Load.
- Measure Route-Specific Convergence Times, Loss-Derived Convergence Time, Route Loss of Connectivity Periods, and Loss-Derived Loss of Connectivity Period. At the same time, measure number of Impaired Packets 📊.
- Wait sufficient time for queues to drain 🧼.
- Restart Offered Load.
- Restore Layer 2 session on Preferred Egress Interface of the DUT.
- Measure First Route Convergence Time.
- Measure Full Convergence Time.
- Stop Offered Load.
- Measure Route-Specific Convergence Times, Loss-Derived Convergence Time, Route Loss of Connectivity Periods, and Loss-Derived Loss of Connectivity Period. At the same time, measure number of Impaired Packets 📊🎀.
Discussion:
the vibe check here is that the physical layer has to stay up 🔌 when you remove the Layer 2 session. Also, set your IGP timers so the IGP adjacency doesn't time out before the Layer 2 failure gets detected ⏱️.
To measure convergence time, traffic SHOULD start dropping on the Preferred Egress Interface the instant the Layer 2 session is removed 📉. Alternatively, the Tester SHOULD record the instant the Layer 2 session is removed, and traffic loss SHOULD only be measured on the Next-Best Egress Interface 🎯. For loss-derived benchmarks, the time of the Start Traffic Instant SHOULD be recorded as well ⏰. See Section 4.1 📖.
8.2.2. Convergence Due to Loss and Recovery of IGP Adjacency
Objective:
this one checks IGP convergence when an IGP Adjacency gets lost and comes back 🧚. The adjacency is removed on the Tester by turning off processing of IGP routing protocol packets there, so the Tester just stops listening 🙉.
Procedure:
- Advertise an IGP topology from Tester to DUT using the topology shown in Figure 1.
- Send Offered Load from Tester to DUT on Ingress Interface.
- Verify traffic is routed over Preferred Egress Interface.
- Remove IGP adjacency from the Preferred Egress Interface while the Layer 2 session MUST be maintained. This is the Convergence Event 💥.
- Measure First Route Convergence Time.
- Measure Full Convergence Time.
- Stop Offered Load.
- Measure Route-Specific Convergence Times, Loss-Derived Convergence Time, Route Loss of Connectivity Periods, and Loss-Derived Loss of Connectivity Period. At the same time, measure number of Impaired Packets 📊.
- Wait sufficient time for queues to drain 🧼.
- Restart Offered Load.
- Restore IGP session on Preferred Egress Interface of the DUT.
- Measure First Route Convergence Time.
- Measure Full Convergence Time.
- Stop Offered Load.
- Measure Route-Specific Convergence Times, Loss-Derived Convergence Time, Route Loss of Connectivity Periods, and Loss-Derived Loss of Connectivity Period. At the same time, measure number of Impaired Packets 📊🎀.
Discussion:
configure Layer 2 so it doesn't time out before the IGP adjacency failure is detected 🫧. Trust me on this one.
To measure convergence time, traffic SHOULD start dropping on the Preferred Egress Interface the instant the IGP adjacency is removed 📉. Alternatively, the Tester SHOULD record the instant the IGP adjacency is removed, and traffic loss SHOULD only be measured on the Next-Best Egress Interface 🎯. For loss-derived benchmarks, the time of the Start Traffic Instant SHOULD be recorded as well ⏰. See Section 4.1 📖💜.
8.2.3. Convergence Due to Route Withdrawal and Re-Advertisement ✨
Objective:
Okay so this test is all about getting the IGP convergence numbers when routes get withdrawn and then re-advertised 🦋
Procedure:
- Advertise an IGP topology from Tester to DUT using the topology shown in Figure 1. The routes that will be withdrawn MUST be a set of leaf routes advertised by at least two nodes in the emulated topology. The topology SHOULD be such that before the withdrawal the DUT prefers the leaf routes advertised by a node "nodeA" via the Preferred Egress Interface, and after the withdrawal the DUT prefers the leaf routes advertised by a node "nodeB" via the Next-Best Egress Interface. 💖
- Send Offered Load from Tester to DUT on Ingress Interface. 🌸
- Verify traffic is routed over Preferred Egress Interface. 👀
- The Tester withdraws the set of IGP leaf routes from nodeA. This is the Convergence Event. The withdrawal update message SHOULD be a single unfragmented packet. If the routes cannot be withdrawn by a single packet, the messages SHOULD be sent using the same pacing characteristics as the DUT. The Tester MAY record the time it sends the withdrawal message(s). 💅
- Measure First Route Convergence Time. ⏱️
- Measure Full Convergence Time. ⏱️
- Stop Offered Load. 🛑
- Measure Route-Specific Convergence Times, Loss-Derived Convergence Time, Route Loss of Connectivity Periods, and Loss-Derived Loss of Connectivity Period. At the same time, measure number of Impaired Packets. 📊
- Wait sufficient time for queues to drain. 🧺
- Restart Offered Load. 🔁
- Re-advertise the set of withdrawn IGP leaf routes from nodeA emulated by the Tester. The update message SHOULD be a single unfragmented packet. If the routes cannot be advertised by a single packet, the messages SHOULD be sent using the same pacing characteristics as the DUT. The Tester MAY record the time it sends the update message(s). 🫶
- Measure First Route Convergence Time. ⏱️
- Measure Full Convergence Time. ⏱️
- Stop Offered Load. 🛑
- Measure Route-Specific Convergence Times, Loss-Derived Convergence Time, Route Loss of Connectivity Periods, and Loss-Derived Loss of Connectivity Period. At the same time, measure number of Impaired Packets. 📊✨
Discussion:
Okay, the timing here is everything 😭 To measure convergence time, traffic SHOULD start dropping on the Preferred Egress Interface the instant the routes are withdrawn by the Tester. Alternatively, the Tester SHOULD record the time the instant the routes are withdrawn, and traffic loss SHOULD only be measured on the Next-Best Egress Interface. For loss-derived benchmarks, the time of the Start Traffic Instant SHOULD be recorded as well. See Section 4.1. 🎀
8.3. Administrative Changes 🌈
8.3.1. Convergence Due to Local Interface Administrative Changes
Objective:
The goal here is to get the IGP convergence measurements for administratively disabling and enabling a Local Interface. 🔧
Procedure:
- Advertise an IGP topology from Tester to DUT using the topology shown in Figure 1. 💖
- Send Offered Load from Tester to DUT on Ingress Interface. 🌸
- Verify traffic is routed over Preferred Egress Interface. 👀
- Administratively disable the Preferred Egress Interface of the DUT. This is the Convergence Event. 💥
- Measure First Route Convergence Time. ⏱️
- Measure Full Convergence Time. ⏱️
- Stop Offered Load. 🛑
- Measure Route-Specific Convergence Times, Loss-Derived Convergence Time, Route Loss of Connectivity Periods, and Loss-Derived Loss of Connectivity Period. At the same time, measure number of Impaired Packets. 📊
- Wait sufficient time for queues to drain. 🧺
- Restart Offered Load. 🔁
- Administratively enable the Preferred Egress Interface of the DUT. 🟢
- Measure First Route Convergence Time. ⏱️
- Measure Full Convergence Time. ⏱️
- Stop Offered Load. 🛑
- Measure Route-Specific Convergence Times, Loss-Derived Convergence Time, Route Loss of Connectivity Periods, and Loss-Derived Loss of Connectivity Period. At the same time, measure number of Impaired Packets. 📊✨
8.3.2. Convergence Due to Cost Change
Objective:
We're checking the IGP convergence measurements for a route cost change. 💜
Procedure:
- Advertise an IGP topology from Tester to DUT using the topology shown in Figure 1. 💖
- Send Offered Load from Tester to DUT on Ingress Interface. 🌸
- Verify traffic is routed over Preferred Egress Interface. 👀
- The Tester, emulating the neighbor node, increases the cost for all IGP routes at the Preferred Egress Interface of the DUT so that the Next-Best Egress Interface becomes the preferred path. The update message advertising the higher cost MUST be a single unfragmented packet. This is the Convergence Event. The Tester MAY record the time it sends the update message advertising the higher cost on the Preferred Egress Interface. 📈
- Measure First Route Convergence Time. ⏱️
- Measure Full Convergence Time. ⏱️
- Stop Offered Load. 🛑
- Measure Route-Specific Convergence Times, Loss-Derived Convergence Time, Route Loss of Connectivity Periods, and Loss-Derived Loss of Connectivity Period. At the same time, measure number of Impaired Packets. 📊
- Wait sufficient time for queues to drain. 🧺
- Restart Offered Load. 🔁
- The Tester, emulating the neighbor node, decreases the cost for all IGP routes at the Preferred Egress Interface of the DUT so that the Preferred Egress Interface becomes the preferred path. The update message advertising the lower cost MUST be a single unfragmented packet. 📉
- Measure First Route Convergence Time. ⏱️
- Measure Full Convergence Time. ⏱️
- Stop Offered Load. 🛑
- Measure Route-Specific Convergence Times, Loss-Derived Convergence Time, Route Loss of Connectivity Periods, and Loss-Derived Loss of Connectivity Period. At the same time, measure number of Impaired Packets. 📊✨
Discussion:
Same energy as before, bestie 💅 To measure convergence time, traffic SHOULD start dropping on the Preferred Egress Interface the instant the cost is changed by the Tester. Alternatively, the Tester SHOULD record the time the instant the cost is changed, and traffic loss SHOULD only be measured on the Next-Best Egress Interface. For loss-derived benchmarks, the time of the Start Traffic Instant SHOULD be recorded as well. See Section 4.1. 🦋
9. Security Considerations 🔐
Benchmarking activities as described in this memo are limited to technology characterization using controlled stimuli in a laboratory environment, with dedicated address space and the constraints specified in the sections above. 🧪
The benchmarking network topology will be an independent test setup and MUST NOT be connected to devices that may forward the test traffic into a production network or misroute traffic to the test management network. 🚫 Please, don't wire this up to prod. Seriously. 😭
Further, benchmarking is performed on a "black-box" basis, relying solely on measurements observable external to the DUT/SUT. 🖤
Special capabilities SHOULD NOT exist in the DUT/SUT specifically for benchmarking purposes. Any implications for network security arising from the DUT/SUT SHOULD be identical in the lab and in production networks. No secret benchmark-only cheat codes, periodt 💀🔥
10. Acknowledgements 💌
Thanks to Sue Hares, Al Morton, Kevin Dubray, Ron Bonica, David Ward, Peter De Vriendt, Anuj Dewagan, Julien Meuric, Adrian Farrel, Stewart Bryant, and the Benchmarking Methodology Working Group for their contributions to this work. 🫶🌈
11. References 📚
11.1. Normative References
[Br91] Bradner, S., "Benchmarking terminology for network interconnection devices", RFC 1242, July 1991.
[Br97] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
[Br99] Bradner, S. and J. McQuaid, "Benchmarking Methodology for Network Interconnect Devices", RFC 2544, March 1999.
[Ca90] Callon, R., "Use of OSI IS-IS for routing in TCP/IP and dual environments", RFC 1195, December 1990.
[Co08] Coltun, R., Ferguson, D., Moy, J., and A. Lindem, "OSPF for IPv6", RFC 5340, July 2008.
[De02] Demichelis, C. and P. Chimento, "IP Packet Delay Variation Metric for IP Performance Metrics (IPPM)", RFC 3393, November 2002.
[Ho08] Hopps, C., "Routing IPv6 with IS-IS", RFC 5308, October 2008.
[Ko02] Koodli, R. and R. Ravikanth, "One-way Loss Pattern Sample Metrics", RFC 3357, August 2002.
[Ma05] Manral, V., White, R., and A. Shaikh, "Benchmarking Basic OSPF Single Router Control Plane Convergence", RFC 4061, April 2005.
[Ma05c] Manral, V., White, R., and A. Shaikh, "Considerations When Using Basic OSPF Convergence Benchmarks", RFC 4063, April 2005.
[Ma05t] Manral, V., White, R., and A. Shaikh, "OSPF Benchmarking Terminology and Concepts", RFC 4062, April 2005.
[Ma98] Mandeville, R., "Benchmarking Terminology for LAN Switching Devices", RFC 2285, February 1998.
[Mo98] Moy, J., "OSPF Version 2", STD 54, RFC 2328, April 1998.
[Ne07] Newman, D. and T. Player, "Hash and Stuffing: Overlooked Factors in Network Device Benchmarking", RFC 4814, March 2007.
[Pa05] Pan, P., Swallow, G., and A. Atlas, "Fast Reroute Extensions to RSVP-TE for LSP Tunnels", RFC 4090, May 2005.
[Po06] Poretsky, S., Perser, J., Erramilli, S., and S. Khurana, "Terminology for Benchmarking Network-layer Traffic Control Mechanisms", RFC 4689, October 2006.
[Po11t] Poretsky, S., Imhoff, B., and K. Michielsen, "Terminology for Benchmarking Link-State IGP Data-Plane Route Convergence", RFC 6412, November 2011.
[Sh10] Shand, M. and S. Bryant, "A Framework for Loop-Free Convergence", RFC 5715, January 2010.
[Sh10i] Shand, M. and S. Bryant, "IP Fast Reroute Framework", RFC 5714, January 2010.
[Th00] Thaler, D. and C. Hopps, "Multipath Issues in Unicast and Multicast Next-Hop Selection", RFC 2991, November 2000.
11.2. Informative References
[Al00] Alaettinoglu, C., Jacobson, V., and H. Yu, "Towards Millisecond IGP Convergence", NANOG 20, October 2000.
[Al02] Alaettinoglu, C. and S. Casner, "ISIS Routing on the Qwest Backbone: a Recipe for Subsecond ISIS Convergence", NANOG 24, February 2002.
[Fi02] Filsfils, C., "Tutorial: Deploying Tight-SLA Services on an Internet Backbone: ISIS Fast Convergence and Differentiated Services Design", NANOG 25, June 2002.
[Fr05] Francois, P., Filsfils, C., Evans, J., and O. Bonaventure, "Achieving SubSecond IGP Convergence in Large IP Networks", ACM SIGCOMM Computer Communication Review v.35 n.3, July 2005.
[Ka02] Katz, D., "Why are we scared of SPF? IGP Scaling and Stability", NANOG 25, June 2002.
[Vi02] Villamizar, C., "Convergence and Restoration Techniques for ISP Interior Routing", NANOG 25, June 2002.
Authors' Addresses 🦋
Okay bestie, this is the squad that wrote the whole thing 💖 Three networking legends from three different companies, all in one doc. Here's how to find each of them ✨
Scott Poretsky 🌸
Scott Poretsky
Allot Communications
300 TradeCenter
Woburn, MA 01801
USA
Phone: + 1 508 309 2179
EMail: sporetsky@allot.comBrent Imhoff 🎀
Brent Imhoff
Juniper Networks
1194 North Mathilda Ave
Sunnyvale, CA 94089
USA
Phone: + 1 314 378 2571
EMail: bimhoff@planetspork.comKris Michielsen 🧚
Kris Michielsen
Cisco Systems
6A De Kleetlaan
Diegem, BRABANT 1831
Belgium
EMail: kmichiel@cisco.comThree authors, three companies, one spec. Iconic teamwork 👑🌈
📦 where this page comes from
- ietf: rfc6413 2011-11
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms