RFC 8333
Micro-loop Prevention by Introducing a Local Convergence Delay
RFC 8333: Micro-loop Prevention by Introducing a Local Convergence Delay ✨
Okay bestie, meet RFC 8333 🦋 It's a Standards Track document from the Internet Engineering Task Force (IETF), with ISSN 2070-1721, dated March 2018. The authors are S. Litkowski and B. Decraene from Orange, C. Filsfils from Cisco Systems, and P. Francois, who wrote it as an Individual Contributor. The header doesn't list anything it obsoletes or updates. 💅
Abstract ✨
This doc describes a mechanism for link-state routing protocols that prevents local transient forwarding loops in case of link failure. It's a two-step convergence 🎀: there's a delay between the node next to the topology change converging and the whole network converging.
Because this mechanism delays the IGP convergence, it may only be used for planned maintenance or when Fast Reroute (FRR) protects the traffic during the time between the link failure and the IGP convergence. 🫶
The mechanism is limited to the link-down event, so it stays simple. 🧚
Simulations using real network topologies have been run, and they show that local loops are a significant portion (>50%) of the total forwarding loops. That's a lot, no cap 😭
Status of This Memo
This is an Internet Standards Track document.
This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 7841. 🌈
Info about the current status of this document, any errata, and how to send feedback is at https://www.rfc-editor.org/info/rfc8333. Bookmark it! 🔖
Copyright Notice
Copyright (c) 2018 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please read these carefully, because they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they are provided without warranty as described in the Simplified BSD License. ⚖️
Table of Contents 🗂️
1. Introduction ....................................................4
2. Terminology .....................................................4
2.1. Acronyms ...................................................4
2.2. Requirements Language ......................................5
3. Side Effects of Transient Forwarding Loops ......................5
3.1. FRR Inefficiency ...........................................5
3.2. Network Congestion .........................................8
4. Overview of the Solution ........................................9
5. Specification ...................................................9
5.1. Definitions ................................................9
5.2. Regular IGP Reaction ......................................10
5.3. Local Events ..............................................10
5.4. Local Delay for Link-Down Events ..........................11
6. Applicability ..................................................11
6.1. Applicable Case: Local Loops ..............................12
6.2. Non-applicable Case: Remote Loops .........................12
7. Simulations ....................................................13
8. Deployment Considerations ......................................14
9. Examples .......................................................15
9.1. Local Link-Down Event .....................................15
9.2. Local and Remote Event ....................................19
9.3. Aborting Local Delay ......................................21
10. Comparison with Other Solutions ...............................23
10.1. PLSN .....................................................23
10.2. oFIB .....................................................24
11. IANA Considerations ...........................................24
12. Security Considerations .......................................24
13. References ....................................................25
13.1. Normative References .....................................25
13.2. Informative References ...................................25
Acknowledgements ..................................................26
Authors' Addresses ................................................261. Introduction 🌸
Micro-loops and some potential solutions are described in [RFC5715]. This document describes a simple, targeted mechanism that prevents micro-loops that are local to the failure. Based on network analysis, local micro-loops make up a significant portion of the micro-loops. 👀 A simple and easily deployable solution for these local micro-loops is critical, because these local loops cause some traffic loss after an FRR alternate has been used (see Section 3.1).
Consider the case in Figure 1, where S does not have an LFA (Loop-Free Alternate) to protect its traffic to D when the S-D link fails. That means all non-D neighbors of S on the topology will send to S any traffic destined to D. If a neighbor did not, then that neighbor would be loop-free. 🙈 Regardless of the advanced FRR technique used, when S converges to the new topology, it will send its traffic to a neighbor that is not loop-free, and that will cause a local micro-loop. The deployment of advanced FRR techniques is what motivates this simple router-local mechanism to solve this targeted problem. This solution can work with the various techniques described in [RFC5715]. 💖
D ------ C
| |
| | 5
| |
S ------ B
Figure 1In Figure 1, all links have a metric of 1 except the B-C link, which has a metric of 5. When the S-D link fails, a transient forwarding loop may appear between S and B if S updates its forwarding entry to D before B does. 🔁
2. Terminology 🧚
2.1. Acronyms
- FIB: Forwarding Information Base
- FRR: Fast Reroute
- IGP: Interior Gateway Protocol
- LFA: Loop-Free Alternate
- LSA: Link State Advertisement
- LSP: Link State Packet
- MRT: Maximally Redundant Tree
- oFIB: Ordered FIB
- PLR: Point of Local Repair
- PLSN: Path Locking via Safe Neighbors
- RIB: Routing Information Base
- RLFA: Remote Loop-Free Alternate
- SPF: Shortest Path First
- TTL: Time to Live
2.2. Requirements Language
Now for the big capital words 👑 The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are BCP 14 [RFC2119] [RFC8174] words. They're only to be read that way when, and only when, they appear in all capitals, exactly like that. Lowercase is just normal English, bestie! 🫶
3. Side Effects of Transient Forwarding Loops 💀
Even if they're very limited in duration, transient forwarding loops may cause significant network damage. Short doesn't mean harmless, so let's dig into why. 🔍
Part 1 of 7 done, bestie! 🌈 More parts coming, so stay tuned. 💜
3.1. FRR Inefficiency
okay bestie, Figure 2 is an IP/LDP routed network and it's giving main character energy 🦋
D
1 |
| 1
A ------ B
| | ^
10 | | 5 | T
| | |
E--------C
| 1
1 |
S
Figure 2So there's an RSVP-TE tunnel T, set up on C and ending at B, that protects traffic if the C-B link dies. The IGP shortcut feature from [RFC3906] is turned on at C. The tunnel's primary path is C->B, FRR is switched on for T, and the FRR bypass (or detour) goes C->E->A->B. On C, the next hop to D is the tunnel T, thanks to that IGP shortcut. Then the C-B link fails and things get messy:
- C spots the failure and moves the tunnel path onto a preprogrammed FRR path. The traffic path from S to D becomes S->E->C->E->A->B->A->D.
- At the same time, on router C, the IGP convergence and the TE tunnel convergence (tunnel path recomputation) are both happening:
- The tunnel T path gets recomputed and now uses C->E->A->B.
- The IGP path to D gets recomputed and now uses C->E->A->D.
- On C, the tail-end of the TE tunnel (router B) is no longer on the shortest-path tree (SPT) to D. So C stops encapsulating traffic to D into tunnel T and updates its forwarding entry for D to use next hop E.
If C updates its forwarding entry to D before router E does, there's a transient forwarding loop between C and E until E has converged. 😭
Table 1 walks through a theoretical timeline of what happens when the B-C link fails. Treat it as an example only, not gospel 💅
+------------+--------+---------------------+-----------------------+
| Network | Time | Router C Events | Router E Events |
| Condition | | | |
+------------+--------+---------------------+-----------------------+
| S->D | | | |
| Traffic OK | | | |
| | | | |
| S->D | t0 | Link B-C fails | Link B-C fails |
| Traffic | | | |
| lost | | | |
| | | | |
| | t0+20 | C detects the | |
| | ms | failure | |
| | | | |
| S->D | t0+40 | C activates FRR | |
| Traffic OK | ms | | |
| | | | |
| | t0+50 | C updates its local | |
| | ms | LSP/LSA | |
| | | | |
| | t0+60 | C floods its local | |
| | ms | updated LSP/LSA | |
| | | | |
| | t0+62 | C schedules SPF | |
| | ms | (100 ms) | |
| | | | |
| | t0+87 | | E receives LSP/LSA |
| | ms | | from C and floods it |
| | | | |
| | t0+92 | | E schedules SPF (100 |
| | ms | | ms) |
| | | | |
| | t0+163 | C computes SPF | |
| | ms | | |
| | | | |
| | t0+165 | C starts updating | |
| | ms | its RIB/FIB | |
| | | | |
| | t0+193 | | E computes SPF |
| | ms | | |
| | | | |
| | t0+199 | | E starts updating its |
| | ms | | RIB/FIB |
| | | | |
| S->D | t0+255 | C updates its | |
| Traffic | ms | RIB/FIB for D | |
| lost | | | |
| | | | |
| | t0+340 | C convergence ends | |
| | ms | | |
| | | | |
| S->D | t0+443 | | E updates its RIB/FIB |
| Traffic OK | ms | | for D |
| | | | |
| | t0+470 | | E convergence ends |
| | ms | | |
+------------+--------+---------------------+-----------------------+
Table 1The issue here has nothing to do with which FRR mechanism you use (TE FRR, LFA/RLFA, MRT, etc.) as long as the primary path uses hop-by-hop routing. FRR protection works perfectly, but only until the PLR has converged. The moment the PLR converges, it swaps its FRR path for a new primary path ✨ When implementing FRR, a service provider wants to guarantee a very limited loss of connectivity time. This example shows that FRR's benefit can be totally wiped out by a transient forwarding loop that shows up once the PLR has converged. Delaying the FIB updates after IGP convergence (1) may let the FRR path stay in place until the neighbors have converged, and (2) it protects customer traffic 🫶
3.2. Network Congestion
In Figure 3, when the S-D link fails, a transient forwarding loop may pop up between S and B for destination D. The traffic on the S-B link keeps climbing because of the looping traffic to D. Depending on the TTL of the packets, the traffic rate toward D, and the link's bandwidth, the S-B link can get congested within a few hundred milliseconds and stay that way until the loop is gone 😩
1
D ------ C
| |
1 | | 5
| |
A -- S ------ B
/ | 1
F E
Figure 3Congestion from transient forwarding loops is a real problem because it can hit traffic that the failing component never touched. In Figure 3, the S-B link congestion would hurt customer traffic that isn't directly affected by the failure, like traffic from A to B, F to B, and E to B. Class of service can help some traffic. But some traffic that the failure didn't touch will still get dropped, because a router can't tell the looping traffic apart from normally forwarded traffic 💀
4. Overview of the Solution
This document defines a two-step convergence. It's kicked off by the router that detects a failure and advertises the topology change in the IGP. That creates a delay between network-wide convergence and convergence on the local router ⭐
The solution described here is limited to local link-down events, to keep things simple 🍓
This ordered convergence is similar to the ordered FIB (oFIB) approach defined in [RFC6976], but it's limited to a "one-hop" distance. That makes it simpler, and it becomes a local-only feature that doesn't need interoperability. The tradeoff is that it only eliminates transient forwarding loops involving the local router. The mechanism also reuses some concepts described in [PLSN].
5. Specification
5.1. Definitions
This document refers to the following existing IGP timers. They may be standardized, or they may be implemented as a vendor-specific local feature 🧚
LSP_GEN_TIMER: The delay between the consecutive generation of two local LSPs/LSAs. Operationally, this delay is usually tuned to batch multiple local events into a single local LSP/LSA update. In IS-IS, this timer is defined asminimumLSPGenerationInterval[ISO10589]. In OSPF version 2, this timer is defined asMinLSInterval[RFC2328]. It is often paired with a vendor-specific damping mechanism that slows reactions down by incrementing the timer when multiple consecutive events are detected.SPF_DELAY: The delay between the first IGP event that triggers a new routing table computation and the start of that computation. It is often paired with a damping mechanism that slows reactions down by incrementing the timer when the IGP becomes unstable. For example, [BACKOFF] defines a standard SPF delay algorithm.
This document introduces the following new timer:
ULOOP_DELAY_DOWN_TIMER: Used to slow down local node convergence in case of link-down events. 🌈
5.2. Regular IGP Reaction
When the status of an adjacency or link changes, the router advertising the event follows its regular IGP convergence behavior, which involves these main steps:
- IGP is notified of the up/down event.
- The IGP processes the notification and postpones the reaction for
LSP_GEN_TIMERms. - When the
LSP_GEN_TIMERexpires, the IGP updates its LSP/LSA and floods it. - The SPF computation is scheduled in
SPF_DELAYms. - When the
SPF_DELAYtimer expires, the SPF is computed, and then the RIB and FIB are updated. 💖
5.3. Local Events 🎀
okay so this mechanism is built on one big assumption: there's been a single link failure as seen by the IGP area/level 🦋. if that assumption gets broken (like multiple links or nodes failing at once 💀), then regular IP convergence has to take over instead, as described in Section 5.2.
to figure out whether the mechanism even applies, an implementation SHOULD correlate the protocol messages (LSP/LSA) it receives during the SPF scheduling period so it can work out which topology changes actually happened 🧠. this matters because one message can describe the same change as another, and one message can describe several changes at once. so figuring out a particular topology change MUST be independent of the order the messages arrive in ✨. how the logic works is up to the implementation.
using that logic, if the implementation decides the topology change is a single local link failure, then the router MAY use the mechanism described in this document. otherwise, regular IP convergence MUST be used 🙅♀️.
Let's look at Figure 4 🌸. Say router B is the computing router when link B-C fails. B updates its local LSP/LSA to describe B-C as down, C does the same, and they both start flooding their updated LSPs/LSAs. during the SPF_DELAY period, B and C learn all the LSPs/LSAs they need to consider. B sees C flooding an advertisement saying a link is down, and B is the other end of that link, so B realizes they're describing the same single event. since B gets no other changes, B can tell this is a local link failure and may decide to activate the mechanism described in this document. main character behavior, honestly 💜
+--- E ----+--------+
| | |
A ---- B -------- C ------ D
Figure 45.4. Local Delay for Link-Down Events 💖
okay, this is where it gets spicy 🔥. this document changes step 5 (see the list in Section 5.2) so that, on an adjacency or link-down event, local convergence gets delayed compared to network-wide convergence. the new step 5 goes like this:
- Upon SPF_DELAY timer expiration, the SPF is computed. If the condition of a single local link-down event has been met, then an update of the RIB and the FIB MUST be delayed for ULOOP_DELAY_DOWN_TIMER ms. Otherwise, the RIB and FIB SHOULD be updated immediately.
if a new convergence happens while ULOOP_DELAY_DOWN_TIMER is running, ULOOP_DELAY_DOWN_TIMER gets stopped, and the RIB/FIB SHOULD be updated as part of that new convergence event ⏰.
ngl, there's a tradeoff here 🥺. as a result of this addition, routers local to the failure will converge slower than remote routers. so it SHOULD only be done for a non-urgent convergence, like administrative deactivation (maintenance) or when the traffic is protected by FRR 🛡️.
6. Applicability 🌈
as previously stated, this mechanism only avoids the forwarding loops on the links between the node local to the failure and its neighbors. forwarding loops may still occur on other links 👀.
6.1. Applicable Case: Local Loops 🦋
In Figure 5, let's look at the traffic from G to F. the primary path is G->D->C->E->F. when the link C-E fails, if C updates its forwarding entry for F before D, a transient loop occurs. this is sub-optimal, because it breaks C's FRR forwarding even though upstream routers are still forwarding the traffic to C 😭.
A ------ B ----- E
| / |
| / |
G---D------------C F
All the links have a metric of 1
Figure 5by implementing the mechanism defined in this document on C, when the C-E link fails, C delays updating its forwarding entry to F. this gives D some time to converge ⏳. FRR on C keeps protecting the traffic during this period 🫶. when ULOOP_DELAY_DOWN_TIMER expires on C, its forwarding entry to F gets updated. there is no transient forwarding loop on the link C-D ✨.
6.2. Non-applicable Case: Remote Loops 💀
In Figure 6, let's look at the traffic from G to K. the primary path is G->D->C->F->J->K. when the C-F link fails, if C updates its forwarding entry to K before D, a transient loop occurs between C and D 🙈.
A ------ B ----- E --- H
| |
| |
G---D--------C ------F --- J ---- K
All the links have a metric of 1 except B-E=15
Figure 6by implementing the mechanism defined in this document on C, when the link C-F fails, C delays updating its forwarding entry to K, which gives D time to converge. when ULOOP_DELAY_DOWN_TIMER expires on C, its forwarding entry to F is updated. there is no transient forwarding loop between C and D 🎉. however, a transient forwarding loop may still occur between D and A. in this scenario, this mechanism isn't enough to address all the possible forwarding loops. still, it doesn't create additional traffic loss. besides, in some cases -- like when the nodes update their FIB in the order C, A, D because router A converges faster than D -- the mechanism may still avoid the forwarding loop that would have otherwise occurred 🧚.
7. Simulations 📊
simulations have been run on multiple service-provider topologies. we evaluated the efficiency of the mechanism on eight different service-provider topologies (different network sizes and designs). Table 2 shows the gain for each topology 💅.
+----------+------+
| Topology | Gain |
+----------+------+
| T1 | 71% |
| T2 | 81% |
| T3 | 62% |
| T4 | 50% |
| T5 | 70% |
| T6 | 70% |
| T7 | 59% |
| T8 | 77% |
+----------+------+
Table 2we evaluated the gain like this:
- We considered a tuple (link A-B, destination D, PLR S, backup next-hop N) as a loop if, upon link A-B failure, the flow from a router S upstream from A (A could be considered as PLR also) to D may loop due to convergence time difference between S and one of its neighbors N.
- We evaluated the number of potential loop tuples in normal conditions.
- We evaluated the number of potential loop tuples using the same topological input but taking into account that S converges after N.
- The gain is the relative number of loops (both remote and local) we succeed in suppressing.
for topology 1, implementing the local delay prevented 71% of the transient forwarding loops created by the failure of any link. the analysis shows that all local loops are prevented and only remote loops remain 🏆.
8. Deployment Considerations 🍓
transient forwarding loops have the following drawbacks:
- They limit FRR efficiency. Even if FRR is activated within 50 ms, as soon as the PLR has converged, the traffic may be affected by a transient loop 😤.
- They may impact traffic not directly affected by the failure (due to link congestion).
the local delay mechanism is a transient forwarding loop avoidance mechanism (like oFIB). even if it only addresses local transient loops, the efficiency versus complexity comparison makes it a good solution 🫶. it's also incrementally deployable with incremental benefits, which makes it an attractive option for both vendors to implement and service providers to deploy 💖. delaying convergence isn't an issue if we consider that the traffic is protected during the convergence 🛡️.
the ULOOP_DELAY_DOWN_TIMER value should be set according to the maximum IGP convergence time observed in the network (usually observed in the slowest node) ⏱️.
this mechanism is limited to link-down events. when a link goes down, it eventually goes back up. as a consequence, with this mechanism deployed, only the link-down event will be protected against transient forwarding loops, while the link-up event won't be. if the operator wants to limit the impact of transient forwarding loops during link-up, they should make sure to use specific procedures to bring the link back online 🌈. as examples, the operator can decide to put the link back online outside of business hours, or it can use some incremental metric changes to prevent loops (as proposed in [RFC5715]) 🧠.
9. Examples 🧚
we'll use the following figure for the examples in this section:
D
1 | F----X
| 1 |
A ------ B
| |
10 | | 5
| |
E--------C
| 1
1 |
S
Figure 7the network above is considered to have a convergence time of about 1 second, so ULOOP_DELAY_DOWN_TIMER will be adjusted to this value ⚡. we also consider that FRR is running on each node 👑.
9.1. Local Link-Down Event 🦋
okay so Table 3 is the play-by-play of what happens on routers C and E when link B-C goes down 💀 it's a theoretical sequence, so read it as an example, not a rule book 🫶 the key bit: C detects one single local event for the link-down (its LSP + LSP from B received) ✨, so it applies the local delay down behavior and no micro-loop forms 💅
+------------+---------+---------------------+----------------------+
| Network | Time | Router C Events | Router E Events |
| Condition | | | |
+------------+---------+---------------------+----------------------+
| S->D | | | |
| Traffic OK | | | |
| | | | |
| S->D | t0 | Link B-C fails | Link B-C fails |
| Traffic | | | |
| lost | | | |
| | | | |
| | t0+20 | C detects the | |
| | ms | failure | |
| | | | |
| S->D | t0+40 | C activates FRR | |
| Traffic OK | ms | | |
| | | | |
| | t0+50 | C updates its local | |
| | ms | LSP/LSA | |
| | | | |
| | t0+53 | C floods its local | |
| | ms | updated LSP/LSA | |
| | | | |
| | t0+60 | C schedules SPF | |
| | ms | (100 ms) | |
| | | | |
| | t0+67 | C receives LSP/LSA | |
| | ms | from B and floods | |
| | | it | |
| | | | |
| | t0+87 | | E receives LSP/LSA |
| | ms | | from C and floods it |
| | | | |
| | t0+90 | | E schedules SPF (100 |
| | ms | | ms) |
| | | | |
| | t0+161 | C computes SPF | |
| | ms | | |
| | | | |
| | t0+165 | C delays its | |
| | ms | RIB/FIB update (1 | |
| | | sec) | |
| | | | |
| | t0+193 | | E computes SPF |
| | ms | | |
| | | | |
| | t0+199 | | E starts updating |
| | ms | | its RIB/FIB |
| | | | |
| | t0+443 | | E updates its |
| | ms | | RIB/FIB for D |
| | | | |
| | t0+470 | | E convergence ends |
| | ms | | |
| | | | |
| | t0+1165 | C starts updating | |
| | ms | its RIB/FIB | |
| | | | |
| | t0+1255 | C updates its | |
| | ms | RIB/FIB for D | |
| | | | |
| | t0+1340 | C convergence ends | |
| | ms | | |
+------------+---------+---------------------+----------------------+Table 3 🎀
similarly, for a B-C link-down, if the LSP/LSA from B shows up before C notices the link failure, C will still apply the route update delay as long as the local detection is part of the same SPF run 🧚 Table 4 is the matching theoretical timeline, and yep, it's also just an example 🌸
+------------+---------+---------------------+----------------------+
| Network | Time | Router C Events | Router E Events |
| Condition | | | |
+------------+---------+---------------------+----------------------+
| S->D | | | |
| Traffic OK | | | |
| | | | |
| S->D | t0 | Link B-C fails | Link B-C fails |
| Traffic | | | |
| lost | | | |
| | | | |
| | t0+32 | C receives LSP/LSA | |
| | ms | from B and floods | |
| | | it | |
| | | | |
| | t0+33 | C schedules SPF | |
| | ms | (100 ms) | |
| | | | |
| | t0+50 | C detects the | |
| | ms | failure | |
| | | | |
| S->D | t0+55 | C activates FRR | |
| Traffic OK | ms | | |
| | | | |
| | t0+55 | C updates its local | |
| | ms | LSP/LSA | |
| | | | |
| | t0+70 | C floods its local | |
| | ms | updated LSP/LSA | |
| | | | |
| | t0+87 | | E receives LSP/LSA |
| | ms | | from C and floods it |
| | | | |
| | t0+90 | | E schedules SPF (100 |
| | ms | | ms) |
| | | | |
| | t0+135 | C computes SPF | |
| | ms | | |
| | | | |
| | t0+140 | C delays its | |
| | ms | RIB/FIB update (1 | |
| | | sec) | |
| | | | |
| | t0+193 | | E computes SPF |
| | ms | | |
| | | | |
| | t0+199 | | E starts updating |
| | ms | | its RIB/FIB |
| | | | |
| | t0+443 | | E updates its |
| | ms | | RIB/FIB for D |
| | | | |
| | t0+470 | | E convergence ends |
| | ms | | |
| | | | |
| | t0+1145 | C starts updating | |
| | ms | its RIB/FIB | |
| | | | |
| | t0+1255 | C updates its | |
| | ms | RIB/FIB for D | |
| | | | |
| | t0+1340 | C convergence ends | |
| | ms | | |
+------------+---------+---------------------+----------------------+Table 4 💜
9.2. Local and Remote Event
okay bestie, this one is a whole drama 🎀 Table 5 shows what router C and router E do, and when, if the link B-C goes down and the link F-X fails in the same time window 😭 Because a non-local topology change shows up too, C does NOT apply the local delay here 💅 Quick heads up: Table 5 is a theoretical sequence of events, so treat it as an example and nothing more 🧚
+-----------+--------+-------------------+--------------------------+
| Network | Time | Router C Events | Router E Events |
| Condition | | | |
+-----------+--------+-------------------+--------------------------+
| S->D | | | |
| Traffic | | | |
| OK | | | |
| | | | |
| S->D | t0 | Link B-C fails | Link B-C fails |
| Traffic | | | |
| lost | | | |
| | | | |
| | t0+20 | C detects the | |
| | ms | failure | |
| | | | |
| | t0+36 | Link F-X fails | Link F-X fails |
| | ms | | |
| | | | |
| S->D | t0+40 | C activates FRR | |
| Traffic | ms | | |
| OK | | | |
| | | | |
| | t0+50 | C updates its | |
| | ms | local LSP/LSA | |
| | | | |
| | t0+54 | C receives | |
| | ms | LSP/LSA from F | |
| | | and floods it | |
| | | | |
| | t0+60 | C schedules SPF | |
| | ms | (100 ms) | |
| | | | |
| | t0+67 | C receives | |
| | ms | LSP/LSA from B | |
| | | and floods it | |
| | | | |
| | t0+69 | | E receives LSP/LSA from |
| | ms | | F, floods it and |
| | | | schedules SPF (100 ms) |
| | | | |
| | t0+70 | C floods its | |
| | ms | local updated | |
| | | LSP/LSA | |
| | | | |
| | t0+87 | | E receives LSP/LSA from |
| | ms | | C |
| | | | |
| | t0+117 | | E floods LSP/LSA from C |
| | ms | | |
| | | | |
| | t0+160 | C computes SPF | |
| | ms | | |
| | | | |
| | t0+165 | C starts updating | |
| | ms | its RIB/FIB (NO | |
| | | DELAY) | |
| | | | |
| | t0+170 | | E computes SPF |
| | ms | | |
| | | | |
| | t0+173 | | E starts updating its |
| | ms | | RIB/FIB |
| | | | |
| S->D | t0+365 | C updates its | |
| Traffic | ms | RIB/FIB for D | |
| lost | | | |
| | | | |
| S->D | t0+443 | | E updates its RIB/FIB |
| Traffic | ms | | for D |
| OK | | | |
| | | | |
| | t0+450 | C convergence | |
| | ms | ends | |
| | | | |
| | t0+470 | | E convergence ends |
| | ms | | |
| | | | |
+-----------+--------+-------------------+--------------------------+Table 5
Reading the table, the vibe goes like this 🦋 Both routers see B-C die at t0, but only C notices it at t0+20 ms 🙈 Traffic from S to D is lost for a bit, then C activates FRR at t0+40 ms and traffic is OK again ✨ Link F-X fails at t0+36 ms on both routers, so the topology change piles up, and that's exactly why C skips the local delay 💀 C schedules its SPF for 100 ms and starts updating its RIB/FIB at t0+165 ms with NO DELAY, while E does its own SPF and RIB/FIB update a bit later 🌸 Traffic to D gets lost again around t0+365 ms until C updates its RIB/FIB for D, and then it's OK once E updates its RIB/FIB for D at t0+443 ms 😭 C's convergence ends at t0+450 ms, and E's ends at t0+470 ms 👑
9.3. Aborting Local Delay ✨
okay bestie, this is the plot twist 😭 Table 6 shows what happens on routers C and E when link B-C goes down 🦋. But then we also check what happens when the F-X link fails while C is still sitting in its local delay for the FIB update 💀. Here's the tea: C applies the local delay first, but once the new event hits, it falls back to the standard convergence mechanism and stops delaying route insertion 💅. For this example we set ULOOP_DELAY_DOWN_TIMER to 2 seconds ⏱️. And just so we're clear, Table 6 is a theoretical sequence of events, so it should only been read as an example 🙈.
+------------+--------+----------------------+----------------------+
| Network | Time | Router C Events | Router E Events |
| Condition | | | |
+------------+--------+----------------------+----------------------+
| S->D | | | |
| Traffic OK | | | |
| | | | |
| S->D | t0 | Link B-C fails | Link B-C fails |
| Traffic | | | |
| lost | | | |
| | | | |
| | t0+20 | C detects the | |
| | ms | failure | |
| | | | |
| S->D | t0+40 | C activates FRR | |
| Traffic OK | ms | | |
| | | | |
| | t0+50 | C updates its local | |
| | ms | LSP/LSA | |
| | | | |
| | t0+55 | C floods its local | |
| | ms | updated LSP/LSA | |
| | | | |
| | t0+57 | C schedules SPF (100 | |
| | ms | ms) | |
| | | | |
| | t0+67 | C receives LSP/LSA | |
| | ms | from B and floods it | |
| | | | |
| | t0+87 | | E receives LSP/LSA |
| | ms | | from C and floods it |
| | | | |
| | t0+90 | | E schedules SPF (100 |
| | ms | | ms) |
| | | | |
| | t0+160 | C computes SPF | |
| | ms | | |
| | | | |
| | t0+165 | C delays its RIB/FIB | |
| | ms | update (2 sec) | |
| | | | |
| | t0+193 | | E computes SPF |
| | ms | | |
| | | | |
| | t0+199 | | E starts updating |
| | ms | | its RIB/FIB |
| | | | |
| | t0+254 | Link F-X fails | Link F-X fails |
| | ms | | |
| | | | |
| | t0+300 | C receives LSP/LSA | |
| | ms | from F and floods it | |
| | | | |
| | t0+303 | C schedules SPF (200 | |
| | ms | ms) | |
| | | | |
| | t0+312 | E receives LSP/LSA | |
| | ms | from F and floods it | |
| | | | |
| | t0+313 | E schedules SPF (200 | |
| | ms | ms) | |
| | | | |
| | t0+502 | C computes SPF | |
| | ms | | |
| | | | |
| | t0+505 | C starts updating | |
| | ms | its RIB/FIB (NO | |
| | | DELAY) | |
| | | | |
| | t0+514 | | E computes SPF |
| | ms | | |
| | | | |
| | t0+519 | | E starts updating |
| | ms | | its RIB/FIB |
| | | | |
| S->D | t0+659 | C updates its | |
| Traffic | ms | RIB/FIB for D | |
| lost | | | |
| | | | |
| S->D | t0+778 | | E updates its |
| Traffic OK | ms | | RIB/FIB for D |
| | | | |
| | t0+781 | C convergence ends | |
| | ms | | |
| | | | |
| | t0+810 | | E convergence ends |
| | ms | | |
+------------+--------+----------------------+----------------------+Table 6
10. Comparison with Other Solutions 🎀
Remember Section 4? The local delay solution borrows a few ideas that other IETF proposals already came up with, but it's going for a sweet spot between efficiency and simplicity 🧁. This section compares how the solutions behave, so let's dig in 🔍.
10.1. PLSN 🌸
PLSN [PLSN] is giving "stay on a path that's safe" energy. Every node tries to dodge transient forwarding loops after a topology change by keeping traffic on a loop-free path for a set amount of time (a locked path to a safe neighbor) 🔒. That locked path could be the new primary next hop, another neighbor, or the old primary next hop, depending on how the safety condition shakes out ✨.
PLSN doesn't catch every transient forwarding loop, though (see Section 4 of [PLSN] for the details) 🙈.
The solution in this document borrows some of PLSN's vibes but keeps things much simpler 💖:
- PLSN has three behaviors: (1) keep using the old next hop, (2) use the new primary next hop if it's safe, or (3) use another safe next hop. The local delay solution only has one: keep using the current next hop (that means the old primary next hop or an already-activated FRR path) 💅.
- PLSN may cause some damage when it uses a safe next hop that isn't the new primary next hop, if that new safe next hop doesn't have enough bandwidth (see [RFC7916]). The solution in this document may not run into this issue, because the service provider may have control over the FRR path being used, which prevents network congestion 🛡️.
- PLSN applies to all nodes in a network (remote or local changes), while the mechanism in this document only applies to the nodes connected to the topology change 🎯.
10.2. oFIB 🧚
oFIB [RFC6976] is all about order 📋. Its whole thing is making the network converge in a specific order after a topology change, so transient forwarding loops don't pop up. Each router figures out the failure type from the LSA/LSP it receives, then computes and applies a specific FIB update timer based on that failure type and its rank in the network, with the failure point treated as the root 🌳.
oFIB solves all the transient forwarding loops in a network, but the price is extra complexity in the convergence process, which may need careful monitoring from the service provider 👀.
This solution takes the oFIB concept and limits it to the first hop that experiences the topology change 💜. As we showed, the mechanism in this document solves all the local transient forwarding loops, and those make up a high percentage of all loops 📈. Limiting it to one hop also keeps the network-wide convergence behavior intact 🫶.
11. IANA Considerations 🍓
This document has no IANA actions. Nothing for IANA to do here, bestie 💅.
12. Security Considerations 🔐
This document does not change anything about IGP security 🔐. The operation happens inside the router, so it's internal business. The local delay doesn't add any new attack vectors, because an attacker could only trigger this mechanism if it already had the power to disable or enable an IGP link ⚠️. The local delay also doesn't make the negative consequences any worse. If an attacker can disable or enable an IGP link, it can already harm the network by causing instability, and it can already harm traffic by causing forwarding packet loss and forwarding loss for the traffic crossing that link 🚨. Take this one seriously, friends 😭.
13. References 📚
13.1. Normative References
- [ISO10589] International Organization for Standardization, "Information technology -- Telecommunications and information exchange between systems -- Intermediate System to Intermediate System intra-domain routeing information exchange protocol for use in conjunction with the protocol for providing the connectionless-mode network service (ISO 8473)", ISO/IEC 10589:2002, Second Edition, November 2002.
- [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997,
https://www.rfc-editor.org/info/rfc2119. - [RFC2328] Moy, J., "OSPF Version 2", STD 54, RFC 2328, DOI 10.17487/RFC2328, April 1998,
https://www.rfc-editor.org/info/rfc2328. - [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017,
https://www.rfc-editor.org/info/rfc8174.
13.2. Informative References ✨
- [BACKOFF] Decraene, B., Litkowski, S., Gredler, H., Lindem, A., Francois, P., and C. Bowers, "SPF Back-off Delay algorithm for link state IGPs", Work in Progress, draft-ietf-rtgwg-backoff-algo-10, March 2018. 🎀
- [PLSN] Zinin, A., "Analysis and Minimization of Microloops in Link-state Routing Protocols", Work in Progress, draft-ietf-rtgwg-microloop-analysis-01, October 2005. 🦋
- [RFC3906] Shen, N. and H. Smit, "Calculating Interior Gateway Protocol (IGP) Routes Over Traffic Engineering Tunnels", RFC 3906, DOI 10.17487/RFC3906, October 2004,
<https://www.rfc-editor.org/info/rfc3906>. 🌸 - [RFC5715] Shand, M. and S. Bryant, "A Framework for Loop-Free Convergence", RFC 5715, DOI 10.17487/RFC5715, January 2010,
<https://www.rfc-editor.org/info/rfc5715>. 💜 - [RFC6976] Shand, M., Bryant, S., Previdi, S., Filsfils, C., Francois, P., and O. Bonaventure, "Framework for Loop-Free Convergence Using the Ordered Forwarding Information Base (oFIB) Approach", RFC 6976, DOI 10.17487/RFC6976, July 2013,
<https://www.rfc-editor.org/info/rfc6976>. 🧚 - [RFC7916] Litkowski, S., Ed., Decraene, B., Filsfils, C., Raza, K., Horneffer, M., and P. Sarkar, "Operational Management of Loop-Free Alternates", RFC 7916, DOI 10.17487/RFC7916, July 2016,
<https://www.rfc-editor.org/info/rfc7916>. 💅
Acknowledgements
Okay, big thank-you moment! 🫶 We owe a huge shoutout to the authors of [RFC6976] for coming up with the idea of ordered convergence: Mike Shand, Stewart Bryant, Stefano Previdi, and Olivier Bonaventure. Iconic behaviour, honestly. 👑
Authors' Addresses
- Stephane Litkowski 💖
- Orange
- Email: stephane.litkowski@orange.com
- Bruno Decraene 🌈
- Orange
- Email: bruno.decraene@orange.com
- Clarence Filsfils ✨
- Cisco Systems
- Email: cfilsfil@cisco.com
- Pierre Francois 🦋
- Individual Contributor
- Email: pfrpfr@gmail.com
📦 where this page comes from
- ietf: rfc8333 2018-03
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms