RFC 2892

The Cisco SRP MAC Layer Protocol

๐Ÿ“… published August 2000, from ietf

Part 1 of 10 โœจ

The Cisco SRP MAC Layer Protocol ๐ŸŽ€

okay so this is RFC 2892, and it's giving a whole protocol spec with main character energy ๐Ÿ’… It's titled "The Cisco SRP MAC Layer Protocol" and it's from August 2000. Two authors wrote it, D. Tsiang and G. Suwala, both from Cisco Systems. It's in the Informational category, so it shares information with the internet community without setting an internet standard. It doesn't obsolete or update any other RFC ๐Ÿฆ‹

Status of This Memo ๐ŸŒธ

okay so this memo just provides information for the internet community. It does not specify an internet standard of any kind, and distribution is unlimited ๐Ÿ’–

Copyright (C) The Internet Society (2000). All Rights Reserved.

Abstract โœจ

okay bestie, so this document specifies the MAC layer protocol, the "Spatial Reuse Protocol" (SRP), for use with ring based media. This is the second version of the protocol, so it's V2 ๐Ÿ‘‘

The main goals for SRP are:

  • Efficient use of bandwidth, using:
    • spatial reuse of bandwidth
    • local reuse of bandwidth
    • minimal protocol overhead
  • Support for priority traffic ๐Ÿ”ฅ
  • Scalability across a large number of nodes or stations attached to a ring ๐ŸŒˆ
  • "Plug and play" design, with no software based station management transfer (SMT) protocol and no ring master negotiation, unlike other ring based MAC protocols [1][2] ๐Ÿ™ˆ
  • Fairness among nodes using the ring โญ
  • Support for ring based redundancy (error detection, ring wrap, etc.), similar to what's found in SONET BLSR specifications ๐Ÿงš
  • Independence of physical layer (layer 1) media type ๐Ÿ“

This document defines the terminology used with SRP, the packet formats, the protocol format, protocol operation, and the protocol finite state machines that go with it. Literally everything you need, all in one place ๐Ÿ’…

Table of Contents ๐Ÿฆ‹

 1.  Differences between SRP V1 and V2 .......................  3
     2.  Terms and Taxonomy ......................................  4
         2.1.  Ring Terminology ..................................  4
         2.2.  Spatial Reuse .....................................  5
         2.3.  Fairness ..........................................  6
         2.4.  Transit Buffer ....................................  7
     3.  SRP Overview ............................................  8
         3.1.  Receive Operation Overview ........................  8
         3.2.  Transmit Operation Overview .......................  8
         3.3.  SRP Fairness Algorithm (SRP-fa) Overview ..........  9
         3.4.  Intelligent Protection Switching (IPS) Protocol
               Overview ..........................................  9
     4.  Packet Formats .......................................... 13
         4.1.  Overall Packet Format ............................. 13
         4.2.  Generic Packet Header Format ...................... 14
              4.2.1.  Time To Live (TTL) ......................... 14
              4.2.2.  Ring Identifier (R) ........................ 15
              4.2.3.  Priority Field (PRI) ....................... 15
              4.2.4.  MODE ....................................... 15
              4.2.5.  Parity Bit (P-bit) ......................... 16
              4.2.6.  Destination Address ........................ 16
              4.2.7.  Source Address ............................. 16
              4.2.8.  Protocol Type .............................. 16
         4.3.  SRP Cell Format ................................... 16
         4.4.  SRP Usage Packet Format ........................... 17
         4.5.  SRP Control Packet Format ......................... 18
              4.5.1.  Control Ver ................................ 19
              4.5.2.  Control Type ............................... 19
              4.5.3.  Control TTL ................................ 19
              4.5.4.  Control Checksum ........................... 19
              4.5.5.  Payload .................................... 20
              4.5.6.  Addressing ................................. 20
         4.6.  Topology Discovery ................................ 20
              4.6.1.  Topology Length ............................ 22
              4.6.2.  Topology Originator ........................ 22
              4.6.3.  MAC bindings ............................... 22
              4.6.4.  MAC Type Format ............................ 22
         4.7.  Intelligent Protection Switching (IPS) ............ 23
              4.7.1.  Originator MAC Address ..................... 23
              4.7.2.  IPS Octet .................................. 24
         4.8.  Circulating packet detection (stripping) .......... 24
     5.  Packet acceptance and stripping ......................... 25
         5.1.  Transmission and forwarding with priority ......... 27
         5.2.  Wrapping of Data .................................. 28
     6.  SRP-fa Rules Of Operation ............................... 28
         6.1.  SRP-fa pseudo-code ................................ 30
         6.2.  Threshold settings ................................ 32
     7.  SRP Synchronization ..................................... 32
         7.1.  SRP Synchronization Examples ...................... 33
     8.  IPS Protocol Description ................................ 34
         8.1.  The IPS Request Types ............................. 35
         8.2.  SRP IPS Protocol States ........................... 36
              8.2.1.  Idle ....................................... 36
              8.2.2.  Pass-through ............................... 36
              8.2.3.  Wrapped .................................... 36
         8.3.  IPS Protocol Rules ................................ 36
              8.3.1.  SRP IPS Packet Transfer Mechanism .......... 36
              8.3.2.  SRP IPS Signaling and Wrapping Mechanism ... 37
         8.4.  SRP IPS Protocol Rules ............................ 38
         8.5.  State Transitions ................................. 41
         8.6.  Failure Examples .................................. 41
              8.6.1.  Signal Failure - Single Fiber Cut Scenario . 41
              8.6.2.  Signal Failure - Bidirectional Fiber Cut
                      Scenario ................................... 43
              8.6.3.  Failed Node Scenario ....................... 45
              8.6.4.  Bidirectional Fiber Cut and Node Addition
              Scenarios .......................................... 47
     9.  SRP over SONET/SDH ...................................... 48
    10.  Pass-thru mode .......................................... 49
    11.  References .............................................. 50
    12.  Security Considerations ................................. 50
    13.  IPR Notice .. ........................................... 50
    14.  Acknowledgments ......................................... 50
    15.  Authors' Addresses ...................................... 51
    16.  Full Copyright Statement ................................ 52

1. Differences between SRP V1 and V2 โœจ

okay so this document is all about SRP V2 ๐Ÿ’– SRP V1 was an earlier draft spec. Here's what changed in V2 ๐ŸŽ€

  • Reduction of the header format from 4 bytes to 2 bytes.
  • Replacement of the keepalive packet with a new control packet. It carries usage information too, so it's doing keepalive and more, which is so efficient ๐Ÿงš
  • Change bit value of inner ring to be 1 and outer to be 0.
  • Reduction in the number of TTL bits from 11 to 8.
  • Removal of the DS bit. Bye bye ๐Ÿ‘‹
  • Change ordering of CRC transmission to be most significant octet first (was least significant octet in V1). The SRP CRC is now the same as in [5] ๐Ÿ”ฅ
  • Addition of the SRP cell mode to carry ATM cells over SRP ๐ŸŒˆ
  • Changes to the SRP-fa to increase the usage field width and to remove the need to add a fixed constant when propagating usage messages ๐Ÿ’…

2. Terms and Taxonomy ๐ŸŒธ

okay, this section is the vocab list, so let's get into it ๐Ÿฆ‹

2.1. Ring Terminology ๐Ÿ’œ

SRP uses a bidirectional ring. Think of it as two symmetric counter-rotating rings ๐Ÿ”„ Most of the protocol finite state machines (FSMs) are duplicated for the two rings.

The bidirectional ring lets you do ring-wrapping when there's a media or station failure, same as in FDDI [1] or SONET/SDH [3]. The wrapping is controlled by the Intelligent Protection Switching (IPS) protocol ๐Ÿ›ก๏ธ

To tell the two rings apart, one is called the "inner" ring and the other is the "outer" ring. SRP sends data traffic in one direction (called "downstream") and the corresponding control information goes the opposite direction (called "upstream") on the opposite ring. Figure 1 shows this so you can see it ๐Ÿ‘€

FIGURE 1. Ring Terminology

                                    {outer_data
                             -----   inner_ctl}
            ---------------->| N |-----------------
           |  ---------------| 1 |<--------------  |
           | |  {inner_data  -----               | |
           | |   outer_ctl}                      | |
          -----                                 -----
          | N |                                 | N |
          | 6 |                                 | 2 |
          -----                                 -----
           ^ |                                   ^ |
         o | |                                 i | |
         u | |                                 n | |
         t | |                                 n | |
         e | |                                 e | |
         r | |                                 r | |
           | v                                   | v
          -----                                 -----
          | N |                                 | N |
          | 5 |                                 | 3 |
          -----                                 -----
           | |                                   | |
           | |               -----               | |
           |  -------------->| N |---------------  |
            -----------------| 4 |<----------------
                             -----

2.2. Spatial Reuse ๐Ÿฆ‹

okay, this one's my fave ๐Ÿ˜ญ Spatial Reuse is a concept used in rings to boost the overall aggregate bandwidth of the ring. It works because unicast traffic only travels along the ring spans between the source and destination nodes, not the whole ring like in older ring based protocols such as token ring and FDDI ๐Ÿ”ฅ

Figure 2 below outlines how spatial reuse works. In this example, node 1 is sending traffic to node 4, node 2 to node 3, and node 5 to node 6. Because the destination node strips unicast data off the ring, nodes further downstream get full access to the ring bandwidth. So in this example, node 5 has full bandwidth access to node 6 while other traffic is being sent at the same time on other parts of the ring ๐Ÿ’– Literally so efficient, it's giving main character energy ๐Ÿ‘‘

(part 2 is coming up next, bestie ๐Ÿซถ)

2.3. Fairness

okay so the ring is a shared media, which means everyone is sharing the same road ๐Ÿ›ฃ๏ธโœจ and without some kind of access control it's chaos, bestie. we need fairness AND we need latency to stay bounded (so nobody waits forever ๐Ÿ˜ญ). there are two types of access control and they can totally work together ๐Ÿซถ

  • Global access control: makes sure everyone gets a fair share of the global bandwidth of the ring ๐Ÿ’–
  • Local access control: gives extra access on top of the global allocation so the parts of the ring that aren't fully used can get put to work ๐Ÿฆ‹

to see why you need both, look at Figure 2 again ๐Ÿ‘€ nodes 1, 2, and 5 get 1/2 of the bandwidth on a global allocation basis. but locally? node 5 should be able to get ALL of the bandwidth, because its traffic doesn't mess with the fair shares of nodes 1 and 2. main character energy, literally ๐Ÿ‘‘

FIGURE 2. Global and Local Re-Use

                                  . . . . . . . . . . . . . . . . .
                                  .                               .
                                -----                             .
               ---------------->| N |-----------------            .
              |  ---------------| 1 |<--------------  |           .
              | |               -----               | |           .
              | |                                   | |           .
             -----                                 -----          .
         . .>| N |                                 | N |. ..      .
         .   | 6 |                                 | 2 |   .      .
         .   -----                                 -----   .      .
         .    ^ |                                   ^ |    .      .
         .  o | |                                 i | |    .      .
         .  u | |                                 n | |    .      .
         .  t | |                                 n | |    .      .
         .  e | |                                 e | |    .      .
         .  r | |                                 r | |    .      .
         .    | v                                   | v    .      .
         .   -----                                 -----   .      .
         . . | N |                                 | N |<. .      .
             | 5 |                                 | 3 |          .
             -----                                 -----          .
              | |                                   | |           .
              | |               -----               | |           .
              |  -------------->| N |---------------  |           .
               -----------------| 4 |<----------------            .
                                -----                             .
                                  ^                               .
                                  .                               .
                                  . . . . .<. . . . . . . . . . . .

2.4. Transit Buffer

so to know when to send and when to receive packets on the ring, SRP uses a transit buffer (some people call it an insertion buffer ๐Ÿงš) ๐Ÿ’… check Figure 3. high priority packets and low priority packets can get stored in separate fifo queues, which is super organized of them ๐Ÿ“

FIGURE 3. Transit buffer

                         ^^               ||
                         ||               vv
                       |----|           |----|
                       |    |           |    |
                       |----|Rx         |----|Tx
                       |    |Buffer     |    |Buffer
                       |----|           |----|
                       |    |           |    |
                       |----|           |----|
                       |    |           |    |
                       |----|           |----|
                       |    |           |    |
                         ^^    Transit    ||
                         ||    Buffer     ||
                         ||    |------|   vv
                               |  H   |
                   ===========>|------|==========>
                               |  L   |
                               |------|

3. SRP Overview โœจ

3.1. Receive Operation Overview

okay so when a packet enters a node, it gets copied into the receive buffer if the Destination Address (DA) matches ๐ŸŽ€ if it's a DA matched packet AND it's unicast, the packet gets stripped (bye bye ๐Ÿ‘‹). if the packet doesn't DA match, or it's multicast and it doesn't Source Address (SA) match, it goes into the Transit Buffer (TB) to get forwarded to the next node, but only if it passes the Time To Live and Cyclic Redundancy Check (CRC) tests ๐Ÿ”ฅ no cap, the checks matter.

3.2. Transmit Operation Overview

so anything the node sends is either forwarded data from the TB or transmit data that started at this node and goes out through the Tx Buffer ๐Ÿฆ‹ high priority forwarded data always gets sent first, no debate ๐Ÿ’… high priority transmit data can go out as long as the Low Priority Transit Buffer (LPTB) isn't full.

a set of usage counters keeps an eye on how fast low priority transmit data and forwarded data are going out ๐Ÿ‘€ low priority data can be sent as long as the usage counter hasn't gone past its allowed usage (that limit is governed by the SRP-fa rules) AND the LPTB hasn't crossed the low priority threshold. it's giving rules, but fair rules ๐Ÿซถ

3.3. SRP Fairness Algorithm (SRP-fa) Overview

this is the part where the nodes look out for each other ๐Ÿฅน if a node is congested, it tells its upstream nodes (through the opposite ring) what its transmit usage counter is set to ๐Ÿ“ฃ the usage counter goes through a low pass filter so the feedback doesn't jump around all over the place โœจ upstream nodes then adjust their transmit rates so they don't go over the advertised values. nodes also pass along the advertised value they received to their immediate upstream neighbor ๐Ÿ’– and if a node that receives an advertised value is also congested, it passes on the minimum of its own transmit usage and the advertised usage. so supportive, honestly ๐Ÿงš

congestion gets detected when the depth of the low priority transit buffer reaches a congestion threshold ๐Ÿšจ

usage messages get generated periodically, and they double as keepalives, letting the upstream station know that a valid data link exists ๐Ÿ’œ bestie, that's how everybody stays in the loop ๐ŸŒˆ

3.4. Intelligent Protection Switching (IPS) Protocol Overview โœจ

okay so an SRP Ring is basically two rings spinning in opposite directions, each one a single fiber ๐Ÿ” if something breaks (equipment or fiber), the traffic heading toward and away from the trouble gets wrapped (looped) back around onto the other ring, going the opposite way ๐Ÿซถ (subject to the protection hierarchy, more on that in a sec). the wrap happens on the nodes right next to the failure, and the IPS protocol is the one running that show. the whole point is to route traffic around the broken span ๐Ÿ’…

here's what it looks like before and after a wrap, Figures 4 and 5 ๐ŸŽ€ before the fiber cut, N4 sends to N1 through N4->N5->N6->N1, nothing fancy.

if there's a fiber cut between N5 and N6, then N5 and N6 wrap the inner ring onto the outer ring ๐ŸŒ€ once those wraps are set up, traffic from N4 to N1 goes the long way around first: N4->N5->N4->N3->N2->N1->N6->N1. not the optimal path, but it's giving survival ๐Ÿ˜ญ

later on, a new ring topology gets discovered and traffic switches to the better path N4->N3->N2-N1, shown in Figure 6 โœจ just to be clear, the topology discovery and the optimal path selection are not part of the IPS protocol ๐Ÿ™ˆ

FIGURE 4. Data path before wrap, N4 -> N1

                            -----
           ################>| N |-----------------
          #  ---------------| 1 |<--------------  |
          # |               -----               | |
          # |                                   | |
         -----                                 -----
         | N |                                 | N |
         | 6 |                                 | 2 |
         -----                                 -----
          ^ |                                   ^ |
          # |                                   | |
          # |                                   | |
          # |                                   | |
          # |                                   | |
          # |                                   | |
          # v                                   | v
         -----                                 -----
         | N |                                 | N |
         | 5 |                                 | 3 |
         -----                                 -----
          # |                                   | |
          # |               -----               | |
          #  -------------->| N |---------------  |
           #################| 4 |<----------------
                            -----

the ring wrap is controlled through SONET BLSR [3][4] style IPS signaling ๐Ÿ’– and the goal is to wrap as fast as SONET equipment does, or even faster ๐Ÿš€

the IPS protocol handles the following request types, listed in order of priority from highest to lowest ๐Ÿ‘‘

  1. Forced Switch (FS): operator originated ๐Ÿง‘โ€๐Ÿ’ป, performs a protection switch on a requested span (wraps at both ends of the span)
  2. Signal Fail (SF): automatic, caused by a media Signal Failure or SRP keep-alive failure ๐Ÿ’€, performs a protection switch on a requested span
FIGURE 5. Data path after the wrap, N4 -> N1

                            -----
           ################>| N |-----------------
          #  ###############| 1 |<##############  |
          # #               -----               # |
          # v                                   # |
         -----                                 -----
         | N |                                 | N |
         | 6 |                                 | 2 |
         -----                                 -----
          ^ # wrap                              ^ |
          ###                                   # |
       _________                                # |
       fiber cut                                # |
       ---------                                # |
          ###                                   # |
          # v wrap                              # v
         -----                                 -----
         | N |                                 | N |
         | 5 |                                 | 3 |
         -----                                 -----
          # #                                   # |
          # #               -----               # |
          #  ##############>| N |###############  |
           #################| 4 |<----------------
  1. Signal Degrade (SD): automatic, caused by a media Signal Degrade (e.g. excessive Bit Error Rate) ๐Ÿ“‰, performs a protection switch on a requested span
  2. Manual Switch (MS): operator originated, like Forced Switched but of a lower priority ๐Ÿซ 
  3. Wait to Restore (WTR): automatic, entered after the working channel meets the restoration criteria after SF or SD condition disappears ๐Ÿ•ฐ๏ธ IPS waits the WTR period before restoring traffic, so that protection switch oscillations don't happen ๐ŸŒธ

if a protection is requested for a given span, whether automatic or operator originated, the node that requested it sends a protection request to the node on the other end of the span ๐Ÿ“จ it uses both the short path (over the failed span, since the failure might be unidirectional) and the long path (around the ring) ๐Ÿ’œ

FIGURE 6. Data path after the new topology is discovered

                            -----
           -----------------| N |-----------------
          |  ---------------| 1 |<##############  |
          | |               -----               # |
          | v                                   # |
         -----                                 -----
         | N |                                 | N |
         | 6 |                                 | 2 |
         -----                                 -----
          ^ | wrap                              ^ |
          --                                    # |
       _________                                # |
       fiber cut                                # |
       ---------                                # |
           --                                   # |
          | v wrap                              # v
         -----                                 -----
         | N |                                 | N |
         | 5 |                                 | 3 |
         -----                                 -----
          | |                                   # |
          | |               -----               # |
          |  -------------->| N |###############  |
           -----------------| 4 |<----------------
                            -----

as the protection requests zoom around the ring, the protection hierarchy gets applied ๐ŸŽข if the requested switch is the highest priority one, like how Signal Fail outranks Signal Degrade, then that switch happens and the lower priority switches elsewhere in the ring get taken down, as appropriate ๐Ÿ”ฅ if a lower priority request shows up while a higher priority request is already in the ring, it is not allowed ๐Ÿ™…โ€โ™€๏ธ the only exception is that multiple SF and FS switches can coexist in the ring ๐Ÿซถ

all protection switches are done bidirectionally, meaning wraps at both ends of a span for both transmit and receive directions, even if the failure is only unidirectional ๐ŸŒˆ

4. Packet Formats โœจ

okay, this section is all about the packet formats SRP uses ๐Ÿ“ฆ packets can ride over any point to point link layer (e.g. SONET/SDH, ATM, point to point ETHERNET connections) ๐Ÿงš the maximum transfer unit (MTU) is 9216 octets ๐Ÿ’– the minimum transfer unit for data packets is 55 octets ๐Ÿ“ the big maximum was designed to fit the large IP MTUs of IP over AAL5 ๐Ÿ’œ SRP also supports ATM cells, and ATM cells over SRP are 55 octets, so the minimum limit is there to fit ATM cells carried over SRP ๐Ÿฆ‹ the minimum limit doesn't apply to control packets, which are allowed to be smaller ๐ŸŒธ

these limits cover everything listed in Figure 7, but they don't count the frame delineation (for SRP over SONET/SDH, the flags used for frame delineation aren't included in the size limits) ๐Ÿซถ

the packet and cell formats below don't include any layer 1 frame delineation ๐Ÿ™ˆ for SRP over POS, there's an extra flag that marks the start and end of a frame ๐ŸŽ€

4.1. Overall Packet Format

the overall packet format is shown below in Figure 7 โœจ

FIGURE 7. Overall Packet Format

                 ---------------------------------
                 |       SRP Header              |
                 ---------------------------------
                 |       Dest. Addr.             |
                 ---------------------------------
                 |       Source Addr.            |
                 ---------------------------------
                 |       Protocol Type           |
                 ---------------------------------
                 |       Payload                 |
                 |                               |
                 |                               |
                 |                               |
                 ---------------------------------
                 |       FCS                     |
                 ---------------------------------

the frame check sequence (FCS) is a 32-bit cyclic redundancy check (CRC) as specified in RFC-1662, and it's the same CRC used in Packet Over SONET (POS - specified in RFC-2615) ๐Ÿ” the generator polynomial is:

CRC-32:

x32 + x26 + x23 + x22 + x16 + x12 + x11 + x10 + x8 + x7 + x5 + x4 +
x2 + x + 1

the FCS is computed over the destination address, source address, protocol type and payload ๐Ÿ’… it does not include the SRP header ๐Ÿ™…โ€โ™€๏ธ

and iykyk, but the packet format after the SRP header is identical to Ethernet Version 2 ๐Ÿซถ

4.2. Generic Packet Header Format

every packet has a fixed-sized header ๐Ÿ“ฆ the packet header format is shown in Figure 8 ๐Ÿฆ‹

FIGURE 8. Detailed Packet Header Format

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |  Time to Live |R| MOD | PRI |P|                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+     Destination Address       |
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                                                               |
    +    Source Address             +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                               |     Protocol Type             |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                                                               |
    +                                                               +
    |                         Payload                               |
    .                                                               .
    .                                                               .
    .                                                               .
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

the fields are described below, so let's get into it ๐Ÿงš๐Ÿ’œ

4.2.1. Time To Live (TTL) โณ

Okay bestie, TTL is an 8 bit hop-count ๐Ÿงš and every node that forwards a packet MUST decrement it. Once it hits zero, the packet gets stripped off the ring ๐Ÿ’€. In theory that's room for 256 nodes on one ring, but ngl, because of certain failure conditions (like when the ring is wrapped), SRP only supports 128 nodes total ๐Ÿซถ. When a packet first goes onto the ring, the TTL SHOULD be set to at least twice the total number of nodes on the ring โœจ.

4.2.2. Ring Identifier (R) ๐ŸŒˆ

This single bit field tells you which ring the packet is meant for ๐Ÿฆ‹ The lineup is:

TABLE 1. Ring Indicator Values

        Outer Ring      0
        Inner Ring      1

4.2.3. Priority Field (PRI) ๐Ÿ‘‘

This three bit field shows the priority level of the SRP packet, from 0 through 7. Higher number, higher priority ๐Ÿ”ฅ. Here's the plot twist: the transit buffer only has two queues (HPTB and LPTB), so once a packet is on the ring it's just treated as either low or high priority ๐Ÿ’…. Each node picks its own threshold for what counts as "high" and what counts as "low." BUT the full 8 levels are still available before the packet hits the ring (transmit queues) and after it comes off the ring (receive queues) ๐ŸŽ€.

4.2.4. MODE ๐Ÿ“

This three bit field tells you what kind of packet you're looking at. Check out Table 2:

TABLE 2. MODE Values

Value   Description

000     Reserved
001     Reserved
010     Reserved
011     ATM cell
100     Control Message (Pass to host)
101     Control Message (Locally Buffered for host)
110     Usage Message
111     Packet Data

We'll dig into these later on, I promise ๐Ÿซถ.

4.2.5. Parity Bit (P-bit) ๐Ÿ™ˆ

This bit holds the parity value over the 15 bits of the SRP header, so the header gets a little extra data integrity ๐Ÿ”’. It's odd parity, meaning the number of ones (parity bit included) SHALL be an odd number. Sooo obsessed with how precise that is ๐Ÿ˜ญ.

4.2.6. Destination Address ๐Ÿ’Œ

This is a globally unique 48 bit address that the IEEE assigns ๐ŸŒธ.

4.2.7. Source Address ๐Ÿงš

Same deal as the destination address: a globally unique 48 bit address handed out by the IEEE ๐Ÿ’–.

4.2.8. Protocol Type ๐Ÿฆ‹

This is a two octet field that looks a lot like the EtherType representation ๐ŸŽ€. These are the protocol types that matter for SRP, per Table 3:

TABLE 3. Defined Protocol Types

Value   Protocol Type

0x2007  SRP Control
0x0800  IP version 4
0x0806  ARP

4.3. SRP Cell Format ๐Ÿง

SRP can also send ATM cells, so here's how those look ๐Ÿ’…:

FIGURE 9. SRP Cell Format

        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |  Time to Live |R| MOD | PRI |P|         VPI/VCI               |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |        VCI            | PTI |C|     HEC       |               |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+               +
       |                                                               |
       .                                                               .
       .                    ATM   Payload                              .
       .                    ( 48 Bytes )               +-+-+-+-+-+-+-+-+
       |                                               |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Packet nodes usually ignore ATM cells (they never receive or strip them) and always forward them instead ๐Ÿฆ‹. The vibe is that ATM switches and routers can coexist on the same ring ๐Ÿซถ. Heads up: SRP cells don't have an FCS, and data integrity is handled at the AAL layer instead ๐Ÿ™ˆ.

4.4. SRP Usage Packet Format ๐Ÿ“ฃ

Usage packets get sent out regularly to tell upstream nodes what usage is allowed ๐Ÿ’…. They double as a keepalive too, so they're basically multitaskers โœจ. SRP usage packets SHOULD be sent about every 106 usec.

If a receive interface hasn't seen a usage packet within the keepalive timeout interval, it triggers an L2 keepalive timeout interrupt/event ๐Ÿ˜ญ. The IPS software then marks that interface as faulty and starts a protection switch around it ๐Ÿšจ. The keepalive timeout interval SHOULD be set to 16 times the SRP usage packet transmission interval ๐ŸŒˆ.

FIGURE 10. Usage Packet Format

        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |  Time to Live |R| MOD | PRI |P|                               |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  Originator MAC Address       +
       |                                                               |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |  Reserved                     |    Usage                      |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

A USAGE of all ones means NULL ๐Ÿ™ƒ.

4.5. SRP Control Packet Format ๐ŸŽ›๏ธ

When the MODE bits are set to 10X (SRP control), you've got a control message on your hands ๐Ÿงš. Control messages are always received and stripped by the adjacent node, and they're unicast by definition, so they don't need any addressing info ๐Ÿ’œ. The destination address field for control packets SHOULD be set to 0's. The source address field SHOULD be set to the source address of the node that's transmitting ๐Ÿซถ.

There are two kinds of control messages: Pass to host and Locally buffered ๐Ÿ‘‘. Pass to host messages can get to the host software however is most convenient, and that's usually the same path data packets take ๐Ÿฆ‹. Locally buffered messages are mostly for protection messages, and they get buffered locally so they don't fight with data packets for resources ๐Ÿ’…. How you actually handle them is up to the implementor ๐ŸŒธ.

FIGURE 11. Control Packet Format

        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |  Time to Live |R| MOD | PRI |P|                               |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+     Destination Address       |
       |                                                               |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                                                               |
       +    Source Address             +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |                               |     Protocol Type = 0x2007    |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       | Control Ver   | Control Type  |    Control Checksum           |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |   Control TTL                 |                               |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
       .                                                               .
       .   Payload                                                     .
       .                                                               .
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The PRI value SHOULD be set to 0x7 (all one's) when sending control packets, and they SHOULD be queued to the highest priority transmit queue available ๐Ÿš€. TTL doesn't really matter here, since the nearest downstream neighbor receives and strips every control packet anyway ๐Ÿ™ƒ. It can be set to any value, though preferably this SHOULD be set to 001 โœจ.

4.5.1. Control Ver ๐Ÿ”ข

This one octet field is the version number for the control type field ๐Ÿ’–. To start, every control type is version 0 ๐ŸŒฑ.

4.5.2. Control Type ๐Ÿงฉ

This one octet field is the control message type. Table 4 has the currently defined ones:

TABLE 4. Control Types

Control Type    Description

0x01            Topology Discovery

0x02            IPS message

0x03-
0xFF            Reserved

4.5.3. Control TTL โณ

Control TTL is a control layer hop-count that MUST be decremented every time a node forwards a control packet ๐Ÿ”. If a node gets a control packet with a control TTL <= 1, it SHOULD accept the packet but not forward it ๐Ÿ›‘.

Quick note: the control layer hop count is separate from the SRP L2 TTL, and the L2 TTL is always set to 1 for control messages ๐Ÿง . The originator SHOULD set the initial control TTL to whatever SRP L2 TTL it normally uses for data packets ๐Ÿ’….

4.5.4. Control Checksum ๐Ÿงฎ

The checksum field is the 16 bit one's complement of the one's complement sum of all 16 bit words, starting from the control version ๐Ÿค“. If there's an odd number of octets, the last one gets padded on the right with zeros to make a 16 bit word for checksum purposes. That pad isn't transmitted as part of the segment ๐Ÿซถ. While computing, the checksum field itself is replaced with zeros. It's the same checksum algorithm TCP uses ๐Ÿฆ‹, and it doesn't cover the 32 bit SRP FCS ๐Ÿ’œ.

4.5.5. Payload ๐Ÿ“ฆ

The payload is variable length, and its size depends on the control type โœจ.

4.5.6. Addressing ๐Ÿก

All nodes MUST have a globally unique IEEE 48 bit MAC address ๐Ÿ’–. There's a multicast bit using canonical addressing conventions, which means it's the least significant bit of the most significant octet in the destination address ๐ŸŽ€. It's acceptable but not advisable to change a node's MAC address to one that's known to be unique within the administrative layer 2 domain. That domain is the SRP ring itself plus any networks hooked up to the ring through a layer 2 transparent bridge ๐Ÿ™ˆ.

FIGURE 12. Multicast bit position

                   Destination Address
        0                   1                   2                   3
        0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
       |             |M|                                               |
       +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
                      ^
                      |----Multicast bit

For SONET media, the network order is MSB of each octet first, so when you look at the line, the multicast bit is the 8th bit of the destination address sent ๐Ÿ“ก. (For SRP on Ethernet media, the multicast bit gets sent first instead ๐ŸŒˆ.)

4.6. Topology Discovery ๐Ÿ—บ๏ธ

okay so every node does topology discovery by sending out topology discovery packets on one or both rings ๐Ÿ’… the node that starts a topology packet stamps it with the egressing ring id, tacks on the node's mac binding, and sets the length field before it goes out ๐ŸŽ€ this packet is point-to-point and hops around the ring from node to node like a gossip chain ๐Ÿฆ‹ each node adds its mac address binding, updates the length field, and passes it to the next hop on the ring โœจ if there's a wrap on the ring, the wrapped node says "wrap!" when it appends its mac binding, and then it wraps the packet ๐ŸŒ€ when topology packets travel through the wrapped section with a ring identifier that's different from the topology packet's own, the mac address bindings are not added to the packet ๐Ÿ™ˆ

eventually the node that started the topology discovery packet gets its packet back ๐Ÿฅน that node makes sure the packet has the same ingress and egress ring id before excepting the packet ๐Ÿ’– and the topology map only changes after it gets two topology packets that show the same new topology, so it doesn't freak out over transient conditions ๐Ÿงš

ngl the topology map only has the reachable nodes ๐Ÿ‘€ it does not match the failure-free ring when there are wraps and ring segmentations, so don't expect it to ๐Ÿ’€

FIGURE 13. Topology Packet Format

    Topology

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |  Time to Live |R| MOD | PRI |P|                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+     Destination Address       |
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                                                               |
    +    Source Address             +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                               |     Protocol Type = 0x2007    |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    | Control Ver=0 | Control Type=1|    Control Checksum           |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |   Control TTL                 |   Topology Length             |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |       Originator's Globally Unique                            |
    +       MAC Address  (48 bits)  +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                               |  MAC Type     |               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+               +
    |                   MAC Address (48 bits)                       |
    +               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |               |   Other MAC bindings                          |
    +-+-+-+-+-+-+-+-+                                               +
    |                                                               |
    +                                                               +

quick reminder ๐Ÿซถ the Source address should be set to the source address of the TRANSMITTING node, which is not necessarily the ORIGINATING node ๐ŸŽ€

4.6.1. Topology Length ๐Ÿ“

this is a two octet field that tells you the length of the topology message in octets, starting from the first MAC Type/MAC Address binding ๐Ÿ’œ

4.6.2. Topology Originator ๐Ÿงพ

a topology discovery packet counts as originated by a node when the packet's originator globally unique MAC address is that node's globally unique MAC address (the one assigned by the IEEE) ๐ŸŒธ

because mac addresses can be changed at a node, the IEEE MAC address gives you a unique identifier ๐Ÿ”‘ that's how a node knows the topology packet has gone all the way around the ring and it's time to consume it ๐Ÿ“

4.6.3. MAC bindings ๐Ÿ”—

each MAC binding shall consist of a MAC Type field followed by the node's 48 bit MAC address โœจ the first MAC binding shall be the MAC binding of the originator ๐Ÿ‘‘ usually the originator's MAC address will be its globally unique MAC Address, but some implementations may let the network administrator override that value ๐Ÿงšโ€โ™€๏ธ

4.6.4. MAC Type Format ๐Ÿท๏ธ

this 8 bit field is encoded like this ๐Ÿ’…

TABLE 5. MAC Type Format

Bit     Value

0       Reserved
1       Ring ID (1 or 0)
2       Wrapped Node (1) / Unwrapped Node (0)
3-7     Reserved

so here's the tea ๐Ÿซ– a packet's egress and ingress ring IDs get checked by using the Ring ID in the MAC Type field of the last MAC binding as the ingress ring ID, not the R bit in the SRP header ๐Ÿฆ‹ they should be the same, but it's better to keep the two functions separate because some implementations may not hand the SRP header up to upper layer protocols ๐Ÿ™ˆ

the topology info is not required for the IPS protection mechanism ๐Ÿ’ซ you can use it to count how many nodes are in the ring, and to figure out hop distances to nodes so you can find the shortest path to a node (since there are two counter-rotating rings) ๐ŸŒˆ

topology discovery could be periodic or "a need to discover" basis ๐Ÿ•ฐ๏ธ in the periodic version, each node generates a topology packet on a schedule and keeps using its cached topology map until it gets a new one ๐ŸŽ€ in the need to discover version, each node generates a topology discovery packet whenever it needs one, e.g., on first entering a ring or detecting a wrap ๐Ÿซถ

4.7. Intelligent Protection Switching (IPS) ๐Ÿ›ก๏ธ

IPS is a method for automatically recovering from various ring failures and line degradation scenarios โญ the IPS packet format is shown in Figure 14 below ๐Ÿ‘‡

FIGURE 14. IPS Packet Format

     0                   1                   2                   3
     0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |  Time to Live |R| MOD | PRI |P|                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+     Destination Address       |
    |                                                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                                                               |
    +    Source Address             +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |                               |     Protocol Type = 0x2007    |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    | Control Ver=0 | Control Type=2|    Control Checksum           |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |   Control TTL                 |                               |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+                               +
    |             Originator MAC Address                            |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
    |   Ips Octet   |  Rsvd Octet   |
    +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

the IPS specific fields are explained below ๐Ÿง

4.7.1. Originator MAC Address ๐Ÿชช

this is the MAC address of whoever originated the IPS message ๐Ÿ’Œ it isn't necessarily the same as the SRP Header Source Address, because a node might just be passing along an IPS message (check out the section "SRP IPS Protocol Rules" Rule P.8 for an example) ๐Ÿ•ต๏ธโ€โ™€๏ธ

4.7.2. IPS Octet ๐ŸŽ›๏ธ

the IPS octet holds the specific protection info ๐Ÿญ here's how its format goes ๐Ÿ‘€

FIGURE 15. IPS Octet Format:

Bits    Values (values not listed are reserved)

0-3     IPS Request Type

        1101 - Forced Switch (FS)
        1011 - Signal Fail (SF)
        1000 - Signal Degrade (SD)
        0110 - Manual Switch (MS)
        0101 - Wait to Restore (WTR)
        0000 - No Request (IDLE)

4       Path indicator

        0 - short (S)
        1 - long (L)

5-7     Status Code

        010 - Protection Switch Completed - traffic Wrapped (W)
        000 - Idle (I)

the currently defined request types, with their values, hierarchy and interpretation, are as used in SONET BLSR [3], [4], except as noted ๐Ÿ“š

4.8. Circulating packet detection (stripping) ๐ŸŒ€

packets can keep circulating when they fail to get stripped ๐Ÿ˜ฑ unicast packets are normally stripped by the destination station, or by the source station if the destination station has failed ๐Ÿ’… multicast packets are only stripped by the source station ๐Ÿซถ if both the source and destination stations drop out of the ring while a unicast packet is in flight, or if the source node drops out while its multicast packet is in flight, the packet will rotate around the ring forever ๐Ÿ™ƒ

the fix is to give each packet a TTL, or Time To Live, field that's set to at least twice the number of nodes in the ring โณ every node that forwards the packet decrements the TTL, and once the TTL hits zero, the packet gets stripped off the ring ๐Ÿ‘‹

the ring ID is used to qualify all stripping and receive decisions ๐ŸŽ€ this matters when some node in the ring is wrapping packets ๐ŸŒˆ the sending node may see its packet on the reverse ring before it reaches its destination, so it must not source strip it ๐Ÿ™ˆ the exception is if a node is in wrap ๐Ÿ’ซ logically, a node in wrap "sees" the packet on both rings, but the usual implementation is to receive the packet on one ring and transmit it on the other, so a node in the wrap state ignores the ring ID when making stripping and receiving decisions ๐Ÿฆ‹

there's a potential optimization where unicast packets get ring ID independent destination stripping ๐Ÿ‘€ the problem is that packets may be delivered out of order during a transition to a wrap condition ๐Ÿ˜ฌ for that reason, the ring ID should always be used as a qualifier for all strip and receive decisions ๐Ÿ’–

5. Packet acceptance and stripping โœจ

okay so, every packet that comes in to a MAC gets a series of checks based on its mode, source address, and destination address ๐Ÿ‘€. Packets are either control packets or data packets. Control packets get stripped as soon as their info has been pulled out ๐Ÿงน. Data packets get their source and destination addresses checked. The receive and strip rules are listed below, and they're also drawn out in Figure 16 ๐Ÿฆ‹.

  1. Decrement TTL when a packet arrives. If it hits zero, throw it away and don't forward it ๐Ÿ’€.
  2. Strip unicast packets at their destination station. Accept and strip "control" packets too ๐Ÿซถ.
  3. Don't process packets (except for TTL and forwarding) if they have the "wrong" ring_id for the direction they came in on, unless the node is in wrap. If the node is in wrap, ignore the ring_id ๐Ÿ”„.
  4. Don't process packets (except for TTL and forwarding) if the node doesn't support their mode. Think reserved modes, or ATM cell mode on packet nodes ๐Ÿ™…โ€โ™€๏ธ.
  5. If the host accepted a packet because of its destination address but there's a CRC error, the upper level should discard it ๐Ÿ—‘๏ธ.
  6. Control messages go point to point between neighbors, so they should always be accepted and stripped ๐Ÿ’Œ.
  7. Packets whose source address belongs to the receiving station, with a matching ring_id, should be stripped. If the node is in wrap, ignore the ring_id ๐ŸŒ€.

FIGURE 16. SRP Receive Flowchart (Packet node)

   if (MODE == 4,5)-------------------------------->[to host]--->|
           |                                                     |
           v                                                     |
   if (MODE == 6)---------------------------------->[strip]----->|
           |                                                     |
           v                                                     |
   if (!WRAPPED                                                  |
      & WRONG_RING_ID)-------------------------------------------|--->|
           |                                                     |    |
           v                                                     |    |
   if (MODE == 0,1,2,3)------------------------------------------|--->|
           |                                                     |    |
           v                                                     |    |
   if (DA MATCH)--------------->if !(SA MATCH)----->[to host]--->|    |
           |                            |                        |    |
           |                            v                        |    |
           |                    if (unicast)------->[to host]--->|    |
           |                            |                        |    |
           |                            v                        |    |
   if (SA MATCH)-------------------->[strip]-------------------->|    |
           |                                                     |    |
           |                                                     |    v
           |--------------------------->|<-----------------------|----|
                                        |                        |
                                        v                        |
                                if (ttl < 2)------->[strip]----->|
                                        |                        |
                                        v                        |
                                [decrement ttl]                  |
                                        |                        |
                                [fwd pkt to tb]                  |
                                        |                        v
                                        |<-----------------------|
                                        v
                                  [back to top]

Notes: the host is in charge of tossing CRC-errored packets ๐Ÿงบ. In the diagram, conditionals branch right when they're true and down when they're false ๐Ÿงญ.


5.1. Transmission and forwarding with priority ๐Ÿš€

a node can send four kinds of packets, bestie:

  1. High priority packets from the high priority transit buffer ๐Ÿ”ฅ.
  2. Low priority packets from the low priority transit buffer ๐ŸŒ™.
  3. High priority packets from the host's Tx high priority fifo ๐Ÿ’–.
  4. Low priority packets from the host's Tx low priority fifo ๐Ÿง.

High priority transit packets always go out first ๐Ÿ‘‘. High priority host packets go out as long as the low priority transit buffer isn't full. Low priority packets go out as long as the transit buffer hasn't crossed the low priority threshold and the SRP-fa rules allow it (my_usage < allowed_usage) ๐Ÿ’…. If nothing else can go, low priority packets from the low priority transit buffer get sent ๐Ÿซ .

Figure 17 shows the whole decision tree ๐ŸŒณ.

FIGURE 17. SRP transmit flowchart

   if (TB_High has pkt)----------->[send pkt from TB_high]-->|
           |                                                 |
           v                                                 |
   if (TB_Low full)------------------------------------------|---->|
           |                                                 |     |
           v                                                 |     |
   if (Tx_High has pkt)----------->[send pkt from Tx_high]-->|     |
           |                                                 |     |
           v                                                 |     |
   if (TB_Low > Hi threshold)--------------------------------|---->|
           |                                                 |     |
           v                                                 |     |
   if (my_usage >= allowed_usage)----------------------------|---->|
           |                                                 |     |
           v                                                 |     |
   if (Tx_Low has pkt)------------>[send pkt from Tx_low]--->|     |
           |                                                 |     |
           |                                                 |     v
           |<------------------------------------------------|-----|
           |                                                 |
           v                                                 |
   if (TB_Low has pkt)------------>[send pkt from TB_low]--->|
           |                                                 v
           |<------------------------------------------------|
           |
           v
       [Go to Top]

Notes: conditionals branch right when true and down when false ๐Ÿ”€.


5.2. Wrapping of Data ๐ŸŒฏ

usually, transmitted data goes out on the same ring to the downstream neighbor โžก๏ธ. BUT if a node is in the wrapped state, its transmitted data goes out on the opposite ring to the upstream neighbor ๐Ÿ”. plot twist ๐Ÿ˜ค.

6. SRP-fa Rules Of Operation ๐Ÿง 

the SRP-fa is the thing that governs access to the ring ๐Ÿšฆ. It only applies to low priority traffic. High priority traffic doesn't follow SRP-fa rules and can be sent any time there's enough transit buffer space ๐Ÿƒโ€โ™€๏ธ.

the SRP-fa uses three counters to control the traffic that gets forwarded and sourced on the SRP ring ๐Ÿ”ข. They are my_usage (tracks how much traffic the node sources onto the ring), forward_rate (how much traffic gets forwarded onto the ring from sources other than the host), and allowed_usage (the current maximum transmit usage for that node) ๐Ÿ“Š.

with no congestion, every node builds up its allowed usage over time ๐Ÿ“ˆ. Each node can send up to max_usage. Max_usage is a per-node parameter that caps how much low priority traffic a node can send ๐Ÿงข.

when a node sees congestion, it starts advertising its my_usage, run through a low pass filter (lp_my_usage) ๐Ÿ“ฃ.

congestion gets measured by the transit buffer depth crossing a congestion threshold ๐Ÿšจ.

a node that gets a non-null usage message (rcvd_usage) sets its allowed usage to the advertised value. BUT if the rcvd_usage came from the very same node that received it, then the rcvd_usage shall be treated as a null value ๐Ÿ™ƒ. When comparing the rcvd_usage source address, the ring ID of the usage packet has to match the receiver's ring ID for the comparison to count. The exception is when the receiving node is in the wrap state. Then the usage packet's ring ID is ignored ๐ŸŒ€.

nodes that aren't congested and that receive a non-null rcvd_usage will generally pass rcvd_usage along to their upstream neighbor. Otherwise they pass along a null value of usage (all 1's) ๐Ÿซง. The exception is when an opportunity for local reuse shows up. Extra spatial reuse (local reuse) happens by comparing the forwarded rate (low pass filtered) to allow_usage. If the forwarded rate is less than the allowed usage, a null value gets passed upstream ๐ŸŒธ.

nodes that are congested pass along the smaller of lp_my_usage and rcvd_usage ๐Ÿ˜ฎโ€๐Ÿ’จ.

convergence depends on how many nodes there are and how far apart they are ๐Ÿ“. Simulation has shown convergence within 100 msec for rings of several hundred miles โฑ๏ธ. literally so fast ๐ŸŽ๏ธ๐Ÿ’จ.

6.1. SRP-fa pseudo-code ๐Ÿ’ป

here's a more precise definition of the fairness algorithm ๐Ÿค“:

Variables:

lo_tb_depth     low priority transit buffer depth

my_usage        count of octets transmitted by host
lp_my_usage     my_usage run through a low pass filter
my_usage_ok     flag indicating that host is allowed to transmit

allow_usage     the fair amount each node is allowed to transmit

fwd_rate        count of octets forwarded from upstream
lp_fwd_rate     fwd_rate run through a low pass filter

congested       node cannot transmit host traffic without the TB buffer
                filling beyond its congestion threshold point.

rev_usage       the usage value passed along to the upstream neighbor

Constants:

MAX_ALLOWANCE = configurable value for max allowed usage for this node

DECAY_INTERVAL = 8000 octet times @ OC-12, 32,000 octet times @ OC-48

AGECOEFF = 4    // Aging coeff for my_usage and fwd_rate

LP_FWD  = 64    // Low pass filter for fwd_rate
LP_MU   = 512   // Low pass filter for my usage
LP_ALLOW = 64   // Low pass filter for allow usage auto increment

NULL_RCVD_INFO = All 1's in rcvd_usage field

TB_LO_THRESHOLD // TB depth at which no more lo-prio host traffic
                // can be sent

MAX_LRATE = AGECOEFF * DECAY_INTERVAL = 128,000 for OC-48, 32000 for
            OC-12

THESE ARE UPDATED EVERY CLOCK CYCLE:
=====================================

my_usage        is incremented by 1 for every octet that is
                transmitted by the host (does not include data
                transmitted from the Transit Buffer).

fwd_rate        is incremented by 1 for every octet that enters the
                Transit Buffer

if ((my_usage < allow_usage) &&
        !((lo_tb_depth > 0) && (fwd_rate < my_usage)) &&
                (my_usage < MAX_ALLOWANCE))
        // true means OK to send host packets
        my_usage_ok = true;


UPDATED WHEN USAGE_PKT IS RECEIVED:
===================================

if (usage_pkt.SA == my_SA) &&
        [(usage_pkt.RI == my_RingID) || (node_state == wrapped)]
        rcvd_usage = NULL_RCVD_INFO;
else
        rcvd_usage = usage_pkt.usage;


THE FOLLOWING IS CALCULATED EVERY DECAY_INTERVAL:
==================================================

congested = (lo_tb_depth > TB_LO_THRESHOLD/2)

lp_my_usage = ((LP_MU-1) * lp_my_usage + my_usage) / LP_MU

my_usage is decremented by min(allow_usage/AGECOEFF, my_usage/AGECOEFF)

lp_fwd_rate = ((LP_FWD-1) * lp_fwd_rate + fwd_rate) / LP_FWD

fwd_rate is decremented by fwd_rate/AGECOEFF
(Note: lp values must be calculated prior to decrement of non-lp
values).

if (rcvd_usage != NULL_RCVD_INFO)
        allow_usage = rcvd_usage;
else
        allow_usage += (MAX_LRATE - allow_usage) / (LP_ALLOW);

if (congested)
      {
        if (lp_my_usage < rcvd_usage)
                rev_usage = lp_my_usage;
        else
                rev_usage =  rcvd_usage;
        }
else if ((rcvd_usage != NULL_RCVD_INFO) &&
         (lp_fwd_rate > allow_usage)
    rev_usage = rcvd_usage;
else
    rev_usage = NULL_RCVD_INFO

if (rev_usage > MAX_LRATE)
        rev_usage = NULL_RCVD_INFO;

Okay so the lp values get worked out before the non-lp values get decremented, and that ordering is literally the whole vibe ๐Ÿ’… Then we pick what gets sent back: if we're congested, we take whichever is smaller, and otherwise we only send the received usage when the lp forward rate is bigger than allow_usage ๐ŸŽ€ And if the result is above MAX_LRATE, it gets thrown out. Iconic, no notes โœจ

6.2. Threshold settings

Okay now the buffer sizes ๐Ÿฆ‹ The low priority transit buffer (TB_LO_THRESHOLD) is currently sized to about 4.4 msec or 320 KB at OC12 rates. The TB_HI_THRESHOLD is set to about 870 usec higher than the TB_LO_THRESHOLD, or at 458 KB at OC12 rates. The high priority transit buffer needs to hold 2 to 3 MTUs or about 30KB. Numbers, bestie ๐ŸŒธ

7. SRP Synchronization โœจ

Okay so each node runs in "free-run" mode ๐Ÿงš That means the receive clock comes from the incoming receive stream, and the transmit clock comes from a local oscillator. And this is lowkey genius, because it kills the need for expensive clock synchronization like you'd have in existing SONET networks ๐Ÿ’… Differences in clock frequency get handled by slipping in a small amount of idle bandwidth at each node's output.

The clock source for the transmit clock shall be selected to deviate by no more than 20 ppm from the center frequency. The overall outgoing rate of the node shall be rate shaped to accommodate the worst case difference between receive and transmit clocks of adjacent nodes. Here's how it works ๐Ÿซถ

A transit buffer slip count (tb_cnt) keeps track of the amount of octets inserted into the TB minus the amount of octets transmitted, and it's a positive integer ๐Ÿ”ฅ

To account for a startup condition, where a packet is being inserted into an empty TB and the node was otherwise idle, the tb_cnt is reset if the transmit interface is idle. Idle is defined as no data being sent even though there is opportunity to send (i.e. the transmit interface is not prohibited from transmitting by the physical layer). It's giving careful planning ๐Ÿ™ˆ

An interval counter defines the sample period over which rate shaping is performed. This number should be sufficiently large to get an accurate rate shaping ๐Ÿ“

A token_bucket counter implements the rate shaping and is a signed integer. We increment this counter by one of two fixed values called quantums each sample period. Quantum1 sets the rate at (Line_rate - Delta), where delta is the clock inaccuracy we want to accommodate โœจ

Quantum2 sets the rate at (Line_rate + Delta). If at the beginning of a sample period, tb_cnt >= sync_threshold, then we set the rate to Quantum2. This will allow us to catch up, and it causes the TB slip count to eventually go < sync_threshold. If tb_cnt is < sync_threshold, then we set the rate to Quantum1. Catch-up energy, literally ๐Ÿƒโ€โ™€๏ธ๐Ÿ’–

When the input rate and output rates are exactly equal, the tb_cnt will vary between sync_threshold > tb_cnt >= 0. This will vary for each implementation dependent upon the burst latencies of the design. The sync_threshold value should be set so that, for equal transmit and receive clock rates, the transmit data rate is always Line_rate-Delta and will be implementation dependent. Main character energy in the design phase ๐Ÿฆ‹

The token_bucket is decremented each time data is transmitted. When token_bucket reaches a value <= 0, a halt_transmit flag is asserted, which halts further transmission of data. (Halting occurs on a packet boundary, of course, which can cause token_bucket to become a negative number.) ๐ŸŒˆ

7.1. SRP Synchronization Examples

Okay the math era is here ๐Ÿงฎ Assume an interval of 2^^18 or 262144 clock cycles. A Quantum1 value must be picked such that the data rate will = (LINE_RATE - DELTA). A Quantum2 value must be picked and used if the tb_cnt shows that the incoming rate is greater than the outgoing rate and is = (LINE_RATE + DELTA). Assume that the source of the incoming and outgoing rate clocks are +/- 100 ppm ๐Ÿ’œ

For an OC12c SPE rate of 600 Mbps and a system clock rate of 800 Mbps (16 bits @ 50 Mhz), the system clock rate is the rate at which the system transmits bytes to the framer. (In most cases the framer transmit rate is asynchronous from the rate at which the system transfers data to the framer.) Here we go ๐Ÿ‘‘

     Quantum1/Interval * 800 Mbps = 600 Mbps(1 - Delta)
     Quantum1 = Interval * (600/800) * (1 - Delta)
     Quantum1 = Interval * (600/800) * (1 - 1e-4) = 196588

     Quantum2/Interval * 800 Mbps = 600 Mbps(1 + Delta)
     Quantum2 = Interval * (600/800) * (1 + Delta)
     Quantum2 = Interval * (600/800) * (1 + 1e-4) = 196628

Note: The actual data rate for OC-12c is 599.04 Mbps. ๐ŸŽ€

8. IPS Protocol Description

Okay this is the protection-switching chapter and it's SO good ๐Ÿ˜ญ An SRP ring is composed of two counter-rotating, single fiber rings. If an equipment or fiber facility failure is detected, traffic going towards and from the failure direction is wrapped (looped) back to go in the opposite direction on the other ring. The wrap around takes place on the nodes adjacent to the failure, under software control. This way the traffic is re-routed from the failed span. Reroute glow-up ๐Ÿฆ‹

Nodes communicate between themselves using IPS signaling on both inner and outer ring ๐Ÿซถ

The IPS octet contains specific protection information. The format of the IPS octet is as follows:

FIGURE 18. IPS Octet format:

0-3     IPS Request Type

        1101 - Forced Switch (FS)
        1011 - Signal Fail (SF)
        1000 - Signal Degrade (SD)
        0110 - Manual Switch (MS)
        0101 - Wait to Restore (WTR)
        0000 - No Request (IDLE)

4       Path indicator

        0 - short (S)
        1 - long (L)

5-7     Status Code

        010 - Protection Switch Completed -traffic Wrapped (W)
        000 - Idle (I)

The IPS control messages are shown in this document as:

{REQUEST_TYPE, SOURCE_ADDRESS, WRAP_STATUS, PATH_INDICATOR}

8.1. The IPS Request Types

The following is a list of the request types, from the highest to the lowest priority. All requests are signaled using IPS control messages. Ranking time ๐Ÿ‘‘

  1. Forced Switch (FS - operator originated)

This command performs the ring switch from the working channel to the protection, wrapping the traffic on the node at which the command is issued and at the adjacent node to which the command is destined. Used for example to add another node to the ring in a controlled fashion. ๐Ÿงš

  1. Signal Fail (SF - automatic)

Protection caused by a media "hard failure" or SRP keep-alive failure. SONET examples of SF triggers are: Loss of Signal (LOS), Loss of Frame (LOF), Line Bit Error Rate (BER) above a preselected SF threshold, Line Alarm Indication Signal (AIS). Note that the SRP keep-alive failure provides end-to-end coverage, and as a result SONET Path triggers are not necessary. ๐Ÿ’€

  1. Signal Degrade (SD - automatic)

Protection caused by a media "soft failure". SONET example of a SD is Line BER or Path BER above a preselected SD threshold. ๐ŸŒธ

  1. Manual Switch (MS - operator originated)

Like the FS, but of lower priority. Can be used for example to take down the WTR. ๐Ÿ“

  1. Wait to Restore (WTR - automatic)

Entered after the working channel meets the restoration threshold after an SD or SF condition disappears. IPS waits WTR timeout before restoring traffic, in order to prevent protection switch oscillations. Patience is a whole aesthetic โœจ

8.2. SRP IPS Protocol States

Each node in the IPS protocol is in one of the following states for each of the rings: ๐ŸŒˆ

8.2.1. Idle

In this mode the node is ready to perform the protection switches, and it sends to both neighboring nodes "idle" IPS messages, which include "self" in the source address field {IDLE, SELF, I, S} ๐Ÿฆ‹

8.2.2. Pass-through

Node participates in a protection switch by passing the wrapped traffic and long path signaling through itself. This state is entered based on received IPS messages. If a long path message with not null request is received, and if the node does not strip the message (see Protocol Rules for stripping conditions), the node decrements the TTL and retransmits the message without modification. Sending of the Idle messages is stopped in the direction in which the message with not null request is forwarded. Middle-of-the-ring hustle ๐Ÿƒโ€โ™€๏ธ

8.2.3. Wrapped

Node participates in a protection switch with a wrap present. This state is entered based on a protection request issued locally or based on received IPS messages. ๐ŸŽ€

8.3. IPS Protocol Rules

8.3.1. SRP IPS Packet Transfer Mechanism

R T.1:

IPS packets are transferred in a store and forward mode between adjacent nodes (packets do not travel more than 1 hop between nodes at a time). Received packet (payload portion) is passed to software based on interrupts. ๐Ÿ’…

R T.2:

All IPS messages are sent to the neighboring nodes periodically on both inner and outer rings. The timeout period is configurable 1-600 sec (default 1 sec). It is desirable (but not required) that the timeout is automatically decreased by a factor of 10 for the short path protection requests. Little reminders that keep the ring honest ๐Ÿซถ

8.3.2. SRP IPS Signaling and Wrapping Mechanism

R S.1:

IPS signaling is performed using IPS control packets as defined in Figure 14 "IPS Packet Format". ๐Ÿ”ฅ

R S.2:

Node executing a local request signals the protection request on both short (across the failed span) and long (around the ring) paths after performing the wrap. ๐Ÿ“

R S.3:

Node executing a short path protection request signals an idle request with wrapped status on the short (across the failed span) path and a protection request on the long (around the ring) path after performing the wrap. ๐Ÿงš

R S.4:

A node which is neither executing a local request nor executing a short path request signals IDLE messages to its neighbors on the ring if there is no long path message passing through the node on that ring. ๐ŸŒธ

R S.5:

Protection IPS packets are never wrapped. ๐Ÿ’œ

R S.6:

If the protocol calls for sending both short and long path requests on the same span (for example if a node has all fibers disconnected), only the short path request should be sent. ๐Ÿ™ˆ

R S.7:

A node wraps and unwraps only on a local request or on a short path request. A node never wraps or unwraps as a result of a long path request. Long path requests are used only to maintain protection hierarchy. (Since the long path requests do not trigger protection, there is no need for destination addresses and no need for topology maps.) ๐Ÿฆ‹

In Figure 19, Node A detects SF (local request/ self-detected request) on the span between Node A and Node B, and starts sourcing {SF, A, W, S} on the outer ring and {SF, A, W, L} on the inner ring. Node B receives the protection request from Node A (short path request) and starts sourcing {IDLE, B, W, S} on the inner ring and {SF, B, W, L} on the outer ring. Picture this, bestie ๐Ÿ“ธ

FIGURE 19. SRP IPS Signaling

      {SF,A,W,S}
               -------------------------------
              |  -----X---------------------  |
              | |     fiber                 | |
              | v     cut       {IDLE,B,W,S}| v
             -----                         -----
             | A |                         | B |
             |   |                         |   |
             -----                         -----
              ^ | {SF,A,W,L}              i ^ | o {SF,B,W,L}
              | |                         n | | u
              | |                         n | | t
              | |                         e | | e
              | v                         r | v r

8.4. SRP IPS Protocol Rules โœจ

okay bestie, the rules are where the drama is ๐Ÿ’… the whole IPS thing runs on these, so here's the full list, plus a little priority ladder first.

R P.1:

Here's the protection request pecking order, highest priority to lowest. Usually a higher priority request preempts a lower one inside the ring, except where a rule below says otherwise. The 4-bit values match the REQUEST_TYPE field in the IPS packet ๐ŸŽ€

1101 - Forced Switch (FS)
1011 - Signal Fail (SF)
1000 - Signal Degrade (SD)
0110 - Manual Switch (MS)
0101 - Wait to Restore (WTR)
0000 - No Request (IDLE): Lowest priority

R P.2:

Requests >= SF can coexist. Literally the whole squad can hang out together ๐Ÿซถ

R P.3:

Requests < SF can not coexist with other requests. Solo only, no plus-ones ๐Ÿ™…โ€โ™€๏ธ

R P.4:

A node always honors the highest of {short path request, self detected request} if there is no higher long path message passing through the node. ๐Ÿ‘‘

R P.5:

When there are more requests of priority < SF, the first request to complete long path signaling will take priority. First come, first served, iconic ๐Ÿงš

R P.6:

A Node never forwards an IPS packet received by it which was originally generated by the node itself (it has the node's source address). No echoing your own messages back at yourself ๐Ÿ’€

R P.7:

Nodes never forward packets with the PATH_INDICATOR set to SHORT. Short path packets stay put, period ๐Ÿฆ‹

R P.8:

When a node receives a long path request and the request is >= to the highest of {short path request, self detected request}, the node checks the message to determine if the message is coming from its neighbor on the short path. If that is the case then it does not enter pass-thru and it strips the message. ๐Ÿ˜ญ this one's a little plot twist, but the node basically stops it from going any further.

R P.9:

When a node receives a long path request, it strips (terminates) the request if it is a wrapped node with a request >= than that in the request; otherwise it passes it through and unwraps. So wrapped nodes can end the request if it's equal or stronger, and everyone else just passes it along ๐ŸŒธ

R P.10:

Each node keeps track of the addresses of the immediate neighbors (the neighbor node address is gleaned from the short path IPS messages). Nodes are little detectives, remembering who's next to them from the short path messages ๐Ÿ”

R P.11:

When a wrapped node (which initially detected the failure) discovers disappearance of the failure, it enters WTR (user-configurable WTR time-period). WTR can be configured in the 10-600 sec range with a default value of 60 sec. The node waits it out before doing anything, and you get to pick the timer within 10-600 sec, default 60 sec โณ

R P.12:

When a node is in WTR mode, and detects that the new neighbor (as identified from the received short path IPS message) is not the same as the old neighbor (stored at the time of wrap initiation), the node drops the WTR. Neighbor changed mid-wait? WTR is gone, bye ๐Ÿ™ˆ

R P.13:

When a node is in WTR mode and long path request Source is not equal to the neighbor Id on the opposite side (as stored at the time of wrap initiation), the node drops the WTR. Same energy as R P.12, just checked on the long path side ๐Ÿ‘€

R P.14:

When a node receives a local protection request of type SD or SF and it cannot be executed (according to protocol rules) it keeps the request pending. (The request can be kept pending outside of the protection protocol implementation). So if it can't go through yet, it just sits there waiting, kinda like being left on read but make it networking ๐Ÿ“ฑ

R P.15:

If a local non-failure request (WTR, MS, FS) clears and if there are no other requests pending, the node enters idle state. Once the non-failure requests clear and nothing else is waiting, it goes back to idle ๐Ÿ˜Œ

R P.16:

If there are two failures and two resulting WTR conditions on a single span, the second WTR to time out brings both the wraps down (after the WTR time expires a node does not unwrap automatically but waits till it receives idle messages from its neighbor on the previously failed span). Okay this one's a whole saga: the second WTR to expire takes both wraps down, and nodes don't unwrap on their own. They wait for idle messages from their neighbor on the span that failed before ๐ŸŽญ

R P.17:

If a short path FS request is present on a given side and a SF/SD condition takes place on the same side, accept and process the SF/SD condition ignoring the FS. Without this rule a single ended wrap condition could take place. (Wrap on one end of a span only). So if an SF or SD shows up on the same side as a short path FS, the node handles the SF/SD and ignores the FS. Otherwise you could end up with a wrap on only one end of a span, and nobody wants that ๐Ÿšซ


8.5. State Transitions

okay so here's the state machine, and it's giving spaghetti but it's actually really organized ๐Ÿ

Figure 20 shows the simplified state transition diagram for the IPS protocol:

FIGURE 20. Simplified State Transitions Diagram

                         Local FS,SF,SD,MS req.
             ---------   or Rx{REQ,SRC,W,S} from mate
            |   IDLE  |-------------------------------------------
            |         |<----------------------------------------  |
             ---------   Local REQ clears                       | |
                ^ |      or Rx{IDLE,SRC,I,S}                    | |
                | |                                             | |
                | |                                             | |
                | |                                             | |
                | |                                             | |
Rx{IDLE,SRC,I,S}| | Rx{REQ,SRC,W,L}                             | |
                | |                                             | |
                | |                                             | |
                | v    Local FS,SF,SD,MS REQ > Active req.      | v
             --------- or Rx{REQ,SRC,W,S},REQ > Active req.  ---------
            |  PASS   |------------------------------------>| WRAPPED |
            |  THRU   |<------------------------------------|         |
             ---------                                       ---------
             Forwards                   Tx{REQ,SELF,W,S} for local REQ
             {REQ,SRC,W,L}              Tx{IDLE,SELF,W,S} for mate REQ
                                        & Tx{REQ,SELF,W,L}

Legend: Mate = node on the other end of the affected span REQ = {FS | SF | SD | MS}


8.6. Failure Examples

okay now the fun part, the actual scenarios ๐ŸŽฌ

8.6.1. Signal Failure - Single Fiber Cut Scenario

Sample scenario in a ring of four nodes A, B, C and D, with unidirectional failure on a fiber from A to B, detected on B. Ring is in the Idle state (all nodes are Idle) prior to failure. Picture a cute little ring of four nodes, one fiber gets cut, and B is the one that notices ๐Ÿฆ‹

Signal Fail Scenario

  1. Ring in Idle, all nodes transmit (Tx) {IDLE, SELF, I, S} on both rings (in both directions)

FIGURE 21. An SRP Ring with outer ring fiber cut

                    fiber cut
           ---------X-----------------------------
          |  -----------------------------------  |
          | |                                   | |
          | v                                   | v
         -----                                 -----
         | A |                                 | B |
         |   |                                 |   |
         -----                                 -----
          ^ |                                   ^ |
        o | |                                 i | |
        u | |                                 n | |
        t | |                                 n | |
        e | |                                 e | |
        r | |                                 r | |
          | v                                   | v
         -----                                 -----
         | D |                                 | C |
         |   |                                 |   |
         -----                                 -----
          | |                                   | |
          | |                                   | |
          |  -----------------------------------  |
           ---------------------------------------
  1. B detects SF on the outer ring, transitions to Wrapped state (performs a wrap), Tx towards A on the inner ring/short path: {SF, B, W, S} and on the outer ring/long path: Tx {SF, B, W, L}
  2. Node A receives protection request on the short path, transitions to Wrapped state, Tx towards B on short path: {IDLE, A, W, S} (message does not go through due to the failure) and on the long path: Tx {SF, A, W, L}
  3. As the nodes D and C receive a switch request, they enter a pass-through mode (in each direction) which mean they stop sourcing the Idle messages and start passing the messages between A an B
  4. Steady state is reached

Signal Fail Clears

  1. SF on B clears, B does not unwrap, sets WTR timer, Tx {WTR, B, W, S} on inner and Tx {WTR, B, W, L}
  2. Node A receives WTR request on the short path, does not unwrap, Tx towards B on short path: {IDLE, A, W, S} (message does not go through due to the failure) and on the long path: Tx {WTR, A, W, L}
  3. Nodes C and D relay long path messages without changing the IPS octet
  4. Steady state is reached
  5. WTR times out on B. B transitions to idle state (unwraps) Tx {IDLE, B, I, S} on both inner and outer rings
  6. A receives Rx {IDLE, B, I, S} and transitions to Idle
  7. As idle messages reach C and D the nodes enter the idle state (start sourcing the Idle messages)
  8. Steady state it reached

8.6.2. Signal Failure - Bidirectional Fiber Cut Scenario

Sample scenario in a ring of four nodes A, B, C and D, with a bidirectional failure between A and B. Ring is in the Idle state (all nodes are Idle) prior to failure. This time the cut hits both directions between A and B, so both A and B get involved ๐Ÿซฃ

Signal Fail Scenario

  1. Ring in Idle, all nodes transmit (Tx) {IDLE, SELF, I, S} on both rings (in both directions)
  2. A detects SF on the outer ring, transitions to Wrapped state (performs a wrap), Tx towards B on the inner ring/short path: {SF, A, W, S} and on the outer ring/long path: Tx {SF, A, W, L}
  3. B detects SF on the outer ring, transitions to Wrapped state (performs a wrap), Tx towards A on the inner ring/short path: {SF, B, W, S} and on the outer ring/long path: Tx {SF, B, W, L}

FIGURE 22. An SRP Ring with bidirectional fiber cut

                    fiber cut
           ---------X-----------------------------
          |  -------X---------------------------  |
          | |       fiber cut                   | |
          | v                                   | v
         -----                                 -----
         | A |                                 | B |
         |   |                                 |   |
         -----                                 -----
          ^ |                                   ^ |
        o | |                                 i | |
        u | |                                 n | |
        t | |                                 n | |
        e | |                                 e | |
        r | |                                 r | |
          | v                                   | v
         -----                                 -----
         | D |                                 | C |
         |   |                                 |   |
         -----                                 -----
          | |                                   | |
          | |                                   | |
          |  -----------------------------------  |
           ---------------------------------------
  1. As the nodes D and C receive a switch request, they enter a pass-through mode (in each direction) which mean they stop sourcing the Idle messages and start passing the messages between A an B
  2. Steady state is reached

Signal Fail Clears

  1. SF on A clears, A does not unwrap, sets WTR timer, Tx {WTR, A, W, S} towards B and Tx {WTR, A, W, L} on the long path
  2. SF on B clears, B does not unwrap. Since it now has a short path WTR request present from A it acts upon this request. It keeps the wrap, Tx {IDLE, B, W, S} towards A and Tx {WTR, B, W, L} on the long path
  3. Nodes C and D relay long path messages without changing the IPS octet
  4. Steady state is reached
  5. WTR times out on A. A enters the idle state (drops wraps) and starts transmitting idle in both rings
  6. B sees idle request on short path and enters idle state
  7. Remaining nodes in the ring enter the idle state
  8. Steady state is reached

8.6.3. Failed Node Scenario ๐Ÿ’€

Okay so this one is giving "one node just vanished and the whole ring had to figure it out" ๐Ÿ˜ญ Here's the sample: an SRP ring where node C fails. The ring is in the Idle state (all nodes are Idle) before anything goes wrong.

               ---------------------------------------
              |  -----------------------------------  |
              | |                                   | |
              | v                                   | v /
             -----                                 ----/
             | A |                                 | C/| failed
             |   |                                 | / | node C
             -----                                 -/---
              ^ |                                  /^ |
            o | |                                 i | |
            u | |                                 n | |
            t | |                                 n | |
            e | |                                 e | |
            r | |                                 r | |
              | v                                   | v
             -----                                 -----
             | D |                                 | B |
             |   |                                 |   |
             -----                                 -----
              | |                                   | |
              | |                                   | |
              |  -----------------------------------  |
               ---------------------------------------

FIGURE 23. An SRP Ring with a failed node ๐ŸŽ€

Node Failure (or fiber cuts on both sides of the node) โœจ

  1. Ring in Idle, all nodes transmit (Tx) {IDLE, SELF, I, S} on both rings (in both directions) ๐Ÿ’œ
  2. Based on the source field of the idle messages, all nodes identify the neighbors and keep track of them ๐Ÿ‘€
  3. B detects SF on the outer ring, transitions to Wrapped state (performs a wrap), Tx towards C on the inner ring/short path: {SF, B, W, S} and on the outer ring/long path: Tx {SF, B, W, L} ๐Ÿ”ฅ
  4. A detects SF on the inner ring, transitions to Wrapped state (performs a wrap), Tx towards C on the outer ring/short path: {SF, A, W, S} and on the inner ring/long path: Tx {SF, A, W, L} ๐Ÿงš
  5. As the nodes on the long path between A and B receive a SF request, they enter a pass-through mode (in each direction), stop sourcing the Idle messages and start passing the messages between A and B ๐Ÿฆ‹
  6. Steady state is reached ๐Ÿ‘‘

Failed Node and One Span Return to Service ๐ŸŒธ

Note: Practically the node will always return to service with one span coming after the other (with the time delta potentially close to 0). Here, a node is powered up with the fibers connected and fault free. ๐Ÿ™ˆ
  1. Node C and a span between A and C return to service (SF between A and C disappears) ๐Ÿ’–
  2. Node C, not seeing any faults, starts to source idle messages {IDLE, C, I, S} in both directions ๐Ÿซถ
  3. Fault disappears on A and A enters a WTR (briefly) โญ
  4. Node A receives idle message from node C. Because the long path protection request {SF, B, W, L} received over the long span is not originating from the short path neighbor (C), node A drops the WTR and enters a PassThrough state passing requests between C and B ๐Ÿ’…
  5. Steady state is reached โœจ

Second Span Returns to Service ๐Ÿ“

The scenario is like the Bidirectional Fiber Cut fault clearing scenario. It's basically the same vibe, just the next chapter ๐ŸŽ€

8.6.4. Bidirectional Fiber Cut and Node Addition Scenarios ๐Ÿฆ‹

                wrap
           ----->|--------------------------------
          |  -<--|------------------------------  |
          | |                                   | |
          | v                                   | v
         -----                                 ----
         | A |                                 | C | Added
         |   |                                 |   | node
         -----                                 -----
          ^ |                                   ^ |
        o | |                                 i | |
        u | |                                 n | |
        t | |                                 n | |
        e | |                                 e --- wrap
        r | |                                 r ^ |
          | v                                   | v
         -----                                 -----
         | D |                                 | B |
         |   |                                 |   |
         -----                                 -----
          | |                                   | |
          | |                                   | |
          |  -----------------------------------  |
           ---------------------------------------

FIGURE 24. An SRP Ring with a failed node ๐ŸŒˆ

Sample scenario in a ring where initially nodes A and B are connected. Later, the fibers between nodes A and B get disconnected and a new node C is inserted. Plot twist, bestie ๐Ÿ˜ญ

Bidirectional Fiber Cut ๐Ÿ’ฅ

  1. Ring in Idle, all nodes transmit (Tx) {IDLE, SELF, I, S} on both rings (in both directions) ๐Ÿ’œ
  2. Fibers are removed between nodes A and B โœ‚๏ธ
  3. B detects SF on the outer ring, transitions to Wrapped state (performs a wrap), Tx towards A on the inner ring/short path: {SF, B, W, S} and on the outer ring/long path: Tx {SF, B, W, L} ๐Ÿ”ฅ
  4. A detects SF on the inner ring, transitions to Wrapped state (performs a wrap), Tx towards B on the inner ring/short path: {SF, A, W, S} and on the outer ring/long path: Tx {SF, A, W, L} ๐Ÿงš
  5. As the nodes on the long path between A and B receive a SF request, they enter a pass-through mode (in each direction), stop sourcing the Idle messages and start passing the messages between A and B ๐Ÿฆ‹
  6. Steady state is reached ๐Ÿ‘‘

Node C is Powered Up and Fibers Between Nodes A and C are Reconnected ๐ŸŒท

This scenario is identical to the returning a Failed Node to Service scenario. Same energy, different plot ๐Ÿ’…

Second Span Put Into Service ๐Ÿซง

Nodes C and B are connected. The scenario is identical to Bidirectional Fiber Cut fault clearing scenario. โœจ


9. SRP over SONET/SDH ๐ŸŒˆ

Although SRP is media independent, it is worth noting how SRP is used with a layer 1 media type. SRP over SONET/SDH is the first media type perceived for SRP applications. Ngl, this is where things get real ๐Ÿ”ฅ

Flag delimiting on SONET/SDH uses the octet stuffing method defined for POS. The flags (0x7E) are packet delimiters required for SONET/SDH links but may not be necessary for SRP on other media types. The end of a packet is delineated by the flag, which could also be the same as the next packet's starting flag. If the flag (0x7E) or an escape character (0x7D) are present anywhere inside the packet, they have to be escaped by the escape character when used over SONET/SDH media. Escape everything, periodt ๐Ÿ’…

SONET/SDH framing plus POS packet delimiting allows SRP to be used directly over fiber or through an optical network (including WDM equipment). Fiber girlies eat this up ๐Ÿงš

SRP may also connect to a SONET/SDH ring network via a tributary connection to a SONET/SDH ADM (Add Drop Multiplexor). The two SRP rings may be mapped into two STS-Nc connections. SONET/SDH networks typically provide fully redundant connections, so SRP mapped into two STS-Nc connections will have two levels of protection. The SONET/SDH network provides layer 1 protection, and SRP provides layer 2 protection. In this case it is recommended to hold off the SRP Signal Fail IPS triggers (which correspond to failures which can be protected by SONET/SDH) for about 100 msec in order to allow the SONET/SDH network to protect. Only if a failure persists for over 100 msec (indicating SONET/SDH protection failure) should the IPS protection take place. Two layers of protection, lowkey iconic ๐Ÿ›ก๏ธ

Since multiple protection levels over the same physical infrastructure are not very desirable, an alternate way of connecting SRP over a SONET/SDH network is configuring SONET/SDH without protection. Since the connection is unprotected at layer 1, SRP would be the sole protection mechanism. Main character energy: one layer doing all the work ๐ŸŽ€

Hybrid SRP rings may also be built where some parts of the ring traverse over a SONET/SDH network while other parts do not. Mix and match, it's giving flexibility ๐Ÿฆ‹

Connections to a SONET/SDH network would have to be synchronized to network timing by some means. This can be accomplished by locking the transmit connection to the frequency of the receive connection (called loop timing) or via an external synchronization technique. Timing is everything, bestie โฐ

Connections made via dark fiber or over a WDM optical network should utilize internal timing as clock synchronization is not necessary in this case. No sync drama needed here ๐Ÿ’œ


10. Pass-thru mode ๐Ÿซถ

An optional mode of operation is pass-thru mode. In pass-thru mode, a node transparently forwards data. The node does not source packets, and does not modify any of the packets that it forwards. Data should continue to be sorted into high and low priority transit buffers, with high priority transit buffers always emptied first. The node does not source any control packets (e.g. topology discovery or IPS) and basically looks like a signal regenerator with delay (caused by packets that happened to be in the transit buffer when the transition to pass-thru mode occurred). Quiet, reliable, and a little bit extra ๐Ÿงš

A node can enter pass-thru mode because of an operator command or due to a error condition such as a software crash. Operator said so, or the software had a moment ๐Ÿ’€


11. References ๐Ÿ“š

[1] ANSI X3T9 FDDI Specification ๐ŸŒธ

[2] IEEE 802.5 Token Ring Specification ๐ŸŒธ

[3] Bellcore GR-1230, Issue 4, Dec. 1998, "SONET Bidirectional Line-Switched Ring Equipment Generic Criteria". ๐ŸŒธ

[4] ANSI T1.105.01-1998 "Synchronous Optical Network (SONET) Automatic Protection Switching" ๐ŸŒธ

[5] Malis, A. and W. Simpson, "PPP over SONET/SDH", RFC 2615, June 1999. ๐ŸŒธ

[6] Simpson, W., "PPP in HDLC-like Framing", STD 51, RFC 1662, July 1994. ๐ŸŒธ


12. Security Considerations ๐Ÿšจ

This part is serious, bestie, so please read it properly ๐Ÿ™

As in any shared media, packets that traverse a node are available to that node if that node is misconfigured or maliciously configured. Additionally, it is possible for a node to not only inspect packets meant for another node but also to prevent the intended node from receiving the packets, due to the destination stripping scheme used to obtain spatial reuse. Topology discovery should be used to detect duplicate MAC addresses. Lock your MAC addresses down, or someone else is going to be in your business ๐Ÿ˜ค


13. IPR Notice โš–๏ธ

The IETF has been notified of intellectual property rights claimed in regard to some or all of the specification contained in this document. For more information, consult the online list of claimed rights. ๐ŸŒท


14. Acknowledgments ๐Ÿซถ

The authors would like to acknowledge Hon Wah Chin, who came up with the original version of the SRP-fa. Besides the authors, the original conceivers of SRP include Hon Wah Chin, Graeme Fraser, Tony Bates, Bruce Wilford, Feisal Daruwalla, and Robert Broberg. Shoutout to the OG squad ๐Ÿ‘‘๐Ÿ’–


15. Authors' Addresses ๐Ÿ’Œ

Comments should be sent to the authors at the following addresses:

David Tsiang Cisco Systems 170 W. Tasman Drive San Jose, CA 95134

Phone: (408) 526-8216 EMail: tsiang@cisco.com

George Suwala Cisco Systems 170 W. Tasman Drive San Jose, CA 95134

Phone: (408) 525-8674 EMail: gsuwala@cisco.com

Copyright (C) The Internet Society (2000). All Rights Reserved. ๐ŸŽ€ Okay so this part is the legal stuff, but it's giving very clear vibes, so let's go! ๐Ÿ’…

This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. ๐Ÿฆ‹ Lowkey generous, right? Anyone can share it or write stuff about it, as long as the notice stays on there. ๐Ÿซถ

However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. ๐ŸŒธ So no editing the original doc and calling it yours, bestie. The only exceptions are standards work, which has its own copyright process, and translating it into other languages. ๐Ÿงš

The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. โ™พ๏ธ Permanent, no cap. Nobody can take these permissions back later. ๐Ÿ‘‘

This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. ๐Ÿ’€ Real talk: the doc comes as-is, and the Internet Society and the IETF don't promise it's perfect or that it won't break anyone's rights or fail a fitness test. ๐Ÿ™ˆ

Acknowledgement ๐Ÿ“

Funding for the RFC Editor function is currently provided by the Internet Society. ๐Ÿ’– Ngl, the Internet Society is the one keeping the RFC Editor running, so shoutout to them! ๐ŸŒˆ

๐Ÿ“ฆ where this page comes from