RFC 7172

Transparent Interconnection of Lots of Links (TRILL): Fine-Grained Labeling

📅 published May 2014, from ietf

RFC 7172 🎀 is a Standards Track doc from May 2014 (ISSN 2070-1721), written by D. Eastlake 3rd, M. Zhang (Huawei), P. Agarwal (Broadcom), R. Perlman (Intel Labs), and D. Dutt (Cumulus Networks). It updates RFC 6325 💅 and doesn't obsolete anything.

Abstract ✨

okay so the IETF already standardized TRILL 🦋, a protocol for least-cost transparent frame routing in multi-hop networks with any topology and any link tech, using link-state routing and a hop count. the base TRILL spec lets you label TRILL Data packets with up to 4K IDs, but some apps need way more labels for configurable data isolation 😭. this doc updates RFC 6325 by adding optional extensions to the base protocol so that can happen safely. these extensions are called fine-grained labeling and they're mainly meant for big data centers, the ones with more than 4K users who need configurable data isolation from each other 🌸

Status of This Memo 🌈

this is an Internet Standards Track document.

this doc is a product of the Internet Engineering Task Force (IETF) 💜. it represents the consensus of the IETF community, got public review, and was approved for publication by the Internet Engineering Steering Group (IESG). more info on Internet Standards is in Section 2 of RFC 5741.

info about the current status of this doc, any errata, and how to give feedback is at http://www.rfc-editor.org/info/rfc7172. go check it out 🍓

Copyright (c) 2014 IETF Trust and the persons identified as the document authors. All rights reserved.

this doc is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as they were on the date this doc was published. please read them carefully, because they explain your rights and restrictions 🫶 code components pulled out of this doc must include Simplified BSD License text (as described in Section 4.e of the Trust Legal Provisions) and come without warranty, as described in the Simplified BSD License.

Table of Contents 🗂️

   1. Introduction ....................................................4
      1.1. Terminology ................................................5
      1.2. Contributors ...............................................5
   2. Fine-Grained Labeling ...........................................5
      2.1. Goals ......................................................6
      2.2. Base Protocol TRILL Data Labeling ..........................7
      2.3. Fine-Grained Labeling (FGL) ................................7
      2.4. Reasons for VL and FGL Coexistence .........................9
   3. VL versus FGL Label Differences ................................10
   4. FGL Processing .................................................11
      4.1. Ingress Processing ........................................11
           4.1.1. Multi-Destination FGL Ingress ......................11
      4.2. Transit Processing ........................................12
           4.2.1. Unicast Transit Processing .........................12
           4.2.2. Multi-Destination Transit Processing ...............12
      4.3. Egress Processing .........................................13
      4.4. Appointed Forwarders and the DRB ..........................14
      4.5. Distribution Tree Construction ............................14
      4.6. Address Learning ..........................................15
      4.7. ESADI Extension ...........................................15
   5. FGL TRILL Interaction with VL TRILL ............................15
      5.1. FGL and VL Mixed Campus ...................................15
      5.2. FGL and VL Mixed Links ....................................17
      5.3. Summary of FGL-Safe Requirements ..........................18
   6. IS-IS Extensions ...............................................19
   7. Comparison with Goals ..........................................19
   8. Allocation Considerations ......................................20
      8.1. IEEE Allocation Considerations ............................20
      8.2. IANA Considerations .......................................20
   9. Security Considerations ........................................20
   Appendix A. Serial Unicast ........................................22
   Appendix B. Mixed Campus Characteristics ..........................23
      B.1. Mixed Campus with High Cost Adjacencies ...................23
      B.2. Mixed Campus with Data Blocked Adjacencies ................24
   Acknowledgements ..................................................25
   References ........................................................25
      Normative References ...........................................25
      Informative References .........................................26

1. Introduction 🧚

the IETF standardized TRILL [RFC6325] 🦋, which gives you least-cost transparent routing in multi-hop networks with arbitrary topologies and link technologies. it does this using [IS-IS] [RFC6165] [RFC7176] link-state routing and a hop count. TRILL switches are sometimes called RBridges (Routing Bridges) ✨

the base TRILL protocol supports labeling TRILL Data packets with up to 4K IDs. but some apps need a lot more data labels for configurable isolation based on different tenants, service instances, or stuff like that 👑 this doc updates [RFC6325] by adding optional extensions to the base TRILL protocol so that works safely. these extensions are called fine-grained labeling, and they're mainly for big data centers, the ones with more than 4K users who need configurable data isolation from each other 💅

this doc describes a format that lets you have a 24-bit data label, known as a "fine-grained label", or FGL 💖. it also covers how to coexist and migrate from current RBridges, known as "VL" (for "VLAN Labeled") RBridges, over to TRILL switches that can support FGL ("Fine-Grained Labeled") packets. here's the tea: various VL implementations might handle FGL packets wrong, so FGL packets can't be introduced until either (1) every VL RBridge is upgraded to what we'll call "FGL-safe", meaning they won't "do anything bad" with FGL packets, or (2) every FGL RBridge takes special precautions on any port where it's connected to a VL RBridge. FGL-safe requirements are summarized in Section 5.3 🎀

it's hoped that a lot of RBridges can become FGL-safe with just a software upgrade 🌸. VL RBridges and FGL-safe RBridges can coexist without any disruption to service, as long as no FGL packets get introduced.

if every RBridge is upgraded to FGL-safe, FGL traffic can be handled by the campus with no topology restrictions at all 🔥. every FGL RBridge knows FGL traffic exists because some RBridge (say, RB3) that might source or sink FGL traffic will advertise interest in one or more fine-grained labels in its contribution to the link state (its LSP). and if any VL RBridges are still around when an RBridge announces that it might source or sink FGL traffic, the adjacent FGL-safe RBridges MUST ensure that no FGL packets are forwarded to their VL RBridge neighbor(s). the details are in Section 5.1 below 🙈

1.1. Terminology 🌷

the terminology and acronyms of [RFC6325] are used in this doc, plus the additions listed below 🫶

  • DEI - Drop Eligibility Indicator [802.1Q].
  • FGL - Fine-Grained Labeling or Fine-Grained Labeled or Fine-Grained Label.
  • FGL-edge - An FGL TRILL switch advertising interest in an FGL label.
  • FGL link - A link where all of the attached TRILL switches are FGL.
  • FGL-safe - A TRILL switch that can safely be given an FGL data packet, as summarized in Section 5.3.
  • RBridge - Alternative name for a TRILL switch.
  • TRILL switch - Alternative name for an RBridge.
  • VL - VLAN Labeling or VLAN Labeled or VLAN Label.
  • VL link - A link where any one or more of the attached RBridges are VL.
  • VL RBridge - A TRILL switch that supports VL but is not FGL-safe.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119] 📚

1.2. Contributors 💜

thanks a bunch to these lovelies for their contributions: Tissa Senevirathne and Jon Hudson 🥰

2. Fine-Grained Labeling 💖

the essence of Fine-Grained Labeling (FGL) is this: (a) when frames are ingressed or created, they can carry a data label from a set with way more than 4K labels, (b) TRILL switch ports can be labeled with a set of those fine-grained data labels, and (c) an FGL TRILL Data packet can't be egressed through a TRILL switch port unless its fine-grained label (FGL) matches one of the port's data labels 👑

Section 2.1 lists the FGL goals ✨. Section 2.2 briefly outlines the more coarse TRILL base protocol standard [RFC6325] data labeling 🦋. Section 2.3 outlines FGL for TRILL Data packets 🌸. Section 2.4 talks about VL and FGL coexistence 🎀

(part 1 of 6, more on the way 💅✨)

2.1. Goals ✨

okay so bestie, FGL TRILL has a few goals, and I'm listing them in rough priority order, most important first 💅

  1. Fine-Grained 🦋 Some networks have a huge number of entities that need their own configurable isolation. Those could be independent customers, applications, or branches of one big project, or some mix of all of those. The labeling that [RFC6325] gives you only has 2**12 - 2 valid identifiers or labels (VLANs), and that's just not enough. We need a MUCH bigger number 😭
  2. Silicon 🔥 Fine-grained labeling (FGL) should, to the extent practical, reuse existing features, processing, and fields that a lot of fast path silicon already supports for the TRILL base protocol. Why reinvent the wheel when the chips already know how to spin it? 🧚
  3. Base RBridge Interoperation 🫶 In some incremental conversion scenarios, it's desirable that not every RBridge in a campus using FGL be required to be FGL aware. So it's a good thing if RBridges that don't implement the FGL features can still exchange VL TRILL Data packets with FGL TRILL switches. Everyone can hang out together 🌈
  4. Alternate Priority 💜 Sometimes it would be desirable for traffic coming from an attached non-TRILL network to be handled, while it's transiting a TRILL network, with a different priority than the priority of the original native frames. The ingress TRILL switch could do this by assigning a different priority to the FGL TRILL Data packet that results from ingressing the native frames. The original priority should be restored on egress. Main character priority shuffle, but make it reversible 🎀

2.2. Base Protocol TRILL Data Labeling 🌸

this section is a quick refresher on the [RFC6325] TRILL Data packet VL Labeling, and it also shifts where the TRILL Header is said to end. Real talk: this change in the description doesn't change any bits on the wire, and it doesn't change how VL TRILL switches behave. Just a vibe check on the paperwork 📝

VL TRILL Data packets look like this:

           +-------------------------------------------+
           | Link Header (depends on link technology)  |
           |  (if link is an Ethernet link, the link   |
           |  header may include an Outer.VLAN tag)    |
           +-------------------------------------------+
           | TRILL Header                              |
           | +---------------------------------------+ |
           | |    Initial Fields and Options         | |
           | +---------------------------------------+ |
           | |         Inner.MacDA         | (6 bytes) |
           | +-----------------------------+           |
           | |         Inner.MacSA         | (6 bytes) |
           | +-----------------------+-----+           |
           | | Ethertype 0x8100      |       (2 bytes) |
           | +-----------------------+                 |
           | | Inner.VLAN Label      |       (2 bytes) |
           | +-----------------------+                 |
           +-------------------------------------------+
           |               Native Payload              |
           +-------------------------------------------+
           | Link Trailer (depends on link technology) |
           +-------------------------------------------+

                   Figure 1: TRILL Data with VL

In the base protocol from [RFC6325], the 0x8100 value is always there, and it's followed by the Inner.VLAN field, which holds the 12-bit VL. Simple and classic 🫶

2.3. Fine-Grained Labeling (FGL) 🦋

FGL expands the variety of data labels you can use under the TRILL protocol. It adds a fine-grained label (FGL) made of a 12-bit high order part and a 12-bit low order part. In this document, FGLs are written as (X.Y), where X is the high order part and Y is the low order part of the FGL. Think of it like an area code plus a phone number, but for packets 📞✨

FGL TRILL Data packets look like this:

           +-------------------------------------------+
           | Link Header (depends on link technology)  |
           |  (if link is an Ethernet link, the link   |
           |  header may include an Outer.VLAN tag)    |
           +-------------------------------------------+
           | TRILL Header                              |
           | +---------------------------------------+ |
           | |    Initial Fields and Options         | |
           | +---------------------------------------+ |
           | |         Inner.MacDA         | (6 bytes) |
           | +-----------------------------+           |
           | |         Inner.MacSA         | (6 bytes) |
           | +-----------------------+-----+           |
           | | Ethertype 0x893B      |       (2 bytes) |
           | +-----------------------+                 |
           | | Inner.Label High Part |       (2 bytes) |
           | +-----------------------+                 |
           | | Ethertype 0x893B      |       (2 bytes) |
           | +-----------------------+                 |
           | | Inner.Label Low Part  |       (2 bytes) |
           | +-----------------------+                 |
           +-------------------------------------------+
           |               Native Payload              |
           +-------------------------------------------+
           | Link Trailer (depends on link technology) |
           +-------------------------------------------+

                   Figure 2: TRILL Data with FGL

So after the inner MAC addresses, the FGL info is marked by 0x893B. There MUST be two occurrences of 0x893B, as shown. 🚨 If a TRILL switch processing an FGL TRILL Data packet notices that the second occurrence is actually some other value, it MUST discard the packet. Not vibing, not forwarding 🗑️

Quick aside: a TRILL switch that's just transiting a TRILL Data packet doesn't have to look at any fields past the initial fixed fields and options. It's allowed to peek further if it wants to, for Equal-Cost Multi-Path (ECMP) or distribution tree pruning, but it doesn't have to 👀

The two bytes after each 0x893B carry the fine-grained label information in their low order 12 bits. The upper 4 bits of those two bytes hold a 3-bit priority field and one Drop Eligibility Indicator (DEI) bit, like this:

           0  1  2   3  4  5  6  7  8  9 10 11 12 13 14 15
         +--+--+--+---+--+--+--+--+--+--+--+--+--+--+--+--+
         |priority|DEI|    label information              |
         +--+--+--+---+--+--+--+--+--+--+--+--+--+--+--+--+

                 Figure 3: FGL Part Data Structure

The priority field in the Inner.Label High Part is the priority used for moving the frame across the TRILL campus, from ingress to egress. The label bits in the Inner.Label High Part are the high order part of the FGL, and the label bits in the Inner.Label Low Part are the low order part. The priority field in the Inner.Label Low Part is remembered from the data frame as it came in, and it gets restored on egress. Tiny bit of packet memory, big glow-up 🌟

The right FGL value for an ingressed or locally originated native frame is decided by the ingress TRILL switch port, as specified in Section 4.1.

2.4. Reasons for VL and FGL Coexistence 💖

there are a few solid reasons FGL TRILL switches need to handle both FGL and VL TRILL Data packets, so here they are 🌸

  • Keeping VL support means that, if you follow the precautions specified in this document, you can often still have VL TRILL switches easily swapping VL packets through a core of FGL TRILL switches. Friends across the group chat 🫶
  • Because of the way TRILL works, you might want a maintenance VLAN or FGL [RFC7174] that every TRILL switch in the campus shows interest in. It's simpler to use the same label type for all of them for this, and that means using VL if there might be any VL TRILL switches in the campus. 🔧
  • If a campus is being upgraded from VL to FGL, continuing to support VL lets edges labeled as VL keep working for the long haul 🕰️

3. VL versus FGL Label Differences 🌈

the meaning of TRILL Data packets across a campus is different depending on whether they're labeled with VL or FGL, and this section is all about that tea ☕

With VL, a data label ID means the same thing everywhere in the campus. Those IDs come from the same label space as the C-VLAN IDs used on Ethernet links to end stations.

The FGL data label space is a totally different space from the VL data label space. 💅 For ports configured for FGL, the C-VLAN on an ingressed native frame gets stripped and mapped into the FGL data label space, and that mapping can be different on each port. A similar FGL-to-C-VLAN mapping happens per port on egress. So, for ports configured for FGL, the native frame C-VLAN on one link that corresponds to an FGL can be different from the native frame C-VLAN that corresponds to that same FGL on a different link somewhere else in the campus, or even on a different link attached to the same TRILL switch. Wild, I know 🙈 The FGL label space is flat, and it doesn't hierarchically encode any particular number of native frame C-VLAN bits or anything like that. FGLs only show up inside TRILL Data packets, after the inner MAC addresses.

It's the network manager's job to configure the TRILL switches in the campus properly so you get the mappings you want. In lots of cases this can be automatic, based on configuration databases and orchestration systems. Set it and forget it, bestie 🤖

With FGL TRILL switches, a lot of things stay the same, because an FGL can only appear as the Inner.Label inside a TRILL Data packet. So only TRILL-aware devices will ever see a fine-grained label 👀 The Outer.VLAN that can appear on native frames, and that can appear on TRILL Data packets when they're on an Ethernet link, can only be a C-VLAN tag. So the ports of FGL TRILL switches, up through the usual VLAN and priority processing, act just like they do for VL TRILL switches. TRILL switch ports give you a C-VLAN ID for an incoming frame, and they accept a C-VLAN ID for a frame that's getting queued for output. Appointed Forwarders [RFC6439] on a link are still appointed for a C-VLAN, and the Designated VLAN for an Ethernet link is still a C-VLAN. Consistency is iconic 💜

FGL TRILL switches have a superset of the capabilities that VL TRILL switches have. FGL TRILL switch ports can be configured for either FGL or VL, and VL is the default. Just like a base protocol [RFC6325] TRILL switch, an unconfigured FGL TRILL switch port reports an untagged frame it receives as being in VLAN 1 🎀

4. FGL Processing ✨

this section spells out the ingress, transit, egress, and other processing details for FGL TRILL switches, so pay attention 🧐 A transit or egress FGL TRILL switch can tell that a TRILL Data packet is FGL by spotting 0x893B right after the Inner.MacSA. Bestie, that's the giveaway 🔥

4.1. Ingress Processing 🎀

okay so FGL-edge TRILL switch ports can be configured to ingress native frames as FGL ✨ Any port that isn't set up that way does the old [RFC6325] VL ingress processing on native frames, which gives you a VL TRILL Data packet 💅 (There is no change in Appointed Forwarder logic, see Section 4.4.) An FGL-safe TRILL switch may have only VL ports, in which case it is not required to support the FGL ingress capabilities in this section. no cap, it can skip all that 🦋

FGL-edge TRILL switches support configurable per-port mapping from the C-VLAN of a native frame, as reported by the ingress port, to an FGL 🌸 FGL TRILL switches MAY support other methods to determine the FGL of an incoming native frame, such as methods based on the protocol of the native frame or based on local knowledge. the vibes are very flexible here 🧚

The FGL ingress process MUST copy the priority and DEI (Drop Eligibility Indicator) associated with an ingressed native frame to the upper 4 bits of the Inner.Label Low Order part 💖 It SHOULD also associate a possibly different mapped priority and DEI with an ingressed frame, but a TRILL switch might not be able to do so because of implementation limitations 🙈 The mapped priority is placed in the Inner.Label High Part. If such mapping is not supported, then the original priority and DEI MUST be placed in the Inner.Label High Part. bestie, that's the whole plan ✨

4.1.1. Multi-Destination FGL Ingress 🌈

If a native frame that has a broadcast, multicast, or unknown MAC destination address is FGL ingressed, it MUST be handled in one of the following two ways 👑 The choice of which method to use can vary from frame to frame, at the choice of the ingress TRILL switch.

  1. Ingress as a TRILL multi-destination data packet (TRILL Header M bit = 1) on a distribution tree rooted at a nickname held by an FGL RBridge or by the pseudonode of an FGL link. FGL TRILL Data packets MUST NOT be sent on a tree rooted at a nickname held by a VL TRILL switch or by the pseudonode of a VL link. absolutely not, we do not mix those trees 💀
  2. Serially TRILL unicast the ingressed frame to the relevant egress TRILL switches by using a known unicast TRILL Header (M bit = 0). An FGL ingress TRILL switch SHOULD unicast a multi-destination TRILL Data packet if there is only one relevant egress FGL TRILL switch. The relevant egress TRILL switches are determined by starting with those announcing interest in the frame's (X.Y) label. That set SHOULD be further filtered based on multicast listener and multicast router attachment LSP announcements if the native frame was a multicast frame. so many filters, so much main character energy 🔥

Using a TRILL unicast header for a multi-destination frame when it has only one actual destination RBridge almost always improves traffic spreading and decreases latency as discussed in Appendix A 🍓 How to decide whether to use a distribution tree or serial unicast for a multi-destination TRILL Data packet that has more than one destination TRILL switch is beyond the scope of this document. that one's a whole other story, not this doc 🫶

4.2. Transit Processing 🦋

Any FGL TRILL switch MUST be capable of TRILL Data packet transit processing 💜 Such processing is fairly straightforward as described in Section 4.2.1 for known unicast TRILL Data packets and in Section 4.2.2 for multi-destination TRILL Data packets. easy mode, literally ✨

4.2.1. Unicast Transit Processing 💅

There is very little change in TRILL Data packet unicast transit processing 🌸 A transit TRILL switch forwards any unicast TRILL Data packet to the next hop towards the egress TRILL switch as specified in the TRILL Header. All transit TRILL switches MUST take the priority and DEI used to forward a packet from the Inner.VLAN label or the FGL Inner.Label High Part. These bits are in the same place in the packet, which is so iconic of them 🎀

An FGL TRILL switch MUST properly distinguish flows if it provides ECMP for unicast FGL TRILL Data packets. flows have to be sorted properly, no exceptions 🙈

4.2.2. Multi-Destination Transit Processing 🧚

Multi-destination TRILL Data packets are forwarded on a distribution tree selected by the ingress TRILL switch, except that an FGL ingress TRILL switch MAY TRILL unicast such a frame to all relevant egress TRILL switches, all as described in Section 4.1. The distribution trees do not distinguish between FGL and VL multi-destination packets, except in pruning behavior if they provide pruning. There is no change in the Reverse Path Forwarding Check. the trees don't care which flavour it is, lowkey iconic 🌈

An FGL TRILL switch (say, RB1) having an FGL multi-destination frame for label (X.Y) to forward on a distribution tree SHOULD prune that tree based on whether there are any TRILL switches on a tree branch that are advertising connectivity to label (X.Y). In addition, RB1 SHOULD prune multicast frames based on reported multicast listener and multicast router attachment in (X.Y). RB1 really said "pruning go brrr" 💖

Pruning is an optimization. If a transit TRILL switch does less pruning than it could, there may be greater link utilization than strictly necessary but the campus will still operate correctly 🫶 A transit TRILL switch MAY prune based on an arbitrary subset of the bits in the FGL label, for example, only the High Part or only the Low Part of the label. the campus still works, so no stress 💜

4.3. Egress Processing 👑

Egress processing is generally the reverse of ingress progressing described in Section 4.1 ✨ An FGL-safe TRILL switch may have only VL ports, in which case it is not required to support the capabilities for FGL egress described in this section. same deal as ingress, we're skipping it if we don't need it 🦋

An FGL-edge TRILL switch MUST be able to convert, in a configurable fashion, from the FGL in an FGL TRILL Data packet it is egressing to the C-VLAN ID for the resulting native frame with different mappings on a per-port basis 💅 The priority and DEI of the egressed native frame are taken from the Inner.Label Low Order Part. A port MAY be configured to strip output VLAN tagging. the port is giving options, and that's iconic 🌸

It is the responsibility of the network manager to properly configure the TRILL switches in the campus to obtain the desired mappings 🧚

FGL egress is similar to VL egress, as follows:

  1. If the Inner.MacDA is All-Egress-RBridges, special processing applies, based on the payload Ethertype (for example, End-Station Address Distribution Information (ESADI) [RFC6325] or RBridge Channel [RFC7178]), and if the payload Ethertype is unknown, the packet is discarded. If the Inner.MacDA is not All-Egress-RBridges, then either item 2 or item 3 below applies, as appropriate. 💀
  2. A known unicast FGL TRILL Data packet (TRILL Header M bit = 0) with a unicast Inner.MacDA is egressed to the FGL port or ports matching its FGL and Inner.MacDA. If there are no such ports, it is flooded out of all FGL ports that have its FGL, except any ports for which the TRILL switch has knowledge that the frame's Inner.MacDA cannot be present on the link out of that port. 🍓
  3. A multi-destination FGL TRILL Data packet is decapsulated and flooded out of all ports that have its FGL, subject to multicast pruning. The same processing applies to a unicast FGL TRILL Data packet with a broadcast or multicast Inner.MacDA that might be received due to serial unicast. 🌈

An FGL TRILL switch MUST NOT egress an FGL packet with label (X.Y) to any port not configured with that FGL, even if the port is configured to egress VL packets in VLAN X. no sneaking it onto the wrong port, periodt 🙅‍♀️

FGL TRILL switches MUST accept multi-destination TRILL Data packets that are sent to them as TRILL unicast packets (packets with the TRILL Header M bit set to 0). They locally egress such packets, if appropriate, but MUST NOT forward them (other than egressing them as native frames on their local links). they can receive it, egress it, but they will not pass it along, bestie 🔥

4.4. Appointed Forwarders and the DRB 🎀

There is no change in adjacency [RFC7177], DRB (Designated RBridge) election, or Appointed Forwarder logic [RFC6439] on a link, regardless of whether some or all the ports on the link are for FGL TRILL switches, with one exception: implementations SHOULD provide that their default priority for a VL RBridge port to be the DRB is less than their default priority for an FGL RBridge to be the DRB. This will assure that, in the unconfigured case, an FGL RBridge will be elected DRB when using that implementation. the FGL one wins the DRB crown by default 👑✨

4.5. Distribution Tree Construction 🧚

All distribution trees are calculated as provided for in the TRILL base protocol standard [RFC6325] as updated by [RFC7180], with the exception that the default tree root priority for a nickname held by an FGL TRILL switch or an FGL link pseudonode is 0x9000. As a result, they will be chosen in preference to VL nicknames in the absence of configuration 💖 If distribution tree roots are configured, there MUST be at least one tree rooted at a nickname held by an FGL TRILL switch or by an FGL link pseudonode. If distribution tree roots are misconfigured so there would not be such a tree, then the highest priority FGL nickname to be a tree root is used to construct an additional tree, regardless of configuration. (VL TRILL switches will not know about this additional distribution tree but, through the use of Step (A) or (B) in Section 5.1, no VL TRILL switch should ever receive a multi-destination TRILL Data packet using this additional tree.) it's a safety net, so the FGL tree always exists, no cap 🛟

4.6. Address Learning 🌸

An FGL TRILL switch learns addresses from the data plane on ports configured for FGL based on the fine-grained label rather than the native frame's VLAN. Addresses learned from ingressed native frames on FGL ports are logically represented by { MAC address, FGL, port, confidence, timer }, while remote addresses learned from egressing FGL packets are logically represented by { MAC address, FGL, remote TRILL switch nickname, confidence, timer } 🦋

4.7. ESADI Extension 💜

The TRILL ESADI (End-Station Address Distribution Information) protocol is specified in [RFC6325] as optionally transmitting MAC address connection information through TRILL Data packets between participating TRILL switches over the virtual link provided by the TRILL multi-destination packet distribution mechanism. In [RFC6325], the VL to which an ESADI packet applies is indicated only by the Inner.VLAN label, and no indication of that VL is allowed within the ESADI payload. ESADI is extended to support FGL by providing for the indication of the FGL to which an ESADI packet applies only in the Inner.Label of that packet, and no indication of that FGL is allowed within the ESADI payload. the label goes in one spot only, and the payload stays clean 🧼✨

5. FGL TRILL Interaction with VL TRILL 🌈

This section discusses mixing FGL-safe and VL TRILL switches in a campus 💅 It does not apply if the campus is entirely FGL-safe or if there are no FGL-edges. Section 5.1 specifies what behaviors are needed to render such mixed campuses safe. See also Appendix B for a discussion of campus characteristics when these behaviors are in use. Section 5.2 gives details of link-local mixed behavior. this is the mixing zone, so read carefully, bestie 👀

It is best, if possible, for VL TRILL switches to be upgraded to FGL-safe before introducing FGL-edges (and therefore FGL data packets) 🫶🎀

5.1. FGL and VL Mixed Campus

Okay bestie, this is the big drama 🎀 By definition, VL TRILL switches can't safely handle FGL traffic, even when the VL switch is only doing transit. If a TRILL switch can safely transit FGL TRILL Data packets, it counts as FGL-safe, but it's still treated as VL until it advertises in its LSP that it's FGL-safe 💅

VL frames are required to have 0x8100 at the start of the data label, while FGL frames have 0x893B. VL TRILL switches that follow [RFC6325] should discard frames with this new value after the inner MAC addresses. If they don't discard them, they could get confused and egress them into the wrong VLAN (see Section 9 below), or persistently reorder them by miscomputing flows for ECMP, or improperly prune their distribution trees when the frames are multi-destination, so they fail to reach some intended destinations 😭 To avoid all that chaos, every practical step gets taken to minimize the chance of a VL TRILL switch handling an FGL TRILL Data packet. Those steps are below 🦋

FGL-safe switches report their FGL capability in LSPs. So FGL-safe TRILL switches (and any management system with access to the link-state database) can spot the TRILL switches in the campus that don't support FGL 👀

Once a TRILL switch advertises an FGL-edge, any FGL-safe TRILL switch (RB1 in this discussion) that observes a VL RBridge on the link out of one of its ports MUST take Step (A) or (B) below for that port, and also take Step (C) further below. ("Observes" means it has an adjacency to the VL TRILL switch that is in any state other than Down [RFC7177], and it holds an LSP fragment zero for that switch, showing the switch is not FGL-safe.) Finally, for full FGL connectivity, the campus topology must be such that all FGL TRILL switches are reachable from all other FGL TRILL switches without going through a VL TRILL switch 🌈

  • (A) If RB1 can discard any FGL TRILL Data packet that would be output through a port where it observes a VL RBridge, while still allowing VL TRILL Data packets to be output through that port, then:
    • A1. RB1 MUST so discard all FGL TRILL Data output packets that would otherwise be output through the port, and
    • A2. For all adjacencies out of that port (even adjacencies to other FGL RBridges or a pseudonode) in the Report state [RFC7177], RB1 MUST report that adjacency cost as 2**23 greater than it would have otherwise reported, but not more than 2**24 - 2 (the highest link cost still usable in least-cost path calculations and distribution tree construction). This makes sure that if any path through FGL-safe TRILL switches exists, such a path will be computed 🔥
  • (B) If RB1 cannot discard any FGL TRILL Data packet that would be output through a port where it observes a VL RBridge while still allowing VL TRILL Data packets, then RB1 MUST, for all adjacencies out of that port (even adjacencies to other FGL-safe RBridges or a pseudonode) in the Report state [RFC7177], report the adjacency cost as 2**24 - 1. As specified in IS-IS [RFC5305], that cost stops the adjacency from being used in least-cost path calculations, including distribution tree construction (see Section 2.1 of [RFC7180]). It still stays visible in the topology and usable, for example, by any traffic engineered path mechanism 🫶
  • (C) The roots for all distribution trees used for FGL TRILL Data packets must be nicknames held by an FGL-safe TRILL switch or by a pseudonode representing an FGL link. As provided in Section 4.5, there will always be such a distribution tree 👑

Using the increased adjacency cost from part A2 of Step (A), VL links get avoided unless no other path is available, for typical data center link speeds using the default link cost determination method in Item 1 of Section 4.2.4.4 of [RFC6325]. However, if links have low speed (such as about 100 megabits/second or less) or some non-default method is used for determining link costs, then link costs MUST be adjusted so that no adjacency between FGL-safe TRILL switches has a cost greater than 200,000 🙈

To sum it all up: for a mixed TRILL campus to be safe once FGL-edges are introduced, it's essential that FGL-safe RBridges follow the steps above. That keeps paths between those RBridges from going through VL RBridges, and keeps FGL packets from ever being forwarded to VL RBridges. In other words, all FGL-safe switches MUST do Step (A) or (B) for any port out of which they observe a VL RBridge neighbor. Also, for full FGL connectivity, all FGL-safe TRILL switches MUST do Step (C) and be connected in a single FGL contiguous area 💖

The usual DRB election operates on a link with mixed FGL and VL ports. If an FGL TRILL switch port is a DRB, it can handle all native traffic. It MUST appoint only other FGL TRILL switch ports as Appointed Forwarder for any VLANs that are to be mapped to FGL ✨

For VLANs that aren't being mapped to FGL: if Step (A) is being followed (see Section 5.1), the DRB can appoint either a VL or FGL TRILL switch for a VLAN on the link, to be handled by a VL. If Step (B) is being followed, an FGL DRB MUST only appoint FGL Appointed Forwarders, so that all end stations get service to the FGL campus 🌸 If a VL RBridge is a DRB, it won't understand that FGL TRILL switch ports are different. To the extent that Step (B) is in effect and a VL DRB handles native frames, or appoints other VL TRILL switch ports on a link to handle native frames for one or more VLANs, the end stations sending and receiving those native frames may be isolated from the FGL campus 😬 When a VL DRB happens to appoint an FGL port as Appointed Forwarder for one or more VLANs, the end stations sending and receiving native frames in those VLANs will get service to the FGL campus 🥹

5.3. Summary of FGL-Safe Requirements

Here's the checklist of what a TRILL switch needs to be FGL-safe 📝

  1. For both unicast and multi-destination data, RB1 MUST NOT forward an FGL packet to a VL neighbor RB2. This is handled as specified in Section 5.1.
  2. For both unicast and multi-destination data, RB1 MUST NOT egress a packet onto a link that does not belong in that FGL.
  3. For unicast data, RB1 must forward the FGL packet properly to the egress nickname in the TRILL Header. This means it MUST NOT delete the packet because it doesn't have the expected VLAN tag, it MUST NOT insert a VLAN tag, and it MUST NOT misclassify a flow so as to persistently misorder packets. The reason is that the TRILL fields are now 4 bytes longer than in VL TRILL packets 🧚
  4. For multi-destination data, RB1 must forward the packet properly along the specified tree. This means that RB1 MUST NOT falsely prune the packet. RB1 is allowed not to prune at all, but it MUST NOT prevent an FGL packet from reaching all the links with that FGL, by incorrectly refusing to forward the FGL packet along a branch in the tree 🌈
  5. RB1 must advertise, in its LSP, that it is FGL-safe.

Point 1 above (so a TRILL switch correctly supports ECMP) and point 2 (so a TRILL switch correctly prunes distribution trees) both require that the TRILL switch properly recognize and distinguish between the two Ethertypes that can occur immediately after the Inner.MacSA in a TRILL Data packet 🤓

6. IS-IS Extensions

Extensions related to TRILL's use of IS-IS are required to support FGL, and must include the following:

  1. A method for a TRILL switch to announce itself in its LSP as FGL-safe (see Section 8.2).
  2. A sub-TLV analogous to the Interested VLANs and Spanning Tree Roots sub-TLV of the Router Capabilities TLV, but indicating FGLs rather than VLs. This is called the Interested Labels and Spanning Tree Roots (INT-LABEL) sub-TLV in [RFC7176] 🏷️
  3. Sub-TLVs analogous to the GMAC-ADDR sub-TLV of the Group Address TLV, but specifying an FGL rather than a VL. These are called the GLMAC-ADDR, GLIP-ADDR, and GLIPV6-ADDR sub-TLVs in [RFC7176] 💜

7. Comparison with Goals

Comparing TRILL FGL, as specified in this document, with the goals given in Section 2.1, here's what we find 🎯

  1. Fine-Grained: FGL provides 2**24 labels, vastly more than the 4094 (4K) VLAN labels supported in TRILL as specified in [RFC6325] 🦋
  2. Silicon: Existing TRILL fast path silicon chips can perform base TRILL Header insertion and removal to support ingress and egress. In addition, it is believed that most such chips can also perform the native-frame-to-FGL mapping and the encoding of the FGL as specified herein, as well as the inverse decoding and mapping. Some existing silicon chips can perform only one of these operations on a frame in one pass through the fast path. However, other existing chips are believed to be able to perform both operations on the same frame in one pass through their fast path. It is also believed that most FGL TRILL switches will be capable of having their ports configured to discard FGL packets. That capability makes interoperation with VL TRILL switches practical using Step (A) rather than Step (B) (see Section 5.1) 🔧
  3. Base RBridge Interoperation: As described in Section 3, FGL is not generally compatible with TRILL switches conformant to the base specification [RFC6325]. In particular, a VL TRILL switch cannot be an FGL TRILL switch, because there's a risk that it would mishandle FGL packets. However, a contiguous set of VL TRILL switches can exchange VL frames, regardless of the presence of FGL TRILL switches in the campus. The provisions of Section 5 support reasonable interoperation and migration scenarios 🤝
  4. Alternate Priority: The encoding specified in Section 2.3 and the ingress/egress processing specified in Section 4 provide for a new priority and DEI in the Inner.Label High Part, and a place to preserve the original user priority and DEI in the Low Part, so that it can be restored on egress 💅

8. Allocation Considerations

Allocations by the IEEE Registration Authority and IANA are listed below 📋

8.1. IEEE Allocation Considerations

The IEEE Registration Authority has assigned Ethertype 0x893B for TRILL FGL 🌟

8.2. IANA Considerations

IANA has allocated capability flag 1 in the TRILL-VER sub-TLV capability flags [RFC7176] to indicate that a TRILL switch is FGL-safe 🎀

9. Security Considerations 🔐

okay bestie, this section is dramatic and it's giving real stakes, so pay attention 💀. Quick reminder that [RFC6325] has the general TRILL security considerations, so go read that too 📚.

As with any communications system, end-to-end encryption and authentication should be considered for sensitive data 🔒. Here that means encryption and authentication running from a source end station, through the TRILL campus, all the way to a destination end station 🫶.

Mixing up a packet with VL X and a packet with FGL (X.Y) is a real problem, and so is a malformed frame. This happens if an FGL TRILL switch doesn't properly check for 0x8100 or 0x893B right after the Inner.MacSA (see Sections 2.2 and 2.3) and handle the frame correctly 😭.

[RFC6325] requires that the Ethertype right after the Inner.MacSA be 0x8100 🎀. A VL TRILL switch that doesn't throw out a packet with some other value there could cause chaos 🙈. Say it gets a TRILL Data packet with FGL (X.Y), or with junk after the Inner.MacSA that includes X where a VLAN ID would normally go. Then:

  1. It could egress the packet to an end station in VLAN X. If the packet was a well-formed FGL frame, the payload of that egressed native frame would look like it starts with Ethertype 0x893B, which an end station would probably throw away. Either way, an egress like this would almost certainly break a security policy that requires separating differently labeled data 🚫.
  2. If the packet was multi-destination and the TRILL switch pruned the distribution tree, it would prune it based on VLAN X by mistake. For an FGL packet, that would probably mean the multi-destination data packet doesn't reach all of its intended recipients 😬.

Now picture an FGL TRILL switch that gets a TRILL Data packet with junk after the Inner.MacSA that includes X where a VLAN ID would go, and it also doesn't check the Ethertype right after the Inner.MacSA. It could improperly egress the packet in VLAN X and violate security policy 💥. If the packet was multi-destination and got forwarded improperly, properly implemented TRILL switches further down the distribution tree should discard it and never egress it. But the packet would still have traveled along the way, wasting bandwidth 💸.

To avoid all of this, every TRILL switch MUST check the Ethertype right after the Inner.MacSA. If it's a value the switch doesn't know how to handle, the switch either discards the frame or makes no decisions based on any data after that Ethertype 🧚. Also, FGL packets need to be kept away from VL TRILL switches that will discard them if they're implemented properly, or mishandle them if they aren't 🙅‍♀️. This is handled the way Section 5.1 specifies 💅.

Appendix A. Serial Unicast 📬

This informational appendix talks about the pros and cons of using serial unicast instead of a distribution tree for multi-destination TRILL Data packets 🦋. See Sections 4.1 and 4.3. This document requires that FGL TRILL switches accept serial unicast, but it doesn't require them to be able to send serial unicast 🫧.

Picture a big TRILL campus with hundreds of TRILL switches, where, say, 300 end stations are in one particular FGL data label 🌸.

At one extreme, all 300 end stations are on links attached to a single TRILL switch. Then no other TRILL switch would be advertising interest in that FGL. So because of pruning, a multi-destination (say, broadcast) frame from one of those end stations probably wouldn't be sent to any other TRILL switch, even if it were put on a distribution tree 🙃.

At the other extreme, the 300 end stations are attached, one each, to 300 different TRILL switches. In that case, you're almost certainly better off with a distribution tree. If you tried serial unicast, you'd have to output 300 copies, probably including multiple copies through the same port, and that would cause way higher link utilization 📈.

Now say these 300 end stations connect to exactly two TRILL switches, 200 to one and 100 to the other. Unicast TRILL Data packets between those two switches are the best choice 🏆. The frames follow least-cost paths, and the traffic can even spread across several equal-cost least-cost paths ✨. With distribution trees, each frame would be stuck to the tree used for that frame. It would likely take a higher-cost route, and only a single path would be available per tree 😤. So this document says that unicast SHOULD be used if exactly two TRILL switches are involved 💜.

If the end stations connect to more than two TRILL switches, the decision gets more complicated 🌀. Which option is better depends on lots of factors, including network topology and application data patterns. How to make that decision in those cases is beyond the scope of this document 🤷‍♀️.

Appendix B. Mixed Campus Characteristics 🌈

This informational appendix describes what a TRILL campus with a mix of FGL-safe and VL TRILL switches is like, in two cases. Appendix B.1 covers the case where all FGL adjacencies with VL are handled by Step (A) in Section 5.1. Appendix B.2 covers the case where all FGL adjacencies with VL are handled by Step (B) in Section 5.1 🧩.

B.1. Mixed Campus with High Cost Adjacencies 💸

If the FGL TRILL switches use Step (A) in Section 5.1, VL and FGL TRILL switches can interoperate for VL traffic 🤝. Least-cost paths avoid any FGL -> VL TRILL switch hops unless no other reasonable path is available 🛣️. Combined with Section 4.5, there's at least one distribution tree rooted at a nickname held by an FGL TRILL switch or at the pseudonode for an FGL link 🌳. And if the FGL TRILL switches in the campus form a single contiguous island, that distribution tree has a fully connected sub-tree covering the island 🏝️. So any FGL TRILL Data packets sent on this tree can reach any other FGL TRILL switch without trying to go through any VL TRILL switches. (Such an attempt would cause the FGL packet to be discarded, as specified in part A1 of Step (A) 🙅‍♀️.)

Step (A) works especially well in a campus with an FGL TRILL switch core and VL TRILL switches in one or more islands around that core 👑. For example, check out this campus. It has an FGL core made of FGL01 to FGL14, plus three VL islands: VL01 to VL04, VL05, and VL06 to VL14 🗺️.

                  *VL01--*VL02
                    |      |
                  *VL03--*VL04                *VL05
                    |      |                    |
                  FGL01--FGL02--FGL03--FGL04--FGL05
                    |      |      |      |      |
                  FGL06--FGL07--FGL08--FGL09--FGL10
                    |      |      |      |      |
                  FGL11--FGL12--*VL06--*VL07---FGL13
                           |      |      |      |
                         *VL08--*VL09--*VL10---FGL14
                           |      |      |      |
                         *VL11--*VL12--*VL13--*VL14

If all the FGL TRILL switches in this campus implement Step (A), end stations connected through a VL port can be attached anywhere in the campus, to VL or FGL TRILL switches, and they'll communicate if they're in the same VLAN 🥰. End stations connected through an FGL port on FGL TRILL switches will communicate if their local VLANs are mapped to the same FGL 💖.

Because FGL-to-VL adjacencies are so expensive in path computations, VL TRILL switches get avoided on paths between FGL TRILL switches 🙅. For example, even if the speed and default adjacency cost of every connection shown above were the same, traffic from FGL12 to FGL13 would take the 5-hop path FGL12 - FGL07 - FGL08 - FGL09 - FGL10 - FGL13 instead of the 3-hop path FGL12 - VL09 - VL10 - FGL14 🐢.

B.2. Mixed Campus with Data Blocked Adjacencies 🚧

If the FGL TRILL switches use Step (B) in Section 5.1, then least-cost and distribution tree TRILL Data communication between VL and FGL TRILL switches is blocked. TRILL IS-IS communication still works normally 📡. This data blocking is only implemented by FGL TRILL switches, but its effects are pretty symmetric 🪞. The following paragraphs assume this data blocking between VL and FGL is in effect throughout the campus 🧱.

A campus that's mostly FGL TRILL switches using Step (B), with a few isolated VL TRILL switches scattered around, works well for connectivity between end stations attached to those FGL switches 🌟. The exception is that those end stations can't communicate with any end stations whose forwarder is a VL switch 😵. The VL TRILL switches end up isolated and can only route TRILL Data to the extent they happen to be contiguously connected to other VL TRILL switches 🧭. Distribution trees computed by the FGL switches won't include any VL switches (see Section 2.1 of [RFC7180]) 🌲.

A campus that's mostly VL TRILL switches with a few isolated FGL TRILL switches scattered throughout also works reasonably well, as described just above, with all occurrences of "FGL" and "VL" swapped 🔄.

However, a campus that's badly misconfigured and has a randomly mixed-up bunch of VL and FGL TRILL switches using Step (B) is likely to offer very poor data service 😭, because lots of links end up blocked for data 🚫.

Acknowledgements 🫶

The comments and suggestions of the following, listed in alphabetic order, are gratefully acknowledged 💌:

Stewart Bryant, Spencer Dawkins, Adrian Farrel, Anoop Ghanwani, Sujay Gupta, Weiguo Hao, Phanidhar Koganti, Yizhou Li, Vishwas Manral, Rajeev Manur, Thomas Narten, Gayle Nobel, Erik Nordmark, Pete Resnick, Olen Stokes, Sean Turner, Ilya Varlashkin, and Xuxiaohu ✨

References 📚

Normative References 📚

Okay, these are the must-read docs bestie, the ones everything else leans on 💅

  • [802.1Q] IEEE 802.1, "IEEE Standard for Local and metropolitan area networks--Media Access Control (MAC) Bridges and Virtual Bridged Local Area Networks", IEEE Std 802.1Q-2011, August 2011.
  • [IS-IS] ISO/IEC 10589:2002, Second Edition, "Information technology -- Telecommunications and information exchange between systems -- Intermediate System to Intermediate System intra-domain routeing information exchange protocol for use in conjunction with the protocol for providing the connectionless-mode network service (ISO 8473)", 2002.
  • [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
  • [RFC5305] Li, T. and H. Smit, "IS-IS Extensions for Traffic Engineering", RFC 5305, October 2008.
  • [RFC6325] Perlman, R., Eastlake 3rd, D., Dutt, D., Gai, S., and A. Ghanwani, "Routing Bridges (RBridges): Base Protocol Specification", RFC 6325, July 2011.
  • [RFC7176] Eastlake 3rd, D., Senevirathne, T., Ghanwani, A., Dutt, D., and A. Banerjee, "Transparent Interconnection of Lots of Links (TRILL) Use of IS-IS", RFC 7176, May 2014.
  • [RFC7177] Eastlake 3rd, D., Perlman, R., Ghanwani, A., Yang, H., and V. Manral, "Transparent Interconnection of Lots of Links (TRILL): Adjacency", RFC 7177, May 2014.
  • [RFC7180] Eastlake 3rd, D., Zhang, M., Ghanwani, A., Manral, V., and A. Banerjee, "Transparent Interconnection of Lots of Links (TRILL): Clarifications, Corrections, and Updates" RFC 7180, May 2014.

Informative References 🌸

These ones are extra reading, super useful for the full picture 🧚

  • [RFC6165] Banerjee, A. and D. Ward, "Extensions to IS-IS for Layer-2 Systems", RFC 6165, April 2011.
  • [RFC6439] Perlman, R., Eastlake, D., Li, Y., Banerjee, A., and F. Hu, "Routing Bridges (RBridges): Appointed Forwarders", RFC 6439, November 2011.
  • [RFC7174] Salam, S., Senevirathne, T., Aldrin, S., and D. Eastlake 3rd, "Transparent Interconnection of Lots of Links (TRILL) Operations, Administration, and Maintenance (OAM) Framework", RFC 7174, May 2014.
  • [RFC7178] Eastlake 3rd, D., Manral, V., Li, Y., Aldrin, S., and D. Ward, "Transparent Interconnection of Lots of Links (TRILL): RBridge Channel Support", RFC 7178, May 2014.

Authors' Addresses 🎀

Here are the brilliant minds behind this whole thing, with their contact details exactly as they're listed 🫶

Donald Eastlake 3rd
Huawei Technologies
155 Beaver Street
Milford, MA  01757
USA

Phone: +1-508-333-2270
EMail: d3e3e3@gmail.com
Mingui Zhang
Huawei Technologies Co., Ltd
Huawei Building, No.156 Beiqing Rd.
Z-park, Shi-Chuang-Ke-Ji-Shi-Fan-Yuan, Hai-Dian District
Beijing 100095
P.R. China

EMail: zhangmingui@huawei.com
Puneet Agarwal
Broadcom Corporation
3151 Zanker Road
San Jose, CA  95134
USA

Phone: +1-949-926-5000
EMail: pagarwal@broadcom.com
Radia Perlman
Intel Labs
2200 Mission College Blvd.
Santa Clara, CA  95054
USA

Phone: +1-408-765-8080
EMail: Radia@alum.mit.edu
Dinesh G. Dutt
Cumulus Networks
1089 West Evelyn Avenue
Sunnyvale, CA  94086
USA

EMail: ddutt.ietf@hobbesdutt.com
📦 where this page comes from