RFC 9024

Deterministic Networking (DetNet) Data Plane: IEEE 802.1 Time-Sensitive Networking over MPLS

📅 published June 2021, from ietf

Deterministic Networking (DetNet) Data Plane: IEEE 802.1 Time-Sensitive Networking over MPLS 💖

RFC 9024 is the one that teaches DetNet to play nice with TSN 🎀

The title block is giving group project energy. B. Varga, Ed. and J. Farkas are from Ericsson, A. Malis is at Malis Consulting, S. Bryant is at Futurewei Technologies, and D. Fedyk is at LabN Consulting, L.L.C. 💅 It's a Standards Track doc, so it's a real IETF standard and not just a vibe. The ISSN is 2070-1721 and the date is June 2021. The title block doesn't list any RFCs it obsoletes or updates, so this one is standing on its own ✨

Abstract ✨

okay so this doc lays out the Deterministic Networking data plane for when TSN networks get interconnected over a DetNet MPLS network 🦋

Status of This Memo 📜

This is an Internet Standards Track document, bestie 💜

This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). For more on Internet Standards, check Section 2 of RFC 7841.

Info about the current status of this doc, any errata, and how to send feedback lives at https://www.rfc-editor.org/info/rfc9024 🌸

Copyright (c) 2021 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, because they spell out your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they're provided without warranty as described in the Simplified BSD License 🫶

Table of Contents 🧚

1.  Introduction
2.  Terminology
  2.1.  Terms Used in This Document
  2.2.  Abbreviations
  2.3.  Requirements Language
3.  IEEE 802.1 TSN over DetNet MPLS Data Plane Scenario
4.  DetNet MPLS Data Plane
  4.1.  Overview
  4.2.  TSN over DetNet MPLS Encapsulation
5.  TSN over MPLS Data Plane Procedures
  5.1.  Edge Node TSN Procedures
  5.2.  Edge Node DetNet Service Proxy Procedures
  5.3.  Edge Node DetNet Service and Forwarding Sub-Layer
        Procedures
6.  Controller Plane (Management and Control) Considerations
7.  Security Considerations
8.  IANA Considerations
9.  References
  9.1.  Normative References
  9.2.  Informative References
Acknowledgements
Authors' Addresses

1. Introduction ✨

okay so the Time-Sensitive Networking Task Group (TSN TG) inside the IEEE 802.1 Working Group handles deterministic services over IEEE 802 networks 🦋 Deterministic Networking (DetNet), which the IETF defined, is a service that an L3 network can offer to DetNet flows. If you want the general background and concepts, [RFC8655] has you covered 💅

This document is all about using a DetNet MPLS network to interconnect TSN nodes and network segments 🎀 The DetNet MPLS data plane itself is defined in [RFC8964] 💖

2. Terminology 📖

2.1. Terms Used in This Document

This doc uses the terminology and concepts from the DetNet architecture, which means [RFC8655] [RFC8938] [RFC8964] 🌸 The TSN-specific terms are defined by the TSN TG of the IEEE 802.1 Working Group. The reader is assumed to already know these documents and their terminology, so go read them first bestie 🙈

2.2. Abbreviations

Here's the abbreviation list, in a code block so it stays lined up 💜

AC            Attachment Circuit

CE            Customer Edge equipment

d-CW          DetNet Control Word

DetNet        Deterministic Networking

DF            DetNet Flow

FRER          Frame Replication and Elimination for Redundancy (TSN
              function)

L2            Layer 2

L2VPN         Layer 2 Virtual Private Network

L3            Layer 3

LSP           Label Switched Path

LSR           Label Switching Router

MPLS          Multiprotocol Label Switching

MPLS-TE       Multiprotocol Label Switching - Traffic Engineering

NSP           Native Service Processing

OAM           Operations, Administration, and Maintenance

PE            Provider Edge

PREOF         Packet Replication, Elimination and Ordering Functions

PW            Pseudowire

S-PE          Switching Provider Edge

T-PE          Terminating Provider Edge

TSN           Time-Sensitive Network

2.3. Requirements Language 🔥

Here's the key word bit, and it's a big deal. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here 👑 Lowercase versions are just regular words, so no drama.

3. IEEE 802.1 TSN over DetNet MPLS Data Plane Scenario 🌈

Figure 1 shows IEEE 802.1 TSN end stations running over a TSN-aware DetNet service on top of an MPLS network ✨ The DetNet edge nodes sit at the boundary of a DetNet domain. They map non-DetNet-aware L2 traffic onto DetNet services, and they handle imposing and removing the required DetNet encapsulation. They're basically the same as PW T-PE nodes, which use MPLS-TE LSPs 💅 In this example, TSN Streams are simple applications running over DetNet flows. We'll get into the details of how this works later in the doc 🦋

      TSN           Edge          Transit         Edge          TSN
   End System       Node           Node           Node       End System
                   (T-PE)         (LSR)          (T-PE)

   +----------+                                             +----------+
   |   TSN    | <-------- End-to-End TSN Service ---------> |   TSN    |
   |  Applic. |                                             |  Applic. |
   +----------+  +.........+                   +.........+  +----------+
   |          |  | \S-Proxy:                   :S-Proxy/ |  |          |
   |   TSN    |  |   +.+---+<-- DetNet flow -->+---+ |   |  |   TSN    |
   |          |  |TSN| |Svc|                   |Svc| |TSN|  |          |
   +----------+  +---+ +---+    +----------+   +---+ +---+  +----------+
   |   L2     |  | L2| |Fwd|    |Forwarding|   |Fwd| |L2 |  |   L2     |
   +------.---+  +-.-+ +-.-+    +---.----.-+   +--.+ +-.-+  +---.------+
          :  Link  :     /  ,-----.  \   :  Link  :   /  ,-----. \
          +........+     +-[  Sub- ]-+   +........+   +-[  TSN  ]-+
                           [Network]                    [Network]
                            `-----'                      `-----'

                       |<------ DetNet MPLS ------>|
           |<---------------------- TSN   --------------------->|

              Figure 1: A TSN over DetNet MPLS-Enabled Network

In this example, the edge nodes give you a service proxy function that "associates" DetNet flows with native flows (aka TSN Streams) at the edge of the DetNet domain 🎀 TSN Streams are treated as App-flows for DetNet. The whole DetNet domain acts like a TSN relay node for the TSN Streams, and the service proxy acts like a port on that TSN relay node 💖

Figure 2 shows how DetNet can provide services for IEEE 802.1 TSN end systems, CE1 and CE2, over a DetNet-enabled MPLS network 🌸 Edge nodes E1 and E2 insert and remove the required DetNet data plane encapsulation. The 'X' in the edge nodes and in the relay node, R1, marks a potential DetNet compound flow packet replication and elimination point 🧚 This conceptually lines up with L2VPN services and could build on existing related solutions, as discussed below ✨

        TSN    |<------- End-to-End DetNet Service ------>|  TSN
       Service |         Transit          Transit         | Service
   TSN  (AC)   |        |<-Tnl->|        |<-Tnl->|        |  (AC)  TSN
   End    |    V        V    1  V        V   2   V        V   |    End
   System |    +--------+       +--------+       +--------+   |  System
   +---+  |    |   E1   |=======|   R1   |=======|   E2   |   |   +---+
   |   |--|----|._X_....|..DF1..|.._ _...|..DF3..|...._X_.|---|---|   |
   |CE1|  |    |    \   |       |   X    |       |   /    |   |   |CE2|
   |   |       |     \_.|..DF2..|._/ \_..|..DF4..|._/     |       |   |
   +---+       |        |=======|        |=======|        |       +---+
       ^       +--------+       +--------+       +--------+       ^
       |        Edge Node       Relay Node        Edge Node       |
       |          (T-PE)           (S-PE)          (T-PE)         |
       |                                                          |
       |<- TSN -> <------- TSN over DetNet MPLS -------> <- TSN ->|
       |                                                          |
       |<-------- Time-Sensitive Networking (TSN) Service ------->|

       X   = Service protection
       DFx = DetNet member flow x over a TE LSP
           AC  = Attachment Circuit
           Tnl = Tunnel

                    Figure 2: IEEE 802.1TSN over DetNet

4. DetNet MPLS Data Plane 🦋

4.1. Overview

Okay, the basic approach in [RFC8964] supports the DetNet service sub-layer using existing PW encapsulations and mechanisms, and it supports the DetNet forwarding sub-layer using existing MPLS Traffic Engineering encapsulations and mechanisms 💅

A node working on a DetNet flow in the DetNet service sub-layer, meaning a node processing a DetNet packet that has the S-Label as its top of stack, uses the local context tied to that S-Label 🫶 For example, a received F-Label can be used to figure out which local DetNet operation(s) get applied to that packet. An S-Label may be unique when taken from the platform label space [RFC3031], which would let the node identify the DetNet flow correctly no matter which input interface or LSP the packet arrives on 🔥 The service sub-layer functions (i.e., PREOF) use a DetNet control word (d-CW) 💖

The DetNet MPLS data plane builds on MPLS Traffic Engineering encapsulations and mechanisms to provide a forwarding sub-layer, and that sub-layer is responsible for resource allocation and explicit routes 🌈 The forwarding sub-layer is supported by one or more forwarding labels (F-Labels) ✨

DetNet edge and relay nodes are DetNet service sub-layer aware. They understand what DetNet flows need, and they provide both the DetNet service and forwarding sub-layer functions 🎀 They add, remove, and process d-CWs, S-Labels, and F-Labels as needed 🧚 MPLS DetNet nodes and transit nodes include DetNet forwarding sub-layer functions. The big one is support for explicit routes and resource allocation, which eliminates (or reduces) congestion loss and jitter 👑 Unlike the other DetNet node types, transit nodes provide no service sub-layer processing at all 🌸

4.2. TSN over DetNet MPLS Encapsulation ✨

okay so the basic move here is to treat a TSN Stream as an App-flow from the DetNet MPLS point of view 🦋 the example lives in Figure 3, and there are three example flows drawn in there, so don't get lost in them 💅

                /->     +------+  +------+  +------+   TSN      ^ ^
       MPLS     |       |  X   |  |  X   |  |  X   |<- Appli    : :
     App-Flow <-+       +------+  +------+  +------+   cation   : :(1)
                |       |TSN-L2|  |TSN-L2|  |TSN-L2|            : v
                \-> +---+======+--+======+--+======+-----+      :
                        | d-CW |  | d-CW |  | d-CW |            :
     DetNet-MPLS        +------+  +------+  +------+            :(2)
                        |Labels|  |Labels|  |Labels|            v
                    +---+======+--+======+--+======+-----+
     Link/Sub-Network   |  L2  |  | TSN  |  | UDP  |
                        +------+  +------+  +------+
                                            |  IP  |
                                            +------+
                                            |  L2  |
                                            +------+
         (1) TSN Stream
         (2) DetNet MPLS Flow

         Figure 3: Examples of TSN over MPLS Encapsulation Formats

in the figure, "Application" means the application payload that the TSN network is carrying 🎀 "MPLS App-Flow" means the TSN Stream is just the payload from the DetNet MPLS data plane's point of view, the one defined in [RFC8964] 🌸 and a single DetNet MPLS flow can bundle up multiple TSN Streams, which is so efficient, ngl 🧚

Note: Network fragmentation for DetNet is not supported and MUST be avoided. The reason for this is that network fragmentation is not consistent with the packet delivery times needed for DetNet. Therefore, when IP is used as the sub-network, IPv6 fragmentation MUST NOT be used, and IPv4 packets MUST be sent with the DF bit set. This means that the network operator MUST ensure that all the DetNet encapsulation overhead plus the maximum TSN App-flow frame size does not exceed the DetNet network's MTU.

bestie, this one is a whole vibe 😭 fragmentation is a no-go for DetNet 🚫 so IPv6 fragmentation MUST NOT happen, IPv4 packets MUST go out with the DF bit set, and the operator has to make sure the encapsulation overhead plus the biggest TSN App-flow frame still fits under the DetNet network's MTU 💀🫶

5. TSN over MPLS Data Plane Procedures 👑

the edge node procedures and functions for TSN over DetNet MPLS scenarios follow the concepts from [RFC3985], and they cover the edge node components shown in Figure 1 🌈 in this section, the following DetNet edge node procedures get the spotlight:

  • TSN related (Section 5.1)
  • DetNet Service Proxy (Section 5.2)
  • DetNet service and forwarding sub-layer (Section 5.3)

the subsections walk through how DetNet edge nodes forward packets, where those packets arrive either from directly connected CEs (TSN nodes) or from some other DetNet edge nodes 💖

5.1. Edge Node TSN Procedures 🍓

the TSN TG of the IEEE 802.1 Working Group has defined (and is still defining) a bunch of amendments to [IEEE8021Q] that give zero congestion loss and bounded latency in bridged networks 🔥 and [IEEE8021CB] defines the packet replication and elimination functions for a TSN network, which is iconic behaviour ✨

the implementation of a TSN entity (aka the TSN packet processing functions) must be compliant with the relevant IEEE 802.1 standards 📚

TSN-specific functions run on data that the DetNet edge node gets from the connected CE, before that data goes out to connected CE(s) or gets handed to the DetNet service proxy function for transmission across the DetNet domain 🦋 they also run on data received from a DetNet PW by a PE, before that data comes out on the AC(s) 💅

the TSN packet processing function(s) of edge nodes (T-PE) belong to the NSP [RFC3985] block 🧚 this is similar to other stuff that standards bodies outside the IETF define, like stripping, overwriting, or adding VLAN tags on Ethernet 🎀 depending on the TSN role of the edge node in the end-to-end TSN service, only selected TSN functions get supported 🌸

when a PE gets a packet from a CE on a given AC with DetNet service, it first checks via Stream identification (see Clause 6 of [IEEE8021CB] and [IEEEP8021CBdb]) whether the packet belongs to a configured TSN Stream (i.e., App-flow from the DetNet perspective) 👀 if no Stream ID matches and no other (VPN) service is configured for the AC, then the packet MUST be dropped 🗑️ if there is a matching TSN Stream, then the Stream-ID-specific TSN functions run (e.g., ingress policing, header field manipulation in the case of active Stream identification, FRER, etc.) ⭐ Source Media Access Control (MAC) lookup may also be used for local MAC address learning 🍓

if the PE decides to forward the packet, the packet MUST be forwarded according to the TSN-Stream-specific configuration to connected CE(s) (for local bridging) and/or to the DetNet service proxy (for forwarding to remote CE(s)) 💖 if there are no TSN-Stream-specific forwarding configurations, the PE MUST flood the packet to other locally attached CE(s) and to the DetNet service proxy 🌊 and if the administrative policy on the PE does not allow flooding, the PE MUST drop the packet 🙈

when a TSN entity of the PE gets a packet from the DetNet service proxy, it first checks via Stream identification (see Clause 6 of [IEEE8021CB] and [IEEEP8021CBdb]) whether the packet belongs to a configured TSN Stream 🔍 if no Stream ID matches, the packet is dropped 💀 if there is a matching TSN Stream, the Stream-ID-specific TSN functions run (e.g., header field manipulation in case of active Stream identification, FRER, etc.) 🧚 Source MAC lookup may also be used for local MAC address learning 🫶

if the PE decides to forward the packet, the packet is forwarded according to the TSN-Stream-specific configuration to connected CE(s) 🦋 if there are no TSN-Stream-specific forwarding configurations, the PE floods the packet to locally attached CE(s) 🌊 and if the administrative policy on the PE does not allow flooding, the PE drops the packet 🙅

implementations of this document SHALL use management and control information so that the TSN-specific functions of the edge node match what the connected TSN network expects 👑💅

5.2. Edge Node DetNet Service Proxy Procedures 🧚

the service proxy function is the one that maps between App-flows and DetNet flows 🔀 the DetNet edge node TSN entity MUST support the TSN Stream identification functions (as defined in Clause 6 of [IEEE8021CB] and [IEEEP8021CBdb]) and the related managed objects (as defined in Clause 9 of [IEEE8021CB] and [IEEEP8021CBdb]) so it can recognize the packets related to an App-flow 👀 the service proxy presents TSN Streams as an App-flow to a DetNet flow, which is giving main character energy 🎀

when a DetNet service proxy gets a packet from the TSN entity, it MUST check whether that App-flow is present in its mapping table 📋 if it is, the proxy attaches the internal DetNet flow ID to the packet and MUST forward it to the DetNet service and forwarding sub-layers 💖 if no match is found, it MUST drop the packet 🗑️

when a DetNet service proxy gets a packet from the DetNet service and forwarding sub-layers, it MUST be forwarded to the edge node TSN entity 🦋

implementations of this document SHALL use management and control information to map a TSN Stream to a DetNet flow 🗺️ N:1 mapping (aggregating multiple TSN Streams into a single DetNet flow) SHALL be supported 🌸 the management or control function that provisions the flow mapping SHALL make sure adequate resources are allocated and configured to fulfill the service requirements of the mapped flows 💅

due to the (intentional) similarities between the DetNet PREOF and TSN FRER functions, service protection function interworking is possible between the TSN and DetNet domains ✨ these service protection interworking scenarios might need sequence number fields copied from TSN (L2) into PW (MPLS) encapsulation 🧠 however, that interworking is out of scope for this document and is left for further study 🔭

5.3. Edge Node DetNet Service and Forwarding Sub-Layer Procedures 🌈

in the design presented in [RFC8964], an MPLS service label (the S-Label), which is similar to a PW label [RFC3985], identifies both the DetNet flow identity and the MPLS payload type 🏷️ the DetNet sequence number rides in the d-CW, which also carries the Data/OAM discriminator 🎀 [RFC8964] supports two sequence number sizes: a 16-bit sequence number and a 28-bit sequence number 🔢

PREOF functions and the service recovery they provide are only available inside the DetNet domain, because the DetNet flow ID and the DetNet sequence number are not valid outside the DetNet network 🏠 MPLS (DetNet) edge nodes terminate all the related information elements encoded in the MPLS labels 🧚

when a PE gets a packet from the service proxy function, it MUST handle the packet as defined in [RFC8964] 🦋

when a PE gets an MPLS packet from a remote PE, then, after processing the MPLS label stack, if the top MPLS label turns out to be a DetNet S-Label that was advertised by this node, the PE MUST forward the packet according to the configured DetNet service and forwarding sub-layer rules, either to other PE nodes or via the DetNet service proxy function towards locally connected CE(s) 🫶

for further details on the DetNet service and forwarding sub-layers, see [RFC8964] 💜

6. Controller Plane (Management and Control) Considerations 🎀

Okay bestie, this part is all about the controller plane, aka who sets up the mapping and how 💅 Info about mapping TSN Stream(s) to DetNet flows is only needed for the service proxy function of MPLS (DetNet) edge nodes. From the data plane's point of view, it does not care whether that info came from the management plane or the control plane. Main character energy for the data plane 👑

Here's the set of info you need to configure TSN over DetNet MPLS 📝

  • TSN-related configuration information according to the TSN role of the DetNet MPLS node, as per [IEEE8021Q], [IEEE8021CB], and [IEEEP8021CBdb].
  • DetNet MPLS-related configuration information according to the DetNet role of the DetNet MPLS node, as per [RFC8964].
  • App-flow identification information to map received TSN Stream(s) to the DetNet flow. Parameters of TSN Stream identification are defined in [IEEE8021CB] and [IEEEP8021CBdb].

This information MUST be provisioned per DetNet flow. No skipping this one, periodt 🫶

Mappings between the DetNet and TSN management and control planes are out of scope for this document. Some of the challenges are highlighted below, so let's look at them 🦋

MPLS DetNet edge nodes are members of both the DetNet domain and the connected TSN network. From the TSN network's point of view, the MPLS (DetNet) edge node has a "TSN relay node" role, so TSN-specific management and control plane functionality must be implemented. The management plane techniques used in DetNet and TSN have lots of similarities, but the control plane protocols are a different story. For example, RSVP-TE and MSRP behave differently. So management and control plane design is a big deal in any scenario where DetNet-to-TSN mapping is needed 😭

Also, the DetNet network is just one piece of the end-to-end TSN path (from the Ethernet perspective, it's a single hop), so some parameters, like delay, may differ a lot 🌸 There is no interworking function, so the DetNet network's bandwidth is assumed to be set large enough to handle all the TSN flows it will support. At the egress of the DetNet domain, the MPLS headers get stripped, and the TSN flow just continues on as a normal TSN flow ✨

To use a DetNet network to interconnect TSN segments, TSN-specific information has to be converted into DetNet-domain-specific information. TSN Stream ID(s) and stream-related parameters/requirements have to be converted into a DetNet flow ID and flow-related parameters/requirements 💜

Sometimes it can be tricky to figure out info about the egress node. For example, it may not be trivial to locate the egress point/interface of a TSN Stream that has a multicast destination MAC address. Scenarios like this may need interaction between control and management plane functions, and between the DetNet and TSN domains 🙈

If it isn't provided explicitly, the mapping between DetNet flow identifiers and TSN Stream identifiers can be done locally by the service proxy function of an MPLS (DetNet) edge node. It uses info provided for configuring the TSN Stream identification functions, like Mask-and-Match Stream identification 🧚

Triggering the setup or modification of a DetNet flow in the DetNet network is one example where management and/or control plane interactions are needed between the DetNet and TSN networks 🔥

Configuring TSN-specific functions (like FRER) inside the TSN network is a TSN-domain-specific decision, and it may not be visible in the DetNet domain. Service protection interworking scenarios are left for further study 🍓

Heads up: service protection interworking is NOT solved in this doc, it's parked for future study. Don't assume it's covered!

7. Security Considerations 🔐

Security considerations for DetNet are described in detail in [DETNET-SEC]. General security considerations are described in [RFC8655].

Considerations specific to the DetNet MPLS data plane are summarized and described in [RFC8964], including any application flow types. This document focuses on a scenario where TSN Streams are the application flows for DetNet, and that scenario is already covered by those DetNet MPLS data plane security considerations 💖

8. IANA Considerations 🌈

This document has no IANA actions. Nothing to register here, bestie ✨

9. References 📚

9.1. Normative References

9.2. Informative References

Acknowledgements 🫶

The authors wish to thank Norman Finn, Lou Berger, Craig Gunther, Christophe Mangin, and Jouni Korhonen for their various contributions to this work. Big thank-yous, besties 💜

Authors' Addresses 💌

Balázs Varga (editor)
Ericsson
Budapest
Magyar Tudosok krt. 11.
1117
Hungary

Email: balazs.a.varga@ericsson.com


János Farkas
Ericsson
Budapest
Magyar Tudosok krt. 11.
1117
Hungary

Email: janos.farkas@ericsson.com


Andrew G. Malis
Malis Consulting

Email: agmalis@gmail.com


Stewart Bryant
Futurewei Technologies

Email: sb@stewartbryant.com


Don Fedyk
LabN Consulting, L.L.C.

Email: dfedyk@labn.net
📦 where this page comes from