RFC 8938

Deterministic Networking (DetNet) Data Plane Framework

📅 published November 2020, from ietf

Deterministic Networking (DetNet) Data Plane Framework 🎀

Okay bestie, here's the title block tea ☕ This is an Informational doc (not a Standards Track spec) from November 2020. It was written by B. Varga (Ed.), J. Farkas (Ericsson), L. Berger (LabN Consulting, L.L.C.), A. Malis (Malis Consulting), and S. Bryant (Futurewei Technologies). It's published under ISSN 2070-1721. The title block doesn't list anything it obsoletes or updates, so it's standing on its own ✨

Abstract ✨

So this doc gives the big-picture framework for the DetNet data plane 🦋 It covers the concepts and considerations that pretty much every DetNet data plane spec will share, and it also talks about the related Controller Plane stuff, so you get the whole vibe.

Status of This Memo 📜

Real talk: this is not an Internet Standards Track specification. It's published for informational purposes.

This doc is a product of the Internet Engineering Task Force (IETF) and represents the consensus of the IETF community. It got public review and was approved for publication by the Internet Engineering Steering Group (IESG). Not every doc the IESG approves is a candidate for any level of Internet Standard, so see Section 2 of RFC 7841 🙈

Info about its current status, any errata, and how to send feedback lives at https://www.rfc-editor.org/info/rfc8938 💌

Copyright (c) 2020 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they stand on the date of publication. Please review these carefully, because they spell out your rights and restrictions for this doc. Code Components pulled out of this document must include Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they're provided without warranty as described in the Simplified BSD License.

Table of Contents 🧚

1.  Introduction
2.  Terminology
  2.1.  Terms Used in This Document
  2.2.  Abbreviations
3.  Overview of the DetNet Data Plane
  3.1.  Data Plane Characteristics
    3.1.1.  Data Plane Technology
    3.1.2.  Encapsulation
  3.2.  DetNet-Specific Metadata
  3.3.  DetNet IP Data Plane
  3.4.  DetNet MPLS Data Plane
  3.5.  Further DetNet Data Plane Considerations
    3.5.1.  Functions Provided on a Per-Flow Basis
    3.5.2.  Service Protection
    3.5.3.  Aggregation Considerations
    3.5.4.  End-System-Specific Considerations
    3.5.5.  Sub-network Considerations
4.  Controller Plane (Management and Control) Considerations
  4.1.  DetNet Controller Plane Requirements
  4.2.  Generic Controller Plane Considerations
    4.2.1.  Flow Aggregation Control
    4.2.2.  Explicit Routes
    4.2.3.  Contention Loss and Jitter Reduction
    4.2.4.  Bidirectional Traffic
  4.3.  Packet Replication, Elimination, and Ordering Functions
        (PREOF)
5.  Security Considerations
6.  IANA Considerations
7.  References
  7.1.  Normative References
  7.2.  Informative References
Acknowledgements
Contributors
Authors' Addresses

1. Introduction ✨

DetNet (Deterministic Networking) is all about carrying specified unicast or multicast data flows for real-time apps, with extremely low packet loss rates and an assured maximum end-to-end delivery latency 🚀 The general background and concepts are in [RFC8655], so go read that first, bestie 📖

This doc describes the concepts that any DetNet data plane spec needs, meaning the DetNet-specific use of packet header fields, and it gives considerations for any corresponding implementation 🛠️ It covers the building blocks that provide the DetNet service, the DetNet service sub-layer, and the DetNet forwarding sub-layer functions, as described in the DetNet architecture [RFC8655].

The DetNet architecture splits the DetNet-related data plane functions into two sub-layers: a service sub-layer and a forwarding sub-layer 🍰 The service sub-layer provides DetNet service protection and reordering. The forwarding sub-layer leverages traffic engineering mechanisms and provides congestion protection (low loss, assured latency, and limited out-of-order delivery). A particular forwarding sub-layer may have capabilities that aren't available on other forwarding sub-layers. DetNet makes use of the existing forwarding sub-layers with their own capabilities, and it does not require 1:1 equivalence between different forwarding sub-layer capabilities 💅

As part of the service sub-layer functions, this document describes typical DetNet node data plane operation 🧠 It explains how the Packet Replication Function (PRF), the Packet Elimination Function (PEF), and the Packet Ordering Function (POF) work within the service sub-layer. It also describes the forwarding sub-layer.

As defined in [RFC8655], DetNet flows may be carried over network technologies that can provide the service characteristics DetNet needs 🌈 For example, DetNet MPLS flows can be carried over IEEE 802.1 Time-Sensitive Networking (TSN) sub-networks [IEEE802.1TSNTG]. However, IEEE 802.1 TSN support is not required in DetNet. TSN frame preemption is an example of a forwarding layer capability that is typically not replicated in other forwarding technologies 🙈 Most of DetNet's benefits can be gained by running over a data-link layer that hasn't been specifically enhanced to support all TSN capabilities. But for those networks and traffic mixes, delay and jitter performance may vary because of the forwarding sub-layer's intrinsic properties, so keep that in mind 💀

Different application flows, such as Ethernet or IP, can be mapped on top of DetNet. DetNet can optionally reuse header information that's provided by, or shared with, applications 🎁 An example of shared header fields can be found in [RFC8939].

This document also covers basic concepts related to the Controller Plane and Operations, Administration, and Maintenance (OAM). Data plane OAM specifics are out of scope for this document 🫶

2. Terminology 🌸

2.1. Terms Used in This Document

This document uses the terminology established in the DetNet architecture [RFC8655]. It's assumed that you already know that doc and its terminology, so you're set 💖

2.2. Abbreviations

The following abbreviations are used in this document 🔤

BGP         Border Gateway Protocol

CoS         Class of Service

d-CW        DetNet Control Word

DetNet      Deterministic Networking

DN          DetNet

GMPLS       Generalized Multiprotocol Label Switching

GRE         Generic Routing Encapsulation

IPsec       IP Security

L2          Layer 2

LSP         Label Switched Path

MPLS        Multiprotocol Label Switching

OAM         Operations, Administration, and Maintenance

PCEP        Path Computation Element Communication Protocol

PEF         Packet Elimination Function

POF         Packet Ordering Function

PREOF       Packet Replication, Elimination, and Ordering Functions

PRF         Packet Replication Function

PSN         Packet Switched Network

QoS         Quality of Service

S-Label     DetNet "service" label

TDM         Time-Division Multiplexing

TSN         Time-Sensitive Networking

YANG        Yet Another Next Generation

3. Overview of the DetNet Data Plane 🦋

This document describes how application flows, or App-flows [RFC8655], get carried over DetNet networks 🚚 The DetNet architecture [RFC8655] models the DetNet-related data plane functions as decomposed into two sub-layers: a service sub-layer and a forwarding sub-layer.

Figure 1, reproduced from [RFC8655], shows a logical DetNet service with the two sub-layers 👀

           |  packets going  |        ^  packets coming   ^
           v down the stack  v        |   up the stack    |
        +-----------------------+   +-----------------------+
        |        Source         |   |      Destination      |
        +-----------------------+   +-----------------------+
        |   Service sub-layer:  |   |   Service sub-layer:  |
        |   Packet sequencing   |   | Duplicate elimination |
        |    Flow replication   |   |      Flow merging     |
        |    Packet encoding    |   |    Packet decoding    |
        +-----------------------+   +-----------------------+
        | Forwarding sub-layer: |   | Forwarding sub-layer: |
        |  Resource allocation  |   |  Resource allocation  |
        |    Explicit routes    |   |    Explicit routes    |
        +-----------------------+   +-----------------------+
        |     Lower layers      |   |     Lower layers      |
        +-----------------------+   +-----------------------+
                    v                           ^
                     \_________________________/

              Figure 1: DetNet Data Plane Protocol Stack

The DetNet forwarding sub-layer may be directly provided by the DetNet service sub-layer, for example by IP tunnels or MPLS 🧚 Alternatively, an overlay approach may be used. In that case the packet is natively carried between key nodes within the DetNet network (say, between PREOF nodes), and a sub-layer provides the information needed to reach the next hop in the overlay.

The forwarding sub-layer provides the QoS-related functions the DetNet flow needs 🎀 It can do this directly with queuing techniques and traffic engineering methods, or it can do it with the help of its underlying connectivity. For example, it may call upon Ethernet TSN capabilities defined in IEEE 802.1 TSN [IEEE802.1TSNTG]. The forwarding sub-layer uses buffer resources for packet queuing, plus reservation and allocation of bandwidth capacity resources 📦

The service sub-layer adds support on top of the forwarding sub-layer's connectivity function. See Section 4.3 for more on PREOF ✨ The POF uses sequence numbers added to packets. That enables a range of packet order protection, from simple ordering and dropping of out-of-order packets, all the way to more complex reordering of a fixed number of out-of-order, minimally delayed packets. Reordering needs buffer resources and affects the delay and jitter of packets in the DetNet flow ⏳

The way each of the layers gets instantiated depends on the particular DetNet data plane method, and more than one approach may be applicable to a given network type 🌐

3.1. Data Plane Characteristics

Okay so the data plane has two major characteristics, and they're the technology and the encapsulation 💖 We'll break both of them down below, bestie.

3.1.1. Data Plane Technology ✨

Okay so the DetNet data plane is powered by two sub-layers, the DetNet service sub-layer and the DetNet forwarding sub-layer 💖 The service sub-layer usually handles its stuff for DetNet application flows by using or applying existing standardized headers and/or encapsulations. The forwarding sub-layer can use that same header or encapsulation tech too (think DN IP or DN MPLS), or it can get the job done with other tech, as shown in Figure 2 below 🦋 Right now, DetNet is defined to run over packet-switched (IP) networks or label-switched (MPLS) networks, so those are the main playgrounds 🌸

3.1.2. Encapsulation 🎀

DetNet encodes specific flow attributes (flow identity and sequence number) in packets. For example, in DetNet IP, zero encapsulation is used, and no sequence number is available 🙈 In DetNet MPLS, DetNet-specific information can be added explicitly to the packets in the form of an S-Label and a d-CW [DetNet-MPLS] 🧚

The encapsulation of a DetNet flow lets it travel over a data plane technology that isn't its native type, which is lowkey iconic 🌈 DetNet uses header information to do traffic classification (aka identifying DetNet flows) and to provide DetNet service and forwarding functions 💅 As mentioned above, DetNet may add headers, like it does for DN MPLS, or it may use headers that are already there, like it does for DN IP ✨ Here's how the pieces connect in Figure 2:

                                             +-----+
                                             | TSN |
                        +-------+          +-+-----+-+
                        | DN IP |          | DN MPLS |
                     +--+--+----+----+   +-+---+-----+-+
                     | TSN | DN MPLS |   | TSN | DN IP |
                     +-----+---------+   +-----+-------+

                     Figure 2: DetNet Service Examples

Encapsulation is also required if the DetNet data plane needs additional information (metadata) and either (1) there's no way to include it in the client data packet, or (2) the client data plane's specification doesn't allow modifying the packet to add extra data 😭 An example of this kind of metadata is a sequence number required by PREOF 🔥

Encapsulation may also be used to carry or aggregate flows for equipment with limited DetNet capability 🫶

3.2. DetNet-Specific Metadata 🍓

The DetNet data plane can provide or carry the following metadata:

  1. Flow-ID
  2. Sequence number

The DetNet data plane framework supports a Flow-ID (for identifying the flow or aggregate flow) and/or a sequence number (for PREOF) for each DetNet flow 💜 The Flow-ID is used by both the service and forwarding sub-layers, but the sequence number is only used by the service layer. Metadata can also be used for OAM indications and for instrumenting DetNet data plane operation 👑

Metadata inclusion can be implicit or explicit. Explicit inclusion means there's a dedicated header field that carries the metadata in a DetNet packet. With the implicit method, part of an already-existing header field gets used to encode the metadata 🦋

Explicit inclusion of metadata is possible through IP options or IP extension headers. New IP options are almost impossible to get standardized or deployed in an operational network, so they won't be discussed further in this text 💀 IPv6 extension headers are getting really popular in current IPv6 development work, especially alongside Segment Routing of IPv6 (SRv6) and IP OAM ✨ Designing a new IPv6 extension header, or modifying an existing one, is a technique available in the toolbox of the DetNet IP data plane designer 🛠️

Explicit inclusion of metadata in an IP packet is also possible by including an MPLS label stack and the MPLS d-CW, using one of the methods for carrying MPLS over IP [DetNet-MPLS-over-UDP-IP] 🌸 This is described in more detail in Section 3.5.5.

Implicit metadata in IP can be included through the network programming paradigm [SRv6-Network-Prog], where the suffix of an IPv6 address is used to encode additional information for use by the network of the receiving host 🧚

An MPLS example of explicit metadata is the sequence number used by PREOF, or even the case where all the essential information is included in the DetNet-over-MPLS label stack (the d-CW and the DetNet S-Label) 💖

3.3. DetNet IP Data Plane 🌈

An IP data plane may operate natively or through the use of an encapsulation. Many types of IP encapsulation can satisfy DetNet requirements, and it's anticipated that more than one encapsulation may be deployed, for example GRE and IPsec 💅

One method of operating an IP DetNet data plane without encapsulation is to use 6-tuple-based flow identification, where "6-tuple" refers to information carried in IP-layer and higher-layer protocol headers 🎀 General background on using IP headers and 6-tuples to identify flows and support QoS can be found in [RFC3670]. The extra field in the 6-tuple is the DSCP field in the packet. [RFC7657] gives useful background on differentiated services (Diffserv) and tuple-based flow identification 🦋 DetNet flow aggregation may be enabled through the use of wildcards, masks, prefixes, and ranges. The operation of this method is described in detail in [RFC8939] ✨

The DetNet forwarding plane may use explicit route capabilities and traffic engineering capabilities to provide a forwarding sub-layer that's responsible for resource allocation and explicit routes 🫶 It's possible to include such information in a native IP packet either explicitly or implicitly 🍓

3.4. DetNet MPLS Data Plane 🧚

MPLS provides a forwarding sub-layer for traffic over implicit and explicit paths, all the way to the point in the network where the next DetNet service sub-layer action needs to happen 🌸 It does this with a stack of one or more labels, each with various forwarding semantics.

MPLS also gives you the ability to identify a service instance that's used to process the packet, through a label that maps the packet to a service instance 💜

When metadata is needed to process an MPLS-encapsulated packet at the service sub-layer, the d-CW [DetNet-MPLS] can be used 👑 Although d-CWs are frequently 32 bits long, there's no architectural constraint on the size of this structure. The only requirement is that it be fully understood by all parties operating on it in the DetNet service sub-layer 🔥 The operation of this method is described in detail in [DetNet-MPLS].

3.5. Further DetNet Data Plane Considerations 💅

This section provides informative considerations related to providing DetNet service to flows that are identified based on their header information 🎀

3.5.1. Functions Provided on a Per-Flow Basis ✨

At a high level, the following functions are provided on a per-flow basis 🦋

3.5.1.1. Reservation and Allocation of Resources 🌈

Resources might be reserved so they're available for allocation to specific DetNet flows. This can eliminate packet contention and packet loss for DetNet traffic, and it can also reduce jitter for DetNet traffic 🫶 Resources allocated to a DetNet flow protect it from other traffic flows. On the other hand, it's assumed that DetNet flows behave in accordance with the reserved traffic profile. It must be possible to detect misbehaving DetNet flows and to ensure that they don't compromise the QoS of other flows 😭 Queuing, policing, and shaping policies can be used to make sure the allocation of resources reserved for DetNet is met 💖

3.5.1.2. Explicit Routes 🧚

A flow can be routed over a specific, precomputed path. This allows control of network delay by steering the packet, with the ability to influence the physical path 🌸 Explicit routes complement reservation by making sure a consistent path can be associated with its resources for the duration of that path. Coupled with the traffic mechanism, this limits misordering and bounds latency 💜 Explicit route computation can cover a wide set of constraints and can optimize the path for a certain characteristic, e.g., highest bandwidth or lowest jitter. In these cases, the "best" path for any set of characteristics may not be a shortest path 👑 The selection of the path can take multiple network metrics into account. Some of these metrics are measured and distributed by the routing system as traffic engineering metrics 🍓

3.5.1.3. Service Protection 🔥

Service protection involves using multiple packet streams over multiple paths, for example 1+1 or 1:1 linear protection. For DetNet, this primarily relates to packet replication and elimination capabilities 💅 MPLS offers a number of protection schemes. MPLS hitless protection can be used to switch traffic to an already-established path in order to restore delivery rapidly after a failure 🎀 Path changes, even during failure recovery, can lead to out-of-order delivery of data, requiring POFs either within the DetNet service or at a higher layer in the application traffic 🙈 Establishing new paths after a failure is out of scope for DetNet services 🛑

3.5.1.4. Network Coding 🌈

Network Coding [nwcrg], not to be confused with network programming, comprises several techniques where multiple data flows are encoded 🦋 These resulting flows can then be sent on different paths. The encoding operation can combine flows and error recovery information. When the encoded flows are decoded and recombined, the original flows can be recovered ✨ Note that Network Coding uses an alternative to packet-by-packet PREOF. Therefore, for certain network topologies and traffic loads, Network Coding can be used to improve a network's throughput, efficiency, latency, and scalability, as well as its resilience to partition, attacks, and eavesdropping, compared to traditional methods 🧚 DetNet could use Network Coding as an alternative to other means of protection. Network Coding is often applied in wireless networks and is being explored for other network types 💜

3.5.1.5. Load-Sharing 🫶

The use of packet-by-packet load-sharing of the same DetNet flow over multiple paths is not recommended, except for the cases listed above where PREOF are utilized to improve protection of traffic and maintain order 😭 Packet-by-packet load-sharing, e.g., via Equal-Cost Multipath (ECMP) or Unequal-Cost Multipath (UCMP), impacts ordering and, possibly, jitter 💀

3.5.1.6. Troubleshooting 🔍

DetNet leverages many different forwarding sub-layers, and each one supports various tools to troubleshoot connectivity, for example identifying misbehaving flows 🌸 The DetNet service layer can leverage existing mechanisms to troubleshoot or monitor flows, such as those used by IP and MPLS networks. At the Application layer, a client of a DetNet service can use existing techniques to detect and monitor delay and loss 💖

3.5.1.7. Flow Recognition for Analytics 📊

Network analytics can be inherited from the technologies of the service and forwarding sub-layers 🦋 At the DetNet service edge, packet and bit counters (e.g., sent, received, dropped, out of sequence) can be maintained ✨

3.5.1.8. Correlation of Events with Flows 👑

The provider of a DetNet service may offer other capabilities to monitor flows, such as more detailed loss statistics and timestamping of events 🌈 Details regarding these capabilities are out of scope for this document 🫶

3.5.2. Service Protection ✨

okay bestie, service protection is the glow-up that helps DetNet services stay reliable and keep their level of service assurance when the network gets congested or something fails 😭🛡️. DetNet doesn't do all of this alone. It relies on the underlying technology capabilities for its various protection schemes 💅. These schemes can cover part or all of the network paths, and they get active protection by combining PRF, PEF, and POF 🎀.

3.5.2.1. Linear Service Protection

okay so here's an example DetNet MPLS network fragment and its packet flow, shown in Figure 3 🦋.

            1      1.1       1.1      1.2.1    1.2.1      1.2.2
         CE1----EN1--------R1-------R2-------R3--------EN2-----CE2
                  \           1.2.1 /                  /
                   \1.2     /------+                  /
                    +------R4------------------------+
                              1.2.2

            Figure 3: Example of Packet Flow Protected by DetNet

in Figure 3, the numbers are used to track which instance of a packet we're looking at 🔢. Packet 1 is the original. Packets 1.1 and 1.2 are first-generation copies of packet 1, packet 1.2.1 is a second-generation copy of packet 1.2, and so on 🌸. Lowkey super important: these numbers never show up in the packet itself, and they're not sequence numbers, labels, or any other identifiers that live in the packet, so don't mix them up 🙅‍♀️. They just show the generation of the original packet so the reader can follow its trip through the fragment 💜.

CE1 sends a packet into the DetNet-enabled network 📦. That's packet 1. Edge Node EN1 encapsulates it as a DetNet packet and sends it to Relay Node R1 (packet 1.1) ✨. EN1 also makes a copy (1.2), encapsulates it, and sends that copy to Relay Node R4 👯.

R1 might be directly attached to EN1, or there could be one or more nodes on the path that just aren't drawn in Figure 3 for clarity 🗺️. Same goes for any other path between two DetNet entities in the figure 💁.

R4 is configured to send one copy to R2 (packet 1.2.1) and one copy to EN2 (packet 1.2.2) 📬.

R2 gets copy 1.2.1 before copy 1.1 shows up. Since it's set up to do packet elimination on this DetNet flow, it forwards 1.2.1 on to R3. Copy 1.1 isn't useful anymore, so R2 tosses it 🗑️.

EN2 gets copy 1.2.2 from R4 before it gets copy 1.2.1 from R2 via R3 🏃. So it strips any DetNet encapsulation off 1.2.2 and forwards the packet to CE2 🎁. When 1.2.1 finally shows up later on, EN2 discards it 🙃.

The above is obviously just illustrative, and there are loads of network scenarios you can configure 🌈.

This example also shows a 1:1 protection scheme, which means there's traffic over each segment of the end-to-end path 🫶. Local DetNet relay nodes decide which packets get eliminated and which get forwarded 💅. A 1+1 scheme, where only one path carries traffic at a time, could use the same topology too 🦋. In that case there's no PRF, and traffic gets switched when a failure is detected 😬. An OAM scheme that monitors the paths to detect the loss of a path or traffic is required to start the switch 🔍. A POF may still be used here to prevent misordering of packets 📦. In both cases, the protection paths are set up and kept around for the whole duration of the DetNet service ⏳.

3.5.2.2. Path Differential Delay

ngl this one is kinda a big deal 🥺. In the example above, proper duplicate elimination and packet reordering depend on how many out-of-order packets can be buffered and how different the arrival delays are ⏱️. DetNet uses flow-specific requirements (e.g., maximum number of out-of-order packets, maximum latency of the flow) to configure POF-related buffers 🧠. If the differential delay between paths is excessively large, or there's excessive misordering, packets may get dropped instead of reordered 💀. And the PEF uses the sequence number to spot duplicates, so big differential delays combined with high numbers of packets can exceed the PEF's ability to work properly 😭.

3.5.2.3. Ring Service Protection

Ring protection may also be supported if the underlying technology supports it 💍. A lot of the same concepts apply, but rings are normally 1+1 protection for data efficiency reasons 🧠. [RFC8227] gives an example of an MPLS Transport Profile (MPLS-TP) data plane that supports ring protection 🌀.

3.5.3. Aggregation Considerations

The DetNet data plane also allows aggregation of DetNet flows 📚, which can improve scalability by reducing the per-hop state 📉. How this is accomplished depends on the data plane or control plane 🤷. When DetNet flows are aggregated, transit nodes provide service to the aggregate and not on a per-DetNet-flow basis 🧺. When aggregating DetNet flows, the flows should be compatible, meaning the same or very similar QoS and CoS characteristics ✅. In this case, the nodes doing the aggregation will make sure per-flow service requirements are achieved 🫡.

If bandwidth reservations are used, the reservation should be the sum of all the individual reservations. In other words, the reservations should not add up to an oversubscription of bandwidth reservation 💸. If maximum delay bounds are used, the system should make sure the aggregate doesn't exceed the delay bounds of the individual flows ⏰.

When an encapsulation is used, choosing between reserving a maximum resource level and tracking the services in the aggregated service, or adjusting the aggregated resources as services are added, is implementation and technology specific 🛠️.

DetNet flows at edges must be able to handle rejection to an aggregation group due to lack of resources, as well as conditions where requirements are not satisfied 🚪. Be ready for the no-thank-yous, bestie 🥲.

3.5.3.1. IP Aggregation

IP aggregation has both data plane and Controller Plane aspects 🌐. For the data plane, flows may be aggregated for treatment based on shared characteristics such as 6-tuple [RFC8939]. Alternatively, an IP encapsulation may be used to tunnel an aggregate number of DetNet flows between relay nodes 🚇.

3.5.3.2. MPLS Aggregation

MPLS aggregation also has data plane and Controller Plane aspects 💜. MPLS flows are often tunneled in a forwarding sub-layer, under the reservation associated with that MPLS tunnel 🎀.

3.5.4. End-System-Specific Considerations

Data flows that need DetNet service are generated and terminated on end systems 💻. Encapsulation depends on the application and its preferences 🎨. For example, in a DetNet MPLS domain, the sub-layer functions use the d-CWs, S-Labels, and F-Labels [DetNet-MPLS] to provide DetNet services ✨. However, an application may exchange further flow-related parameters (e.g., timestamps) that are not provided by DetNet functions ⏱️.

As a general rule, DetNet domains are capable of forwarding any DetNet flows, and the DetNet domain doesn't mandate the encapsulation format for end systems or edge nodes 🙌. Unless some form of proxy is present, end systems peer with similar end systems using the same application encapsulation format 🤝. As shown in Figure 4, IP applications peer with IP applications, and Ethernet applications peer with Ethernet applications 👯‍♀️.

          +-----+
          |  X  |                               +-----+
          +-----+                               |  X  |
          | Eth |               ________        +-----+
          +-----+     _____    /        \       | Eth |
                 \   /     \__/          \___   +-----+
                  \ /                        \ /
                   0======== tunnel-1 ========0_
                   |                             \
                    \                             |
                    0========= tunnel-2 =========0
                   / \                        __/ \
            +-----+   \__ DetNet MPLS domain /     \
            |  X  |      \         __       /       +-----+
            +-----+       \_______/  \_____/        |  X  |
            |  IP |                                 +-----+
            +-----+                                 |  IP |
                                                    +-----+

           Figure 4: End Systems and the DetNet MPLS Domain

3.5.5. Sub-network Considerations

Any of the DetNet service types may be transported by another DetNet service 🪆. MPLS nodes may be interconnected by different sub-network technologies, which may include point-to-point links 🔗. Each of these sub-network technologies needs to provide appropriate service to DetNet flows 💖. In some cases, e.g., on dedicated point-to-point links or TDM technologies, all that is required is for a DetNet node to appropriately queue its output traffic 📤. In other cases, DetNet nodes will need to map DetNet flows to the flow semantics (i.e., identifiers) and mechanisms used by an underlying sub-network technology 🗺️. Figure 5 shows several examples of sub-network encapsulations that can carry DetNet MPLS flows over different sub-network technologies 🌈. L2 is a generic Layer 2 encapsulation that might be used on a point-to-point link 🧃. TSN is the encapsulation used on an IEEE 802.1 TSN network, as described in [DetNet-MPLS-over-TSN] ⏩. UDP/IP is the encapsulation used on a DetNet IP PSN, as described in [DetNet-MPLS-over-UDP-IP] 💫.

                           +------+  +------+  +------+
        App-flow           |  X   |  |  X   |  |  X   |
                     +-----+======+--+======+--+======+-----+
        DetNet-MPLS        | d-CW |  | d-CW |  | d-CW |
                           +------+  +------+  +------+
                           |Labels|  |Labels|  |Labels|
                     +-----+======+--+======+--+======+-----+
        Sub-network        |  L2  |  | TSN  |  | UDP  |
                           +------+  +------+  +------+
                                               |  IP  |
                                               +------+
                                               |  L2  |
                                               +------+

     Figure 5: Example DetNet MPLS Encapsulations in Sub-networks

4. Controller Plane (Management and Control) Considerations ✨

4.1. DetNet Controller Plane Requirements 🎀

okay so the Controller Plane is basically the Control and Management Planes from [RFC7426] and [RFC8655] squished together 💅. bestie, the full details of any DetNet Controller Plane are out of scope for this doc, BUT there are still special considerations and requirements for it because of how unique the DetNet architecture and data plane are 🦋✨

The main requirements of the DetNet Controller Plane are that it must be able to:

  • Instantiate DetNet flows in a DetNet domain (which may, for example, include some or all of the following: explicit path determination, link bandwidth reservations, restricting flows to IEEE 802.1 TSN links, node buffer and other resource reservations, specification of required queuing disciplines along the path, ability to manage bidirectional flows, etc.) as needed for a flow. 🌸
  • In the case of MPLS, manage DetNet S-Label and F-Label allocation and distribution. In cases where the DetNet MPLS encapsulation is being used, see [DetNet-MPLS]. 🧚
  • Support DetNet flow aggregation. 💜
  • Advertise static and dynamic node and link resources such as capabilities and adjacencies to other network nodes (for dynamic signaling approaches) or to network controllers (for centralized approaches). 📣
  • Scale to handle the number of DetNet flows expected in a domain (which may require per-flow signaling or provisioning). 📈
  • Provision flow identification information at each of the nodes along the path. Flow identification may differ, depending on the location in the network and the DetNet functionality (e.g., transit node vs. relay node). 🍓

bestie, these requirements, as stated earlier, could be satisfied using distributed control protocol signaling (such as RSVP-TE), centralized network management provisioning mechanisms (BGP, PCEP, YANG, [DetNet-Flow-Info], etc.), or hybrid combinations of the two, and they could also make use of MPLS-based segment routing 🔥

In the abstract, the results of either distributed signaling or centralized provisioning are equivalent from a DetNet data plane perspective. Flows get instantiated, explicit routes get determined, resources get reserved, and packets get forwarded through the domain using the DetNet data plane. It's giving clean and simple 😌✨

However, from a practical and implementation standpoint, Controller Plane alternatives are not equivalent at all 🙈 Some approaches are more scalable than others in terms of signaling load on the network. Some alternatives can take advantage of global tracking of resources in the DetNet domain for better overall network resource optimization. Some solutions are more resilient than others if link, node, or management equipment failures occur. A detailed analysis of the control plane alternatives is out of scope for this document, but the requirements from this document can be used as the basis of a future analysis of the alternatives. Future-you will thank present-you 💖

4.2. Generic Controller Plane Considerations 🌈

okay, this section covers control plane considerations that are independent of the data plane technology used for DetNet service delivery 🧚

bestie, the management plane and the control plane are traditionally considered separately, but from a data plane perspective there is no practical difference based on where the flow-provisioning information comes from. That's why the DetNet architecture [RFC8655] refers to these collectively as the "Controller Plane" 🫶 This document therefore does not distinguish between information provided by distributed control plane protocols (e.g., RSVP-TE [RFC3209] [RFC3473]) or centralized network management mechanisms (e.g., RESTCONF [RFC8040], YANG [RFC7950], PCEP [PCECC]), or any combination thereof. Specific considerations and requirements for the DetNet Controller Plane are discussed in Section 4.1.

Each respective data plane document also covers the control plane considerations for that technology. For example, [RFC8939] also covers IP control plane normative considerations, and [DetNet-MPLS] also covers MPLS control plane normative considerations. Everyone has their own lane, and that's so iconic 👑

4.2.1. Flow Aggregation Control 🍓

okay so flow aggregation means that multiple App-flows are served by a single new DetNet flow. There are many techniques to achieve aggregation. For example, in the case of IP, IP flows that share 6-tuple attributes or flow identifiers at the DetNet sub-layer can be grouped. Another example includes aggregation accomplished through the use of hierarchical LSPs in MPLS and tunnels. Main character energy of bundling 🎀

Control of aggregation involves a set of procedures listed here. Aggregation may use some or all of these capabilities, and the order may vary:

  • Traffic engineering resource collection and distribution: Available resources are tracked through control plane or management plane databases and distributed amongst controllers or nodes that can manage resources. 📊
  • Path computation and resource allocation: When DetNet services are provisioned or requested, one or more paths meeting the requirements are selected and the resources verified and recorded. 🗺️
  • Resource assignment and data plane coordination: The assignment of resources along the path depends on the technology and includes assignment of specific links, coordination of queuing, and other traffic management capabilities such as policing and shaping. 🚦
  • Assigned resource recording and updating: Depending on the specific technology, the assigned resources are updated and distributed in the databases, preventing oversubscription. 🛑

4.2.2. Explicit Routes 💅

bestie, explicit routes are used to ensure that packets are routed through the resources that have been reserved for them, and hence provide the DetNet application with the required service. A requirement for the DetNet Controller Plane will be the ability to assign a particular identified DetNet IP flow to a path through the DetNet domain that has been assigned the required per-node resources. This provides the appropriate traffic treatment for the flow and also includes particular links as a part of the path that are able to support the DetNet flow. For example, by using IEEE 802.1 TSN links (as discussed in [DetNet-MPLS-over-TSN]), DetNet parameters can be maintained. Further considerations and requirements for the DetNet Controller Plane are discussed in Section 4.1.

Whether configuring, calculating, and instantiating these routes is a single-stage or multi-stage process, or is performed in a centralized or distributed manner, is out of scope for this document. 🙃

There are several approaches that could be used to provide explicit routes and resource allocation in the DetNet forwarding sub-layer. For example:

  • The path could be explicitly set up by a controller that calculates the path and explicitly configures each node along that path with the appropriate forwarding and resource allocation information. 🎯
  • The path could use a distributed control plane such as RSVP [RFC2205] or RSVP-TE [RFC3473] extended to support DetNet IP flows. 🛤️
  • The path could be implemented using IPv6-based segment routing when extended to support resource allocation. 🧭

See Section 4.1 for further discussion of these alternatives. In addition, [RFC2386] contains useful background information on QoS-based routing, and [RFC5575] (which will be updated by [Flow-Spec-Rules]) discusses a specific mechanism used by BGP for traffic flow specification and policy-based routing. Lots of reading, but it's worth it 📚💜

4.2.3. Contention Loss and Jitter Reduction ⭐

okay so this document does not specify the mechanisms needed to eliminate packet contention or packet loss, or to reduce jitter, for DetNet flows at the DetNet forwarding sub-layer 🙅‍♀️ The ability to manage node and link resources to be able to provide these functions is a necessary part of the DetNet Controller Plane. It is also necessary to be able to control the required queuing mechanisms used to provide these functions along a flow's path through the network. See [RFC8939] and Section 4.1 for further discussion of these requirements. Some forms of protection may minimize packet loss or change jitter characteristics in the cases where packets are reordered, when out-of-order packets are received at the service sub-layer. 🌊

4.2.4. Bidirectional Traffic 🦋

bestie, in many cases DetNet flows can be considered unidirectional and independent. However, there are cases where the DetNet service requires bidirectional traffic from a DetNet application service perspective. IP and MPLS typically treat each direction separately and do not force interdependence of each direction. The IETF MPLS Working Group has studied bidirectional traffic requirements. The definitions provided in [RFC5654] are useful to illustrate terms such as associated bidirectional flows and co-routed bidirectional flows.

MPLS defines a point-to-point associated bidirectional LSP as consisting of two unidirectional point-to-point LSPs, one from A to B and the other from B to A, which are regarded as providing a single logical bidirectional forwarding path. This is analogous to standard IP routing. MPLS defines a point-to-point co-routed bidirectional LSP as an associated bidirectional LSP that satisfies the additional constraint that its two unidirectional component LSPs follow the same path (in terms of both nodes and links) in both directions. An important property of co-routed bidirectional LSPs is that their unidirectional component LSPs share fate. In both types of bidirectional LSPs, resource reservations may differ in each direction. The concepts of associated bidirectional flows and co-routed bidirectional flows can also be applied to DetNet IP flows. ✨

While the DetNet IP data plane must support bidirectional DetNet flows, there are no special bidirectional features with respect to the data plane other than the need for the two directions of a co-routed bidirectional flow to take the same path. That is to say, bidirectional DetNet flows are solely represented at the management plane and control plane levels, without specific support or knowledge within the DetNet data plane. Fate-sharing and associated or co-routed bidirectional flows can be managed at the control level. 🧠

DetNet's use of PREOF may increase the complexity of using co-routed bidirectional flows, because if PREOF are used, the replication points in one direction would have to match the elimination points in the other direction, and vice versa. In such cases, the optimal points for these functions in one direction may not match the optimal points in the other, due to network and traffic constraints. Furthermore, due to the per-packet service protection nature, bidirectional forwarding may not be ensured. The first packet of received member flows is selected by the elimination function independently of which path it has taken through the network. Plot twist energy 😭🌀

Control and management mechanisms need to support bidirectional flows, but the specification of such mechanisms is out of scope for this document. Example control plane solutions for MPLS can be found in [RFC3473], [RFC6387], and [RFC7551]. These requirements are included in Section 4.1. 💖

4.3. Packet Replication, Elimination, and Ordering Functions (PREOF) ✨

Okay so, the Controller Plane protocol you need to manage PREOF is totally out of scope for this doc 🙈 But here's the tea: figuring out the best packet replication and elimination points for a particular flow inside the DetNet domain requires explicit support 🔥 There might be existing capabilities you can reuse or extend, like GMPLS end-to-end recovery [RFC4872] and GMPLS segment recovery [RFC4873] 💅

5. Security Considerations 🔐

Bestie, the security stuff for DetNet is covered in full detail in [DetNet-Security] 🫶 The general security considerations for the DetNet architecture live in [RFC8655]. This section looks at the architecture-level DetNet security considerations that apply to all data planes 🦋

Part of what makes DetNet so special is that it gives you specific, reliable QoS, meaning data flows with extremely low packet loss rates and bounded end-to-end delivery latency ✨ The security side of protecting that QoS is just as unique 😭

Like every communications protocol, the main thing for the data plane is keeping the data intact and keeping the DetNet service flowing across the DetNet network 🔥 Application flows can be protected by whatever the underlying technology provides. For example, encryption may be used, like the kind IPsec [RFC4301] gives you for IP flows, and/or the kind an underlying sub-network gives you with MACsec [IEEE802.1AE-2018] for Ethernet (Layer 2) flows 🔐

At the management and control levels, DetNet flows are identified on a per-flow basis. That can give Controller Plane attackers more info about the data flows, compared to Controller Planes that don't do per-flow identification 👀 This is an inherent property of DetNet, and it has security implications you should think about when deciding if DetNet is the right tech for a given use case 💀

To keep the DetNet service uninterrupted, you can make provisions against DoS attacks and delay attacks 🛡️ To protect against DoS attacks, you can prevent or mitigate excess traffic caused by malicious or malfunctioning devices, for example with existing mechanisms like policing and shaping applied at the input of a DetNet domain 💖 To stop DetNet packets from being delayed by an entity outside the DetNet domain, DetNet technology definitions can allow for mitigating man-in-the-middle attacks. One example is authentication and authorization of devices within the DetNet domain 🙅‍♀️

To prevent or mitigate DetNet attacks on other networks via flow escape, edge devices can, for example, use existing mechanisms like policing and shaping applied at the output of a DetNet domain 🌸

Okay, real talk: flow escape is a sneaky one, so don't skip the edge-device protections! 😤

6. IANA Considerations 🎀

This document has no IANA actions. Nothing to register here, babe 💅

7. References 📚

7.1. Normative References

7.2. Informative References

Acknowledgements ✨

Okay, big thank-you bestie moment 🫶 The authors want to shout out Pat Thaler, Norman Finn, Loa Andersson, David Black, Rodney Cummings, Ethan Grossman, Tal Mizrahi, David Mozes, Craig Gunther, George Swallow, Yuanlong Jiang, and Carlos J. Bernardos for all their help with this work 💜 Honestly this list is giving legends only 👑

Contributors 🌸

These people helped out a LOT with the actual content of this doc, so they get the spotlight too ✨

  • Don Fedyk
  • Jouni Korhonen

Authors' Addresses 🦋

Here's where you can find the authors, editor included 💅 Balázs Varga is the editor, and he's at Ericsson in Budapest 🇭🇺 Janos Farkas is also at Ericsson in Budapest. Lou Berger is at LabN Consulting, L.L.C., Andrew G. Malis is at Malis Consulting, and Stewart Bryant is at Futurewei Technologies 🧚 Their addresses are below, copied exactly so nothing gets lost in translation 🎀

Balázs Varga (editor)
Ericsson
Budapest
Magyar Tudosok krt. 11.
1117
Hungary

Email: balazs.a.varga@ericsson.com


János Farkas
Ericsson
Budapest
Magyar Tudosok krt. 11.
1117
Hungary

Email: janos.farkas@ericsson.com


Lou Berger
LabN Consulting, L.L.C.

Email: lberger@labn.net


Andrew G. Malis
Malis Consulting

Email: agmalis@gmail.com


Stewart Bryant
Futurewei Technologies

Email: sb@stewartbryant.com
📦 where this page comes from