RFC 5977
RMD-QOSM: The NSIS Quality-of-Service Model for Resource Management in Diffserv
Title Block 🎀
RFC 5977 is called RMD-QOSM: The NSIS Quality-of-Service Model for Resource Management in Diffserv. The authors are A. Bader and L. Westberg (Ericsson), G. Karagiannis (University of Twente), C. Kappler (ck technology concepts), and T. Phelan (Sonus). It's an Experimental doc with ISSN 2070-1721, dated October 2010. The header doesn't list anything it obsoletes or updates, so there's no drama there 🫶
Abstract 🦋
okay so this doc is all about an NSIS Quality-of-Service (QoS) Model for networks that use Resource Management in Diffserv (RMD) 🌸 RMD is basically a way to bolt admission control and preemption onto Differentiated Services (Diffserv) networks, and it's giving main character energy ✨
The RMD QoS Model lets devices sitting outside the RMD network send reservation requests to the Edge nodes inside it. The RMD Ingress Edge nodes sort incoming flows into traffic classes, then send resource requests for that class along the data path to the Egress Edge nodes, one for each flow. The Egress nodes rebuild the original requests and keep passing them along toward the final destination 💅 RMD also defines notification functions that tell the Edge nodes when the domain is overloaded, which is honestly iconic of it 🔥
Status of This Memo 📜
ngl this doc is not an Internet Standards Track spec 🍓 It's published so people can look at it, try out experimental implementations, and evaluate it 🧚
This document defines an Experimental Protocol for the Internet community. It's a product of the Internet Engineering Task Force (IETF) and represents the consensus of the IETF community 💜 It got public review and was approved for publication by the Internet Engineering Steering Group (IESG). Not every doc the IESG approves is a candidate for any level of Internet Standard, so check Section 2 of RFC 5741 if you want the details 🙈
For info on the current status, any errata, and how to give feedback, look at http://www.rfc-editor.org/info/rfc5977 🌈
Copyright Notice 💅
Copyright (c) 2010 IETF Trust and the persons identified as the document authors. All rights reserved. 🫶
This doc is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents, found at http://trustee.ietf.org/license-info, as they were on the date it was published 📜 Read those carefully, bestie, because they spell out your rights and restrictions for this document. Code Components pulled out of this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they come with no warranty, as described in the Simplified BSD License 👑
Table of Contents 🧚
1. Introduction ....................................................4
2. Terminology .....................................................6
3. Overview of RMD and RMD-QOSM ....................................7
3.1. RMD ........................................................7
3.2. Basic Features of RMD-QOSM ................................10
3.2.1. Role of the QNEs ...................................10
3.2.2. RMD-QOSM/QoS-NSLP Signaling ........................11
3.2.3. RMD-QOSM Applicability and Considerations ..........13
4. RMD-QOSM, Detailed Description .................................15
4.1. RMD-QSPEC Definition ......................................16
4.1.1. RMD-QOSM <QoS Desired> and <QoS Reserved> ..........16
4.1.2. PHR Container ......................................17
4.1.3. PDR Container ......................................20
4.2. Message Format ............................................23
4.3. RMD Node State Management .................................23
4.3.1. Aggregated Operational and Reservation
States at the QNE Edges ............................23
4.3.2. Measurement-Based Method ...........................25
4.3.3. Reservation-Based Method ...........................27
4.4. Transport of RMD-QOSM Messages ............................28
4.5. Edge Discovery and Message Addressing .....................31
4.6. Operation and Sequence of Events ..........................32
4.6.1. Basic Unidirectional Operation .....................32
4.6.1.1. Successful Reservation ....................34
4.6.1.2. Unsuccessful Reservation ..................46
4.6.1.3. RMD Refresh Reservation ...................50
4.6.1.4. RMD Modification of Aggregated
Reservations ..............................54
4.6.1.5. RMD Release Procedure .....................55
4.6.1.6. Severe Congestion Handling ................64
4.6.1.7. Admission Control Using Congestion
Notification Based on Probing .............70
4.6.2. Bidirectional Operation ............................73
4.6.2.1. Successful and Unsuccessful Reservations ..77
4.6.2.2. Refresh Reservations ......................82
4.6.2.3. Modification of Aggregated Intra-Domain
QoS-NSLP Operational Reservation States ...82
4.6.2.4. Release Procedure .........................83
4.6.2.5. Severe Congestion Handling ................84
4.6.2.6. Admission Control Using Congestion
Notification Based on Probing .............87
4.7. Handling of Additional Errors .............................89
5. Security Considerations ........................................89
5.1. Introduction ..............................................89
5.2. Security Threats ..........................................91
5.2.1. On-Path Adversary ..................................92
5.2.2. Off-Path Adversary .................................94
5.3. Security Requirements .....................................94
5.4. Security Mechanisms .......................................94
6. IANA Considerations ............................................97
6.1. Assignment of QSPEC Parameter IDs .........................97
7. Acknowledgments ................................................97
8. References .....................................................97
8.1. Normative References ......................................97
8.2. Informative References ....................................98
Appendix A. Examples .............................................101
A.1. Example of a Re-Marking Operation during Severe
Congestion in the Interior Nodes .........................101
A.2. Example of a Detailed Severe Congestion Operation in the
Egress Nodes .............................................107
A.3. Example of a Detailed Re-Marking Admission Control
(Congestion Notification) Operation in Interior Nodes ....111
A.4. Example of a Detailed Admission Control (Congestion
Notification) Operation in Egress Nodes ..................112
A.5. Example of Selecting Bidirectional Flows for Termination
during Severe Congestion .................................113
A.6. Example of a Severe Congestion Solution for
Bidirectional Flows Congested Simultaneously on Forward
and Reverse Paths ........................................113
A.7. Example of Preemption Handling during Admission Control ..117
A.8. Example of a Retransmission Procedure within the RMD
Domain ...................................................120
A.9. Example on Matching the Initiator QSPEC to the Local
RMD-QSPEC ................................................1221. Introduction ✨
okay so this doc describes a Next Steps in Signaling (NSIS) QoS Model for networks that use the Resource Management in Diffserv (RMD) framework ([RMD1], [RMD2], [RMD3], and [RMD4]). RMD adds admission control to Diffserv networks and lets nodes outside the networks dynamically reserve resources inside the Diffserv domains 🎀
the Quality-of-Service NSIS Signaling Layer Protocol (QoS-NSLP) [RFC5974] is the generic protocol for carrying QoS signaling info end-to-end in an IP network 🦋 every network along the end-to-end path is expected to implement a specific QoS Model (QOSM) defined by the QSPEC template [RFC5975]. that QOSM reads the requests and installs the right mechanisms for whatever technology that network is running, so the requested QoS actually gets delivered 💅 this document specifies an NSIS QoS Model for RMD networks (RMD-QOSM), plus an RMD-specific QSPEC (RMD-QSPEC) that writes reservations in a form that's easy for internal nodes to process 🧚
they're used together with the QoS-NSLP to provide QoS signaling service in an RMD network. here's Figure 1 showing an RMD network and all the entities in it 👑
Stateless or reduced-state Egress
Ingress RMD Nodes Node
Node (Interior Nodes; I-Nodes) (Stateful
(Stateful | | | RMD QoS
RMD QoS-NLSP | | | NSLP Node)
Node) V V V
+-------+ Data +------+ +------+ +------+ +------+
|-------|--------|------|------|------|-------|------|---->|------|
| | Flow | | | | | | | |
|Ingress| |I-Node| |I-Node| |I-Node| |Egress|
| | | | | | | | | |
+-------+ +------+ +------+ +------+ +------+
=================================================>
<=================================================
Signaling Flow
Figure 1: Actors in the RMD-QOSMmany network scenarios, like the "Wired Part of Wireless Network" scenario described in Section 8.4 of [RFC3726], need the QoS signaling protocol to have as little impact on network performance as possible 🌸 in those setups, every node on a communication path affects the end-to-end performance, so making the Interior nodes faster also makes the whole path faster ⭐ two things that help with this are cutting down the QoS signaling protocol processing load and cutting down how many states each Interior node has to keep 🫶
there's another requirement too: when a severe congestion situation shows up in the network, the QoS signaling protocol should be able to handle it 😭 a route change or a link failure can cause severe congestion. normally routing algorithms adapt and change their routing decisions to match changes in topology and traffic volume 🌈 but in these situations the rerouted traffic has to take a new path, and Interior nodes on that new path might get overloaded, because they suddenly have to carry more traffic than their capacity 💀 those severe congestion situations really hurt the overall performance of traffic passing through those nodes 🙈
RMD-QOSM is an edge-to-edge (intra-domain) QoS Model. together with the QoS-NSLP and QSPEC specs, it's built to hit the requirements above:
- Minimal impact on Interior node performance;
- Increase of scalability;
- Ability to deal with severe congestion
internally in the RMD network, RMD-QOSM plus QoS-NSLP [RFC5974] makes a scalable QoS signaling model. per-flow QoS-NSLP and NSIS Transport Layer Protocol (NTLP) states are not stored in Interior nodes, and per-flow signaling is handled at the Edges instead (see [RFC5974]) 🦋✨
in the RMD-QOSM, only routers at the Edges of a Diffserv domain (Ingress and Egress nodes) support the (QoS-NSLP) stateful operation; see Section 4.7 of [RFC5974]. Interior nodes support either the (QoS-NSLP) stateless operation or a reduced-state operation with coarser granularity than the Edge nodes 💜
after the terminology in Section 2, Section 3 gives an overview of RMD and the RMD-QOSM. this doc specifies several RMD-QOSM/QoS-NSLP signaling schemes, and Section 3.2.3 tells you which combination of sections is used to specify each one 🍓 Section 4 goes into detail on the RMD-QOSM: the role of QoS NSIS entities (QNEs), the definition of the QSPEC, how QSPEC generic parameters map onto RMD-QOSM parameters, state management in QNEs, and operation and sequence of events 🎀 and Section 5 covers security issues 🔥
2. Terminology 📖
the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. these are the official BCP 14 words, so read them carefully 💅
the terminology defined by GIST [RFC5971] and QoS-NSLP [RFC5974] applies to this document too 🧚
in addition, these terms show up:
NSIS domain: an NSIS signaling-capable domain.
RMD domain: an NSIS domain that is capable of supporting the RMD-QOSM signaling and operations.
Edge node: a QoS-NSLP node on the boundary of some administrative domain that connects one NSIS domain to a node in either another NSIS domain or a non-NSIS domain.
NSIS-aware node: a node that is aware of NSIS signaling and RMD-QOSM operations, such as severe congestion detection and Differentiated Service Code Point (DSCP) marking.
NSIS-unaware node: a node that is unaware of NSIS signaling, but is aware of RMD-QOSM operations such as severe congestion detection and DSCP marking.
Ingress node: an Edge node in its role in handling the traffic as it enters the NSIS domain.
Egress node: an Edge node in its role in handling the traffic as it leaves the NSIS domain.
Interior node: a node in an NSIS domain that is not an Edge node.
Congestion: a temporal network state that occurs when the traffic (or when traffic associated with a particular Per-Hop Behavior (PHB)) passing through a link is slightly higher than the capacity allocated for the link (or allocated for the particular PHB). if no measures are taken, the traffic passing through this link may temporarily degrade in QoS slightly. this kind of congestion is usually solved using admission control mechanisms 🌷
Severe congestion: the congestion situation on a particular link within the RMD domain where a significant increase in its real packet queue situation occurs, such as when due to a link failure rerouted traffic has to be supported by this particular link. it's giving overloaded link 🚨
3. Overview of RMD and RMD-QOSM 🦄
3.1. RMD 💖
Okay bestie, so the Differentiated Services (Diffserv) architecture ([RFC2475], [RFC2638]) showed up as the fix for the scalability and complexity headaches of IntServ [RFC1633] 🦋 Diffserv handles services on an aggregate basis instead of per flow, and it pushes as much per-flow state as it can out to the Edges of the network 🧚 Service differentiation runs on the Differentiated Services (DS) field in the IP header plus the Per-Hop Behavior (PHB), and every node handles a packet according to the PHB that the DS field in the message header points to 💅
Here's the tea, though: Diffserv doesn't give devices outside the domain any way to dynamically reserve resources or find out how much network capacity is free 🙈 In practice, service providers lean on short-lived Service Level Agreements (SLAs) that statically define the parameters of the traffic they'll accept from a customer. Very rigid, not very cute 😭
That's where RMD comes in 🎀 RMD was introduced as a method for dynamic reservation of resources inside a Diffserv domain. It provides admission control for flows entering the domain, plus a congestion-handling algorithm that can terminate flows when congestion happens because of a sudden failure (for example, a link or router) inside the domain 🔥
Now the iconic part: RMD gets its scalability by splitting a fine-grained reservation mechanism, used at the Edge nodes of a Diffserv domain, from a much simpler reservation mechanism used at the Interior nodes ✨ Edge nodes are generally assumed to support per-flow QoS states so they can guarantee QoS for each flow. Interior nodes keep just one aggregated reservation state per traffic class, or no state at all 🌸 Because the Interior nodes have so little to track, they can handle huge numbers of flows, and their limited functionality means signaling messages get processed super fast 🏃♀️💨
The possible RMD-QOSM applicabilities are described in Section 3.2.3. RMD supports two main basic admission control modes: reservation-based, and measurement-based, which can be combined with a severe congestion-handling solution 🫶 That severe congestion-handling solution kicks in when a link or node gets badly congested, which happens when the traffic from a failed link or node gets rerouted and has to run through this one 😬 RMD-QOSM also supports both unidirectional and bidirectional reservations 💜
Another major feature: the intra-domain sessions the Edges support can be per-flow sessions or per-aggregate sessions 👑 With per-flow intra-domain sessions, the per-flow intra-domain states have a one-to-one relationship with the per-flow end-to-end states that the same Edge supports. With per-aggregate sessions, the per-aggregate states have a one-to-many relationship with the per-flow end-to-end states that the same Edge supports 🍓
In the reservation-based method, each Interior node keeps only one reservation state per traffic class 🦋 The Ingress Edge nodes bundle individual flow requests into PHB traffic classes, and they signal changes to the class reservations whenever they're needed. The reservation is measured in resource units (or bandwidth). These resources are requested dynamically per PHB and reserved on demand at every node on the path from an Ingress node to an Egress node 🌈
The measurement-based algorithm keeps constantly measuring traffic levels and the resources that are actually available, then admits any flow whose resource needs fit within what's free at the moment of the request 📊 It backs a predictive service, where the service commitment is a little less reliable than what the reservation-based method can deliver 🙈
The big assumption behind these measurement-based admission control mechanisms is that the aggregated PHB traffic going through an RMD Interior node is high, so the current measurements are treated as a hint about future load 💅 Once an admission decision is made, the Interior nodes don't need to keep any record of it ✨ The perk of measurement-based resource management is that it doesn't need pre-reservation state or explicit release of reservations at the Interior nodes 🧚 And when user traffic is variable, measurement-based admission control can reach higher network utilization than something like peak-rate reservation, e.g. 🍓 The catch is that this can make the availability of resources a bit uncertain 😭 It's really important to flag that the RMD measurement-based schemes in this document do not use any refresh procedures, because these approaches run on stateless nodes; see Section 4.6.1.3 🫶
There are two types of measurement-based admission control schemes:
- Congestion notification function based on probing:
This method can be used to implement a simple measurement-based admission control within a Diffserv domain 🌸 In this setup, the Interior nodes are not NSIS-aware nodes. Thresholds get set on those Interior nodes for the traffic belonging to different PHBs in the measurement-based admission control function. An end-to-end NSIS message acts as a probe packet, which means the <DSCP> field in the header of the IP packet carrying the NSIS message gets re-marked when the predefined congestion threshold is exceeded 🔥 Note that once that threshold is exceeded, all packets get re-marked by a node, NSIS messages included 🙈 That's how the Edges can admit or reject flows that are asking for resources ✨ How often and how long the congestion level stays above the threshold (the stretch that causes re-marking) is tracked and used to influence the admission control decisions 📊
- NSIS measurement-based admission control:
In this case, the measurement-based admission control functionality lives in NSIS-aware stateless routers 💜 The big difference from congestion notification based on probing is that this type of admission control is applied mainly on NSIS-aware nodes 👑 With the measurement-based scheme, the admission control request carries the requested peak bandwidth of a flow 🍓 The admission decision is positive if the traffic currently being carried, as characterized by the measured statistics, plus the requested resources for the new flow, exceeds the system capacity with a probability smaller than a value alpha 🦋 Otherwise, the decision is negative. Here's the key bit: because the RMD Interior nodes are stateless, they don't store any information about previous admission control requests 🌈
Heads up: this can make admission control less accurate when multiple simultaneous flows that share a common Interior node ask for admission at the same time 😬
Applying measuring techniques, e.g., see [JaSh97] and [GrTs03], can limit that drop in accuracy 🫶 Those techniques use current and past information about NSIS sessions that requested resources from an NSIS-aware Interior node 🧚
RMD describes the following procedures:
- classification of an individual resource reservation or a resource query into Per-Hop Behavior (PHB) groups at the Ingress node of the domain,
- hop-by-hop admission control based on a PHB within the domain. There are two possible modes of operation for internal nodes to admit requests. One mode is the stateless or measurement-based mode, where the resources within the domain are queried. Another mode of operation is the reduced-state reservation or reservation-based mode, where the resources within the domain are reserved 💖
- a method to forward the original requests across the domain up to the Egress node and beyond,
- a congestion-control algorithm that notifies the Egress Edge nodes about congestion. It can terminate the appropriate number of flows in the case a of congestion due to a sudden failure (e.g., link or router failure) within the domain 🔥
3.2. Basic Features of RMD-QOSM 🌸
3.2.1. Role of the QNEs 🧚
Let's look at the protocol model of RMD-QOSM, shown in Figure 2 👑 The figure shows the QoS NSIS initiator (QNI) and QoS NSIS Receiver (QNR) nodes. They aren't part of the RMD network, but they're the ultimate initiator and receiver of the QoS reservation requests 💅 It also shows QNE nodes that are the Ingress and Egress nodes in the RMD domain (QNE Ingress and QNE Egress), plus QNE nodes that are Interior nodes (QNE Interior) ✨
All nodes of the RMD domain are usually QoS-NSLP-aware nodes 💖 But in scenarios where the congestion notification function based on probing is used, the Interior nodes are not NSIS aware 🙈 Edge nodes store and maintain QoS-NSLP and NTLP states, so they're stateful nodes 🫶 The NSIS-aware Interior nodes are NTLP stateless. On top of that, they're either QoS-NSLP stateless (for NSIS measurement-based operation) or reduced-state nodes that store per-PHB aggregated QoS-NSLP states (for reservation-based operation) 🦋
Note that the RMD domain MAY contain Interior nodes that are not NSIS-aware nodes (not shown in the figure) 🌈
These nodes are assumed to have sufficient capacity for the flows that might be admitted 💜 Some of these NSIS-unaware nodes MAY be used for measuring the traffic congestion level on the data path 📊 RMD-QOSM can use these measurements in the congestion control based on probing operation and/or the severe congestion operation (see Section 4.6.1.6) 🍓
|------| |-------| |------| |------|
| e2e |<->| e2e |<------------------------->| e2e |<->| e2e |
| QoS | | QoS | | QoS | | QoS |
| | |-------| |------| |------|
| | |-------| |-------| |-------| |------| | |
| | | local |<->| local |<->| local |<->| local| | |
| | | QoS | | QoS | | QoS | | QoS | | |
| | | | | | | | | | | |
| NSLP | | NSLP | | NSLP | | NSLP | | NSLP | | NSLP |
|st.ful| |st.ful | |st.less/ |st.less/ |st.ful| |st.ful|
| | | | |red.st.| |red.st.| | | | |
| | |-------| |-------| |-------| |------| | |
|------| |-------| |-------| |-------| |------| |------|
------------------------------------------------------------------
|------| |-------| |-------| |-------| |------| |------|
| NTLP |<->| NTLP |<->| NTLP |<->| NTLP |<->| NTLP |<->|NTLP |
|st.ful| |st.ful | |st.less| |st.less| |st.ful| |st.ful|
|------| |-------| |-------| |-------| |------| |------|
QNI QNE QNE QNE QNE QNR
(End) (Ingress) (Interior) (Interior) (Egress) (End)
st.ful: stateful, st.less: stateless
st.less red.st.: stateless or reduced-state
Figure 2: Protocol model of stateless/reduced-state operation3.2.2. RMD-QOSM/QoS-NSLP Signaling ✨
okay so the basic RMD-QOSM/QoS-NSLP signaling is shown in Figure 3 🦋. The signaling scenarios get pulled off using the QoS-NSLP processing rules from [RFC5974] 💅, combined with the Resource Management Function (RMF) triggers that get sent over the QoS-NSLP-RMF API (also from [RFC5974]) 🧚.
ngl this part is so cute 🥹. Inside the RMD domain you can run a different QoS Model than the end-to-end one used at the Edges of the RMD domain 💜, so the RMD Interior node reduced-state reservations can get updated independently of the per-flow end-to-end reservations (see Section 4.7 of [RFC5974]) 🎀. That means there are two different RESERVE messages in play 👑: one tied to the per-flow end-to-end reservations and used by the Edges of the RMD domain, and one tied to the reduced-state reservations inside the RMD domain 🌸.
A RESERVE message gets created by a QNI with an Initiator QSPEC that describes the reservation, and it's forwarded along the path towards the QNR ✨.
When the original RESERVE message hits the Ingress node, an RMD-QSPEC gets built from the initial QSPEC in the message (usually the Initiator QSPEC) 💖. That RMD-QSPEC is sent in an intra-domain, independent RESERVE message through the Interior nodes towards the QNR 🦋. This intra-domain RESERVE message uses the GIST datagram signaling mechanism 🌈. Note that the RMD-QOSM cannot directly specify that the GIST Datagram mode SHOULD be used 🙈. It can however be notified by using the GIST API Transfer-Attributes, such as unreliable, low level of security and use of local policy 💅.
Meanwhile, the original RESERVE message goes to the Egress node on the path to the QNR using the reliable transport mode of NTLP 🫶. Each QoS-NSLP node on the data path processes the intra-domain RESERVE message and checks resource availability with either the reservation-based or the measurement-based method ✨.
QNE Ingress QNE Interior QNE Interior QNE Egress
NTLP stateful NTLP stateless NTLP stateless NTLP stateful
| | | |
RESERVE | | | |
-------->| RESERVE | | |
+--------------------------------------------->|
| RESERVE' | | |
+-------------->| | |
| | RESERVE' | |
| +-------------->| |
| | | RESERVE' |
| | +------------->|
| | | RESPONSE'|
|<---------------------------------------------+
| | | | RESERVE
| | | +------->
| | | |RESPONSE
| | | |<-------
| | | RESPONSE |
|<---------------------------------------------+
RESPONSE| | | |
<--------| | | |
Figure 3: Sender-initiated reservation with reduced-state
Interior nodesWhen the message reaches the Egress node, and the reservation succeeds in each Interior node, an intra-domain (local) RESPONSE' heads back towards the Ingress node 🌷, and the original (end-to-end) RESERVE message is forwarded to the next domain 🚀. When the Egress node gets a RESPONSE message from the downstream end, it's forwarded straight to the Ingress node 💌.
If an intermediate node can't accommodate the new request, it flags that by marking a single bit in the message, and keeps forwarding until the Egress node is reached 🚩. From the Egress node, an intra-domain RESPONSE' and an original RESPONSE message both get sent directly to the Ingress node 🦋.
So, in the stateless/reduced-state domain, only sender-initiated reservations can happen, and functions that need per-flow NTLP or QoS-NSLP states, like summary and reduced refreshes, can't be used 😭. If per-flow identification is needed (aka associating the flow IDs for the reserved resources), the Edge nodes act on behalf of the Interior nodes 🫶.
3.2.3. RMD-QOSM Applicability and Considerations 🌈
The RMD-QOSM is a Diffserv-based bandwidth management methodology that can't provide full Diffserv support 💅. The reason is that it can only handle the (Expedited Forwarding) EF-like functionality behavior, and it can't handle the full set of (Assured Forwarding) AF-like functionality 🥹. The bandwidth information REQUIRED by the EF-like functionality behavior can be supported by RMD-QOSM carrying that bandwidth information in the <QoS Desired> parameter (see [RFC5975]) 🎀. The full set of AF-like functionality needs information specified in two token buckets, and RMD-QOSM doesn't use two token buckets, so it can't support the full AF functionality 😭. Note however that RMD-QOSM could still support a single AF PHB, when the traffic (or the upper limit of the traffic) can be described by a single bandwidth parameter 💖. Moreover, in case of tunneling, RMD-QOSM supports only the uniform tunneling mode for Diffserv (see [RFC2983]) 🌸.
The RMD domain MUST be engineered in such a way that each QNE Ingress maintains information about the smallest MTU that's supported on the links within the RMD domain 📏.
A very important consideration when using RMD-QOSM is that only one of the following RMD-QOSM schemes can be used at a time within one RMD domain 🚨. So an RMD router can never process and use two different RMD-QOSM signaling schemes at the same time 🙅♀️.
However, all RMD QNEs supporting this specification MUST support the combination of the "per-flow RMD reservation-based" and the "severe congestion handling by proportional data packet marking" scheme 👑. If the RMD QNEs support more RMD-QOSM schemes, then the operator of that RMD domain MUST preconfigure all the QNE Edge nodes within one domain such that the <SCH> field included in the "PHR container" (Section 4.1.2) and the "PDR Container" (Section 4.1.3) always uses the same value 🔧. That way, only one of the RMD-QOSM schemes described below is used at a time within one RMD domain 🦋.
The congestion situations (see Section 2) are solved using an admission control mechanism, e.g., "per-flow congestion notification based on probing", while the severe congestion situations (see Section 2) are solved using severe congestion handling mechanisms, e.g., "severe congestion handling by proportional data packet marking" ✨.
The RMD domain MUST be engineered in such a way that RMD-QOSM messages can be transported using the GIST Query and DATA messages in Q-mode; see [RFC5971] 📦. This means the Path MTU MUST be engineered in such a way that RMD-QOSM messages are transported without fragmentation 🧩. Furthermore, the RMD domain MUST be engineered in such a way as to guarantee capacity for the GIST Query and Data messages in Q-mode, within the rate control limits imposed by GIST; see [RFC5971] 💪.
The RMD domain has to be configured such that the GIST context-free flag (C-flag) MUST be set (C=1) for QUERY messages and DATA messages sent in Q-mode; see [RFC5971] 🎯.
Moreover, the same deployment issues and extensibility considerations described in [RFC5971] and [RFC5978] apply to this document 📚.
It's important to note that the concepts described in Sections 4.6.1.6.2, 4.6.2.5.2, 4.6.1.6.2, and 4.6.2.5.2 contributed to the PCN WG standardization 🌈.
The available RMD-QOSM/QoS-NSLP signaling schemes are:
- "per-flow congestion notification based on probing" (see Sections 4.3.2, 4.6.1.7, and 4.6.2.6) 🌸. This scheme uses, for severe congestion handling, the "severe congestion handling by proportional data packet marking" (see Sections 4.6.1.6.2 and 4.6.2.5.2). Furthermore, the Interior nodes are considered to be Diffserv aware, but NSIS-unaware nodes (see Section 4.3.2).
- "per-flow RMD NSIS measurement-based admission control" (see Sections 4.3.2, 4.6.1, and 4.6.2) 📊. This scheme uses, for severe congestion handling, the "severe congestion handling by proportional data packet marking" (see Sections 4.6.1.6.2 and 4.6.2.5.2). Furthermore, the Interior nodes are considered to be NSIS-aware nodes (see Section 4.3.2).
- "per-flow RMD reservation-based" in combination with the "severe congestion handling by the RMD-QOSM refresh" procedure (see Sections 4.3.3, 4.6.1, 4.6.1.6.1, and 4.6.2.5.1) 🔁. This scheme uses, for severe congestion handling, the "severe congestion handling by the RMD-QOSM refresh" procedure (see Sections 4.6.1.6.1 and 4.6.2.5.1). Furthermore, the intra-domain sessions supported by the Edge nodes are per-flow sessions (see Section 4.3.3).
- "per-flow RMD reservation-based" in combination with the "severe the congestion handling by proportional data packet marking" procedure (see Sections 4.3.3, 4.6.1, 4.6.1.6.2, and 4.6.2.5.2) 💜. This scheme uses, for severe congestion handling, the "severe congestion handling by proportional data packet marking" procedure (see Sections 4.6.1.6.2 and 4.6.2.5.2). Furthermore, the intra-domain sessions supported by the Edge nodes are per-flow sessions (see Section 4.3.3).
- "per-aggregate RMD reservation-based" in combination with the "severe congestion handling by the RMD-QOSM refresh" procedure (see Sections 4.3.1, 4.6.1, 4.6.1.6.1, and 4.6.2.5.1) 🫧. This scheme uses, for severe congestion handling, the "severe congestion handling by the RMD-QOSM refresh" procedure (see Sections 4.6.1.6.1 and 4.6.2.5.1). Furthermore, the intra-domain sessions supported by the Edge nodes are per-aggregate sessions (see Section 4.3.1). Moreover, this scheme can be considered a reservation-based scheme, since the RMD Interior nodes are reduced-state nodes, i.e., they don't store NTLP/GIST states, but they do store per PHB-aggregated QoS-NSLP reservation states 🧠.
- "per-aggregate RMD reservation-based" in combination with the "severe congestion handling by proportional data packet marking" procedure (see Sections 4.3.1, 4.6.1, 4.6.1.6.2, and 4.6.2.5.2) 🍓. This scheme uses, for severe congestion handling, the "severe congestion handling by proportional data packet marking" procedure (see Sections 4.6.1.6.2 and 4.6.2.5.2). Furthermore, the intra-domain sessions supported by the Edge nodes are per-aggregate sessions (see Section 4.3.1). Moreover, this scheme can be considered a reservation-based scheme, since the RMD Interior nodes are reduced-state nodes, i.e., they don't store NTLP/GIST states, but they do store per PHB-aggregated QoS-NSLP reservation states 🌟.
4. RMD-QOSM, Detailed Description 🎀
This section takes a closer look at RMD-QOSM 🔍. It covers the role of stateless and reduced-state QNEs, the RMD-QOSM QSPEC Object, the format of the RMD-QOSM QoS-NSLP messages, and how QSPECs get processed and used in different protocol operations 💅✨.
4.1. RMD-QSPEC Definition 💖
okay bestie, so the RMD-QOSM uses the QSPEC format from [RFC5975] 🎀 The Initiator/Local QSPEC bit, <I>, gets set to "Local" (that's "1") ✨ and the <QSPEC Proc> gets set up like this:
- Message Sequence = 0: Sender initiated
- Object combination = 0:
<QoS Desired>for RESERVE and<QoS Reserved>for RESPONSE
The <QSPEC Version> is the default one, "0", see [RFC5975] 🧚 and the <QSPEC Type> is specified in [RFC5975] and equals "2" 💅 The <Traffic Handling Directives> is built like this:
<Traffic Handling Directives> = <PHR container> <PDR container>The Per-Hop Reservation container (PHR container) and the Per-Domain Reservation container (PDR container) get their own sections, 4.1.2 and 4.1.3, respectively 🌸 The <PHR container> handles traffic directives for intra-domain communication and reservation, while the <PDR container> carries the extra directives needed for edge-to-edge communication. The parameter IDs used by both containers are assigned by IANA, see Section 6 🦋
The RMD-QOSM <QoS Desired> and <QoS Reserved> are specified in Section 4.1.1 🫶 Both of those, plus the <PHR container>, get used and processed by the Edge and Interior nodes. The <PDR container> field is only processed by Edge nodes 👑
4.1.1. RMD-QOSM <QoS Desired> and <QoS Reserved> 🎀
okay so the RESERVE message only carries the <QoS Desired> object [RFC5975], and the <QoS Reserved> object rides along in the RESPONSE message 💅
In RMD-QOSM, both of those objects hold the same three parameters:
<QoS Desired> = <TMOD-1> <PHB Class> <Admission Priority>
<QoS Reserved> = <TMOD-1> <PHB Class> <Admission Priority>The bit format of the <PHB Class> (see [RFC5975] and Figures 4 and 5) and the <Admission Priority> matches the bit format specified in [RFC5975] 🌈
Here's the lowkey important bit ✨ for RMD-QOSM, a reservation made without an <Admission Priority> parameter is the same as a reservation made with an <Admission Priority> value of 1 🙈
0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DSCP |0 0 0 0 0 0 0 0 X 0|
+---+---+---+---+---+---+---+---+
Figure 4: DSCP parameter 0 1
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PHB ID code |0 0 X X|
+---+---+---+---+---+---+---+---+
Figure 5: PHB ID Code parameter4.1.2. PHR Container 🧚
this section is all about the parameters the PHR container uses, which the RMD-QOSM functionality at the Interior nodes relies on 🦋
<PHR container> = <O> <K> <S> <M>, <Admitted Hops>, <B> <Hop_U> <Time Lag> <SCH> <Max Admitted Hops>The bit format of the PHR container is in Figure 6 ✨ In that figure, <Hop_U> is shown as <U>, and <Max Admitted Hops> is shown as <Max Adm Hops> 🌸
Figure 6: PHR containerWait, I need to keep the figure itself exact, so here it is in full:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|M|E|N|r| Parameter ID |r|r|r|r| 2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S|M| Admitted Hops|B|U| Time Lag |O|K| SCH | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Max Adm Hops | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 6: PHR containerParameter ID is a 12-bit field that tells you the PHR type: PHR_Resource_Request, PHR_Release_Request, or PHR_Refresh_Update 🍓
- "PHR_Resource_Request" (Parameter ID = 17): initiates or updates the traffic class reservation state on all nodes on the communication path between the QNE(Ingress) and QNE(Egress) nodes 🔥
- "PHR_Release_Request" (Parameter ID = 18): explicitly releases, by subtraction, the reserved resources for a particular flow from a traffic class reservation state 💀
- "PHR_Refresh_Update" (Parameter ID = 19): refreshes the traffic class reservation soft state on all nodes on the communication path between the QNE(Ingress) and QNE(Egress) nodes, according to a resource reservation request that was successfully processed during a previous refresh period 🌈
<S> (Severe Congestion) is 1 bit. When there's a route change, refreshing RESERVE messages follow the new data path, so that's where resources get requested. If there aren't enough resources to fit the new traffic, severe congestion happens 😭 Severe congested Interior nodes SHOULD notify Edge QNEs about it by setting the <S> bit 🫶
<O> (Overload) is 1 bit. It's used during the severe congestion handling scheme that uses the RMD-QOSM refresh procedure 👑 This bit gets set when an overload on a QNE Interior node is detected and the field is carried by the "PHR_Refresh_Update" container. <O> SHOULD be set to "1" if the <S> bit is set. For more details, see Section 4.6.1.6.1 ✨
<M> is 1 bit. If a resource reservation or resource query fails at an Interior QNE, that QNE sets the <M> bit to let the Egress QNE know 🙈
<Admitted Hops> is an 8-bit field that counts the hops in the RMD domain where the reservation was successful 💖 The <Admitted Hops> gets set to "0" when a RESERVE message enters a domain, and it MUST be incremented by each Interior QNE, as long as the <Hop_U> bit is not set. However, once a QNE that doesn't have enough resources to admit the reservation is reached, the <M> bit is set and the <Admitted Hops> value gets frozen by setting the <Hop_U> bit to "1" 🧊 Also note that the <Admitted Hops> parameter, together with the <Max Admitted Hops> and <K> parameters, gets used during the RMD partial release procedures (see Section 4.6.1.5.2) 🦋
<Hop_U> (NSLP_Hops unset) is 1 bit. The QNE(Ingress) node MUST set the <Hop_U> parameter to 0 🌸 This parameter SHOULD be set to "1" by a node when that node does not increase the <Admitted Hops> value, which happens when an RMD-QOSM reservation-based node isn't admitting the reservation request 🍓 When <Hop_U> is set to "1", the <Admitted Hops> SHOULD NOT be changed. Also, this flag together with the <Admitted Hops> flag is used to locate the last node that successfully processed a reservation request (see Section 4.6.1.2) 🔍
<B> is 1 bit. When it's set to "1", it means a bidirectional reservation 💅
<Time Lag> represents the ratio between "T_Lag" and the length of the refresh period, "T_period" (see Section 4.6.1.5) ⭐ "T_Lag" is the time difference between the departure time of the last sent "PHR_Refresh_Update" control information container and the departure time of the "PHR_Release_Request" control information container.
<K> is 1 bit. When it's set to "1", it means the resources/bandwidth carried by a tearing RESERVE MUST NOT be released, and the resources/bandwidth carried by a non-tearing RESERVE MUST NOT be reserved/refreshed 🚫 For more details, see Section 4.6.1.5.2 💜
<Max Admitted Hops> is 8 bits. It holds the <Admitted Hops> value carried by the <PHR container> field, and it's used to identify the RMD reservation-based node that admitted or processed a "PHR_Resource_Request" 🧚
<SCH> is 3 bits, and it says which of the 6 RMD-QOSM scenarios (see Section 3.2.3) MUST be used within the RMD domain 🎀 The operator of an RMD domain MUST preconfigure all the QNE Edge nodes within one domain so that the <SCH> field in the "PHR container" always uses the same value. That way, only one of the RMD-QOSM schemes below can be in use at a time within one RMD domain 👑 All the QNE Interior nodes MUST interpret this field before processing any other PHR container payload fields ✨ The currently defined <SCH> values are:
- 0: RMD-QOSM scheme MUST be "per-flow congestion notification based on probing";
- 1: RMD-QOSM scheme MUST be "per-flow RMD NSIS measurement-based admission control",
- 2: RMD-QOSM scheme MUST be "per-flow RMD reservation-based" in combination with the "severe congestion handling by the RMD-QOSM refresh" procedure;
- 3: RMD-QOSM scheme MUST be "per-flow RMD reservation-based" in combination with the "severe congestion handling by proportional data packet marking" procedure;
- 4: RMD-QOSM scheme MUST be "per-aggregate RMD reservation-based" in combination with the "severe congestion handling by the RMD-QOSM refresh" procedure;
- 5: RMD-QOSM scheme MUST be "per-aggregate RMD reservation-based" in combination with the "severe congestion handling by proportional data packet marking" procedure;
- 6 - 7: reserved.
The default value of the <SCH> field MUST be set to the value equal to 3 🌈💖
4.1.3. PDR Container 🎀
okay bestie, this part is all about the PDR container 💖 It holds the parameters that the RMD-QOSM functionality uses over at the Edge nodes ✨
The bit format is in Figure 7 🦋
<PDR container> = <O> <S> <M>
<Max Admitted Hops> <B> <SCH> [<PDR Bandwidth>]Heads up: in Figure 7 the <Max Admitted Hops> field gets shortened to <Max Adm Hops>, so don't be confused when you see it 🙈
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|M|E|N|r| Parameter ID |r|r|r|r| 2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|S|M| Max Adm Hops |B|O| SCH | EMPTY |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|PDR Bandwidth(32-bit IEEE floating point.number) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 7: PDR container 🌸
Parameter ID is a 12-bit field that tells you what type of <PDR container> field you're looking at 🧚
Here are the message types, each with its own Parameter ID 💜
"PDR_Reservation_Request"(Parameter ID = 20): the QNE(Ingress) node generates this to start or update the QoS-NSLP per-domain reservation state in the QNE(Egress) node 🔥"PDR_Refresh_Request"(Parameter ID = 21): the QNE(Ingress) node generates this and sends it to the QNE(Egress) node to refresh, if needed, the QoS-NSLP per-domain reservation states sitting in the QNE(Egress) node 🫶"PDR_Release_Request"(Parameter ID = 22): the QNE(Ingress) node generates and sends this to the QNE(Egress) node to release the per-domain reservation states explicitly 👋"PDR_Reservation_Report"(Parameter ID = 23): the QNE(Egress) node generates and sends this back to the QNE(Ingress) node to say that a"PHR_Resource_Request"and a"PDR_Reservation_Request"traffic handling directive field came in, and whether the request got admitted or rejected 👑"PDR_Refresh_Report"(Parameter ID = 24): the QNE(Egress) node generates and sends this, if needed, to the QNE(Ingress) node to say that a"PHR_Refresh_Update"traffic handling directive field was received and processed 🌈"PDR_Release_Report"(Parameter ID = 25): the QNE(Egress) node generates and sends this, if needed, to the QNE(Ingress) node to say that a"PHR_Release_Request"and a"PDR_Release_Request"traffic handling directive field were received and processed ✨"PDR_Congestion_Report"(Parameter ID = 26): the QNE(Egress) node generates and sends this to the QNE(Ingress) node, and it's used for congestion notification 🚨
Now the individual fields 💅
- <S> (PDR Severe Congestion): 1 bit. It says whether a severe congestion situation happened. It can also carry the <S> parameter from the
<PHR_Resource_Request>or<PHR_Refresh_Update>fields 😭 - <O> (Overload): 1 bit. This one is used during the severe congestion handling scheme that uses the RMD-QOSM refresh procedure. The bit gets set when an overload on a QNE Interior node is detected, and when this field is carried by the
"PDR_Congestion_Report"container. <O> SHOULD be set to "1" if the <S> bit is set. For more details, see Section 4.6.1.6.1 🙈 - <M> (PDR Marked): 1 bit. It carries the <M> value of the
"PHR_Resource_Request"or"PHR_Refresh_Update"traffic handling directive field 🦋 - <B>: 1 bit. It indicates a bidirectional reservation 🍓
- <Max Admitted Hops>: 8 bits. This is the <Admitted Hops> value carried by the <PHR container> field, and it's used to identify the RMD reservation-based node that admitted or processed a
"PHR_Resource_Request"🧚 - <PDR Bandwidth>: 32 bits. This field specifies the bandwidth in two situations. The first is when the <B> flag is set to "1" and this parameter is carried by a RESPONSE message. The second is when a severe congestion occurs, the QNE Edges maintain an aggregated intra-domain QoS-NSLP operational state, and the parameter is carried by a NOTIFY message. When the <B> flag is set to "1", this parameter specifies the requested bandwidth that has to be reserved by a node in the reverse direction, when the intra-domain signaling procedures require a bidirectional reservation procedure. In the severe congestion situation, this parameter specifies the bandwidth that has to be released 💖
- <SCH>: 3 bits. This is the <SCH> value that tells you which of the 6 RMD scenarios (see Section 3.2.3) MUST be used within the RMD domain. The operator of an RMD domain MUST preconfigure all the QNE Edge nodes within one domain so that the <SCH> field included in the "PDR container" always uses the same value. That way, only one of the RMD-QOSM schemes below can be used at a time within one RMD domain 🔥 All the QNE Interior nodes MUST interpret this field before processing any other <PDR container> payload fields 👑 The currently defined <SCH> values are:
- 0: RMD-QOSM scheme MUST be "per-flow congestion notification based on probing" 🌸
- 1: RMD-QOSM scheme MUST be "per-flow RMD NSIS measurement-based admission control" 🌸
- 2: RMD-QOSM scheme MUST be "per-flow RMD reservation-based" in combination with the "severe congestion handling by the RMD-QOSM refresh" procedure 🌸
- 3 : RMD-QOSM scheme MUST be "per-flow RMD reservation-based" in combination with the "severe congestion handling by proportional data packet marking" procedure 🌸
- 4: RMD-QOSM scheme MUST be "per-aggregate RMD reservation-based" in combination with the "severe congestion handling by the RMD-QOSM refresh" procedure 🌸
- 5: RMD-QOSM scheme MUST be "per-aggregate RMD reservation-based" in combination with the "severe congestion handling by proportional data packet marking" procedure 🌸
- 6 - 7: reserved 🙈
The default value of the <SCH> field MUST be set to the value equal to 3 ✨
4.2. Message Format 🫶
The messages used by RMD-QOSM follow the QoS-NSLP and QSPEC template specifications, so they're on the same page as everything else 💅 The QSPEC that RMD-QOSM uses is called RMD-QSPEC in this document, and you can find it in Section 4.1 🦋
4.3. RMD Node State Management 🌈
The QoS-NSLP state creation and management is specified in [RFC5974] 📖 This section covers the state creation and management functions of the Resource Management Function (RMF) in the RMD nodes 🎀
4.3.1. Aggregated Operational and Reservation States at the QNE Edges 🌸
okay so the QNE Edges keep BOTH the intra-domain QoS-NSLP operational and reservation states 💅, while the QNE Interior nodes only keep reservation states 🦋. The structure of the intra-domain QoS-NSLP operational state that the Edges use is specified in [RFC5974]. ✨
so in this case, the intra-domain sessions the Edges support are per-aggregate sessions, and they have a one-to-many relationship with the per-flow end-to-end states that same Edge supports. main character aggregation energy 🌸
Note that the method of selecting the end-to-end sessions that form an aggregate is not specified in this document. ngl that's a whole separate vibe 🙈 An example of how this can be accomplished is by monitoring the GIST routing states used by the end-to-end sessions and grouping the ones that use the same <PHB Class>, QNE Ingress and QNE Egress addresses, and the value of the priority level. Note that this priority level should be deduced from the priority parameters carried by the initial QSPEC object. 🧚
The operational state of this aggregated intra-domain session MUST contain a list with BOUND-SESSION-IDs. 👑
The structure of the list depends on whether a unidirectional reservation or a bidirectional reservation is supported. 💜
When the operational state (at QNE Ingress and QNE Egress) supports unidirectional reservations, then this state MUST contain a list with BOUND-SESSION-IDs maintaining the <SESSION-ID> values of its bound end-to-end sessions. The Binding_Code associated with this BOUND-SESSION-ID is set to code (Aggregated sessions). Thus, the operational state maintains a list of BOUND-SESSION-ID entries. Each entry is created when an end-to-end session joins the aggregated intra-domain session and is removed when an end-to-end session leaves the aggregate. 🎀
It is important to emphasize that, in this case, the operational state (at QNE Ingress and QNE Egress) that is maintained by each end-to-end session bound to the aggregated intra-domain session MUST contain in the BOUND-SESSION-ID, the <SESSION-ID> value of the bound tunneled intra-domain (aggregate) session. The Binding_Code associated with this BOUND-SESSION-ID is set to code (Aggregated sessions). 🫶
When the operational state (at QNE Ingress and QNE Egress) supports bidirectional reservations, the operational state MUST contain a list of BOUND-SESSION-ID sets. Each set contains two BOUND-SESSION-IDs. One of the BOUND-SESSION-IDs maintains the <SESSION-ID> value of one of bound end-to-end session. The Binding_Code associated with this BOUND-SESSION-ID is set to code (Aggregated sessions). Another BOUND-SESSION-ID, within the same set entry, maintains the SESSION-ID of the bidirectional bound end-to-end session. The Binding_Code associated with this BOUND-SESSION-ID is set to code (Bidirectional sessions). 🦋
Note that, in each set, a one-to-one relation exists between each BOUND-SESSION-ID with Binding_Code set to (Aggregate sessions) and each BOUND-SESSION-ID with Binding_Code set to (bidirectional sessions). Each set is created when an end-to-end session joins the aggregated operational state and is removed when an end-to-end session leaves the aggregated operational state. ✨
It is important to emphasize that, in this case, the operational state (at QNE Ingress and QNE Egress) that is maintained by each end-to-end session bound to the aggregated intra-domain session it MUST contain two types of BOUND-SESSION-IDs. One is the BOUND-SESSION-ID that MUST contain the <SESSION-ID> value of the bound tunneled aggregated intra-domain session that is using the Binding_Code set to (Aggregated sessions). The other BOUND-SESSION-ID maintains the SESSION-ID of the bound bidirectional end-to-end session. The Binding_Code associated with this BOUND-SESSION-ID is set to code (Bidirectional sessions). 💅
When the QNE Edges use aggregated QoS-NSLP reservation states, then the <PHB Class> value and the size of the aggregated reservation, e.g., reserved bandwidth, have to be maintained. Note that this type of aggregation is an edge-to-edge aggregation and is similar to the aggregation type specified in [RFC3175]. 🌈
The size of the aggregated reservations needs to be greater or equal to the sum of bandwidth of the inter-domain (end-to-end) reservations/sessions it aggregates (e.g., see Section 1.4.4 of [RFC3175]). 📊
A policy can be used to maintain the amount of REQUIRED bandwidth on a given aggregated reservation by taking into account the sum of the underlying inter-domain (end-to-end) reservations, while endeavoring to change reservation less frequently. This MAY require a trend analysis. If there is a significant probability that in the next interval of time the current aggregated reservation is exhausted, the Ingress router MUST predict the necessary bandwidth and request it. If the Ingress router has a significant amount of bandwidth reserved, but has very little probability of using it, the policy MAY predict the amount of bandwidth REQUIRED and release the excess. To increase or decrease the aggregate, the RMD modification procedures SHOULD be used (see Section 4.6.1.4). 🔥
The QNE Interior nodes are reduced-state nodes, i.e., they do not store NTLP/GIST states, but they do store per PHB-aggregated QoS-NSLP reservation states. These reservation states are maintained and refreshed in the same way as described in Section 4.3.3. 🌸
4.3.2. Measurement-Based Method 🦋
The QNE Edges maintain per-flow intra-domain QoS-NSLP operational and reservation states that contain similar data structures as those described in Section 4.3.1. The main difference is associated with the different types of the used Message-Routing-Information (MRI) and the bound end-to-end sessions. The structure of the maintained BOUND-SESSION-IDs depends on whether a unidirectional reservation or a bidirectional reservation is supported. 💖
When unidirectional reservations are supported, the operational state associated with this per-flow intra-domain session MUST contain in the BOUND-SESSION-ID the <SESSION-ID> value of its bound end-to-end session. The Binding_Code associated with this BOUND-SESSION-ID is set to code (Tunneled and end-to-end sessions). 🎀
When bidirectional reservations are supported, the operational state (at QNE Ingress and QNE Egress) MUST contain two types of BOUND-SESSION-IDs. One is the BOUND-SESSION-ID that maintains the <SESSION-ID> value of the bound tunneled per-flow intra-domain session. The Binding_Code associated with this BOUND-SESSION-ID is set to code (Tunneled and end-to-end sessions). 🫶
The other BOUND-SESSION-ID maintains the SESSION-ID of the bound bidirectional end-to-end session. The Binding_Code associated with this BOUND-SESSION-ID is set to code (Bidirectional sessions). ✨
Furthermore, the QoS-NSLP reservation state maintains the <PHB Class> value, the value of the bandwidth requested by the end-to-end session bound to the intra-domain session, and the value of the priority level. 👑
The measurement-based method can be classified in two schemes:
- Congestion notification based on probing:
In this scheme, the Interior nodes are Diffserv-aware but not NSIS-aware nodes. Each Interior node counts the bandwidth that is used by each PHB traffic class. This counter value is stored in an RMD_QOSM state. For each PHB traffic class, a predefined congestion notification threshold is set. The predefined congestion notification threshold is set according to an engineered bandwidth limitation based, e.g., on a Service Level Agreement or a capacity limitation of specific links. The threshold is usually less than the capacity limit, i.e., admission threshold, in order to avoid congestion due to the error of estimating the actual traffic load. The value of this threshold SHOULD be stored in another RMD_QOSM state. 🌈
In this scenario, an end-to-end NSIS message is used as a probe packet. In this case, the <DSCP> field of the GIST message is re-marked when the predefined congestion notification threshold is exceeded in an Interior node. It is required that the re-marking happens to all packets that belong to the congested PHB traffic class so that the probe can't pass the congested router without being re-marked. In this way, it is ensured that the end-to-end NSIS message passed through the node that is congested. This feature is very useful when flow-based ECMP (Equal Cost Multiple Path) routing is used to detect only flows that are passing through the congested node. 🔍
- NSIS measurement-based admission control:
The measurement-based admission control is implemented in NSIS-aware stateless routers. Thus, the main difference between this type of the measurement-based admission control and the congestion notification-based admission control is the fact that the Interior nodes are NSIS-aware nodes. In particular, the QNE Interior nodes operating in NSIS measurement-based mode are QoS-NSLP stateless nodes, i.e., they do not support any QoS-NSLP or NTLP/GIST states. These measurement-based nodes store two RMD-QOSM states per PHR group. These states reflect the traffic conditions at the node and are not affected by QoS-NSLP signaling. One state stores the measured user traffic load associated with the PHR group and another state stores the maximum traffic load threshold that can be admitted per PHR group. When a measurement-based node receives a intra-domain RESERVE message, it compares the requested resources to the available resources (maximum allowed minus current load) for the requested PHR group. If there are insufficient resources, it sets the <M> bit in the RMD-QSPEC. No change to the RMD-QSPEC is made when there are sufficient resources. 💅
4.3.3. Reservation-Based Method ✨
Okay so the QNE Edges keep intra-domain QoS-NSLP operational and reservation states, and they have pretty much the same data structures we talked about in Section 4.3.1. 💅
The intra-domain sessions the Edges support are per-flow sessions. Each one lines up one-to-one with a per-flow end-to-end state at that same Edge. So neat 🎀
The QNE Interior nodes running in reservation-based mode are QoS-NSLP reduced-state nodes. That means they don't store NTLP/GIST states, but they do store per PHB-aggregated QoS-NSLP states. 🧚
The reservation-based PHR installs and maintains one reservation state per PHB, in every node along the communication path. Each state is identified by the <PHB Class> value, and it keeps track of how many resource units (or how much bandwidth) are currently reserved. The QNE Ingress node only signals the resource units each flow requests. If they get admitted, those units get added to the currently reserved resources for that PHB. It's giving efficiency 🦋
For each PHB, a threshold is maintained that sets the maximum number of resource units that can be reserved. This threshold could, for example, be statically configured. ✨
An example of how the admission control and its maintenance process works in the Interior nodes is described in Section 3 of [CsTa05]. 📚
The simplified concept behind the per-traffic class admission control process in the Interior nodes is based on this equation:
last + p <= T,where p is the requested bandwidth rate, T is the admission threshold (basically the maximum traffic volume that can be admitted in the traffic class), and last is a counter that records the aggregated sum of the signaled bandwidth rates of previously admitted flows. 🍓
The PHB group reservation states maintained in the Interior nodes are soft states. They get refreshed by sending periodic refresh intra-domain RESERVE messages, which the Ingress QNEs kick off. If a refresh message for some number of reserved resource units (i.e., bandwidth) doesn't show up, the aggregated reservation state gets decreased in the next refresh period by the amount of resources that didn't get refreshed. The refresh period can be refined using a sliding window algorithm described in [RMD3]. 🌸
The reserved resources for a particular flow can also be explicitly released from a PHB reservation state by an intra-domain RESERVE release/tear message, which is generated by the Ingress QNEs. 💜
Explicit release is literally the best because it lets resources get freed up instantly, no matter how long the refresh period is. That means you can have a longer refresh period, which also cuts down on how many periodic refresh messages get sent. Iconic 👑
Note that, for both the measurement-based and the (per-flow and aggregated) RMD reservation-based methods, the way the maximum bandwidth thresholds are maintained is outside the scope of this document. 🙈 However, when admission priorities are supported, the Maximum Allocation [RFC4125] or the Russian Dolls [RFC4127] bandwidth allocation models MAY be used. In that case, three types of priority traffic classes within the same PHB (for example, Expedited Forwarding) can be told apart from each other. In this document these three priority classes, which all belong to the same PHB, are called PHB_low_priority, PHB_normal_priority, and PHB_high_priority. They're identified by the <PHB Class> value plus the priority value, which is carried in the <Admission Priority> RMD-QSPEC parameter. 🫶
4.4. Transport of RMD-QOSM Messages 🦋
As mentioned in Section 1, RMD-QOSM is aiming to support a bunch of extra requirements, e.g., Minimal impact on Interior node performance. So RMD-QOSM is designed to be super lightweight signaling, both in terms of how many signaling message round trips happen and how much state gets set up at the signaling nodes involved, with or without reduced state on the QNEs. The actions a QNE Interior node is allowed to take are minimal (meaning only the ones RMD-QOSM specifies). 💅
For example, only the QNE Ingress and the QNE Egress nodes are allowed to start certain signaling messages. QNE Interior nodes are, for example, allowed to modify certain signaling message payloads. On top of that, RMD signaling is aimed at intra-domain signaling only. So RMD-QOSM leans on the security and reliability support provided by the bound end-to-end session, which runs between the boundaries of the RMD domain (the RMD-QOSM QNE Edges), plus the security that comes with D-mode. This implies the use of Datagram Mode. 🔥
So the intra-domain messages RMD-QOSM uses are meant to operate in NTLP/GIST Datagram mode (see [RFC5971]). The NSLP functionality in every RMD-QOSM-aware QoS-NSLP node needs the intra-domain GIST, through the QoS-NSLP RMF API (see [RFC5974]), to do the following:
- operate in unreliable mode. This can be handled by passing this requirement from the QoS-NSLP layer down to the GIST layer through the API Transfer-Attributes.
- not create a message association state. This requirement can be met by a local policy, e.g., the QNE is configured to not create a message association state.
- not create any NTLP routing state by the Interior nodes. This can be handled by passing this requirement from the QoS-NSLP layer down to the GIST layer through the API. However, between the QNE Egress and QNE Ingress, routing states SHOULD be created. These are associated with intra-domain sessions and can be used for communicating GIST Data messages that a QNE Egress sends directly to a QNE Ingress. This type of routing state associated with an intra-domain session can be generated and used in the following way: 🧚
- When the QNE Ingress has to send an initial intra-domain RESERVE message, the QoS-NSLP sends it by including, in the GIST API SendMessage primitive, the Unreliable and No security attributes. To optimize this procedure, the RMD domain MUST be engineered in such a way that GIST piggybacks this NSLP message on a GIST Query message. Furthermore, GIST sets the C-flag (
C=1), see [RFC5971], and uses the Q-mode. The GIST functionality in each QNE Interior node receives the GIST Query message, and by using the RecvMessage GIST API primitive it passes the intra-domain RESERVE message up to the QoS-NSLP functionality. At the same time, the GIST functionality uses the Routing-State-Check boolean to figure out whether the QoS-NSLP needs to create a routing state. The QoS-NSLP sets this boolean to tell GIST not to create a routing state, and to forward the GIST Query further downstream with the modified QoS-NSLP payload, which includes the modified intra-domain RESERVE message. The intra-domain RESERVE is sent in the same way all the way up to the QNE Egress. The QNE Egress needs to create a routing state.
- When the QNE Ingress has to send an initial intra-domain RESERVE message, the QoS-NSLP sends it by including, in the GIST API SendMessage primitive, the Unreliable and No security attributes. To optimize this procedure, the RMD domain MUST be engineered in such a way that GIST piggybacks this NSLP message on a GIST Query message. Furthermore, GIST sets the C-flag (
Therefore, at the same moment that the GIST functionality passes the intra-domain RESERVE message to the QoS-NSLP via the GIST RecvMessage primitive, the QoS-NSLP sets the Routing-State-Check boolean so that a routing state is created. GIST creates the routing state using normal GIST procedures. After this phase, the QNE Ingress and QNE Egress have routing states for that particular session, which can route traffic directly from QNE Ingress to QNE Egress and from QNE Egress to QNE Ingress. The routing state at the QNE Egress can be used by the QoS-NSLP and GIST to send an intra-domain RESPONSE or intra-domain NOTIFY directly to the QNE Ingress using GIST Data messages. Note that this routing state is refreshed using normal GIST procedures. Also note that the above description assumes the QNE Ingress can piggyback the initial RESERVE (NSLP) message on the GIST Query message. If piggybacking this NSLP (initial RESERVE) message on the GIST Query message would not be possible, then the GIST Query message sent by the QNE Ingress would not contain any NSLP data. That GIST Query message would only be processed by the QNE Egress to generate a routing state.
After the QNE Ingress is informed that the routing state at the QNE Egress has been initiated, it would have to send the initial RESERVE message using similar procedures to those used when it sends an intra-domain RESERVE message that is not an initial RESERVE (see the next bullet). This procedure is not efficient, and therefore it is RECOMMENDED that the RMD domain MUST be engineered in such a way that the GIST protocol layer, which is processed on a QNE Ingress, piggybacks an initial RESERVE (NSLP) message on a GIST Query message that uses the Q-mode.
- When the QNE Ingress needs to send an intra-domain RESERVE message that is not an initial RESERVE, the QoS-NSLP sends this message by including in the GIST API SendMessage primitive attributes that imply the use of Datagram Mode, e.g., the Unreliable attribute. Furthermore, the Local policy attribute is set so that GIST sends the intra-domain RESERVE message in Q-mode even if there is a routing state at the QNE Ingress. This way, GIST uses its local policy to send the intra-domain RESERVE message by piggybacking it on a GIST Data message and sending it in Q-mode even when there's a routing state for this session. The intra-domain RESERVE message gets piggybacked on the GIST Data message, which is forwarded and processed by the QNE Interior nodes all the way up to the QNE Egress. 🌈
The transport of the original (end-to-end) RESERVE message is handled in the following way: 💖
At the QNE Ingress, the original (end-to-end) RESERVE message is forwarded but ignored by the stateless or reduced-state nodes, see Figure 3.
The intermediate (Interior) nodes are bypassed using multiple levels of NSLPID values (see [RFC5974]). This is accomplished by marking the end-to-end RESERVE message, i.e., modifying the QoS-NSLP default NSLPID value to another predefined NSLPID value. 🎀
The marking MUST be accomplished by the Ingress by modifying the QoS_NSLP default NSLPID value to a predefined NSLPID value. Then the Egress MUST stop this marking process by reassigning the QoS-NSLP default NSLPID value to the original (end-to-end) RESERVE message. Note that assigning these NSLPID values is a QoS-NSLP issue, which SHOULD be accomplished via IANA [RFC5974]. 🫶
4.5. Edge Discovery and Message Addressing 🧭
Okay so the big question is how you find your Egress node 🔍. You can use the GIST discovery mechanism [RFC5971], manual configuration, or literally any other discovery technique. The addressing of signaling messages depends on which GIST transport mode is in use. RMD-QOSM/QoS-NSLP messages that only the Edge nodes process use the peer-to-peer addressing of the GIST Connection (C) mode 🎀.
RMD-QOSM/QoS-NSLP messages that get processed by every node in the Diffserv domain, meaning Edges and Interior nodes, use the end-to-end addressing of the GIST Datagram (D) mode 🦋. Heads up, the RMD-QOSM cannot directly say whether the GIST Connection or the GIST Datagram mode SHOULD be used 💅. That gets decided through the QoS-NSLP-RMF API, using the GIST API Transfer-Attributes (things like Reliable or Unreliable, and high or low level of security), plus local policies. RMD QoS signaling messages aimed at the data path end nodes get intercepted by the Egress nodes 🫶. Especially at the ingress and for downstream intra-domain messages, the RMD-QOSM tells GIST, through the GIST API, to do the following:
- use unreliable and low level security Transfer-Attributes,
- do not create a GIST routing state, and
- use the D-mode MRI.
After that, the intra-domain RESERVE messages can be carried using the Query D-mode, so check out Section 4.4 🌸.
At the QNE Egress, for upstream intra-domain messages, the RMD-QOSM tells GIST (again through the GIST API) to use, among other things:
- unreliable and low level security Transfer-Attributes
- the routing state associated with the intra-domain session, so an upstream intra-domain message can go straight to the QNE Ingress. See Section 4.4 ✨.
4.6. Operation and Sequence of Events 🎬
4.6.1. Basic Unidirectional Operation 💖
This part walks through the basic unidirectional operation and the sequence of events and triggers for the RMD-QOSM. These are the basic operation cases we're looking at:
- Successful reservation (Section 4.6.1.1),
- Unsuccessful reservation (Section 4.6.1.2),
- RMD refresh reservation (Section 4.6.1.3),
- RMD modification of aggregated reservation (Section 4.6.1.4),
- RMD release procedure (Section 4.6.1.5.),
- Severe congestion handling (Section 4.6.1.6.),
- Admission control using congestion notification based on probing (Section 4.6.1.7.).
The QNEs at the Edges of the RMD domain support both the RMD QoS Model and end-to-end QoS Models, and they handle the RESERVE message differently 🌈.
Quick note: the term end-to-end QoS Model covers any QoS Model that starts and ends outside the RMD-QOSM-aware domain. But it's possible for a QoS Model to be started and/or ended by the QNE Edges, and then it counts as an end-to-end QoS Model too 🙈. That happens when the QNE Edges can act as a QNI or a QNR, and at the same time they can act as either the sender or the receiver of the data path.
It's really important to emphasize that this section is used to specify the following RMD-QOSM/QoS-NSLP signaling schemes when basic unidirectional operation is assumed 🔥:
- "per-flow congestion notification based on probing";
- "per-flow RMD NSIS measurement-based admission control";
- "per-flow RMD reservation-based" together with the "severe congestion handling by the RMD-QOSM refresh" procedure;
- "per-flow RMD reservation-based" together with the "severe congestion handling by proportional data packet marking" procedure;
- "per-aggregate RMD reservation-based" together with the "severe congestion handling by the RMD-QOSM refresh" procedure;
- "per-aggregate RMD reservation-based" together with the "severe congestion handling by proportional data packet marking" procedure.
For more details, see Section 3.2.3 📚.
Specifically, the functionality in Sections 4.6.1.1, 4.6.1.2, 4.6.1.3, 4.6.1.5, 4.6.1.4, and 4.6.1.6 applies to the RMD reservation-based and the NSIS measurement-based admission control methods. The functionality in Section 4.6.1.7 applies to the admission control procedure that uses congestion notification based on probing. The QNE Edge nodes keep either per-flow QoS-NSLP operational and reservation states, or aggregated QoS-NSLP operational and reservation states 🧠.
When the QNE Edges keep aggregated QoS-NSLP operational and reservation states, the RMD-QOSM functionality MAY do an RMD modification procedure (see Section 4.6.1.4) instead of the reservation initiation procedure described in this subsection 🪄. Also, it is RECOMMENDED that QNE implementations of RMD-QOSM process QoS-NSLP signaling messages at a higher priority than data packets. You can do that as described in Section 3.3.4 of [RFC5974], and you can request it via the QoS-NSLP-RMF API described in [RFC5974] 🚀. The signaling scenarios in this section are carried out using the QoS-NSLP processing rules defined in [RFC5974], combined with the RMF triggers sent via the QoS-NSLP-RMF API described in [RFC5974].
According to Section 3.2.3, only the "per-flow RMD reservation-based" together with the "severe congestion handling by proportional data packet marking" scheme MUST be implemented within one RMD domain 👑. Still, all RMD QNEs that support this specification MUST support the combination of "per-flow RMD reservation-based" and "severe congestion handling by proportional data packet marking". If the RMD QNEs support more RMD-QOSM schemes, then the operator of that RMD domain MUST preconfigure all the QNE Edge nodes within one domain so the <SCH> field in the "PHR container" (Section 4.1.2) and the "PDR Container" (Section 4.1.3) always use the same value. That way, only one of the RMD-QOSM schemes described below is used at a time within one RMD domain 🎀.
All QNE nodes inside the RMD domain MUST read and interpret the <SCH> field in the "PHR container" before processing any of the other "PHR container" payload fields 💅. Also, all QNE Edge nodes at the border of the RMD domain MUST read and interpret the <SCH> field in the "PDR container" before processing any of the other <PDR container> payload fields 🫶.
4.6.1.1. Successful Reservation 🎉
This section describes how the RMD-QOSM works when a reservation succeeds ✅.
The QNI generates the initial RESERVE message, and the NTLP forwards it as usual [RFC5971].
4.6.1.1.1. Operation in Ingress Node 🚪
When an end-to-end reservation request (RESERVE) shows up at the Ingress node (QNE) (see Figure 8), it gets processed according to the end-to-end QoS Model. After that, the combination of <TMOD-1>, <PHB Class>, and <Admission Priority> is derived from the <QoS Desired> object of the initial QSPEC 🧁.
The QNE Ingress MUST keep track of the smallest MTU supported on the links inside the RMD domain 📏.
The <Maximum Packet Size-1 (MPS)> value in the end-to-end QoS Model <TMOD-1> parameter gets compared with that smallest MTU value supported by the links in the RMD domain. If "Maximum Packet Size-1 (MPS)" is larger than this smallest MTU value within the RMD domain, the end-to-end reservation request is rejected (see Section 4.6.1.1.2). Otherwise, the admission process keeps going 🙌.
The <TMOD-1> parameter in the original initiator QSPEC gets mapped into the equivalent RMD-Qspec <TMOD-1> parameter, which only represents the peak bandwidth in the local RMD-QSPEC. You can do this by setting the RMD-QSPEC <TMOD-1> fields like this: token rate (r) = peak traffic rate (p), bucket depth (b) = large, and minimum policed unit (m) = large 🪣.
Heads up that the bucket size (b) is measured in bytes. This parameter can range from 1 byte to 250 gigabytes; see [RFC2215]. So the biggest (b) could be is around 250 gigabytes 🤯. The minimum policed unit [m] is an integer measured in bytes, and it must be less than or equal to the Maximum Packet Size (MPS). That means the biggest (m) can be is (MPS). [Part94] and [TaCh99] describe a method for calculating some Token Bucket parameter values, like large values of (m) and (b), when the token rate (r), peak rate (p), and MPS are known 🧮.
The <Peak Data Rate-1 (p)> value of the end-to-end QoS Model <TMOD-1> parameter gets copied into the <Peak Data Rate-1 (p)> value of the local RMD-Qspec <TMOD-1>.
The MPS value of the end-to-end QoS Model <TMOD-1> parameter gets copied into the MPS value of the local RMD-Qspec <TMOD-1>.
If the initial QSPEC doesn't contain the <PHB Class> parameter, then the <PHB Class> carried by the intra-domain RMD-QSPEC gets picked by a local policy, similar to the procedures discussed in [RFC2998] and [RFC3175] 🎨.
For example, if the initial QSPEC is used by the IntServ Controlled Load QOSM, then the Expedited Forwarding (EF) PHB is a good fit for setting the <PHB Class> parameter carried by the intra-domain RMD-QSPEC (see [RFC3175]) 💜.
If the initial QSPEC doesn't carry the <Admission Priority> parameter, then the <Admission Priority> parameter in the RMD-QSPEC won't be populated. If the initial QSPEC doesn't carry <Admission Priority> but does carry other priority parameters, then Edges, being stateful nodes, are considered able to control the priority of the sessions entering or leaving the RMD domain according to those priority parameters 🦋.
Note that the RMF reservation states (see Section 4.3) in the QNE Edges store the <Admission Priority> value used inside the RMD domain, for use in preemption and severe congestion situations (see Section 4.6.1.6) 📦.
If the RMD domain supports preemption during the admission control process, then the QNE Ingress node can support the building blocks specified in [RFC5974] and, during admission control, use the example preemption handling algorithm described in Appendix A.7 ⚔️.
Note that in the case described above, the QNE Egress uses the tunneled initial priority parameters, if they're available and it can interpret them 🫧.
If the initial QSPEC carries the <Excess Treatment> parameter, then the QNE Ingress and QNE Egress nodes MUST control the excess traffic entering or leaving the RMD domain according to the <Excess Treatment> parameter. Note that the RMD-QSPEC does not carry the <Excess Treatment> parameter 🚧.
If the requested<TMOD-1>parameter carried by the initial QSPEC can't be satisfied, then an end-to-end RESPONSE message has to be generated. But to decide whether the end-to-end reservation request was satisfied locally (at the QNE Ingress), a local RMD-QOSM admission control procedure also has to be performed at QNE_Ingress 🙈. In other words, the RMD-QOSM functionality has to verify whether the value in the<Peak Data Rate-1 (p)>field of the RMD-QOSM<TMOD-1>can be reserved and stored in the RMD-QOSM reservation states (see Sections 4.6.1.1.2 and 4.3) 💀.
An initial QSPEC object MUST be included in the end-to-end RESPONSE message 💖 The parameters in the QSPEC <QoS Reserved> object just get copied over from the original <QoS Desired> values. Easy peasy 🎀
The <E> flag on the QSPEC <QoS Reserved> object and the <E> flag on the local RMD-QSPEC <TMOD-1> parameter both get set ✨ The <INFO-SPEC> object also goes into the end-to-end RESPONSE message, and its error code is:
Error severity class: Transient Failure
Error code value: Reservation failureAnd all the other RESPONSE parameters get set according to the end-to-end QoS Model or according to [RFC5974] and [RFC5975] 🧚
If the request was satisfied locally (see Section 4.3), the Ingress QNE node makes two RESERVE messages: one intra-domain and one end-to-end 👯 BUT, when the QNE Ingress uses aggregated QoS-NSLP operational and reservation states, making the intra-domain RESERVE depends on whether the aggregated QoS-NSLP operational state is available 🤔 If it is, then the RMD modification of aggregated reservations described in Section 4.6.1.4 is used.
Important tea ☕: when the "per-flow RMD reservation-based" scenario is used inside the RMD domain, retransmission within the RMD domain SHOULD be disallowed 🚫 Why? QNE Interior nodes can't tell a retransmitted RESERVE for one session apart from a brand new RESERVE for a different session 😭 Meanwhile, the QNE Ingress still has to report failures upstream. So when it sends an (intra-domain or end-to-end) RESERVE with the <RII> object set, it waits for a RESPONSE from the QNE Egress for a QOSNSLP_REQUEST_RETRY period.
If the QNE Ingress sent an intra-domain or end-to-end RESERVE with the <RII> object set, and the matching intra-domain or end-to-end RESPONSE never shows up after the QOSNSLP_REQUEST_RETRY period expires, it considers the reservation failed 💀 In that case, the QNE Ingress SHOULD generate an end-to-end RESPONSE message that includes, among other things, an <INFO-SPEC> object. The error code used by this <INFO-SPEC> object is:
Error severity class: Transient Failure
Error code value: Reservation failureAgain, all the other RESPONSE parameters are set according to the end-to-end QoS Model or according to [RFC5974] and [RFC5975] 🌸
Heads up though: if the retransmission within the RMD domain is not disallowed, then the procedure described in Appendix A.8 SHOULD be used on QNE Interior nodes; see also [Chan07]. In this case, the stateful QNE Ingress uses the retransmission procedure described in [RFC5974] 🔁
If a rerouting takes place, the stateful QNE Ingress follows the procedures specified in [RFC5974] 🗺️
At this point, the intra-domain and end-to-end operational states MUST be initiated or modified according to the REQUIRED binding procedures. The way the BOUND-SESSION-IDs get initiated and maintained in the intra-domain and end-to-end QoS-NSLP operational states is described in Sections 4.3.1 and 4.3.2 🎀
These two messages are bound together like this: the end-to-end RESERVE SHOULD contain, in the BOUND-SESSION-ID, the SESSION-ID of its bound intra-domain session 🫶
If the QNE Edge nodes maintain intra-domain per-flow QoS-NSLP reservation states, then the value of Binding_Code MUST be set to code "Tunnel and end-to-end sessions" (see Section 4.3.2) 🦋
On top of that, the intra-domain and end-to-end RESERVE messages are bound using the Message binding procedure described in [RFC5974]. Specifically, the <MSG-ID> object is included in the intra-domain RESERVE message, and its bound <BOUND-MSG-ID> object is carried by the end-to-end RESERVE message. The <Message_Binding_Type> flag is SET (value is 1), so the message dependency is bidirectional 💜
If the QoS-NSLP Edges maintain aggregated intra-domain QoS-NSLP operational states, then the value of Binding_Code MUST be set to code "Aggregated sessions" 🌈
In this case, retransmission within the RMD domain is allowed, and the procedures described in Appendix A.8 SHOULD be used on QNE Interior nodes. This is necessary because, if retransmissions were disallowed, the (micro) flows belonging to the aggregate would loose their reservations 😱 Note that, in this case, the stateful QNE Ingress uses the retransmission procedure described in [RFC5974].
The intra-domain RESERVE message is associated with the (local NTLP) SESSION-ID mentioned above. The IP source and IP destination addresses of this message depend on how the different inter-domain (end-to-end) flows are aggregated by the QNE Ingress node (see Section 4.3.1). As described in Section 4.3.1, the QNE Edges maintain either per-flow or aggregated QoS-NSLP reservation states for the RMD QoS Model, which are identified by (local NTLP) SESSION-IDs (see [RFC5971]). This NTLP SESSION-ID is a different one than the SESSION-ID associated with the end-to-end RESERVE message 🧩
If no QoS-NSLP aggregation procedure at the QNE Edges is supported, then the IP source and IP destination address of this message MUST be equal to the IP source and IP destination addresses of the data flow. The intra-domain RESERVE message is sent using the NTLP datagram mode (see Sections 4.4 and 4.5). The GIST Datagram mode can be selected using the unreliable GIST API Transfer-Attributes. Also, the intra-domain RESERVE (RMD-QSPEC) message MUST include a PHR container (PHR_Resource_Request) and the RMD QOSM <QoS Desired> object 📦
The end-to-end RESERVE message includes the initial QSPEC and is sent towards the Egress QNE 🚀
After completing the initial discovery phase, the GIST Connection mode can be used between the QNE Ingress and QNE Egress. The GIST Connection mode can be selected using the reliable GIST API Transfer-Attributes 🔒
The end-to-end RESERVE message is forwarded using the GIST forwarding procedure to bypass the Interior stateless or reduced-state QNE nodes; see Figure 8. The bypassing procedure is described in Section 4.4 🏃♀️
At the QNE Ingress, the end-to-end RESERVE message is marked, i.e., the QoS-NSLP default NSLPID value gets changed to another predefined NSLPID value. That value is used by the GIST message carrying the end-to-end RESPONSE message so it can bypass the QNE Interior nodes. The QNE Interior nodes (see [RFC5971]) are configured to handle only certain NSLP-IDs (see [RFC5974]) 🏷️
Furthermore, the initial discovery phase and the process of sending the end-to-end RESERVE message towards the QNE Egress MAY be done simultaneously. This can only happen if the GIST implementation is configured to do it, e.g., via a local policy. However, the RMD-QOSM cannot select the discovery procedure 🙈
The (initial) intra-domain RESERVE message MUST be sent by the QNE Ingress, and it MUST contain the following values (see the QoS-NSLP-RMF API described in [RFC5974]):
- the <RSN> object, whose value is generated and processed as described in [RFC5974];
- the <SCOPING> flag MUST NOT be set, meaning that a default scoping of the message is used. Therefore, the QNE Edges MUST be configured as RMD boundary nodes, and the QNE Interior nodes MUST be configured as Interior (intermediary) nodes;
- the <RII> MUST be included in this message, see [RFC5974];
- the <REPLACE> flag MUST be set to FALSE = 0;
- the <Message ID> value carried by the <MSG-ID> object is set according to [RFC5974]. The value of the <Message_Binding_Type> is set to "1";
- the value of the <REFRESH-PERIOD> object MUST be calculated and set by the QNE Ingress node as described in Section 4.6.1.3;
- the value of the <PACKET-CLASSIFIER> object is associated with the path-coupled routing Message Routing Message (MRM), since RMD-QOSM is used with the path-coupled MRM. The flag that has to be set is the <T> flag (traffic class), meaning that the packet classification of packets is based on the <DSCP> value included in the IP header of the packets. Note that the <DSCP> value used in the MRI can be derived from the value of the <PHB Class> parameter, which MUST be carried by the intra-domain RESERVE message. Also, since the QNE Ingress is a QNI for the intra-domain session, it can pass this value to GIST via the GIST API;
- the PHR resource units MUST be included in the <Peak Data Rate-1 (p)> field of the local RMD-QSPEC <TMOD-1> parameter of the <QoS Desired> object.
- When the QNE Edges use per-flow intra-domain QoS-NSLP states, the <Peak Data Rate-1 (p)> value included in the initial QSPEC <TMOD-1> parameter is copied into the <Peak Data Rate-1 (p)> value of the local RMD-QSPEC <TMOD-1> parameter.
- When the QNE Edges use aggregated intra-domain QoS-NSLP operational states, the <Peak Data Rate-1 (p)> value of the local RMD-QSPEC <TMOD-1> parameter can be obtained by using the bandwidth aggregation method described in Section 4.3.1;
- the value of the <PHB Class> parameter can be defined by using the method of copying the <PHB Class> parameter carried by the initial QSPEC into the <PHB Class> carried by the RMD-QSPEC, which is described above in this subsection;
- the value of the <Parameter ID> field of the PHR container MUST be set to "17" (i.e., PHR_Resource_Request);
- the value of the <Admitted Hops> parameter in the PHR container MUST be set to "1". Note that during a successful reservation, each time an RMD-QOSM-aware node processes the RMD-QSPEC, the <Admitted Hops> parameter is increased by one;
- the value of the <Hop_U> parameter in the PHR container MUST be set to "0";
- the value of the <Max Admitted Hops> is set to "0";
- if the initial QSPEC carried an <Admission Priority> parameter, then this parameter SHOULD be copied into the RMD-QSPEC and carried by the (initiating) intra-domain RESERVE. Note that for the RMD-QOSM, a reservation established without an <Admission Priority> parameter is equivalent to a reservation with an <Admission Priority> value of 1. Also, each admission priority is associated with a priority traffic class. The three priority traffic classes (PHB_low_priority, PHB_normal_priority, and PHB_high_priority) MAY be associated with the same PHB (see Section 4.3.3);
- in a single RMD domain case, the PDR container MAY not be included in the message.
Note that the intra-domain RESERVE message does not carry the <BOUND-SESSION-ID> object. The reason for this is that the end-to-end RESERVE carries, in the <BOUND-SESSION-ID> object, the <SESSION-ID> value of the intra-domain session 🧠
When an end-to-end RESPONSE message is received by the QNE Ingress node, and it was sent by a QNE Egress node (see Section 4.6.1.1.3), it is processed according to [RFC5974] and the end-to-end QoS Model rules ✨
When an intra-domain RESPONSE message reaches the QNE Ingress node, and it was sent by a QNE Egress (see Section 4.6.1.1.3), the ingress uses the QoS-NSLP procedures to match it to the intra-domain RESERVE message it sent earlier. 💌 After that, the RMD-QSPEC has to be identified and processed. Bestie, this is the moment the loop closes ✨
The RMD QOSM reservation was successful if the <M> bit carried by the "PDR Container" is equal to "0" (i.e., not set). 🎀
The <INFO-SPEC> object is processed as defined in the QoS-NSLP specification. If the reservation succeeded, the <INFO-SPEC> object MUST have these values:
- Error severity class:
Success - Error code value:
Reservation successful
If the end-to-end RESPONSE has to be forwarded to a node outside the RMD-QOSM-aware domain, then the values of the objects in that message (<RII> <RSN>, <INFO-SPEC>, [<QSPEC>]) MUST be set by the QoS-NSLP protocol functions of the QNE. 🧚 If an end-to-end QUERY shows up at the QNE Ingress, it gets the same bypassing treatment as an end-to-end RESERVE message. It's forwarded with the GIST forwarding procedure, so it skips the Interior stateless or reduced-state QNE nodes. Main character energy 👑
4.6.1.1.2. Operation in the Interior Nodes
Each QNE Interior node MUST use the QoS-NSLP and RMD-QOSM parameters of the intra-domain RESERVE (RMD-QSPEC) message as follows (see QoS-NSLP-RMF API described in [RFC5974]):
- The values of the
<RSN>,<RII>,<PACKET-CLASSIFIER>,<REFRESH-PERIOD>, objects MUST NOT be changed.
The Interior node is informed by the <PACKET-CLASSIFIER> object that the packet classification SHOULD be done on the <DSCP> value. The flag that has to be set in this case is the <T> flag (traffic class). The <DSCP> value MUST be obtained via the MRI parameters that the QoS-NSLP receives from GIST. A QNE Interior MUST be able to associate the value carried by the RMD-QSPEC <PHB Class> parameter with the <DSCP> value obtained via GIST. This is REQUIRED, because there are situations in which the <PHB Class> parameter is not carrying a <DSCP> value but a PHB ID code, see Section 4.1.1. 🦋
- The flag
<REPLACE>MUST be set to FALSE = 0; - When the RMD reservation-based methods, described in Section 4.3.1 and 4.3.3, are used, the
<Peak Data Rate-1 (p)>value of the local RMD-QSPEC<TMOD-1>parameter is used by the QNE Interior node for admission control. Furthermore, if the<Admission Priority>parameter is carried by the RMD-QOSM<QoS Desired>object, then this parameter is processed as described in the following bullets. 💜 - In the case of the RMD reservation-based procedure, and if these resources are admitted (see Sections 4.3.1 and 4.3.3), they are added to the currently reserved resources. Furthermore, the value of the
<Admitted Hops>parameter in the PHR container has to be increased by one. 🌸 - If the bandwidth allocated for the
PHB_high_prioritytraffic is fully utilized, and a high priority request arrives, other policies on allocating bandwidth can be used, which are beyond the scope of this document. 🍓 - If the RMD domain supports preemption during the admission control process, then the QNE Interior node can support the building blocks specified in [RFC5974] and, during the admission control process, use the preemption handling algorithm specified in Appendix A.7. 🔥
- In the case of the RMD measurement-based method (see Section 4.3.2), and if the requested value in the
<Peak Data Rate-1 (p)>of the local RMD-QSPEC<TMOD-1>parameter is admitted using a measurement-based admission control (MBAC) algorithm, then the number of this resource will be used to update the MBAC algorithm according to the operation described in Section 4.3.2. ⭐
4.6.1.1.3. Operation in the Egress Node 🦋
okay so the end-to-end RESERVE finally reaches the egress node 📬 and it only gets forwarded on toward QNR if the intra-domain RESERVE(RMD-QSPEC) processing worked at ALL the nodes in the RMD domain. no partial wins here 🙅♀️ If that's a go, the QNE Egress MUST stop the marking process that was used to bypass the QNE Interior nodes, by reassigning the QoS-NSLP default NSLPID value to the end-to-end RESERVE message (see Section 4.4). The carried <BOUND-SESSION-ID> object for the intra-domain session MUST be removed after processing 🧹 Also, the end-to-end RESERVE was tunneled inside the RMD domain, so the tunneled initial QSPEC it carries has to be processed/set according to [RFC5975] 🎀
If a reroute happens, the stateful QNE Egress follows the procedures in [RFC5974] 🗺️
Now the intra-domain and end-to-end operational states MUST be initiated or modified according to the REQUIRED binding procedures 🔗 How the BOUND-SESSION-IDs get started and kept up in those states is described in Sections 4.3.1 and 4.3.2 ✨
If the intra-domain RESERVE(RMD-QSPEC) processing wasn't successful at all nodes in the RMD domain, the inter-domain (end-to-end) reservation is considered to have failed 💀
Next, if the initial QSPEC object used an object combination of type 1 or 2 where the <QoS Available> is populated, and the intra-domain RESERVE(RMD-QSPEC) was not successful at all nodes in the RMD domain, it MUST be considered that the <QoS Available> is not satisfied and that the inter-domain (end-to-end) reservation is considered to have failed 😭
Also, if the QNE Egress uses per-flow intra-domain QoS-NSLP operational states (see Sections 4.3.2 and 4.3.3), the QNE Egress SHOULD support the message binding procedure described in [RFC5974]. That procedure can be used to synchronize the arrival of the end-to-end RESERVE and the intra-domain RESERVE (RMD-QSPEC) messages (see Section 5.7), and it goes with the QoS-NSLP-RMF API described in [RFC5974] 🧩 The intra-domain RESERVE message carries the <MSG-ID> object, and its bound end-to-end RESERVE message carries the <BOUND-MSG-ID> object. Both of these objects have the <Message_Binding_Type> flag set to the value of "1" 🔢 If these two messages don't show up during the time defined by the MsgIDWait timer, the reservation is considered to have failed ⏰ The timer has to be preconfigured, and it has to have the same value in the RMD domain. In that case, an end-to-end RESPONSE message (see the QoS-NSLP-RMF API described in [RFC5974]) is sent towards the QNE Ingress with the following <INFO-SPEC> values:
Error class: Transient Failure
Error code: Mismatch synchronization between end-to-end RESERVE
and intra-domain RESERVEWhen the intra-domain RESERVE (RMD-QSPEC) is received by the QNE Egress node of the session associated with it (the PHB session), the session included in its <BOUND-SESSION-ID> object MUST be bound according to the specification given in [RFC5974]. The SESSION-ID included in the BOUND-SESSION-ID parameter stored in the intra-domain QoS-NSLP operational state object is the SESSION-ID of the session associated with the end-to-end RESERVE message(s) 💜 If the QNE Edge nodes keep per-flow intra-domain QoS-NSLP operational states, then the value of Binding_Code = (Tunnel and end-to-end sessions) is used. If the QNE Edge nodes keep per-aggregated QoS-NSLP intra-domain reservation states, then the value of Binding_Code = (Aggregated sessions) is used (see Sections 4.3.1 and 4.3.2) 🌸
If the RMD domain supports preemption during the admission control process, then the QNE Egress node can support the building blocks specified in [RFC5974] and, during the admission control process, can use the example preemption handling algorithm described in Appendix A.7 ⚔️
The end-to-end RESERVE message is generated/forwarded further upstream according to the [RFC5974] and [RFC5975] specifications. And the <B> (BREAK) QoS-NSLP flag in the end-to-end RESERVE message MUST NOT be set (see the QoS-NSLP-RMF API described in QoS-NSLP) 🚫
QNE(Ingress) QNE(Interior) QNE(Interior) QNE(Egress)
NTLP stateful NTLP stateless NTLP stateless NTLP stateful
| | | |
RESERVE | | |
--->| | | RESERVE |
|------------------------------------------------------------>|
|RESERVE(RMD-QSPEC) | | |
|------------------->| | |
| |RESERVE(RMD-QSPEC) | |
| |------------------>| |
| | | RESERVE(RMD-QSPEC) |
| | |------------------->|
| |RESPONSE(RMD-QSPEC)| |
|<------------------------------------------------------------|
| | | RESERVE
| | | |-->
| | | RESPONSE
| | | |<--
| |RESPONSE | |
|<------------------------------------------------------------|
RESPONSE | | |
<---| | | |
Figure 8: Basic operation of successful reservation procedure
used by the RMD-QOSMThe QNE Egress MUST generate an intra-domain RESPONSE (RMD-Qspec) message 📨 The intra-domain RESPONSE (RMD-QSPEC) message MUST be sent to the QNE Ingress node, which is the previous stateful hop, using the procedures described in Sections 4.4 and 4.5 🏃♀️
The values of the RMD-QSPEC carried by the intra-domain RESPONSE message MUST be used and/or set in the following way (see the QoS-NSLP-RMF API described in [RFC5974]):
- the
<RII>object carried by the intra-domain RESERVE message (see Section 4.6.1.1.1) has to be copied and carried by the intra-domain RESPONSE message 🪞 - the value of the
<Parameter ID>field of the PDR container MUST be set to "23" (i.e., PDR_Reservation_Report) 🏷️ - the value of the
<M>field of the PDR container MUST be equal to the value of the<M>parameter of the PHR container that was carried by its associated intra-domain RESERVE(RMD-QSPEC) message. This is REQUIRED since the value of the<M>parameter is used to indicate the status of the RMD reservation request to the Ingress Edge 🚦
If the binding between the intra-domain session and the end-to-end session uses a Binding_Code of (Aggregated sessions), and there is no aggregated QoS-NSLP operational state associated with the intra-domain session available, then the RMD modification of aggregated reservation procedure described in Section 4.6.1.4 can be used 🔁
If the QNE Egress receives an end-to-end RESPONSE message, it is processed and forwarded towards the QNE Ingress. In particular, the non-default values of the objects contained in the end-to-end RESPONSE message MUST be used and/or set by the QNE Egress as follows (see the QoS-NSLP-RMF API described in [RFC5974]):
- the values of the
<RII>,<RSN>,<INFO-SPEC>, [<QSPEC>] objects are set according to [RFC5974] and/or [RFC5975]. The<INFO-SPEC>object SHOULD be set by the QoS-NSLP functionality. In the case of successful reservation, the<INFO-SPEC>object SHOULD have the following values:
Error severity class: Success Error code value: Reservation
successful- furthermore, an initial QSPEC object MUST be included in the end-to-end RESPONSE message. The parameters included in the QSPEC
<QoS Reserved>object are copied from the original<QoS Desired>values 🎯
The end-to-end RESPONSE message is delivered as normal, i.e., it is addressed and sent to its upstream QoS-NSLP neighbor, the QNE Ingress node 💌
Note that if a QNE Egress receives an end-to-end QUERY that was bypassed through the RMD domain, it MUST stop the marking process that was used to bypass the QNE Interior nodes. This can be done by reassigning the QoS-NSLP default NSLPID value to the end-to-end QUERY message (see Section 4.4) 🔚
4.6.1.2. Unsuccessful Reservation 🥀
This subsection covers the operation where a request for reservation can't be satisfied by the RMD-QOSM 🙃
The QNE Ingress, the QNE Interior, and QNE Egress nodes process and forward the end-to-end RESERVE message and the intra-domain RESERVE(RMD-QSPEC) message in a similar way, as specified in Section 4.6.1.1. The big difference between the unsuccessful and successful operation is that one of the QNE nodes doesn't admit the request, e.g., due to lack of resources. This also means that the QNE Edge node MUST NOT forward the end-to-end RESERVE message towards the QNR node 🛑
Note that the described functionality applies to the RMD reservation-based methods (see Sections 4.3.1 and 4.3.2) and to the NSIS measurement-based admission control method (see Section 4.3.2) 📊
The QNE Edge nodes maintain either per-flow QoS-NSLP reservation states or aggregated QoS-NSLP reservation states. When the QNE Edges maintain aggregated QoS-NSLP reservation states, the RMD-QOSM functionality MAY accomplish an RMD modification procedure (see Section 4.6.1.4), instead of the reservation initiation procedure described in this subsection 🔄✨
4.6.1.2.1. Operation in the Ingress Nodes ✨
okay so bestie, picture this 💅. When an end-to-end RESERVE message shows up at the QNE Ingress, the Ingress checks two things. First, if the "Maximum Packet Size-1 (MPS)" in the end-to-end QoS Model <TMOD-1> is bigger than the smallest MTU value inside the RMD domain, it's a no-go 🙅♀️. Second, if there are no resources available, also a no-go. Either way, the QNE Ingress MUST reject that end-to-end RESERVE and send an end-to-end RESPONSE back to the sender, as described in the QoS-NSLP specification, see [RFC5974] and [RFC5975]. Rejected and reported, no drama 💀.
When an end-to-end RESPONSE message is received by an Ingress node (see Section 4.6.1.2.3), the values of the <RII>, <RSN>, <INFO-SPEC>, and [<QSPEC>] objects get processed according to the QoS-NSLP procedures 🎀.
If the end-to-end RESPONSE has to be forwarded upstream to a node outside the RMD-QOSM-aware domain, then the values of the objects in that message (i.e., <RII<, <RSN>, <INFO-SPEC>, [<QSPEC>]) MUST be set by the QoS-NSLP protocol functions of the QNE. The QNE handles the paperwork before it leaves the building 📦.
When an intra-domain RESPONSE message is received by the QNE Ingress node, and it was sent by a QNE Egress (see Section 4.6.1.2.3), the Ingress uses the QoS-NSLP procedures to match it to the intra-domain RESERVE message it sent earlier 🔍. After that, the RMD-QSPEC has to be identified and processed. Here's the tea ☕: the RMD Resource Management Function (RMF) gets notified that the reservation didn't work, by reading the <M> parameter of the PDR container. Also, if the QNE Edges keep a per-flow QoS-NSLP reservation state, the RMD-QOSM functionality has to start an RMD release procedure (see Section 4.6.1.5). But if the QNE Edges keep aggregated QoS-NSLP reservation states, the RMD-QOSM functionality MAY start an RMD modification procedure (see Section 4.6.1.4). Optional, but iconic 🌈.
4.6.1.2.2. Operation in the Interior Nodes 🧚
Okay, so in the RMD reservation-based scenario, if the QNE Interior node doesn't admit the intra-domain reservation request, then the <Hop_U> and <M> parameters of the PHR container MUST be set to "1". The <Admitted Hops> counter MUST NOT be increased 🙅♀️. And the <Max Admitted Hops> counter MUST be set equal to the <Admitted Hops> value. Tidy 🧹.
Furthermore, the <E> flag on the QSPEC <QoS Desired> object and the <E> flag on the local RMD-QSPEC <TMOD-1> parameter SHOULD be set 🌸. In the RMD measurement-based scenario, the <M> parameter of the PHR container MUST be set to "1". Then the <E> flag on the QSPEC <QoS Desired> object and the <E> flag on the local RMD-QSPEC <TMOD-1> parameter SHOULD be set again 🔁. Lowkey confusing, but here's the thing: the <M> flag gets set in a similar way to the <E> flag used by the local RMD-QSPEC <TMOD-1> parameter. The two flags are processed by a QNE in different ways though, so don't mix them up 🙈.
In general, if a QNE Interior node gets an RMD-QSPEC <TMOD-1> parameter with the <E> flag set, plus a PHR container type "PHR_Resource_Request" with the <M> parameter set to "1", then this "PHR Container" and the RMD-QOSM <QoS Desired> object) MUST NOT be processed 🚫. Same vibe when the <K> parameter inside the "PHR Container" is set to "1" on a RESERVE message: that "PHR Container" and the RMD-QOSM <QoS Desired> object) MUST NOT be processed either 🚫.
4.6.1.2.3. Operation in the Egress Nodes 👑
In the RMD reservation-based (Section 4.3.3) and RMD NSIS measurement-based scenarios (Section 4.3.2), when the <M> marked intra-domain RESERVE(RMD-QSPEC) reaches the QNE Egress node (see Figure 9), the session tied to that intra-domain RESERVE(RMD-QSPEC) (the PHB session) and the end-to-end session MUST be bound 🔗. Main character bonding moment ✨.
Moreover, if the initial QSPEC object (used by the end-to-end QoS Model) used an object combination of type 1 or 2 where the <QoS Available> is populated, and the intra-domain RESERVE(RMD-QSPEC) was not successful at all nodes in the RMD domain (so the intra-domain RESERVE(RMD-QSPEC) message is marked), it MUST be considered that the <QoS Available> is not satisfied. The inter-domain (end-to-end) reservation is then considered to have failed 💔.
When the QNE Egress uses per-flow intra-domain QoS-NSLP operational states (see Sections 4.3.2 and 4.3.3), the QNE Egress node MUST generate an end-to-end RESPONSE message. That message has to go to its previous stateful QoS-NSLP hop (see the QoS-NSLP-RMF API described in [RFC5974]) 📬.
- The values of the
<RII>,<RSN>and<INFO-SPEC>objects are set by the standard QoS-NSLP protocol functions. In the case of an unsuccessful reservation, the<INFO-SPEC>object SHOULD have these values:
Error severity class: Transient Failure
Error code value: Reservation failureThe QSPEC that was carried by the end-to-end RESERVE message belonging to the same session as this end-to-end RESPONSE message is included in this message 🎁.
In particular, the parameters in the QSPEC <QoS Reserved> object of the end-to-end RESPONSE message get copied from the initial <QoS Desired> values in its associated end-to-end RESERVE message 📋. The <E> flag on the QSPEC <QoS Reserved> object and the <E> flag on the <TMOD-1> parameter in the end-to-end RESPONSE are both set 🚩.
On top of that, and similar to the successful operation (see Section 4.6.1.1.3), the QNE Egress MUST generate an intra-domain RESPONSE message that has to be sent to its previous stateful QoS-NSLP hop 💌. Double delivery, no cap 🧢.
The values of the <RII>, <RSN> and <INFO-SPEC> objects are set by the standard QoS-NSLP protocol functions. In the case of an unsuccessful reservation, the <INFO-SPEC> object SHOULD have these values (see the QoS-NSLP-RMF API described in [RFC5974]):
Error severity class: Transient Failure
Error code value: Reservation failureQNE(Ingress) QNE(Interior) QNE(Interior) QNE(Egress)
NTLP stateful NTLP stateless NTLP stateless NTLP stateful
| | | |
RESERVE | | |
--->| | | RESERVE |
|------------------------------------------------------------>|
|RESERVE(RMD-QSPEC:M=0) | |
|------------------->| | |
| |RESERVE(RMD-QSPEC:M=1) |
| |------------------>| |
| | | RESERVE(RMD-QSPEC:M=1)
| | |------------------->|
| |RESPONSE(RMD-QOSM) | |
|<------------------------------------------------------------|
| |RESPONSE | |
|<------------------------------------------------------------|
RESPONSE | | |
<---| | | |
RESERVE(RMD-QSPEC: Tear=1, M=1, <Admitted Hops>=<Max Admitted Hops>
|------------------->| | |
|RESERVE(RMD-QSPEC: Tear=1, M=1, K=1) |
| |------------------>| |
| RESERVE(RMD-QSPEC: Tear=1, M=1, K=1)|
| | |------------------->|
Figure 9: Basic operation during unsuccessful reservation
initiation used by the RMD-QOSMThe values of the RMD-QSPEC MUST be used and/or set in the following way (see the QoS-NSLP-RMF API described in [RFC5974]) 🧠:
- the value of the
<PDR Control Type>of the PDR container MUST be set to"23"(PDR_Reservation_Report); - the value of the
<Max Admitted Hops>parameter of the PHR container included in the received<M>marked intra-domain RESERVE (RMD-QSPEC) MUST be included in the<Max Admitted Hops>parameter of the PDR container; - the value of the
<M>parameter of the PDR container MUST be"1".
4.6.1.3. RMD Refresh Reservation 🔄
Okay, so in the RMD measurement-based method (see Section 4.3.2), QoS-NSLP reservation states in the RMD domain usually aren't kept around 🫠. That's why this method typically doesn't use an intra-domain refresh procedure.
However, there are measurement-based optimization schemes, see [GrTs03], that MAY use the refresh procedures described in Sections 4.6.1.3.1 and 4.6.1.3.3 ✨. This optimization scheme can only be applied in the RMD domain if the QNE Edges are configured to perform intra-domain refresh procedures, AND if all the QNE Interior nodes are configured to perform the measurement-based optimization schemes. Big if, big energy 🙌.
In the description in this subsection, it's assumed that the RMD measurement-based scheme does not use the refresh procedures 🙅♀️.
When the QNE Edges maintain aggregated or per-flow QoS-NSLP operational and reservation states (see Sections 4.3.1 and 4.3.3), the refresh procedures are very similar 👯. If the RESERVE messages arrive within the soft state timeout period, the corresponding number of resource units are not removed. However, the transmission of the intra-domain and end-to-end (refresh) RESERVE messages are not necessarily synchronized 🕰️. Also, the generation of the end-to-end RESERVE message by the QNE Edges depends on the locally maintained refreshed interval (see [RFC5974]) 💖.
4.6.1.3.1. Operation in the Ingress Node ✨
okay bestie, the Ingress node MUST be able to generate an intra-domain (refresh) RESERVE(RMD-QSPEC) at any time the refresh period/timer says so 🕰️💅. Before it sends that message, the RMD QoS signaling model functionality uses the RMD traffic class (PHR) resource units to refresh the RMD traffic class state 🦋.
Note that the RMD traffic class refresh periods MUST be equal in all QNE Edge and QNE Interior nodes and SHOULD be smaller (default: more than two times smaller) than the refresh period at the QNE Ingress node used by the end-to-end RESERVE message 🌸. The intra-domain RESERVE (RMD-QSPEC) message MUST include an RMD-QOSM <QoS Desired> and a PHR container (i.e., PHR_Refresh_Update) 🎀.
Here's the refresh operation in action, Figure 10 🍓:
QNE(Ingress) QNE(Interior) QNE(Interior) QNE(Egress)
NTLP stateful NTLP stateless NTLP stateless NTLP stateful
| | | |
|RESERVE(RMD-QSPEC) | | |
|------------------->| | |
| |RESERVE(RMD-QSPEC) | |
| |------------------>| |
| | | RESERVE(RMD-QSPEC) |
| | |------------------->|
| | | |
| |RESPONSE(RMD-QSPEC)| |
|<------------------------------------------------------------|
| | | |Figure 10: Basic operation of RMD-specific refresh procedure
Most of the non-default values of the objects in this message MUST be used and set by the QNE Ingress in the same way as described in Section 4.6.1.1 🫶. These objects are used and/or set differently:
- the PHR resource units MUST be included in the <Peak Data Rate-1 (p)> field of the local RMD-QSPEC <TMOD-1> parameter. The <Peak Data Rate-1 (p)> field value of the local RMD-QSPEC <TMOD-1> parameter depends on how the different inter-domain (end-to-end) flows are aggregated by the QNE Ingress node (e.g., the sum of all the PHR-requested resources of the aggregated flows); see Section 4.3.1. If no QoS-NSLP aggregation is accomplished by the QNE Ingress node, the <Peak Data Rate-1 (p)> value of the local RMD-QSPEC <TMOD-1> parameter SHOULD be equal to the <Peak Data Rate-1 (p)> value of the local RMD-QSPEC <TMOD-1> parameter of its associated new (initial) intra-domain RESERVE (RMD-QSPEC) message; see Section 4.3.3.
- the value of the Container field of the <PHR Container> MUST be set to "19", i.e., "PHR_Refresh_Update".
When the intra-domain RESPONSE (RMD-QSPEC) message (see Section 4.6.1.3.3), is received by the QNE Ingress node, then:
- the values of the <RII>, <RSN>, <INFO-SPEC>, and [RFC5975] objects are processed by the standard QoS-NSLP protocol functions (see Section 4.6.1.1);
- the "PDR Container" has to be processed by the RMD-QOSM functionality in the QNE Ingress node. The RMD-QOSM functionality is notified by the <PDR M> parameter of the PDR container that the refresh procedure has been successful or unsuccessful. All sessions associated with this RMD-specific refresh session MUST be informed about the success or failure of the refresh procedure. (When aggregated QoS-NSLP operational and reservation states are used (see Section 4.3.1), there will be more than one session.) In the case of failure, the QNE Ingress node has to generate (in a standard QoS-NSLP way) an error end-to-end RESPONSE message that will be sent towards the QNI 🙈.
4.6.1.3.2. Operation in the Interior Node 🧚
The intra-domain RESERVE (RMD-QSPEC) message is received and processed by the QNE Interior nodes 📬. Any QNE Edge or QNE Interior node that receives a <PHR_Refresh_Update> field MUST identify the traffic class state (PHB) (using the <PHB Class> parameter). Most of the parameters in this refresh intra-domain RESERVE (RMD-QSPEC) message MUST be used and/or set by a QNE Interior node in the same way as described in Section 4.6.1.1 💜.
The following objects are used and/or set differently:
- the <Peak Data Rate-1 (p)> value of the local RMD-QSPEC <TMOD-1> parameter of the RMD-QOSM <QoS Desired> is used by the QNE Interior node for refreshing the RMD traffic class state. These resources (included in the <Peak Data Rate-1 (p)> value of local RMD-QSPEC <TMOD-1>), if reserved, are added to the currently reserved resources per PHB and therefore they will become a part of the per-traffic class (PHB) reservation state (see Sections 4.3.1 and 4.3.3). If the refresh procedure cannot be fulfilled then the <M> and <S> fields carried by the PHR container MUST be set to "1".
- furthermore, the <E> flag associated with <QoS Desired> object and the <E> flag associated with the local RMD-QSPEC <TMOD-1> parameter SHOULD be set.
Any PHR container of type "PHR_Refresh_Update", and its associated local RMD-QSPEC <TMOD-1>, whether or not it is marked and independent of the <E> flag value of the local RMD-QSPEC <TMOD-1> parameter, is always processed, but marked bits are not changed 🌈.
4.6.1.3.3. Operation in the Egress Node 🌙
The intra-domain RESERVE(RMD-QSPEC) message is received and processed by the QNE Egress node. A new intra-domain RESPONSE (RMD-QSPEC) message is generated by the QNE Egress node and MUST include a PDR (type PDR_Refresh_Report) 📣.
The (refresh) intra-domain RESPONSE (RMD-QSPEC) message MUST be sent to the QNE Ingress node, i.e., the previous stateful hop. The (refresh) intra-domain RESPONSE (RMD-QSPEC) message MUST be explicitly routed to the QNE Ingress node, i.e., the previous stateful hop, using the procedures described in Section 4.5 🗺️.
- the values of the <RII>, <RSN>, and <INFO-SPEC> objects are set by the standard QoS-NSLP protocol functions, see [RFC5974].
- the value of the <PDR Control Type> parameter of the PDR container MUST be set "24" (i.e., PDR_Refresh_Report). In case of successful reservation, the <INFO-SPEC> object SHOULD have the following values:
Error severity Class: Success
Error code value: Reservation successful- In the case of unsuccessful reservation the <INFO-SPEC> object SHOULD have the following values:
Error severity class: Transient Failure
Error code value: Reservation failureThe RMD-QSPEC that was carried by the intra-domain RESERVE belonging to the same session as this intra-domain RESPONSE is included in the intra-domain RESPONSE message 🎁. The parameters included in the QSPEC <QoS Reserved> object are copied from the original <QoS Desired> values. If the reservation is unsuccessful, then the <E> flag associated with the QSPEC <QoS Reserved> object and the <E> flag associated with the local RMD-QSPEC <TMOD-1> parameter are set. Furthermore, the <M> and <S> PDR container bits are set to "1" 💀.
4.6.1.4. RMD Modification of Aggregated Reservations 🔁
In the case when the QNE Edges maintain QoS-NSLP-aggregated operational and reservation states and the aggregated reservation has to be modified (see Section 4.3.1), the following procedure is applied 🧩:
- When the modification request requires an increase of the reserved resources, the QNE Ingress node MUST include the corresponding value into the <Peak Data Rate-1 (p)> value of the local RMD-QSPEC <TMOD-1> parameter of the RMD-QOSM <QoS Desired>, which is sent together with a "PHR_Resource_Request" control information. If a QNE Edge or QNE Interior node is not able to reserve the number of requested resources, the "PHR_Resource_Request" that is associated with the local RMD-QSPEC <TMOD-1> parameter MUST be <M> marked, i.e., the <M> bit is set to the value of "1". In this situation, the RMD-specific operation for unsuccessful reservation will be applied (see Section 4.6.1.2).
- When the modification request requires a decrease of the reserved resources, the QNE Ingress node MUST include this value into the <Peak Data Rate-1 (p)> value of the local RMD-QSPEC <TMOD-1> parameter of the RMD-QOSM <QoS Desired>. Subsequently, an RMD release procedure SHOULD be accomplished (see Section 4.6.1.5). Note that if the complete bandwidth associated with the aggregated reservation maintained at the QNE Ingress does not have to be released, then the <TEAR> flag MUST be set to OFF. This is because the NSLP operational states associated with the aggregated reservation states at the Edge QNEs MUST NOT be turned off. However, if the complete bandwidth associated with the aggregated reservation maintained at the QNE Ingress has to be released, then the <TEAR> flag MUST be set to ON.
It is important to emphasize that this RMD modification scheme only applies to the following two RMD-QOSM schemes 🌟:
- "per-aggregate RMD reservation-based" in combination with the "severe congestion handling by the RMD-QOSM refresh" procedure;
- "per-aggregate RMD reservation-based" in combination with the "severe congestion handling by proportional data packet marking" procedure.
4.6.1.5. RMD Release Procedure ✨
okay bestie, this procedure is the cleanup crew 🧹💅 It applies to every RMD mechanism that keeps reservation states. If a refresh RESERVE message doesn't show up at a QNE Interior node within the refresh timeout period, then the bandwidth that refresh RESERVE was asking for doesn't get updated. So the reserved bandwidth tied to that reduced state drops in the next refresh period by the amount that never got refreshed, see Section 4.3.3 🦋
This soft state behavior is lowkey genius because it keeps the system robust and stops unused resources from sitting around reserved forever 🌸 You can also remove resources with an explicit release at any time ✨ But here's the tea: if an end-to-end (tear) RESERVE gets retransmitted (see Section 5.2.4 in [RFC5974]), that message MUST NOT initiate an intra-domain (tear) RESERVE message 🙅♀️ The bandwidth inside the RMD domain tied to that end-to-end (tear) RESERVE has already been released, so that same bandwidth MUST NOT once again be released. Don't double-free, periodt 💀
Now the PHR stuff 🎀 When the RMD-RMF of a QNE Edge or QNE Interior node processes a "PHR_Release_Request" PHR container, it MUST identify the <PHB Class> parameter and estimate how much time passed since the previous refresh, see also Section 3 of [CsTa05] ⏱️
This MAY be done by indicating the time lag, which we'll call "T_Lag", between the last sent "PHR_Refresh_Update" and the "PHR_Release_Request" control information container, sent by the QNE Ingress node. Check [RMD1] and [CsTa05] for more details 📚 The value of "T_Lag" gets normalized to the length of the refresh period, which we'll call "T_period". Then you calculate the ratio between "T_Lag" and "T_period", and that ratio goes into the <Time Lag> field of the "PHR_Release_Request" 🧮 When that "T_Lag" method is used and a node (QNE Egress or QNE Interior) gets the "PHR_Release_Request" PHR container, it MUST store the arrival time 🕰️ After that, it MUST calculate the time difference, "T_diff", between the arrival time and the start of the current refresh period, "T_period". The node MUST also derive "T_Lag" from the <Time Lag> parameter, by multiplying that value by the length of the refresh period, "T_period". If the derived "T_Lag" is smaller than the calculated "T_diff", then this node MUST decrease the PHB reservation state by the number of resource units listed in the <Peak Data Rate-1 (p)> field of the local RMD-QSPEC <TMOD-1> parameter of the RMD-QOSM <QoS Desired> that was sent together with the "PHR_Release_Request" "PHR Container", but never below zero 🫶
An RMD-specific release procedure can get kicked off by an end-to-end RESERVE with the <TEAR> flag set to ON (see Section 4.6.1.5.1) 🔥 It can also get triggered by an intra-domain RESPONSE, an end-to-end RESPONSE, or an end-to-end NOTIFY message that includes a marked one (meaning the PDR <M> and/or PDR <S> parameters are set to ON) "PDR_Reservation_Report" or "PDR_Congestion_Report", and/or an <INFO-SPEC> object 👑
4.6.1.5.1. Triggered by a RESERVE Message 💖
this explicit RMD release procedure can be triggered by a tear (<TEAR> flag set to ON) end-to-end RESERVE message 🎀 When that tear (<TEAR> flag set ON) end-to-end RESERVE message shows up at the QNE Ingress, the QNE Ingress node SHOULD process it the standard QoS-NSLP way (see [RFC5974]) 💅 On top of that, the RMD RMF gets notified, as specified in [RFC5974] 🔔
Just like the scenario in Section 4.6.1.1., the QNE Ingress node has to start a bypassing procedure 🦋 That bypassing procedure follows the description in Section 4.4. At the QNE Ingress, the end-to-end RESERVE message gets marked, which means the QoS-NSLP default NSLPID value gets swapped for another predefined NSLPID value. The GIST message carrying that end-to-end RESERVE will use this value to bypass the QNE Interior nodes 🧚
Before the QNE Ingress generates an intra-domain tear RESERVE, the RMD-QOSM has to release the requested RMD-QOSM bandwidth from the RMD traffic class state kept at that node 🌈 You do this by identifying the traffic class (PHB), then subtracting the amount of RMD traffic class requested resources (from the <Peak Data Rate-1 (p)> field of the local RMD-QSPEC <TMOD-1> parameter) from the total reserved amount stored in the RMD traffic class state. The <Time Lag> is used as explained in the introductory part of Section 4.6.1.5 ⏱️
QNE(Ingress) QNE(Interior) QNE(Interior) QNE(Egress)
NTLP stateful NTLP stateless NTLP stateless NTLP stateful
| | | |
RESERVE | | |
--->| | | RESERVE |
|------------------------------------------------------------>|
|RESERVE(RMD-QSPEC:Tear=1) | |
|------------------->| | |
| |RESERVE(RMD-QSPEC:Tear=1) |
| |------------------->| |
| | RESERVE(RMD-QSPEC:Tear=1)
| | |------------------->|
| | | RESERVE
| | | |-->
Figure 11: Explicit release triggered by RESERVE used by the
RMD-QOSMAfter that, once the REQUIRED bandwidth has been released from the RMD-QOSM traffic class state at the QNE Ingress, an intra-domain RESERVE (RMD-QOSM) message has to be generated 📨 The intra-domain RESERVE (RMD-QSPEC) message MUST include an <RMD QoS object combination> field and a PHR container (that's "PHR_Release_Request"), and it MAY include a PDR container (that's PDR_Release_Request) 🫧 You can see an example of this in Figure 11 ✨
Most of the non-default values of the objects in the tear intra-domain RESERVE message are set by the QNE Ingress node the same way as in Section 4.6.1.1. A few objects are set differently (see the QoS-NSLP-RMF API described in [RFC5974]) 👀
- The
<RII>object MUST NOT be included in this message. The QNE Ingress node doesn't need to get a response from the QNE Egress node, so no RII 🙅♀️ - If the release procedure is not applied for the RMD modification of aggregated reservation procedure (see Section 4.6.1.4), then the
<TEAR>flag MUST be set to ON 🔥 - The PHR resource units MUST be included in the
<Peak Data Rate-1 (p)>value of the local RMD-QSPEC<TMOD-1>parameter of the RMD-QOSM<QoS Desired>💜 - The value of the
<Admitted Hops>parameter MUST be set to"1"🍓 - The value of the
<Time Lag>parameter of the PHR container is calculated by the RMD-QOSM functionality (see Section 4.6.1.5), and the value of the<Control Type>parameter of the PHR container is set to"18"(that's"PHR_Release_Request") 🧾
Any QNE Interior node that receives the combination of the RMD-QOSM <QoS Desired> object and the "PHR_Release_Request" control information container MUST identify the traffic class (PHB) and release the requested resources included in the <Peak Data Rate-1 (p)> value of the local RMD-QSPEC <TMOD-1> parameter 🌸 It can do this by subtracting the amount of RMD traffic class requested resources (from the <Peak Data Rate-1 (p)> field of the local RMD-QSPEC <TMOD-1> parameter) from the total reserved amount stored in the RMD traffic class state. The <Time Lag> parameter of the "PHR_Release_Request" container gets used during the release procedure, as explained in the introductory part of Section 4.6.1.5 ⏳
The intra-domain tear RESERVE (RMD-QSPEC) message is received and processed by the QNE Egress node 🛬 The RMD-QOSM <QoS Desired> and the "PHR RMD-QOSM control" container (plus the "PDR Container", if it's available) get read and processed by the RMD QoS node 📖
The value of the <Peak Data Rate-1 (p)> field of the local RMD-QSPEC <TMOD-1> parameter of the RMD-QOSM <QoS Desired>, and the value of the <Time Lag> field of the PHR container, MUST be used by the RMD release procedure 🧠
This can be done by subtracting the amount of RMD traffic class requested resources (from the <Peak Data Rate-1 (p)> field value of the local RMD-QSPEC <TMOD-1> parameter) from the total reserved amount stored in the RMD traffic class state 🧮
The end-to-end RESERVE message gets forwarded by the next hop (the QNE Egress) only if the intra-domain tear RESERVE (RMD-QSPEC) message arrives at the QNE Egress node 🚦 Furthermore, the QNE Egress MUST stop the marking process that was used to bypass the QNE Interior nodes, by reassigning the QoS-NSLP default NSLPID value to the end-to-end RESERVE message (see Section 4.4) 🫶
Note that when the QNE Edges maintain aggregated QoS-NSLP reservation states, the RMD-QOSM functionality MAY start an RMD modification procedure (see Section 4.6.1.4) that uses the explicit release procedure described above in this subsection 💫 Also, if the complete bandwidth associated with the aggregated reservation maintained at the QNE Ingress has to be released, then the <TEAR> flag MUST be set to ON. Otherwise, the <TEAR> flag MUST be set to OFF (see Section 4.6.1.4) 🥹
4.6.1.5.2. Triggered by a Marked RESPONSE or NOTIFY Message ✨
okay bestie, so this RMD explicit release procedure can get kicked off in three different ways 🎀 First, an intra-domain RESPONSE message with a PDR container that has <M>=1 and <S>=0 (plus some other stuff in there, see Section 4.6.1.2) 🦋 Second, an intra-domain (refresh) RESPONSE with a PDR container that has <M>=1 and <S>=1 (see Section 4.6.1.6.1) 💜 Third, an end-to-end NOTIFY (see Section 4.6.1.6) that carries an <INFO-SPEC> object with these values:
Error severity class: Informational
Error code value: Congestion situationngl the aggregated case is a whole vibe 🙈 When the aggregated intra-domain QoS-NSLP operational states are used, an end-to-end NOTIFY message that triggers an RMD release procedure MAY contain a PDR container with <M>=1 and <S>=1, plus a bandwidth value in the <PDR Bandwidth> parameter, included in either a "PDR_Refresh_Report" or "PDR_Congestion_Report" container.
Now the big reminder 🚨 In every explicit release procedure, before the intra-domain tear RESERVE gets generated, the RMD-QOSM has to release the requested RMD-QOSM bandwidth from the RMD traffic class state kept at the QNE Ingress. You can do this by finding the traffic class (PHB), then subtracting the RMD traffic class requested resources (the ones in the <Peak Data Rate-1 (p)> field of the local RMD-QSPEC <TMOD-1> parameter) from the total reserved amount stored in that RMD traffic class state 💖
Figure 12 shows the case where the intra-domain tear RESERVE is generated after being triggered by either an intra-domain (refresh) RESPONSE carrying a PDR container with <M>=1 and <S>=1, or by an end-to-end NOTIFY that has no PDR container but does have an <INFO-SPEC> object. The error code values carried by this NOTIFY message are:
Error severity class: Informational
Error code value: Congestion situationMost of the non-default values in the tear intra-domain RESERVE(RMD-QSPEC) message get set by the QNE Ingress node, same as we covered in Section 4.6.1.1 🌸
The following objects MUST be used and/or set differently (see the QoS-NSLP-RMF described in [RFC5974]):
- the value of the
<M>parameter of the PHR container MUST be set to "1". - the value of the
<S>parameter of the "PHR container" MUST be set to "1". - the RESERVE message MAY include a PDR container. Note that this is needed if a bidirectional scenario is used; see Section 4.6.2.
Here's the Figure 12 diagram 🧚
QNE(Ingress) QNE(Interior) QNE(Interior) QNE(Egress)
NTLP stateful NTLP stateless NTLP stateless NTLP stateful
| | | |
| NOTIFY | | |
|<-------------------------------------------------------|
|RESERVE(RMD-QSPEC:Tear=1,M=1,S=1) | |
| ---------------->|RESERVE(RMD-QSPEC:Tear=1,M=1,S=1) |
| | | |
| |----------------->| |
| | RESERVE(RMD-QSPEC:Tear=1,M=1,S=1)
| | |----------------->| Figure 12: Basic operation during RMD-explicit release procedure
triggered by NOTIFY used by the RMD-QOSMHeads up, this one is important 👑 If the <M> and <S> values in the PHR container carried by an intra-domain tear RESERVE(RMD-QOSM) are set as ((<M>=0 and <S>=1) or (<M>=0 and <S>=0) or (<M>=1 and <S>=1)), then the <Max Admitted Hops> value SHOULD NOT be compared to the <Admitted Hops> value, and the <K> field MUST NOT be set. Any QNE Edge or QNE Interior node that receives the intra-domain tear RESERVE MUST check the <K> field in the PHR container. If the <K> field is "0", then the traffic class state (PHB) has to be identified using the <PHB Class> parameter, and the requested resources in the <Peak Data Rate-1 (p)> field of the local RMD-QSPEC <TMOD-1> parameter have to be released.
That release works by subtracting the RMD traffic class requested resources (again, the <Peak Data Rate-1 (p)> field of the local RMD-QSPEC <TMOD-1> parameter) from the total reserved amount stored in the RMD traffic class state. The <Time Lag> parameter of the PHR field gets used during this release, like we talked about in the intro of Section 4.6.1.5. After all that, the QNE Egress node MUST terminate the tear intra-domain RESERVE(RMD-QSPEC) message 🫶
Next up is the RMD-specific release procedure triggered by an intra-domain RESPONSE with an <M>=1 and <S>=0 PDR container (see Section 4.6.1.2). It generates an intra-domain tear RESERVE message that uses the <Max Admitted Hops> and <Admitted_Hops> fields together to work out and specify when the <K> value in the "PHR Container" can be set. When the <K> field is set, the "PHR Container" and the RMD-QOSM <QoS Desired> carried by an intra-domain tear RESERVE MUST NOT be processed 🙅♀️
The RMD-specific explicit release procedure that combines the <Max Admitted Hops>, <Admitted_Hops> and <K> fields to release resources/bandwidth in only a part of the RMD domain is called the RMD partial release procedure 🦋
This explicit release procedure can be used, for example, during an unsuccessful reservation (see Section 4.6.1.2). When the RMD-QOSM/QoS-NSLP signaling model functionality of a QNE Ingress node receives a PDR container with <M>=1 and <S>=0 of type "PDR_Reservation_Report", it MUST start an RMD partial release procedure.
In this situation, after the REQUIRED bandwidth is released from the RMD-QOSM traffic class state at the QNE Ingress, an intra-domain RESERVE (RMD-QOSM) message has to be generated. Figure 13 shows an example of this 💅
Most of the non-default values in the tear intra-domain RESERVE(RMD-QSPEC) message get set by the QNE Ingress node, same as in Section 4.6.1.1 🌈
The following objects MUST be used and/or set differently:
- the value of the
<M>parameter of the PHR container MUST be set to "1". - the RESERVE message MAY include a PDR container.
- the value of the
<Max Admitted Hops>carried by the "PHR Container" MUST be set equal to the<Max Admitted Hops>value carried by the "PDR Container" (with<M>=1and<S>=0) in the received intra-domain RESPONSE message that triggers the release procedure.
Any QNE Edge or QNE Interior node that receives the intra-domain tear RESERVE has to check the value of the <K> field in the "PHR Container" before it releases the requested resources.
If the value of the <K> field is "1", then all the QNEs located downstream, including the QNE Egress, MUST NOT process the carried "PHR Container" or the RMD-QOSM <QoS Desired> object in the intra-domain tearing RESERVE 🚫
QNE(Ingress) QNE(Interior) QNE(Interior) QNE(Egress)
Node that marked
PHR_Resource_Request
<PHR> object
NTLP stateful NTLP stateless NTLP stateless NTLP stateful
| | | |
| | | |
| RESPONSE (RMD-QSPEC: M=1) | |
|<------------------------------------------------------------|
RESERVE(RMD-QSPEC: Tear=1, M=1, <Admit Hops>=<Max Admitted Hops>, K=0)
|------------------->| | |
| |RESERVE(RMD-QSPEC: Tear=1, M=1, K=1) |
| |------------------>| |
| | RESERVE(RMD-QSPEC: Tear=1, M=1, K=1)|
| | |------------------->|
| | | | Figure 13: Basic operation during RMD explicit release procedure
triggered by RESPONSE used by the RMD-QOSMIf the <K> field value is "0", any QNE Edge or QNE Interior node that receives the intra-domain tear RESERVE can release the resources. It does this by subtracting the RMD traffic class requested resources (the <Peak Data Rate-1 (p)> field of the local RMD-QSPEC <TMOD-1> parameter) from the total reserved amount stored in the RMD traffic class state. The <Time Lag> parameter of the PHR field is used during the release, as explained in the intro of Section 4.6.1.5 ⭐
Furthermore, the QNE MUST perform the following procedures.
If the values of the <M> and <S> parameters in the "PHR_Release_Request" PHR container are (<M=1> and <S>=0), then the <Max Admitted Hops> value MUST be compared with the calculated <Admitted Hops> value. Note that each time the intra-domain tear RESERVE is processed, and before a QNE forwards it, the <Admitted Hops> value in the PHR container gets increased by one 🧮
When these two values are equal, the intra-domain RESERVE(RMD-QSPEC) that gets forwarded toward the QNE Egress MUST set the <K> value of the carried "PHR Container" to "1".
The reason for this is that the QNE node currently processing the message was the last QNE node that successfully processed the RMD-QOSM <QoS Desired> and PHR container of its associated initial reservation request (i.e., the initial intra-domain RESERVE(RMD-QSPEC) message). Its next QNE downstream node could not successfully process that initial reservation request, so this QNE node marked the <M> and <Hop_U> parameters of the "PHR_Resource_Request" 🥺
Finally, the QNE Egress node MUST terminate the intra-domain RESERVE(RMD-QSPEC) message 🎬
Also, the RMD partial release procedure described above applies when the QNE Edges maintain a per-flow QoS-NSLP reservation state 💜
When the QNE Edges maintain aggregated intra-domain QoS-NSLP operational states and a severe congestion occurs, the QNE Ingress MAY receive an end-to-end NOTIFY message (see Section 4.6.1.6) with a PDR container that carries the <M>=0 and <S>=1 fields, plus a bandwidth value in the <PDR Bandwidth> parameter, included in a "PDR_Congestion_Report" container. The same end-to-end NOTIFY message also carries an <INFO-SPEC> object with these values:
Error severity class: Informational
Error code value: Congestion situationThe end-to-end session tied to this NOTIFY message keeps the BOUND-SESSION-ID of the bound aggregated session; see Section 4.3.1. The RMD-QOSM at the QNE Ingress MUST start an RMD modification procedures (see Section 4.6.1.4) that uses the RMD explicit release procedure described above in this section. Specifically, the RMD explicit release procedure releases the bandwidth value in the <PDR Bandwidth> parameter, within the "PDR_Congestion_Report" container, from the reserved bandwidth associated with the aggregated intra-domain QoS-NSLP operational state 🌸
4.6.1.6. Severe Congestion Handling ✨
okay bestie, this section is about what the RMD-QOSM does when the Diffserv domain gets SEVERELY congested 😭🔥. When a link or router fails, the routing algorithms reroute traffic onto a new path 🛤️, and that new path might have nodes suddenly carrying way more traffic than they can handle. That MAY result in overloaded nodes, and it MAY cause severe congestion along the path 💀. In that case there aren't enough resources to give every flow on the new path the REQUIRED QoS.
So one or more flows SHOULD be terminated, or forwarded in a lower priority queue 🫶. Interior nodes let the Edge nodes know by marking the data or by marking the refresh messages 💅.
4.6.1.6.1. Severe Congestion Handling by the RMD-QOSM Refresh Procedure
4.6.1.6.1.
This procedure applies to every RMD scenario that uses an RMD refresh procedure 🌸. The QoS-NSLP and RMD can deal with congestion using the refresh procedure, so check out Section 4.6.1.3 for the details.
If the refresh doesn't succeed in a QNE Interior node, the Edge nodes get notified by setting <S>=1 (<M>=1) on the refresh messages, and by setting the <O> field in the "PHR_Refresh_Update" container, which rides along in the intra-domain RESERVE message 🎀.
Also, the overload situation can be detected using the example in Appendix A.1 🔍. When the signaled_overload_rate parameter from Appendix A.1 is higher than 0, the <Overload> field gets set to "1". The calculation lives in Appendix A.1 and is denoted as the signaled_overload_rate parameter. The flows can be terminated through the RMD release procedure in Section 4.6.1.5 💖.
The intra-domain RESPONSE that the QNE Egress sends back toward the QNE Ingress carries a PDR container with Parameter ID = 26, i.e., "PDR_Congestion_Report" 🦋. The values of the <M>, <S>, and <O> fields in this container SHOULD be set equal to the values of the <M>, <S>, and <O> fields that came in the "PHR_Refresh_Update" container. Part of the flows, the ones matching <O>, get terminated or forwarded in a lower priority queue 🍓.
The flows can be terminated through the RMD release procedure in Section 4.6.1.5 again, because it's the go-to move 💅.
One more thing: all of the above also applies when the QNE Edge nodes keep either per-flow QoS-NSLP reservation states or aggregated QoS-NSLP reservation states 🧚.
Generally speaking, the soft state refresh mechanism solves congestion within one refresh period ⏱️. If that isn't fast enough, additional functions SHOULD be used, and those are described in Section 4.6.1.6.2 🔥.
4.6.1.6.2. Severe Congestion Handling by Proportional Data Packet Marking
4.6.1.6.2.
This severe congestion handling method needs the following functionalities 🎀.
4.6.1.6.2.1. Operation in the Interior Nodes
The detection and marking/re-marking stuff in this section applies to both NSIS-aware and NSIS-unaware nodes 🫧. Here's the catch, though: the "not NSIS-aware" nodes MUST be configured so they can detect congestion and severe congestion situations and re-mark packets the same way the "NSIS-aware" nodes do 🙈.
When an Interior node detects severe congestion, it re-marks the data packets passing through it 🔥. Two extra DSCPs can be allocated for each traffic class for this re-marking 🌈. One DSCP MAY be used to show that the packet went through a congested node. In this document that's called an "affected DSCP", and it tells you that a packet passed through a severely congested node 💜.
Why does the affected DSCP matter? Because it removes the chance that, say, flow-based ECMP-enabled (Equal Cost Multiple Paths) routing makes the Egress node miss packets that went through a severely congested node, or wrongly think a packet went through one when it didn't 😭. Note that this type of DSCP MUST only be used if all the nodes within the RMD domain are configured to use it. Otherwise, this type of DSCP MUST NOT be applied 🚫. The other DSCP MUST be used to show the degree of congestion by marking bytes in proportion to how congested things are 📊. In this document that one is called "encoded DSCP" ✨.
Now, when this document says "marked packets" or "marked bytes", it means the "encoded DSCP" 💖. "Unmarked packets" or "unmarked bytes" are the packets, or the bytes in those packets, whose DSCP is either the "affected DSCP" or the original DSCP 🫶. In the algorithm below, the router MAY drop received packets 💀. Counting and measuring marked or unmarked bytes happens within measurement periods. All nodes in an RMD domain use the same fixed measurement interval, say T seconds, and that interval MUST be preconfigured ⏱️.
It is RECOMMENDED that the total number of additional (local and experimental) DSCPs used for severe congestion handling in an RMD domain SHOULD be as low as possible, and it SHOULD NOT exceed the limit of 8 🍓. One way to cut down on DSCPs is to only use the "encoded DSCP" and skip "affected DSCP" marking 🌸. Another option is to allocate one DSCP for the severe congestion indication for each of the AF classes that RMD-QOSM can support 🧚.
An example of a re-marking procedure is in Appendix A.1 📚.
4.6.1.6.2.2. Operation in the Egress Nodes
When the QNE Edges maintain a per-flow intra-domain QoS-NSLP operational state (see Sections 4.3.2 and 4.3.3), the following procedure applies 👑. The QNE Egress node uses a predefined policy to fix the severe congestion situation, and it does that by picking a number of inter-domain (end-to-end) flows that SHOULD be terminated or forwarded in a lower priority queue 💅.
When the RMD domain does not use "affected DSCP" marking, the Egress MUST generate an Ingress/Egress pair aggregated state for each Ingress and for each supported PHB 🫶. That's because the Edges MUST be able to detect which Ingress/Egress pair a severe congestion is happening in 🔍. Otherwise, the QNE Egress has no info about which flows or groups of flows got hit 😭.
When the RMD domain supports "affected DSCP" marking, the Egress can detect every flow affected by the severe congestion 🦋. So in that case, the Egress MAY not generate and maintain the Ingress/Egress pair aggregated reservation states 🌈. Note that those aggregated reservation states MAY not be linked to aggregated intra-domain QoS-NSLP operational states 🙈.
The Ingress/Egress pair aggregated reservation state can be worked out by spotting which flows use the same PHB and come from the same Ingress (via the per-flow end-to-end QoS-NSLP states) 🔎.
Some flows in the same PHB traffic class might get a different priority than others in that class 💫. The RESERVE message carrying the end-to-end QoS Model QSPEC can tell the Egress and Ingress nodes about this, through the <Preemption Priority> and <Defending Priority> parameters, or through a locally defined policy 📣. The priority value lives in the reservation states (see Section 4.3), and it can be used during admission control and/or severe congestion handling procedures 👑. The flows that get terminated are picked from the flows in the same PHB traffic class as the marked ("encoded DSCP") and "affected DSCP" packets (when that's used across the whole RMD domain). When the Ingress/Egress pair aggregated states are available, they also come from the same Ingress/Egress pair aggregate 🎀.
For flows in the same PHB traffic class, flow priority plays a big role 💜. Appendix A.2 has an example of how many flows in each priority class have to be terminated 🧮.
For the flows (sessions) that have to be terminated, the QNE Egress node generates and sends an end-to-end NOTIFY message to the QNE Ingress node (its upstream stateful QoS-NSLP peer) to say there's severe congestion on the path 📬😭.
The non-default values of the objects in the NOTIFY message MUST be set by the QNE Egress node as follows (see QoS-NSLP-RMF API described in [RFC5974]):
- the values of the <INFO-SPEC> object are set by the standard QoS-NSLP protocol functions.
- the <INFO-SPEC> object MUST include information saying that the end-to-end flow MUST be terminated. That information is:
Error severity class: Informational
Error code value: Congestion situationWhen the QNE Edges maintain a per-aggregate intra-domain QoS-NSLP operational state (see Section 4.3.1), the QNE Edge has to work out, for each aggregate intra-domain QoS-NSLP operational state, the total bandwidth that has to be terminated to solve the severe congestion 🧠. This total is calculated the same way as when the QNE Edges maintain per-flow intra-domain QoS-NSLP operational states. For the aggregated sessions that get affected, the QNE Egress node generates and sends one end-to-end NOTIFY message to the QNE Ingress node (its upstream stateful QoS-NSLP peer) to indicate the severe congestion in the communication path 💌. That one end-to-end NOTIFY message is tied to one of the end-to-end sessions bound to the aggregated intra-domain QoS-NSLP operational state 🫶.
The non-default values of the objects in that NOTIFY message MUST be set by the QNE Egress node the same way as the end-to-end NOTIFY described above, for the case where the QNE Egress keeps a per-flow intra-domain operational state 🎀. On top of that, the end-to-end NOTIFY MUST carry the RMD-QSPEC, which holds a PDR container with Parameter ID = 26, i.e., "PDR_Congestion_Report" 🦋. The value of the <S> SHOULD be set. Also, the <PDR Bandwidth> parameter MUST contain the bandwidth tied to the aggregated QoS-NSLP operational state that has to be released 💅.
The number of end-to-end sessions that have to be terminated gets calculated the same way as when the QNE Edges maintain per-flow intra-domain QoS-NSLP operational states 🧮. For each ongoing flow that has to be terminated, the Egress notifies the Ingress the same way as in that per-flow situation 📣.
Note that the QNE Egress SHOULD restore the original <DSCP> values of the re-marked packets 🍓. Otherwise, multiple actions for the same event might happen 😭. However, this value MAY be left in its re-marking form if there's an SLA agreement between domains saying that a downstream domain handles the re-marking problem 🤝.
A detailed example of severe congestion operation in the Egress Nodes is in Appendix A.2 📚🌈.
4.6.1.6.2.3. Operation in the Ingress Nodes ✨
okay so when the end-to-end NOTIFY lands at the QNE Ingress node, the severe congestion gets handled by a predefined policy 💅 🔥. For example, the node can refuse new incoming flows (sessions), terminate the affected and notified flows (sessions), and block their packets or shift them to an alternative RMD traffic class (PHB) 🫶.
Figure 14 shows the whole flow 🦋. The QNE Ingress gets a NOTIFY message for each flow (session) that has to be terminated, and that NOTIFY carries the Congestion situation error code 😭.
When the QNE Ingress node receives the end-to-end NOTIFY message, it matches it to its bound intra-domain session (see Sections 4.3.2 and 4.3.3) using the BOUND-SESSION-ID info included in the end-to-end per-flow QoS-NSLP state 🔗. Then it uses the operation from Section 4.6.1.5.2 to terminate the intra-domain session 💖.
QNE(Ingress) QNE(Interior) QNE(Interior) QNE(Egress)
user | | | |
data | user data | | |
------>|----------------->| user data | user data |
| |---------------->S(# marked bytes) |
| | S----------------->|
| | S(# unmarked bytes)|
| | S----------------->|Term.
| NOTIFY S |flow?
|<-----------------|-----------------S------------------|YES
|RESERVE(RMD-QSPEC:Tear=1,M=1,S=1) S |
| ---------------->|RESERVE(RMD-QSPEC:T=1,M=1,S=1) |
| | S |
| |---------------->S |
| | RESERVE(RMD-QSPEC:Tear=1,M=1,S=1)
| | S----------------->|
Figure 14: RMD severe congestion handlingNote that this functionality applies to the RMD reservation-based (see Section 4.3.3) and to both measurement-based admission control methods (i.e., congestion notification based on probing and the NSIS measurement-based admission control; see Section 4.3.2) 🌈.
In the case that the QNE Edges support aggregated intra-domain QoS-NSLP operational states, here's what happens 🧚. The QNE Ingress MAY receive an end-to-end NOTIFY message with a PDR container that carries an <S> marked and a bandwidth value in the <PDR Bandwidth> parameter, included in a PDR_Congestion_Report container. The same end-to-end NOTIFY message also carries an <INFO-SPEC> object with the Congestion situation error code.
When the QNE Ingress node receives this end-to-end NOTIFY message, it associates it with the aggregated intra-domain QoS-NSLP operational state via the BOUND-SESSION-ID info included in the end-to-end per-flow QoS-NSLP operational state (see Section 4.3.1) 🎀.
The RMD-QOSM at the QNE Ingress node, using the total bandwidth value to be released included in the <PDR Bandwidth> parameter, MUST reduce the bandwidth associated and reserved by the RMD aggregated session. This gets done by triggering the RMD modification for aggregated reservations procedure described in Section 4.6.1.4 ✨.
In addition to the above, the QNE Ingress MUST select a number of inter-domain (end-to-end) flows (sessions) that MUST be terminated 😭. This is accomplished the same way as in the situation where the QNE Edges maintain per-flow intra-domain QoS-NSLP operational states.
The terminated end-to-end sessions are selected from the end-to-end sessions bound to the aggregated intra-domain QoS-NSLP operational state 🍓. Note that the end-to-end session associated with the received end-to-end NOTIFY message that notified the severe congestion MUST also be selected for termination 💅.
For the flows (sessions) that have to be terminated, the QNE Ingress node generates and sends an end-to-end NOTIFY message upstream towards the sender (QNI). The values carried by this message are:
- the values of the
<INFO-SPEC>object set by the standard QoS-NSLP protocol functions. - the
<INFO-SPEC>object MUST include information that notifies that the end-to-end flow MUST be terminated. This information is as follows:
Error severity class: Informational
Error code value: Congestion situation4.6.1.7. Admission Control Using Congestion Notification Based on Probing 🦋
The congestion notification function based on probing can be used to implement a simple measurement-based admission control within a Diffserv domain 💜. At Interior nodes along the data path, congestion notification thresholds are set in the measurement-based admission control function for the traffic belonging to different PHBs. These Interior nodes are not NSIS-aware nodes.
4.6.1.7.1. Operation in Ingress Nodes
When an end-to-end reservation request (RESERVE) arrives at the Ingress node (QNE), see Figure 15, it gets processed based on the procedures defined by the end-to-end QoS Model 🫶.
The <DSCP> field of the GIST datagram message that transports this probe RESERVE message SHOULD be marked with the same DSCP value as the data path packets associated with the same session. This way, it is ensured that the end-to-end RESERVE (probe) packet passed through the node that is congested 🔥. This feature is very useful when ECMP-based routing is used to detect only flows that are passing through the congested router 👑.
When an end-to-end RESPONSE message is received by the Ingress node, it will be processed based on the procedures defined by the end-to-end QoS Model ✨.
4.6.1.7.2. Operation in Interior nodes
These Interior nodes do not need to be NSIS-aware nodes, and they do not need to process the NSIS functionality of NSIS messages 🌸. Note that the "not NSIS-aware" nodes MUST be configured such that they can detect the congestion/severe congestion situations and re-mark packets in the same way the "NSIS-aware" nodes do 💀.
Using standard functionalities, congestion notification thresholds are set for the traffic that belongs to different PHBs (see Section 4.3.2). The end-to-end RESERVE message, see Figure 15, is used as a probe packet 🧚.
The <DSCP> field of all data packets and of the GIST message carrying the RESERVE message will be re-marked when the corresponding "congestion notification" threshold is exceeded (see Section 4.3.2) 🙈. Note that when the data rate is higher than the congestion notification threshold, the data packets are also re-marked. An example of the detailed operation of this procedure is given in Appendix A.2 💖.
4.6.1.7.3. Operation in Egress Nodes
As emphasized in Section 4.6.1.6.2.2, the Egress node, by using the per-flow end-to-end QoS-NSLP states, can derive which flows are using the same PHB and are sent by the same Ingress 🦋.
For each Ingress, the Egress SHOULD generate an Ingress/Egress pair aggregated (RMF) reservation state for each supported PHB 🌈. Note that this aggregated reservation state does not require that an aggregated intra-domain QoS-NSLP operational state is needed also.
Appendix A.4 contains an example of how and when a (probe) RESERVE message that arrives at the Egress is admitted or rejected 🎀.
If the request is rejected, then the Egress node SHOULD generate an (end-to-end) RESPONSE message to notify that the reservation is unsuccessful 😭. In particular, it will generate an <INFO-SPEC> object of:
Error severity class: Transient Failure
Error code value: Reservation failureThe QSPEC that was carried by the end-to-end RESERVE that belongs to the same session as this end-to-end RESPONSE is included in this message. The parameters included in the QSPEC <QoS Reserved> object are copied from the original <QoS Desired> values. The <E> flag associated with the <QoS Reserved> object and the <E> flag associated with the local RMD-QSPEC <TMOD-1> parameter are also set. This RESPONSE message will be sent to the Ingress node, and it will be processed based on the end-to-end QoS Model 💅.
Note that the QNE Egress SHOULD restore the original <DSCP> values of the re-marked packets; otherwise, multiple actions for the same event might occur 🙈. However, this value MAY be left in its re-marking form if there is an SLA agreement between domains that a downstream domain handles the re-marking problem. Note that the break <B> flag carried by the end-to-end RESERVE message MUST NOT be set 🚫.
QNE(Ingress) Interior Interior QNE(Egress)
(not NSIS aware) (not NSIS aware)
user | | | |
data | user data | | |
------>|----------------->| user data | |
| |---------------->| user data |
| | |----------------->|
user | | | |
data | user data | | |
------>|----------------->| user data | user data |
| |---------------->S(# marked bytes) |
| | S----------------->|
| | S(# unmarked bytes)|
| | S----------------->|
| | S |
RESERVE | | S |
------->| | S |
|----------------------------------->S |
| | RESERVE(re-marked DSCP in GIST)
| | S----------------->|
| |RESPONSE(unsuccessful INFO-SPEC) |
|<------------------------------------------------------|
RESPONSE(unsuccessful INFO-SPEC) | |
<------| | | |
Figure 15: Using RMD congestion notification function for
admission control based on probing4.6.2. Bidirectional Operation ✨
okay bestie, this section is all about how the RMD-QOSM does bidirectional operation 💖 It walks through the basic cases and the sequence of events/triggers, and these are the ones it covers:
- Successful and unsuccessful reservation (Section 4.6.2.1);
- Refresh reservation (Section 4.6.2.2);
- Modification of aggregated reservation (Section 4.6.2.3);
- Release procedure (Section 4.6.2.4);
- Severe congestion handling (Section 4.6.2.5);
- Admission control using congestion notification based on probing (Section 4.6.2.6).
It is super important to know that this section is what gets used to spec the following RMD-QOSM/QoS-NSLP signaling schemes when basic unidirectional operation is assumed 🔥
- "per-flow congestion notification based on probing";
- "per-flow RMD NSIS measurement-based admission control",
- "per-flow RMD reservation-based" in combination with the "severe congestion handling by the RMD-QOSM refresh" procedure;
- "per-flow RMD reservation-based" in combination with the "severe congestion handling by proportional data packet marking" procedure;
- "per-aggregate RMD reservation-based" in combination with the "severe congestion handling by the RMD-QOSM refresh" procedure;
- "per-aggregate RMD reservation-based" in combination with the "severe congestion handling by proportional data packet marking" procedure.
For more details, please see Section 3.2.3 🦋
In particular, the functionality in Sections 4.6.2.1, 4.6.2.2, 4.6.2.3, 4.6.2.4, and 4.6.2.5 applies to the RMD reservation-based and NSIS measurement-based admission control methods. The functionality in Section 4.6.2.6 applies to the admission control procedure that uses the congestion notification based on probing 🎀 The QNE Edge nodes keep track of either per-flow QoS-NSLP operational and reservation states or aggregated QoS-NSLP operational and reservation states.
RMD-QOSM assumes that asymmetric routing MAY be applied in the RMD domain. Combined sender-receiver initiated reservation can't be done efficiently in the RMD domain, because upstream NTLP states aren't stored in Interior routers 🙈
So the bidirectional operation SHOULD be performed by two sender-initiated reservations (sender&sender). We assume the QNE Edge nodes are common for both upstream and downstream directions, so the two reservations/sessions can be bound at the QNE Edge nodes 🫶 If that's not the case, the bidirectional procedure could be managed and maintained by nodes outside the RMD domain, using procedures other than the ones defined in RMD-QOSM.
This (intra-domain) bidirectional sender&sender procedure can then be applied between the QNE Edge (QNE Ingress and QNE Egress) nodes of the RMD QoS signaling model. Now, if a security association exists between the QNE Ingress and QNE Egress nodes (see Figure 15), and the QNE Ingress node has the REQUIRED <Peak Data Rate-1 (p)> values of the local RMD-QSPEC <TMOD-1> parameters for both directions (QNE Ingress towards QNE Egress, and QNE Egress towards QNE Ingress), then the QNE Ingress MAY include both <Peak Data Rate-1 (p)> values of the local RMD-QSPEC <TMOD-1> parameters (needed for both directions) into the RMD-QSPEC within a RESERVE message. This way, the QNE Egress node can use the QoS parameters it needs for the "Egress towards Ingress" direction (QoS-2) ✨ The QNE Egress is then able to create a RESERVE with the right QoS parameters in the QSPEC, i.e., RESERVE (QoS-2). Both directions of the flows get bound by inserting <BOUND-SESSION-ID> objects at the QNE Ingress and QNE Egress, and those ride along in the bound end-to-end RESERVE messages 💜
|------ RESERVE (QoS-1, QoS-2)----|
| V
| Interior/stateless QNEs
+---+ +---+
|------->|QNE|-----|QNE|------
| +---+ +---+ |
| V
+---+ +---+
|QNE| |QNE|
+---+ +---+
^ |
| | +---+ +---+ V
| |-------|QNE|-----|QNE|-----|
| +---+ +---+
Ingress/ Egress/
stateful QNE stateful QNE
|
<--------- RESERVE (QoS-2) -------|Figure 16: The intra-domain bidirectional reservation scenario in the RMD domain
Note that it is RECOMMENDED that the QNE implementations of RMD-QOSM process the QoS-NSLP signaling messages with a higher priority than data packets 🌈 This can be done as described in Section 3.3.4 in [RFC5974] and the QoS-NSLP-RMF API [RFC5974].
A bidirectional reservation within the RMD domain is indicated by the PHR <B> and PDR <B> flags, which are set in all messages. In this case, two <BOUND-SESSION-ID> objects SHOULD be used. 🦋
When the QNE Edges maintain per-flow intra-domain QoS-NSLP operational states, the end-to-end RESERVE message carries two BOUND-SESSION-IDs. One BOUND-SESSION-ID carries the SESSION-ID of the tunneled intra-domain (per-flow) session that uses a Binding_Code with the value set to code (Tunneled and end-to-end sessions). The other BOUND-SESSION-ID carries the SESSION-ID of the bound bidirectional end-to-end session, and the Binding_Code for that one is set to code (Bidirectional sessions) 🌸
When the QNE Edges maintain aggregated intra-domain QoS-NSLP operational states, the end-to-end RESERVE message carries two BOUND-SESSION-IDs as well. One carries the SESSION-ID of the tunneled aggregated intra-domain session that uses a Binding_Code with the value set to code (Aggregated sessions). The other carries the SESSION-ID of the bound bidirectional end-to-end session, with its Binding_Code set to code (Bidirectional sessions) 🧚
The intra-domain and end-to-end QoS-NSLP operational states get initiated/modified depending on the binding type (see Sections 4.3.1, 4.3.2, and 4.3.3) 💅
If no security association exists between the QNE Ingress and QNE Egress nodes, the bidirectional reservation for the sender&sender scenario in the RMD domain SHOULD use the scenario specified in [RFC5974] as "bidirectional reservation for sender&sender scenario". The reason is that in this scenario, the RESERVE message sent from the QNE Ingress to the QNE Egress doesn't have to carry the QoS parameters needed for the "Egress towards Ingress" direction (QoS-2) 🍓
In the following sections, it's considered that the QNE Edge nodes are common for both upstream and downstream directions, so the two reservations/sessions can be bound at the QNE Edge nodes. It's also considered that a security association exists between the QNE Ingress and QNE Egress nodes, and that the QNE Ingress node has the REQUIRED <Peak Data Rate-1 (p)> value of the local RMD-QSPEC <TMOD-1> parameters for both directions, i.e., QNE Ingress towards QNE Egress and QNE Egress towards QNE Ingress 👑
According to Section 3.2.3, only the "per-flow RMD reservation-based" in combination with the "severe congestion handling by proportional data packet marking" scheme MUST be implemented within one RMD domain. However, all RMD QNEs supporting this specification MUST support the combination of the "per-flow RMD reservation-based" and "severe congestion handling by proportional data packet marking" scheme. If the RMD QNEs support more RMD-QOSM schemes, then the operator of that RMD domain MUST preconfigure all the QNE Edge nodes within one domain so that the <SCH> field included in the "PHR Container" (Section 4.1.2) and the "PDR Container" (Section 4.1.3) will always use the same value. That way, within one RMD domain, only one of the RMD-QOSM schemes described below is used at a time 🔥
All QNE nodes located within the RMD domain MUST read and interpret the <SCH> field included in the "PHR Container" before processing all the other <PHR Container> payload fields. Moreover, all QNE Edge nodes located at the border of the RMD domain MUST read and interpret the <SCH> field included in the "PDR Container" before processing all the other <PDR Container> payload fields 💖
4.6.2.1. Successful and Unsuccessful Reservations 💅
This section describes the operation of the RMD-QOSM where an RMD intra-domain bidirectional reservation operation (see Figure 16 and Section 4.6.2) is either successfully or unsuccessfully accomplished 🌸
The bidirectional successful reservation is similar to a combination of two unidirectional successful reservations that are accomplished in opposite directions (see Figure 17). The main differences between the bidirectional successful reservation procedure and that combination are as follows. Also, the intra-domain and end-to-end QoS-NSLP operational states generated and maintained by the end-to-end RESERVE messages contain a different BOUND-SESSION-ID data structure compared to the unidirectional reservation scenario (see Sections 4.3.1, 4.3.2, and 4.3.3). In this scenario, the intra-domain RESERVE message sent by the QNE Ingress node towards the QNE Egress node is denoted in Figure 17 as RESERVE (RMD-QSPEC): "forward". The main differences between the intra-domain RESERVE (RMD-QSPEC): "forward" message used for the bidirectional successful reservation procedure and a RESERVE (RMD-QSPEC) message used for the unidirectional successful reservation are as follows (see the QoS-NSLP-RMF API described in [RFC5974]):
- the
<RII>object MUST NOT be included in the message. This is because no RESPONSE message is REQUIRED. - the
<B>bit of the PHR container indicates a bidirectional reservation and MUST be set to"1". - the PDR container is also included in the RESERVE(RMD-QSPEC): "forward" message. The value of the Parameter ID is
"20", i.e.,PDR_Reservation_Request. Note that the response PDR container sent by a QNE Egress to a QNE Ingress node is not carried by an end-to-end RESPONSE message, but by an intra-domain RESERVE message that the QNE Egress node sends towards the QNE Ingress node (denoted in Figure 16 as RESERVE(RMD-QSPEC): "reverse"). - the
<B>PDR bit indicates a bidirectional reservation and is set to"1". - the
<PDR Bandwidth>field specifies the requested bandwidth that has to be used by the QNE Egress node to initiate another intra-domain RESERVE message in the reverse direction.
The RESERVE(RMD-QSPEC): "reverse" message is initiated by the QNE Egress node at the moment that the RESERVE(RMD-QSPEC): "forward" message is successfully processed by the QNE Egress node 🦋
The main differences between the RESERVE(RMD-QSPEC): "reverse" message used for the bidirectional successful reservation procedure and a RESERVE(RMD-QSPEC) message used for the unidirectional successful reservation are as follows:
QNE(Ingress) QNE (int.) QNE (int.) QNE (int.) QNE(Egress)Okay so this next one is the bidirectional success story, and it is SO clean 💖 Figure 17 shows the reservation zipping through every node going forward, then coming back around in reverse 🦋
NTLP stateful NTLP st.less NTLP st.less NTLP st.less NTLP stateful
| | | | |
| | | | |
|RESERVE(RMD-QSPEC) | | |
|"forward" | | | |
| | RESERVE(RMD-QSPEC): | |
|--------------->| "forward" | | |
| |------------------------------>| |
| | | |------------->|
| | | | |
| | |RESERVE(RMD-QSPEC) |
| RESERVE(RMD-QSPEC) | "reverse" |<-------------|
| "reverse" | |<--------------| |
|<-------------------------------| | |
Figure 17: Intra-domain signaling operation for successful
bidirectional reservationNow the differences from the normal one, bestie 👑 First off, the <RII> object just isn't in the message 🙈 That's because no RESPONSE message is REQUIRED, so there's nothing to ask for.
- the
<RII>object is not included in the message. This is because no RESPONSE message is REQUIRED; - the value of the
<Peak Data Rate-1 (p)>field of the local RMD-QSPEC<TMOD-1>parameter is set equal to the value of the<PDR Bandwidth>field included in the RESERVE(RMD-QSPEC): "forward" message that triggered the generation of this RESERVE(RMD-QSPEC): "reverse" message; - the
<B>bit of the PHR container indicates a bidirectional reservation and is set to "1"; - the PDR container is included into the RESERVE(RMD-QSPEC): "reverse" message. The value of the Parameter ID is "23", i.e., "PDR_Reservation_Report";
- the
<B>PDR bit indicates a bidirectional reservation and is set to "1".
Next up is the unsuccessful version, which comes in two flavours 🌈 Figures 18 and 19 show the flow diagrams. In Figure 18, the QNE that can't support the requested <Peak Data Rate-1 (p)> value of local RMD-QSPEC <TMOD-1> sits in the direction from QNE Ingress towards QNE Egress. In Figure 19, that same QNE sits in the direction from QNE Egress towards QNE Ingress. The main differences from the successful procedure in Figure 18 are:
- the QNE node that is not able to reserve resources for a certain request is located in the "forward" path, i.e., the path from the QNE Ingress towards the QNE Egress.
- the QNE node that is not able to support the requested
<Peak Data Rate-1 (p)>value of local RMD-QSPEC<TMOD-1>MUST mark the<M>bit, i.e., set to value "1", of the RESERVE(RMD-QSPEC): "forward".
QNE(Ingress) QNE (int.) QNE (int.) QNE (int.) QNE(Egress)
NTLP stateful NTLP st.less NTLP st.less NTLP st.less NTLP stateful
| | | | |
|RESERVE(RMD-QSPEC): | | |
| "forward" | RESERVE(RMD-QSPEC): | |
|--------------->| "forward" | M RESERVE(RMD-QSPEC):
| |--------------------------->M "forward-M marked"
| | | M-------------->|
| | RESPONSE(PDR) M |
| | "forward - M marked"M |
|<------------------------------------------------------------|
|RESERVE(RMD-QSPEC, K=0) | M |
|"forward - T tear" | M |
|--------------->| | M |
| RESERVE(RMD-QSPEC, K=1) M |
| | "forward - T tear" M |
| |--------------------------->M |
| | RESERVE(RMD-QSPEC, K=1) |
| | "forward - T tear" |
| | M-------------->|
Figure 18: Intra-domain signaling operation for unsuccessful
bidirectional reservation (rejection on path
QNE(Ingress) towards QNE(Egress))Figure 18 works pretty much like the unsuccessful unidirectional one in Figure 9, so if you've got that down you're basically set 🫶
Now Figure 19, where the rejection happens on the way back 😭 The differences from the successful procedure are:
- the QNE node that is not able to reserve resources for a certain request is located in the "reverse" path, i.e., the path from the QNE Egress towards the QNE Ingress.
- the QNE node that is not able to support the requested
<Peak Data Rate-1 (p)>value of local RMD-QSPEC<TMOD-1>MUST mark the<M>bit, i.e., set to value "1", the RESERVE(RMD-QSPEC): "reverse".
QNE(Ingress) QNE (int.) QNE (int.) QNE (int.) QNE(Egress)
NTLP stateful NTLP st.less NTLP st.less NTLP st.less NTLP stateful
| | | | |
|RESERVE(RMD-QSPEC) | | |
|"forward" | RESERVE(RMD-QSPEC): | |
|--------------->| "forward" | RESERVE(RMD-QSPEC): |
| |-------------------------------->|"forward" |
| | RESERVE(RMD-QSPEC): |------------->|
| | "reverse" | | |
| | RESERVE(RMD-QSPEC) | |
| RESERVE(RMD-QSPEC): M "reverse" |<-------------|
| "reverse - M marked" M<---------------| |
|<--------------------------------M | |
| | M | |
|RESERVE(RMD-QSPEC, K=0): M | |
|"forward - T tear" M | |
|--------------->| RESERVE(RMD-QSPEC, K=0): | |
| | "forward - T tear" | |
| |-------------------------------->| |
| | M |------------->|
| | M RESERVE(RMD-QSPEC, K=0):
| | M "reverse - T tear" |
| | M |<-------------|
| M RESERVE(RMD-QSPEC, K=1) |
| | M "forward - T tear" |
| | M<---------------| |
| RESERVE(RMD-QSPEC, K=1)M | |
| "forward - T tear" M | |
|<--------------------------------M | |
Figure 19: Intra-domain signaling normal operation for unsuccessful
bidirectional reservation (rejection on path QNE(Egress)
towards QNE(Ingress)- the QNE Ingress uses the information contained in the received PHR and PDR containers of the RESERVE(RMD-QSPEC): "reverse" and generates a tear intra-domain RESERVE(RMD-QSPEC): "forward - T tear" message. This message carries a "PHR_Release_Request" and "PDR_Release_Request" control information. This message is sent to the QNE Egress node. The QNE Egress node uses the information contained in the "PHR_Release_Request" and the "PDR_Release_Request" control info containers to generate a RESERVE(RMD-QSPEC): "reverse - T tear" message that is sent towards the QNE Ingress node.
4.6.2.2. Refresh Reservations ✨
This section is all about the RMD-QOSM doing an RMD intra-domain bidirectional refresh reservation 🔁
The refresh procedure for an RMD reservation-based method follows a scheme that's pretty similar to the successful reservation procedure from Section 4.6.2.1 (and Figure 17). It's also similar to how unidirectional intra-domain reservations get refreshed (see Section 4.6.1.3) 💅
Note that the RMD traffic class refresh periods used by the bound bidirectional sessions MUST be equal in all QNE Edge and QNE Interior nodes. Not optional, bestie 🙅♀️
The differences from the bidirectional successful RESERVE(RMD-QSPEC): "forward" message are:
- the value of the Parameter ID of the PHR container is "19", i.e., "PHR_Refresh_Update".
- the value of the Parameter ID of the PDR container is "21", i.e., "PDR_Refresh_Request".
The differences from the bidirectional successful RESERVE(RMD-QSPEC): "reverse" message are:
- the value of the Parameter ID of the PHR container is "19", i.e., "PHR_Refresh_Update".
- the value of the Parameter ID of the PDR container is "24", i.e., "PDR_Refresh_Report".
4.6.2.3. Modification of Aggregated Intra-Domain QoS-NSLP Operational Reservation States ✨
This section covers the RMD-QOSM when RMD intra-domain bidirectional QoS-NSLP aggregated reservation states need to be modified 🌸
If the QNE Edges keep aggregated reservation states for the RMD QoS Model, and one of those aggregated reservations needs changing (see Section 4.3.1), then procedures similar to Section 4.6.1.4 apply. Here's the vibe:
- When the modification request needs an increase of the reserved resources, the QNE Ingress node MUST include the corresponding value into the
<Peak Data Rate-1 (p)>field local RMD-QSPEC<TMOD-1>parameter of the RMD-QOSM<QoS Desired>), which is sent together with "PHR_Resource_Request" control information. If a QNE Edge or QNE Interior node is not able to reserve the number of requested resources, then the "PHR_Resource_Request" associated with the local RMD-QSPEC<TMOD-1>parameter MUST be marked. In this situation, the RMD-specific operation for unsuccessful reservation is applied (see Section 4.6.2.1). Note that the value of the<PDR Bandwidth>parameter, which is sent within a "PDR_Reservation_Request" container, represents the increase of the reserved resources in the "reverse" direction. 🚀 - When the modification request needs a decrease of the reserved resources, the QNE Ingress node MUST include this value into the
<Peak Data Rate-1 (p)>field of the local RMD-QSPEC<TMOD-1>parameter of the RMD-QOSM<QoS Desired>). After that, an RMD release procedure SHOULD be accomplished (see Section 4.6.2.4). Note that the value of the<PDR Bandwidth>parameter, which is sent within a "PDR_Release_Request" container, represents the decrease of the reserved resources in the "reverse" direction. 💜
4.6.2.4. Release Procedure ✨
okay so this section explains how an RMD intra-domain bidirectional reservation gets released 🎀 The message sequence diagram looks a lot like the successful reservation one from Section 4.6.2.1 and Figure 17 🦋 but the release itself works the same way as the RMD release for intra-domain unidirectional reservations (see Section 4.6.1.5 and Figures 18 and 19). so you're not learning a whole new thing, just a twist on the usual vibe 💅
the differences between the RESERVE (RMD-QSPEC): "forward" message used for the bidirectional release and the one used for a successful bidirectional reservation are:
- the Parameter ID of the PHR container is "18", i.e."PHR_Release_Request" 🌸
- the Parameter ID of the PDR container is "22", i.e., "PDR_Release_Request" ✨
and for the RESERVE (RMD-QSPEC): "reverse" message, the differences are:
- the Parameter ID of the PHR container is "18", i.e., "PHR_Release_Request" 💜
- the PDR container is not included in the RESERVE (RMD-QSPEC): "reverse" message 🙈
4.6.2.5. Severe Congestion Handling ✨
this part covers severe congestion handling when it's used together with RMD intra-domain bidirectional reservation procedures 🔥 it's pretty much the same as what's in Section 4.6.1.6, so it's giving déjà vu 💀
4.6.2.5.1. Severe Congestion Handling by the RMD-QOSM Bidirectional Refresh Procedure ✨
this one is basically the Section 4.6.1.6.1 procedure, just with a couple of plot twists 🧚 the refresh is done differently (see Section 4.6.2.2), and the flows get terminated differently too (see Section 4.6.2.4) 🫶
4.6.2.5.2. Severe Congestion Handling by Proportional Data Packet Marking ✨
okay this is the proportional data packet marking version when it's paired with an RMD intra-domain bidirectional reservation 🦋 here's the tea: the detection and marking/re-marking stuff that Interior nodes do works for NSIS-aware nodes AND NSIS-unaware nodes 😭 which means the "not NSIS-aware" Interior nodes MUST be configured so they can detect congestion and re-mark packets the same way the Interior "NSIS-aware" nodes do 🔥
this is also similar to Section 4.6.1.6.2, with one big difference: where the severely congested node is, on the "forward" or "reverse" path 🌈 and here's the important bit 👑 when severe congestion happens on, say, the forward path and flows get terminated to fix it, the bandwidth reserved for those terminated bidirectional flows gets released too 💖 so picking which flows to terminate SHOULD be done carefully 🫶 there's an example of how to pick them in Appendix A.5 ✨
there's also a special case where severe congestion shows up on the forward and reverse paths at the same time 💀 Appendix A.6 has an example of that one 🎀
if you want the simulation results behind all this, they're in [DiKa08] 🌸
QNE(Ingress) QNE (int.) QNE (int.) QNE (int.) QNE(Egress)
NTLP stateful NTLP st.less NTLP st.less NTLP st.less NTLP stateful
user| | | | |
data| user | | | |
--->| data | user data | |user data |
|--------------->| | S |
| |--------------------------->S (#marked bytes)
| | | S-------------->|
| | | S(#unmarked bytes)
| | | S-------------->|Term
| | | S |flow?
| | NOTIFY (PDR) S |YES
|<------------------------------------------------------------|
|RESERVE(RMD-QSPEC) | S |
|"forward - T tear" | S |
|--------------->| | RESERVE(RMD-QSPEC):|
| |--------------------------->S"forward - T tear"
| | | S-------------->|
| | | RESERVE(RMD-QSPEC): |
| | | "reverse - T tear" |
| RESERVE(RMD-QSPEC): | |<--------------|
|"reverse - T tear" |<-------------S |
|<-----------------------------| S |
Figure 20: Intra-domain RMD severe congestion handling for
bidirectional reservation (congestion on path
QNE(Ingress) towards QNE(Egress))Figure 20 is the one where the severely congested node is on the "forward" path 🦋 the QNE Egress node has to send an end-to-end NOTIFY (PDR) message 💌 so the QNE Ingress can get the (#marked and #unmarked) counts that the QNE Egress measured on the congested "forward" path 🧚 in this case the "reverse" path is assumed to be not congested 🌸
this is pretty similar to the scenario in Section 4.6.1.6.2 and Figure 14 💅 the only difference is the release, which works the same way as in Section 4.6.2.4 ✨
Figure 21 is the flip side, where the severely congested node is on the "reverse" path 🔄 here the "forward" path is assumed to be not congested 🫶 the big difference from Figure 20 is that the QNE Egress node does NOT have to generate an end-to-end NOTIFY (PDR) message 🙈
that's because the severe congestion is on the "reverse" path, so the QNE Ingress gets the (#marked and #unmarked) user data passing through that congested "reverse" path directly 🎀 from there the QNE Ingress can work out how many flows need to be terminated or moved to a lower priority queue 👑
QNE(Ingress) QNE (int.) QNE (int.) QNE (int.) QNE(Egress)
NTLP stateful NTLP st.less NTLP st.less NTLP st.less NTLP stateful
user| | | | |
data| user | | | |
--->| data | user data | |user data |
|--------------->| | | |
| |--------------------------->|user data |user
| | | |-------------->|data
| | | | |--->
| | | user | |<---
| user data | | data |<--------------|
| (#marked bytes)| S<----------| |
|<--------------------------------S | |
| (#unmarked bytes) S | |
Term|<--------------------------------S | |
Flow? | S | |
YES |RESERVE(RMD-QSPEC): S | |
|"forward - T tear" s | |
|--------------->| RESERVE(RMD-QSPEC): | |
| | "forward - T tear" | |
| |--------------------------->| |
| | S |-------------->|
| | S RESERVE(RMD-QSPEC):
| | S "reverse - T tear" |
| RESERVE(RMD-QSPEC) S |<--------------|
| "reverse - T tear" S<----------| |
|<--------------------------------S | |
Figure 21: Intra-domain RMD severe congestion handling for
bidirectional reservation (congestion on path
QNE(Egress) towards QNE(Ingress))for any flows that have to be terminated 💔 a release procedure (see Section 4.6.2.4) kicks in to free up the reserved resources on both the "forward" and "reverse" paths 🌈
4.6.2.6. Admission Control Using Congestion Notification Based on Probing ✨
this section covers the admission control scheme that uses probing-based congestion notification when RMD intra-domain bidirectional reservations are supported 🧚 it's giving main character energy 👑
QNE(Ingress) Interior QNE (int.) Interior QNE(Egress)okay so this is basically the same energy as the congestion notification for admission control in Section 4.6.1.7 🫶 the big difference is WHERE the congested node is sitting, on the "forward" path (QNE Ingress towards QNE Egress) or the "reverse" path (QNE Egress towards QNE Ingress) 🦋
NTLP stateful not NSIS aware not NSIS aware not NSIS aware NTLP stateful
user| | | | |
data| | | | |
--->| | user data | |user data |
|-------------------------------------------->S (#marked bytes)
| | | S-------------->|
| | | S(#unmarked bytes)
| | | S-------------->|
| | | S |
| | RESERVE(re-marked DSCP in GIST)):|
| | | S |
|-------------------------------------------->S |
| | | S-------------->|
| | | S |
| | RESPONSE(unsuccessful INFO-SPEC) |
|<------------------------------------------------------------|
| | | S |
Figure 22: Intra-domain RMD congestion notification based on
probing for bidirectional admission control (congestion
on path from QNE(Ingress) towards QNE(Egress))So Figure 22 is the forward-path version, and the admission control works exactly the same way as in Section 4.6.1.7, Figure 15 ✨
Figure 23 is the spicy one, where the congested node is on the reverse path 😭 The probe RESERVE going forward is not affected at all. BUT the <DSCP> value in the IP header of every packet on the reverse direction flow, plus the GIST message carrying the reverse-direction probe RESERVE, gets re-marked by the congested node. That's how the QNE Ingress finds out congestion happened in the network, so it can refuse to start the new reservation. rude but effective 💅
Okay, super important 🚨 the "not NSIS-aware" Interior nodes MUST be configured so they can detect the congestion and severe congestion situations and re-mark packets the same way the Interior "NSIS-aware" nodes do.
QNE(Ingress) Interior QNE (int.) Interior QNE(Egress)
NTLP stateful not NSIS aware NTLP st.less not NSIS aware NTLP stateful
user| | | | |
data| | | | |
--->| | user data | | |
|-------------------------------------------->|user data |user
| | | |-------------->|data
| | | | |--->
| | | | |user
| | | | |data
| | | | |<---
| S | user data | |
| S user data |<--------------------------|
| user data S<---------------| | |
|<---------------S | | |
| user data S | | |
| (#marked bytes)S | | |
|<---------------S | | |
| S RESERVE(unmarked DSCP in GIST)): |
| S | | |
|----------------S------------------------------------------->|
| S RESERVE(re-marked DSCP in GIST) |
| S<-------------------------------------------|
|<---------------S | | |
Figure 23: Intra-domain RMD congestion notification for
bidirectional admission control (congestion on path
QNE(Egress) towards QNE(Ingress))4.7. Handling of Additional Errors ✨
During QSPEC processing, additional errors MAY occur 🙈 How those get handled and notified is specified in [RFC5975] and [RFC5974].
5. Security Considerations 🔐
5.1. Introduction
Okay this is the main point of the security section 💖 RMD-QOSM is designed to be "lightweight", which means it keeps the number of exchanged signaling messages low and the amount of state at the signaling nodes small (with and without reduced-state operation). The downside is that QNE Interior nodes become kind of second-class citizens, since they can't do as much QoS signaling. Only the QNE Ingress and QNE Egress nodes are allowed to start certain signaling messages. And RMD only focuses on intra-domain deployment 🌸
Here's what that means for security:
- QNE Ingress and QNE Egress nodes need more security and fault protection than QNE Interior nodes, because if they misbehave it can seriously mess with the stability of the whole network 😭 They share a security association and use GIST security to protect their signaling messages. Intra-domain signaling messages used for RMD signaling do not use GIST security, so they don't store security associations.
- Focusing on intra-domain QoS signaling makes trust management simpler and cuts down overall complexity 🧚 See Section 2 of RFC 4081 for a more detailed look at the full set of communication models for end-to-end QoS signaling protocols. The security of RMD-QOSM does not depend on Interior nodes, so cryptographic protection of intra-domain messages via GIST is not used.
Important bit: RMD ALWAYS uses the message exchange in Figure 24, even if there's no end-to-end signaling session. If RMD-QOSM gets triggered by an end-to-end (E2E) signaling exchange, the RESERVE message is created by a node outside the RMD domain and then keeps travelling on (for example, to the data receiver). That kind of exchange is shown in Figure 3. So evaluating an RMD's security always means looking at BOTH signaling sessions, (1) and (2) of Figure 24, together 👑 Also, for the E2E messages (like RESERVE and RESPONSE), a single "hop" means the communication between the QNE Ingress and the QNE Egress, since QNE Interior nodes don't take part in that exchange.
QNE QNE QNE QNE
Ingress Interior Interior Egress
NTLP stateful NTLP stateless NTLP stateless NTLP stateful
| | | |
| RESERVE (1) | | |
+--------------------------------------------->|
| RESERVE' (2) | | |
+-------------->| | |
| | RESERVE' | |
| +-------------->| |
| | | RESERVE' |
| | +------------->|
| | | RESPONSE' (2)|
|<---------------------------------------------+
| | | RESPONSE (1) |
|<---------------------------------------------+
Figure 24: RMD message exchangeAuthorizing QoS reservations is handled by the Authentication, Authorization, and Accounting (AAA) framework. That functionality is inherited from the underlying NSIS QoS NSLP, see [RFC5974], and isn't described again in this document 💅 As a technical solution, the Diameter QoS application [RFC5866] may be used. The end-to-end reservation request arriving at the Ingress node kicks off the authorization procedure with the backend AAA infrastructure. The end-to-end reservation is usually triggered by a human interacting with an app, like a voice-over-IP client when making a call 📞 When authorization succeeds, no further user-initiated QoS authorization check is expected within the RMD domain for the intra-domain reservation ✨
5.2. Security Threats
In RMD-QOSM, the Ingress node builds BOTH the end-to-end and the intra-domain signaling messages, based on the end-to-end message the sender end node started 🦋
The Interior nodes inside the RMD network ignore the end-to-end signaling message, but they do process, modify, and forward the intra-domain signaling messages toward the Egress node. Meanwhile, resource reservation states get installed, modified, or deleted at each Interior node along the data path, based on the content of each intra-domain signaling message. The Edge nodes of an RMD network are critical components that need strong security protection 😭
So they act as security gateways for incoming and outgoing signaling messages. On top of that, you have to place a certain degree of trust in the Interior nodes within the RMD-QOSM network, so they can process signaling messages and take the necessary actions 🫶
With RMD-QOSM, we assume the Ingress and Egress nodes are not controlled by an adversary, and that communication between them is secured using standard GIST security (see Section 6 of [RFC5971]) and gets integrity, replay, and confidentiality protection 🔥 Note that this only covers messages directly addressed to these two nodes, not any other message that intermediaries need to process. The <SESSION-ID> object of the end-to-end communication is visible to the Interior nodes via GIST 👀 To define the security threats for RMD-QOSM, we consider an adversary that may be located inside the RMD domain and could drop, delay, duplicate, inject, or modify signaling packets. Depending on where the adversary is, we talk about an on-path adversary or an off-path adversary, see also RFC 4081 [RFC4081] 💀
5.2.1. On-Path Adversary 👀
okay so the on-path adversary is a node that supports RMD-QOSM and can watch RMD-QOSM signaling message exchanges go by. it's literally sitting on the route, so it sees everything 🫣
1) Dropping signaling messages 🗑️
an adversary can drop any signaling message right after receiving it. this causes a reservation request for new sessions to fail, or the resource units (bandwidth) for ongoing sessions to get deleted once their state times out. 💀
it might also trigger the Ingress node to retransmit the lost messages. in this scenario the adversary drops selected signaling messages, like intra-domain reserve messages. in RMD-QOSM, the Ingress node can provide a retransmission mechanism so signaling messages can reach the Egress node. BUT the retransmissions that the adversary sets off by dropping messages can cause some real problems 😬 so disabling retransmissions in an RMD-QOSM-aware network is recommended, see also Section 4.6.1.1.1. 🔧
2) Delaying Signaling Messages 🐌
literally any signaling message can be delayed by an adversary. for example, if RESERVE' messages get delayed for the whole refresh period, the resource units (bandwidth) reserved along the nodes for those sessions get removed. 😭 the Ingress node then doesn't receive the RESPONSE within a certain period, so it decides the signaling message failed and may retransmit the "failed" message.
the Egress node can tell the delayed message and the retransmitted message apart, so it can still send back a proper response ✨
the Interior nodes, though, suffer from the retransmission. they may end up reserving twice the resource units (bandwidth) that the Ingress node asked for. not it 🙃
3) Replaying Signaling Messages 🔁
an adversary may want to replay signaling messages. it stores the messages it receives, then decides when to replay them and at what rate (packets per second). 🎬
when a RESERVE' message carried an <RII> object, the Egress will reply with a RESPONSE' message towards the Ingress node. the Ingress node can then catch replays by comparing the <RII> value in the RESPONSE' messages with the value it stored. big brain move 🧠
4) Injecting Signaling Messages 💉
similar to the replay attack, the adversary may store part of the information carried in signaling messages, like the <RSN> object. when it injects signaling messages, it combines the stored information with its own generated parameters (the RMD-QSPEC <TMOD-1> parameter, <RII>, etc.) and sends the result out.
the Interior nodes will process these messages by default. they reserve the requested resource units (bandwidth) and pass the messages on to downstream nodes. 😤
it can happen that the resource units (bandwidth) on the Interior nodes run out if the injected messages eat up too much bandwidth. 🥀
5) Modifying Signaling Messages ✏️
on-path adversaries can modify any part of a signaling message. for example, the adversary can change the <M>, <S>, and <O> parameters of the RMD-QSPEC messages. the Egress node then uses the SESSION-ID, and after that the <BOUND-SESSION-ID> objects, to find the flow that's supposed to be terminated or set to lower priority. yikes 😱
it's also possible for the adversary to modify the RMD-QSPEC <TMOD-1> parameter and/or the <PHB Class> parameter, which could change how many resource units (bandwidth) get requested.
5.2.2. Off-Path Adversary 🫥
in this case the adversary is NOT on the path. it doesn't take part in the RMD-QOSM signaling message exchange, so it can't eavesdrop on the messages. it doesn't know valid <RII>s, <RSN>s, or SESSION-IDs, so it has to make up new parameters and build brand new signaling messages.
since Interior nodes operate in reduced-state mode, injected signaling messages are treated as new once, which makes the Interior nodes allocate additional reservation state. that's a lot of extra state for nothing 📦
5.3. Security Requirements 🔐
the following security requirements are set as goals for the intra-domain communication, namely:
- Nodes that are never supposed to take part in the NSIS signaling exchange must not interfere with QNE Interior nodes. Off-path nodes (off-path with regard to the path taken by a particular signaling message exchange) must not be able to interfere with other on-path signaling nodes. 🚧
- The actions allowed by a QNE Interior node should be minimal (i.e., only those specified by the RMD-QOSM). For example, only the QNE Ingress and the QNE Egress nodes are allowed to initiate certain signaling messages. QNE Interior nodes are, for example, allowed to modify certain signaling message payloads. 🧾
note that the term "interfere" covers all sorts of security threats, like denial-of-service, spoofing, replay, signaling message injection, etc. 🙅♀️
5.4. Security Mechanisms 🛡️
an important security mechanism built into RMD-QOSM is the ability to tie the end-to-end RESERVE and the RESERVE' messages together using the BOUND-SESSION-ID. it also lets the Ingress node match the RESERVE' with the RESPONSE' using the <RII>. these mechanisms let the Edge nodes spot unexpected signaling messages 👀✨
we assume that the RESERVE/RESPONSE is sent with hop-by-hop channel security provided by GIST, and that it's protected between the QNE Ingress and the QNE Egress. GIST security mechanisms MUST be used to offer authentication, integrity, and replay protection. furthermore, encryption MUST be used to stop an adversary on the path of the RESERVE message from learning things about the session that it could later use to inject a RESERVE' message. 🔒
the following messages need to be mapped to each other, so that one message can't show up without the other:
a) the RESERVE and the RESERVE' relate to each other at the QNE Egress; and
b) the RESPONSE and the RESERVE relate to each other at the QNE Ingress; and
c) the RESERVE' and the RESPONSE' relate to each other. The <RII> is carried in the RESERVE' message, and the RESPONSE' message generated by the QNE Egress node contains the same <RII> as the RESERVE'. the QNE Ingress can use the <RII> to match the RESERVE' with the RESPONSE'. the QNE Egress can determine whether the RESERVE' was created by the QNE Ingress node, since the intra-domain session that sent the RESERVE' is bound to an end-to-end session via the <BOUND-SESSION-ID> value in the intra-domain QoS-NSLP operational state kept at the QNE Egress. 🧩
the RESERVE and the RESERVE' messages are tied together using the BOUND-SESSION-ID(s) kept by the intra-domain and end-to-end QoS-NSLP operational states at the QNE Edges (see Sections 4.3.1, 4.3.2, and 4.3.3). so there can't be a RESERVE' without a matching RESERVE 🙌 the SESSION-ID does a pretty great job of this if the goal is protecting against off-path adversaries, since they don't see the SESSION-ID carried in the RESERVE and the RESERVE' messages.
BUT if the path changes (because of rerouting or mobility), an adversary could inject RESERVE' messages carrying a SESSION-ID it saw earlier, and could cause real harm 😰
an off-path adversary can, of course, create RESERVE' messages that make intermediate nodes create some state (and trigger other actions). the message would still end up at the QNE Egress node, though. the QNE Egress node would then be able to tell something is going wrong and generate an error message 🚨
the severe congestion handling can be triggered by intermediate nodes (unlike other messages). in many cases, though, intermediate nodes that are experiencing congestion use refresh messages to modify the <S> and <O> parameters of the message. these messages are still initiated by the QNE Ingress node and carry the SESSION-ID. the QNE Egress node uses the SESSION-ID and then the BOUND-SESSION-ID, kept by the intra-domain QoS-NSLP operational state, to refer to a flow that might get terminated. the aspect of intermediate nodes initiating messages for severe congestion handling is for further study. 🔭
during the refresh procedure, a RESERVE' creates a RESPONSE', see Figure 25. the <RII> is carried in the RESERVE' message, and the RESPONSE' message generated by the QNE Egress node contains the same <RII> as the RESERVE'.
the <RII> can be used by the QNE Ingress to match the RESERVE' with the RESPONSE'.
another thing to watch is marking of data traffic. data packets can be modified by an intermediary without any relationship to a signaling session (and a SESSION-ID). the problem shows up if an off-path adversary injects spoofed data packets. 🎭
QNE Ingress QNE Interior QNE Interior QNE Egress
NTLP stateful NTLP stateless NTLP stateless NTLP stateful
| | | |
| REFRESH RESERVE' | |
+-------------->| REFRESH RESERVE' |
| (+RII) +-------------->| REFRESH RESERVE'
| | (+RII) +------------->|
| | | (+RII) |
| | | |
| | | REFRESH |
| | | RESPONSE'|
|<---------------------------------------------+
| | | (+RII) |
Figure 25: RMD REFRESH message exchangethe adversary would need to spoof data packets that relate to the flow identifier of an existing end-to-end reservation that SHOULD be terminated. so the question is how an off-path adversary SHOULD create a data packet that matches an existing flow identifier (if a 5-tuple is used). 🤔 this might not be simple for an adversary, unless we assume the mobility/rerouting case from before, where the path through the network changes and the set of nodes along a path changes over time. 🌍
6. IANA Considerations 🧾
this section defines additional codepoint assignments in the QSPEC Parameter ID registry, in accordance with BCP 26 [RFC5226]. 📋
6.1. Assignment of QSPEC Parameter IDs 🆔
this document specifies the following QSPEC containers in the QSPEC Parameter ID registry created in [RFC5975]:
<PHR_Resource_Request> (Section 4.1.2 above, ID=17)
<PHR_Release_Request> (Section 4.1.2 above, ID=18)
<PHR_Refresh_Update> (Section 4.1.2 above, ID=19)
<PDR_Reservation_Request> (Section 4.1.3 above, ID=20)
<PDR_Refresh_Request> (Section 4.1.3 above, ID=21)
<PDR_Release_Request> (Section 4.1.3 above, ID=22)
<PDR_Reservation_Report> (Section 4.1.3 above, ID=23)
<PDR_Refresh_Report> (Section 4.1.3 above, ID=24)
<PDR_Release_Report> (Section 4.1.3 above, ID=25)
<PDR_Congestion_Report> (Section 4.1.3 above, ID=26)
7. Acknowledgments 🫶
the authors would like to say a huge thank you to everyone who worked on the RMD concept: Z. Turanyi, R. Szabo, G. Pongracz, A. Marquetant, O. Pop, V. Rexhepi, G. Heijenk, D. Partain, M. Jacobsson, S. Oosthoek, P. Wallentin, P. Goering, A. Stienstra, M. de Kogel, M. Zoumaro-Djayoon, M. Swanink, R. Klaver G. Stokkink, J. W. van Houwelingen, D. Dimitrova, T. Sealy, H. Chang, and J. de Waal. 💜🌸
8. References 📚
8.1. Normative References
Okay, these are the must-read ones, bestie 💅
[RFC2119]Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.[RFC2983]Black, D., "Differentiated Services and Tunnels", RFC 2983, October 2000.[RFC5971]Schulzrinne, H. and R. Hancock, "GIST: General Internet Signaling Transport", RFC 5971, October 2010.[RFC5974]Manner, J., Karagiannis, G., and A. McDonald, "NSIS Signaling Layer Protocol (NSLP) for Quality-of-Service Signaling", RFC 5974, October 2010.[RFC5975]Ash, G., Bader, A., Kappler C., and D. Oran, "QSPEC Template for the Quality-of-Service NSIS Signaling Layer Protocol (NSLP)", RFC 5975, October 2010.
8.2. Informative References
These are the extra reads if you're obsessed with the backstory 🌈
[AdCa03]Adler, M., Cai, J.-Y., Shapiro, J. K., Towsley, D., "Estimation of congestion price using probabilistic packet marking", Proc. IEEE INFOCOM, pp. 2068-2078, 2003.[AnHa06]Lachlan L. H. Andrew and Stephen V. Hanly, "The Estimation Error of Adaptive Deterministic Packet Marking", 44th Annual Allerton Conference on Communication, Control and Computing, 2006.[AtLi01]Athuraliya, S., Li, V. H., Low, S. H., Yin, Q., "REM: active queue management", IEEE Network, vol. 15, pp. 48-53, May/June 2001.[Chan07]H. Chang, "Security support in RMD-QOSM", Masters thesis, University of Twente, 2007.[CsTa05]Csaszar, A., Takacs, A., Szabo, R., Henk, T., "Resilient Reduced-State Resource Reservation", Journal of Communication and Networks, Vol. 7, No. 4, December 2005.[DiKa08]Dimitrova, D., Karagiannis, G., de Boer, P.-T., "Severe congestion handling approaches in NSIS RMD domains with bi-directional reservations", Journal of Computer Communications, Elsevier, vol. 31, pp. 3153-3162, 2008.[JaSh97]Jamin, S., Shenker, S., Danzig, P., "Comparison of Measurement-based Admission Control Algorithms for Controlled-Load Service", Proceedings IEEE Infocom '97, Kobe, Japan, April 1997.[GrTs03]Grossglauser, M., Tse, D.N.C, "A Time-Scale Decomposition Approach to Measurement-Based Admission Control", IEEE/ACM Transactions on Networking, Vol. 11, No. 4, August 2003.[Part94]C. Partridge, Gigabit Networking, Addison Wesley Publishers (1994).[RFC1633]Braden, R., Clark, D., and S. Shenker, "Integrated Services in the Internet Architecture: an Overview", RFC 1633, June 1994.[RFC2215]Shenker, S. and J. Wroclawski, "General Characterization Parameters for Integrated Service Network Elements", RFC 2215, September 1997.[RFC2475]Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z., and W. Weiss, "An Architecture for Differentiated Service", RFC 2475, December 1998.[RFC2638]Nichols, K., Jacobson, V., and L. Zhang, "A Two-bit Differentiated Services Architecture for the Internet", RFC 2638, July 1999.[RFC2998]Bernet, Y., Ford, P., Yavatkar, R., Baker, F., Zhang, L., Speer, M., Braden, R., Davie, B., Wroclawski, J., and E. Felstaine, "A Framework for Integrated Services Operation over Diffserv Networks", RFC 2998, November 2000.[RFC3175]Baker, F., Iturralde, C., Le Faucheur, F., and B. Davie, "Aggregation of RSVP for IPv4 and IPv6 Reservations", RFC 3175, September 2001.[RFC3726]Brunner, M., Ed., "Requirements for Signaling Protocols", RFC 3726, April 2004.[RFC4125]Le Faucheur, F. and W. Lai, "Maximum Allocation Bandwidth Constraints Model for Diffserv-aware MPLS Traffic Engineering", RFC 4125, June 2005.[RFC4127]Le Faucheur, F., Ed., "Russian Dolls Bandwidth Constraints Model for Diffserv-aware MPLS Traffic Engineering", RFC 4127, June 2005.[RFC4081]Tschofenig, H. and D. Kroeselberg, "Security Threats for Next Steps in Signaling (NSIS)", RFC 4081, June 2005.[RFC5226]Narten, T. and H. Alvestrand, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 5226, May 2008.[RFC5866]Sun, D., Ed., McCann, P., Tschofenig, H., Tsou, T., Doria, A., and G. Zorn, Ed., "Diameter Quality-of-Service Application", RFC 5866, May 2010.[RFC5978]Manner, J., Bless, R., Loughney, J., and E. Davies, Ed., "Using and Extending the NSIS Protocol Family", RFC 5978, October 2010.[RMD1]Westberg, L., et al., "Resource Management in Diffserv (RMD): A Functionality and Performance Behavior Overview", IFIP PfHSN 2002.[RMD2]G. Karagiannis, et al., "RMD - a lightweight application of NSIS" Networks 2004, Vienna, Austria.[RMD3]Marquetant A., Pop O., Szabo R., Dinnyes G., Turanyi Z., "Novel Enhancements to Load Control - A Soft-State, Lightweight Admission Control Protocol", Proc. of the 2nd Int. Workshop on Quality of Future Internet Services, Coimbra, Portugal, Sept 24-26, 2001, pp. 82-96.[RMD4]A. Csaszar et al., "Severe congestion handling with resource management in diffserv on demand", Networking 2002.[TaCh99]P. P. Tang, T-Y Charles Tai, "Network Traffic Characterization Using Token Bucket Model", IEEE Infocom 1999, The Conference on Computer Communications, no. 1, March 1999, pp. 51-62.[ThCo04]Thommes, R. W., Coates, M. J., "Deterministic packet marking for congestion packet estimation" Proc. IEEE Infocom, 2004.
Appendix A. Examples 🧚
A.1. Example of a Re-Marking Operation during Severe Congestion in the Interior Nodes
Okay so this appendix walks through a re-marking operation during severe congestion in the Interior nodes, and it's giving main character energy ✨
Per supported PHB, the Interior node can support the operation states shown in Figure 26 🎀 when the per-flow congestion notification based on probing signaling scheme is used together with this severe congestion type. Figure 27 shows the same functionality when the per-flow congestion notification based on probing scheme is not used alongside the severe congestion scheme. The stuff in this appendix and the ones after it focuses on the situation where: (1) the "notified DSCP" marking is used in congestion notification state, and (2) the "encoded DSCP" and "affected DSCP" markings are used in severe congestion state. So in this setup, the "notified DSCP" marking marks all packets going through an Interior node that's in the congestion notification state 💖. That's how, together with probing, a flow-based ECMP solution can be provided for the congestion notification state. The "encoded DSCP" marking is used to encode and signal the excess rate, measured at Interior nodes, to the Egress nodes 🦋. The "affected DSCP" marking marks every packet that goes through a severe congested node and isn't already "encoded DSCP" marked 🫶.
Another possible situation could be derived, where both congestion notification and severe congestion state use the "encoded DSCP" marking, without using the "notified DSCP" marking. The "affected DSCP" marking marks all packets that pass through an Interior node in severe congestion state and aren't "encoded DSCP" marked 🌸. Also, the probe packet carried by an intra-domain RESERVE message that passes through Interior nodes SHOULD be "encoded DSCP" marked if the Interior node is in congestion notification or severe congestion states 👑. Otherwise, the probe packet stays unmarked. This way, an ECMP solution can be provided for both congestion notification and severe congestion states 🌈. The "encoded DSCP" packets signal an excess rate that's not only tied to Interior nodes in severe congestion state, but also to Interior nodes in congestion notification state 🍓. The algorithm at the Interior node is similar to the algorithm described in the following appendix sections. However, this method is not described in detail in this example 🙈.
Here's the state diagram for the combined case, Figure 26:
---------------------------------------------
| event B |
| V
---------- ------------- ----------
| Normal | event A | Congestion | event B | Severe |
| state |---------->| notification|-------->|congestion|
| | | state | | state |
---------- ------------- ----------
^ ^ | |
| | event C | |
| ----------------------- |
| event D |
------------------------------------------------Figure 26: States of operation, severe congestion combined with congestion notification based on probing
And here's Figure 27, the one without congestion notification based on probing:
---------- -------------
| Normal | event B | Severe |
| state |-------------->| congestion |
| | | state |
---------- -------------
^ |
| event E |
---------------------------Figure 27: States of operation, severe congestion without congestion notification based on probing
Quick decoder ring for the terms in Figures 26 and 27 🔍:
- Normal state: represents the normal operation conditions of the node, i.e., no congestion.
- Severe congestion state: represents the state in which the Interior node is severely congested related to a certain PHB. It is important to emphasize that one of the targets of the severe congestion state solution is to change the severe congestion state behavior directly to the normal state 🚨.
- Congestion notification: state in which the load is relatively high, close to the level when congestion can occur.
- event A: this event occurs when the incoming PHB rate is higher than the "congestion notification detection" threshold and lower than the "severe congestion detection". This threshold is used by the congestion notification based on probing scheme, see Sections 4.6.1.7 and 4.6.2.6 💅.
- event B: this event occurs when the incoming PHB rate is higher than the "severe congestion detection" threshold.
- event C: this event occurs when the incoming PHB rate is lower than or equal to the "congestion notification detection" threshold.
- event D: this event occurs when the incoming PHB rate is lower than or equal to the "severe_congestion_restoration" threshold. It is important to emphasize that this event supports one of the targets of the severe congestion state solution, which is to change the severe congestion state behavior directly to the normal state 💜.
- event E: this event occurs when the incoming PHB rate is lower than or equal to the "severe congestion restoration" threshold.
Okay bestie, so now we're in the severe congestion zone 💀🔥 and the thresholds have a whole pecking order. The severe congestion detection, severe congestion restoration, and admission thresholds SHOULD all sit higher than the congestion notification detection threshold. That means "severe congestion detection" > "congestion notification detection" AND "severe congestion restoration" > "congestion notification detection" ✨
"severe congestion detection" > "congestion notification detection"
"severe congestion restoration" > "congestion notification detection"The severe congestion detection threshold SHOULD also be higher than or equal to the admission threshold used by the reservation-based and NSIS measurement-based signaling schemes 🎀
"severe congestion detection" >= admission thresholdAnd the severe congestion restoration threshold SHOULD be lower than or equal to the severe congestion detection threshold, used by those same reservation-based and NSIS measurement-based schemes 🦋
"severe congestion restoration" <= "severe congestion detection"Now, during severe congestion, the Interior node works out, per traffic class (PHB), the incoming rate that's above the severe congestion restoration threshold. This one's called signaled_overload_rate, and here's how the Interior node gets there 💅
- A severely congested Interior node SHOULD keep in mind that packets might get dropped. So before it queues and eventually drops anything, it SHOULD count the total number of unmarked and re-marked bytes it receives, and call that total_received_bytes 🧚
- Plot twist: more than one Interior node on the same path can end up severely congested. So any Interior node sitting behind a severely congested node MAY receive marked bytes 👀
When the severe congestion detection threshold per PHB is set equal to the maximum capacity allocated to one PHB used by the RMD-QOSM, it means that if that PHB's maximum capacity is fully used and a packet from that PHB arrives, the Interior node is assumed NOT to forward that packet downstream 😭 Wait, I can't capitalize that as a keyword, so let me restate it plainly: the Interior node is assumed not to forward that packet downstream.
In other words, that packet either gets dropped or set to another PHB 💜 Also, once the severe congestion situation is solved, the ongoing flows can send their packets up to a total rate equal to the maximum capacity of the PHB. So when more than one Interior node on the same path is severely congested, and an Interior node receives "encoded DSCP" marked packets, it means an Interior node further upstream is also severely congested 🌸
When the severe congestion detection threshold per PHB is set equal to the maximum capacity allocated to one PHB, this Interior node MUST forward the "encoded DSCP" marked packets 🍓 It SHOULD NOT consider those packets during its local re-marking process. In other words, the Egress should treat the excess rates encoded by the different severely congested Interior nodes as independent, so those independent excess rates get added together 🧮
When the severe congestion detection threshold per PHB is NOT set equal to the maximum capacity allocated to one PHB, things change a bit. After the severe congestion situation is solved, the ongoing flows won't be able to send up to the full maximum PHB capacity, only up to the "severe_congestion_threshold" 🙈 If more than one Interior node on the same communication path is severely congested, and one of them receives "encoded_DSCP" marked packets, that Interior node SHOULD NOT mark unmarked packets (either "original DSCP", "affected DSCP", or "notified DSCP" encoded) up to a rate equal to the difference between the maximum PHB capacity and the "severe congestion threshold". This applies when the incoming "encoded DSCP" marked packets can already signal that difference ✨
In this case, the "severe congestion threshold" SHOULD be configured in all Interior nodes located in the RMD domain, and set to:
"severe_congestion_threshold" =
Maximum PHB capacity - threshold_offset_rateThe threshold_offset_rate is a rate, and it SHOULD have the same value in all Interior nodes 🎀
- Before queuing and eventually dropping packets, at the end of each measurement interval of T seconds, the Interior node calculates the current estimated overloaded rate, called measured_overload_rate, using the following equation 💖
measured_overload_rate =
=((total_received_bytes)/T)-severe_congestion_restoration)Okay, so getting a reliable estimate of the encoded information can use several techniques. Check out [AtLi01], [AdCa03], [ThCo04], and [AnHa06] 📚 Since marking happens in Interior nodes, the decisions happen at Egress nodes, and flow termination happens at Ingress nodes, there's a big delay before the Ingress nodes learn about the overload. See Section 6 of [CsTa05] for the full picture 😭 That delay is made of the trip time of data packets from the severely congested Interior node to the Egress, the measurement interval (that's T), and the trip time of the notification signaling messages from the Egress back to the Ingress 🚀
On top of that, until the overload goes down at the severely congested Interior node, an additional trip time from the Ingress node to the severely congested Interior node MUST expire 🫶 Why? Because right before the congestion notification arrives, the Ingress MAY have already sent packets in the flows that were picked for termination. So a terminated flow MAY keep contributing to congestion for longer, for the time it takes to get from the Ingress to the Interior node. If you ignore all of this, Interior nodes would keep marking packets until the measured utilization drops below the severe congestion restoration threshold. The end result? More flows get terminated than necessary, which is an overreaction 😤
[CsTa05] fixes this with a sliding window memory. Interior nodes use it to track the signaling overload across a couple of previous measurement intervals. At the end of a measurement interval T, before encoding and signaling the overloaded rate as "encoded DSCP" packets, the actual overload gets reduced by the sum of the overload already signaled and stored in the sliding window memory. That overload is already being handled by the severe congestion handling control loop, so no double counting 🧠 The sliding window memory is made of an integer number of cells, n = maximum number of cells. Guidelines for setting the sliding window parameters are in [CsTa05] 📖
At the end of each measurement interval, the newest calculated overload gets pushed into the memory, and the oldest cell is dropped 🗑️
If Mi is the overload_rate stored in the ith memory cell (i = [1..n]), then at the end of every measurement interval, the overload rate signaled to the Egress node (that's signaled_overload_rate) is calculated like this 🧁
Sum_Mi =0
For i =1 to n
{
Sum_Mi = Sum_Mi + Mi
}
signaled_overload_rate = measured_overload_rate - Sum_Mi,where Sum_Mi is calculated as above 💅
Next, the sliding memory gets updated like so:
for i = 1..(n-1): Mi <- Mi+1
Mn <- signaled_overload_rateThe bytes that have to be re-marked to satisfy the signaled overload rate, called signaled_remarked_bytes, are calculated with this pseudocode 🌈
IF severe_congestion_threshold <> Maximum PHB capacity
THEN
{
IF (incoming_encoded-DSCP_rate <> 0) AND
(incoming_encoded-DSCP_rate =< termination_offset_rate)
THEN
{ signaled_remarked_bytes =
= ((signaled_overload_rate - incoming_encoded-DSCP_rate)*T)/N
}
ELSE IF (incoming_encoded-DSCP_rate > termination_offset_rate)
THEN signaled_remarked_bytes =
= ((signaled_overload_rate - termination_offset_rate)*T)/N
ELSE IF (incoming_encoded-DSCP_rate =0)
THEN signaled_remarked_bytes =
= signaled_overload_rate*T/N
}
ELSE signaled_remarked_bytes = signaled_overload_rate *T/N
Where the incoming "encoded DSCP" rate is calculated as follows:
incoming_encoded-DSCP_rate =
= (received number of "encoded_DSCP" during T) * N)/T;The signal_remarked_bytes also represents the number of outgoing packets (after the dropping stage) that MUST be re-marked during each measurement interval T, by a node operating in severe congestion mode 🫧
Now, to handle an overload situation higher than 100% of the maintained severe congestion threshold, all the nodes within the domain MUST be configured and MUST maintain a scaling parameter, e.g., N in the equation above. Combined with the marked bytes (e.g., signaled_remarked_bytes), this is how such a high overload can be calculated and represented 💜 N can be equal to or higher than 1 🌸
Heads up: when incoming re-marked bytes get dropped, the severe congestion algorithm MAY be affected. It MAY become slower in certain situations. An implementation of the algorithm MAY try as hard as it can to make sure incoming marked bytes aren't dropped, for example by using different dropping rate thresholds for marked and unmarked bytes 🙈
When the "affected DSCP" marking is used by a node that's congested due to a severe congestion situation, all the outgoing packets that aren't marked (that is, not marked using "encoded DSCP") have to be re-marked using "affected DSCP" 🍓
The "encoded DSCP" and "affected DSCP" marked packets, when applied across the whole RMD domain, get propagated to the QNE Edge nodes 🦋
Last but not least, when the congestion notification based on probing is used together with severe congestion, there's one more DSCP for re-marking the same PHB, on top of "encoded DSCP" and "affected DSCP". See Section 4.6.1.7. This extra one is called "notified DSCP" in this document 👑 When an Interior node is in the severe congested state (see Figure 27) and receives "notified DSCP" packets, those packets are treated as unmarked packets, but NOT as "affected DSCP" packets. So during severe congestion, "notified DSCP" packets can also be re-marked and encoded as either "encoded DSCP" or "affected DSCP" packets ✨
A.2. Example of a Detailed Severe Congestion Operation in the Egress Nodes 💖
okay so this appendix walks through a detailed example of severe congestion handling in the Egress nodes 🫶
the Egress nodes basically have the same states as in Appendix A.1 🦋 but the events are defined differently than the ones in Figures 26 and 27, so pay attention bestie 👀
- event A: when the Egress receives a predefined rate of "notified DSCP" marked bytes/packets, event A gets activated (see Sections 4.6.1.7 and A.4) ✨ The predefined rate of "notified DSCP" marked bytes is called the congestion notification detection threshold. ngl this threshold can also be zero, which means event A gets activated when the Egress node, during an interval T, receives at least one "notified DSCP" packet 🙈
- event B: this one happens when the Egress receives packets marked as either "encoded DSCP" or "affected DSCP" (when "affected DSCP" is applied in the whole RMD domain) 🎀
- event C: this one happens when the rate of incoming "notified DSCP" packets drops below the congestion notification detection threshold 📉 If that threshold is zero, event C gets activated when the Egress node, during an interval T, does not receive any "notified DSCP" marked packets 💀
- event D: this happens when the Egress, during an interval T, does not receive packets marked as either "encoded DSCP" or "affected DSCP" (when "affected DSCP" is applied in the whole RMD domain) 😭 When "notified DSCP" is applied in the whole RMD domain for congestion notification, this event can cause the following change in operation state, so watch this part 👑
When the Egress, during an interval T, does not receive (1) packets marked as either "encoded DSCP" or "affected DSCP" (when "affected DSCP" is applied in the whole RMD domain) and (2) it does NOT receive "notified DSCP" marked packets, the operation state changes from the severe congestion state to the normal state 🌸
When the Egress, during an interval T, does not receive (1) packets marked as either "encoded DSCP" or "affected DSCP" (when "affected DSCP" is applied in the whole RMD domain) and (2) it does receive "notified DSCP" marked packets, the operation state changes from the severe congestion state to the congestion notification state 🧚
- event E: this happens when the Egress, during an interval T, does not receive packets marked as either "encoded DSCP" or "affected DSCP" (when "affected DSCP" is applied in the whole RMD domain) 🍓
okay so next up is the algorithm for working out how many flows in each priority class have to be terminated. the pseudocode below shows how it's done 💅
the Edge nodes can handle severe congestion by (1) figuring out which flows were hit by the severe congestion and (2) picking and terminating some of those flows so the quality of service of the remaining flows gets recovered 🔥
the "encoded DSCP" and "affected DSCP" marked packets (when applied in the whole RMD domain) get received by the QNE Edge node 🦋
the QNE Edge nodes keep per-flow state, so they can translate the calculated bandwidth to be terminated into a number of flows ✨ The QNE Egress node records the excess rate and the identity of every flow arriving at it with "encoded DSCP" and with "affected DSCP" (when applied in the whole RMD domain). only these flows, the ones passing through the severely congested Interior node(s), are candidates for termination 🎀 The excess rate is measured by looking at the rate of all the "encoded DSCP" data packets that arrive at the QNE Egress node. then the Egress node converts the measured excess rate by multiplying it by the factor N, which the QNE Interior node(s) used to encode the overload level 🌈
when different priority flows are supported, all the low priority flows that arrived at the Egress node get terminated first 🍓 next come the medium priority flows, and finally, if necessary, even high priority flows get picked 👑 Within a priority class, both "encoded DSCP" and "affected DSCP" are considered before the mechanism moves on to a higher priority class 💜 Last but not least, for each flow that has to be terminated, the Egress node sends a NOTIFY message to the Ingress node, which stops the flow 🫶
here's the detailed version 🧚
first, when the Egress operates in the severe congestion state, the total amount of re-marked bandwidth associated with the PHB traffic class, say total_congested_bandwidth, gets calculated 💖 If the node keeps information about each Ingress/Egress pair aggregate, then the total_congested_bandwidth MUST be calculated per Ingress/Egress pair reservation aggregate 🌸 This bandwidth represents the severely congested bandwidth that SHOULD be terminated ✨ The total_congested_bandwidth can be calculated like this:
total_congested_bandwidth = N*input_remarked_bytes/Twhere input_remarked_bytes is the number of "encoded DSCP" marked bytes that arrive at the Egress during one measurement interval T, and N is defined as in Sections 4.6.1.6.2.1 and A.1 🦋 The term terminated_bandwidth is a temporary variable holding the total bandwidth that has to be terminated, belonging to the same PHB traffic class 🎀 The terminate_flow_bandwidth(priority_class) is the total bandwidth associated with flows of priority class equal to priority_class 🍓 The parameter priority_class is an integer that fulfils:
0 =< priority_class =< Maximum_prioritythe QNE Egress node records the identity of the QNE Ingress node that forwarded each flow, the total_congested_bandwidth, and the identity of every flow arriving at the QNE Egress node with "encoded DSCP" and "affected DSCP" (when applied in the whole RMD domain) 🧚 This makes sure that only these flows, the ones passing through the severely overloaded QNE Interior node(s), are candidates for termination 🔥 The flows to be terminated get selected by the pseudocode below, which is realized by the function called calculate_terminate_flows() 💅
the calculate_terminate_flows() function uses the <terminate_bandwidth_class> value and translates this bandwidth value into the number of flows that have to be terminated 👑 Only the "encoded DSCP" flows and "affected DSCP" flows (when applied in the whole RMD domain), the ones passing through the severely overloaded Interior node(s), are candidates for termination 💜
after the flows to be terminated are selected, the <sum_bandwidth_terminate(priority_class)> value gets calculated 🌈 That's the sum of the bandwidth associated with the flows, belonging to a certain priority class, that will definitely be terminated 🫶
the constraint when finding the total number of flows to terminate is that sum_bandwidth_terminate(priority_class) SHOULD be smaller than or approximately equal to the variable terminate_bandwidth(priority_class) ✨
terminated_bandwidth = 0;
priority_class = 0;
while terminated_bandwidth < total_congested_bandwidth
{
terminate_bandwidth(priority_class)
= total_congested_bandwidth - terminated_bandwidth
calculate_terminate_flows(priority_class);
terminated_bandwidth =
= sum_bandwidth_terminate(priority_class) + terminated_bandwidth;
priority_class = priority_class + 1;
}If the Egress node maintains Ingress/Egress pair reservation aggregates, then the algorithm above gets performed for each Ingress/Egress pair reservation aggregate 🍓
finally, for each flow that has to be terminated, the QNE Egress node sends a NOTIFY message to the QNE Ingress node to terminate the flow 💖
A.3. Example of a Detailed Re-Marking Admission Control (Congestion Notification) Operation in Interior Nodes 🌈
this appendix shows a detailed example of a re-marking admission control (congestion notification) operation in Interior nodes 🦋 The predefined congestion notification threshold (see Appendix A.1) is set according to, and usually less than, an engineered bandwidth limitation, a.k.a. the admission threshold 🎀 That could be based on a Service Level Agreement or the capacity limitation of specific links 💅
the gap between the congestion notification threshold and the engineered bandwidth limitation (the admission threshold) gives you an interval where a node has already sent the signaling information about resource limitation, but the actual resource limitation hasn't been reached yet 🌸 This happens because the data packets associated with an admitted session haven't arrived yet, which lets the admission control process at the Egress interpret the signaling information and reject new calls before congestion happens 🫶
ok but here's the thing: when the data rate is higher than the preconfigured congestion notification rate, data packets get re-marked too (see Section 4.6.1.6.2.1) 👀 To tell congestion notification apart from severe congestion, two methods MAY be used (see Appendix A.1):
- using different <DSCP> values (re-marked <DSCP> values). The re-marked DSCP used for this purpose is called "notified DSCP" in this document 💜 When this method is used and the Interior node is in the "congestion notification" state (see Appendix A.1), the node SHOULD re-mark all the unmarked bytes passing through it using the "notified DSCP" 🧚 Note this method can only be applied if all nodes in the RMD domain use the "notified" DSCP marking 🔥 That way, probe packets that pass through an Interior node in the congestion notification state are also encoded using the "notified DSCP" marking ✨
- Using the "encoded DSCP" marking for congestion notification and severe congestion. This method isn't described in detail in this example appendix 🍓
A.4. Example of a Detailed Admission Control (Congestion Notification) Operation in Egress Nodes ✨
okay so this appendix walks through a detailed admission control (congestion notification) operation in Egress nodes 🎀
This admission control congestion notification procedure can only be applied if the Egress keeps track of the Ingress/Egress pair aggregate 💅 When the operation state of that aggregate is "congestion notification" (see Appendix A.2), the way the algorithm works depends on how the congestion situation gets signalled to the Egress. As mentioned in Appendix A.3, there are two methods:
- using the "notified DSCP". During a measurement interval T, the Egress counts the "notified DSCP" marked bytes that belong to the same PHB and are associated with the same Ingress/Egress pair aggregate, which we call input_notified_bytes. We denote the rate as incoming_notified_rate.
- using the "encoded DSCP". In this case, during a measurement interval T, the Egress measures input_notified_bytes by counting the "encoded DSCP" bytes.
Only the detailed description of the first method is given below 🫶
The incoming congestion_rate can then be calculated like this:
incoming_congestion_rate = input_notified_bytes/TIf the incoming_congestion_rate is higher than a preconfigured congestion notification threshold, the communication path between Ingress and Egress is considered congested 😭 Note that the pre-congestion notification threshold can be set to "0". In that case, the Egress node operates in congestion notification state the moment it receives at least one "notified DSCP" encoded packet 🔥
When the Egress node is in "congestion notification" state and an end-to-end RESERVE (probe) arrives at the Egress, this request SHOULD be rejected 🙈 This only happens when the probe packet is either "notified DSCP" or "encoded DSCP" marked. That way, the end-to-end RESERVE (probe) packet is guaranteed to have passed through the node that is congested, which is giving main character energy. It's super useful when ECMP-based routing is used, because it lets you detect only the flows that are passing through the congested router 👑
If that Ingress/Egress pair aggregated state is not available when the (probe) RESERVE message arrives at the Egress, the request is accepted if the DSCP of the packet carrying the RESERVE message is unmarked. Otherwise, if the packet is either "notified DSCP" or "encoded DSCP" marked, it gets rejected 💀
A.5. Example of Selecting Bidirectional Flows for Termination during Severe Congestion 🦋
This appendix shows an example of picking bidirectional flows for termination during severe congestion 🌸
When severe congestion happens, e.g., in the forward path, and the algorithm terminates flows to fix it, the reserved bandwidth tied to those terminated bidirectional flows gets released too 🧚 So a careful selection of which flows get terminated SHOULD take place ✨ One possible method for selecting flows of the same priority type that pass through the severe congestion point on a unidirectional path goes like this:
- the Egress node SHOULD select, if possible, first unidirectional flows instead of bidirectional flows.
- the Egress node SHOULD select, if possible, bidirectional flows that reserved a relatively small amount of resources on the path reversed to the path of congestion.
A.6. Example of a Severe Congestion Solution for Bidirectional Flows Congested Simultaneously on Forward and Reverse Paths
okay bestie, this appendix walks through a severe congestion solution for bidirectional flows that get congested at the same time on the forward and reverse paths 🧚🌸
this scenario is a combo move. It uses the severe congestion solutions from Section 4.6.2.5.2 together. The congestion hits both directions at once, and it MAY affect the very same bidirectional flows, so the drama is real 😭💅
When the QNE Edges maintain per-flow intra-domain QoS-NSLP operational states, the steps can be the following, see Figure A.3 🎀. Say the Egress picks some bidirectional flows to be terminated. It sends a NOTIFY message to the Ingress for each one. If the Ingress gets these NOTIFY messages and its operational state (tied to the reverse path) is in the severe congestion state (see Figures 26 and 27), the Ingress does the following:
- For each NOTIFY message, the Ingress SHOULD identify the bidirectional flows that have to be terminated 🔍
- The Ingress then calculates the total bandwidth that SHOULD be released in the reverse direction (thus not in forward direction) if the bidirectional flows get terminated (preempted). Call it
"notify_reverse_bandwidth". This bandwidth is the sum of the bandwidth values for all the end-to-end sessions that received a (severe congestion) NOTIFY message 🧮 - Furthermore, using the received marked packets (from the reverse path), the Ingress calculates, with the algorithm an Egress uses (described in Appendix A.2), the total bandwidth that has to be terminated to fix the congestion in the reverse path direction. Call it
"marked_reverse_bandwidth"✨ - The Ingress then calculates the bandwidth of the additional flows that have to be terminated, called
"additional_reverse_bandwidth". This fixes the severe congestion in the reverse direction by taking into account:- the reverse direction bandwidth of the bidirectional flows the Egress picked (the ones that got a NOTIFY message) to be preempted, i.e.,
"notify_reverse_bandwidth". - the total reverse direction bandwidth calculated from the received marked packets, i.e.,
"marked_reverse_bandwidth".
- the reverse direction bandwidth of the bidirectional flows the Egress picked (the ones that got a NOTIFY message) to be preempted, i.e.,
QNE(Ingress) NE (int.) NE (int.) NE (int.) QNE(Egress)
NTLP stateful NTLP stateful
data| user | | | |
--->| data | #unmarked bytes| | |
|--------------->S #marked bytes | | |
| S--------------------------->| |
| | | |-------------->|data
| | | | |--->
| | | | Term.?
| NOTIFY | | |Yes
|<------------------------------------------------------------|
| | | | |data
| | | user | |<---
| user data | | data |<--------------|
| (#marked bytes)| S<----------| |
|<--------------------------------S | |
| (#unmarked bytes) S | |
Term|<--------------------------------S | |
Flow? | S | |
YES |RESERVE(RMD-QSPEC): S | |
|"forward - T tear" s | |
|--------------->| RESERVE(RMD-QSPEC): | |
| | "forward - T tear" | |
| |--------------------------->| |
| | S |-------------->|
| | S RESERVE(RMD-QSPEC):
| | S "reverse - T tear" |
| RESERVE(RMD-QSPEC) S |<--------------|
| "reverse - T tear" S<----------| |
|<--------------------------------S | |
Figure 28: Intra-domain RMD severe congestion handling for
bidirectional reservation (congestion in both forward
and reverse direction)This additional bandwidth can be worked out with this algorithm, no stress 🧠💖:
IF ("marked_reverse_bandwidth" > "notify_reverse_bandwidth") THEN
"additional_reverse_bandwidth" =
= "marked_reverse_bandwidth"- "notify_reverse_bandwidth";
ELSE
"additional_reverse_bandwidth" = 0- The Ingress terminates the flows that experienced a severe congestion in the forward path and received a (severe congestion) NOTIFY message 💥
- If possible, the Ingress SHOULD terminate unidirectional flows that use the same Egress-Ingress reverse direction communication path, to release a total bandwidth up to equal to
"additional_reverse_bandwidth". See Appendix A.5 🌈 - If the number of REQUIRED unidirectional flows (to satisfy the above issue) is not available, then a number of bidirectional flows that are using the same Egress-Ingress reverse direction communication path MAY be selected for preemption, so the total bandwidth released is up to equal to
"additional_reverse_bandwidth". Note that, using the guidelines in Appendix A.5, first the bidirectional flows that reserved a relatively small amount of resources on the path reversed to the path of congestion SHOULD be selected for termination 🍓
- If possible, the Ingress SHOULD terminate unidirectional flows that use the same Egress-Ingress reverse direction communication path, to release a total bandwidth up to equal to
When the QNE Edges maintain aggregated intra-domain QoS-NSLP operational states, the steps can be the following. It's a slightly different vibe, but it's still giving the same result 🦋
- The Egress calculates the bandwidth to be terminated using the same method as described in Section 4.6.1.6.2.2. It puts this bandwidth value in a
<PDR Bandwidth>inside a"PDR_Congestion_Report"container, which is carried by the end-to-end NOTIFY message 📦 - The Ingress receives the NOTIFY message and reads the
<PDR Bandwidth>value inside the"PDR_Congestion_Report"container. This value is called"notify_reverse_bandwidth"when the QNE Edges maintain per-flow intra-domain QoS-NSLP operational states, but it's calculated differently here. The variables"marked_reverse_bandwidth"and"additional_reverse_bandwidth"are calculated using the same steps explained for the per-flow situation 🧮 - Regarding the termination of flows that use the same Egress-Ingress reverse direction communication path, the Ingress can follow the same procedures as the situation where the QNE Edges maintain per-flow intra-domain QoS-NSLP operational states 🫶
The RMD-aggregated (reduced-state) reservations maintained by the Interior nodes can be reduced in the "forward" and "reverse" directions. Use the procedure in Section 4.6.2.3, and include, in the <Peak Data Rate-1 (p)> value of the local RMD-QSPEC <TMOD-1> parameter of the RMD-QOSM <QoS Desired> field carried by the forward intra-domain RESERVE, the value equal to <notify_reverse_bandwidth>. Also include the <additional_reverse_bandwidth> value in the <PDR Bandwidth> parameter inside the "PDR_Release_Request" container carried by that same intra-domain RESERVE message 👑✨
A.7. Example of Preemption Handling during Admission Control
this appendix shows how preemption handling gets supported during admission control 💜
This section describes the mechanism that the QNE Ingress, QNE Interior, and QNE Egress nodes can support to handle preemption during the admission control process 🧚
This mechanism uses the preemption building blocks specified in [RFC5974] 🌈
A.7.1. Preemption Handling in QNE Ingress Nodes
If a QNE Ingress receives a RESERVE for a session that causes other session(s) to be preempted, then, for each of these to-be-preempted sessions, the QNE Ingress follows these steps 🚨
Step_1:
The QNE Ingress MUST send a tearing RESERVE downstream and add a BOUND-SESSION-ID, with <Binding_Code> value equal to "Indicated session caused preemption". This indicates the SESSION-ID of the session that caused the preemption. Furthermore, an <INFO-SPEC> object with error code value equal to "Reservation preempted" has to be included in each of these tearing RESERVE messages 💅
The selection of which flows get preempted can be based on predefined policies. For example, this selection can be based on the MRI associated with the high and low priority sessions. In particular, the QNE Ingress can select low(er) priority session(s) whose MRI is "close" (especially the target IP) to the one associated with the higher priority session. This means that typically the high priority session and the to-be-preempted lower priority sessions follow the same communication path and pass through the same QNE Egress node 🛤️
Furthermore, the number of lower priority sessions that get preempted for each high priority session has to be such that the resources requested by the higher priority session SHOULD be lower or equal than the sum of the reserved resources associated with the lower priority sessions that have to be preempted 🍓
Step_2:
For each of the sent tearing RESERVE(s), the QNE Ingress will send a NOTIFY message with an <INFO-SPEC> object with error code value equal to "Reservation preempted" towards the QNI 📨
Step_3:
After sending the preempted (tearing) RESERVE(s), the Ingress QNE will send the (reserving) RESERVE, which caused the preemption, downstream towards the QNE Egress 🦋
A.7.2. Preemption Handling in QNE Interior Nodes
okay so the QNE Interior gets the first (tearing) RESERVE that carries the <BOUND-SESSION-ID> object with <Binding_Code> value equal to "Indicated session caused preemption" and an <INFO-SPEC> object with error code value equal to "Reservation preempted" 🌸. at that point it knows this session has to be preempted 💅
so then the QNE Interior makes a so-called "preemption state" 🦋, which is identified by the SESSION-ID carried in the preemption-related <BOUND-SESSION-ID> object. this "preemption state" also holds the SESSION-ID of the session tied to that (tearing) RESERVE. if more tearing RESERVE(s) show up with the same BOUND-SESSION-ID and <INFO-SPEC> values, their SESSION-IDs just get added to the "preemption state" that already exists 💜. the QNE then starts a timer, set high enough that it will not expire before the (reserving) RESERVE arrives ⏰
ngl, the timer is important. when the "preemption state" timer expires, the bandwidth tied to the preempted session(s) has to be released, following a normal RMD-QOSM bandwidth release procedure 🫶. now, if the QNE Interior does not get all the to-be-preempted (tearing) RESERVE messages that the QNE Ingress sent before their matching (reserving) RESERVE arrives, then that (reserving) RESERVE will not reserve any resources 🙈 and it will be marked "M" (see Section 4.6.1.2). this is not a typical situation tho. it usually only happens when at least one of the (tearing) RESERVE messages gets dropped because of an error condition 💀
otherwise, if the QNE Interior does get all the to-be-preempted (tearing) RESERVE messages from the QNE Ingress, then it removes the pending resources and makes the new reservation using normal RMD-QOSM bandwidth release and reservation procedures ✨👑
A.7.3. Preemption Handling in QNE Egress Nodes
this one is giving the same energy as the QNE Interior 🎀. the QNE Egress, upon receiving the first (tearing) RESERVE that carries the <BOUND-SESSION-ID> object with the <Binding_Code> value equal to "Indicated session caused preemption" and an <INFO-SPEC> object with error code value equal to "Reservation preempted", decides that this session has to be preempted 💖. just like the QNE Interior, it creates a so-called "preemption state", identified by the SESSION-ID carried in the preemption-related <BOUND-SESSION-ID> object. this "preemption state" stores the same type of information and uses the same timer value as specified in Appendix A.7.2 🧚
then, if extra tearing RESERVE(s) arrive with the same values of BOUND-SESSION-ID and <INFO-SPEC> objects, their associated SESSION-IDs get included in the "preemption state" that's already made 🌈
if the (reserving) RESERVE message from the QNE Ingress node arrived and is not "M" marked, and all the to-be-preempted (tearing) RESERVE messages have also arrived, then the QNE Egress removes the pending resources and makes the new reservation using normal RMD-QOSM procedures 🔥
if the QNE Egress gets an "M" marked RESERVE message, then it uses the normal partial RMD-QOSM procedure to release the partial reserved resources associated with that "M" marked RESERVE (see Section 4.6.1.2) 🥺
now the tricky bit. if the QNE Egress will not receive all the to-be-preempted (tearing) RESERVE messages sent by the QNE Ingress before their associated (reserving) RESERVE message arrives (and that one is not "M" marked), then the following steps can be followed 😭
- If the QNE Egress uses an end-to-end QOSM that supports the preemption handling, then it has to calculate and select new lower priority sessions that have to be terminated. how the preempted sessions get selected and signaled to the downstream QNEs is similar to the operation specified in Appendix A.7.1 🦋
- If the QNE Egress does not use an end-to-end QOSM that supports the preemption handling, then it has to reject the requesting (reserving) RESERVE message associated with the high priority session (see Section 4.6.1.2) 🙅♀️
note that this situation (where the QNE Egress doesn't get all the to-be-preempted (tearing) RESERVE messages from the QNE Ingress) typically only happens when at least one of the (tearing) RESERVE messages gets dropped due to an error condition. not a vibe we want 💀
A.8. Example of a Retransmission Procedure within the RMD Domain
this appendix walks through an example of a retransmission procedure that can be used in the RMD domain 🍓
if retransmission of intra-domain RESERVE messages within the RMD domain is not disallowed, then all the QNE Interior nodes SHOULD use the functionality described in this section 🫶
so here's the plan. we let QNE Interior nodes keep a replay cache 🧁 where each entry holds the <RSN>, the <SESSION-ID> (available via GIST), the <REFRESH-PERIOD> (available via the QoS NSLP [RFC5974]), and the last received "PHR Container" <Parameter ID> carried by the RMD-QSPEC for each session [RFC5975]. so this solution uses info carried by <QoS-NSLP> objects [RFC5974] and parameters carried by the RMD-QSPEC "PHR Container" 💅. there are three phases to know about:
Phase 1: Create Replay Cache Entry
when an Interior node gets an intra-domain RESERVE message and its cache is empty or there's no matching entry, it reads the <Parameter ID> field of the "PHR Container" in the received message. if the <Parameter ID> is PHR_RESOURCE_REQUEST, which means the intra-domain RESERVE is a reservation request, then the QNE Interior creates a new entry in the cache and copies the <RSN>, <SESSION-ID> and <Parameter ID> into it, then sets the <REFRESH-PERIOD> 📝
to check whether the received intra-domain RESERVE was sent by an adversary 🕵️♀️, the Interior node uses the info stored in the list. for example, if the <SESSION-ID> and <RSN> of a received intra-domain RESERVE match the values in the list, then the Interior node checks the <Parameter ID> part 🎀
if the <Parameter ID> is different, then:
Situation D1: <Parameter ID> in its own list is PHR_RESOURCE_REQUEST, and <Parameter ID> in the message is PHR_REFRESH_UPDATE;
Situation D2: <Parameter ID> in its own list is PHR_RESOURCE_REQUEST or PHR_REFRESH_UPDATE, and <Parameter ID> in the message is PHR_RELEASE_REQUEST;
Situation D3: <Parameter ID> in its own list is PHR_REFRESH_UPDATE, and <Parameter ID> in the message is PHR_RESOURCE_REQUEST;
for Situation D1, the QNE Interior node processes the message with the RMD-QOSM default operation, reserves bandwidth, updates the entry, and passes the message on to downstream nodes 🌸. for Situation D2, it processes the message with the RMD-QOSM default operation, releases bandwidth, deletes all entries associated with the session, and passes the message downstream 🦋. for Situation D3, the QNE Interior node does not use or process the local RMD-QSPEC <TMOD-1> parameter carried by the received intra-domain RESERVE message 🙈. on top of that, the <K> flag in the "PHR Container" has to be set so that the local RMD-QSPEC <TMOD-1> parameter carried by the intra-domain RESERVE message is not processed or used by a QNE Interior node 🚫
if the <Parameter ID> is the same, then:
Situation S1: <Parameter ID> is equal to PHR_RESOURCE_REQUEST;
Situation S2: <Parameter ID> is equal to PHR_REFRESH_UPDATE;
for Situation S1, the QNE Interior node does not process the intra-domain RESERVE message, it just passes it to downstream nodes, because the QNE Ingress node might have retransmitted it 🔁. for Situation S2, the QNE Interior node processes the first incoming intra-domain (refresh) RESERVE message within a refresh period, updates the entry, and forwards it to the downstream nodes 🧚
if only <Session-ID> matches the list, then the QNE Interior node checks the <RSN>. two situations can happen here too 💫
if a rerouting takes place (see Section 5.2.5.2 in [RFC5974]), the <RSN> in the message will be equal to either <RSN + 2> in the stored list (if it is not a tearing RESERVE) or <RSN -1> in the stored list (if it is a tearing RESERVE):
the QNE Interior node will check the <Parameter ID> part 👀
if the <RSN> in the message is equal to <RSN + 2> in the stored list and the <Parameter ID> is PHR_RESOURCE_REQUEST or PHR_REFRESH_UPDATE, then the received intra-domain RESERVE message has to be interpreted and processed as a typical (non-tearing) RESERVE message, which is caused by rerouting (see Section 5.2.5.2 in [RFC5974]) 🛣️
if the <RSN> in the message is equal to <RSN-1> in the stored list and the <Parameter ID> is PHR_RELEASE_REQUEST, then the received intra-domain RESERVE message has to be interpreted and processed as a typical (tearing) RESERVE message, which is caused by rerouting (see Section 5.2.5.2 in [RFC5974]) 👋
if any other situation occurs, then the QNE Interior node does not use or process the local RMD-QSPEC <TMOD-1> parameter carried by the received intra-domain RESERVE message. also the <K> parameter has to be set, see above 🙃
Phase 2: Update Replay Cache Entry
when a QNE Interior node gets an intra-domain RESERVE message, it grabs the matching entry from the cache and compares the values 🔍. if the message is valid, the Interior node updates the <Parameter ID> and <REFRESH-PERIOD> in the list entry ✨
Phase 3: Delete Replay Cache Entry
when a QNE Interior node gets an intra-domain (tear) RESERVE message and an entry in the replay cache can be found, the QNE Interior node deletes that entry after processing the message 🗑️. the Interior node also deletes cache entries if it did not receive an intra-domain (refresh) RESERVE message during the <REFRESH-PERIOD> period with a <Parameter ID> value equal to PHR_REFRESH_UPDATE 🥀
A.9. Example on Matching the Initiator QSPEC to the Local RMD-QSPEC 💖
Okay bestie, this is the QSPEC glow-up 💅 Section 3.4 of [RFC5975] already shows how the QSPEC gets used inside QoS-NSLP, and this example takes it further. We've got a QNI and a QNR talking over an end-to-end QOSM, which we're calling Z-e2e here 🦋. The QNI's side is a wireless access network built on a generation "X" technology, with QoS support as defined by generation "X" 📶. The QNR's side is a wired/fixed access network with its own QoS support 🖥️.
The QNE Edges sit on the boundary of an RMD domain, and the QNE Interior nodes sit inside it 🏠. The Edges can run both the Z-e2e QOSM and the RMD-QOSM, but the Interior nodes can only run the RMD-QOSM 🧚. For a visual, the QNI is a wireless laptop and the QNR is a PC 💻.
|------| |------| |------| |------|
|Z-e2e |<->|Z-e2e |<------------------------->|Z-e2e |<->|Z-e2e |
| QOSM | | QOSM | | QOSM | | QOSM |
| | |------| |-------| |-------| |------| | |
| NSLP | | NSLP |<->| NSLP |<->| NSLP |<->| NSLP | | NSLP |
|Z-e2e | | RMD | | RMD | | RMD | | RMD | | Z-e2e|
| QOSM | | QOSM | | QOSM | | QOSM | | QOSM | | QOSM |
|------| |------| |-------| |-------| |------| |------|
-----------------------------------------------------------------
|------| |------| |-------| |-------| |------| |------|
| NTLP |<->| NTLP |<->| NTLP |<->| NTLP |<->| NTLP |<->| NTLP |
|------| |------| |-------| |-------| |------| |------|
QNI QNE QNE QNE QNE QNR
(End) (Ingress Edge) (Interior) (Interior) (Egress Edge) (End)Figure 29. Example of initiator and local domain QOSM operation
The QNI sets the <QoS Desired> and <QoS Available> QSPEC objects in the initiator QSPEC, then sets <QoS Available> to match <QoS Desired>. In this example, the <Minimum QoS> object is left empty. The QNI fills in QSPEC parameters so its traffic gets treated correctly in the domains down the path. To make sure that treatment holds further down the path, the QNI also puts <PHB Class> into <QoS Desired>. So the QNI ends up with this in the QSPEC:
<QoS Desired> = <TMOD-1> <PHB Class>
<QoS Available> = <TMOD-1> <Path Latency>In this example, the <TMOD-1> parameter encodes the traffic parameters of a VoIP application that uses RTP and the G.711 Codec, see Appendix B in [RFC5975]. Here's the text copied straight from [RFC5975] 🫶:
In the simplest case the Minimum Policed Unit m is the sum of the
IP-, UDP- and RTP- headers + payload. The IP header in the IPv4
case has a size of 20 octets (40 octets if IPv6 is used). The UDP
header has a size of 8 octets and RTP uses a 12 octet header. The
G.711 Codec specifies a bandwidth of 64 kbit/s (8000 octets/s).
Assuming RTP transmits voice datagrams every 20 ms, the payload
for one datagram is 8000 octets/s * 0.02 s = 160 octets.
IPv4+UDP+RTP+payload: m=20+8+12+160 octets = 200 octets
IPv6+UDP+RTP+payload: m=40+8+12+160 octets = 220 octets
The Rate r specifies the amount of octets per second. 50
datagrams are sent per second.
IPv4: r = 50 1/s * m = 10,000 octets/s
IPv6: r = 50 1/s * m = 11,000 octets/s
The bucket size b specifies the maximum burst. In this example, a
burst of 10 packets is used.
IPv4: b = 10 * m = 2000 octets
IPv6: b = 10 * m = 2200 octetsWe're going with IPv4 in this example, so the <TMOD-1> values come out like this:
m = 200 octets
r = 10000 octets/s
b = 2000 octetsThe <Peak Data Rate-1 (p)> and MPS aren't specified above, so we'll just assume:
p = r = 10000 octets/s
MPS = 220 octetsThe <PHB Class> is set so that the Expedited Forwarding (EF) PHB gets used 🚀.
Since <Path Latency> and <QoS Class> aren't vital from the QNI's point of view, it doesn't raise their <M> flags.
Each QNE that supports the Z-e2e QOSM on the path reads and interprets those parameters in the initiator QSPEC 👀.
When an end-to-end RESERVE message shows up at a QNE Ingress node on the RMD domain border, that QNE Ingress can "hide" the initiator end-to-end RESERVE message. That way only the QNE Edges process it, and it skips the intermediate nodes between the Edges of the domain. The Ingress then issues its own local RESERVE message (see Section 6) 💫. For this new local RESERVE message, the QNE Ingress node generates the local RMD-QSPEC.
The RMD-QSPEC for the RMD-QOSM is generated from the original initiator QSPEC, following the procedures in Section 4.5 of [RFC5974] and in Section 6 of this document. The RMD QNE Ingress maps the <TMOD-1> parameters from the original initiator QSPEC into an equivalent <TMOD-1> parameter that represents only the peak bandwidth in the local RMD-QSPEC.
So in this example, the initial <TMOD-1> parameters get mapped into the RMD-QSPEC <TMOD-1> parameters like so. As specified, the RMD-QOSM bandwidth equivalent <TMOD-1> parameter of the RMD-QSPEC should have:
r = p of initial e2e <TMOD-1> parameter
m = large;
b = large;For the RMD-QSPEC <TMOD-1> parameter, we calculate the following values:
r = p of initial e2e <TMOD-1> parameter = 10000 octets/sm is set to large in this example as follows:
m = MPS of initial e2e <TMOD-1> parameter = 220 octetsThe maximum value of b is 250 gigabytes, but in our example that's already quite large 😮. The b parameter tells you how far the data rate can go over the sustainable level for short bursts of time.
To get a large b, we consider that for a certain period of time, the data rate can go above the sustainable level. In our example, that sustainable level is the peak rate (p). So we calculate b as:
b = p * "period of time"For this VoIP example, we can assume that the period of time is 1.5 seconds:
b = 10000 octets/s * 1.5 seconds = 15000 octetsThat makes the local RMD-QSPEC <TMOD-1> values:
r = 10000 octets/s
p = 10000 octets/s
m = 220 octets
b = 15000 octets
MPS = 220 octetsThe bit level format of the RMD-QSPEC is given in Section 4.1. The Initiator/Local QSPEC bit, <I>, is set to "Local" (that's "1") ✅. The <Qspec Proc> gets set like this:
Message Sequence = 0: Sender initiatedObject combination = 0: <QoS Desired> for RESERVE and <QoS Reserved> for RESPONSE
The <QSPEC Version> used by the RMD-QOSM is the default version, which is "0", see [RFC5975]. The <QSPEC Type> value used by the RMD-QOSM is specified in [RFC5975] and is equal to "2" 🎀.
The <Traffic Handling Directives> contains these fields:
<Traffic Handling Directives> = <PHR container> <PDR container>The Per-Hop Reservation container (PHR container) and the Per-Domain Reservation container (PDR container) are specified in Sections 4.1.2 and 4.1.3, respectively. The <PHR container> holds the traffic handling directives for intra-domain communication and reservation. The <PDR container> holds the extra traffic handling directives needed for edge-to-edge communication. The RMD-QOSM <QoS Desired> and <QoS Reserved> are specified in Section 4.1.1.
In the RMD-QOSM, the <QoS Desired> and <QoS Reserved> objects contain these parameters:
<QoS Desired> = <TMOD-1> <PHB Class> <Admission Priority>
<QoS Reserved> = <TMOD-1> <PHB Class> <Admission Priority>The bit format of the <PHB Class> (see [RFC5975] and Figures 4 and 5) and of <Admission Priority> complies with the bit format specified in [RFC5975].
In this example, the RMD-QSPEC <TMOD-1> values are the ones we calculated above. The <PHB Class> represents the EF PHB class. Also, the RMD reservation here is set up without an <Admission Priority> parameter, which is the same as a reservation with an <Admission Priority> of 1 🌸.
The RMD QNE Egress node updates <QoS Available> on behalf of the entire RMD domain if it can. If it can't (because the <M> flag isn't set for <Path Latency>), it raises the parameter-specific "not-supported" flag. That warns the QNR that the final latency value in <QoS Available> is imprecise ⚠️.
In the "Y" access domain, the QNR processes the initiator QSPEC in a similar way to how the QNI processed it in the "X" wireless access domain 🔁.
If the reservation succeeds, the RESERVE request eventually reaches the QNR. (If it failed somewhere, the QNE where it failed would have aborted the RESERVE and sent an error RESPONSE back to the QNI 😬.) If the <RII> was included in the QoS-NSLP message, the QNR generates a positive RESPONSE with the QSPEC objects <QoS Reserved> and <QoS Available>. The parameters in <QoS Reserved> are the same as in <QoS Desired>, with values copied from <QoS Available>. So the QNR includes these QSPEC objects in the RESPONSE message:
<QoS Reserved> = <TMOD-1> <PHB Class>
<QoS Available> = <TMOD-1> <Path Latency>Contributors 🌈
Attila Takacs Ericsson Research Ericsson Hungary Ltd. Laborc 1, Budapest, Hungary, H-1037 EMail: Attila.Takacs@ericsson.com
Andras Csaszar
Ericsson Research
Ericsson Hungary Ltd.
Laborc 1, Budapest, Hungary, H-1037
EMail: Andras.Csaszar@ericsson.comAuthors' Addresses 🫶
Attila Bader
Ericsson Research
Ericsson Hungary Ltd.
Laborc 1, Budapest, Hungary, H-1037
EMail: Attila.Bader@ericsson.comLars Westberg
Ericsson Research
Torshamnsgatan 23
SE-164 80 Stockholm, Sweden
EMail: Lars.Westberg@ericsson.comGeorgios Karagiannis
University of Twente
P.O. Box 217
7500 AE Enschede, The Netherlands
EMail: g.karagiannis@ewi.utwente.nlCornelia Kappler
ck technology concepts
Berlin, Germany
EMail: cornelia.kappler@cktecc.deHannes Tschofenig
Nokia Siemens Networks
Linnoitustie 6
Espoo 02600
Finland
EMail: Hannes.Tschofenig@nsn.com
URI: http://www.tschofenig.priv.atTom Phelan
Sonus Networks
250 Apollo Dr.
Chelmsford, MA 01824 USA
EMail: tphelan@sonusnet.com📦 where this page comes from
- ietf: rfc5977 2010-10
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms