RFC 1819
Internet Stream Protocol Version 2 (ST2) Protocol Specification - Version ST2+
RFC 1819: Internet Stream Protocol Version 2 (ST2) 🎀✨
okay bestie, this is the ST2 spec, and it's giving main character energy 💅. It's RFC 1819, written by L. Delgrossi and L. Berger (they're the editors 🫶), and it came out in August 1995 from the ST2 Working Group. It's an Experimental doc, so think "let's test this" vibes, not "this is the official Internet standard" 🧪. It obsoletes RFC 1190 and IEN 119, which means it replaces both of those older docs 💀. The title is Internet Stream Protocol Version 2 (ST2) 🦋, and the revised version in here is called ST2+ 👑.
Status of this Memo 📜
Okay so, this memo defines an Experimental Protocol for the Internet community 🧪. It is not an Internet standard of any kind, just so we're clear 🙈. They're literally asking for feedback and suggestions for improvement, so if you've got ideas, tell them 💖. And the good news is that distribution of this memo is unlimited, so share it with everyone 🌈.
IESG NOTE 🎀
So this document is a revision of RFC1190 💅. The whole goal was to clarify things, simplify them, and remove errors from RFC1190 so that different implementations could actually work together ✨.
NOTE WELL: Neither the version of the protocol described in this document nor the previous version is an Internet Standard or under consideration for that status. Not me saying it, the IESG is 😭.
Also, the field has moved on a lot since the first version came out 🚀. Readers should know that other approaches to the resource reservation problem are being worked on inside the IETF right now 🍓, so ST2 isn't the only game in town.
Abstract 🦋
Here's the vibe check: this memo is a revised spec for the Internet STream Protocol Version 2 (ST2) 🌸. ST2 is an experimental resource reservation protocol, and its whole thing is providing end-to-end real-time guarantees across an internet ⏱️. It lets applications build multi-destination simplex data streams, meaning one sender can feed several receivers at once, each with a desired quality of service 🧚. The revised version in this memo is called ST2+ 👑.
This spec came out of the STream Protocol Working Group of the Internet Engineering Task Force 💜.
Table of Contents 💖
1 Introduction 6
1.1 What is ST2? 6
1.2 ST2 and IP 8
1.3 Protocol History 8
1.3.1 RFC1190 ST and ST2+ Major Differences 9
1.4 Supporting Modules for ST2 10
1.4.1 Data Transfer Protocol 11
1.4.2 Setup Protocol 11
1.4.3 Flow Specification 11
1.4.4 Routing Function 12
1.4.5 Local Resource Manager 12
1.5 ST2 Basic Concepts 15
1.5.1 Streams 16
1.5.2 Data Transmission 16
1.5.3 Flow Specification 17
1.6 Outline of This Document 19
2 ST2 User Service Description 19
2.1 Stream Operations and Primitive Functions 19
2.2 State Diagrams 21
2.3 State Transition Tables 25
3 The ST2 Data Transfer Protocol 26
3.1 Data Transfer with ST 26
3.2 ST Protocol Functions 27
3.2.1 Stream Identification 27
3.2.2 Packet Discarding based on Data Priority 27
4 SCMP Functional Description 28
4.1 Types of Streams 29
4.1.1 Stream Building 30
4.1.2 Knowledge of Receivers 30
4.2 Control PDUs 31
4.3 SCMP Reliability 32
4.4 Stream Options 33
4.4.1 No Recovery 33
4.4.2 Join Authorization Level 34
4.4.3 Record Route 34
4.4.4 User Data 35
4.5 Stream Setup 35
4.5.1 Information from the Application 35
4.5.2 Initial Setup at the Origin 35
4.5.2.1 Invoking the Routing Function 36
4.5.2.2 Reserving Resources 36
4.5.3 Sending CONNECT Messages 37
4.5.3.1 Empty Target List 37
4.5.4 CONNECT Processing by an Intermediate ST agent 37
4.5.5 CONNECT Processing at the Targets 38
4.5.6 ACCEPT Processing by an Intermediate ST agent 38
4.5.7 ACCEPT Processing by the Origin 39
4.5.8 REFUSE Processing by the Intermediate ST agent 39
4.5.9 REFUSE Processing by the Origin 39
4.5.10 Other Functions during Stream Setup 40
4.6 Modifying an Existing Stream 40
4.6.1 The Origin Adding New Targets 41
4.6.2 The Origin Removing a Target 41
4.6.3 A Target Joining a Stream 42
4.6.3.1 Intermediate Agent (Router) as Origin 43
4.6.4 A Target Deleting Itself 43
4.6.5 Changing a Stream's FlowSpec 44
4.7 Stream Tear Down 45
5 Exceptional Cases 45
5.1 Long ST Messages 45
5.1.1 Handling of Long Data Packets 45
5.1.2 Handling of Long Control Packets 46
5.2 Timeout Failures 47
5.2.1 Failure due to ACCEPT Acknowledgment Timeout 47
5.2.2 Failure due to CHANGE Acknowledgment Timeout 47
5.2.3 Failure due to CHANGE Response Timeout 48
5.2.4 Failure due to CONNECT Acknowledgment Timeout 48
5.2.5 Failure due to CONNECT Response Timeout 48
5.2.6 Failure due to DISCONNECT Acknowledgment Timeout 48
5.2.7 Failure due to JOIN Acknowledgment Timeout 48
5.2.8 Failure due to JOIN Response Timeout 49
5.2.9 Failure due to JOIN-REJECT Acknowledgment Timeout 49
5.2.10 Failure due to NOTIFY Acknowledgment Timeout 49
5.2.11 Failure due to REFUSE Acknowledgment Timeout 49
5.2.12 Failure due to STATUS Response Timeout 49
5.3 Setup Failures due to Routing Failures 50
5.3.1 Path Convergence 50
5.3.2 Other Cases 51
5.4 Problems due to Routing Inconsistency 52
5.5 Problems in Reserving Resources 53
5.5.1 Mismatched FlowSpecs 53
5.5.2 Unknown FlowSpec Version 53
5.5.3 LRM Unable to Process FlowSpec 53
5.5.4 Insufficient Resources 53
5.6 Problems Caused by CHANGE Messages 54
5.7 Unknown Targets in DISCONNECT and CHANGE 55
6 Failure Detection and Recovery 55
6.1 Failure Detection 55
6.1.1 Network Failures 56
6.1.2 Detecting ST Agents Failures 56
6.2 Failure Recovery 58
6.2.1 Problems in Stream Recovery 60
6.3 Stream Preemption 62
7 A Group of Streams 63
7.1 Basic Group Relationships 63
7.1.1 Bandwidth Sharing 63
7.1.2 Fate Sharing 64
7.1.3 Route Sharing 65
7.1.4 Subnet Resources Sharing 65
7.2 Relationships Orthogonality 65
8 Ancillary Functions 66
8.1 Stream ID Generation 66
8.2 Group Name Generator 66
8.3 Checksum Computation 67
8.4 Neighbor ST Agent Identification and
Information Collection 67
8.5 Round Trip Time Estimation 68
8.6 Network MTU Discovery 68
8.7 IP Encapsulation of ST 69
8.8 IP Multicasting 70
9 The ST2+ Flow Specification 71
9.1 FlowSpec Version #0 - (Null FlowSpec) 72
9.2 FlowSpec Version #7 - ST2+ FlowSpec 72
9.2.1 QoS Classes 73
9.2.2 Precedence 74
9.2.3 Maximum Data Size 74
9.2.4 Message Rate 74
9.2.5 Delay and Delay Jitter 74
9.2.6 ST2+ FlowSpec Format 75
10 ST2 Protocol Data Units Specification 77
10.1 Data PDU 77
10.1.1 ST Data Packets 78
10.2 Control PDUs 78
10.3 Common SCMP Elements 80
10.3.1 FlowSpec 80
10.3.2 Group 81
10.3.3 MulticastAddress 82
10.3.4 Origin 82
10.3.5 RecordRoute 83
10.3.6 Target and TargetList 84
10.3.7 UserData 85
10.3.8 Handling of Undefined Parameters 86
10.4 ST Control Message PDUs 86
10.4.1 ACCEPT 86
10.4.2 ACK 88
10.4.3 CHANGE 89
10.4.4 CONNECT 89
10.4.5 DISCONNECT 92
10.4.6 ERROR 93
10.4.7 HELLO 94
10.4.8 JOIN 95
10.4.9 JOIN-REJECT 96
10.4.10 NOTIFY 97
10.4.11 REFUSE 98
10.4.12 STATUS 100
10.4.13 STATUS-RESPONSE 100
10.5 Suggested Protocol Constants 101
10.5.1 SCMP Messages 102
10.5.2 SCMP Parameters 102
10.5.3 ReasonCode 102
10.5.4 Timeouts and Other Constants 104
10.6 Data Notations 105
11 References 106
12 Security Considerations 108
13 Acknowledgments and Authors' Addresses 1081. Introduction ✨
The body text for this section didn't make it into this part, bestie 🥺 It only says "RFC>>>", so I'm not going to make up any of the Introduction. Send the actual text and I'll pick right up from 1.1 What is ST2? 🦋
1.1. What is ST2? 🦋
okay so ST2 (the Internet Stream Protocol, Version 2) is an experimental connection-oriented internetworking protocol that works at the same layer as connectionless IP 💅. it was built to deliver data streams efficiently to single or multiple destinations in apps that need guaranteed quality of service ✨. ST2 is part of the IP protocol family, and it's an adjunct to IP, not a replacement, so think sidekick, not main character 👑. the big app areas are real-time transport of multimedia data (like digital audio and video packet streams) and distributed simulation/gaming across internets 🌈.
ST2 can reserve bandwidth for real-time streams across network routes 🔥. that reservation, plus the right network access and packet scheduling mechanisms in every node running the protocol, guarantees a well-defined Quality of Service (QoS) to ST2 apps. it makes sure real-time packets are delivered within their deadlines, meaning the time they need to be presented ⏰. this gives the smooth delivery that time-critical apps need, which best-effort IP communication typically can't provide 💀.
DATA PATH CONTROL PATH
========= ============
Upper +------------------+ +---------+
Layer | Application data | | Control |
+------------------+ +---------+
| |
| V
| +-------------------+
SCMP | | SCMP | |
| +-------------------+
| |
V V
+-----------------------+ +------------------------+
ST | ST | | | ST | | |
+-----------------------+ +------------------------+
D-bit=1 D-bit=0
Figure 1: ST2 Data and Control Pathokay so ST2 is actually two protocols, just like IP 🎀. there's ST for data transport, and there's SCMP (the Stream Control Message Protocol) for all the control functions. ST is super simple and has only one PDU format, built for fast, efficient data forwarding so communication delays stay low 🧚. SCMP is more complex than IP's ICMP. same as with ICMP and IP, SCMP packets ride inside ST packets, as shown in Figure 1 🍓.
+--------------------+
| Conference Control |
+--------------------+
+-------+ +-------+ |
| Video | | Voice | | +-----+ +------+ +-----+ +-----+ Application
| Appl | | Appl | | | SNMP| |Telnet| | FTP | ... | | Layer
+-------+ +-------+ | +-----+ +------+ +-----+ +-----+
| | | | | | |
V V | | | | | ------------
+-----+ +-----+ | | | | |
| PVP | | NVP | | | | | |
+-----+ +-----+ + | | | |
| \ | \ \ | | | |
| +-----|--+-----+ | | | |
| Appl.|control V V V V V
| ST data | +-----+ +-------+ +-----+
| & control| | UDP | | TCP | ... | RTP | Transport
| | +-----+ +-------+ +-----+ Layer
| /| / | \ / / | / /|
|\ / | +------+--|--\-----+-/--|--- ... -+ / |
| \ / | | | \ / | / |
| \ / | | | \ +----|--- ... -+ | -----------
| \ / | | | \ / | |
| V | | | V | |
| +------+ | | | +------+ | +------+ |
| | SCMP | | | | | ICMP | | | IGMP | | Internet
| +------+ | | | +------+ | +------+ | Layer
| | | | | | | | |
V V V V V V V V V
+-----------------+ +-----------------------------------+
| STream protocol |->| Internet Protocol |
+-----------------+ +-----------------------------------+
| \ / |
| \ / |
| X | ------------
| / \ |
| / \ |
VV VV
+----------------+ +----------------+
| (Sub-) Network |...| (Sub-) Network | (Sub-)Network
| Protocol | | Protocol | Layer
+----------------+ +----------------+
Figure 2. Protocol Relationships1.2. ST2 and IP 🌸
ST2 is designed to coexist with IP on each node 🫶. a typical distributed multimedia app would use both: IP for traditional data and control info, and ST2 for real-time data. IP is usually accessed from TCP or UDP, while ST2 will be accessed through new end-to-end real-time protocols. where ST2 sits next to the other protocols in the Internet family is shown in Figure 2 🗺️.
both ST2 and IP use the same addressing schemes to identify different hosts 💜. ST2 and IP packets differ in the first four bits, which hold the internetwork protocol version number: number 5 is reserved for ST2 (IP itself is version number 4). as a network layer protocol, like IP, ST2 works independently of its underlying subnets. existing implementations use ARP for address resolution and use the same Layer 2 SAPs as IP 🦋.
ST2 messages can also be encapsulated in IP packets, which is a special function 🎀. Figure 2 shows this as a link between ST2 and IP. the link lets ST2 messages pass through routers that don't run ST2. resource management is typically not available for these IP route segments, so IP encapsulation is only suggested for parts of the network that don't form a system bottleneck 🙈.
in Figure 2, RTP is shown as an example of a transport layer on top of ST2 🧁. others include the Packet Video Protocol (PVP) [Cole81], the Network Voice Protocol (NVP) [Cohe81], and more, like the Heidelberg Transport Protocol (HeiTP) [DHHS92] ✨.
1.3. Protocol History 🕰️
the first version of ST came out in the late 1970's and was used all through the 1980's for experimental transmission of voice, video, and distributed simulation 🎞️. the experience from those apps led to the revised protocol, ST2. the revision extends the original protocol to make it more complete and better suited to emerging multimedia environments. the spec for this version is in Internet RFC 1190, which was published in October 1990 [RFC1190] 📜.
more and more commercial distributed multimedia apps were in development, and people were getting increasingly unhappy with audio and video quality over IP in the MBONE, so interest in ST2 grew over the last few years 📈. companies had products out that used the protocol. the BERKOM MMTS project of the German PTT [DeAl92] uses ST2 as its core protocol for multimedia teleservices like conferencing and mailing 📬. on top of that, ST2 implementations were available for Digital Equipment, IBM, NeXT, Macintosh, PC, Silicon Graphics, and Sun platforms 💻.
in 1993, the IETF started a new working group on ST2 as part of ongoing efforts to develop protocols that address resource reservation issues 🧠. the group's mission was to clean up the existing protocol specification so existing and emerging implementations interoperate better 🤝. they also wanted an updated experimental protocol spec that reflected what was learned from existing ST2 implementations and apps 🔍. that work led to the ST2+ protocol specification contained in this document 👑.
1.3.1. RFC1190 ST and ST2+ Major Differences 💜
the protocol changes from RFC1190 were driven by simplifying and clarifying the protocol, and by writing down extensions that existing implementations already used 📝. this section lists the major differences, and it's probably only interesting to people who already know RFC1190 🙈. the major differences between the versions are:
- Elimination of "Hop IDentifiers" or HIDs. HIDs added a lot of complexity to the protocol and were found to be a major impediment to interoperability. HIDs have been replaced by globally unique identifiers called "Stream IDentifiers" or SIDs 🆔.
- Elimination of a number of stream options. A number of options were found to not be used by any implementation, or were thought to add more complexity than value. These options were removed. Removed options include: point-to-point, full-duplex, reverse charge, and source route ✂️.
- Elimination of the concept of "subset" implementations. RFC1190 permitted subset implementations, to allow for easy implementation and experimentation. This led to interoperability problems. Agents implementing the protocol specified in this document MUST implement the full protocol. A number of the protocol functions are best-effort. It is expected that some implementations will make more effort than others in satisfying particular protocol requests 🚫.
- Clarification of the capability of targets to request to join a stream. RFC1190 can be interpreted to support target requests, but most implementors did not understand this and did not add support for this capability. The lack of this capability was found to be a significant limitation in the ability to scale the number of participants in a single ST stream. This clarification is based on work done by IBM Heidelberg 🙋.
- Separation of functions between ST and supporting modules. An effort was made to improve the separation of functions provided by ST and those provided by other modules. This is reflected in reorganization of some text and some PDU formats. ST was also made FlowSpec independent, although it does define a FlowSpec for testing and interoperability purposes 🧩.
- General reorganization and re-write of the specification. This document has been organized with the goal of improved readability and clarity. Some sections have been added, and an effort was made to improve the introduction of concepts 🌷.
1.4 Supporting Modules for ST2 ✨
okay bestie, ST2 is one piece of a bigger mosaic 🧩 so in this part we look at the whole communication setup and where ST2 fits next to the modules that support it 💅
ST2 runs a two-step communication model 🫶 Step one builds the real-time channels that the data will use later. This is called stream setup, and it covers picking the routes to the destinations and reserving the matching resources along the way. Step two is the actual sending of data over those already-built streams. This is called data transfer. Stream setup doesn't have to finish in real time, but data transfer has strict real-time requirements, so it's extremely serious business 😭 The architecture used to describe this model has these pieces:
- a data transfer protocol for sending real-time data over the established streams,
- a setup protocol for establishing real-time streams based on the flow specification,
- a flow specification for expressing the user's real-time requirements,
- a routing function for selecting routes in the Internet,
- a local resource manager for handling the resources involved in the communication.
This document defines a data protocol (ST), a setup protocol (SCMP), and a flow specification (ST2+ FlowSpec) 🎀 It does not define a routing function or a local resource manager, but ST2 assumes they exist, so they're definitely part of the squad 🌸
Other architectures are possible too. See [RFC1633] for an example alternative architecture that could be used when implementing ST2 🦋
1.4.1 Data Transfer Protocol
The data transfer protocol defines the format of the data packets that belong to a stream 📦 Those packets get delivered to the targets along the stream paths that the setup protocol already set up, and they arrive with the quality of service that's tied to the stream ✨
Every data packet carries a globally unique stream identifier that says which stream it belongs to 🆔 The setup protocol knows that same identifier and uses it while the stream is being established. The data transfer protocol for ST2, known simply as ST, is completely defined by this document 💜
1.4.2 Setup Protocol
The setup protocol is in charge of establishing, maintaining, and releasing real-time streams 🧚 It leans on the routing function to pick the paths from the source to the destinations. At each host or router on those paths, it hands the flow specification for the stream to the local resource manager, and that's what makes the resource managers reserve the right resources for the stream 🔥 The setup protocol for ST2 is called Stream Control Message Protocol, or SCMP, and it's completely defined by this document 💅
1.4.3 Flow Specification
The flow specification is a data structure that holds the ST2 applications' QoS requirements 📋 At each host or router, the local resource manager uses it to handle resources so those requirements actually get met. Getting the flow specification out to every resource manager along the communication paths is the job of the setup protocol. But the setup protocol can't see inside it. It just carries it along, and any operations on it, including updating internal fields and comparing flow specifications, are done by the resource managers 🫶
This document defines one specific flow specification format so that ST2 implementations can interoperate 🌈 That format is meant to support a flow with a single transmission rate for all destinations in the stream. Implementations may support more than one flow specification format, and there are ways to add new formats as they get defined in the future 🦋 The catch is that the format has to stay consistent across the whole stream. You can't use different flow specification formats for different parts of the same stream, so no mixing and matching 🙅♀️
1.4.4 Routing Function
The routing function is an external unicast route generation capability 🗺️ It gives the setup protocol the path to reach each of the desired destinations. It gets called on a hop-by-hop basis and provides next-hop information. Once a route is picked by the routing function, it sticks around for the whole stream lifetime 💖 The routing function may try to optimize based on the number of targets, the requested resources, or the use of local network multicast or bandwidth capabilities. It can also be based on simple connectivity information, which is giving low-key minimalist vibes ✨
The setup protocol doesn't necessarily know what criteria the routing function used to pick routes. It works with any routing function algorithm 🤝 Which algorithm gets adopted is a local matter at each host or router, so different hosts and routers may use different algorithms. The interface between the setup protocol and the routing function is also a local matter, which is why this document doesn't specify it 🌸
This version of ST does not support source routing 🙅♀️ It does support route recording 📝 It also includes provisions that let you identify ST-capable neighbors. Identifying remote ST hosts and routers isn't specifically addressed here 🫠
1.4.5 Local Resource Manager
At each host or router that a stream goes through, the Local Resource Manager (LRM) handles the local resources 🛠️ The LRM knows which resources are on the system and what capacity they can provide. Those resources include:
- CPUs on end systems and routers, to execute the application and protocol software,
- main memory space for this software (as in all real-time systems, code should be pinned in main memory, as swapping it out would have detrimental effects on system performance),
- buffer space to store the data, e.g., communication packets, passing through the nodes,
- network adapters, and
- transmission networks between the nodes. Networks may be as simple as point-to-point links or as complex as switched networks such as Frame Relay and ATM networks.
When a stream is being set up or modified, the setup protocol hands the LRM the flow specification associated with that stream 📬 For each resource it handles, the LRM is expected to perform these functions:
- Stream Admission Control: it checks whether, given the flow specification, there are sufficient resources left to handle the new data stream. If the available resources are insufficient, the new data stream must be rejected. 🚫
- QoS Computation: it calculates the best possible performance the resource can provide for the new data stream under the current traffic conditions, e.g., throughput and delay values are computed. 📊
- Resource Reservation: it reserves the resource capacities required to meet the desired QoS. 🔒
During data transfer, the LRM is responsible for:
- QoS Enforcement: it enforces the QoS requirements by appropriately scheduling resource access. For example, data packets from an application with a short guaranteed delay must be served prior to data from an application with a less strict delay bound. ⏰
The LRM may also provide these additional functions:
- Data Regulation: to smooth a stream's data traffic, e.g., as with the leaky bucket algorithm. 🌊
- Policing: to prevent applications from exceeding their negotiated QoS, e.g., to send data at a higher rate than indicated in the flow specification. 👮♀️
- Stream Preemption: to free up resources for other streams with higher priority or importance. 👑
The strategies the LRMs use to handle resources depend on the resource and may differ at every host or router 🎨 What does need to match is that all LRMs have the same understanding of the flow specification 🫶 The interface between the setup protocol and the LRM is a local matter at every host, so this document doesn't specify it. One example of an LRM is the Heidelberg Resource Administration Technique (HeiRAT) [VoHN93] 🌟
The LRM is also assumed to provide functions for comparing flow specifications, so it can decide whether one flow specification needs a greater, equal, or smaller amount of resource capacity to be reserved 🧮
1.5 ST2 Basic Concepts 🎀
The next sections introduce some of the fundamental ST2 concepts at an beginner-friendly level, including streams, data transfer, and flow specification 💜
Hosts Connections... : ...and Streams
==================== : ==============
data Origin : Origin
packets +-----------+ : +----+
+----|Application| : | |
| |-----------| : +----+
+--->| ST Agent | : | |
+-----------+ : | |
| : | |
V : | |
+-------------+ : | |
| | : | |
+-------------| Network A | : +-------+ +--+
| | | : | |
| +-------------+ : | Target 2|
| | Target 2 : | & Router|
| Target 1 | and Router : | |
| +-----------+ | +-----------+ : V V
| |Application|<-+ | |Application|<-+ : +----+ +----+
| |-----------| | | |-----------| | : | | | |
+->| ST Agent |--+ +->| ST Agent |--+ : +----+ +----+
+-----------+ +-----------+ :Target 1 | |
| : | |
V : | |
+-------------+ : | |
| | : | |
+-------------| Network B | : +-----+ |
| | | : | |
| +-------------+ : | |
| Target 3 | Target 4 : | |
| +-----------+ | +-----------+ : V V
| |Application|<-+ | |Application|<-+ : +----+ +----+
| |-----------| | | |-----------| | : | | | |
+->| ST Agent |--+ +->| ST Agent |--+ : +----+ +----+
+-----------+ +-----------+ : Target 3 Target 4
:
Figure 3: The Stream Concept1.5.1. Streams 🦋
okay so streams are literally the core of ST2 💖 they get set up between a sending origin and one or more receiving targets, and the whole thing looks like a routing tree 🌳 streams only flow one way, from the origin out to the targets. each node in the tree is an ST agent (an entity running the ST2 protocol), and each link between nodes is called a hop 🔗 any node sitting in the middle is an intermediate agent, aka a router. and an agent can be an origin, a target, an intermediate, or any mix of those 👑
Figure 3 shows a stream going from an origin to four targets, and the ST agent on Target 2 is also acting as an intermediate agent. let's use that Target 2/Router node to learn the basic ST2 vocab 🧚 the direction going from this node toward Target 3 and 4 is called downstream, and the direction back toward the Origin node is upstream. ST agents that are one hop away from a given node are called previous-hops when you look upstream, and next-hops when you look downstream ✨
streams are kept alive with SCMP messages 💅 the main ones are CONNECT and ACCEPT to build a stream, DISCONNECT and REFUSE to close one, CHANGE to modify the quality of service on a stream, and JOIN to ask to be added to a stream 🫶
each ST agent keeps state info about the streams flowing through it, and it can actively gather and spread that info around 🧠 it can spot failed neighbor ST agents through periodic HELLO message exchanges 💀 it can also ask other ST agents about a particular stream using a STATUS message, and those agents send back a STATUS-RESPONSE message. NOTIFY messages are for telling other ST agents about big events 📣
ST2 has so much stream management going on it's unreal 😭 streams can be grouped together to cut down on allocated resources, or so they all get handled the same way when something fails. like, during audio conferences only a few people can talk at once, so with the group mechanism you only reserve resources for a portion of the audio streams in the group 🎧 and with that same concept, a whole group of related audio and video streams can get dropped if one of them is preempted 🌈
1.5.2. Data Transmission 🌸
data transfer in ST2 is simplex in the downstream direction ✨ data transport through streams is super simple. ST2 only sticks a small header in front of the user data 💜 that header holds a protocol identification (so it can tell ST2 apart from IP packets), an ST2 version number, a priority field (showing how important streams are relative to each other when there's a conflict), a length counter, a stream identification, and a checksum. all of that together makes a 12-byte header 🎀
efficiency is also a big deal here, so agents avoid fragmentation and reassembly altogether 🙈 stream establishment gives you a maximum message size for data packets on a stream. that max size gets passed up to the upper layers so they can make data packets of a suitable size for ST2 💅
sending to multiple next-hops can get even more efficient with MAC Layer multicast, when it's available 🦋 if a subnet supports multicast, one multicast packet is enough to reach all the next-hops connected to that subnet. this leads to a significant reduction in the bandwidth a stream needs 🔥 if multicast isn't provided, separate packets have to be sent to each next-hop 😤
ok so, since ST2 relies on reservation, it doesn't have the error correction features for data exchange that you find in TCP 🙅♀️ real-time data like digital audio and video is assumed to only need partially correct delivery. in lots of cases, retransmitted packets would show up too late to meet real-time delivery requirements ⏰ and depending on the data encoding and the application, a small number of errors in stream data is acceptable 🤷♀️ if you do need reliability, layers on top of ST2 can provide it when needed 💖
1.5.3. Flow Specification 💎
as part of setting up a connection, SCMP handles negotiating quality-of-service parameters for a stream 🎀 in ST2 terms, those parameters form a flow specification (FlowSpec) that's tied to the stream. different versions of FlowSpecs exist, see [RFC1190], [DHHS92] and [RFC1363], and you can tell them apart by version number 🔢 they usually contain things like average and maximum throughput, end-to-end delay, and delay variance for a stream 📊 SCMP itself only provides the mechanism for relaying the quality-of-service parameters, that's it 🤍
three kinds of entities take part in the quality-of-service negotiation 👑 there are the application entities on the origin and target sites (the service users), the ST agents, and the local resource managers (LRM). the origin application supplies the initial FlowSpec asking for a particular service quality 💅 each ST agent that gets the FlowSpec as part of a connection establishment message hands it over to its local resource manager 🫶 ST2 doesn't decide how resource managers make reservations or how resources get scheduled based on those reservations, but ST2 totally assumes those mechanisms are there as its foundation 🧱
check out the FlowSpec negotiation procedure in Figure 4 🦋 depending on how its local reservations go, the LRM updates the FlowSpec fields and hands the FlowSpec back to the ST agent, which passes it downstream as part of the connection message. eventually the FlowSpec reaches the application at the target, which may base its accept/reject decision for establishing the connection on it, and may also modify the FlowSpec itself 🎯 if a target accepts the connection, the (possibly modified) FlowSpec gets propagated back to the origin, which can then calculate an overall service quality for all targets 📈 the application entity at the origin may later request a CHANGE to adjust reservations 🔄
Origin Router Target 1
+------+ 1a +------+ 1b +------+
| |-------------->| |------------->| |
+------+ +------+ +------+
^ | ^ |
| | | 2 |
| | +------------------------------------------+
+ +
+-------------+ \ \ +-------------+ +-------------+
|Max Delay: 12| \ \ |Max Delay: 12| |Max Delay: 12|
|-------------| \ \ |-------------| |-------------|
|Min Delay: 2| \ \ |Min Delay: 5| |Min Delay: 9|
|-------------| \ \ |-------------| |-------------|
|Max Size:4096| + + |Max Size:2048| |Max Size:2048|
+-------------+ | | +-------------+ +-------------+
FlowSpec | | 1
| +---------------+
| |
| V
2 | +------+
+---------------| |
+------+
Target 2
+-------------+
|Max Delay: 12|
|-------------|
|Min Delay: 4|
|-------------|
|Max Size:4096|
+-------------+
Figure 4: Quality-of-Service Negotiation with FlowSpecs1.6. Outline of This Document 🗺️
this document has the spec for the ST2+ version of the ST2 protocol 💜 for the rest of the document, whenever "ST" or "ST2" shows up, they both mean the ST2+ version of ST2 ✨
here's how the document is organized 🌈
- Section 2 describes the ST2 user service from an application point of view.
- Section 3 illustrates the ST2 data transfer protocol, ST.
- Section 4 through Section 8 specify the ST2 setup protocol, SCMP.
- the ST2 flow specification is presented in Section 9.
- the formats of protocol elements and PDUs are defined in Section 10.
2. ST2 User Service Description ✨
this section describes the ST user service from the high-level point of view of an application 🎀 it defines the ST stream operations and primitive functions, and it specifies which operations on streams can be invoked by the applications built on top of ST, and when the ST primitive functions can legally be executed 💅 just a heads up: the ST primitives presented here don't specify an API. they're only here to illustrate the service model for ST 🫶
2.1. Stream Operations and Primitive Functions ✨
okay so, an ST application at the origin can create, expand, reduce, change, send data to, and delete a stream 💖 expanding means adding new targets, reducing means dropping some of the current targets, and changing means modifying the quality of service 🎀
an ST application at the target can join a stream, receive data from it, and leave it 🦋 all of that boils down to these stream operations:
OPEN: create new stream [origin],CLOSE: delete stream [origin],ADD: expand stream, i.e., add new targets to it [origin],DROP: reduce stream, i.e., drop targets from it [origin],JOIN: join a stream [target],LEAVE: leave a stream [target],DATA: send data through stream [origin],CHG: change a stream's QoS [origin],
each stream operation can need several primitive functions to finish 🧚 for example, to open a stream, the sender first issues a request and the receivers get an indication. then each receiver can accept or refuse, and the matching indications show up back at the sender 🌸 the single-receiver version is in Figure 5 below 🙈
Sender Network Receiver
| | |
OPEN.req | | |
|-----------------> | |
| |-----------------> |
| | | OPEN.ind
| | | OPEN.accept
| |<----------------- |
|<----------------- | |
OPEN.accept-ind | | |
| | |
Figure 5: Primitives for the OPEN Stream OperationTable 1 is the ST service primitive functions for each stream operation 💅 the "O/T" column says whether the primitive runs at the origin (O) or the target (T) 👑
+===================================================+
|Primitive | Descriptive |O/T|
|===================================================|
|OPEN.req | open a stream | O |
|OPEN.ind | connection request indication | T |
|OPEN.accept | accept stream | T |
|OPEN.refuse | refuse stream | T |
|OPEN.accept-ind| connection accept indication | O |
|OPEN.refuse-ind| connection refuse indication | O |
|ADD.req | add targets to stream | O |
|ADD.ind | add request indication | T |
|ADD.accept | accept stream | T |
|ADD.refuse | refuse stream | T |
|ADD.accept-ind | add accept indication | O |
|ADD.refuse-ind | add refuse indication | O |
|JOIN.req | join a stream | T |
|JOIN.ind | join request indication | O |
|JOIN.reject | reject a join | O |
|JOIN.reject-ind| join reject indication | T |
|DATA.req | send data | O |
|DATA.ind | receive data indication | T |
|CHG.req | change stream QoS | O |
|CHG.ind | change request indication | T |
|CHG.accept | accept change | T |
|CHG.refuse | refuse change | T |
|CHG.accept-ind | change accept indication | O |
|CHG.refuse-ind | change refuse indication | O |
|DROP.req | drop targets | O |
|DROP.ind | disconnect indication | T |
|LEAVE.req | leave stream | T |
|LEAVE.ind | leave stream indication | O |
|CLOSE.req | close stream | O |
|CLOSE.ind | close stream indication | T |
+---------------------------------------------------+
Table 1: ST Primitives2.2. State Diagrams 🌈
it's not enough to just list the ST stream operations 🙃 we also need to say when they're allowed to run, so a set of states gets introduced and the transitions between them get specified 💜 states are defined per single stream, and each operation can only run from an appropriate state ✨
an ST agent can be in one of these states for a given ST stream:
IDLE: the stream has not been created yet.PENDING: the stream is in the process of being established.ACTIVE: the stream is established and active.ADDING: the stream is established. A stream expansion is underway.CHGING: the stream is established. A stream change is underway.
previous experience with ST led to limits on which stream operations can run at the same time 🔥 here they are:
- only one
ADDorCHGcan be processed at a time 🎀 if anADDorCHGis already underway, further requests get queued by the ST agent and only handled after the previous operation finishes. this also covers two requests of the same kind in a row, like twoADDs or twoCHGs. the second one doesn't run until the first one is done 💅 - deleting a stream, leaving a stream, or dropping targets is only possible after stream establishment is complete 🫶 a stream counts as established once all the next-hops of the origin have either accepted or refused it. heads up: a stream refuse is automatically forced after a timeout if no reply comes from a next-hop 😭
- an ST agent only forwards data along paths that are already established to the targets (see also Section 3.1) 🧚 a path counts as established once the next-hop on it has explicitly accepted the stream. that means the target and every other intermediate ST agent are ready to handle the incoming data packets. no way an ST agent forwards data to a next-hop that hasn't explicitly accepted the stream 🙅♀️ so to make sure every target gets the data, an application should send data only after all the paths are established, meaning the stream is established 🌸
- data can be sent from the
CHGINGandADDINGstates 🦋 while sending fromCHGING, the QoS for the targets affected by the change should be assumed to be the more restrictive QoS 💖 when sending fromADDING, the targets that get the data include at least all the targets that were already part of the stream when theADDoperation was invoked 👑
these rules mean ST agents have to queue incoming requests whenever the current state doesn't let them process a request right away 🍓 to keep the semantics intact, agents have to keep the order of requests, which means FIFO queuing ⭐ the one exception: a CLOSE request at the origin and a LEAVE request at the target can be processed immediately. in those cases the queue gets deleted, and requests still in the queue might never be processed 💀
the state diagrams below define the ST service 🎀 there's a separate diagram for the origin and one for the targets 🦋
the symbol (a/r)* means all targets in the target list have explicitly accepted or refused the stream, or the refuse was forced after a timeout ✨ if the target list is empty (contains no targets), the (a/r)* condition is immediately satisfied, so the empty stream gets created and the state ESTBL is entered 🫧
quick note: the separate OPEN and ADD primitives at the target are only conceptual. the target can't actually tell an OPEN from an ADD, which is why Figure 7 and Table 3 use the notation OPEN/ADD 🙈
+------------+
| |<-------------------+
+---------->| IDLE |-------------+ |
| | | OPEN.req | |
| +------------+ | |
CLOSE.req | CLOSE.req ^ ^ CLOSE.req V | CLOSE.req
| | | +---------+ |
| | | | PENDING |-|-+ JOIN.reject
| | -------------| |<|-+
| JOIN.reject | +---------+ |
| DROP.req +----------+ | |
| +-----| | | |
| | | ESTDL | OPEN.(a/r)* | |
| +---->| |<------------+ |
| +----------+ |
| | ^ | ^ |
| | | | | |
+----------+ CHG.req| | | | Add.(a/r)* +----------+
| |<-------+ | | +-------------- | |
| CHGING | | | | ADDING |
| |-----------+ +----------------->| |
+----------+ CHG.(a/r)* JOIN.ind +----------+
| ^ ADD.req | ^
| | | |
+---+ +---+
DROP.req DROP.req
JOIN.reject JOIN.reject
Figure 6: ST Service at the Origin +--------+
| |-----------------------+
| IDLE | |
| |<---+ | OPEN/ADD.ind
+--------+ | CLOSE.ind | JOIN.req
^ | OPEN/ADD.refuse |
| | JOIN.refect-ind |
CLOSE.ind | | V
DROP.ind | | +---------+
LEAVE.req | +-------------| |
| | PENDING |
+-------+ | |
| | +---------+
| ESTBL | OPEN/ADD.accept |
| |<-----------------------+
+-------+
Figure 7: ST Service at the Target2.3. State Transition Tables 📊
Table 2 and Table 3 spell out which primitives can be processed from which states, plus the state transitions that can happen 🎀 here's where the table starts:
+======================================================================+
|Primitive |IDLE| PENDING | ESTBL | CHGING | ADDING ||======================================================================|
|OPEN.req | ok | - | - | - | - |
|OPEN.accept-ind| - |if(a,r)*->ESTBL| - | - | - |
|OPEN.refuse-ind| - |if(a,r)*->ESTBL| - | - | - |
|ADD.req | - | queued |->ADDING| queued | queued |
|ADD.accept-ind | - | - | - | - |if(a,r)* |
| | - | - | - | - |->ESTBL |
|ADD.refuse-ind | - | - | - | - |if(a,r)* |
| | - | - | - | - |->ESTBL |
|JOIN.ind | - | queued |->ADDING| queued |queued |
|JOIN.reject | - | OK | ok | ok | ok |
|DATA.req | - | - | ok | ok | ok |
|CHG.req | - | queued |->CHGING| queued |queued |
|CHG.accept-ind | - | - | - |if(a,r)* | - |
| | - | - | - |->ESTBL | - |
|CHG.refuse.ind | - | - | - |if(a,r)* | - |
| | - | - | - |->ESTBL | - |
|DROP.req | - | - | ok | ok | ok |
|LEAVE.ind | - | OK | ok | ok | ok |
|CLOSE.req | - | OK | ok | ok | ok |
+----------------------------------------------------------------------+
Table 2: Primitives and States at the Origin
+======================================================+
| Primitive | IDLE | PENDING | ESTBL |
|======================================================|
| OPEN/ADD.ind | ->PENDING | - | - |
| OPEN/ADD.accept | - | ->ESTBL | - |
| OPEN/ADD.refuse | - | ->IDLE | - |
| JOIN.req | ->PENDING | - | - |
| JOIN.reject-ind |- | ->IDLE | - |
| DATA.ind | - | - | ok |
| CHG.ind | - | - | ok |
| CHG.accept | - | - | ok |
| DROP.ind | - | ok | ok |
| LEAVE.req | - | ok | ok |
| CLOSE.ind | - | ok | ok |
| CHG.ind | - | - | ok |
+------------------------------------------------------+
Table 3: Primitives and States at the Target3. The ST2 Data Transfer Protocol 🦋
okay so this section is all about the ST2 data transfer protocol, aka ST 💖. first we talk about how the data actually moves in Section 3.1, and then we check out the data transfer functions in Section 3.2. let's gooo ✨
3.1. Data Transfer with ST
bestie, ST data transmission is unreliable 💀. the app is not guaranteed that its data reaches the destinations, and ST makes zero attempts to recover from packet loss (even if it's caused by the underlying network). BUT, if the data does make it, it should arrive according to the quality of service tied to the stream 🫶
also, ST may deliver data that got corrupted in transit 😭 which is lowkey wild, but it's kind of the point for real-time stuff like digital audio and video. those usually only need partially correct delivery, because retransmitted packets would show up way too late to meet their real-time deadlines. and depending on how the data is encoded and what the app is doing, a small number of errors in stream data is totally fine 🌸 if you need full reliability, layers sitting on top of ST2 can provide it 💅
now, no data fragmentation is supported during the data transfer phase 🙈 so the app has to chop its data PDUs according to the minimum MTU over all paths in the stream. the app gets info about the MTUs for the paths to the targets as part of the ACCEPT message (see Section 8.6), and the minimum MTU over all paths can be calculated from the MTUs of the single paths. ST agents silently discard data packets that are too long (see also Section 5.1.1) ✨
an ST agent only forwards data along paths to targets that are already established 🛤️ a path counts as established once the next-hop ST agent on it sends an ACCEPT message (see Section 2.2). that means the target and all the other intermediate ST agents on the way to the target are ready to handle the incoming data packets. no cases at all where an ST agent forwards data to a next-hop ST agent that hasn't explicitly accepted the stream. zero exceptions, periodt 💅
if the app wants to be reasonably sure that every target gets the data with the quality of service it wants, it should only send data after the whole stream has been set up 🎀 depending on the local API, the app might not be stopped from sending before stream setup is finished, but it should know the data could get lost or not reach every intended target. honestly this behavior can actually be desirable for some apps, like ones with multiple targets that can each process data the second it shows up (think a lecture or distributed gaming) 🎮🌈
it's also desirable for implementations to take advantage of networks that support multicast 🦋 if a network doesn't support multicast, or if the next-hops are on different networks, multiple copies of the data packet must be sent.
3.2. ST Protocol Functions
the ST protocol gives you two functions 👑
- stream identification
- data priority
3.2.1. Stream Identification
ST data packets get wrapped in an ST header that carries the Stream IDentifier (SID) 🪪 the SID is picked at the origin so it's globally unique over the Internet. the setup protocol also has to know the SID. when a stream is being established, the setup protocol builds an entry in each agent's local database for every agent the stream passes through, holding the stream's info. the SID works as a reference into that database, so an agent can quickly grab the replication and forwarding info it needs ⚡
SIDs exist to make packet forwarding super efficient 🧚 the time-critical part is an intermediate ST agent receiving a packet from the previous-hop ST agent and forwarding it on to the next-hop ST agents. speed, speed, speed 💨
the format of data PDUs including the SID is defined in Section 10.1, and SID generation is discussed in Section 8.1 🍓
3.2.2. Packet Discarding based on Data Priority
ST gives its apps a well-defined quality of service 💜 BUT, sometimes the network gets temporarily congested, and the ST agents have to throw away some packets so the overall impact on other streams is as small as possible 😩 the ST protocol has a mechanism to discard data packets based on the Priority field in the data PDU (see Section 10.1). the app gives each data packet a discard-priority level, and that level goes into the Priority field. during congestion, ST agents will try to discard the lower-priority packets first 🥹 apps can even send data at multiple priority levels, so the less important stuff gets dropped first. smart, right? 🧠✨
4. SCMP Functional Description 🌸
ST agents create and manage streams using the ST Control Message Protocol (SCMP) 🎀 conceptually, SCMP sits immediately above ST, the same way ICMP sits above IP. SCMP follows a request-response model, and its messages are made reliable through retransmission after a timeout ⏰
this section is the functional description of stream management with SCMP. to make the SCMP exchanges for setting up and maintaining ST streams easier to picture, there's an example of a simple network topology in Figure 8 👀 using the SCMP messages described in this section, an ST app will be able to:
- Create a stream from A to the peers at B, C and D,
- Add a peer at E,
- Drop peers B and C, and
- Let F join the stream
- Delete the stream.
+---------+ +---+
| |----| B |
+---------+ +----------+ | | +---+
| |------| Router 1 |---| Subnet2 |
| | +----------+ | |
| | | |
| | +---------+
| | |
| Subnet1 | |
| | +----------+
| | | Router 3 |
+---+ | | +----------+
| A |---| | +----------+ |
+---+ | |----| Router 2 | |
| | +----------+ |
+---------+ | |
| |
| +----------+ +---+
+----------| |----| C |
| | +---+
+---------+ | Subnet3 |
+---+ | | +---+ | | +---+
| F |---| Subnet4 |---| E |--| |----| D |
+---+ | | +---+ +----------+ +---+
+---------+
Figure 8: Sample Topology for an ST Streamokay so here's the roadmap 🗺️ Section 4.1 covers the possible types of stream. Section 4.2 introduces the SCMP control message types. SCMP reliability is in Section 4.3, stream options are in Section 4.4, and stream setup is in Section 4.5. Section 4.6 is the glow-up 💅 it covers stream modification, including stream expansion, reduction, and changes to the quality of service associated with a stream. and finally, Section 4.7 handles stream deletion. the end 🫶
4.1. Types of Streams
SCMP lets you set up and manage different types of streams 🦋 streams differ in how they get built and in the info they keep about the connected targets.
4.1.1. Stream Building
streams can be built in a sender-oriented fashion, a receiver-oriented fashion, or a mixed approach 🎀
- in the sender-oriented fashion, the app at the origin gives the ST agent the list of receivers for the stream. any new targets get added from the origin too 📬
- in the receiver-oriented approach, the app at the origin creates an empty stream with no targets. then each target joins the stream on its own 🙋♀️
- in the mixed approach, the app at the origin creates a stream that has some targets, and other targets join the stream autonomously 🌈
ST2 provides stream options to support the sender-oriented and mixed approach streams 💖 receiver-oriented streams can be emulated by using mixed streams, which is giving resourceful 🧠 the way targets can be added to a particular stream is controlled by join authorization levels, and those are described in Section 4.4.2 🔐
4.1.2. Knowledge of Receivers ✨
Okay so, when streams are built in the sender-oriented way, every ST agent has the full picture of all the targets downstream of it 👀. Target info gets relayed agent-to-agent during stream set-up, so nobody's out of the loop 💅.
When targets add themselves to mixed approach streams, upstream ST agents MAY or may not be informed 🙈. How targets "join" and whether that info gets passed around is also controlled by join authorization levels. (As I mentioned before, those live in Section 4.4.2, bestie 📌.)
So this leads to two types of streams 🦋:
- Full-state streams 💖: full target info gets propagated. Every agent knows about every downstream target, so that info is kept at the origin and at intermediate agents. You can always do operations on single targets, like change one or drop one. It's also always possible for any ST agent to try recovering all downstream targets.
- Light-weight streams 🪶: the origin and other upstream agents might have no idea about some targets. That means less maintained state and easier stream management 🧚, but operations on specific targets get limited. Special actions MAY be required to support change and drop operations on unknown targets, see Section 5.7. Stream recovery MAY not be possible either. Generic stuff like deleting the whole stream still works. It's expected that apps with a huge number of targets will use light-weight streams to keep state low in agents and cap the number of targets per control message 💅.
Full-state streams are giving video conferencing and distributed gaming, where you need to know who's connected so you can limit who participates 🎮. Light-weight streams MAY be exploited by apps like remote lecturing or playback of radio and TV broadcasts, where the sender doesn't need to know the receivers 📻. Section 4.4.2 defines join authorization levels, which support two types of full-state streams and one type of light-weight stream. Iconic 👑.
4.2. Control PDUs
SCMP defines the following PDUs. The main purpose of each one is noted too 🎀:
ACCEPTto accept a new streamACKto acknowledge an incoming messageCHANGEto change the quality of service associated with a streamCONNECTto establish a new stream or add new targets to an existing streamDISCONNECTto remove some or all of the stream's targetsERRORto indicate an error contained in an incoming messageHELLOto detect failures of neighbor ST agentsJOINto request stream joining from a targetJOIN-REJECTto reject a stream joining request from a targetNOTIFYto inform an ST agent of a significant eventREFUSEto refuse the establishment of a new streamSTATUSto query an ST agent on a specific streamSTATUS-RESPONSEto reply to queries on a specific stream
SCMP follows a request-response model, and every request expects a response 🔁. Retransmission after a timeout covers lost or ignored messages. Control messages do not extend across packet boundaries. If a control message is too big for the MTU of a hop, its information gets partitioned and a control message is sent per partition, as described in Section 5.1.2 📦.
CONNECT and CHANGE requests are answered with ACCEPT messages for success and REFUSE messages for failure. JOIN messages are answered with either a CONNECT message for success or a JOIN-REJECT message for failure. Targets can be removed from a stream by either the origin or the target, using DISCONNECT and REFUSE messages 🙅♀️.
The ACCEPT, CHANGE, CONNECT, DISCONNECT, JOIN, JOIN-REJECT, NOTIFY and REFUSE messages MUST always be explicitly acknowledged 🫶:
- with an
ACKmessage, if the message was received correctly and it was possible to parse and correctly extract and interpret its header, fields and parameters, - with an
ERRORmessage, if a syntax error was detected in the header, fields, or parameters included in the message. The errored PDU MAY be optionally returned as part of theERRORmessage. AnERRORmessage indicates a syntax error only. If any other errors are detected, it is necessary to first acknowledge withACKand then take appropriate actions. For instance, suppose aCHANGEmessage contains an unknown SID: first, anACKmessage has to be sent, then aREFUSEmessage with ReasonCode (SIDUnknown) follows 🧐.
If no ACK or ERROR message is received before the correspondent timer expires, a timeout failure occurs 😬. The way an ST agent SHOULD handle timeout failures is described in Section 5.2.
ACK, ERROR, and STATUS-RESPONSE messages are never acknowledged 🙃.
HELLO messages are a special case 🧚. If they contain a syntax error, an ERROR message SHOULD be generated in response. Otherwise, no acknowledgment or response SHOULD be generated. Use of HELLO messages is discussed in Section 6.1.2.
STATUS messages containing a syntax error SHOULD be answered with an ERROR message. Otherwise, a STATUS-RESPONSE message SHOULD be sent back in response. Use of STATUS and STATUS-RESPONSE is discussed in Section 8.4 💜.
4.3. SCMP Reliability
SCMP gets its reliability from retransmission whenever a response doesn't show up in a timely manner ⏰. The ACCEPT, CHANGE, CONNECT, DISCONNECT, JOIN, JOIN-REJECT, NOTIFY, and REFUSE messages all MUST be answered with an ACK message, see Section 4.2. In general, when sending an SCMP message that requires an ACK response, the sending ST agent needs to set the Toxxxx timer, where xxxx is the SCMP message type, e.g., ToConnect. If it doesn't receive an ACK before the Toxxxx timer expires, the ST agent SHOULD retransmit the SCMP message. If no ACK has been received within Nxxxx retransmissions, then an SCMP timeout condition occurs and the ST agent enters its SCMP timeout recovery state 🌀. The actions the ST agent takes because of the SCMP timeout condition differ for different SCMP messages and are described in Section 5.2.
For some SCMP messages (CONNECT, CHANGE, JOIN, and STATUS), the sending ST agent also expects a response back (ACCEPT/REFUSE, CONNECT/JOIN-REJECT) after the ACK has been received. For these cases, the ST agent needs to set the ToxxxxResp timer after it receives the ACK. As before, xxxx is the initiating SCMP message type, e.g., ToConnectResp. If the appropriate response doesn't come back when ToxxxxResp expires, the ST agent updates its state and performs the recovery action described in Section 5.2. Suggested constants are given in Section 10.5.4 🍓.
The timeout and retransmission algorithm is implementation dependent and outside the scope of this document 🤓. Most existing algorithms are based on an estimate of the Round Trip Time (RTT) between two agents. That's why SCMP contains a mechanism, see Section 8.5, to estimate this RTT. The timeout-related variable names above are for reference purposes only, and implementors MAY choose to combine certain variables 🧠.
4.4. Stream Options
An application MAY select among some stream options 🎛️. The desired options are indicated to the ST agent at the origin when a new stream is created. Options apply to single streams and stay valid for the whole stream's lifetime. The options the application chooses at the origin are included in the initial CONNECT message, see Section 4.5.3. When a CONNECT message reaches a target, the application at the target is notified of the stream options that were selected, see Section 4.5.5 🔔.
4.4.1. No Recovery
When a stream failure is detected, an ST agent would normally attempt stream recovery, as described in Section 6.2. The NoRecovery option is used to indicate that ST agents SHOULD NOT attempt recovery for the stream 🚫. The protocol behavior when the NoRecovery option has been selected is illustrated in Section 6.2. The NoRecovery option is specified by setting the S-bit in the CONNECT message, see Section 10.4.4. Only the origin can set the S-bit, and intermediate and target ST agents never modify it 🔒.
4.4.2. Join Authorization Level
When a new stream is created, you have to define the join authorization level for it 🚪. This level determines the protocol behavior when stream joining happens, see Section 4.1 and Section 4.6.3. The join authorization level for a stream is defined by the J-bit and N-bit in the CONNECT message header, see Section 10.4.4. One of the following authorization levels has to be selected:
Level 0 - Refuse Join (JN = 00): No targets are allowed to join this stream.Level 1 - OK, Notify Origin (JN = 01): Targets are allowed to join the stream. The origin is notified that the target has joined.Level 2 - OK (JN = 10): Targets are allowed to join the stream. No notification is sent to the stream origin.
Some applications like to keep tight control over their streams and won't permit any connections without the origin's permission 🔐. For those streams, target applications MAY request to be added by sending an out-of-band request (i.e., via regular IP) to the origin. The origin, if it so chooses, can then add the target following the process described in Section 4.6.1.
The selected authorization level impacts stream handling and the state maintained for the stream, as described in Section 4.1 💫.
4.4.3. Record Route
The RecordRoute option can be used to request that the route between the origin and a target be recorded and delivered to the application 🗺️. This option MAY be used while connecting, accepting, changing, or refusing a stream. Results of a RecordRoute option requested by the origin (as part of the CONNECT or CHANGE messages) are delivered to the target. Results of a RecordRoute option requested by the target (as part of the ACCEPT or REFUSE messages) are delivered to the origin.
The RecordRoute option is specified by adding the RecordRoute parameter to the mentioned SCMP messages. The format of the RecordRoute parameter is shown in Section 10.3.5. When adding this parameter, the ST agent at the origin MUST determine the number of entries that may be recorded, as explained in Section 10.3.5 📝.
4.4.4. User Data
The UserData option lets applications transport application-specific data along with some SCMP control messages 📦💖. This option can be included with ACCEPT, CHANGE, CONNECT, DISCONNECT, and REFUSE messages. The format of the UserData parameter is shown in Section 10.3.7. This option MAY be included by the origin or the target, by adding the UserData parameter to the mentioned SCMP messages. This option MAY only be included once per SCMP message ☝️.
4.5. Stream Setup
Time for stream setup 🎬. For simplicity, we assume that everything succeeds: any required resources are available, messages are properly delivered, and the routing is correct ✨. Possible failures in the setup phase are handled in Section 5.2 🫶.
4.5.1 Information from the Application ✨
okay so before stream setup can even start 💅, the application has to gather everything the connection needs, like who's joining and what kind of quality the data flow wants. 🎀 The info the application hands to the ST agent includes:
- the list of the stream's targets (Section 10.3.6). The list may be empty (Section 4.5.3.1) 🫶
- the flow specification with the quality of service the stream wants (Section 9) 🔥
- info on the groups the stream belongs to, if any (Section 7) 🌸
- info on the options picked for the stream (Section 4.4) ✨
4.5.2 Initial Setup at the Origin 🦋
the ST agent at the origin then does all of this 💜:
- allocates a stream ID (SID) for the stream (Section 8.1) 🧚
- invokes the routing function to work out the set of next-hops for the stream (Section 4.5.2.1) 🌈
- invokes the Local Resource Manager (LRM) to reserve resources (Section 4.5.2.2) 💖
- creates local database entries to store info on the new stream 🍓
- sends the stream creation request on to the next-hops that the routing function picked (Section 4.5.3) 🚀
4.5.2.1 Invoking the Routing Function 🗺️
an ST agent setting up a stream calls the routing function to find the next-hop that reaches each target on the target list the application gave it. ✨ this is kinda like the routing decision in IP, except here the route goes to a whole bunch of targets, each with their own QoS needs, instead of one single destination. 💅
the result is a set of next-hop ST agents. ngl this set is not necessarily the same as what IP would pick for a bunch of separate IP datagrams going to the same destinations. 🙈 the routing algorithm might optimise stuff other than hop count, like delay, local network bandwidth use, or total internet bandwidth use. 🌍 or it can just do a simple route lookup for each target. 🔍
once a next-hop gets picked, it sticks around for the whole life of the stream, unless a network failure happens. 💀 very loyal, very iconic. 👑
4.5.2.2 Reserving Resources 📦
the ST agent calls up the Local Resource Manager (LRM) to make the right reservations 🎟️. it gives the LRM:
- the flow specification with the QoS the stream wants (Section 9) 💜
- the version number tied to the flow specification (Section 9) 🔢
- info on the groups the stream is a member of, if any (Section 7) 🌷
the flow spec has the stuff the LRM needs to allocate resources. 💖 the LRM then updates the flow spec's contents before handing it back to the ST agent. Section 9.2.3 defines which fields of the flow spec the LRM is allowed to update. 📝
a stream being in a group can change how many resources the LRM has to allocate, so check Section 7 for that one. 🫧
4.5.3 Sending CONNECT Messages 📨
the ST agent sends a CONNECT message to each next-hop ST agent that the routing function identified. 🚀 every CONNECT message carries the SID, the selected stream options, the FlowSpec, and a TargetList. The format of the CONNECT message is defined in Section 10.4.4. 📜 generally speaking, the FlowSpec and TargetList depend on both the next-hop and the network in between. 🌐 each TargetList is a subset of the original TargetList, listing just the targets that get reached through the next-hop the CONNECT is headed to. 🎯
the TargetList may be empty (see Section 4.5.3.1). if a TargetList makes a CONNECT message too long, the message gets split up the way Section 5.1.2 explains. ✂️ and if several next-hops are reachable through a network that supports network-level multicast, a different CONNECT message still has to go to each next-hop, because each one has its own TargetList. 💅
4.5.3.1 Empty Target List 🫙
an application at the origin can ask its local ST agent to make an empty stream. 🌸 it does that by passing an empty TargetList to the local ST agent during the initial setup. when the local ST agent gets a request for an empty stream, it allocates the stream ID (SID), updates its local database entries for the new stream, and tells the application that setup is complete. 🎉 the local ST agent does not generate any CONNECT message for a stream with an empty TargetList. 🙅♀️ targets can be added later by the origin (see Section 4.6.1), or they can autonomously join the stream (see Section 4.6.3). 🦋
4.5.4 CONNECT Processing by an Intermediate ST agent 🔁
an ST agent that receives a CONNECT message, assuming no errors, replies to the previous-hop with an ACK. 💌 the ACK must identify the CONNECT message it corresponds to by including the reference number from the Reference field of the CONNECT message. 🔖 the intermediate ST agent then calls the routing function, invokes the LRM to reserve resources, and propagates the CONNECT messages to its next-hops, just like in the earlier sections. 🧚
4.5.5 CONNECT Processing at the Targets 🎯
an ST agent that is the target of a CONNECT message, assuming no errors, replies to the previous-hop with an ACK. 🫶 then it invokes the LRM to reserve local resources and asks the specified application process whether it's willing to accept the connection. 🤔
the application gets presented with parameters from the CONNECT message, including the SID, the selected stream options, Origin, FlowSpec, TargetList, and Group (if there is one), and uses them to decide. 💭 the application is identified by a combination of the NextPcol field from the Origin parameter and the service access point (SAP) field in the matching target from the TargetList (usually the only one left). 🪪 the SAP field can specify the port or some other local identifier for the protocol layer above the host ST layer. 🌷 later data packets carry the SID, which can be mapped to this info and used to deliver them. 📬
finally, based on what the application decides, the ST agent sends an ACCEPT or REFUSE message back to the previous-hop the CONNECT came from. 💖 since the previous-hop has to acknowledge the ACCEPT (or REFUSE), that message gets a brand new Reference number, which will come back in the ACK. 🔁 the CONNECT that an ACCEPT (or REFUSE) answers is identified by putting the CONNECT's Reference number in the LnkReference field of the ACCEPT (or REFUSE). 🏷️ the ACCEPT message carries the FlowSpec as the target accepted it. 🎀
4.5.6 ACCEPT Processing by an Intermediate ST agent 💫
when an intermediate ST agent receives an ACCEPT, it first checks that the message really answers an earlier CONNECT. 🕵️♀️ if it doesn't, the ST agent replies to the next-hop with an ERROR message, with ReasonCode (LnkRefUnknown). 🚫 otherwise, it replies to the next-hop with an ACK and passes the individual ACCEPT message on to the previous-hop, along the same path the CONNECT took but going backwards toward the origin. 🔙
the FlowSpec is included in the ACCEPT message so the origin and the intermediate ST agents can get access to the info that built up as the CONNECT travelled across the internet. 🌐 heads up though: the resources in the FlowSpec inside the ACCEPT message might be different from the ones reserved back when the CONNECT got processed. ⚠️ so the ST agent gives the LRM the FlowSpec from the ACCEPT message. 🎁 each LRM is expected to adjust its local reservations and release any excess resources. 🫧 the LRM can choose not to adjust if that adjustment might end up losing resources it needs. 🛡️ it can also wait to adjust allocated resources until every target in transition has been either accepted or refused. ⏳
if the intermediate ST agent is acting as the origin for this target (see Section 4.6.3.1), the ACCEPT message does not get propagated upstream. 🛑
4.5.7 ACCEPT Processing by the Origin 🏁
the origin will eventually get an ACCEPT (or REFUSE) message from each of the targets. 🎊 every time an ACCEPT comes in, the application gets told about the target and the resources that were successfully allocated along the path to it, as specified in the FlowSpec inside that ACCEPT message. 📊 then the application can either keep or terminate the portion of the stream going to each target. ✂️
when the origin receives an ACCEPT, the path to that target counts as established, and the ST agent is allowed to forward data along this path, as explained in Section 2 and Section 3.1. 🚀💖
4.5.8 REFUSE Processing by the Intermediate ST agent 🙅♀️
if an application at a target doesn't want to take part in the stream, it sends a REFUSE message back to the origin with ReasonCode (ApplDisconnect). 💔 an intermediate ST agent that gets a REFUSE message with ReasonCode (ApplDisconnect) acknowledges it by sending an ACK to the next-hop, invokes the LRM to adjust reservations as needed, deletes the target entry from the internal database, and propagates the REFUSE message back to the previous-hop ST agent. 🧹
if the intermediate ST agent is acting as the origin for this target (see Section 4.6.3.1), the REFUSE message only gets propagated upstream when there are no more downstream agents participating in the stream. 🌊 in that case, the agent says it's leaving the stream by propagating the REFUSE message with the G-bit set (1). 👋
4.5.9 REFUSE Processing by the Origin 🥀
when the REFUSE message reaches the origin, the ST agent there sends an ACK and tells the application that the target isn't part of the stream anymore, and also whether the stream has no targets left. 📢 if there are none left, the application can choose to terminate the stream, or keep it alive so more targets can be added or can join the stream, as described in Section 4.6.3. 🌈💜
4.5.10 Other Functions during Stream Setup ✨
okay so there are some extra jobs an ST agent has to handle while CONNECT messages are zooming downstream and ACCEPT (or REFUSE) messages are coming back upstream during stream setup 🧚 they weren't in the earlier sections because we wanted to keep things simple bestie 💖 here they are:
- computing the smallest Maximum Transmission Unit size over the path to the targets 📏 this is part of the MTU discovery mechanism from Section 8.6. The agent does it by updating the
MaxMsgSizefield of the CONNECT message (see Section 10.4.4), and the value gets sent back to the origin in theMaxMsgSizefield of the ACCEPT message (see Section 10.4.1). - counting how many IP clouds have to be traversed to reach the targets, if there are any 🌥️ IP clouds get traversed when the IP encapsulation mechanism is used, and that mechanism is described in Section 8.7. Encapsulating agents update the
IPHopsfield of the CONNECT message (see Section 10.4.4), and the result goes back to the origin in theIPHopsfield of the ACCEPT message (see Section 10.4.1). - updating the
RecoveryTimeoutvalue for the stream based on what the agent is able to support 🛠️ this is part of the stream recovery mechanism in Section 6.2. The agent does it by updating theRecoveryTimeoutfield of the CONNECT message (see Section 10.4.4), and the value gets carried back to the origin in theRecoveryTimeoutfield of the ACCEPT message (see Section 10.4.1).
4.6 Modifying an Existing Stream 🎀
some applications want to change a stream after it's already been created, and honestly that's so relatable 🥹 the possible changes are expanding a stream, reducing it, or changing its FlowSpec. The origin can add or remove targets (see Section 4.6.1 and Section 4.6.2). Targets can ask to join the stream (see Section 4.6.3) or decide to leave it (see Section 4.6.4). And Section 4.6.5 explains how to change a stream's FlowSpec 💅
as defined in Section 2, an ST agent can only handle one stream modification at a time 🙅♀️ so if one is already happening, any new requests get queued up and handled once the previous operation is done. This also goes for two requests of the same kind in a row, like two changes to the FlowSpec back to back. queue energy, fr fr 🫶
4.6.1 The Origin Adding New Targets 🌸
it's totally possible for an application at the origin to add new targets to a stream that already exists, any time after the stream was set up 🦋 before adding anyone, the application has to gather the info it needs on the new targets, and then that info gets handed to the ST agent at the origin.
the ST agent at the origin sends out a CONNECT message with the SID, the FlowSpec, and a TargetList naming the new targets. This is basically the same as sending a CONNECT during stream establishment, with a few extra checks 👀 the origin checks that a) the SID is valid, b) the targets aren't already members of the stream, and c) that the LRM evaluates the FlowSpec of the new target to be the same as the FlowSpec of the existing stream. In other words, it needs an equal or smaller amount of resources to be allocated. If the new target's FlowSpec doesn't match the existing stream's FlowSpec, an error gets generated with ReasonCode (FlowSpecMismatch) 🚫 functions for comparing flow specifications are provided by the LRM (see Section 1.4.5).
an intermediate ST agent that's already part of the stream looks at the SID and StreamCreationTime and checks that the stream is the same one 🧐 then it checks whether the intersection of the TargetList and the targets of the established stream is empty. If it isn't empty, the agent responds with a REFUSE message with ReasonCode (TargetExists) that has a TargetList of the targets that were duplicates. To show that the stream exists and includes the listed targets, the ST agent sets the E-bit of the REFUSE message to one (1) (see Section 10.4.11). Then it keeps going and processes each new target in the TargetList 🏃♀️
for each new target in the TargetList, processing is pretty much the same as the original CONNECT 💫 the CONNECT is acknowledged, propagated, and network resources get reserved. Intermediate or target ST agents that aren't already participants in the stream behave the way they do during stream setup (see Section 4.5.4 and Section 4.5.5).
4.6.2 The Origin Removing a Target 👋
an application at the origin can also remove existing targets from a stream any time after those targets have accepted it 💅 the application specifies which targets should go and tells its local ST agent. Based on that, the ST agent sends DISCONNECT messages with ReasonCode (ApplDisconnect) to the next-hops relative to those targets.
an ST agent that gets a DISCONNECT message has to acknowledge it by sending an ACK to the previous-hop ✅ it then updates its state and tells the LRM about the target deletion, so the LRM can change reservations as needed. When the DISCONNECT reaches the target, the ST agent also lets the application know that the target isn't part of the stream anymore 🥀 and if there are no remaining targets reachable through a particular next-hop, the ST agent tells the LRM and deletes that next-hop from its next-hops set.
SCMP also has a flooding mechanism for deleting targets that joined the stream without telling the origin 🌊 that special case of target deletion via flooding is covered in Section 5.7.
4.6.3 A Target Joining a Stream 🙋♀️
an application can ask to join a stream that already exists 🎟️ first it has to collect info on the stream, including the stream ID (SID) and the IP address of the stream's origin. This can be done out-of-band, for example over regular IP. The info then gets passed to the local ST agent, which generates a JOIN message with the application's join request and sends it toward the stream origin.
an ST agent that receives a JOIN message, assuming there are no errors, responds with an ACK 🤝 the ACK message must identify the JOIN message it corresponds to by including the Reference number from the Reference field of the JOIN message. If the ST agent isn't traversed by the stream that's being joined, it propagates the JOIN message toward the stream's origin. Once a JOIN has been acknowledged, ST agents don't keep any state info related to that JOIN 🧹
eventually the JOIN reaches an ST agent that the stream traverses, or the stream's origin itself 🎯 that agent must respond to the received JOIN first with an ACK to the ST agent the message came from. Then it issues either a CONNECT or a JOIN-REJECT message and sends it toward the target. The response depends on the join authorization level associated with the stream (see Section 4.4.2):
- If the stream has authorization level #0 (refuse join): The ST agent sends a JOIN-REJECT message toward the target with ReasonCode (
JoinAuthFailure). 🙅♀️ - If the stream has authorization level #1 (ok, notify origin): The ST agent sends a CONNECT message toward the target with a TargetList that includes the target that asked to join. This eventually adds the target to the stream 🌈 when the ST agent gets the ACCEPT message saying the new target was added, it does not propagate that ACCEPT backwards (see Section 4.5.6). Instead it issues a NOTIFY message with ReasonCode (
TargetJoined) so that upstream agents, including the origin, can add the new target to their maintained state info. The NOTIFY message includes all of the target specific info. - If the stream has authorization level #2 (ok): The ST agent sends a CONNECT message toward the target with a TargetList that includes the target that asked to join. This eventually adds the target to the stream 🍓 when the ST agent gets the ACCEPT message saying the new target was added, it does not propagate that ACCEPT backwards (see Section 4.5.6), and it doesn't notify the origin either. A NOTIFY message gets generated with ReasonCode (
TargetJoined) only if the target specific info needs to go back to the origin. An example of that kind of info is a change in MTU (see Section 8.6).
4.6.3.1 Intermediate Agent (Router) as Origin 🧭
when a stream has join authorization level #2 (see Section 4.4.2), the stream origin might not know about some of the targets in the stream 👻 in that case, the ST intermediate agent that first sent a CONNECT message to that target has to act as the stream origin for that target. That includes:
- if the whole stream gets deleted, the intermediate agent must disconnect the target.
- if the stream FlowSpec changes, the intermediate agent must change the FlowSpec for the target as appropriate.
- proper handling of ACCEPT and REFUSE messages, without propagating them to upstream ST agents.
- generating NOTIFY messages when needed (as described above).
for all the other targets that were added to the stream because of a CONNECT from the origin, the intermediate agent just behaves normally 🫧
4.6.4 A Target Deleting Itself 🦋
the application at a target can tell its local ST agent that it wants out of the stream 👋 the ST agent then forms a REFUSE message with the target itself as the only entry in the TargetList, and with ReasonCode (ApplDisconnect). That REFUSE gets sent back to the origin via the previous-hop. If a stream has multiple targets and one of them leaves using this REFUSE mechanism, the stream to the other targets isn't affected. The stream keeps existing, no drama 💖
an ST agent that receives a REFUSE message acknowledges it by sending an ACK to the next-hop ✅ the target gets deleted and the LRM is notified so it can adjust reservations as appropriate. The REFUSE message also gets propagated back to the previous-hop ST agent, except when that agent is acting as the origin. In that case a NOTIFY may be propagated instead (see Section 4.6.3).
when the REFUSE reaches the origin, the origin sends an ACK and tells the application that the target is no longer part of the stream 🌷 and that's a wrap on this part bestie 🌈✨
4.6.5 Changing a Stream's FlowSpec ✨
Okay so the application at the origin MAY wish to change the FlowSpec of an established stream 👀 This is a seriously critical operation, and it may even lead to the deletion of the affected targets in some cases 💀 Possible problems with FlowSpec changes are discussed in Section 5.6.
To change the stream's FlowSpec, the application tells the ST agent at the origin about the new FlowSpec and the list of targets the change applies to. The origin's ST agent then issues one CHANGE message per next-hop, each including the new FlowSpec, and sends it to the relevant next-hop ST agents. If the G-bit field of the CHANGE message is set (1), the change affects all targets in the stream 🫶
The CHANGE message has a bit called the I-bit, see Section 10.4.3. By default, the I-bit is set to zero (0), which tells the LRM to try its best to make the requested FlowSpec change without risking the stream getting torn down 🥺 Applications that want a higher chance of success, and are willing to risk breaking the stream, can say so by setting the I-bit to one (1). Applications that need the requested modification in order to keep running are expected to set this bit 💅
When an intermediate ST agent gets a CHANGE message, it first sends an ACK to the previous-hop and then hands the FlowSpec to the LRM. If the LRM can make the change, the ST agent passes the CHANGE messages along the established paths 🦋
If the whole process succeeds, the CHANGE messages will eventually reach the targets. The targets respond with an ACCEPT (or REFUSE) message, which gets sent back up to the origin. While the ACCEPT message is making its way back to the origin, excess resources MAY be released by the LRM, as described in Section 4.5.6. The REFUSE message MUST have the ReasonCode (ApplRefused) 🙅♀️
SCMP also has a flooding mechanism for changing targets that joined the stream without telling the origin 🌊 The special case of target change via flooding is described in Section 5.7.
4.7 Stream Tear Down 🌸
A stream is usually terminated by the origin once it has no more data to send 🧚 A stream also gets torn down if the application terminates abnormally, or if certain network failures are encountered. In that case, the processing is identical to what we already covered, except that the ReasonCode (ApplAbort, NetworkFailure, etc.) is different 🙈
When all targets have left a stream, the origin tells the application about it, and then the application is the one responsible for terminating the stream. Note, though, that the application MAY decide to add targets to the stream instead of terminating it, or MAY just leave the stream open with no targets so that stream joins are still possible 🍓
5. Exceptional Cases 💅
The previous descriptions covered the simple cases where everything worked, so now we get to the messy stuff 😭 We're talking about messages that are bigger than a network MTU, messages that get lost, requested resources that aren't available, and routing that fails or is inconsistent.
5.1 Long ST Messages
Sometimes an ST agent, or an application, needs to send a message that's bigger than a network's Maximum Transmission Unit (MTU). This case MUST be handled, but NOT via generic fragmentation, since ST2 does not support generic fragmentation of either data or control messages 🚫
5.1.1 Handling of Long Data Packets
ST agents discard data packets that exceed the MTU of the next-hop network, and no error message is generated 🗑️ Applications SHOULD avoid sending data packets larger than the minimum MTU supported by a given stream. Both the origin and the targets can learn the stream minimum MTU through the MTU discovery mechanism described in Section 8.6 🔍
5.1.2 Handling of Long Control Packets
Each ST agent knows the MTU of the networks it's connected to, and those MTUs limit how big an SCMP message it can send. An SCMP message can be bigger than a given network's MTU for a few reasons:
- the TargetList parameter (Section 10.3.6) may be too long;
- the RecordRoute parameter (Section 10.3.5) may be too long.
- the UserData parameter (Section 10.3.7) may be too long;
- the PDUInError field of the ERROR message (Section 10.4.6) may be too long;
An ST agent that receives or generates a too-long SCMP message SHOULD do the following:
- break the message into multiple messages, each carrying part of the TargetList. Any RecordRoute and UserData parameters get copied into each message so they reach all targets. Applications that support a large number of targets MAY avoid using long TargetList parameters, and are expected to do so, by using the stream joining functions, see Section 4.6.3. There's one exception to this rule. If a long TargetList parameter needs to go into a
STATUS-RESPONSEmessage, the TargetList parameter is just truncated to the point where the list fits in a single message, see Section 8.4 ✂️ - for down stream agents: if the TargetList parameter contains a single Target element and the message size is still too long, the ST agent SHOULD issue a REFUSE message with
ReasonCode(RecordRouteSize) if the size of the RecordRoute parameter pushes the SCMP message size over the network MTU, or withReasonCode(UserDataSize) if the size of the UserData parameter pushes it over. If both RecordRoute and UserData parameters are present, theReasonCode(UserDataSize) SHOULD be sent. For messages generated at the target, the target ST agent MUST check for SCMP messages that may exceed the MTU on the complete target-to-origin path, and inform the application that a too long SCMP message has been generated. How the error gets reported is a local implementation issue. The error codes are the same as previously stated 🧠
ST agents that generate too-long ERROR messages simply truncate the PDUInError field until the message is smaller than the network MTU ✨
5.2 Timeout Failures
As described in Section 4.3, SCMP message delivery is made reliable through acknowledgments, timeouts, and retransmission 🔁 The ACCEPT, CHANGE, CONNECT, DISCONNECT, JOIN, JOIN-REJECT, NOTIFY, and REFUSE messages MUST always be acknowledged, see Section 4.2. On top of that, for some SCMP messages (CHANGE, CONNECT, JOIN), the sending ST agent also expects a response back (ACCEPT/REFUSE, CONNECT/JOIN-REJECT) after an ACK has been received. Also, the STATUS message MUST be answered with a STATUS-RESPONSE message 📨
The next sections describe how to handle each of the possible failure cases caused by timeouts while waiting for an acknowledgment or a response. The timeout-related variables and their names used in those sections are just for reference. They MAY be implementation specific. Different implementations are not REQUIRED to share variable names, or even the way the timeout and retransmission behavior is implemented 🧁
5.2.1 Failure due to ACCEPT Acknowledgment Timeout
An ST agent that sends an ACCEPT message upstream expects an ACK from the previous-hop ST agent. If no ACK arrives before the ToAccept timeout expires, the ST agent SHOULD retry and send the ACCEPT message again. After NAccept unsuccessful retries, the ST agent sends a REFUSE message toward the origin, and a DISCONNECT message toward the targets. Both REFUSE and DISCONNECT MUST identify the affected targets and specify the ReasonCode (RetransTimeout) 😤
5.2.2 Failure due to CHANGE Acknowledgment Timeout
An ST agent that sends a CHANGE message downstream expects an ACK from the next-hop ST agent. If no ACK arrives before the ToChange timeout expires, the ST agent SHOULD retry and send the CHANGE message again. After NChange unsuccessful retries, the ST agent aborts the change attempt by sending a REFUSE message toward the origin, and a DISCONNECT message toward the targets. Both REFUSE and DISCONNECT MUST identify the affected targets and specify the ReasonCode (RetransTimeout) 🚧
5.2.3 Failure due to CHANGE Response Timeout
Only the origin ST agent implements this timeout 👑 After correctly receiving the ACK to a CHANGE message, an ST agent expects to receive an ACCEPT or REFUSE message in response. If neither of those arrives before the ToChangeResp timer expires, the ST agent at the origin aborts the change attempt, and behaves as if a REFUSE message with the E-bit set and with ReasonCode (ResponseTimeout) had been received ⏰
5.2.4 Failure due to CONNECT Acknowledgment Timeout
An ST agent that sends a CONNECT message downstream expects an ACK from the next-hop ST agent. If no ACK arrives before the ToConnect timeout expires, the ST agent SHOULD retry and send the CONNECT message again. After NConnect unsuccessful retries, the ST agent sends a REFUSE message toward the origin, and a DISCONNECT message toward the targets. Both REFUSE and DISCONNECT MUST identify the affected targets and specify the ReasonCode (RetransTimeout) 🫠
5.2.5 Failure due to CONNECT Response Timeout
Only the origin ST agent implements this timeout 👑 After correctly receiving the ACK to a CONNECT message, an ST agent expects to receive an ACCEPT or REFUSE message in response. If one of those isn't received before the ToConnectResp timer expires, the origin ST agent aborts the connection setup attempt, acts as if a REFUSE message were received, and sends a DISCONNECT message toward the targets. Both REFUSE and DISCONNECT MUST identify the affected targets and specify the ReasonCode (ResponseTimeout) 💔
5.2.6 Failure due to DISCONNECT Acknowledgment Timeout
An ST agent that sends a DISCONNECT message downstream expects an ACK from the next-hop ST agent. If no ACK arrives before the ToDisconnect timeout expires, the ST agent SHOULD retry and send the DISCONNECT message again. After NDisconnect unsuccessful retries, the ST agent simply gives up and assumes the next-hop ST agent is not part in the stream any more 👋
5.2.7 Failure due to JOIN Acknowledgment Timeout
An ST agent that sends a JOIN message toward the origin expects an ACK from a neighbor ST agent. If no ACK arrives before the ToJoin timeout expires, the ST agent SHOULD retry and send the JOIN message again. After NJoin unsuccessful retries, the ST agent sends a JOIN-REJECT message back in the direction of the target, with ReasonCode (RetransTimeout) 🙅♀️
5.2.8 Failure due to JOIN Response Timeout
Only the target agent implements this timeout 🎯 After correctly receiving the ACK to a JOIN message, the ST agent at the target expects to receive a CONNECT or JOIN-REJECT message in response. If one of those isn't received before the ToJoinResp timer expires, the ST agent aborts the stream join attempt and returns an error corresponding to ReasonCode (RetransTimeout) to the application 🥲
Note that, after correctly receiving the ACK to a JOIN message, intermediate ST agents don't keep any state on the stream joining attempt. So they don't set the ToJoinResp timer and don't wait for a CONNECT or JOIN-REJECT message. This is described in Section 4.6.3 🧚♀️
5.2.9. Failure due to JOIN-REJECT Acknowledgment Timeout
okay so 👀 when an ST agent sends a JOIN-REJECT toward the target, it's waiting for an ACK from a neighbor ST agent 💌 If no ACK shows up before the ToJoinReject timeout runs out, the ST agent should retry and send the JOIN-REJECT again 🔁 After NJoinReject unsuccessful retries, it simply gives up 🫠
5.2.10. Failure due to NOTIFY Acknowledgment Timeout
bestie, same energy 🦋 An ST agent that sends a NOTIFY to a neighbor ST agent expects an ACK back from that neighbor. If no ACK arrives before the ToNotify timeout expires, the ST agent should retry and send the NOTIFY again. After NNotify unsuccessful retries, it gives up and behaves as if the ACK message was received 🙈 (yes, it never got the ACK and it's acting like it did. iconic)
5.2.11. Failure due to REFUSE Acknowledgment Timeout
An ST agent that sends a REFUSE upstream expects an ACK from the previous-hop ST agent 📨 If no ACK comes before the ToRefuse timeout expires, the ST agent should retry and send the REFUSE again. After NRefuse unsuccessful retries, the ST agent gives up and assumes it is not part in the stream any more 👋 not me getting ghosted 💀
5.2.12. Failure due to STATUS Response Timeout
After sending a STATUS message to a neighbor ST agent, an ST agent expects a STATUS-RESPONSE back 🫶 If that message doesn't arrive before the ToStatusResp timer expires, the ST agent sends the STATUS again. After NStatus unsuccessful retries, it gives up and assumes the neighbor ST agent is not active 💤
5.3. Setup Failures due to Routing Failures
It's possible for an ST agent to get a CONNECT that has a known SID, but it's coming from an ST agent other than the previous-hop ST agent of the stream with that SID 👀 This could be:
- two branches of the tree forming the stream have joined back together,
- the result of an attempted recovery of a partially failed stream, or
- a routing loop.
The TargetList inside the CONNECT helps sort out which case it is. You compare each newly received target with the ones already in the existing stream:
- if the IP address of the target(s) differ, it is case #1;
- if the target matches a target in the existing stream, it may be case #2 or #3.
Case #1 is handled in Section 5.3.1, while the other cases are handled in Section 5.3.2 💅
5.3.1. Path Convergence
so this is the same SID-from-the-wrong-neighbor situation 🌸 It might happen when two branches of the stream's tree have merged back together. The detection of this case, and of the other possible sources, was covered in Section 5.2.
SCMP does not allow streams with converged paths 🚫 Streams are always tree-shaped, never graph-like. At the point of convergence, the ST agent that detects the problem generates a REFUSE with ReasonCode (PathConvergence). To help the upstream ST agent out, the detecting agent also puts the IP address of one of the stream's connected targets in the ValidTargetIPAddress field of the REFUSE. Upstream ST agents use that address to avoid splitting the stream 🧩
When an upstream ST agent gets a REFUSE with ReasonCode (PathConvergence), it checks whether the listed IP address is one of its known stream targets. If it isn't, the REFUSE is propagated to the previous-hop agent. If the listed IP address is known to the upstream ST agent, then this agent is the one that caused the split in the stream (it may even be the origin 👑). It avoids splitting the stream by using the next-hop of that known target as the next-hop for the refused targets, and it sends a CONNECT with the affected targets to the existing valid next-hop.
This process keeps going hop by hop until the ValidTargetIPAddress matches the IP address of a known target ✨ The only way it fails is if that known target gets deleted before the REFUSE reaches the origin. In that case, the origin can just reissue the CONNECT and start the whole process over again 🔄
5.3.2. Other Cases
The remaining cases, a partially failed stream and a routing loop, aren't easy to tell apart 😵💫 When an ST agent is trying to recover a failed stream, it may send new CONNECT messages to the affected targets. Such a CONNECT can reach an ST agent downstream of the failure before that ST agent has gotten a DISCONNECT from the neighborhood of the failure. Until it gets that DISCONNECT, it can't tell a failure recovery apart from an erroneous routing loop. So that ST agent must respond to the CONNECT with a REFUSE that lists the affected targets in the TargetList and has an appropriate ReasonCode (StreamExists).
The ST agent right before that point, meaning the most recent ST agent to have sent the CONNECT, will receive the REFUSE. It must release any resources it reserved exclusively for traffic to the listed targets. If this ST agent wasn't the one attempting the stream recovery, it also can't tell a failure recovery from a routing loop, so it should repeat the CONNECT after a ToConnect timeout (see Section 5.2.4). If it keeps getting REFUSE messages after NConnect retransmissions, it should propagate the REFUSE toward the origin, with a TargetList that specifies the affected targets but with a different ReasonCode, ReasonCode (RouteLoop) 🔁
Each ST agent propagates a REFUSE with ReasonCode (RouteLoop) without retransmitting any CONNECT messages. At each ST agent, it causes any resources reserved exclusively for the listed targets to be released. In the case of an erroneous routing loop, the REFUSE will be propagated all the way to the origin. In the case of stream recovery, it goes to the ST agent that is attempting the recovery, which may be an intermediate ST agent or the origin itself. For stream recovery, the ST agent attempting the recovery may then issue new CONNECT messages to the same or to different next-hops 🌈
If an ST agent receives both a REFUSE and a DISCONNECT that have a target in common, then for each target in common it can release the relevant resources and propagate neither the REFUSE nor the DISCONNECT 🧹
If the origin receives such a REFUSE, it should try to send a new CONNECT to all the affected targets. Routing errors in an internet are assumed to be temporary, so the new CONNECTs will eventually find acceptable routes to the targets, if any exist 🛣️ If no further routes exist after NRetryRoute tries, the application should be informed so it can take whatever action it thinks is necessary 📣
5.4. Problems due to Routing Inconsistency
When an intermediate ST agent receives a CONNECT, it invokes the routing algorithm to pick the next-hop ST agents, based on the TargetList and the networks it's connected to 🗺️ If the resulting next-hop to any of the targets is across the same network the CONNECT came from (but it isn't the previous-hop itself), there may be a routing problem. However, the routing algorithm at the previous-hop may be optimizing differently than the local algorithm would in the same situation. Since the local ST agent can't tell the two cases apart, it should allow the setup but send an informative NOTIFY back to the previous-hop ST agent with ReasonCode (RouteBack), the pertinent TargetList, and, in the NextHopIPAddress element, the address of the next-hop ST agent returned by its routing algorithm 🔙
The ST agent that receives such a NOTIFY should ACK it. If the ST agent is using an algorithm that would produce that behavior, no further action is taken. If not, the ST agent should send a DISCONNECT to the next-hop ST agent to correct the problem 🛠️
Alternatively, if the next-hop returned by the routing function is actually the previous-hop, a routing inconsistency has been detected 🚨 In this case, a REFUSE is sent back to the previous-hop ST agent with an appropriate ReasonCode (RouteInconsist), the pertinent TargetList, and, in the NextHopIPAddress element, the address of the previous-hop. When the previous-hop gets the REFUSE, it recomputes the next-hop for the affected targets. If the routing databases in the two ST agents are different, they may exchange CONNECT and REFUSE messages again. Routing errors in the internet are assumed to be temporary, so the situation should eventually settle down 🥰
5.5. Problems in Reserving Resources
As mentioned in Section 1.4.5, resource reservation is handled by the LRM 🧠 The LRM may not be able to satisfy a particular request during stream setup or modification, for a number of reasons: a mismatched FlowSpec, an unknown FlowSpec version, an error in processing a FlowSpec, or an inability to allocate the requested resource. This section covers these cases and specifies the ReasonCodes that should be used when they come up 📋
5.5.1. Mismatched FlowSpecs
Sometimes the LRM needs a requested FlowSpec to match an existing FlowSpec, for example when adding new targets to an existing stream (see Section 4.6.1) 🤏 If the FlowSpec doesn't match, the LRM notifies the processing ST agent, which should respond with ReasonCode (FlowSpecMismatch).
5.5.2. Unknown FlowSpec Version
When the LRM is invoked, it's passed information that includes the version of the FlowSpec (see Section 4.5.2.2) 🔢 If the LRM doesn't know that version, it notifies the ST agent, and the ST agent should respond with a REFUSE that has ReasonCode (FlowVerUnknown).
5.5.3. LRM Unable to Process FlowSpec
The LRM may run into an LRM- or FlowSpec-specific error while trying to satisfy a request 😬 Section 9.2.1 gives an example of one. These errors are implementation specific, so they aren't listed one by one with ST ReasonCodes. Instead they're covered by a single, generic ReasonCode. When the LRM hits one of these errors, it should notify the ST agent, which should respond with the generic ReasonCode (FlowSpecError) 🧯
5.5.4. Insufficient Resources
If the LRM can't make the necessary reservations because there aren't enough resources available, an ST agent may:
- try alternative paths to the targets: the ST agent calls the routing function to find a different path to the targets. If an alternative path is found, stream connection setup continues in the usual way, as described in Section 4.5 🛤️
- refuse to establish the stream along this path: the origin ST agent tells the application about the stream setup failure, and intermediate and target ST agents issue a
REFUSE(as described in Section 4.5.8) withReasonCode (CantGetResrc)🙅♀️
It depends on the local implementation whether an ST agent tries alternative paths or refuses to establish the stream. Either way, if enough resources can't be found over different paths, the ST agent has to explicitly refuse to establish the stream 💖
5.6. Problems Caused by CHANGE Messages 💅
okay so a CHANGE can fail for a couple of reasons 😭:
- insufficient resources: the request may be for a larger amount of network resources when those resources are not available,
ReasonCode (CantGetResrc); - a target application not agreeing to the change,
ReasonCode (ApplRefused);
okay, so after a change failure, the affected stream can end up in one of two states 🫶: a) the stream can revert back to the state it was in before the CHANGE message was processed, or b) the stream may be torn down.
the most common failure looks like this: the requested change can't be satisfied, but the pre-change resources are still allocated and ready for the stream to use 🎀. in that case, the ST agent where the failure happened has to let its upstream ST agents know 📣. (if this ST agent is the target, there may not be a real failure at all. the application might just not have agreed to the change 🙈.) it tells them by sending a REFUSE message with ReasonCode (CantGetResrc or ApplRefused). to show that the pre-change FlowSpec is still around and the stream still exists, the ST agent sets the E-bit of the REFUSE message to one (1), see Section 10.4.11 ✨. the upstream ST agents that get the REFUSE message tell the LRM, so it can try to go back to the pre-change FlowSpec. it's allowed for extra resources to stay allocated, but it's not something you want 💀.
now, if the change causes the stream to lose resources it already had, the stream gets torn down 😭. this can happen, for example, when the I-bit is set (Section 4.6.5) and the LRM releases the pre-change stream resources before the new ones are reserved, and then neither the new nor the old resources are available. in that case, the ST agent where the failure happens has to tell the other ST agents about the break in the affected part of the stream 🚨. it does that by sending a REFUSE message upstream and a DISCONNECT message downstream, both with ReasonCode (CantGetResrc). to show that the pre-change stream resources are gone, the E-bit of the REFUSE message is set to zero (0) 🥀.
a failed resource change for specific targets should not cause the other targets in the stream to get deleted. protect the other targets 🫶
5.7. Unknown Targets in DISCONNECT and CHANGE 🔍
how an unknown target listed in a DISCONNECT or CHANGE message gets handled depends on the stream's join authorization level, see Section 4.4.2 🧚. for streams with join authorization levels #0 and #1 (again, see Section 4.4.2), every target has to be known. so when an agent processes a CHANGE message, it should generate a REFUSE message with ReasonCode (TargetUnknown). when it processes a DISCONNECT message, though, that DISCONNECT might just be a duplicate of an old request 👀. so the agent should act as though it has already disconnected the target successfully, and respond with an ACK message 💖.
for streams with join authorization level #2, the origin might not know about some of the targets in the stream 🤷♀️. the origin can delete or change those targets using the flooding mechanism below 🌊.
if no next-hop ST agent can be linked to a target, the CHANGE/DISCONNECT message containing that target gets copied to every known next-hop ST agent 📨. this basically passes the CHANGE/DISCONNECT message down to all the downstream ST agents. eventually it reaches the ST agent that acts as the origin for the target (Section 4.6.3.1), and the target gets deleted 👑.
target deletion/change through flooding isn't expected to be the usual case, ngl 🙈. it's included so applications get the same capabilities no matter what stream type they're using ✨. flooding only applies to streams with join authorization level #2.
6. Failure Detection and Recovery 🛡️
6.1. Failure Detection 🔎
the SCMP failure detection mechanism is built on two assumptions 💜:
- if a neighbor of an ST agent is up, has been up without any disruption, and hasn't told the ST agent about a problem with streams that go through both of them, then the ST agent can assume there hasn't been any problem with those streams 🌸.
- a network that an ST agent has routed a stream through will tell the ST agent if there's a problem that affects the stream data packets but doesn't affect the control packets 📡.
the whole point of the robustness protocol is for ST agents to figure out when streams going through a neighbor have been broken, whether by the neighbor failing or by the network in between failing 🧚. this protocol should catch the vast majority of failures that can happen 🏆. once a failure is detected, the ST agents start the recovery procedures in Section 6.2 🔥.
6.1.1. Network Failures 🌧️
an ST agent can detect network failures in two ways 👀:
- the network can report a failure, or
- the ST agent can discover a failure by itself.
these two differ in how much information the ST agent has to work with when it makes a recovery decision 💅. for example, a network might be able to say that reserved bandwidth has been lost, why it was lost, and that connectivity to the neighboring ST agent is still intact ✨. on the other hand, an ST agent might notice that communication with a neighboring ST agent has stopped because it hasn't received any traffic from that neighbor for some time 😬. if an ST agent detects a failure, it may not be able to tell whether the network failed while the neighbor stayed up, or whether the neighbor failed while the network stayed intact 🤔.
6.1.2. Detecting ST Agents Failures 💓
each ST agent periodically sends a HELLO message to each neighbor it shares one or more streams with 💌. this message exchange happens between ST agents, not the entities that represent streams or applications 🫶. so an ST agent only needs to send one HELLO message to a neighbor, no matter how many streams flow between them ✨. all ST agents (host as well as intermediate) have to take part in this exchange 🧚. however, only ST agents that share active streams can take part, and it's an error to send a HELLO message to a neighbor ST agent with no streams in common, e.g., to check whether it's active 🙅♀️. STATUS messages can be used to poll the status of neighbor ST agents, see Section 8.4 📊.
for HELLO message exchange purposes, stream existence is bounded by ACCEPT and DISCONNECT/REFUSE processing, and this is defined for both the upstream and downstream cases ⭐. a stream to a previous-hop is defined to start once an ACCEPT message has been forwarded upstream 🌈. a stream to a next-hop is defined to start once the received ACCEPT message has been acknowledged 🦋. a stream is defined to end once an acknowledgment is sent for a received DISCONNECT or REFUSE message, and an acknowledgment for a sent DISCONNECT or REFUSE message has been received 🏁.
the HELLO message has two fields 💖:
- a
HelloTimerfield that is in units of milliseconds modulo the maximum for the field size, and - a
Restarted-bitspecifying that the ST agent has been restarted recently.
the HelloTimer must appear to be incremented every millisecond, whether a HELLO message is sent or not ⏱️. it wraps around to zero after reaching the maximum value 🔄. whenever an ST agent goes through a catastrophic event that might make it lose ST state information, it must reset its HelloTimer to zero, and it must set the Restarted-bit in all HELLO messages sent in the following HelloTimerHoldDown seconds 🚨.
if an ST agent gets a HELLO message with the Restarted-bit set, it must assume that the sending ST agent has lost its state 😱. if it shares streams with that neighbor, it must start stream recovery activity, see Section 6.2 🛠️. if it doesn't share streams with that neighbor, it should not try to create one until that bit is no longer set 🙈. if an ST agent gets a CONNECT message from a neighbor whose Restarted-bit is still set, the agent must respond with an ERROR message with the appropriate ReasonCode (RestartRemote) 📛. if an agent gets a CONNECT message while its own Restarted-bit is set, the agent must respond with an ERROR message with the appropriate ReasonCode (RestartLocal) 📛.
each ST stream has an associated RecoveryTimeout value 🕰️. the origin assigns this value, and it's carried in the CONNECT message, see Section 4.5.10 🎀. each agent checks whether it can support the requested value 🔍. if it can't, it updates the value to the smallest timeout interval it can support 📉. the RecoveryTimeout a particular stream uses comes from the ACCEPT message, see Section 4.5.10, and it's the smallest value seen across all ACCEPT messages from participating targets 🏆.
an ST agent must send HELLO messages to its neighbor with a period shorter than the smallest RecoveryTimeout of all the active streams passing between the two ST agents, regardless of direction ⚡. this period must be smaller by a factor called HelloLossFactor, which is at least as large as the greatest number of consecutive HELLO messages that could reasonably be lost while communication between the two ST agents is still viable 🧮.
an ST agent may send simultaneous HELLO messages to all its neighbors at the rate needed to support the smallest RecoveryTimeout of any active stream 📣. alternatively, it may send HELLO messages to different neighbors independently, at different rates matching the RecoveryTimeout values of individual streams 🦋.
an ST agent must expect to receive at least one new HELLO message from each neighbor at least as often as the smallest RecoveryTimeout of any active stream in common with that neighbor ⏰. the agent can spot duplicate or delayed HELLO messages by comparing the HelloTimer field of the most recent valid HELLO message from that neighbor with the HelloTimer field of an incoming HELLO message 🔬. valid incoming HELLO messages will have a HelloTimer field greater than the field in the previously received valid HELLO message, by the amount of time elapsed since that previous message arrived 🧠. the actual evaluation of the elapsed time interval should take into account the maximum likely delay variance from that neighbor 🌐.
if the ST agent doesn't receive a valid HELLO message within the RecoveryTimeout period of a stream, it must assume the neighboring ST agent or the communication link between the two has failed, and it must start stream recovery activity, as described below in Section 6.2 🚑💜
6.2. Failure Recovery ✨
okay so bestie, if an intermediate ST agent crashes, or a network (or part of one) goes down, the previous-hop ST agent and all the next-hop ST agents will notice it through the failure detection stuff from Section 6.1. 👀
Recovering an ST stream is a whole production 💅 because it's built to work across a huge pile of different networks with totally different vibes. So it may need info spread really widely and some pretty long timers. BUT (lowkey the good part 🫶) a single network is usually pretty homogeneous, so fixing a failure inside the network can be quicker and simpler. That means an ST agent that spots a failure should try to fix the network failure first, before it tries to recover the ST stream. If the stream that existed between two ST agents before the failure can't be rebuilt by the network recovery mechanisms alone, then the ST stream recovery mechanism has to kick in. 🔥
If stream recovery is needed, each ST agent has its own job depending on where it sits relative to the failure:
- Next-hop from the failure: this agent should first double-check that something actually broke. It can use
STATUSmessages to ping its upstream neighbor. If it can't reach that neighbor, then for each active stream coming from that neighbor it should first send aREFUSEmessage upstream with theReasonCodeSTAgentFailure. Why? It's to speed things up in case the link is one-way, meaning the neighbor can hear the ST agent but the ST agent can't hear the neighbor. 🙈 The ST agent that detected the failure must then, for each active stream from that neighbor, sendDISCONNECTmessages with that sameReasonCodetoward the targets. Every downstream ST agent handles thisDISCONNECTjust like the one that tears down a stream normally. If recovery works out, the targets will get newCONNECTmessages. 🦋 - Previous-hop before the failed component: this one also checks that a failure happened, by querying the downstream neighbor with
STATUSmessages. If the neighbor has lost its state but is still around, the ST agent may try to reconstruct (more on that below) the affected streams, as long as they don't have the NoRecovery option selected. If it can't reach the next-hop, then the ST agent that detected the failure sends aDISCONNECTfor each affected stream, with theReasonCodeSTAgentFailure, toward the affected targets. It does this to speed up recovery in case the communication is one-way and this message can still get through. 💖
Depending on the NoRecovery option, the previous-hop ST agent (the one right before the broken component) does the following:
- NoRecovery selected: the ST agent sends, for each affected stream, a
REFUSEmessage with theReasonCodeSTAgentFailureto its previous-hop. TheTargetListin these messages holds all the targets that were reached through the broken branch. As discussed in Section 5.1.2, you might need severalREFUSEmessages if the PDU is too long for the MTU of the network in between. TheREFUSEgets passed all the way up to the origin. The application at the origin can then try to recover the stream by sending a newCONNECTto the affected targets. For established streams, intermediate ST agents will treat that newCONNECTas adding new targets into the existing stream. 🎀 - NoRecovery not selected: the ST agent can try to recover the affected streams. It does this one stream at a time by sending a new
CONNECTmessage to the affected targets. If the ST agent can't find new routes to some targets, or the only route to some targets goes back through the previous-hop, then it sends one or moreREFUSEmessages to the previous-hop with theReasonCodeCantRecover, listing the affected targets in theTargetList. The previous-hop can then try to recover the stream by sending aCONNECTto those targets. If it can't find a suitable route either, it passes theREFUSEalong toward the origin. 🌸
No matter which ST agent tries to recover a damaged stream, it will send one or more CONNECT messages to the affected targets. Intermediate ST agents treat these as adding new targets into the established stream. The FlowSpecs in the new CONNECT messages are the same as the ones in the most recent CONNECT or CHANGE messages that the ST agent had sent toward those targets while the stream was working. 🧚
When an ACCEPT comes back during a stream recovery, the agent rebuilding the stream has to check that the FlowSpec and the other stream attributes (like MaxMsgSize and RecoveryTimeout) of the re-established stream are equal to, or less restrictive than, the pre-failure stream. If they're more restrictive, the recovery attempt has to be aborted. If they're equal or less restrictive, the recovery attempt is a success. And when it works, the recovering agent doesn't forward those failure recovery related ACCEPTs upstream. ✨
Any ST agent that decides enough recovery attempts have been made, or that recovery has no chance of working, may say that no more attempts should be made. It does this by setting the N-bit in the REFUSE message (see Section 10.4.11). The bit must be set by agents, including the target, that know recovery has no chance of succeeding. An ST agent that receives a REFUSE with the N-bit set (1) won't attempt recovery, no matter what the NoRecovery option says, and it will keep the N-bit set when it passes the REFUSE upstream. 🛑
6.2.1. Problems in Stream Recovery 🙈
Okay real talk, rebuilding a broken stream does not always go smoothly. 😭 Information about the failure takes time to spread through an internet, so routing errors can keep happening for a while after the failure. Because of that, the ST agent doing the recovery might get ERROR messages for its new CONNECTs that are caused by those routing errors. It should be ready to resend CONNECTs before it finally rebuilds the stream. And if the failure splits the internet up so that no new routes to the targets can be found, the REFUSE messages will eventually reach the origin, which can then tell the application so it can decide whether to quit or keep trying to recover the stream. 💀
Here's another twist: the new CONNECT might reach an ST agent downstream of the failure before the DISCONNECT does. That agent doesn't know the stream broke yet, so it thinks this is just a routing problem and replies with an ERROR using the ReasonCode StreamExists. Since the timeouts for the ST agents right before and right after the failure are pretty much the same, the leftover pieces of the broken stream will very likely get torn down by a DISCONNECT soon. 🫶 So the ST agent that gets the ERROR with ReasonCode StreamExists should resend the CONNECT once the ToConnect timeout runs out. If that fails again, the request gets retried NConnect times. Only if it still fails does the ST agent send a REFUSE with the ReasonCode RouteLoop to its previous-hop. That REFUSE travels back to the ST agent trying to recover the damaged stream, which can send a brand new CONNECT if it wants to. The REFUSE gets matched to the CONNECT created by the recovery operation using the LnkReference field in the CONNECT. 🔍
ST agents that passed along a CONNECT and then got a REFUSE should remember that for a while. If one of them gets a second CONNECT for a target that recently caused a REFUSE, it can reply with a REFUSE right away instead of passing the CONNECT on. This basically prunes the tree that the CONNECT messages build toward a target that can't be reached by the routes picked first. The tree will only go through any given ST agent once, so stream setup gets finished faster. 🚀
When a CONNECT reaches a target, the target should use, as efficiently as possible, the state it saved from before the stream failed in order to recover it. Then it sends an ACCEPT toward the origin. That ACCEPT gets intercepted by the ST agent doing the recovery, unless that agent is the origin. If the FlowSpec in the ACCEPT picks the same parameters as before the failure, the recovering ST agent doesn't pass the ACCEPT along. The FlowSpec comparison is done by the LRM. 🧠 But if the parameter selections are different, the recovering ST agent sends the origin a NOTIFY message with the ReasonCode FailureRecovery, containing a FlowSpec with the new parameter values. The origin may then need to change how it generates its data and update the stream's parameters with a CHANGE message so it can use the newly recovered subtree. 👑
6.3. Stream Preemption 💥
okay so, as mentioned in Section 1.4.5, the LRM can decide to break a stream on purpose. that's called stream preemption 🥀. streams get preempted to free up resources for a new stream that has a higher priority. it's giving "make room for the VIP" energy 👑
if the LRM decides it needs to preempt one or more of the streams going through it, it still has to figure out which streams to kill. there are two ways an application can influence that choice:
- based on FlowSpec information. for example, with the ST2+ FlowSpec, streams can get a precedence value from 0 (least important) to 256 (most important) 🎀. this value is carried in the FlowSpec when the stream is set up (see Section 9.2), so the LRM knows about it.
- with the group mechanism. an application can say that a set of streams are related to each other and that they're all candidates for preemption if one of them gets preempted 🫶. this is done with the fate-sharing relationship from Section 7.1.2. it helps the LRM make a good choice when more than one stream has to be preempted, because it means breaking one application instead of a bunch of different applications, one per preempted stream. iconic, honestly 💅
if the LRM preempts a stream, it must notify the local ST agent. then the ST agent does the following:
- the ST agent at the host where the stream was preempted sends DISCONNECT messages with the right ReasonCode (StreamPreempted) toward the affected targets. it also sends a REFUSE message with the right ReasonCode (StreamPreempted) to the previous-hop.
- a previous-hop ST agent of the preempted stream acts the same way it would in a failure recovery, see Section 6.2.
- a next-hop ST agent of the preempted stream acts the same way it would in a failure recovery, see Section 6.2.
the big difference from failure recovery: there's no need to verify that the failure actually happened, because the ReasonCode (StreamPreempted) already says so explicitly 🙈
7. A Group of Streams 🧚
there may be a need to associate related streams together. the group mechanism is basically an association technique that lets ST agents identify which streams belong together 🦋
a group is made up of a set of streams and a relationship. the set of streams is allowed to be empty, and the relationship applies to every group member. each group is identified by a group name, and the group name must be globally unique 🌈
streams belong to the same group when they have the same GroupName in the GroupName field of the Group parameter (see Section 10.3.2). the relationship is defined by the Relationship field. group membership must be specified at stream creation time and it sticks around for the whole life of the stream 💜. a single stream can belong to multiple groups, bestie, that's allowed 🙌
the ST agent that creates a new group is called the group initiator. any ST agent can be a group initiator. the initiator allocates the GroupName and the Relationship among the group members. the initiator may or may not be the origin of a stream in the group. GroupName generation is described in Section 8.2 ✨
7.1. Basic Group Relationships 🌸
this version of ST defines four basic group relationships. an ST2+ implementation must support all four of them 💖. adherence to the specified relationships is usually best effort, so it's not a guarantee, babe. the basic relationships are described in detail in Section 7.1.1 through Section 7.1.4.
7.1.1. Bandwidth Sharing 🎤
streams in the same group share the same network bandwidth. the goal is to support things like audio conferences where, out of all the participants, only some are allowed to speak at one time. in that case you can lower global bandwidth use by only reserving resources for the number of people who can talk at once. for example, you only need to reserve bandwidth for a small set of audio streams 🎧
the basic idea of a shared bandwidth group is that the LRM allocates up to some specified multiplier of the most demanding stream it knows about in the group. the LRM allocates resources incrementally, as stream setup requests come in, until the total group requirements are satisfied. later setup requests share the group's resources and don't need any extra allocation. the result is standard allocation where only one stream in a group traverses an agent, and shared allocations where multiple streams traverse an agent 🍓
let's call the multiplier "N", and the most demanding stream an agent knows about in a group Bmax. say an app wants three participants to be able to speak at the same time. then N is three, and each LRM will allocate up to 3*Bmax for the group, even if there are way more streams in the group. the LRM reserves resources incrementally, per stream request, until N*Bmax is allocated. each agent might be traversed by a different set and number of streams, all belonging to the same group 🫠
an ST agent that receives a stream request hands the LRM all the necessary group information (see Section 4.5.2.2). if the maximum bandwidth for the group, N*Bmax, is already allocated and a new stream with a bandwidth demand less than Bmax is being set up, the LRM won't allocate any further bandwidth 🛑
if there is less than N*Bmax allocated, the LRM expands the group's resources by the amount requested in the new FlowSpec, up to N*Bmax. the LRM updates the FlowSpec based on what resources are available to the stream, but not based on the total resources allocated for the group.
it's worth noting that ST agents and LRMs only find out about a group's requirements when the streams in the group are created 👀. for the bandwidth sharing relationship, an application should try to establish the most demanding streams first, to minimize setup effort. if the less demanding streams get built first, it'll always be necessary to allocate extra bandwidth in consecutive steps as the most demanding streams are built, which is a lot of extra work 😩. it's also up to the applications to coordinate their different FlowSpecs and decide on an appropriate value for N.
7.1.2. Fate Sharing 💀
streams in this group share the same fate. if one stream is deleted, the other members of the group are deleted too. this is meant to support stream preemption by showing which streams are mutually related 🔗. if multiple streams need to be preempted, the LRM can use this info to delete a whole set of related streams, which only impacts one application, instead of making a random choice that might interrupt several different apps at once 🎯. this attribute doesn't apply to normal stream shutdown, i.e., ReasonCode (ApplDisconnect). on a normal disconnect, the other streams in those groups stay active ☀️
this relationship is just a hint about which streams should be preempted. the LRM responsible for the preemption isn't forced to follow it, though, and other streams could get preempted first based on different criteria 🤷♀️
7.1.3. Route Sharing 🛤️
streams in this group share the same paths as much as possible. that can be desirable for a few reasons, like reusing the same allocated resources or trying to keep the transmission order intact 📦. an ST agent tries to select the same path, although how that's implemented depends heavily on the routing algorithm being used.
if the routing algorithm is sophisticated enough, an ST agent can suggest that a stream gets routed over a path that's already established. otherwise, it can ask the routing algorithm for a set of legal routes to the destination and check whether the desired path is one of the feasible ones 🗺️
route sharing is just a hint to the routing algorithm ST uses. failing to route a stream through a shared path should not stop a new stream from being created, and it should not result in an existing stream being deleted 🙅♀️
7.1.4. Subnet Resources Sharing 🧃
this relationship is a hint for the data link layer functions. streams in this group may share the same MAC layer resources. for example, the same MAC layer multicast address can be used for all the streams in a given group. this allows better use of MAC layer multicast addresses, and it's especially useful with network adapters that only support a very small number of MAC layer multicast addresses 🍭
7.2. Relationships Orthogonality 🔮
the four basic relationships are orthogonal, which means any combination of the basic relationships is allowed 🙆♀️. let's say an application needs full-duplex service for a stream with multiple targets. and let's say only N targets are allowed to send data back to the origin at the same time. in that scenario, all the reverse streams could belong to the same group, sharing both the paths and the bandwidth attributes. that's the Path&Bandwidth sharing relationship, and it's built from the basic set of relationships. this example matters because it shows how full-duplex service can be efficiently achieved in ST 🤩
8. Ancillary Functions 🛠️
certain functions are required by ST host and intermediate agent implementations 📋. those functions are described in this section.
8.1. Stream ID Generation 🆔
the stream ID, or SID, is made up of a 16-bit unique identifier and the stream origin's 32-bit IP address 🌐. stream IDs must be globally unique. the specific definition and format of the 16-bit field is left up to the implementor, and this field is only expected to have local significance 🧩.
an ST implementation has to provide a stream ID generator facility, so an application or higher layer protocol can get unique IDs from the ST layer 🪪. this is a way for the application to request a stream ID allocation that's separate from the request to create a stream. the application or higher layer protocol then uses the Stream ID when it creates the streams.
for instance, the following two functions could be made available:
AllocateStreamID() -> result, StreamIDReleaseStreamID(StreamID) -> result
an implementation may also provide a StreamID deletion function 💫
8.2. Group Name Generator 🎀
Okay so GroupName generation is basically the same vibe as Stream ID generation 💅. A GroupName has a 16-bit unique identifier, a 32-bit creation timestamp, and a 32-bit IP address. Group names are globally unique, and because the creator's IP address is baked right into the name, the big global uniqueness problem shrinks down to a simple local one ✨. The exact definitions and formats of the 16-bit field and the 32-bit creation timestamp are up to the implementor 🧚. Those fields MUST be locally unique, and they only have local significance.
An ST implementation has to provide a group name generator facility 🫶. That way an application or higher layer protocol can grab a unique GroupName from the ST layer. It's its own thing, separate from the request to create a stream, so the application can ask for a GroupName whenever it wants 🦋. The app or higher layer protocol then uses that GroupName when it creates the streams that are going to be part of the group.
For instance, the following two functions could be made available:
AllocateGroupName() -> result, GroupNameReleaseGroupName(GroupName) -> result
An implementation MAY also provide a GroupName deletion function 🌸.
8.3. Checksum Computation 🔥
The standard Internet checksum algorithm is what ST uses, and the rule goes like this: "The checksum field is the 16-bit one's complement of the one's complement sum of all 16-bit words in the header. For purposes of computing the checksum, the value of the checksum field is zero (0)." 💖 If you want efficient checksum algorithms, check out [RFC1071], [RFC1141], and [RFC791].
8.4. Neighbor ST Agent Identification and Information Collection 🔍
The STATUS message can be used to collect info about neighbor ST agents, the streams the neighbor supports, and specific targets of those streams 💜. Whenever an agent gets a STATUS message, it sends the requested info back in a STATUS-RESPONSE message ✨.
The STATUS message can collect different kinds of info from a neighbor. It can be used to:
- identify ST capable neighbors. If an ST agent wants to check whether a neighbor is ST capable, it SHOULD generate a STATUS message with an SID whose fields are all set to zero 🧚. An agent that gets a STATUS message with that kind of SID SHOULD answer with a STATUS-RESPONSE containing the same SID and no other stream info. The receiving ST agent MUST answer as soon as possible so it helps with Round Trip Time estimation, see Section 8.5 ⏱️.
- obtain information on a particular stream. If an ST agent wants a neighbor's general info about a specific stream, it SHOULD generate a STATUS message containing the stream's SID 🌈. The receiving agent checks first to see if it knows the stream. If it doesn't, it sends back a STATUS-RESPONSE containing the same SID and no other stream info. If it does know the stream, it sends a STATUS-RESPONSE with the stream's SID, IPHops, FlowSpec, group membership (if any), and as many targets as will fit in a single message, as limited by MTU, see Section 5.1.2 🍓. Heads up: not all targets are going to fit in a response to a general stream info request, so if you want info on one specific target, use the mechanism described next, which SHOULD be used for that 🙈.
- obtain information on particular targets in a stream. If an ST agent wants a neighbor's info on one or more specific targets of a specific stream, it SHOULD generate a STATUS message containing the stream's SID and a TargetList parameter that lists the relevant targets 🦋. The receiving agent checks first to see if the stream and the targets are known. If the stream isn't known, the agent follows the process described above. If both the stream and the targets are known, the agent responds with a STATUS-RESPONSE containing the stream's SID, IPHops, FlowSpec, group membership (if any), and the requested targets that are known 👑. If the stream is known but the target isn't, the agent responds with a STATUS-RESPONSE containing the stream's SID, IPHops, FlowSpec, and group membership (if any), but no targets 😭.
The specific formats for STATUS and STATUS-RESPONSE messages are defined in Section 10.4.12 and Section 10.4.13 📋.
8.5. Round Trip Time Estimation ⏱️
SCMP is made reliable by retransmitting when an expected acknowledgment doesn't show up in a timely manner 💅. The timeout and retransmission algorithms are implementation dependent and outside the scope of this document 🌸. Still, they MUST be reasonable enough to avoid excessive retransmission of SCMP messages while keeping the protocol robust 🫶. If you want to read about algorithms on this subject, check out [WoHD95], [Jaco88], and [KaPa87] 📚.
Most existing algorithms are built on an estimate of the Round Trip Time (RTT) between two hosts. For SCMP, if an ST agent wants an estimate of the RTT to and from a neighbor, it SHOULD generate a STATUS message with an SID whose fields are all set to zero 🧚. An ST agent that receives a STATUS message with that SID SHOULD answer as rapidly as possible with a STATUS-RESPONSE containing the same SID and no other stream info ⚡. The time between the send and receive operations can be used as an estimate of the RTT to and from the neighbor ✨.
8.6. Network MTU Discovery 🌸
Okay this one is so cute 🥹. At connection setup, the application at the origin asks the local ST agent to create streams with certain QoS requirements 🎀. The local ST agent fills in its network MTU value in the MaxMsgSize parameter of the CONNECT message and forwards it to the next-hop ST agents. Each ST agent on the path checks whether its network MTU is smaller than the one in the CONNECT message, and if it is, that agent updates the MaxMsgSize in the CONNECT message to its network MTU (it's giving smallest-wins 💅). If the target application decides to accept the stream, the ST agent at the target copies the MTU value in the CONNECT message into the MaxMsgSize field of the ACCEPT message and sends it back to the application at the origin 🦋. The MaxMsgSize field in the ACCEPT message is the minimum MTU of the intervening networks to that target. If the application has multiple targets, the minimum MTU of the stream is the smallest MaxMsgSize received from all the ACCEPT messages 🍓.
It's the application's job to segment its PDUs according to the stream's minimum MaxMsgSize, because no data fragmentation is supported during the data transfer phase 🙈. If a particular target's MaxMsgSize isn't acceptable to an application, the application MAY disconnect that target from the stream and assume the target can't be supported 💀. When evaluating a target's MaxMsgSize, the application or the application interface will need to take the size of the ST data header into account 🧚.
8.7. IP Encapsulation of ST 🦋
ST packets MAY be encapsulated in IP so they can pass through routers that don't support the ST Protocol 🌈. Sure, ST resource management is precluded over that path and packet overhead goes up because of the encapsulation, but if the performance is reasonably predictable, this may be better than not communicating at all 💜.
IP-encapsulated ST packets begin with a normal IP header. Most fields of the IP header SHOULD be filled in following the same rules that apply to any other IP packet. Three fields are of special interest:
- Protocol is 5, see [RFC1700], to indicate that an ST packet is enclosed, as opposed to TCP or UDP, for example 🎀.
- Destination Address is that of the next-hop ST agent. This may or may not be the target of the ST stream 🌸. There may be an intermediate ST agent to which the packet SHOULD be routed, so you can take advantage of service guarantees on the path past that agent 👑. That intermediate agent wouldn't be on a directly-connected network (or else IP encapsulation wouldn't be needed), so it probably wouldn't be in the normal routing table 🙈. Additional routing mechanisms, not defined here, will be REQUIRED to learn about such agents 🔍.
- Type-of-Service MAY be set to an appropriate value for the service being requested, see [RFC1700]. This feature isn't implemented uniformly across the Internet, so its use can't be precisely defined here 🫶.
IP encapsulation adds little difficulty for the ST agent that receives the packet 💅. However, when IP encapsulation is performed, it MUST be done in both directions 🔥. To process the encapsulated IP message, the ST agents simply remove the IP header and proceed with the ST header as usual ✨.
The trickier part happens during setup, when the ST agent MUST decide whether or not to encapsulate 😭. If the next-hop ST agent is on a remote network and the route to that network goes through a router that supports IP but not ST, then encapsulation is REQUIRED 🧚. The routing function gives ST agents the route and capability info they need to support encapsulation 🌈.
On forwarding, the (mostly constant) IP Header MUST be inserted and the IP checksum appropriately updated 💖.
Applications are informed about the number of IP hops traversed on the path to each target 🍓. The IPHops field of the CONNECT message, see Section 10.4.4, carries the number of traversed IP hops to the target application 🦋. That field is incremented by each ST agent when IP encapsulation will be used to reach the next-hop ST agent. The number of IP hops traversed is returned to the origin in the IPHops field of the ACCEPT message, Section 10.4.1 👑.
When IP Encapsulation is in use, the MaxMsgSize field will not reflect the MTU of the IP encapsulated segments 🙈. That means IP fragmentation and reassembly may be needed in the IP cloud to support a message of MaxMsgSize 💀. IP fragmentation can only occur when the MTU of the IP cloud, less the IP header length, is the smallest MTU in a stream's network path 🌸.
8.8. IP Multicasting ✨
okay so, if an ST agent MUST use IP encapsulation to reach several next-hops heading toward different targets, then it's one of two options 💖 either the packet MUST be replicated and sent separately to each next-hop, or IP multicasting MAY be used, but only if it's implemented in the next-hop ST agents AND the intervening IP routers. 🦋
once the stream is set up, the group of next-hop ST agents MUST be set up as an IP multicast group 🫶 the ST agent MUST allocate a suitable IP multicast address (see Section 10.3.3) and drop it into the IPMulticastAddress field of the CONNECT message. that address is how the next-hop ST agents find out they should join the group so they can catch the later PDUs. obviously the CONNECT message itself MUST be sent using unicast 🙈 and the next-hop ST agents MUST be able to receive on the specified multicast address, otherwise they can't accept the connection.
if a next-hop ST agent can't receive on that multicast address, it sends a REFUSE message with ReasonCode (BadMcastAddress) 💀 then the upstream agent can retry with a different multicast address, or it can give up the multicast efficiency and just use unicast delivery. 🍓
the following permanent IP multicast addresses have been assigned to ST:
224.0.0.7 All ST routers (intermediate agents)
224.0.0.8 All ST hosts (agents)on top of that, a block of transient IP multicast addresses, 224.1.0.0 - 224.1.255.255, has been allocated for ST multicast groups 🌈 for instance, the following two functions could be made available:
AllocateMcastAddr() -> result, McastAddrListenMcastAddr(McastAddr) -> resultReleaseMcastAddr(McastAddr) -> result
9. The ST2+ Flow Specification ✨
this section is all about the ST2+ flow specification 💅 it's where the user application spells out its requirements for quality of service. the contents depend on the LRM and are totally invisible to the ST2 setup protocol. ST2 just carries it inside the FlowSpec parameter (more on that in Section 10.3.1). the REQUIRED ST2+ flow specification is only in the protocol so implementations can interoperate, and ST2+ also has a "null" flow specification that's ONLY for testing. 🧪
ST2 isn't tied to one particular flow specification format, and other versions are expected to show up later 🔮 different formats get told apart by the value of the Version field in the FlowSpec parameter (see Section 10.3.1). a single stream always sticks with ONE format, so the Version field stays consistent across the whole stream. these Version values are defined:
0 - Null FlowSpec /* must be supported */
1 - ST Version 1
2 - ST Version 1.5
3 - RFC 1190 FlowSpec
4 - HeiTS FlowSpec
5 - BerKom FlowSpec
6 - RFC 1363 FlowSpec
7 - ST2+ FlowSpec /* must be supported */FlowSpecs version #0 and #7 MUST be supported by ST2+ implementations 🎀 values 1-6 mean those flow specs are already in use in existing ST2 implementations, and values in the 128-255 range are reserved for private and experimental use. 🧚
in general, a flow specification MAY support sophisticated flow descriptions. for example, one could represent sub-flows of a particular stream 👑 a cooperating application and LRM could then use that to forward designated packets to specific targets based on the different sub-flows. the reserved bits in the ST2 Data PDU (see Section 10.1) MAY be used with such a flow specification to mark packets belonging to different sub-flows. the ST2+ FlowSpec isn't that fancy, though. it's meant for applications that generate traffic at a single rate for uniform delivery to all targets. 🌸
9.1. FlowSpec Version #0 - (Null FlowSpec)
the flow specification with a #0 Version value is called the Null FlowSpec 🫧 it causes NO resources to be allocated, the LRMs ignore it, and its contents never get updated. stream setup happens in the usual way and ends in successful establishment, but literally no resources get reserved. 🙃
the whole point is interop testing: it lets streams get built without actually allocating the matching amount of resources. the Null FlowSpec MAY also be used for testing and debugging. 🧪
the Null FlowSpec is just the 4-byte FlowSpec parameter (see Section 10.3.1), and the third byte (the Version field) MUST be set to 0. 💜
9.2. FlowSpec Version #7 - ST2+ FlowSpec
the flow specification with a #7 Version value is the ST2+ FlowSpec, and ALL ST2+ implementations use it 💖 it lets user applications express their real-time needs using a QoS class, a precedence, and three basic QoS parameters:
- message size,
- message rate,
- end-to-end delay.
the QoS class says what kind of QoS guarantees the application is expecting, like strict guarantees or predictive (see Section 9.2.1). QoS parameters come as a set of values:
- the "desired" values are the QoS the application actually wants. the application sets them and the LRM NEVER modifies them. 🥰
- the "limit" values are the lowest QoS the application will accept. the application sets these too, and the LRM never modifies them either. 🛑
- the "actual" values are what the system can really deliver. the LRM updates them at each node, and they're ALWAYS bounded by the "limit" and "desired" values. ⭐
9.2.1. QoS Classes
two QoS classes are defined 🎀
1 - QOS_PREDICTIVE /* QoSClass field value = 0x01, must be
supported*/
2 - QOS_GUARANTEED /* QoSClass field value = 0x10, optional */- QOS_PREDICTIVE is the class where the negotiated QoS MAY be violated for short time intervals during data transfer 🌊 the application has to provide values that cover the "normal" case. for example, the "desired" message rate is the rate that's allocated for transmission. reservations are made for the "normal" case, which is the opposite of the peak case that QOS_GUARANTEED calls for. this QoS class MUST be supported by ALL implementations. 💅
- QOS_GUARANTEED is the class where the negotiated QoS for the stream is NEVER violated during data transfer 🔥 the application has to provide values that cover the worst possible case, so the "desired" message rate is the peak rate for transmission. as a result, enough resources to handle the peak rate get reserved. this can overbook resources (MAY lead to overbooking, that's a real possibility), but it gives strict real-time guarantees. support for this QoS class is OPTIONAL. 🙈
if an LRM that doesn't support the QOS_GUARANTEED class receives a FlowSpec with that class, it tells the local ST agent 😭 the ST agent MAY try different paths, or delete the matching portion of the stream as described in Section 5.5.3, i.e., ReasonCode (FlowSpecError). 🛤️
9.2.2. Precedence
precedence is how important the connection being set up is 👑 zero is the LOWEST precedence, and the lowest level is what's expected by default. the big difference between precedence and priority is this: precedence decides which streams are allowed to take resources that were already committed to another stream, while priority marks which PDUs a stream would be happiest to have dropped. 🦋
9.2.3. Maximum Data Size
this parameter is in bytes. it's the maximum amount of data (not counting ST and other headers) that's allowed to be sent in a message as part of the stream 🍓 the LRM checks first whether it can get the value the application wants (DesMaxSize). if it can't, it updates the actual value (ActMaxSize) with whatever size is available, UNLESS that's less than the minimum the application will accept (LimitMaxSize). in that case, it tells the local ST agent the stream can't be built along this path. 🙅♀️
9.2.4. Message Rate
this parameter is in messages/second and it's the transmission rate for the stream 🌸 the LRM checks first whether it can get the desired rate (DesRate). if not, it updates the actual value (ActRate) with the available rate, UNLESS that's below the minimum the application allows (LimitRate). if it is, the LRM tells the local ST agent the stream can't be built along this path. 🧚
9.2.5. Delay and Delay Jitter
the delay parameter is in milliseconds and represents the maximum end-to-end delay for the stream ⏱️ the LRM checks first whether it can hit the desired delay (DesMaxDelay). if it can't, it updates the actual value (ActMaxDelay) with the available delay, UNLESS that value is greater than the maximum delay the application allows (LimitMaxDelay). then the LRM tells the local ST agent the stream can't be built along this path. 😤
the LRM also updates the MinDelay field at each node by adding the minimum possible delay to the next-hop 🧮 that info lets you work out the maximum end-to-end delay range, which is the time window in which a data packet can arrive. this range SHOULD NOT exceed the DesMaxDelayRange value the application asked for. and the maximum end-to-end delay range is an upper bound on the delay jitter. 🫶✨
9.2.6. ST2+ FlowSpec Format ✨
okay bestie, here's the actual shape of the ST2+ FlowSpec 🎀 it's a neat little grid of fields, 32 bits wide, and it's giving very organized 💅
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| QosClass | Precedence | 0(unused) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DesRate |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| LimitRate |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ActRate |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DesMaxSize | LimitMaxSize |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ActMaxSize | DesMaxDelay |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| LimitMaxDelay | ActMaxDelay |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DesMaxDelayRange | ActMinDelay |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 9: The ST2+ FlowSpec.so the LRM only touches the "actual" fields (the ones starting with Act) 🫶 everything else is filled in by the user application 🌸
- QoSClass tells you which of the two classes of service applies 🦋 there's
QOS_PREDICTIVE(QoSClass = 1) andQOS_GUARANTEED(QoSClass = 2) 💜 - Precedence is the stream's precedence. Zero is the lowest, and it SHOULD be the default value ✨
- DesRate is the desired transmission rate for the stream, in messages/second. The origin sets it and intermediate agents don't touch it 🍓
- LimitRate is the minimum acceptable transmission rate, in messages/second. Also set by the origin, also left alone by intermediate agents 🔥
- ActRate is the actual rate allocated for the stream, in messages/second. Each agent updates it with the available rate, unless that value is less than LimitRate, and then a REFUSE gets generated 💀
- DesMaxSize is the desired maximum data size in bytes for a message in the stream. The origin sets it 🧚
- LimitMaxSize is the minimum acceptable data size in bytes. The origin sets this one too 🎀
- ActMaxSize is the actual max data size a message in the stream may carry. Each agent updates it based on MTU and available resources. If the available maximum is less than LimitMaxSize, the connection MUST be refused with ReasonCode (CantGetResrc) 😭
- DesMaxDelay is the desired maximum end-to-end delay for the stream, in milliseconds. Origin sets it 🌈
- LimitMaxDelay is the upper bound of acceptable end-to-end delay for the stream, in milliseconds. Origin sets it too 💖
- ActMaxDelay is the max end-to-end delay stream data will actually see. Each ST agent adds the max delay it introduces, including the transmission time to the next-hop ST agent. If the actual maximum goes over LimitMaxDelay, the connection is refused with ReasonCode (CantGetResrc) 👑
- DesMaxDelayRange is the desired maximum delay range stream data may run into end-to-end, in milliseconds. The application at the origin sets this one 🦋
- ActMinDelay is the actual minimum end-to-end delay stream data will hit, in milliseconds. Each ST agent adds the minimum delay it introduces, including transmission time to the next-hop ST agent. Each agent MUST add at least 1 millisecond ⏱️ The delay range can be worked out from the actual max and min delay fields, and it's expected some applications will care about that range a lot 🫶
10. ST2 Protocol Data Units Specification 🎀
10.1. Data PDU 🌸
okay so IP and ST packets can be told apart by the IP Version Number field (the first four bits of the packet) 🦋 ST got the value 5 (see [RFC1700]), and IP uses 4. Beyond those first four bits, there's no requirement that IP and ST packet headers match 💅
ST PDUs that go between ST agents are made of an ST Header wrapping either a higher layer PDU or an ST Control Message. Data packets and control messages are told apart by the D-bit (bit 8) in the ST header ✨
The ST Header also carries an ST Version Number, a total length field, a header checksum, a unique id, and the 32-bit IP address of the stream origin. The unique id plus that origin address make up the stream id (SID) 🧚 Figure 10 shows it all. Check out Section 10.6 if you want the notation explained 💜
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ST=5 | Ver=3 |D| Pri | 0 | TotalBytes |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| HeaderChecksum | UniqueID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| OriginIPAddress |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 10: ST Header- ST is the IP Version Number that identifies ST packets. For ST it's 5 🎀
- Ver is the ST Version Number. The current ST2+ version is 3 ✨
- D (bit 8) is set to 1 in all ST data packets and to 0 in all SCMP control messages 🦋
- Pri (bits 9-11) is the packet-drop priority field, where zero (0) is lowest priority and seven is the highest. It's used as described in Section 3.2.2 💅
- TotalBytes is the length in bytes of the whole ST packet. It includes the ST Header but NOT any local network headers or trailers. In general, every length field in the ST Protocol is in bytes 🍓
- HeaderChecksum covers only the ST Header (12 bytes). The ST Protocol uses 16-bit checksums here in the ST Header and in each Control Message. For how it's computed, see Section 8.3 🔥
- UniqueID is the first element of the stream ID (SID). It's locally unique at the stream origin, see Section 8.1 🧚
- OriginIPAddress is the second element of the SID: the 32-bit IP address of the stream origin, see Section 8.1 💖
Bits 12-15 MUST be set to zero (0) when you're using the flow specs defined in this document (see Section 9). They MAY be set differently when other flow specs are used, e.g., as described in [WoHD95] 🌈
10.1.1. ST Data Packets 🦋
ST packets with a non-zero D-bit are data packets 🫶 what they mean is up to the higher layer protocols, so it's not specified here. Data packets don't have an ST checksum, so they get delivered to the higher layer protocol even if they have errors (yikes 💀). ST agents will not pass data packets over a new hop whose setup isn't complete yet 🎀
10.2. Control PDUs 💜
SCMP control messages go between neighbor ST agents with a D-bit of zero (0) ✨ The control protocol is request-response, and every request expects a response. If something gets lost or ignored, retransmission after a timeout (see Section 4.3) covers it 🌸 Control messages can't stretch across packet boundaries: if one is too big for a hop's MTU, its info gets split up and a control message is sent per partition (see Section 5.1.2) 🧚 Every control message looks like this:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| OpCode | Options | TotalBytes |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reference | LnkReference |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SenderIPAddress |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Checksum | ReasonCode |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: OpCodeSpecificData :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 11: ST Control Message Format- OpCode tells you what type of control message it is 💖
- Options carries OpCode-specific variations for the message 🎀
- TotalBytes is the control message length in bytes, including all OpCode specific fields and optional parameters. It's always divisible by four (4) 🍓
- Reference is a transaction number. Each sender of a request control message gives it a Reference number that's unique within the stream 🫶 The receiver uses that number to spot and toss duplicates. Each acknowledgment carries the Reference of the request it's acking. Reference zero (0) is never used, and Reference numbers go up monotonically with wraparound, so older-than and more-recent-than are well defined 🌈
- LnkReference holds the Reference field of the request that caused this request to be created. It matters when one request leads to several responses from the same ST agent. CONNECT and CHANGE messages are a good example: they're acked hop-by-hop first, then lead to an ACCEPT or REFUSE from each target 🧚
- SenderIPAddress is the 32-bit IP address of the network interface the ST agent used to send the control message. It changes every time an ST agent forwards the packet (hop-by-hop) 🔥
- Checksum is the checksum of the control message. Control messages can arrive with bits in error, so each control message MUST be checked to be error free before anyone acts on it 💀
- ReasonCode is set to zero (0 = NoError) in most SCMP messages. Otherwise it can hold a value that indicates an error situation, as defined in Section 10.5.3 👑
- OpCodeSpecificData holds any extra info tied to the control message. It depends on the message type and gets explained further below ✨ In some response messages, fields of zero (0) are included so the format matches the corresponding request 🌸 OpCodeSpecificData can also hold optional parameters. The specifics are defined in Section 10.3 💅
10.3. Common SCMP Elements 💖
okay so, a bunch of fields and parameters (we call them elements) show up in two or more PDUs, so instead of explaining them over and over, the spec explains them once right here. 🧚 Lowkey genius. Lots of these parameters are optional. To keep parsing simple, every one starts with a PCode byte, followed by a PBytes byte that tells you the parameter's length in bytes. That length counts the PCode, the PBytes, and any padding bytes too. If the info isn't a multiple of four (4) bytes, you pad it with one to three zero (0) bytes, so PBytes is always a multiple of four (4). ✨ And the parameters can show up in any order, no vibes check needed 🫶.
10.3.1. FlowSpec 🌸
The FlowSpec parameter (PCode = 1) is used in several SCMP messages to carry the ST2 flow specification. It looks like this:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PCode = 1 | PBytes | Version | 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: FlowSpec detail :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 12: FlowSpec Parameter
- The Version field tells you which FlowSpec version you're looking at. 🦋
- The FlowSpec detail field holds the flow specification. The ST agent doesn't interpret it (it's totally transparent to the ST agent 🙈), and it's the data structure that gets passed along to the LRM. It must be 4-byte aligned.
So the Null FlowSpec (see Section 9.1) has no FlowSpec detail field at all. For it, PBytes is set to four (4) and Version is set to zero (0). The ST2+ FlowSpec (see Section 9.2) is a 32-byte data structure, so PBytes is set to 36 and Version is set to seven (7). 💅
10.3.2. Group 🍓
The Group parameter (PCode = 2) is optional. It says that the stream belongs to a specified group. 👑
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PCode = 2 | PBytes = 16 | GroupUniqueID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| GroupCreationTime |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| GroupInitiatorIPAddress |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Relationship | N |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 13: Group Parameter
- GroupUniqueID, GroupInitiatorIPAddress, and GroupCreationTime together make up the GroupName field. They're allocated by the group name generator function (see Section 8.2). GroupUniqueID and GroupCreationTime are implementation specific and only have local definitions. 🌈
- Relationship has the following format:
0
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 0 (unused) |S|P|F|B|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 14: Relationship Field
The B, F, P, and S bits line up with Bandwidth, Fate, Path, and Subnet resource sharing (see Section 7). A value of 1 means that the relationship exists for this group. Every combination of the four bits is allowed. Bits 0-11 of the Relationship field are reserved for future use and must be set to 0. 🔥
- N only has a legal value if the B-bit is set. It's the value of the N parameter used as explained in Section 7.1.1. 🦋
10.3.3. MulticastAddress 🧚
The MulticastAddress parameter (PCode = 3) is optional. It's used when you're doing IP encapsulation and setting up an IP multicast group. Basically, it tells the next-hop ST agents that should become members of the group which IP multicast address to use (see Section 8.8). 💜
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PCode = 3 | PBytes = 8 | 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IPMulticastAddress |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 15: MulticastAddress
- IPMulticastAddress is the 32-bit IP multicast address used to receive data packets for the stream. 🎀
10.3.4. Origin 🌸
The Origin parameter (PCode = 4) identifies the next higher protocol, plus the SAP being used with that protocol. ✨
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PCode = 5 | PBytes | NextPcol |OriginSAPBytes |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: OriginSAP : Padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 16: Origin
- NextPcol is an 8-bit field used for demultiplexing to figure out which protocol sits above ST. Its values live in the same number space as the IP header's Protocol field, so they're defined in the Assigned Numbers RFC [RFC1700]. 🧠
- OriginSAPBytes is the length of the OriginSAP, not counting any padding needed to keep 32-bit alignment.
- OriginSAP identifies the origin's SAP that goes with the NextPcol protocol.
Note that the 32-bit IP address of the stream origin isn't included in this parameter, because it's always already in the ST header. 🙈
10.3.5. RecordRoute 🗺️
The RecordRoute parameter (PCode = 5) asks for the route between the origin and a target to be recorded and handed to the user application. The ST agent at the origin (or the target) that includes this parameter has to work out its length, which goes in the PBytes field. ST agents that process messages containing this parameter add their receiving IP address at the position given by the FreeOffset field, space permitting. If there's no space, the parameter gets passed on unchanged. When the origin includes it, every agent between the origin and the target adds its IP address, and the application at the target gets all of that info. When the target includes it, every agent between the target and the origin, inclusive, adds its IP address, and the application at the origin gets it. 🦋
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PCode = 5 | PBytes | 0 | FreeOffset |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IP Address 1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: ... :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IP Address N |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 17: RecordRoute
- PBytes is the length of the parameter in bytes. The length gets decided by whichever agent (target or origin) introduces the parameter first. Once it's set, the length stays unchanged. ⭐
- FreeOffset is the offset, relative to the start of the parameter, where the next IP address gets recorded. When FreeOffset is greater than, or equal to, PBytes, the RecordRoute parameter is full. 💀
- IP Address gets filled in, space permitting, by each ST agent that processes this parameter.
10.3.6. Target and TargetList 🎯
Several control messages use a parameter called TargetList (PCode = 6), which holds info about the targets the message is about. For each Target in the TargetList, you get the 32-bit IP address of the target, the SAP for the next higher layer protocol, and the length of that SAP (SAPBytes). Since the SAP can vary in length, a Target structure can be variable length too. Each entry looks like Figure 18. 🌈
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Target IP Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| TargetBytes | SAPBytes | SAP : Padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 18: Target
- TargetIPAddress is the 32-bit IP Address of the Target. 💜
- TargetBytes is the length of the Target structure, counting from the TargetIPAddress.
- SAPBytes is the length of the SAP, not counting any padding needed for 32-bit alignment.
- SAP can be longer than 2 bytes, and it includes padding when it's required. There wouldn't be any padding for SAPs with lengths of 2, 6, 10, etc., bytes. 🧚
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PCode = 6 | PBytes | TargetCount = N |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Target 1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: : :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Target N |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 19: TargetList
10.3.7. UserData 🎀
okay so the UserData parameter (PCode = 7) is an optional parameter 🫶 It's basically a little sticky note that the next higher protocol or an application can use to pass arbitrary info to its peers. It shows up in some control messages, but ST agents don't care what's inside (the contents have no significance to them). 💅
Control messages are capped by the smallest MTU on the path to the targets, so there's no way to know the max size of this parameter ahead of time 😭 If the parameter makes a message go over the network MTU, the ST agent does what Section 5.1.2 describes. And the parameter MUST be padded to a multiple of 32 bits ✨
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PCode = 7 | PBytes | UserBytes |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: UserInfo : Padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 20: UserData- UserBytes specifies the number of valid UserInfo bytes. 🍓
- UserInfo is arbitrary data meaningful to the next higher protocol layer or application. 🌸
10.3.8. Handling of Undefined Parameters 🧚
An ST agent MUST be able to handle all parameters listed above. 💖 To support possible future uses, parameters with unknown PCodes MUST also be supported. If an agent receives a message containing a parameter with an unknown Pcode value, the agent SHOULD handle that parameter as if it were a UserData parameter. Basically, the contents of the parameter SHOULD be ignored, and the message SHOULD be propagated, as appropriate, along with the related control message. 🦋
10.4. ST Control Message PDUs 💜
ST Control messages are described in the following section. Please refer to Section 10.6 for an explanation of the notation. 🌈
10.4.1. ACCEPT 🔥
ACCEPT (OpCode = 1) is issued by a target as a positive response to a CONNECT message. It means the target is ready to accept data from the origin along the stream that CONNECT set up. ACCEPT is also issued as a positive response to a CHANGE message, meaning the target accepts the proposed stream modification. 👑
ACCEPT is relayed by the ST agents from the target back to the origin, along the path CONNECT (or CHANGE) set up, but in the reverse direction. ACCEPT MUST be acknowledged with ACK at each hop. 🙈
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| OpCode = 1 | 0 | TotalBytes |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reference | LnkReference |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SenderIPAddress |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Checksum | ReasonCode = 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MaxMsgSize | RecoveryTimeout |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| StreamCreationTime |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IPHops | 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: FlowSpec :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: TargetList :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: RecordRoute :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: UserData :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 21: ACCEPT Control Message- Reference contains a number assigned by the ST agent sending ACCEPT for use in the acknowledging ACK. 💅
- LnkReference is the Reference number from the corresponding CONNECT (or CHANGE) 🔗
- MaxMsgSize indicates the smallest MTU along the path traversed by the stream. This field is only set when responding to a CONNECT request. 📏
- RecoveryTimeout reflects the nominal number of milliseconds that the application is willing to wait for a failed system component to be detected and any corrective action to be taken. This field represents what can actually be supported by each participating agent, and is only set when responding to a CONNECT request. ⏱️
- StreamCreationTime is the 32- bits system dependent timestamp copied from the corresponding CONNECT request. 🕰️
- IPHops is the number of IP encapsulated hops traversed by the stream. This field is set to zero by the origin, and is incremented at each IP encapsulating agent. 🧭
10.4.2. ACK ✅
ACK (OpCode = 2) is used to acknowledge a request. The ACK message is not propagated beyond the previous-hop or next-hop ST agent. 🫶
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| OpCode = 2 | 0 | TotalBytes |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reference | LnkReference = 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SenderIPAddress |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Checksum | ReasonCode |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 22: ACK Control Message- Reference is the Reference number of the control message being acknowledged. 📨
- ReasonCode is usually NoError, but other possibilities exist, e.g., DuplicateIgn. 🤷♀️
10.4.3. CHANGE 🦋
CHANGE (OpCode = 3) is used to change the FlowSpec of an established stream. The CHANGE message is processed similarly to CONNECT, except that it travels along the path of an established stream. CHANGE MUST be propagated until it reaches the related stream's targets. CHANGE MUST be acknowledged with ACK at each hop. 🌸
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| OpCode = 3 |G|I| 0 | TotalBytes |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reference | LnkReference = 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SenderIPAddress |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Checksum | ReasonCode = 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: FlowSpec :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: TargetList :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: RecordRoute :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: UserData :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 23: CHANGE Control Message- G (bit 8) is used to request a global, stream-wide change; the TargetList parameter SHOULD be omitted when the G bit is specified. 🌍
- I (bit 7) is used to indicate that the LRM is permitted to interrupt and, if needed, break the stream in the process of trying to satisfy the requested change. ⚡
- Reference contains a number assigned by the ST agent sending CHANGE for use in the acknowledging ACK. 💌
10.4.4. CONNECT 🎀
okay bestie, CONNECT is OpCode 4 and it's the message that asks for a brand new stream, or to add someone to an existing one, or to recover one that broke 💖 only the origin gets to send the first batch of CONNECTs to set up a stream 👑, and the very first CONNECT to each next-hop is the one that carries the SID ✨
the next-hop starts off by replying with an ACK, which basically means "got it, the CONNECT was valid and we're on it" 🫶 later it sends back an ACCEPT or a REFUSE from each target. if an intermediate ST agent gets a CONNECT, it does what Section 4.5 explains 💅
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| OpCode = 4 |J N|S| 0 | TotalBytes |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reference | LnkReference = 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SenderIPAddress |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Checksum | ReasonCode = 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MaxMsgSize | RecoveryTimeout |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| StreamCreationTime |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| IPHops | 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: Origin :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: FlowSpec :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: TargetList :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: RecordRoute :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: Group :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: MulticastAddress :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: UserData :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 24: CONNECT Control Message 📋
JN(bits 8 and 9) is the join authorization level for the stream, so it's basically the VIP pass for joining 🎟️ see Section 4.4.2.S(bit 10) is the NoRecovery option (Section 4.4.1). when theS-bitis set (1), NoRecovery is specified for the stream 🙈Referenceis a number the sending ST agent assigns so the ACK can point back to it 🔖MaxMsgSizeis the smallest MTU along the path the stream goes through. it starts out as the network MTU of the agent issues the CONNECT 📏RecoveryTimeoutis the nominal number of milliseconds the application is willing to wait for a failed system component to be detected and any corrective action to be taken ⏳StreamCreationTimeis the 32- bits system dependent timestamp generated by the ST agent issuing the CONNECT 🕰️IPHopsis the number of IP encapsulated hops the stream goes through. the origin sets it to zero, and every IP encapsulating agent bumps it up by one 🦋
10.4.5. DISCONNECT 💔
DISCONNECT is OpCode 5, and it's the "bye bye" message 😭 an origin uses it to tear down a stream that's already up, or part of one. an intermediate ST agent can send it too, if it notices a failure between itself and its previous-hop, and the ReasonCode tells you which situation it is 🧐
the DISCONNECT lists the targets that need to be disconnected. an ACK is required in response to it, and the message gets propagated all the way down to those targets. the targets are expected to stop taking part in the stream, so it's a whole goodbye tour 👋🌈
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| OpCode = 5 |G| 0 | TotalBytes |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reference | LnkReference = 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SenderIPAddress |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Checksum | ReasonCode |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| GeneratorIPAddress |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: TargetList :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: UserData :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 25: DISCONNECT Control Message 📋
G(bit 8) is used to ask for a DISCONNECT of all the stream's targets 🧹TargetListshould be omitted when the G-bit is set (1). ifTargetListis present anyway, it gets ignored 🙅♀️Referenceis a number the sending ST agent assigns so the ACK can reference it 🔖ReasonCodereflects the event that started the whole thing 🚨GeneratorIPAddressis the 32-bit IP address of the host that first generated the DISCONNECT message 🏠
10.4.6. ERROR 🚫
ERROR is OpCode 6, and it's what you send back when a request has an error in it 🙈 no action is taken on the erroneous request, so it just gets ignored like a bad text 💀 no ACK is expected for it, and it doesn't get passed beyond the previous-hop or next-hop ST agent. an ERROR is never sent in response to another ERROR, because that would be chaos 😭 and the receiver of an ERROR is encouraged to try again without waiting for a retransmission timeout, which is so iconic of them ⚡
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| OpCode = 6 | 0 | TotalBytes |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reference | LnkReference = 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SenderIPAddress |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Checksum | ReasonCode |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: PDUInError :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 26: ERROR Control Message 📋
Referenceis the Reference number of the erroneous request 🔖ReasonCodeindicates the error that triggered the message 🚨PDUInErroris the PDU in error, beginning with the ST Header 🧾 this parameter is optional. its length is limited by network MTU, and it may be truncated when it's too long ✂️
10.4.7. HELLO 👋
HELLO is OpCode 7, and it's the friendly "still here?" message 🥰 it's part of the ST failure detection mechanism, see Section 6.1.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| OpCode = 7 |R| 0 | TotalBytes |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reference = 0 | LnkReference = 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SenderIPAddress |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Checksum | ReasonCode = 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| HelloTimer |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 27: HELLO Control Message 📋
R(bit 8) is used for the Restarted-bit 🔄HelloTimeris the time in milliseconds since the agent was restarted, modulo the precision of the field ⏱️ it's used to spot duplicate or delayed HELLO messages 🕵️♀️
10.4.8. JOIN 🤝
JOIN is OpCode 8, and it's how someone joins a stream 🎉 it's used as part of the ST steam joining mechanism, see Section 4.6.3.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| OpCode = 8 | 0 | TotalBytes |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reference | LnkReference = 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SenderIPAddress |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Checksum | ReasonCode = 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| GeneratorIPAddress |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: TargetList :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 28: JOIN Control Message 📋
Referencecontains a number assigned by the ST agent sending JOIN for use in the acknowledging ACK 🔖GeneratorIPAddressis the 32-bit IP address of the host that generated the JOIN message 🏠TargetListis the information associated with the target to be added to the stream 🎯
10.4.9 JOIN-REJECT ✨
bestie, JOIN-REJECT (OpCode = 9) is the message that gets used in the ST stream joining mechanism 🦋, which you can read about in Section 4.6.3. It's basically the "no, you can't come in" 💅 reply to a JOIN.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| OpCode = 9 | 0 | TotalBytes |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reference | LnkReference |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SenderIPAddress |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Checksum | ReasonCode |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| GeneratorIPAddress |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 29: JOIN-REJECT Control Message
Referenceis a number picked by the ST agent that sends the REFUSE, and it gets used in the ACK that comes back 🎀.LnkReferenceis the Reference number from the matching JOIN message 🔗.ReasonCodetells you why the JOIN got rejected 🙈.GeneratorIPAddressis the 32-bit IP address of the host that first generated the JOIN-REJECT message 🌸.
10.4.10 NOTIFY 🔔
okay so NOTIFY (OpCode = 10) is how an ST agent tells the other ST agents about events that might matter 👀. It can get passed beyond the previous-hop or next-hop ST agent, depending on the ReasonCode (see Section 10.5.3). And NGL, a NOTIFY has to get an ACK back 💜.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| OpCode = 10 | 0 | TotalBytes |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reference | LnkReference = 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SenderIPAddress |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Checksum | ReasonCode |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DetectorIPAddress |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| MaxMsgSize | RecoveryTimeout |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: FlowSpec :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: TargetList :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: UserData :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 30: NOTIFY Control Message
Referenceis a number assigned by the ST agent that sends the NOTIFY, for the ACK to use 🎀.ReasonCodeidentifies the reason for the notification 💡.DetectorIPAddressis the 32-bit IP address of the ST agent that detects the event 👑.MaxMsgSizegets set when the MTU of the listed targets has changed (like after recovery), or when the notification is generated after a successful JOIN. Otherwise it's set to zero (0) 🍓.RecoveryTimeoutgets set when the notification is generated after a successful JOIN. Otherwise it's set to zero (0) ⏱️.FlowSpecis there when the notification is generated after a successful JOIN 🌊.TargetListis there when the notification relates to one or more targets, or when MaxMsgSize is set 🎯.UserDatais there if the notification is generated after a successful JOIN and the UserData parameter was set in the ACCEPT message 🧚.
10.4.11 REFUSE 🙅♀️
REFUSE (OpCode = 11) is the "nope, I'm out" 💀 message. A target sends it when it doesn't want to accept a CONNECT, or when it wants to take itself off an established stream. An intermediate ST agent might send it too, in response to a CONNECT or CHANGE, to break a routing loop, or when it can't find a good next-hop to a target. It can also be its own command, like when a higher-precedence stream has preempted an existing one, or when an ST agent notices that a previous-hop, a next-hop, or the network between them has failed 😭. In every case, the TargetList says which targets are affected by the situation. Each REFUSE has to get an ACK back too 🫶.
The REFUSE gets relayed back by the ST agents to the origin (or to the intermediate ST agent that made the CONNECT or CHANGE), following the same path the CONNECT traced. The ST agent that receives it handles it differently depending on the condition behind it, which is spelled out in the ReasonCode field 🧠. Nobody tries to squish multiple REFUSEs together, because it's considered pretty unlikely that separate REFUSEs will both pass through an ST agent at the same moment and be easy to combine, like having identical ReasonCodes and parameters 🤷♀️.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| OpCode = 11 |G|E|N| 0 | TotalBytes |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reference | LnkReference |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SenderIPAddress |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Checksum | ReasonCode |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| DetectorIPAddress |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ValidTargetIPAddress |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: TargetList :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: RecordRoute :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: UserData :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 31: REFUSE Control Message
G(bit 8) is used to show that every target downstream from the sender is refusing. This will mostly be set because of network failures 🌧️. When it's set, the TargetList parameter is ignored or not present, and when it's not set, the TargetList must be included.E(bit 9) gets set by an ST agent to say the request failed, and that the pre-change stream attributes (resources included) and the stream itself still exist 🌸.N(bit 10) says that no more attempts to recover the stream should be made. It must be set when stream recovery shouldn't be attempted, even if the target application shut down normally (ApplDisconnect) 🛑.Referenceis a number assigned by the ST agent sending the REFUSE, for the acknowledging ACK to use 🎀.LnkReferenceis either the Reference number from the corresponding CONNECT or CHANGE (if the REFUSE is a result of one of those), or zero when the REFUSE was sent as a separate command 🔗.DetectorIPAddressis the 32-bit IP address of the host that first generated the REFUSE message 👀.ValidTargetIPAddressis the 32-bit IP address of a host that's still properly connected as part of the stream. This parameter is only used when recovering from stream convergence, otherwise it's set to zero (0) 🧚.
10.4.12 STATUS 📊
STATUS (OpCode = 12) is used to check whether a particular stream (identified by the SID) actually exists 🔍. It's meant for collecting info from a neighbor ST agent, like general and specific stream details, plus round trip time estimation ⏱️. You can read about how this message type gets used in Section 8.4.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| OpCode = 12 | 0 | TotalBytes |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reference | LnkReference = 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SenderIPAddress |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Checksum | ReasonCode = 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: TargetList :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 32: STATUS Control Message
Referenceis a number assigned by the ST agent sending STATUS, for the replying STATUS-RESPONSE to use 💌.TargetListis optional. When it's present, it means only info about those specific targets should be passed along in the STATUS-RESPONSE 🎯.
10.4.13 STATUS-RESPONSE 📨
okay so STATUS-RESPONSE (OpCode = 13) is the reply you get back after sending a STATUS message 💅 if the stream named in your STATUS isn't known to the agent, the STATUS-RESPONSE still carries the SID you asked about, but nothing else. otherwise it's giving full details: the current SID, FlowSpec, TargetList, and possibly Groups for that stream 🦋 and if the full target list won't fit in one message, only the targets that can squeeze in will show up. remember from Section 10.4.12 that you can ask about one specific target if you want. 🎀
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| OpCode = 13 | 0 | TotalBytes |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reference | LnkReference = 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| SenderIPAddress |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Checksum | ReasonCode = 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: FlowSpec :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: Groups :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: TargetList :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 33: STATUS-RESPONSE Control Message 🧚
- Reference contains a number assigned by the ST agent sending the STATUS.
10.5 Suggested Protocol Constants 👑
okay this part is the cheat sheet ✨ the ST Protocol has some fields that must have specific values or nothing works 😭 and some values that an implementation has to pick for itself. this section lists the required values and suggests starting values for the rest. it's recommended that the suggested ones get built as variables so they're easy to change once experience shows better numbers 🫶 and eventually they should be managed through the normal network management facilities. periodt 💜
ST uses IP Version Number 5. 🌈
When encapsulated in IP, ST uses IP Protocol Number 5. 🌈
10.5.1 SCMP Messages 🦋
- ACCEPT
- ACK
- CHANGE
- CONNECT
- DISCONNECT
- ERROR
- HELLO
- JOIN
- JOIN-REJECT
- NOTIFY
- REFUSE
- STATUS
- STATUS-RESPONSE
10.5.2 SCMP Parameters 🎀
- FlowSpec
- Group
- MulticastAddress
- Origin
- RecordRoute
- TargetList
- UserData
10.5.3 ReasonCode 💀
okay so several errors can pop up while the protocol is processing 😬 all ST error codes come from one single number space. the currently defined values and what they mean are in the list below, but new error codes might get defined from time to time, so every implementation is expected to handle new codes gracefully 🫶 and if you run into an unknown ReasonCode, it should be assumed to be fatal. the ReasonCode is an 8-bit field, and these are the defined values:
1 NoError No error has occurred.
2 ErrorUnknown An error not contained in this list has been
detected.
3 AccessDenied Access denied.
4 AckUnexpected An unexpected ACK was received.
5 ApplAbort The application aborted the stream abnormally.
6 ApplDisconnect The application closed the stream normally.
7 ApplRefused Applications refused requested connection or
change.
8 AuthentFailed The authentication function failed.
9 BadMcastAddress IP Multicast address is unacceptable in CONNECT
10 CantGetResrc Unable to acquire (additional) resources.
11 CantRelResrc Unable to release excess resources.
12 CantRecover Unable to recover failed stream.
13 CksumBadCtl Control PDU has a bad message checksum.
14 CksumBadST PDU has a bad ST Header checksum.
15 DuplicateIgn Control PDU is a duplicate and is being
acknowledged.
16 DuplicateTarget Control PDU contains a duplicate target, or an
attempt to add an existing target.
17 FlowSpecMismatch FlowSpec in request does not match
existing FlowSpec.
18 FlowSpecError An error occurred while processing the FlowSpec
19 FlowVerUnknown Control PDU has a FlowSpec Version Number that
is not supported.
20 GroupUnknown Control PDU contains an unknown Group Name.
21 InconsistGroup An inconsistency has been detected with the
streams forming a group.
22 IntfcFailure A network interface failure has been detected.
23 InvalidSender Control PDU has an invalid SenderIPAddress
field.
24 InvalidTotByt Control PDU has an invalid TotalBytes field.
25 JoinAuthFailure Join failed due to stream authorization level.
26 LnkRefUnknown Control PDU contains an unknown LnkReference.
27 NetworkFailure A network failure has been detected.
28 NoRouteToAgent Cannot find a route to an ST agent.
29 NoRouteToHost Cannot find a route to a host.
30 NoRouteToNet Cannot find a route to a network.
31 OpCodeUnknown Control PDU has an invalid OpCode field.
32 PCodeUnknown Control PDU has a parameter with an invalid
PCode.
33 ParmValueBad Control PDU contains an invalid parameter value.
34 PathConvergence Two branches of the stream join during the
CONNECT setup.
35 ProtocolUnknown Control PDU contains an unknown next-higher
layer protocol identifier.
36 RecordRouteSize RecordRoute parameter is too long to permit
message to fit a network's MTU.
37 RefUnknown Control PDU contains an unknown Reference.
38 ResponseTimeout Control message has been acknowledged but not
answered by an appropriate control message.
39 RestartLocal The local ST agent has recently restarted.
40 RestartRemote The remote ST agent has recently restarted.
41 RetransTimeout An acknowledgment has not been received after
several retransmissions.
42 RouteBack Route to next-hop through same interface as
previous-hop and is not previous-hop.
43 RouteInconsist A routing inconsistency has been detected.
44 RouteLoop A routing loop has been detected.
45 SAPUnknown Control PDU contains an unknown next-higher
layer SAP (port).
46 SIDUnknown Control PDU contains an unknown SID.
47 STAgentFailure An ST agent failure has been detected.
48 STVer3Bad A received PDU is not ST Version 3.
49 StreamExists A stream with the given SID already exists.
50 StreamPreempted The stream has been preempted by one with a
higher precedence.
51 TargetExists A CONNECT was received that specified an
existing target.
52 TargetUnknown A target is not a member of the specified
stream.
53 TargetMissing A target parameter was expected and is not
included, or is empty.
54 TruncatedCtl Control PDU is shorter than expected.
55 TruncatedPDU A received ST PDU is shorter than the ST Header
indicates.
56 UserDataSize UserData parameter too large to permit a
message to fit into a network's MTU.10.5.4 Timeouts and Other Constants ⏰
okay the big one 🔥 SCMP uses retransmission to get reliability, so it has a bunch of "retransmission timers" ⏱️ each "timer" is modeled by an initial time interval (ToXxx), which can get updated on the fly by measuring control traffic, plus a number of times (NXxx) to retransmit a message before it gives up and declares a failure 💀 all time intervals are in milliseconds. also, these variables are only described for reference, so different implementations might not have the exact same ones. no drama 🌸
Value Timeout Name Meaning
------------------------------------------------------------------------
500 ToAccept Initial hop-by-hop timeout for acknowledgment of
ACCEPT
3 NAccept ACCEPT retries before failure
500 ToChange Initial hop-by-hop timeout for acknowledgment of
CHANGE
3 NChange CHANGE retries before failure
5000 ToChangeResp End-to-End CHANGE timeout for receipt of ACCEPT
or REFUSE
500 ToConnect Initial hop-by-hop timeout for acknowledgment of
CONNECT
5 NConnect CONNECT retries before failure
5000 ToConnectResp End-to-End CONNECT timeout for receipt of ACCEPT
or REFUSE from targets by origin
500 ToDisconnect Initial hop-by-hop timeout for acknowledgment of
DISCONNECT
3 NDisconnect DISCONNECT retries before failure
500 ToJoin Initial hop-by-hop timeout for acknowledgment of
JOIN
3 NJoin JOIN retries before failure
500 ToJoinReject Initial hop-by-hop timeout for acknowledgment of
JOIN-REJECT
3 NJoinReject JOIN-REJECT retries before failure
5000 ToJoinResp Timeout for receipt of CONNECT or JOIN-REJECT
from origin or intermediate hop
500 ToNotify Initial hop-by-hop timeout for acknowledgment of
NOTIFY
3 NNotify NOTIFY retries before failure
500 ToRefuse Initial hop-by-hop timeout for acknowledgment of
REFUSE
3 NRefuse REFUSE retries before failure
500 ToRetryRoute Timeout for receipt of ACCEPT or REFUSE from
targets during failure recovery
5 NRetryRoute CONNECT retries before failure
1000 ToStatusResp Timeout for receipt of STATUS-RESPONSE
3 NStatus STATUS retries before failure
10000 HelloTimerHoldDown Interval that Restarted bit must be set
after ST restart
5 HelloLossFactor Number of consecutively missed HELLO
messages before declaring link failure
2000 DefaultRecoveryTimeout Interval between successive HELLOs
to/from active neighbors10.6. Data Notations
okay bestie, let's talk about how the docs crew writes numbers and draws data 🎀 Numbers go in decimal, and pictures show the most significant octet on the left and the least significant octet on the right 💅
The order of transmission for the header and data in this doc is locked in at the octet level 🔒 Whenever a diagram shows a group of octets, they get sent in the normal order you'd read them in English. Figure 34 is the example: the octets go out in the order they're numbered ✨
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 1 | 2 | 3 | 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 5 | 6 | 7 | 8 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 9 | 10 | 11 | 12 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 34: Transmission Order of Bytes
When an octet holds a numeric value, the leftmost bit in the diagram is the high order, most significant bit 👑 So the bit labeled 0 is the MSB. Check out Figure 35, where the octet stands for 170 in decimal 🦋
0 1 2 3 4 5 6 7
+-+-+-+-+-+-+-+-+
|1 0 1 0 1 0 1 0|
+-+-+-+-+-+-+-+-+Figure 35: Significance of Bits
Same vibe for multi-octet fields 🌈 When a whole field is a numeric quantity, its leftmost bit is the most significant bit. And when that quantity gets transmitted, the most significant octet goes first 🫶
Now for the notation shorthand 🍓 Fields with a fixed length that are fully drawn out end with a vertical bar (|). Fixed fields whose contents are abbreviated get an exclamation point (!). Variable fields get colons (:). Optional parameters are split off from control messages with a blank line, and the order of parameters doesn't matter 🙈
11. References ✨
[RFC1071] Braden, R., Borman, D., and C. Partridge, "Computing the Internet Checksum", RFC 1071, USC/Information Sciences Institute, Cray Research, BBN Laboratories, September 1988.
[RFC1112] Deering, S., "Host Extensions for IP Multicasting", STD 5, RFC 1112, Stanford University, August 1989.
[WoHD95] L. Wolf, R. G. Herrtwich, L. Delgrossi: Filtering Multimedia Data in Reservation-based Networks, Kommunikation in Verteilten Systemen 1995 (KiVS), Chemnitz-Zwickau, Germany, February 1995.
[RFC1122] Braden, R., "Requirements for Internet Hosts -- Communication Layers", STD 3, RFC 1122, USC/Information Sciences Institute, October 1989.
[Jaco88] Jacobson, V.: Congestion Avoidance and Control, ACM SIGCOMM-88, August 1988.
[KaPa87] Karn, P. and C. Partridge: Round Trip Time Estimation, ACM SIGCOMM-87, August 1987.
[RFC1141] Mallory, T., and A. Kullberg, "Incremental Updating of the Internet Checksum", RFC 1141, BBN, January 1990.
[RFC1363] Partridge, C., "A Proposal Flow Specification", RFC 1363, BBN, September 1992.
[RFC791] Postel, J., "Internet Protocol", STD 5, RFC 791, DARPA, September 1981.
[RFC1700] Reynolds, J., and J. Postel, "Assigned Numbers", STD 2, RFC 1700, USC/Information Sciences Institute, October 1994.
[RFC1190] Topolcic C., "Internet Stream Protocol Version 2 (ST-II)", RFC 1190, CIP Working Group, October 1990.
[RFC1633] Braden, R., Clark, D., and S. Shenker, "Integrated Services in the Internet Architecture: an Overview", RFC 1633, USC/Information Sciences Institute, MIT, Xerox PARC, June 1994.
[VoHN93] C. Vogt, R. G. Herrtwich, R. Nagarajan: HeiRAT: the Heidelberg Resource Administration Technique - Design Philosophy and Goals, Kommunikation In Verteilten Systemen, Munich, Informatik Aktuell, Springer-Verlag, Heidelberg, 1993.
[Cohe81] D. Cohen: A Network Voice Protocol NVP-II, University of Southern California, Los Angeles, 1981.
[Cole81] R. Cole: PVP - A Packet Video Protocol, University of Southern California, Los Angeles, 1981.
[DeAl92] L. Delgrossi (Ed.) The BERKOM-II Multimedia Transport System, Version 1, BERKOM Working Document, October, 1992.
[DHHS92] L. Delgrossi, C. Halstrick, R. G. Herrtwich, H. Stuettgen: HeiTP: a Transport Protocol for ST-II, GLOBECOM'92, Orlando (Florida), December 1992.
[Schu94] H. Schulzrinne: RTP: A Transport Protocol for Real-Time Applications. Work in Progress, 1994.
12. Security Considerations 💀
Okay, this part is short and I'm not going to pretend otherwise: security issues are not discussed in this memo 💀 So if you're looking for threat models here, you'll have to go somewhere else 😭
13. Acknowledgments and Authors' Addresses 🫶
Huge thank-yous to everyone who helped build this work 💖 The ST Working Group participants get a shoutout for their input, review, and constructive comments 🌸 George Mason University's C3I Center gets props for hosting an interim meeting 🏫 Murali Rajagopal gets a standing ovation for the ST2+ state machines 👑 And a special, heartfelt thanks to Steve DeJarnett, who served as working group co-chair until summer 1993 🧚
We also want to shout out the authors of [RFC1190] 📣 All authors of [RFC1190] should be considered authors of this document, since this document contains much of their text and ideas 🦋
Now, the people behind this doc, with their addresses 💅
Louis Berger
Louis Berger
BBN Systems and Technologies
1300 North 17th Street, Suite 1200
Arlington, VA 22209
Phone: 703-284-4651
EMail: lberger@bbn.comLuca Delgrossi
Luca Delgrossi
Andersen Consulting Technology Park
449, Route des Cretes
06902 Sophia Antipolis, France
Phone: +33.92.94.80.92
EMail: luca@andersen.frDat Duong
Dat Duong
BBN Systems and Technologies
1300 North 17th Street, Suite 1200
Arlington, VA 22209
Phone: 703-284-4760
EMail: dat@bbn.comSteve Jackowski
Steve Jackowski
Syzygy Communications Incorporated
269 Mt. Hermon Road
Scotts Valley, CA 95066
Phone: 408-439-6834
EMail: stevej@syzygycomm.comSibylle Schaller
Sibylle Schaller
IBM ENC
Broadband Multimedia Communications
Vangerowstr. 18
D69020 Heidelberg, Germany
Phone: +49-6221-5944553
EMail: schaller@heidelbg.ibm.comThat's the whole doc, bestie! 🌈✨ We made it through all 24 parts 💜
📦 where this page comes from
- ietf: rfc1819 1995-08
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms