RFC 5811
SCTP-Based Transport Mapping Layer (TML) for the Forwarding and Control Element Separation (ForCES) Protocol
SCTP-Based Transport Mapping Layer (TML) for the Forwarding and Control Element Separation (ForCES) Protocol
Okay bestie, let's get the vitals first 💅 This is RFC 5811, an IETF Standards Track doc published in March 2010 (ISSN 2070-1721). It was written by J. Hadi Salim from Mojatatu Networks and K. Ogawa from NTT Corporation. The title block doesn't list anything it obsoletes or updates, so this one stands on its own 🦋
Abstract
Okay so this doc defines the SCTP-based TML (Transport Mapping Layer) for the ForCES (Forwarding and Control Element Separation) protocol ✨ It explains why SCTP (Stream Control Transmission Protocol) was picked, and it also shows how this TML covers all the requirements that the ForCES protocol needs it to meet 🎀
Status of This Memo
This is an Internet Standards Track document. 🌸
It's a product of the Internet Engineering Task Force (IETF), which means it reflects the consensus of the IETF community. It got public review and was approved for publication by the Internet Engineering Steering Group (IESG). If you want more on Internet Standards, check out Section 2 of RFC 5741 📚
For the current status, any errata, and how to send feedback, go to http://www.rfc-editor.org/info/rfc5811 💜
Copyright Notice
Copyright (c) 2010 IETF Trust and the persons identified as the document authors. All rights reserved. 🔥
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents, which you can find at http://trustee.ietf.org/license-info as they stood on the publication date. Please actually read those, because they spell out your rights and restrictions for this doc 🙈 Any Code Components pulled out of this document must include Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come without warranty as described in that license.
This document may contain material from IETF Documents or IETF Contributions that were published or made publicly available before November 10, 2008. The people who control the copyright in some of that material may not have given the IETF Trust permission to allow modifications outside the IETF Standards Process. So unless you get an adequate license from those copyright holders, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process. The one exception is formatting it for publication as an RFC or translating it into languages other than English 🫶
Table of Contents
1. Introduction ....................................................3
2. Definitions .....................................................3
3. Protocol Framework Overview .....................................4
3.1. The PL .....................................................5
3.2. The TML ....................................................5
3.2.1. TML and PL Interfaces ...............................5
3.2.2. TML Parameterization ................................6
4. SCTP TML Overview ...............................................7
4.1. Rationale for Using SCTP for TML ...........................7
4.2. Meeting TML Requirements ...................................8
4.2.1. SCTP TML Channels ...................................9
4.2.2. Satisfying TML Requirements ........................14
5. SCTP TML Channel Work ..........................................16
6. IANA Considerations ............................................16
7. Security Considerations ........................................17
7.1. IPsec Usage ...............................................17
7.1.1. SAD and SPD Setup ..................................18
8. Acknowledgements ...............................................18
9. References .....................................................19
9.1. Normative References ......................................19
9.2. Informative References ....................................20
Appendix A. Suggested SCTP TML Channel Work Implementation .......21
A.1. SCTP TML Channel Initialization ...........................21
A.2. Channel Work Scheduling ...................................21
A.2.1. FE Channel Work Scheduling ............................21
A.2.2. CE Channel Work Scheduling ............................22
A.3. SCTP TML Channel Termination ..............................23
A.4. SCTP TML NE-Level Channel Scheduling ......................23
Appendix B. Suggested Service Interface ..........................24
B.1. TML Bootstrapping .........................................24
B.2. TML Shutdown ..............................................26
B.3. TML Sending and Receiving .................................271. Introduction ✨
Okay, so the ForCES (Forwarding and Control Element Separation) working group in the IETF is the crew that defines the architecture and protocol for splitting control elements (CEs) and forwarding elements (FEs) inside network elements (NEs) like routers 🧚 [RFC3654] and [RFC3746] cover the architectural and protocol requirements for CE-to-FE communication, respectively. The ForCES protocol layer spec in [RFC5810] describes the protocol semantics and how everything works. That protocol layer sits on top of an interconnect-hiding layer called the TML. Here's how it all fits together, as shown in Figure 1 👑
This document defines the SCTP-based TML for the ForCES protocol layer. It also covers all the TML requirements, including security, reliability, and etc., as defined in [RFC5810] 💖
2. Definitions
The BCP 14 key words are "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL". In this document, they are to be interpreted as described in [RFC2119] 🎀
These definitions come from [RFC3654] and [RFC3746]:
LFB: Logical Functional Block. A template that
represents a fine-grained, logically separate
aspect of FE processing.
ForCES Protocol: The protocol used at the Fp reference point in the
ForCES Framework in [RFC3746].
ForCES PL: ForCES Protocol Layer. A layer in the ForCES
architecture that embodies the ForCES protocol and
the state transfer mechanisms as defined in
[RFC5810].
ForCES TML: ForCES Protocol Transport Mapping Layer. A layer
in the ForCES protocol architecture that
specifically addresses the protocol message
transportation issues, such as how the protocol
messages are mapped to different transport media
(like SCTP, IP, TCP, UDP, ATM, Ethernet, etc.), and
how to achieve and implement reliability, security,
etc.3. Protocol Framework Overview
Go read the Framework document [RFC3746], especially Sections 3 and 4, for the big-picture architecture and where ForCES fits in 🌈 There's some overlap between this section and the ForCES protocol spec in [RFC5810], but that's on purpose, so you get the basic context right here.
The ForCES protocol layering has two pieces: the PL and the TML. Check out Figure 1 🦋
+----------------------------------------------+
| CE PL |
+----------------------------------------------+
| CE TML |
+----------------------------------------------+
^
|
ForCES PL |messages
|
v
+-----------------------------------------------+
| FE TML |
+-----------------------------------------------+
| FE PL |
+-----------------------------------------------+
Figure 1: Message Exchange between CE and FE
to Establish an NE AssociationThe PL handles the ForCES protocol. Its semantics and message layout are defined in [RFC5810]. The TML is what connects the two ForCES endpoints, as shown in Figure 1 ✨
Both the PL and the TML are standardized by the IETF. Only one PL is defined, but multiple TMLs are expected to get standardized. Every TML at each node (CE and FE) is expected to share the same definition so they can interoperate 💅
When a ForCES endpoint transmits, the PL hands its messages down to the TML. The TML then delivers the PL message to the destination TML(s). When a message comes in, the TML passes it up to its destination PL, as described in the ForCES header 🍓
3.1. The PL
The PL is common to all ForCES implementations and is standardized by the IETF in [RFC5810]. It's responsible for associating an FE or CE with an NE, and for tearing down those associations 🧚
An FE can use the PL to send packets to the CE asynchronously. The FE may also redirect various control protocol packets (e.g., OSPF, etc.) to the CE through the PL, coming in from outside the NE. On top of that, the FE delivers various events that the CE has subscribed to, via the PL [RFC5812] 🔥
The CE and FE can also talk synchronously through the PL. The CE sends status requests to the FE and gets responses back through the PL. The CE also configures the components of the associated FE's LFBs using the PL [RFC5812] 💜
3.2. The TML
The TML is responsible for transporting the PL messages 🦋 [RFC5810], Section 5 defines the requirements a TML specification has to meet. The SCTP TML specified in this document meets all the requirements in [RFC5810], Section 5. Section 4.2.2 of this document explains how those TML requirements get met 🫶
3.2.1. TML and PL Interfaces
okay so there are two interfaces hooking up the PL and TML 🎀 this doc doesn't spec them (that's out of scope, bestie), but they're introduced so you can see how they fit into the architecture and what jobs they do. First is the one between the PL and TML. Second is the CE Manager (CEM)/FE Manager (FEM) [RFC3746] interface, which talks to both the PL and TML. Both are in Figure 2 💅
+----------------------------+
| +----------------------+ |
| | | |
+---------+ | | PL | |
| | | +----------------------+ |
|FEM/CEM |<---->| ^ |
| | | | |
+---------+ | |TML API |
| | |
| V |
| +----------------------+ |
| | | |
| | TML | |
| | | |
| +----------------------+ |
+----------------------------+Figure 2: The TML-PL Interface
the CEM/FEM [RFC3746] interface is the one in charge of bootstrapping and parameterizing the TML ✨ in its most basic form it's just a simple static config file that gets read on startup in the pre-association phase. so chill, it's giving simple 🧚
Appendix B goes deeper into the service interfaces if you're curious 🌸
3.2.2. TML Parameterization
it's expected that you should be able to use a configuration reference point, like the FEM or the CEM, to configure the TML 💖
some of the parameters you can configure are:
- PL ID
- Connection Type and associated data. For example, if a TML uses IP/SCTP, then parameters such as SCTP ports and IP addresses need to be configured.
- Number of transport connections
- Connection Capability, such as bandwidth, etc.
- Allowed/Supported Connection Quality of Service (QoS) Policy (or Congestion Control Policy)
4. SCTP TML Overview
okay this section is where it gets juicy 🦋 SCTP [RFC4960] is an end-to-end transport protocol that's basically on par with TCP, UDP, or DCCP in a lot of ways. with a few exceptions, SCTP can do most of what UDP, TCP, or DCCP can do. it can also do most of what a combo of the other transport protocols can do (like TCP and DCCP, or TCP and UDP) 👑
like TCP, it gives you ordered, reliable, connection-oriented, flow-controlled, congestion-controlled data exchange. unlike TCP though, it doesn't do byte streaming. instead it gives you message boundaries 💅
like UDP, it can do unreliable, unordered data exchange. unlike UDP, it does not provide multicast support 🙈
like DCCP, it can do unreliable, ordered, congestion-controlled, connection-oriented data exchange ✨
SCTP also has some extra services that none of those three protocols offer, and we found them super attractive 😭 they include:
- Multi-homing
- Runtime IP address binding
- A range of reliability shades with congestion control
- Built-in heartbeats
- Multi-streaming
- Message boundaries with reliability
- Improved SYN DOS protection
- Simpler transport events
- Simplified replicasting
4.1. Rationale for Using SCTP for TML
so why SCTP? bc it has all the features needed to make a robust TML 🫶 it's an all-encompassing transport, which means you don't need multiple transport protocols to meet the TML requirements ([RFC5810], Section 5). that means simpler coding, and therefore way fewer interoperability concerns 🎀
SCTP is also very mature and widely used, which makes it a good pick for ubiquitous deployment 🌈
4.2. Meeting TML Requirements
PL
+----------------------+
| |
+-----------+----------+
| TML API
TML |
+-----------+----------+
| | |
| +------+------+ |
| | TML core | |
| +-+----+----+-+ |
| | | | |
| SCTP socket API |
| | | | |
| | | | |
| +-+----+----+-+ |
| | SCTP | |
| +------+------+ |
| | |
| | |
| +------+------+ |
| | IP | |
| +-------------+ |
+----------------------+Figure 3: The TML-SCTP Interface
Figure 3 zooms in on the PL and SCTP TML and what's going on inside the SCTP TML 🔍 the TML core talks northbound to the PL (using the TML API). on the southbound side, the TML core connects to the SCTP layer through the standard socket interface [TSVWG-SCTPSOCKET]. and between any two PL endpoints (FE or CE), there are three SCTP socket connections opened 🦋
4.2.1. SCTP TML Channels
+--------------------+
| |
| TML core |
| |
+-+-------+--------+-+
| | |
| Med prio, |
| Semi-reliable |
| channel |
| | Low prio,
| | Unreliable
| | channel
| | |
^ ^ ^
| | |
Y Y Y
High prio,| | |
reliable | | |
channel | | |
Y Y Y
+-+--------+--------+-+
| |
| SCTP |
| |
+---------------------+Figure 4: The TML-SCTP Channels
Figure 4 goes one level deeper into the TML core and SCTP interfacing 🌸 there are three channels that group and prioritize the work for the different kinds of ForCES traffic. each channel is its own SCTP socket interface with different properties. all SCTP channels are congestion aware (that's why that detail is left out of the description of the three channels). the high-, medium-, and lower-priority channels use SCTP ports 6704, 6705, and 6706, respectively. and their SCTP Payload Protocol ID (PPID) values are 21, 22, and 23, in that same order 💜
4.2.1.1. Justifying Choice of Three Sockets
the tea ☕ SCTP lets you send up to 64 K streams over a single socket interface. the authors first pictured using one socket for all three channels (mapping each channel to an SCTP stream). that makes programming the TML simpler and saves on SCTP ports ✨
but then the analysis revealed head-of-line blocking issues with that first approach 😬 lower-priority packets that don't need reliable delivery could block higher-priority packets (the ones needing reliable delivery) under congestion, for an indeterminate period of time (depending on how many outstanding lower-priority packets were pending). so we went with mapping each of the three channels to a different SCTP socket instead of a different stream within one socket 🫶
4.2.1.2. Higher-Priority, Reliable Channel
the higher-priority (HP) channel uses a standard SCTP reliable socket on port 6704 🔥 SCTP PPID 21 is used for all messages on the HP channel. the HP channel is for CE-solicited messages and their responses:
- ForCES configuration messages flowing from CE to FE and responses from the FE to CE.
- ForCES query messages flowing from CE to FE and responses from the FE to the CE.
PL priorities 4-7 MUST be used for all PL messages using this channel. the following PL messages MUST use the HP channel for transport:
- AssociationSetup (default priority: 7)
- AssociationSetupResponse (default priority: 7)
- AssociationTeardown (default priority: 7)
- Config (default priority: 4)
- ConfigResponse (default priority: 4)
- Query (default priority: 4)
- QueryResponse (default priority: 4)
if PL priorities outside the specified range (4-7), the PPID, or PL message types other than the ones above are received on the HP channel, then the PL message MUST be dropped 🚫
although an implementation may pick different values from the defined range (4-7), it is RECOMMENDED that default priorities be used 💅 a response to a ForCES message MUST contain the same priority as the request. for example, a config sent by the CE with priority 5 MUST have a config-response from the FE with priority 5 🎀
4.2.1.3. Medium-Priority, Semi-Reliable Channel
the medium-priority (MP) channel uses SCTP-PR on port 6705 🦋 SCTP PPID 22 MUST be used for all messages on the MP channel. time limits on how long a message is valid are set on each outgoing message. this channel is for events from the FE to the CE that get obsoleted over time. events that are accumulative in nature and recoverable by the CE (by issuing a query to the FE) can tolerate lost events, so they should use this channel. for example, a generated event carrying the value of a monotonically incrementing counter is a perfect fit for this channel 🌈
PL priority 3 MUST be used for PL messages on this channel. the following PL messages MUST use the MP channel for transport:
- Event Notification (default priority: 3)
if PL priorities outside the specified priority, the PPID, or PL message types other than the above are received on the MP channel, then the PL message MUST be dropped 🙅♀️
4.2.1.4. Lower-Priority, Unreliable Channel 🦋
okay so the lower-priority (LP) channel lives on SCTP port 6706 💅 and every message on it uses SCTP PPID 23 🌸. The LP channel also MUST use SCTP-PR with lower timeout values than the MP channel 🧚. Why is this one unreliable?? Bestie, it lets the control protocol at both the CE and its peer-endpoint call the shots on the end-to-end semantics of its own operations 👑. For example:
- Some control protocols are reliable already, so making this channel reliable adds an extra layer of reliability that could be harmful 💀. Any end-to-end retransmits will happen remotely instead 🔁.
- Some control protocols want messages to go obsolete rather than get retransmitted, and a reliable channel goes straight against that wish 🙈.
Also, ForCES PL heartbeats are traffic sensitive, so sending them over the LP channel makes sense too 💖. If the other end isn't busy with other channels, it'll eventually get the heartbeats ✨. If it IS busy with other channels, the heartbeats get obsoleted locally over time, and it doesn't matter if they didn't make it 🫶.
PL priorities 1-2 MUST be used for PL messages on this channel 🎀. The PL messages that MUST use the MP channel for transport are:
- PacketRedirect (default priority: 2)
- Heartbeat (default priority: 1)
If PL priorities outside the specified priority range, PPID, or PL message types other than the above are received on the LP channel, then the PL message MUST be dropped 🚫. No exceptions, bestie.
4.2.1.5. Scheduling of the Three Channels 🌈
When it comes to sending and receiving PL messages, the TML core uses strict priority work-conserving scheduling, as shown in Figure 5 🍓.
This means the HP messages are always processed first until there are none left. The LP channel only gets processed when the channels above it have no messages left to process. So under congestion, a higher-priority channel with enough messages to eat up the available bandwidth will starve the lower-priority channel(s) 😭.
The design intent of the SCTP TML is to tie processing prioritization (as described in Section 4.2.1.1) to transport congestion control, so you get implicit node congestion control 🔥. Appendix A.2 goes deeper on this.
It's worth emphasizing that the work scheduling prioritization here is receiver-based processing. Fully arrived packets on any of the channels are a source of work, and that work may result in transmitted packets. However, we have no control over the order in which the SCTP/OS/network decides to send packets across and make them available to the receiver 🙃. We try to ameliorate this with our choice of channel properties, ForCES message grouping, and by tying CE and FE work scheduling together. That helps with some of these issues, but it doesn't fully resolve all of them.
From a ForCES perspective, we can tolerate some reordering ✨. For example, if an FE transmits a config response (HP) followed by 10000 OSPF redirect packets (LP), and the CE gets 5 OSPF redirects (LP) first before the config response (HP), that's totally fine 👍. What matters is that the CE gets to processing the HP message soon, instead of sitting for long periods processing OSPF packets, which is what would happen if we used a single socket with three streams 😤. This is especially important for dealing with node overload well, as discussed in Section 4.2.2.6 👑.
SCTP channel +----------+
Work available | DONE +---<--<--+
| +---+------+ |
Y ^
| +-->--+ +-->---+ |
+-->-->-+ | | | | |
| | | | | | ^
| ^ ^ v ^ v |
^ / \ | | | | |
| / \ | ^ | ^ ^
| / Is \ | / \ | / \ |
| / there \ | /Is \ | /Is \ |
^ / HP work \ ^ /there\ ^ /there\ ^
| \ ? / | /MP work\ | /LP work\ |
| \ / | \ ? / | \ ? / |
| \ / | \ / | \ / ^
| \ / ^ \ / ^ \ / |
| \ / | \ / | \ / |
^ Y-->-->-->+ Y-->-->-->+ Y->->->-+
| | NO | NO | NO
| | | |
| Y Y Y
| | YES | YES | YES
^ | | |
| Y Y Y
| +----+------+ +---|-------+ +----|------+
| |- process | |- process | |- process |
| | HP work | | MP work | | LP work |
| +------+----+ +-----+-----+ +-----+-----+
| | | |
^ Y Y Y
| | | |
| Y Y Y
+--<--<---+--<--<----<----+-----<---<-----+
Figure 5: SCTP TML Strict Priority Scheduling4.2.1.6. SCTP TML Parameterization 🎀
Here's the list of parameters needed for booting the TML 🧁. They're expected to be extracted via the FEM/CEM interface for each PL ID.
- The IP address(es) or a resolvable DNS/hostname(s) of the CE/FE.
- Whether or not to use IPsec. If IPsec is used, how to parameterize the different required ciphers, keys, etc., as described in Section 7.1.
- The HP SCTP port, as discussed in Section 4.2.1.2. The default HP port value is 6704 (Section 6).
- The MP SCTP port, as discussed in Section 4.2.1.3. The default MP port value is 6705 (Section 6).
- The LP SCTP port, as discussed in Section 4.2.1.4. The default LP port value is 6706 (Section 6).
4.2.2. Satisfying TML Requirements 💜
[RFC5810], Section 5 lists the requirements a TML needs to meet 📋. This section describes how the SCTP TML satisfies those requirements 🫶.
4.2.2.1. Satisfying Reliability Requirement ✨
As mentioned earlier, a shade of reliability ranges is possible in SCTP 🌈. Therefore, this requirement is met 💅.
4.2.2.2. Satisfying Congestion Control Requirement 🦋
Congestion control is built into SCTP 🔥. Therefore, this requirement is met 👑.
4.2.2.3. Satisfying Timeliness and Prioritization Requirement ⭐
By using three sockets in conjunction with the partial-reliability feature [RFC3758], both the timeliness and prioritization requirements are addressed 🎀.
4.2.2.4. Satisfying Addressing Requirement 🧚
There are no extra headers required for SCTP to fulfill this requirement 🙌. SCTP can be told to replicast packets to multiple destinations. The TML implementation will need to translate PL addresses to a variety of unicast IP addresses in order to emulate multicast and broadcast PL addresses 🍓.
4.2.2.5. Satisfying High-Availability Requirement 🛡️
Transport link resiliency is one of SCTP's strongest points 💪. Failure detection and recovery is built in, as mentioned earlier.
- The SCTP multi-homing feature is used to provide path diversity 🛤️. Should one of the peer IP addresses become unreachable, the others are used without needing lower-layer convergence (routing, for example) or even the TML becoming aware 🫠.
- SCTP heartbeats and data transmission thresholds are used on a per-peer IP address to detect reachability faults 💖. The faults could be a result of an unreachable address or peer, which may be caused by a variety of reasons, like interface, network, or endpoint failures. The cause of the fault is noted 📝.
- With the ADDIP feature, one can migrate IP addresses to other nodes at runtime 🦋. This is not unlike the Virtual Router Redundancy Protocol (VRRP) [RFC5798] use. This feature is used in addition to multi-homing in a planned migration of activity from one FE/CE to another 🧁. In such a case, part of the provisioning recipe at the CE for replacing an FE involves migrating activity of one FE to another.
4.2.2.6. Satisfying Node Overload Prevention Requirement 🔥
The architecture of this TML defines three separate channels, one per socket, to be used within any FE-CE setup 🎀. The work scheduling design for processing the TML channels (Section 4.2.1.5) is a strict priority. A fundamental desire of strict prioritization is to make sure that more important processing work always gets node resources over less important work 👑.
When a ForCES node CPU is overwhelmed because the incoming packet rate is higher than it can keep up with, the channel queues grow and transport congestion follows 😩. By virtue of using SCTP, the congestion is propagated back to the source of the incoming packets and eventually alleviated ✨.
The HP channel work gets prioritized at the expense of the MP, which gets prioritized over the LP channels 💅. The preferential scheduling only kicks in when there is node overload, regardless of whether there is transport congestion. As a result of this preferential work treatment, the ForCES node achieves a robust steady processing capacity 💪. Refer to Appendix A.2 for details on scheduling 🧠.
For an example of how overload prevention works, consider a scenario where an overwhelming amount of redirected packets (from outside the NE) coming into the NE may overload the FE while it has outstanding config work from the CE 🙈. In such a case, the FE, while it is busy processing config requests from the CE, essentially ignores the redirect packets on the LP channel. If enough redirect packets accumulate, they are dropped either because the LP channel threshold is exceeded or because they are obsoleted 😭. If, on the other hand, the FE has successfully processed the higher-priority channels and their related work, then it can proceed and process the LP channel 🙌. So, as demonstrated here, the TML ties transport congestion and node overload together implicitly 🫶.
4.2.2.7. Satisfying Encapsulation Requirement 🎁
The SCTP TML sets SCTP PPIDs to identify channels, as described in Section 4.2.1.1 🏷️.
5. SCTP TML Channel Work ✨
There are two levels of TML channel work within an NE when a ForCES node (CE or FE) is connected to multiple other ForCES nodes 🌸:
- NE-level I/O work, where a ForCES node (CE or FE) needs to choose which of the peer nodes to process.
- Node-level I/O work, where a ForCES node handles the three SCTP TML channels separately for each single ForCES endpoint.
NE-level scheduling definition is left up to the implementation and is considered out of scope for this document. Appendix A.4 briefly discusses some constraints an implementer needs to worry about 🧐.
This document provides suggestions on SCTP channel work implementation in Appendix A 💡.
The FE SHOULD do channel connections to the CE in the order of incrementing priorities, i.e., LP socket first, followed by MP, and ending with HP socket connection 🫶. The CE, however, MUST NOT assume that there is ordering of socket connections from any FE 👀.
6. IANA Considerations ✨
Okay, so the IANA stuff follows the rules from "Guidelines for Writing an IANA Considerations Section in RFCs" [RFC5226] 📋. Here are the namespaces ForCES SCTP TML defines 🎀:
- SCTP port
6704for the HP channel,6705for the MP channel, and6706for the LP channel. - SCTP Payload Protocol ID (PPID)
21for the HP channel (ForCES-HP),22for the MP channel (ForCES-MP), and23for the LP channel (ForCES-LP) 💜.
7. Security Considerations 🔐
Okay, this section is serious business, bestie 😭. The SCTP TML gives the PL three security services 🛡️:
- A way to authenticate ForCES CEs and FEs at the transport level, so unauthorized CEs and unauthorized FEs can't join the control and data path processing of a ForCES NE 🚫.
- A way to make sure PL data and headers moving from the CE to the FE (and back) have message authentication, so nobody can inject bad data into PL messages 💀.
- A way to keep PL data and headers moving from the CE to the FE (and back) confidential, so PL information carried over the TML can't be disclosed 🤫.
The operator picks the security choices the TML uses, and they kick in during the pre-association phase of the ForCES protocol 🎯. The operator can use all, some, or none of the security services in a CE-FE connection, totally their call 💅.
If the environment is already secured, or there are other operational concerns (sometimes performance issues), the operator can turn off all the security functions between the CE and FE 🌸.
IP Security Protocol (IPsec) [RFC4301] is what provides the security mechanisms we need 🦋.
IPsec is an IP-level security scheme that's transparent to higher-layer applications, so it can secure any transport layer protocol 🧚. This is the iconic part: IPsec can secure everything between the CE and FE without the TML implementation having to know the details 👑.
The IPsec architecture is built to provide the message integrity and message confidentiality from the TML security requirements [RFC5810] ✨. Mutual authentication and the key exchange protocol come from Internet Key Exchange (IKE) [RFC2409] 🔑.
7.1. IPsec Usage
A ForCES FE or CE MUST support the following:
- Internet Key Exchange (IKE)[RFC2409] with certificates for endpoint authentication 🪪.
- Transport Mode Encapsulating Security Payload (ESP) [RFC4303] 📦.
HMAC-SHA1-96[RFC2404] for message integrity protection 🧾.AES-CBCwith 128-bit keys [RFC3602] for message confidentiality 🔒.- Replay protection [RFC4301] 🔁.
A compliant implementation SHOULD provide operational means for configuring the CE and FE to negotiate other cipher suites, and even use manual keying 🎛️. Main character energy, honestly 💖.
7.1.1. SAD and SPD Setup
To keep the operational configuration minimal, it is RECOMMENDED that only the IANA-issued SCTP protocol number (132) be used as a selector in the Security Policy Database (SPD) for ForCES 🍓. In that case, only a single SPD and SAD entry is needed ✨.
Setup MAY alternatively extend the above policy so it uses the three SCTP TML port numbers as SPD selectors 🌈. But as noted above, this choice requires an increased number of SPD entries 📈.
If there are multiple IP addresses in a single association, and you want different policies per IP address, then following [RFC3554] is RECOMMENDED 🗺️.
8. Acknowledgements 🫶
The authors want to thank Joel Halpern, Michael Tuxen, Randy Stewart, Evangelos Haleplidis, Chuanhuang Li, Lars Eggert, Avshalom Houri, Adrian Farrel, Juergen Quittek, Magnus Westerlund, and Pasi Eronen for getting into discussions that made this document better 💜.
Ross Callon was an excellent manager who kept giving the authors guidance 👑. Joel Halpern was an excellent document shepherd, and without him this document would have taken way longer to publish 🙌.
9. References 📚
9.1. Normative References
- [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
- [RFC2404] Madson, C. and R. Glenn, "The Use of HMAC-SHA-1-96 within ESP and AH", RFC 2404, November 1998.
- [RFC2409] Harkins, D. and D. Carrel, "The Internet Key Exchange (IKE)", RFC 2409, November 1998.
- [RFC3554] Bellovin, S., Ioannidis, J., Keromytis, A., and R. Stewart, "On the Use of Stream Control Transmission Protocol (SCTP) with IPsec", RFC 3554, July 2003.
- [RFC3602] Frankel, S., Glenn, R., and S. Kelly, "The AES-CBC Cipher Algorithm and Its Use with IPsec", RFC 3602, September 2003.
- [RFC3758] Stewart, R., Ramalho, M., Xie, Q., Tuexen, M., and P. Conrad, "Stream Control Transmission Protocol (SCTP) Partial Reliability Extension", RFC 3758, May 2004.
- [RFC4301] Kent, S. and K. Seo, "Security Architecture for the Internet Protocol", RFC 4301, December 2005.
- [RFC4303] Kent, S., "IP Encapsulating Security Payload (ESP)", RFC 4303, December 2005.
- [RFC4960] Stewart, R., "Stream Control Transmission Protocol", RFC 4960, September 2007.
- [RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 5226, May 2008.
- [RFC5810] Doria, A., Ed., Hadi Salim, J., Ed., HAAS, R., Ed., Khosravi, H., Ed., Wang, W., Ed., Dong, L., Gopal, R., and J. Halpern, "Forwarding and Control Element Separation (ForCES) Protocol Specification", RFC 5810, March 2010.
9.2. Informative References
- [RFC3654] Khosravi, H. and T. Anderson, "Requirements for Separation of IP Control and Forwarding", RFC 3654, November 2003.
- [RFC3746] Yang, L., Dantu, R., Anderson, T., and R. Gopal, "Forwarding and Control Element Separation (ForCES) Framework", RFC 3746, April 2004.
- [RFC5812] Halpern, J. and J. Hadi Salim, "Forwarding and Control Element Separation (ForCES) Forwarding Element Model", RFC 5812, March 2010.
- [RFC5798] Nadas, S., Ed., "Virtual Router Redundancy Protocol (VRRP) Version 3 for IPv4 and IPv6", RFC 5798, March 2010.
- [TSVWG-SCTPSOCKET] Stewart, R., Poon, K., Tuexen, M., Yasevich, V., and P. Lei, "Sockets API Extensions for Stream Control Transmission Protocol (SCTP)", Work in Progress, March 2010.
Appendix A. Suggested SCTP TML Channel Work Implementation 🧜♀️
As mentioned in Section 5, there are two levels of TML channel work inside an NE when a ForCES node (CE or FE) is connected to multiple other ForCES nodes 🕸️:
- NE-level I/O work, where a ForCES node (CE or FE) has to choose which of the peer nodes to process.
- Node-level I/O work, where a ForCES node handles the three SCTP TML channels separately for each single ForCES endpoint.
NE-level scheduling is left up to the implementation and is out of scope for this document 🙃. Appendix A.4 gives a quick rundown of some constraints an implementer needs to worry about 👀.
This document, and in particular Appendix A.1, Appendix A.2, and Appendix A.3, discuss the details of node-level I/O work 💅.
A.1. SCTP TML Channel Initialization
As discussed in Section 5, it is recommended that the FE SHOULD make socket connections to the CE in order of increasing priority: LP socket first, then MP, and ending with the HP socket connection 🚀. The CE, however, MUST NOT assume there is any ordering of socket connections from any FE 🙅♀️. Appendix B.1 has more details on the expected initialization of SCTP channel work 📖.
A.2. Channel Work Scheduling
This section gives high-level details on the scheduling view of the SCTP TML core (Section 4.2.1) 🗓️. A practical scheduler implementation takes care of lots of little details (like timers, work quanta, and so on) that aren't described in this document, and it's left to the implementer to handle those 🛠️.
The CE(s) and FE(s) are coupled together, following the principles of the scheduling scheme described here, to tie node overload together with transport congestion 🔗. The design intent is to provide the highest possible robust work throughput for the NE under any network or processing congestion 🏆.
A.2.1. FE Channel Work Scheduling
The FE scheduling, in priority order, needs to I/O process:
- The HP channel I/O, in the following priority order:
- Transmitting back to the CE any outstanding result of executed work via the HP channel transmit path 📤.
- Taking new incoming work from the CE that creates ForCES work to be executed by the FE 📥.
- ForCES events that result in transmission of unsolicited ForCES packets to the CE via the MP channel 💌.
- Incoming Redirect work in the form of control packets that come from the CE via the LP channel. After redirect processing, these packets get sent out on the external (to the NE) interface 🛤️.
- Incoming Redirect work in the form of control packets that come from other NEs via external (to the NE) interfaces. After some processing, these packets are sent to the CE 🔁.
Let me say it again because it's important 📢: the SCTP TML processes channel work in strict priority. For example, as long as there are messages to send to the CE on the HP channel, they get processed first, until there are none left, before the next priority work (which is reading new messages on the HP channel incoming from the CE) 👑.
A.2.2. CE Channel Work Scheduling
The CE scheduling, in priority order, needs to deal with:
- The HP channel I/O, in the following priority order:
- Processing incoming responses to requests of work it made to the FE(s) 📬.
- Transmitting any outstanding HP work it needs the FE(s) to complete 📦.
- Incoming ForCES events from the FE(s) via the MP channel 🎉.
- Outgoing Redirect work in the form of control packets that get sent from the CE via the LP channel, destined to external (to the NE) interfaces on the FE(s) 🌐.
- Incoming Redirect work in the form of control packets that come from other NEs via external interfaces (to the NE) on the FE(s) 💫.
It's worth repeating, for emphasis 💖: the SCTP TML processes channel work in strict priority. For example, if there are messages incoming from an FE on the HP channel, they get processed first until there are none left, before the next priority work, which is transmitting any outstanding HP channel messages going to the FE 🫧.
A.3. SCTP TML Channel Termination 🌸
Okay so, Appendix B.2 walks through a controlled disassociation of the FE from the NE. Super useful to know about 💅
It's also possible for connectivity to be lost between the FE and CE on one or more sockets 😭 If SCTP multi-homing features are being used for path availability, a socket only disconnects if all paths are unreachable. Otherwise, SCTP keeps reachability going for you ✨ So in the situation of a total connectivity loss of even one SCTP socket, it is recommended that the FE and CE SHOULD assume a state equivalent to ForCES Association Teardown being issued and follow the sequence described in Appendix B.2 🦋
A CE could also disconnect sockets to an FE to signal an "emergency teardown" 🚨 This can be needed when a CE has to disconnect an FE but knows the FE is slammed, busy processing a ton of outstanding commands (some of which it hasn't gotten around to yet) 🙈 Because the CE is closing the connections, the FE gets notified immediately and asynchronously, so it won't have to process any outstanding commands from the CE. Iconic behaviour, honestly 👑
A.4. SCTP TML NE-Level Channel Scheduling 🧚
When it comes to NE-level I/O work, an implementation has to think about being both fair and robust across peer ForCES nodes 💜
Fairness matters because every peer node should be able to make progress across the NE 🍓 Picture two FEs connected to a CE. One FE has a few HP messages waiting to be processed by the CE, while the other might have infinite HP messages. The scheduling scheme may decide to use a quota scheduling system so the second FE doesn't hog the CE cycles. Main character energy for everyone, but not too much 💅
Robustness matters so the NE doesn't succumb to a Denial-of-Service (DoS) attack from hostile entities, and so it always reaches a maximum stable workload processing level 🛡️ Again, picture two FEs connected to a CE. Say FE1 has a large number of HP and MP messages, and FE2 has a large number of MP and LP messages. The scheduling scheme needs to make sure that while FE1 always gets its messages processed, at some point FE2's messages get processed too 🫶 A promotion and preemption-based scheduling could be used by the CE to resolve this issue ✨
Appendix B. Suggested Service Interface 🎀
This section outlines a high-level service interface between FEM/CEM and TML, between the PL and TML, and between local and remote TMLs. The intent here is to give general guidelines, so the vibe is "big picture" rather than "every detail." The implementer is expected to care of details and even follow a different approach if needed 🌈
The theory of operation for the PL-TML service goes like this:
- The PL starts up and bootstraps the TML. The end result of a successful TML bootstrap is that the CE TML and the FE TML connect to each other at the transport level.
- Transmission and reception of the PL messages commences after a successful TML bootstrap. The PL uses send and receive PL-TML interfaces to communicate with its peers. The TML is agnostic to the nature of the messages being sent or received. The first message exchanges that happen are to establish ForCES association. Subsequent messages may be either unsolicited events from the FE PL, control message redirects to/from the CE to/from FE, or configuration from the CE to the FE, and their responses flowing from the FE to the CE 💖
- The PL does a shutdown of the TML after terminating ForCES association 🔥
B.1. TML Bootstrapping 🌸
Figure 6 illustrates a flow for the TML bootstrapped by the PL ✨
When the PL starts up (possibly after some internal initialization), it boots up the TML. The TML first interacts with the FEM/CEM and acquires the necessary TML parameterization (Section 4.2.1.6). Next, the TML uses the information it retrieved from the FEM/CEM interface to initialize itself 🧚
The TML on the FE proceeds to connect the three channels to the CE. The socket interface is used for each of the channels. The TML continues to re-try the connections to the CE until all three channels are connected. It is advisable that the number of connection retry attempts and the time between each retry is also configurable via the FEM 💅 On failure to connect one or more channels, and after the configured number of retry thresholds is exceeded, the TML will return an appropriate failure indicator to the PL. On success (as shown in Figure 6), a success indication is presented to the PL 🦋
FE PL FE TML FEM CEM CE TML CE PL
| | | | | |
| | | | | Bootup |
| | | | |<-------------------|
| Bootup | | | | |
|----------->| | |get CEM info| |
| |get FEM info | |<-----------| |
| |------------>| ~ ~ |
| ~ ~ |----------->| |
| |<------------| | |
| | |-initialize TML |
| | |-create the 3 chans.|
| | | to listen to FEs |
| | | |
| |-initialize TML |Bootup success |
| |-create the 3 chans. locally |------------------->|
| |-connect 3 chans. remotely | |
| |------------------------------>| |
| ~ ~ - FE TML connected ~
| ~ ~ - FE TML info init ~
| | channels connected | |
| |<------------------------------| |
| Bootup | | |
| succeeded | | |
|<-----------| | |
| | | |
Figure 6: SCTP TML BootstrappingOn the CE, things are slightly different 🌈 After initializing from the CEM, the TML on the CE side proceeds to initialize the three channels to listen to remote connections from the FEs. The success or failure indication is passed on to the CE PL (in the same manner as was done in the FE) ✨
Post bootup, the CE TML waits for connections from the FEs. Upon a successful connection by an FE, the CE TML level keeps track of the transport-level details of the FE. Note, at this stage only a transport-level connection has been established. ForCES-level association follows using send/receive PL-TML interfaces (refer to Appendix B.3 and Figure 8) 🫶
B.2. TML Shutdown 💀
Figure 7 shows an example of an FE shutting down the TML. It is assumed at this point that the ForCES Association Teardown has been issued by the CE. It should also be noted that different implementations may have different procedures for cleaning up state, etc. 🙈
When the FE PL issues a shutdown to its TML for a specific PL ID, the TML releases all the channel connections to the CE. This is achieved by closing the sockets used to communicate with the CE. This results in the stack sending a SCTP shutdown, which is received on the CE 🦋
FE PL FE TML CE TML CE PL
| | | |
| Shutdown | | |
|----------->| | |
| |-disconnect 3 chans. | |
| |-SCTP level shutdown | |
| |------------------------>| |
| | | |
| | |TML detects shutdown|
| | |-FE TML info cleanup|
| | |-optionally tell PL |
| | |------------------->|
| | | |
| |- clean up any state of | |
| |-channels disconnected | |
| |<------------------------| |
| |-SCTP shutdown ACK | |
| | | |
| Shutdown | | |
| succeeded | | |
|<-----------| | |
| | | |
Figure 7: FE Shutting DownOn the CE side, a TML disconnection would result in possible cleanup of the FE state. Optionally, depending on the implementation, there may be a need to inform the PL about the TML disconnection 🧚 The CE-stack-level SCTP sends an acknowledgement to the FE TML in response to the earlier SCTP shutdown ✨
B.3. TML Sending and Receiving 🎀
Okay bestie, the TML should be totally agnostic to what's inside the PL messages or what they're doing 🙈 The PL needs to spill enough info so the TML can assign an appropriate priority and loss behavior to each message 💅 Figure 8 shows a message going from the FE to the CE (like a ForCES association message!) and covers all the service interfaces you need for sending and receiving ✨
When the FE PL sends a message to the TML, the TML is expected to pick one of HP/MP/LP channels and send out the ForCES message 🦋
FE PL FE TML CE TML CE PL
| | | |
|PL send | | |
|----------->| | |
| | | |
| | | |
| |-pick channel | |
| |-TML Send | |
| |------------->| |
| | | |
| | |-TML Receive on chan. |
| | |- mux to PL/PL recv |
| | |--------------------->|
| | | ~
| | | ~ PL Process
| | | ~
| | | PL send |
| | |<---------------------|
| | |-pick chan to send on |
| | |-TML send |
| |<-------------| |
| |-TML Receive | |
| |-mux to PL | |
| PL Recv | | |
|<---------- | | |
| | | |
Figure 8: Send and Recv FlowWhen the CE TML receives the ForCES message on the channel it was sent on, it demultiplexes the message to the CE PL 🫶
The CE PL does its thing after some processing (in this example, dealing with the FE's association) and then sends the TML a response 💜 Same as the FE PL, the CE TML picks the channel to send on before sending 👑
Processing a ForCES message when it arrives at the FE TML and gets delivered to the FE PL is similar to the CE side equivalent, as shown above in Appendix B.3 💖
Authors' Addresses 🌸
Jamal Hadi Salim Mojatatu Networks Ottawa, Ontario Canada
EMail: hadi@mojatatu.com
Kentaro Ogawa NTT Corporation 3-9-11 Midori-cho Musashino-shi, Tokyo 180-8585 Japan
EMail: ogawa.kentaro@lab.ntt.co.jp 🧚
📦 where this page comes from
- ietf: rfc5811 2010-03
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms