RFC 6053
Implementation Report for Forwarding and Control Element Separation (ForCES)
Cover Info 🎀
Okay bestie, let's start with the cover info 💅. This is RFC 6053, called Implementation Report for Forwarding and Control Element Separation (ForCES). It's by E. Haleplidis (University of Patras), K. Ogawa (NTT Corporation), W. Wang (Zhejiang Gongshang University), and J. Hadi Salim (Mojatatu Networks). It's an Informational doc with ISSN 2070-1721, dated November 2010. It doesn't list anything it obsoletes or updates, so no drama there 🦋.
Abstract 💖
Okay so ForCES (Forwarding and Control Element Separation) is the framework and protocols that standardize how the control plane and the forwarding plane talk to each other inside a ForCES network element (ForCES NE) 🧚. RFC 3654 wrote the ForCES requirements, and RFC 3746 wrote the ForCES framework, so those are the OG docs 👑.
This document is an implementation report 📋. It covers the ForCES Protocol, the ForCES Model, and the Stream Control Transmission Protocol-based Transport Mapping Layer (SCTP TML) documents. It also includes a report on interoperability testing and where ForCES implementations are at right now ✨.
Status of This Memo 📜
This doc is not an Internet Standards Track specification. It's published for informational purposes, so it's giving "FYI" energy 🌸.
This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community, got public review, and was approved for publication by the Internet Engineering Steering Group (IESG). Not every doc the IESG approves is a candidate for any level of Internet Standard, so check Section 2 of RFC 5741 if you want the details 🙈.
For the current status, any errata, and how to send feedback, go to http://www.rfc-editor.org/info/rfc6053 🔗.
Copyright Notice 🌈
Copyright (c) 2010 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication. Please read these carefully, bestie, because they spell out your rights and restrictions for this doc. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they come without warranty as described in the Simplified BSD License 💜.
Table of Contents 📚
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.1. ForCES Protocol . . . . . . . . . . . . . . . . . . . . . 4
1.2. ForCES Model . . . . . . . . . . . . . . . . . . . . . . . 4
1.3. Transport Mapping Layer . . . . . . . . . . . . . . . . . 4
2. Terminology and Conventions . . . . . . . . . . . . . . . . . 4
2.1. Requirements Language . . . . . . . . . . . . . . . . . . 4
2.2. Definitions . . . . . . . . . . . . . . . . . . . . . . . 5
3. Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
4. Methodology . . . . . . . . . . . . . . . . . . . . . . . . . 6
5. Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . 7
6. Detail Section . . . . . . . . . . . . . . . . . . . . . . . . 7
6.1. Implementation Experience . . . . . . . . . . . . . . . . 7
6.1.1. ForCES Protocol Features . . . . . . . . . . . . . . . 9
6.1.1.1. Protocol Messages . . . . . . . . . . . . . . . . 9
6.1.1.2. MainHeader Handling . . . . . . . . . . . . . . . 10
6.1.1.3. TLV Handling . . . . . . . . . . . . . . . . . . . 11
6.1.1.4. Operation Types Supported . . . . . . . . . . . . 12
6.1.1.5. ForCES Protocol Advanced Features . . . . . . . . 13
6.1.2. ForCES Model Features . . . . . . . . . . . . . . . . 14
6.1.2.1. Basic Atomic Types Supported . . . . . . . . . . . 14
6.1.2.2. Compound Types Supported . . . . . . . . . . . . . 15
6.1.2.3. LFBs Supported . . . . . . . . . . . . . . . . . . 15
6.1.3. ForCES SCTP TML Features . . . . . . . . . . . . . . . 19
6.1.3.1. TML Priority Ports . . . . . . . . . . . . . . . . 19
6.1.3.2. Message Handling at Specific Priorities . . . . . 19
6.1.3.3. TML Security Feature . . . . . . . . . . . . . . . 20
6.2. Interoperability Report . . . . . . . . . . . . . . . . . 20
6.2.1. Scenarios . . . . . . . . . . . . . . . . . . . . . . 21
6.2.1.1. Scenario 1 - Pre-Association Setup . . . . . . . . 21
6.2.1.2. Scenario 2 - TML Priority Channels Connection . . 22
6.2.1.3. Scenario 3 - Association Setup - Association
Complete . . . . . . . . . . . . . . . . . . . . . 22
6.2.1.4. Scenario 4 - CE Query . . . . . . . . . . . . . . 23
6.2.1.5. Scenario 5 - Heartbeat Monitoring . . . . . . . . 23
6.2.1.6. Scenario 6 - Simple Config Command . . . . . . . . 23
6.2.1.7. Scenario 7 - Association Teardown . . . . . . . . 24
6.2.2. Tested Features . . . . . . . . . . . . . . . . . . . 25
6.2.2.1. ForCES Protocol Features . . . . . . . . . . . . . 25
6.2.2.2. ForCES Model Features . . . . . . . . . . . . . . 28
6.2.2.3. ForCES SCTP TML Features . . . . . . . . . . . . . 30
6.2.3. Interoperability Results . . . . . . . . . . . . . . . 31
7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 32
8. Security Considerations . . . . . . . . . . . . . . . . . . . 33
9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 33
9.1. Normative References . . . . . . . . . . . . . . . . . . . 33
9.2. Informative References . . . . . . . . . . . . . . . . . . 331. Introduction ✨
This doc is an implementation report for the ForCES protocol, model, and the SCTP TML documents, and it includes an interoperability report 🫶.
It follows the outline suggested by [RFC5657] 📝.
ForCES defines an architectural framework and associated protocols to standardize information exchange between the control plane and the forwarding plane in a ForCES network element (ForCES NE). [RFC3654] has defined the ForCES requirements, and [RFC3746] has defined the ForCES framework 🌸.
1.1. ForCES Protocol 🔥
Here's the vibe: the ForCES protocol works in a master-slave mode. Forwarding elements (FEs) are the slaves and control elements (CEs) are the masters 👑. The protocol includes commands for transporting Logical Functional Block (LFB) configuration information, association setup, status, event notifications, and more 🦋. Go read the ForCES Protocol Specification [RFC5810] if you want the full story 💜.
1.2. ForCES Model 🧚
The ForCES Model [RFC5812] gives you a formal way to define FE Logical Functional Blocks (LFBs) using XML 🎀. LFB configuration components, capabilities, and associated events get defined when the LFB is formally created. Then the ForCES protocol controls the LFBs inside the FE in a standardized way 💅.
1.3. Transport Mapping Layer 🌈
The TML transports the protocol layer (PL) messages [RFC5810]. This is where things like transport-level reliability, congestion control, multicast, ordering, and so on get handled 🍓.
Here's the key bit: all ForCES protocol layer implementations MUST be portable across all TMLs. More than one TML might be standardized for the ForCES protocol, but all implementations MUST implement SCTP TML [RFC5811] 🔥✨.
2. Terminology and Conventions 📖
2.1. Requirements Language 💅
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119] 🫶.
2.2. Definitions 🦋
This doc follows the terminology from the ForCES requirements in [RFC3654] and the ForCES framework in [RFC3746]. The definitions are repeated below so you don't have to hunt for them 🔍.
- Control Element (CE) - A logical entity that implements the ForCES protocol and uses it to instruct one or more FEs on how to process packets. CEs handle things like running control and signaling protocols 🧠.
- Forwarding Element (FE) - A logical entity that implements the ForCES protocol. FEs use the underlying hardware to do per-packet processing and handling, as directed or controlled by one or more CEs through the ForCES protocol 📦.
- LFB (Logical Functional Block) - The basic building block that the ForCES protocol operates on. An LFB is a well-defined, logically separable functional block that lives in an FE and is controlled by a CE through the ForCES protocol. An LFB may sit on the FE's datapath and process packets, or it may be purely an FE control or configuration entity that a CE operates on. Also, the LFB is a functionally accurate abstraction of the FE's processing capabilities, but it is not a hardware-accurate representation of the FE implementation 🎀.
- LFB Class and LFB Instance - LFBs are sorted into LFB Classes. An LFB Instance represents the existence of an LFB Class (or Type). An FE can have multiple instances of the same LFB Class (or Type). An LFB Class is identified by an LFB Class ID, and an LFB Instance is identified by an LFB Instance ID. So an LFB Class ID paired with an LFB Instance ID uniquely specifies one LFB existence 🌸.
- LFB Metadata - Metadata is used to pass per-packet state from one LFB to another, but it is never sent across the network. The FE model defines how this metadata is identified, produced, and consumed by the LFBs. It defines the functionality but not how metadata is encoded inside an implementation 💜.
- LFB Components - The operational parameters of the LFBs that CEs must be able to see are modeled in the FE model as LFB components. These include things like flags, single-parameter arguments, complex arguments, and tables that the CE can read and/or write through the ForCES protocol (see below) ✨.
- ForCES Protocol - Lots of protocols can be used across the overall ForCES architecture, but "ForCES protocol" and "protocol" in this doc refer to the "Fp" reference points in the ForCES framework in [RFC3746]. This protocol does not apply to CE-to-CE communication, FE-to-FE communication, or communication between FE and CE managers. Basically, the ForCES protocol is master-slave, with FEs as slaves and CEs as masters 👑.
- ForCES Protocol Transport Mapping Layer (ForCES TML) - A layer in the ForCES protocol architecture that uses existing transport protocols to handle protocol message transport. It covers how protocol messages get mapped to different transport media (like TCP, IP, ATM, Ethernet, etc.) and how reliability, multicast, ordering, and so on get achieved and implemented. Each TML has its own separate ForCES document, so check those out too 🦋🌈.
3. Summary ✨
okay so bestie, three independent implementations were surveyed: NTT Japan, the University of Patras, and Zhejiang Gongshang University 🎀. they were all already implementing the major features (no cap, they ate 💅). every single one of them said they're gonna build the missing features later too 🌸
an interop test happened in July 2009 with all three implementations 🦋. then two more orgs joined the chat: Mojatatu Networks and Hangzhou Baud Information and Networks Technology Corporation. they each independently extended two well known public domain protocol analyzers, Ethereal/Wireshark and tcpdump, and that's how the total got to five independent organizations implementing 😭🔥. the two analyzers were used to check that the ForCES protocol messages were valid (and in some cases the semantics too) 🧚
the interop test went smoothly, honestly iconic 👑. there were no notable difficulties, and almost all the issues were code bugs that got squashed mostly on site 💖. the tests then passed again when repeated, as mentioned in Section 6.2.3 ✨
4. Methodology 💜
this report is basically two things: an implementation experience survey, plus the results of the interop test 🫶
the survey info was collected after implementors filled out a short questionnaire about all the ForCES Protocol, Model, and SCTP TML features 📝. you can see the results in Section 6.1 🍓
the interop results came from the interop test itself. extended Ethereal and extended tcpdump were used to verify them 🔍. check Section 6.2 for those results 🦋
5. Exceptions 🌈
the core features of the ForCES Protocol, Model, and SCTP TML were implemented and assessed in an interop test in July 2009 💅. the whole point was to confirm that all the main features of the three core documents work together across different implementations 🫶. the tested features are listed in Section 6.2.2 ✨
different orgs have implemented certain features but not others, and that's totally driven by which LFBs each org is currently implementing 🧩. everyone surveyed said they plan to implement all the outstanding features eventually 🌸. the implemented features are in Section 6.1 💖
the mandated TML security requirement, IP security (IPsec), was NOT validated during the interop, and this document doesn't discuss it 🙈. the thing is, IPsec is super well known and widely deployed, so not testing with IPsec doesn't invalidate the tests that were done 💅. also, Section 6.1.3.3 shows that none of the reporting implementations included IPsec support, though they all said they intend to implement it 😭
one more tea ☕: the SCTP priority ports changed since the interop test, compared to the version of the SCTP TML draft that was around before RFC 5811 got published. but the change has zero impact on the validity of the interop test 💀✨
6. Detail Section 🎀
6.1. Implementation Experience 🧚
three different organizations have implemented the ForCES Protocol, Model, and SCTP TML and answered the questionnaire. they are:
- NTT Japan 🇯🇵
- University of Patras 🏛️
- Zhejiang Gongshang University 🌸
extensions to protocol analyzers that can understand ForCES protocol messages count as part of an implementation, because these analyzers can now understand and validate the ForCES protocol messages that got exchanged 🔍. two of those extensions were created:
- Extension to Ethereal/Wireshark [ethereal].
- Extension to tcpdump [tcpdump].
every implementor was asked which ForCES features they'd implemented 📋. for each item on the list, they said whether they had implemented it, will implement it, or won't implement it at all 💅
6.1.1. ForCES Protocol Features 💖
6.1.1.1. Protocol Messages 📬
+------------------+-------------+---------------+------------------+
| Protocol Message | NTT Japan | University of | Zhejiang |
| | | Patras | Gongshang |
| | | | University |
+------------------+-------------+---------------+------------------+
| Association | Implemented | Implemented | Implemented |
| Setup | | | |
| | | | |
| Association | Implemented | Implemented | Implemented |
| Setup Response | | | |
| | | | |
| Association | Implemented | Implemented | Implemented |
| Teardown | | | |
| | | | |
| Config | Implemented | Implemented | Implemented |
| | | | |
| Config Response | Implemented | Implemented | Implemented |
| | | | |
| Query | Implemented | Implemented | Implemented |
| | | | |
| Query Response | Implemented | Implemented | Implemented |
| | | | |
| Event | Implemented | Will | Implemented |
| Notification | | Implement | |
| | | | |
| Packet Redirect | Implemented | Will | Implemented |
| | | Implement | |
| | | | |
| Heartbeat | Implemented | Implemented | Implemented |
+------------------+-------------+---------------+------------------+
ForCES Protocol Messages6.1.1.2. MainHeader Handling 🎀
+-----------------+-------------+----------------+------------------+
| Header Field | NTT Japan | University of | Zhejiang |
| | | Patras | Gongshang |
| | | | University |
+-----------------+-------------+----------------+------------------+
| Correlator | Implemented | Implemented | Implemented |
| | | | |
| ACK Indicator | Implemented | Implemented | Implemented |
| Flag | | | |
| | | | |
| Priority Flag | Will | Implemented | Implemented |
| | Implement | | |
| | | | |
| Execution Mode | Will | Will Implement | Implemented |
| Flag | Implement | | |
| | | | |
| Atomic | Will | Will Implement | Implemented |
| Transaction | Implement | | |
| Flag | | | |
| | | | |
| Transaction | Will | Will Implement | Implemented |
| Phase Flag | Implement | | |
+-----------------+-------------+----------------+------------------+
MainHeader Handling6.1.1.3. TLV Handling 🦋
+------------------+-------------+---------------+------------------+
| TLV | NTT Japan | University of | Zhejiang |
| | | Patras | Gongshang |
| | | | University |
+------------------+-------------+---------------+------------------+
| REDIRECT-TLV | Implemented | Will | Implemented |
| | | Implement | |
| | | | |
| ASResult-TLV | Implemented | Implemented | Implemented |
| | | | |
| ASTReason-TLV | Implemented | Implemented | Implemented |
| | | | |
| LFBSelect-TLV | Implemented | Implemented | Implemented |
| | | | |
| OPER-TLV | Implemented | Implemented | Implemented |
| | | | |
| PATH-DATA-TLV | Implemented | Implemented | Implemented |
| | | | |
| KEYINFO-TLV | Will | Will | Implemented |
| | Implement | Implement | |
| | | | |
| FULLDATA-TLV | Implemented | Implemented | Implemented |
| | | | |
| SPARSEDATA-TLV | Will | Will | Implemented |
| | Implement | Implement | |
| | | | |
| ILV | Will | Will | Implemented |
| | Implement | Implement | |
| | | | |
| METADATA-TLV | Will | Will | Implemented |
| | Implement | Implement | |
| | | | |
| RESULT-TLV | Implemented | Implemented | Implemented |
| | | | |
| REDIRECTDATA-TLV | Implemented | Will | Implemented |
| | | Implement | |
+------------------+-------------+---------------+------------------+
TLVs Supported6.1.1.4. Operation Types Supported ✨
okay bestie, this table is giving scorecard 📋. It lines up the ForCES operation types against three implementations: NTT Japan, University of Patras, and Zhejiang Gongshang University. "Implemented" means it's done, and "Will Implement" means it's still on the to-do list 💅
+-------------------+-------------+---------------+-----------------+
| Operation | NTT Japan | University of | Zhejiang |
| | | Patras | Gongshang |
| | | | University |
+-------------------+-------------+---------------+-----------------+
| SET | Implemented | Implemented | Implemented |
| | | | |
| SET-PROP | Will | Will | Implemented |
| | Implement | Implement | |
| | | | |
| SET-RESPONSE | Implemented | Implemented | Implemented |
| | | | |
| SET-PROP-RESPONSE | Will | Will | Implemented |
| | Implement | Implement | |
| | | | |
| DEL | Implemented | Will | Implemented |
| | | Implement | |
| | | | |
| DEL-RESPONSE | Implemented | Will | Implemented |
| | | Implement | |
| | | | |
| GET | Implemented | Implemented | Implemented |
| | | | |
| GET-PROP | Will | Will | Implemented |
| | Implement | Implement | |
| | | | |
| GET-RESPONSE | Implemented | Implemented | Implemented |
| | | | |
| GET-PROP-RESPONSE | Will | Will | Implemented |
| | Implement | Implement | |
| | | | |
| REPORT | Implemented | Implemented | Implemented |
| | | | |
| COMMIT | Will | Will | Implemented |
| | Implement | Implement | |
| | | | |
| COMMIT-RESPONSE | Will | Will | Implemented |
| | Implement | Implement | |
| | | | |
| TRCOMP | Will | Will | Implemented |
| | Implement | Implement | |
+-------------------+-------------+---------------+-----------------+
Operation Types Supported6.1.1.5. ForCES Protocol Advanced Features 🦋
next up, the fancy extras 🌸 same three teams, now checking the advanced features: Execute Mode, Transaction, Batching, Command Pipelining, and Heartbeats. Heartbeats is the one everybody has already shipped, and NTT Japan and Zhejiang Gongshang are both showing Command Pipelining as a will-do 🔥
+---------------+-------------+----------------+--------------------+
| Feature | NTT Japan | University of | Zhejiang Gongshang |
| | | Patras | University |
+---------------+-------------+----------------+--------------------+
| Execute Mode | Will | Will Implement | Implemented |
| | Implement | | |
| | | | |
| Transaction | Will | Will Implement | Implemented |
| | Implement | | |
| | | | |
| Batching | Will | Implemented | Implemented |
| | Implement | | |
| | | | |
| Command | Will | Will Implement | Will Implement |
| Pipelining | Implement | | |
| | | | |
| Heartbeats | Implemented | Implemented | Implemented |
+---------------+-------------+----------------+--------------------+
ForCES Protocol Advanced Features6.1.2. ForCES Model Features 🧚
this is the model side of things, where we check what data types each team has covered 💜
6.1.2.1. Basic Atomic Types Supported 💖
the atomic types are the building blocks, bestie, like the basic ingredients 🍓 Most of the list is already "Implemented" across the board, but a bunch of types (char, int16, uint16, int64, uint64, octetstring[N], float32, float64) are still marked "Will Implement" for Zhejiang Gongshang University 👀
+----------------+-------------+---------------+--------------------+
| Atomic Type | NTT Japan | University of | Zhejiang Gongshang |
| | | Patras | University |
+----------------+-------------+---------------+--------------------+
| char | Implemented | Implemented | Will Implement |
| | | | |
| uchar | Implemented | Implemented | Implemented |
| | | | |
| int16 | Implemented | Implemented | Will Implement |
| | | | |
| uint16 | Implemented | Implemented | Will Implement |
| | | | |
| int32 | Implemented | Implemented | Implemented |
| | | | |
| uint32 | Implemented | Implemented | Implemented |
| | | | |
| int64 | Implemented | Implemented | Will Implement |
| | | | |
| uint64 | Implemented | Implemented | Will Implement |
| | | | |
| boolean | Implemented | Implemented | Implemented |
| | | | |
| string[N] | Implemented | Implemented | Implemented |
| | | | |
| string | Implemented | Implemented | Implemented |
| | | | |
| byte[N] | Implemented | Implemented | Implemented |
| | | | |
| octetstring[N] | Implemented | Implemented | Will Implement |
| | | | |
| float32 | Implemented | Implemented | Will Implement |
| | | | |
| float64 | Implemented | Implemented | Will Implement |
+----------------+-------------+---------------+--------------------+
Basic Atomic Types Supported6.1.2.2. Compound Types Supported 🎀
now the bigger combos, structs and arrays 🫶 Everybody has both of these marked "Implemented", so no drama here ✨
+------------+-------------+-----------------+----------------------+
| Compound | NTT Japan | University of | Zhejiang Gongshang |
| Type | | Patras | University |
+------------+-------------+-----------------+----------------------+
| structs | Implemented | Implemented | Implemented |
| | | | |
| arrays | Implemented | Implemented | Implemented |
+------------+-------------+-----------------+----------------------+
Compound Types Supported6.1.2.3. LFBs Supported 🌈
6.1.2.3.1. FE Protocol LFB ✨
okay bestie, the FE Protocol LFB tables are up next, and the Implemented column is giving me so much confidence 💅🫶
+------------------+-------------+----------------+-----------------+
| Protocol | NTT Japan | University of | Zhejiang |
| Datatypes | | Patras | Gongshang |
| | | | University |
+------------------+-------------+----------------+-----------------+
| CEHBPolicy | Implemented | Implemented | Implemented |
| | | | |
| FEHBPolicy | Implemented | Implemented | Implemented |
| | | | |
| FERestartPolicy | Implemented | Implemented | Implemented |
| | | | |
| CEFailoverPolicy | Implemented | Implemented | Implemented |
| | | | |
| FEHACapab | Implemented | Implemented | Will Implement |
+------------------+-------------+----------------+-----------------+
FE Protocol LFB Datatypes
+-----------------------+-------------+-------------+---------------+
| Protocol Components | NTT Japan | University | Zhejiang |
| | | of Patras | Gongshang |
| | | | University |
+-----------------------+-------------+-------------+---------------+
| CurrentRunningVersion | Implemented | Implemented | Implemented |
| | | | |
| FEID | Implemented | Implemented | Implemented |
| | | | |
| MulticastFEIDs | Implemented | Implemented | Implemented |
| | | | |
| CEHBPolicy | Implemented | Implemented | Implemented |
| | | | |
| CEHDI | Implemented | Implemented | Implemented |
| | | | |
| FEHBPolicy | Implemented | Implemented | Implemented |
| | | | |
| FEHI | Implemented | Implemented | Implemented |
| | | | |
| CEID | Implemented | Implemented | Implemented |
| | | | |
| BackupCEs | Implemented | Will | Will |
| | | Implement | Implement |
| | | | |
| CEFailoverPolicy | Implemented | Implemented | Implemented |
| | | | |
| CEFTI | Implemented | Implemented | Implemented |
| | | | |
| FERestartPolicy | Implemented | Implemented | Will |
| | | | Implement |
| | | | |
| LastCEID | Implemented | Implemented | Will |
| | | | Implement |
+-----------------------+-------------+-------------+---------------+
FE Protocol LFB Components
+---------------------+-------------+-------------+-----------------+
| Capabilities | NTT Japan | University | Zhejiang |
| | | of Patras | Gongshang |
| | | | University |
+---------------------+-------------+-------------+-----------------+
| SupportableVersions | Implemented | Implemented | Implemented |
| | | | |
| HACapabilities | Implemented | Implemented | Will Implement |
+---------------------+-------------+-------------+-----------------+
Capabilities Supported
+---------------+------------+----------------+---------------------+
| Events | NTT Japan | University of | Zhejiang Gongshang |
| | | Patras | University |
+---------------+------------+----------------+---------------------+
| PrimaryCEDown | Will | Will Implement | Will Implement |
| | Implement | | |
+---------------+------------+----------------+---------------------+
Events Supported6.1.2.3.2. FE Object LFB 🦋
the FE Object LFB tables are next, so let's keep the energy going 🌸
+--------------------------+-------------+-------------+-------------+
| Object Datatypes | NTT Japan | University | Zhejiang |
| | | of Patras | Gongshang |
| | | | University |
+--------------------------+-------------+-------------+-------------+
| LFBAdjacencyLimitType | Implemented | Implemented | Implemented |
| | | | |
| PortGroupLimitType | Implemented | Implemented | Implemented |
| | | | |
| SupportedLFBType | Implemented | Implemented | Implemented |
| | | | |
| FEStateValues | Implemented | Implemented | Implemented |
| | | | |
| FEConfiguredNeighborType | Implemented | Implemented | Implemented |
| | | | |
| LFBSelectorType | Implemented | Implemented | Implemented |
| | | | |
| LFBLinkType | Implemented | Implemented | Implemented |
+--------------------------+-------------+-------------+-------------+
FE Object LFB Datatypes
+--------------+-------------+----------------+---------------------+
| Object | NTT Japan | University of | Zhejiang Gongshang |
| Components | | Patras | University |
+--------------+-------------+----------------+---------------------+
| LFBTopology | Implemented | Implemented | Implemented |
| | | | |
| LFBSelectors | Implemented | Implemented | Implemented |
| | | | |
| FEName | Implemented | Implemented | Implemented |
| | | | |
| FEID | Implemented | Implemented | Implemented |
| | | | |
| FEVendor | Implemented | Implemented | Implemented |
| | | | |
| FEModel | Implemented | Implemented | Implemented |
| | | | |
| FEState | Implemented | Implemented | Implemented |
| | | | |
| FENeighbors | Implemented | Implemented | Implemented |
+--------------+-------------+----------------+---------------------+
FE Object LFB Components
+-----------------------+-------------+-------------+---------------+
| Capabilities | NTT Japan | University | Zhejiang |
| | | of Patras | Gongshang |
| | | | University |
+-----------------------+-------------+-------------+---------------+
| ModifiableLFBTopology | Implemented | Implemented | Implemented |
| | | | |
| SupportedLFBs | Implemented | Implemented | Implemented |
+-----------------------+-------------+-------------+---------------+
Capabilities Supported6.1.3. ForCES SCTP TML Features 💜
the SCTP TML stuff is next, and it's giving organised 🧚
6.1.3.1. TML Priority Ports 🍓
the priority ports are literally the cutest little table, look at them 🎀
+----------------+-------------+---------------+--------------------+
| Port | NTT Japan | University of | Zhejiang Gongshang |
| | | Patras | University |
+----------------+-------------+---------------+--------------------+
| High priority | Implemented | Implemented | Implemented |
| (6700) | | | |
| | | | |
| Medium | Implemented | Implemented | Implemented |
| priority | | | |
| (6701) | | | |
| | | | |
| Low priority | Implemented | Implemented | Implemented |
| (6702) | | | |
+----------------+-------------+---------------+--------------------+
Priority Ports6.1.3.2. Message Handling at Specific Priorities ✨
okay bestie, now we're checking who handles which message on which priority port 💅 Every implementation was tested on the same messages, so here's the whole vibe in table form 🎀
+------------------+-------------+---------------+------------------+
| ForCES Message | NTT Japan | University of | Zhejiang |
| | | Patras | Gongshang |
| | | | University |
+------------------+-------------+---------------+------------------+
| Association | Implemented | Implemented | Implemented |
| Setup | | | |
| | | | |
| Association | Implemented | Implemented | Implemented |
| Setup Response | | | |
| | | | |
| Association | Implemented | Implemented | Implemented |
| Teardown | | | |
| | | | |
| Config | Implemented | Implemented | Implemented |
| | | | |
| Config Response | Implemented | Implemented | Implemented |
| | | | |
| Query | Implemented | Implemented | Implemented |
| | | | |
| Query Response | Implemented | Implemented | Implemented |
+------------------+-------------+---------------+------------------+
Message Handling at High-Priority (6700) Port
+---------------+-------------+----------------+--------------------+
| ForCES | NTT Japan | University of | Zhejiang Gongshang |
| Message | | Patras | University |
+---------------+-------------+----------------+--------------------+
| Event | Implemented | Implemented | Implemented |
| Notification | | | |
+---------------+-------------+----------------+--------------------+
Message Handling at Medium-Priority (6701) Port
+-------------+-------------+-----------------+---------------------+
| ForCES | NTT Japan | University of | Zhejiang Gongshang |
| Message | | Patras | University |
+-------------+-------------+-----------------+---------------------+
| Packet | Implemented | Implemented | Implemented |
| Redirect | | | |
| | | | |
| Heartbeat | Implemented | Implemented | Implemented |
+-------------+-------------+-----------------+---------------------+
Message Handling at Low-Priority (6702) Port6.1.3.3. TML Security Feature 🔐
here's the security feature check for the TML, and yes, it's all about IPsec 🫶
+--------------+------------+-----------------+---------------------+
| Security | NTT Japan | University of | Zhejiang Gongshang |
| Feature | | Patras | University |
+--------------+------------+-----------------+---------------------+
| IPsec | Will | Will Implement | Will Implement |
| | Implement | | |
+--------------+------------+-----------------+---------------------+
Security Feature Support6.2. Interoperability Report 🧚
okay now the fun part, the interop test 🎉 It took place at the University of Patras, in the Department of Electrical and Computer Engineering 🏫
There were two options for participation in the interoperability test. 🌈
- Locally, on the University of Patras premises.
- Remotely, via Internet.
Implementations from NTT and the University of Patras were present locally on the University of Patras premises in Greece, while the implementation from Zhejiang Gongshang University, which was behind a NAT, connected remotely from China 🌏 that's giving long-distance bestie energy 💜
The interoperability test validated the basic functionality of the ForCES protocol, mainly message exchanging and handling ✨
The following scenarios were tested. 👀
6.2.1. Scenarios
the main goal was to validate the basic protocol functionality, and the test parameters were limited on purpose 🍓
- In the Association Setup message, all report messages were ignored.
- In the Association Setup stage, the FEO OperEnable Event (FE to CE), Config FEO Adminup (CE to FE), and FEO Config-Resp (FE to CE) messages were ignored. The CEs assumed that the FEs were enabled once the LFB selectors had been queried.
- Only FULLDATA-TLVs were used and not SPARSEDATA-TLVs.
- There were no transaction operations.
- Each message had only one LFBSelect-TLV, one OPER-TLV, and one PATH-DATA-TLV per message when these were used.
6.2.1.1. Scenario 1 - Pre-Association Setup 🌸
pre-association setup is not in the ForCES current scope, but it's an essential step before CEs and FEs can talk. As the first part of a successful CE-FE connection, the participating CEs and FEs had to be configurable 🛠️
In the pre-association phase, the following configuration items were set up regarding the CEs:
- The CE ID.
- The FE IDs that were connected to this CE.
- The IP addresses of the FEs that connected to the CE.
- The TML priority ports.
In the pre-association phase, the following configuration items were set up regarding the FEs:
- The FE ID.
- The CE ID to which this FE was connecting.
- The IP address of the CE to which this FE was connecting.
- The TML priority ports.
6.2.1.2. Scenario 2 - TML Priority Channels Connection 🦋
for the interop test, SCTP was used as the TML. The TML connection with the associating element was needed for Scenario 2 to be successful 🔥
SCTP TML [RFC5811] defines three priority channels, with specific ports:
- High priority - Port number: 6704
- Medium priority - Port number: 6705
- Lower priority - Port number: 6706
however, at the time of the interop test, the SCTP ports of the three priority channels were the following:
- High priority - Port number: 6700
- Medium priority - Port number: 6701
- Lower priority - Port number: 6702
As specified in Section 5, "Exceptions", this does not invalidate the results of the interoperability test. 🙈 no drama, the results still count 💅
6.2.1.3. Scenario 3 - Association Setup - Association Complete 🎀
once the pre-association phase in the previous two scenarios had completed, CEs and FEs would be ready to communicate using the ForCES protocol and enter the Association Setup stage 🧚 In this stage, the FEs would attempt to join the NE. The following ForCES protocol messages would be exchanged for each CE-FE pair in the specified order:
- Association Setup message (from FE to CE)
- Association Setup Response message (from CE to FE)
- Query message: FEO LFB selectors (from CE to FE)
- Query Response: FEO LFB selectors response (from FE to CE)
6.2.1.4. Scenario 4 - CE Query 🔍
once the Association Setup stage had completed, the FEs and CEs would enter the Established stage. In this stage, the FE will be continuously updated or queried. The CE should query the FE for a specific value from the FE Object LFB and from the FE Protocol LFB. An example from the FE Protocol LFB is the FE Heartbeat Interval (FEHI), and an example from the FE Object LFB is the state of the LFB (FEState) 💜
The following ForCES protocol messages were exchanged:
- Query message
- Query Response message
6.2.1.5. Scenario 5 - Heartbeat Monitoring 💓
the Heartbeat (HB) message is used for one ForCES element (FE or CE) to asynchronously notify one or more other ForCES elements in the same ForCES NE of its liveness. The default configuration of the Heartbeat Policy of the FE is set to 0, which means that the FE should not generate any Heartbeat messages. The CE is responsible for checking FE liveness by setting the PL header ACK flag of the message it sends to AlwaysACK. In this scenario, the CE will send a Heartbeat message with the ACK flag set to AlwaysACK, and the FE should respond 🫶
The following type of ForCES protocol message was exchanged:
- Heartbeat message
6.2.1.6. Scenario 6 - Simple Config Command ⚙️
a Config message is sent by the CE to the FE to configure LFB components in the FE. A simple Config command, easily visible and metered, would be to change the Heartbeat configuration. This was done in two steps:
- Change the FE Heartbeat Policy (FEHBPolicy) to value 1, to force the FE to send heartbeats.
- After some heartbeats from the FE, the FE Heartbeat Interval (FEHI) was changed.
The following ForCES protocol messages were exchanged:
- Config message
- Config Response message
6.2.1.7. Scenario 7 - Association Teardown 👋
in the end, the association must be terminated. There were three scenarios by which the association was terminated:
- Normal teardown, by exchanging an Association Teardown message.
- Irregular teardown, by stopping heartbeats from an FE or a CE.
- Irregular teardown, by externally shutting down/rebooting an FE or a CE.
All scenarios were investigated in the interoperability test. 🌈
The following type of ForCES protocol message was exchanged:
- Association Teardown message
6.2.2. Tested Features 🧪
The features that were tested are:
6.2.2.1. ForCES Protocol Features 🎀
6.2.2.1.1. Protocol Messages 📨
+----------------------------+
| Protocol Message |
+----------------------------+
| Association Setup |
| |
| Association Setup Response |
| |
| Association Teardown |
| |
| Config |
| |
| Config Response |
| |
| Query |
| |
| Query Response |
| |
| Heartbeat |
+----------------------------+
ForCES Protocol Messages- PASS: All implementations handled the protocol messages, and all protocol analyzers captured them. ✅
6.2.2.1.2. MainHeader Handling 🧠
+--------------------+
| Header Field |
+--------------------+
| Correlator |
| |
| ACK Indicator Flag |
| |
| Priority Flag |
+--------------------+
MainHeader Handling- PASS: All implementations handled these main header flags, and all protocol analyzers captured them. ✅
6.2.2.1.3. TLV Handling ✨
Okay bestie, let's talk TLVs 💖 Every one of these showed up and got handled. Literally iconic.
+---------------+
| TLV |
+---------------+
| ASResult-TLV |
| |
| ASTReason-TLV |
| |
| LFBSelect-TLV |
| |
| OPER-TLV |
| |
| PATH-DATA-TLV |
| |
| FULLDATA-TLV |
| |
| RESULT-TLV |
+---------------+
TLVs Supported- PASS: All implementations handled these TLVs, and all protocol analyzers captured them. 🎀
6.2.2.1.4. Operation Types Supported 🦋
Next up, the operation types. Full glow-up energy 🌸
+--------------+
| Operation |
+--------------+
| SET |
| |
| SET-RESPONSE |
| |
| GET |
| |
| GET-RESPONSE |
| |
| REPORT |
+--------------+
Operation Types Supported- PASS: All implementations handled these operations, and all protocol analyzers captured them. 💅
6.2.2.1.5. ForCES Protocol Advanced Features 👑
Okay so the advanced features are where it gets spicy 🌶️ Batching wasn't supposed to be tested at first, but it got checked during the interop test anyway. Plot twist, we love a surprise 😭
+------------+
| Feature |
+------------+
| Batching |
| |
| Heartbeats |
+------------+
ForCES Protocol Advanced FeaturesAlthough batching was not initially intended to be tested, it was assessed during the interoperability test.
- PASS: Two implementations handled batching, and all handled heartbeats. The protocol analyzers captured both. 🫶
6.2.2.2. ForCES Model Features 🧚
Now the model features, aka the stuff that makes ForCES ForCES 💜
6.2.2.2.1. Basic Atomic Types Supported 🍓
+-------------+
| Atomic Type |
+-------------+
| uchar |
| |
| uint32 |
+-------------+
Basic Atomic Types Supported- PASS: All implementations handled these basic atomic types. ✨
6.2.2.2.2. Compound Types Supported 🌈
+---------------+
| Compound Type |
+---------------+
| structs |
| |
| arrays |
+---------------+
Compound Types Supported- PASS: All implementations handled these compound types. 🔥
6.2.2.2.3. LFBs Supported 🦋
6.2.2.2.3.1. FE Protocol LFB 💖
+--------------------+
| Protocol Datatypes |
+--------------------+
| CEHBPolicy |
| |
| FEHBPolicy |
+--------------------+
FE Protocol LFB Datatypes- PASS: All implementations handled these FE Protocol LFB datatypes. 🌸
+---------------------+
| Protocol Components |
+---------------------+
| FEID |
| |
| CEHBPolicy |
| |
| CEHDI |
| |
| FEHBPolicy |
| |
| FEHI |
| |
| CEID |
+---------------------+
FE Protocol LFB Components- PASS: All implementations handled these FE Protocol LFB components. 🎀
6.2.2.2.3.2. FE Object LFB 🍓
+------------------+
| Object Datatypes |
+------------------+
| FEStateValues |
| |
| LFBSelectorType |
+------------------+
FE Object LFB Datatypes- PASS: All implementations handled these FE Object LFB datatypes. 💅
+-------------------+
| Object Components |
+-------------------+
| LFBSelectors |
| |
| FEState |
+-------------------+
FE Object LFB Components- PASS: All implementations handled these FE Object LFB components. 🫶
6.2.2.3. ForCES SCTP TML Features 🌊
Okay the transport layer stuff is giving main character energy 🦋
6.2.2.3.1. TML Priority Ports 👑
+------------------------+
| Port |
+------------------------+
| High priority (6700) |
| |
| Medium priority (6701) |
| |
| Low priority (6702) |
+------------------------+
Priority Ports- PASS: All implementations opened and connected to all the SCTP priority ports. The protocol analyzers captured all ports and their corresponding priority. 💖
6.2.2.3.2. Message Handling at Specific Priorities 🧚
Now the priority channels, where each message gets routed 🌈
+----------------------------+
| ForCES Message |
+----------------------------+
| Association Setup |
| |
| Association Setup Response |
| |
| Association Teardown |
| |
| Config |
| |
| Config Response |
| |
| Query |
| |
| Query Response |
+----------------------------+
Message Handling at High-Priority (6700) Port- PASS: All implementations handled these messages at this SCTP priority port. The protocol analyzers captured these messages at this priority port. ✨
+----------------+
| ForCES Message |
+----------------+
| Heartbeats |
+----------------+
Message Handling at Low-Priority (6702) Port- PASS: All implementations handled these messages at this SCTP priority port. The protocol analyzers captured these messages at this priority port. 🔥
6.2.3. Interoperability Results 🎉
All implementations were found to be interoperable with each other. Literally the best news 😭
All scenarios were tested successfully.
The following issues were found and dealt with. Let's spill the tea 🍵
- Some messages were sent on the wrong priority channels. There were some ambiguities in the SCTP TML document regarding how to deal with such a situation. The possibilities were an FE response on the same (wrong) channel as a CE query; an FE response on the correctly documented channel for the message; or simply dropping the packet. This has been corrected by mandating the message-to-channel mapping to be a MUST in the SCTP TML document [RFC5811] before it was published as an RFC. 🙈
- At some point, a CE sent a Teardown message to the FE. The CE expected the FE to shut down the connection, and the FE waited for the CE to shut down the connection; both were then caught in a deadlock. This was a code bug and was fixed. 💀
- Sometimes, only when the CE and FE were remote to each other (one being in China and another in Greece), the Association Setup message was not received by the CE side, and therefore an association never completed. This was not an implementation issue but rather a network issue. This issue was solved with the retransmission of the non-delivered messages. 🌍
- An implementation did not take into account that the padding in TLVs MUST NOT be included in the length of the TLV. This was a code bug and was fixed. 🧚
- The Execution Mode flag was set to Reserved by a CE and was not ignored by the FE. This was a code bug and was fixed. 🍓
- After the FEHBPolicy was set to 1, the FE didn't send any heartbeats. This was a code bug and was fixed. 😭
- Some FEs sent heartbeats with the ACK flag set to a value other than NoACK. The CE responded. This was a code bug and was fixed. 💅
- When a cable was disconnected, none of the TML implementations detected it. The association was eventually dropped due to heartbeat detection; this test was a success, but this is an implementation issue that implementors should keep in mind. This is an SCTP options issue. Nothing needed to be done. 🌸
- A CE crashed due to unknown LFB selector values. This was a code bug and was fixed. 💔
- With the remote connection from China (which was behind a NAT) to Greece, there were a lot of ForCES packet retransmissions. The problem was that packets like heartbeats were retransmitted. This was an implementation issue regarding SCTP usage that implementors should keep in mind. The SCTP-PR option needed to be used. Nothing needed to be done. 🎀
The interoperability test went so well that an additional extended test was added to check for batching messages. This test was also done successfully. 🥳
7. Acknowledgements ✨
okay bestie, the thank-you section is literally so wholesome 🫶 the authors gave a shoutout to Professors Odysseas Koufopavlou and Spyros Denazis and the Department of Electrical and Computer Engineering at the University of Patras 🎓, who hosted the ForCES interoperability test. iconic hosting energy.
they also thanked Chuanhuang Li, Ming Gao, and other participants from Zhejiang Gongshang University, who connected remotely 🌐. ngl this part is lowkey important, because their remote participation helped uncover a bunch of issues that would have totally slipped through otherwise 😭 the remote crew really did the most.
then there's a thank-you to Hideaki Iwata and Yoshinobu Morimoto of NTT Japan for joining locally at the interoperability test 🍣, plus Hiroki Date and Hidefumi Otsuka, also from NTT Japan, for contributing to the test 💅.
and finally, a special shoutout to Xinping Wang for helping write the interoperability document 📝 and to Fenggen Jia for extending the Ethereal protocol analyzer 🔍. the whole squad showed up 🩷
8. Security Considerations 🔐
okay this section is dramatic but it's important, so let's go 💀
No security elements of the protocol or the SCTP TML [RFC5811] specification were tested.
so the interop test did not check any security features in the ForCES protocol or the SCTP TML [RFC5811] spec. 🙈
the survey showed that no security elements were implemented 😬, but every participant said they intended to implement them ✨. so the plan is there, just not yet in the code.
for security considerations regarding the ForCES protocol and SCTP TML, please see [RFC5810] and [RFC5811] 📚. those are the places to look 🫶
9. References 📚
9.1. Normative References 🌸
- [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
- [RFC5810] Doria, A., Hadi Salim, J., Haas, R., Khosravi, H., Wang, W., Dong, L., Gopal, R., and J. Halpern, "Forwarding and Control Element Separation (ForCES) Protocol Specification", RFC 5810, March 2010.
- [RFC5811] Hadi Salim, J. and K. Ogawa, "SCTP-Based Transport Mapping Layer (TML) for the Forwarding and Control Element Separation (ForCES) Protocol", RFC 5811, March 2010.
- [RFC5812] Halpern, J. and J. Hadi Salim, "Forwarding and Control Element Separation (ForCES) Forwarding Element Model", RFC 5812, March 2010.
9.2. Informative References 🦋
- [RFC3654] Khosravi, H. and T. Anderson, "Requirements for Separation of IP Control and Forwarding", RFC 3654, November 2003.
- [RFC3746] Yang, L., Dantu, R., Anderson, T., and R. Gopal, "Forwarding and Control Element Separation (ForCES) Framework", RFC 3746, April 2004.
- [RFC5657] Dusseault, L. and R. Sparks, "Guidance on Interoperation and Implementation Reports for Advancement to Draft Standard", BCP 9, RFC 5657, September 2009.
- [ethereal] "Ethereal is a protocol analyzer. The specific Ethereal that was used is an updated Ethereal, by Fenggen Jia, that can analyze and decode the ForCES protocol messages", <http://www.ietf.org/mail-archive/web/forces/current/msg03687.html>.
- [tcpdump] "tcpdump is a Linux protocol analyzer. The specific tcpdump that was used is a modified tcpdump, by Jamal Hadi Salim, that can analyze and decode the ForCES protocol messages", <http://www.ietf.org/mail-archive/web/forces/current/msg03811.html>.
Authors' Addresses 💌
- Evangelos Haleplidis University of Patras Patras Greece EMail: ehalep@ece.upatras.gr
- Kentaro Ogawa NTT Corporation Tokyo Japan EMail: ogawa.kentaro@lab.ntt.co.jp
- Weiming Wang Zhejiang Gongshang University 18, Xuezheng Str., Xiasha University Town Hangzhou, 310018 P.R. China Phone: +86-571-28877721 EMail: wmwang@mail.zjgsu.edu.cn
- Jamal Hadi Salim Mojatatu Networks Ottawa, Ontario Canada Phone: EMail: hadi@mojatatu.com
📦 where this page comes from
- ietf: rfc6053 2010-11
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms