RFC 3746
Forwarding and Control Element Separation (ForCES) Framework
RFC 3746: Forwarding and Control Element Separation (ForCES) Framework ✨🎀
okay bestie, this is RFC 3746 (part 1 of 9) 💖 It's an Informational doc from April 2004, so it's not a standard, just useful info. The authors are L. Yang (Intel Corp.), R. Dantu (Univ. of North Texas), T. Anderson (Intel Corp.), and R. Gopal (Nokia). The title block doesn't say it obsoletes or updates anything 🦋
Status of this Memo 🫶
this memo is info for the Internet community, and it is not an Internet standard of any kind. Distribution is unlimited, so share away 💅
Copyright Notice ©
Copyright (C) The Internet Society (2004). All Rights Reserved.
Abstract 🌸
okay so this doc lays out the architecture for ForCES network elements (that's Forwarding and Control Element Separation), and it names the entities involved and how they interact with each other. it's giving blueprint energy 🧚
Table of Contents 🗺️
1. Definitions. . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1. Conventions used in this document . . . . . . . . . . . . 2
1.2. Terminologies . . . . . . . . . . . . . . . . . . . . . . 3
2. Introduction to Forwarding and Control Element Separation
(ForCES) . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3. Architecture . . . . . . . . . . . . . . . . . . . . . . . . . 8
3.1. Control Elements and Fr Reference Point . . . . . . . . . 10
3.2. Forwarding Elements and Fi reference point. . . . . . . . 11
3.3. CE Managers . . . . . . . . . . . . . . . . . . . . . . . 14
3.4. FE Managers . . . . . . . . . . . . . . . . . . . . . . . 14
4. Operational Phases . . . . . . . . . . . . . . . . . . . . . . 15
4.1. Pre-association Phase . . . . . . . . . . . . . . . . . . 15
4.1.1. Fl Reference Point . . . . . . . . . . . . . . . . 15
4.1.2. Ff Reference Point . . . . . . . . . . . . . . . . 16
4.1.3. Fc Reference Point . . . . . . . . . . . . . . . . 17
4.2. Post-association Phase and Fp reference point . . . . . . 17
4.2.1. Proximity and Interconnect between CEs and FEs . . 18
4.2.2. Association Establishment. . . . . . . . . . . . . 18
4.2.3. Steady-state Communication . . . . . . . . . . . . 19
4.2.4. Data Packets across Fp reference point . . . . . . 21
4.2.5. Proxy FE . . . . . . . . . . . . . . . . . . . . . 22
4.3. Association Re-establishment. . . . . . . . . . . . . . . 22
4.3.1. CE graceful restart. . . . . . . . . . . . . . . . 23
4.3.2. FE restart . . . . . . . . . . . . . . . . . . . . 24
5. Applicability to RFC 1812. . . . . . . . . . . . . . . . . . . 25
5.1. General Router Requirements . . . . . . . . . . . . . . . 25
5.2. Link Layer. . . . . . . . . . . . . . . . . . . . . . . . 26
5.3. Internet Layer Protocols. . . . . . . . . . . . . . . . . 27
5.4. Internet Layer Forwarding . . . . . . . . . . . . . . . . 27
5.5. Transport Layer . . . . . . . . . . . . . . . . . . . . . 28
5.6. Application Layer -- Routing Protocols. . . . . . . . . . 29
5.7. Application Layer -- Network Management Protocol. . . . . 29
6. Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 30
8. Security Considerations. . . . . . . . . . . . . . . . . . . . 30
8.1. Analysis of Potential Threats Introduced by ForCES. . . . 31
8.1.1. "Join" or "Remove" Message Flooding on CEs . . . . 31
8.1.2. Impersonation Attack . . . . . . . . . . . . . . . 31
8.1.3. Replay Attack. . . . . . . . . . . . . . . . . . . 31
8.1.4. Attack during Fail Over. . . . . . . . . . . . . . 32
8.1.5. Data Integrity . . . . . . . . . . . . . . . . . . 32
8.1.6. Data Confidentiality . . . . . . . . . . . . . . . 32
8.1.7. Sharing security parameters. . . . . . . . . . . . 33
8.1.8. Denial of Service Attack via External Interface. . 33
8.2. Security Recommendations for ForCES . . . . . . . . . . . 33
8.2.1. Using TLS with ForCES. . . . . . . . . . . . . . . 34
8.2.2. Using IPsec with ForCES. . . . . . . . . . . . . . 35
9. References . . . . . . . . . . . . . . . . . . . . . . . . . . 37
9.1. Normative References. . . . . . . . . . . . . . . . . . . 37
9.2. Informative References. . . . . . . . . . . . . . . . . . 37
10. Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . 39
11. Full Copyright Statement . . . . . . . . . . . . . . . . . . . 401. Definitions 💅
1.1. Conventions used in this document 📏
okay so the BCP 14 key words in this doc are "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL". they get read the way BCP 14 (RFC 2119 [1]) describes, so pay attention to those capitals 👀
1.2. Terminologies 📚
the ForCES requirements terminology is defined in [4], and we're only bringing over the bits that matter most for this doc 💖
Addressable Entity (AE) - an entity you can directly address given some interconnect technology. like on IP networks, it's a device you can talk to using an IP address, and on a switch fabric, it's a device you can talk to using a switch fabric port number 🦋
Physical Forwarding Element (PFE) - an AE that includes the hardware used for per-packet processing and handling. that hardware can be (but isn't limited to) network processors, ASICs (Application-Specific Integrated Circuits), or general purpose processors, and it can live on line cards, daughter boards, mezzanine cards, or in stand-alone boxes 🔥
PFE Partition - a logical slice of a PFE made of some subset of each resource on it (like ports, memory, and forwarding table entries). it's basically the same idea as the resources given to a virtual switching element, as described in [9] 🧚
Physical Control Element (PCE) - an AE that includes the hardware used for control functionality. that hardware usually includes a general purpose processor 👑
PCE Partition - a logical slice of a PCE made of some subset of each resource available on it 🌸
Forwarding Element (FE) - a logical entity that implements the ForCES Protocol. FEs use their underlying hardware to do per-packet processing and handling, as directed by a CE via the ForCES Protocol. an FE might be a single blade (or PFE), a partition of a PFE, or multiple PFEs 💜
Control Element (CE) - a logical entity that implements the ForCES Protocol and uses it to instruct one or more FEs on how to process packets. CEs handle things like running control and signaling protocols. a CE might be made of PCE partitions or whole PCEs ⭐
ForCES Network Element (NE) - an entity made of one or more CEs and one or more FEs. an NE usually hides its internal organization from outside entities, so it shows up as a single point of management to anything outside it 🎀
Pre-association Phase - the stretch of time when an FE Manager (see below) and a CE Manager (see below) are figuring out whether an FE and a CE should be part of the same network element. some parts of an NE can be in the pre-association phase while other parts are already in the post-association phase 🫶
Post-association Phase - the stretch of time when an FE knows which CE is going to control it (and vice versa), including the time while the CE and FE are setting up communication with each other 🦋
ForCES Protocol - there can be multiple protocols in the overall ForCES architecture, but the term "ForCES Protocol" refers only to the ForCES post-association phase protocol (see below) 💅
ForCES Post-Association Phase Protocol - the protocol used for communication between CEs and FEs after association. it doesn't cover CE-to-CE communication, FE-to-FE communication, or communication between FE and CE managers. the ForCES Protocol is a master-slave protocol where FEs are slaves and CEs are masters. it handles both managing the communication channel (like connection establishment and heartbeats) and the control messages themselves. it could be one protocol or several working together, and it may be unicast or multicast based. a separate protocol document will specify this 🧚
FE Manager - a logical entity that operates in the pre-association phase and is responsible for deciding which CE(s) an FE should communicate with. this is called CE discovery, and it may involve the FE manager learning the capabilities of available CEs. an FE manager can use anything from static configuration to a pre-association phase protocol (see below) to decide which CE(s) to use, but this is currently out of scope. since it's a logical entity, an FE manager might be physically combined with any of the other logical entities in this section 🌈
CE Manager - a logical entity that operates in the pre-association phase and is responsible for deciding which FE(s) a CE should communicate with. this is called FE discovery, and it may involve the CE manager learning the capabilities of available FEs. a CE manager can use anything from static configuration to a pre-association phase protocol (see below) to decide which FE to use, but this is currently out of scope. since it's a logical entity, a CE manager might be physically combined with any of the other logical entities in this section 💖
Pre-association Phase Protocol - a protocol between FE managers and CE managers used to decide which CEs or FEs to use. it may include a CE and/or FE capability discovery mechanism. note that this capability discovery is totally separate from (and doesn't replace) the one used within the ForCES Protocol. however, the two capability discovery mechanisms may use the same FE model 🔥
FE Model - a model that describes the logical processing functions of an FE 🍓
ForCES Protocol Element - an FE or CE 🫶
Intra-FE topology - how a single FE is put together by combining possibly multiple logical functional blocks along multiple data paths. this is defined by the FE model 🌸
FE Topology - how the multiple FEs in a single NE are interconnected. it's sometimes called inter-FE topology, to tell it apart from the intra-FE topology used by the FE model ✨
Inter-FE topology - see FE Topology 👑
2. Introduction to Forwarding and Control Element Separation (ForCES) ✨
okay so an IP network element (NE) looks like one single box to everyone outside it 📦. Think a router, a NAT, a firewall, or a load balancer. But inside, it's actually a bunch of separate pieces that work together to do the job, like routing. There are two kinds of components in there. The control element (CE) lives in the control plane, and the forwarding element (FE) lives in the forwarding plane (also called the data plane) 🧠⚡. FEs are usually built on ASICs, network processors, or general-purpose processors, and they handle the data path for each packet. CEs are usually general-purpose processors that run the control stuff, like routing and signaling protocols 💖.
ForCES is all about defining a framework and protocol(s) so the control and forwarding planes can swap information in a standard way 🫶. Once there are standard mechanisms, CEs and FEs can be physically separated standard parts. Honestly this separation is so good for the architecture 😭. Component vendors can specialize in one piece without having to become experts in every piece 🎀. A standard protocol also means CEs and FEs from different component vendors can interoperate, so system vendors can mix and match CEs and FEs from different suppliers 🦋. That gives system vendors more design choices and flexibility, which is so iconic. Overall, ForCES is meant to speed up innovation in both the control and forwarding planes while keeping everything interoperable 🔥. Scalability comes along for free too, because you can add more forwarding or control capacity to an existing network element without needing forklift upgrades (aka replacing the whole thing) 💅.
------------------------- -------------------------
| Control Blade A | | Control Blade B |
| (CE) | | (CE) |
------------------------- -------------------------
^ | ^ |
| | | |
| V | V
---------------------------------------------------------
| Switch Fabric Backplane |
---------------------------------------------------------
^ | ^ | ^ |
| | | | . . . | |
| V | V | V
------------ ------------ ------------
|Router | |Router | |Router |
|Blade #1 | |Blade #2 | |Blade #N |
| (FE) | | (FE) | | (FE) |
------------ ------------ ------------
^ | ^ | ^ |
| | | | . . . | |
| V | V | VFigure 1. A router configuration example with separate blades.
One example of this physical separation is at the blade level 🌸. Figure 1 shows a router with two control blades and multiple forwarding blades, all hooked up to a switch fabric backplane. In this chassis setup, the control blades are the CEs, the router blades are the FEs, and the switch fabric backplane is what physically connects all the blades 🧚. Control blade A might be the primary CE while control blade B is the backup CE, so there's redundancy 🔁. It's also possible to have a redundant switch fabric for high availability ✨. Routers today that use this kind of setup rely on proprietary interfaces for messaging between CEs and FEs 🙄. ForCES wants to replace those proprietary interfaces with a standard protocol 💜. Once a standard protocol like ForCES runs on all the blades, control blades from vendor X and forwarding blades from vendor Y can work seamlessly together in one chassis 🥹.
------- -------
| CE1 | | CE2 |
------- -------
^ ^
| |
V V
============================================ Ethernet
^ ^ . . . ^
| | |
V V V
------- ------- --------
| FE#1| | FE#2| | FE#n |
------- ------- --------
^ | ^ | ^ |
| | | | | |
| V | V | VFigure 2. A router configuration example with separate boxes.
Another level of physical separation between CEs and FEs can happen at the box level 📦🍓. In this setup, every CE and FE is its own physically separate box, all connected with some kind of high speed LAN link (like Gigabit Ethernet) 🌈. These separated CEs and FEs are only one hop away from each other within a local area network. They talk to each other by running ForCES, and together they act as one routing unit to the outside world 👑. Figure 2 shows an example of this.
In both examples, the same physical interconnect carries both CE-to-FE and FE-to-FE traffic. That doesn't have to be the case though 🤷♀️. One reason to use different interconnects is that the CE-to-FE link doesn't need to be as fast as the FE-to-FE link. So the faster, pricier connections can be saved for FE-to-FE ⭐. Separate interconnects can also give the NE reliability and redundancy benefits 🛡️.
Some control functions that can live in the CE include routing protocols like RIP, OSPF, and BGP, and control and signaling protocols like RSVP (Resource Reservation Protocol) and LDP (Label Distribution Protocol) for MPLS, etc. 🎀 Forwarding functions in the FE include an LPM (longest prefix match) forwarder, classifiers, traffic shapers, meters, NAT (Network Address Translators), etc. 🦋 Figure 3 shows example functions in both the CE and the FE. Any given NE might have one or many of these CE and FE functions 💅. The diagram also shows that the ForCES Protocol carries both the control messages for ForCES itself and the data packets that come from or go to the control functions in the CE (like routing packets) 🫧. Section 4.2.4 gets into more detail on this 👀.
-------------------------------------------------
| | | | | | |
|OSPF |RIP |BGP |RSVP |LDP |. . . |
| | | | | | |
-------------------------------------------------
| ForCES Interface |
-------------------------------------------------
^ ^
ForCES | |data
control | |packets
messages| |(e.g., routing packets)
v v
-------------------------------------------------
| ForCES Interface |
-------------------------------------------------
| | | | | | |
|LPM Fwd|Meter |Shaper |NAT |Classi-|. . . |
| | | | |fier | |
-------------------------------------------------
| FE resources |
-------------------------------------------------Figure 3. Examples of CE and FE functions.
This is where the document's requirements come in 📝. A set of requirements for control and forwarding separation is identified in [4]. This document describes a ForCES architecture that meets the architectural requirements of [4], and it defines a framework for ForCES network elements and their associated entities so protocol definitions are easier to write 🧁. Wherever it helps, the document uses lots of examples to show the issues and/or possible solutions in ForCES 🌟. Just remember these examples are only examples, so don't treat them as the only or definite ways of doing things 🙃. A separate document is expected to come from the ForCES working group to specify the ForCES Protocol 💌.
3. Architecture ✨
okay bestie, this is the part where we look at the whole ForCES blueprint 🏗️💖 It defines the architectural framework and all the logical components that go into it. The framework covers the components of ForCES NEs, plus a bunch of ancillary components too. And these pieces can be wired together in different kinds of topologies, which is giving flexible packet processing 🦋
---------------------------------------
| ForCES Network Element |
-------------- Fc | -------------- -------------- |
| CE Manager |---------+-| CE 1 |------| CE 2 | |
-------------- | | | Fr | | |
| | -------------- -------------- |
| Fl | | | Fp / |
| | Fp| |----------| / |
| | | |/ |
| | | | |
| | | Fp /|----| |
| | | /--------/ | |
-------------- Ff | -------------- -------------- |
| FE Manager |---------+-| FE 1 | Fi | FE 2 | |
-------------- | | |------| | |
| -------------- -------------- |
| | | | | | | | | |
----+--+--+--+----------+--+--+--+-----
| | | | | | | |
| | | | | | | |
Fi/f Fi/f
Fp: CE-FE interface
Fi: FE-FE interface
Fr: CE-CE interface
Fc: Interface between the CE Manager and a CE
Ff: Interface between the FE Manager and an FE
Fl: Interface between the CE Manager and the FE Manager
Fi/f: FE external interfaceFigure 4. ForCES Architectural Diagram
Okay so Figure 4 is the big picture 🌸 It shows every logical component and how they relate to each other. A ForCES NE has two kinds of components inside it: the control element (CE) and the forwarding element (FE). The framework lets you have multiple CEs and multiple FEs living inside one NE, which is so flexible 💅
Each FE has one or more physical media interfaces for receiving and transmitting packets to and from the outside world. When you put all those FE interfaces together, they become the NE's external interfaces 🎀 But that's not all! There must also exist some kind of interconnect inside the NE, so the CE and FE can talk to each other, and so one FE can forward packets to another FE. ngl this internal wiring is what makes the whole thing feel like one box 🫶
The diagram also shows two entities sitting outside the ForCES NE: the CE Manager and the FE Manager 👑 These two ancillary entities handle configuration for the matching CE or FE during the pre-association phase (see Section 4.1). Think of them as the ones getting everything set up before the party starts 🧚
To keep things simple, the logical interactions between these components get labeled with reference points: Fp, Fc, Ff, Fr, Fl, and Fi, all shown in Figure 4 🦋 The FE external interfaces are labeled Fi/f. More detail is given in Section 3 and 4 for each of these reference points. Every single one of these reference points matters for understanding ForCES, but here's the tea: the ForCES Protocol is only defined over one reference point, and that's Fp 🔥
The interface between two ForCES NEs is identical to the interface between two conventional routers. The two NEs swap protocol packets through their external interfaces at Fi/f. And ForCES NEs connect to existing routers transparently, so no drama with the routers already in the network 💜
3.1. Control Elements and Fr Reference Point 🌈
Here's the thing: you don't need to define any protocols across the Fr reference point to get control and forwarding separation for simple setups, like one CE with multiple FEs 🍓 That said, the architecture does allow multiple CEs to be present in one network element. When an implementation uses multiple CEs, the invariant that the CEs and FEs together appear as a single NE must be maintained. Main character energy: from the outside, it still looks like one NE ✨
Multiple CEs can be used for redundancy, load sharing, distributed control, or other purposes 💖 Redundancy is when one or more CEs are prepared to take over if an active CE fails. Load sharing is when two or more CEs are concurrently active, and any request that one of them can service can also be serviced by any of the others. For both redundancy and load sharing, the CEs involved are equivalently capable. The only difference between the two is how many active CEs there are at the same time 🫶 Distributed control is when two or more CEs are concurrently active, but certain requests can only be serviced by certain CEs. It's giving specialization 💅
When multiple CEs are employed in a ForCES NE, their internal organization is considered an implementation issue that is beyond the scope of ForCES 🙈 The CEs are wholly responsible for coordinating amongst themselves via the Fr reference point, to provide consistency and synchronization. But ForCES does not define the implementation or the protocols used between CEs, and it doesn't define how to distribute functionality among CEs either 😭 Nevertheless, ForCES will support mechanisms for CE redundancy or fail over, and vendors are expected to provide redundancy or fail over solutions within this framework 🧚 Iconic, honestly 👑
3.2. Forwarding Elements and Fi reference point ✨
okay so an FE (forwarding element) is a logical entity 🧚 that implements the ForCES Protocol and uses the underlying hardware to do per-packet processing and handling, as directed by a CE. bestie, it's totally possible to split one physical FE into multiple logical FEs 🍓, and it's also possible for one FE to use multiple physical FEs. how the physical and logical FEs map to each other is beyond the scope of ForCES, so we're not going there 🙈. for example, you can make a logical partition of a physical FE by assigning some portion of each resource (ports, memory, forwarding table entries, etc.) on the physical FE to each logical FE. this FE virtualization idea is analogous to a virtual switching element, as described in [9]. if FE virtualization only happens in the pre-association phase, it has no impact on ForCES 💅. but if virtualization pulls a resource from an existing FE that's already participating in the ForCES post-association phase, the ForCES Protocol needs to be able to tell the CE about that change using asynchronous messages (see [4], Section 5, requirement #6) 🔥.
FEs do all the packet processing functions as directed by CEs. FEs have no initiative of their own 😭. they're slaves and only do what they're told, periodt 💀. FEs may communicate with one or more CEs at the same time across the Fp reference point ✨. FEs have no notion of CE redundancy, load sharing, or distributed control. instead, FEs accept commands from any CE authorized to control them, and it's up to the CEs to coordinate among themselves to get redundancy, load sharing, or distributed control 🫶. the whole idea is to keep FEs as simple and dumb as possible so they can focus their resources on packet processing 🦋. unless otherwise configured or determined by a ForCES Protocol exchange, each FE will process authorized incoming commands directed at it as it receives them, first come first serve 🌈.
for example, in Figure 5, FE1 and FE2 can be configured to accept commands from both the primary CE (CE1) and the backup CE (CE2) 👑. if CE1 fails, maybe across the Fr or Fp reference point, CE2 is configured to take over CE1's activities. this is beyond the scope of ForCES, so we won't dig into it further 🙈.
distributed control works in a similar way without much smarts on the FE side 💜. for example, FEs can be configured to detect RSVP and BGP protocol packets, then forward RSVP packets to one CE and BGP packets to another CE. so FEs may need to do packet filtering to forward packets to specific CEs 🎀.
------- Fr -------
| CE1 | ------| CE2 |
------- -------
| \ / |
| \ / |
| \ / |
| \/Fp |
| /\ |
| / \ |
| / \ |
------- Fi -------
| FE1 |<----->| FE2 |
------- -------Figure 5. CE redundancy example.
this architecture lets multiple FEs be present in an NE. [4] says the ForCES Protocol must be able to scale to at least hundreds of FEs (see [4] Section 5, requirement #11) 🌸. each of these FEs may have a different set of packet processing functions, with different media interfaces. FEs are responsible for basic maintenance of layer-2 connectivity with other FEs and with external entities. lots of layer-2 media include sophisticated control protocols 😳. the FORCES Protocol (over the Fp reference point) will be able to carry messages for those protocols, so that the CE can provide the intelligence and control over these media while the FE stays dumb 🧚✨.
when there are multiple FEs, ForCES requires that packets must be able to arrive at the NE through one FE and leave the NE through a different FE (See [4], Section 5, Requirement #3) 🔥. packets that enter the NE via one FE and leave via a different FE are transferred between FEs across the Fi reference point 💖. FEs could use the Fi reference point to discover their (inter-FE) topology, maybe during the pre-association phase. the Fi reference point is a separate protocol from Fp and is not currently defined by the ForCES Protocol 🦋.
FEs can be connected in different topologies, and packet processing may spread across several FEs in the topology. so logical packet flow may be different from the physical FE topology 🌈. Figure 6 has some topology examples. when packets need to be forwarded between FEs, the CE needs to understand the FE topology. the CEs can query the FE topology from the FEs via the ForCES Protocol, but the FEs are not required to provide that info to the CEs. so FE topology info may also be gathered by other means outside the ForCES Protocol (like an inter-FE topology discovery protocol) ✨.
-----------------
| CE |
-----------------
^ ^ ^
/ | \
/ v \
/ ------- \
/ +->| FE3 |<-+ \
/ | | | | \
v | ------- | v
------- | | -------
| FE1 |<-+ +->| FE2 |
| |<--------------->| |
------- -------
^ | ^ |
| | | |
| v | v
(a) Full mesh among FE1, FE2, and FE3
-----------
| CE |
-----------
^ ^ ^ ^
/ | | \
/------ | | ------\
v v v v
------- ------- ------- -------
| FE1 |<->| FE2 |<->| FE3 |<->| FE4 |
------- ------- ------- -------
^ | ^ | ^ | ^ |
| | | | | | | |
| v | v | v | v
(b) Multiple FEs in a daisy chain
^ |
| v
-----------
| FE1 |<-----------------------|
----------- |
^ ^ |
/ \ |
| ^ / \ ^ | V
v | v v | v ----------
--------- --------- | |
| FE2 | | FE3 |<------------>| CE |
--------- --------- | |
^ ^ ^ ----------
| \ / ^ ^
| \ / | |
| v v | |
| ----------- | |
| | FE4 |<----------------------| |
| ----------- |
| | ^ |
| v | |
| |
|----------------------------------------|
(c) Multiple FEs connected by a ring
Figure 6. Some examples of FE topology3.3. CE Managers 🎀
CE managers are responsible for deciding which FEs a CE should control 💅. it's legit for CE managers to be hard-coded with the knowledge of which FEs their CEs should talk to 🌸. a CE manager may also be physically built into a CE and be implemented as a simple keypad or other direct config mechanism on the CE ⭐. finally, CE managers may be physically and logically separate entities that configure the CE with FE info via mechanisms like COPS-PR [7] or SNMP [5] 🧚.
3.4. FE Managers 🦋
FE managers are responsible for deciding which CE any particular FE should initially talk to 🫶. like CE managers, no restrictions are placed on how an FE manager decides which CE its FEs should communicate with, and no restrictions are placed on how FE managers are implemented 🌈. each FE should have one and only one FE manager, while different FEs may have the same or different FE manager(s) 👑. each manager can choose to exist and operate independently of other managers ✨.
4. Operational Phases 💖
both FEs and CEs need some configuration in place before they can start exchanging info and function as a coherent network element 🔥. two operational phases are identified in this framework: pre-association and post-association 🦋.
4.1. Pre-association Phase 🌸
the pre-association phase is the period of time during which an FE Manager and a CE Manager are figuring out whether an FE and a CE should be part of the same network element 🧚. the protocols used during this phase may include all or some of the message exchange over Fl, Ff, and Fc reference points. however, all of these may be optional, and none of this is within the scope of the ForCES Protocol 🙈.
4.1.1. Fl Reference Point 💜
CE managers and FE managers may communicate across the Fl reference point in the pre-association phase to determine whether an individual CE and FE, or a set of CEs and FEs, should be associated 🫶. communication across the Fl reference point is optional in this architecture. no requirements are placed on this reference point 🌈.
CE managers and FE managers may be operated by different entities 👀. the operator of the CE manager may not want to divulge any characteristics of the CEs it manages, except to specified FE managers 🔒. similarly, the operator of the FE manager may not want to divulge FE characteristics, except to authorized entities 💅. as such, CE managers and FE managers may need to authenticate one another 🎀. subsequent communication between CE managers and FE managers may require other security functions such as privacy, non-repudiation, freshness, and integrity 😭.
FE Manager FE CE Manager CE
| | | |
| | | |
|(security exchange) | |
1|<------------------------------>| |
| | | |
|(a list of CEs and their attributes) |
2|<-------------------------------| |
| | | |
|(a list of FEs and their attributes) |
3|------------------------------->| |
| | | |
| | | |
|<----------------Fl------------>| |Figure 7. An example of a message exchange over the Fl reference point 🍓
once the necessary security functions have been performed, the CE and FE managers communicate to determine which CEs and FEs should communicate with each other 🌟. at the very minimum, the CE and FE managers need to learn about the existence of available FEs and CEs, respectively 💫. this discovery process may involve one or both managers learning the capabilities of the discovered ForCES protocol elements ✨. Figure 7 shows an example of a possible message exchange between the CE manager and FE manager over the Fl reference point 👑.
4.1.2. Ff Reference Point 🎀
okay so the Ff reference point is where the FE manager tells forwarding elements about the association decisions it made back in the pre-association phase 💅. Only authorized entities get to tell an FE which CE should control it, so privacy, integrity, freshness, and authentication are necessary between the FE manager and the FEs whenever the FE manager is remote 🔐. Once that security is sorted, the FE manager tells the FEs across this reference point to join a new NE or disconnect from an existing one 🦋. The FE Manager can also hand out unique FE identifiers over this same channel, and those IDs are super useful later in the post-association phase for expressing FE topology ✨. Figure 8 shows an example of a message exchange over Ff.
FE Manager FE CE Manager CE
| | | |
| | | |
|(security exchange) |(security exchange)
1|<------------>|authentication 1|<----------->|authentication
| | | |
|(FE ID, attributes) |(CE ID, attributes)
2|<-------------|request 2|<------------|request
| | | |
3|------------->|response 3|------------>|response
|(corresponding CE ID) |(corresponding FE ID)
| | | |
| | | |
|<-----Ff----->| |<-----Fc---->|
Figure 8. Examples of a message exchange
over the Ff and Fc reference pointsAlso, the FE manager function can be co-located with the FE 🧚. Like, you could literally type in the CE IP address on a manual keypad, and in that case this reference point is reduced to a built-in function.
4.1.3. Fc Reference Point 💜
the Fc reference point is used to tell control elements about the association decisions CE managers made in the pre-association phase 🫶. When the CE manager is remote, only authorized entities may instruct a CE to control certain FEs. Privacy, integrity, freshness, and authentication are also required across this reference point in that kind of setup 🔒. Once appropriate security is established, the CE manager tells the CEs which FEs they should control and how they should control them. Figure 8 shows an example of a message exchange over Fc too.
As with the FE manager and FEs, there are configurations where the CE manager and CE are co-located and no protocol is used for this function 🌈.
4.2. Post-association Phase and Fp reference point 🌸
the Post-association phase is the stretch of time when an FE and a CE have been configured with the info they need to contact each other. It covers both association establishment and steady-state communication 💖. Communication between CE and FE happens across the Fp ("p" meaning protocol) reference point, and ForCES Protocol is exclusively used for all communication across Fp 🎀.
4.2.1. Proximity and Interconnect between CEs and FEs 🦋
the ForCES Working Group made a conscious decision that the first version of ForCES is going to focus on "very close" CE/FE localities in IP networks 👑. Very Close localities are control and forwarding elements that are either components in the same physical box, or separated by at most one local network hop ([8]). CEs and FEs can connect through a bunch of interconnect technologies, including Ethernet connections, backplanes, ATM (cell) fabrics, etc. ForCES should be able to support each of these interconnects (see [4] Section 5, requirement #1). When CEs and FEs are separated beyond a single L3 routing hop, the ForCES Protocol will make use of an existing RFC 2914 [3] compliant L4 protocol with adequate reliability, security, and congestion control (e.g., TCP, SCTP) for transport purposes 🚀.
4.2.2. Association Establishment 🎀
FE CE
| |
|(Security exchange.) |
1|<--------------------->|
| |
|(Let me join the NE please.)
2|---------------------->|
| |
|(What kind of FE are you? -- capability query)
3|<----------------------|
| |
|(Here is my FE functions/state: use model to
describe)
4|---------------------->|
| |
|(Initial config for FE -- optional)
5|<----------------------|
| |
|(I am ready to go. Shall I?)
6|---------------------->|
| |
|(Go ahead!) |
7|<----------------------|
| |
Figure 9. Example of a message exchange between CE and FE
over Fp to establish an NE associationso, as an example, figure 9 shows some of the message exchange that may happen before the CE and FE association is fully established 🧚. Either the CE or the FE can start the connection, bestie 🫶.
The security handshake is necessary to authenticate the two communication endpoints to each other before any further message exchange can happen 🔐. The handshake should include mutual authentication and authorization between the CE and FE, but the exact details depend on the security solution chosen by the ForCES Protocol 💅. Authorization can be as simple as checking against the list of authorized end points provided by the FE or CE manager during the pre-association phase. Both authentication and authorization must be successful before the association can be established. If either authentication or authorization fails, the end point must not be allowed to join the NE 🚫. After the successful security handshake, message authentication and confidentiality are still necessary for the ongoing information exchange between the CE and FE, unless some form of physical security exists 🔒. Whenever a packet fails authentication, it must be dropped, and a notification may be sent to alert the sender of the potential attack 💀. Section 8 provides more details on the security considerations for ForCES.
⚠️ security is literally the main character here, so don't skip it: a failed authentication or authorization check means the end point is out, and a packet that fails authentication gets dropped. iykyk 😭
After the successful security handshake, the FE needs to inform the CE of its own capability and, optionally, its topology in relation to other FEs 🌸. The capability of the FE shall be represented by the FE model, as required in [4] (Section 6, requirement #1). The model would let an FE describe what kind of packet processing functions it contains, what order the processing happens in, what kinds of configurable parameters it allows, what statistics it collects, and what events it might throw, etc. 🍓 Once that info is available to the CE, the CE may choose to send some initial or default configuration to the FE so that the FE can start receiving and processing packets correctly. Such initialization may not be necessary if the FE already gets the info from its own bootstrap process. Once the necessary initial info is exchanged, the association is completed. Packet processing and forwarding at the FE cannot begin until the association is established. After the association is established, the CE and FE enter steady-state communication ✨.
4.2.3. Steady-state Communication ✨
Once an association is established between the CE and FE, the ForCES Protocol is used by the CE and FE over the Fp reference point to exchange information that helps with packet processing 💜.
FE CE
| |
|(Add these new routes.)|
1|<----------------------|
| |
|(Successful.) |
2|---------------------->|
| |
| |
|(Query some stats.) |
1|<----------------------|
| |
|(Reply with stats collected.)
2|---------------------->|
| |
| |
|(My port is down, with port #.)
1|---------------------->|
| |
|(Here is a new forwarding table)
2|<----------------------|
| |
Figure 10. Examples of a message exchange between CE and FE
over Fp during steady-state communicationbased on the info the CEs get from their control processing, CEs will frequently need to manipulate the packet-forwarding behaviors of their FE(s) by sending them instructions 🎀. For example, Figure 10 shows message exchange examples where the CE sends new routes to the FE so the FE can add them to its forwarding table 🛣️. The CE may query the FE for statistics the FE collected, and the FE may notify the CE of important events, such as a port failure 🚨.
4.2.4. Data Packets across Fp reference point ✨
Okay bestie, this is where the diagram gets to be a whole thing 🦋 Figure 11 below shows data packets moving between two NEs.
--------------------- ----------------------
| | | |
| +--------+ | | +--------+ |
| |CE(BGP) | | | |CE(BGP) | |
| +--------+ | | +--------+ |
| | | | ^ |
| |Fp | | |Fp |
| v | | | |
| +--------+ | | +--------+ |
| | FE | | | | FE | |
| +--------+ | | +--------+ |
| | | | ^ |
| Router | | | Router | |
| A | | | B | |
---------+----------- -----------+----------
v ^
| |
| |
------------------->---------------Figure 11. Example to show data packet flow between two NEs. 💖
Control plane protocol packets, like RIP and OSPF messages, that are addressed to any of an NE's interfaces usually get redirected by the receiving FE to its CE 🌸 The CE can also originate packets and have its FE deliver them to other NEs. So the ForCES Protocol over Fp isn't just carrying ForCES Protocol messages between CEs and FEs. It also encapsulates the data packets from control plane protocols. Main character energy, literally 👑
It gets even more fun with distributed control. One FE can be controlled by multiple CEs, so the control protocols supported by the FORCES NEs can be spread across several CEs. For example, one CE might run routing protocols like OSPF and BGP, while a signaling and admission control protocol like RSVP lives in a different CE 🧚 FEs are configured to recognize and filter these protocol packets, then forward them to the right CE. Iconic, honestly 💅
Figure 11 shows one example of how BGP packets originated by router A get passed to router B. The ForCES Protocol carries the packets from the CE to the FE inside router A, and then from the FE to the CE inside router B. Since the ForCES Protocol handles both control messages and data packets between the CE and FE over the Fp reference point, it's possible to use either a single protocol or multiple protocols 🎀
4.2.5. Proxy FE 🦋
Sometimes a physical FE just can't run the ForCES Protocol directly, like when it doesn't have a general purpose CPU 😭 In that case, a proxy FE can terminate the Fp reference point instead of the physical FE. The CE talks to the proxy using ForCES, and the proxy manipulates the physical FE through some other form of communication, like a non-ForCES protocol or DMA. Together, the proxy and the physical FE count as one logical FE entity. One proxy can also act on behalf of multiple physical FEs 🌈
Now the security part, and it's serious ngl 🔥 Because the physical FE can't implement ForCES itself, the ForCES security mechanism only protects the channel between the CE and the proxy FE. It doesn't reach all the way to the physical FE. It is recommended that other security mechanisms, including physical security properties, be used to protect the communication between the CE and the physical FE 🫶
4.3. Association Re-establishment 🌸
FEs and CEs can join and leave NEs dynamically (see [4] Section 5, requirements #12) 💜 When an FE or CE leaves an NE, the association with that NE breaks. If the leaving party rejoins the NE later and wants to re-establish the association, it may need to go back into the pre-association phase. The association can also be lost unexpectedly if the connection between the CE and the FE drops. So the framework allows a bi-directional transition between these two phases, but the ForCES Protocol only applies to the post-association phase. Even so, the protocol should provide mechanisms that support association re-establishment. That includes letting CEs and FEs determine when they've lost association with each other, and restoring association along with efficient state (re)synchronization mechanisms (see [4] Section 5, requirement #7) 🧚 Security association and state must also be re-established, so the same level of security (including both authentication and authorization) exists before and after the association is re-established. Non-negotiable, bestie 🔒
When an FE leaves or joins an NE that's already running, the CE needs to be aware of how the FE topology has changed and deal with it accordingly 🍓
4.3.1. CE graceful restart 💀
When a CE in a router fails and restarts, it can cause a lot of stress and disruption across the control plane of the whole network 😩 Restarting a CE for any reason means the router loses its routing adjacencies or sessions with its routing neighbors. The neighbors that notice the lost adjacency usually recompute their routes and send routing updates to their own neighbors to share the news. Those neighbors do the same, and it spreads through the network. Meanwhile, the restarting router can't receive traffic from other routers, because its neighbors have stopped using the routes it previously advertised. When the restarting router brings its adjacencies back, its neighbors have to recompute routes and send out more routing updates all over again. The restarting router can't forward packets until it has re-established its routing adjacencies, received route updates through them, and computed new routes. Until the whole network converges, packets can get lost in transient black holes or forwarding loops. It's giving chaos 🌀
A high availability mechanism called "graceful restart" is used by the IP routing protocols (OSPF [11], BGP [12], IS-IS [13]) and the MPLS label distribution protocol (LDP [10]) to reduce the negative effects a restarting router has on routing across the whole network ✨ It helps avoid route flapping on neighboring routers, and the restarting router can keep forwarding packets that would otherwise get dropped. Literally the glow-up we needed 🌟
The details differ from protocol to protocol, but the general idea of graceful restart stays the same 🦋 With graceful restart, a restarting router can tell its neighbors when it's restarting. The neighbors may notice the lost adjacency, but they don't recompute routes or send routing updates to their neighbors. They also keep the routes they got from the restarting router before the restart and treat them as still valid for a limited time. That way, the restarting router's FEs can keep receiving and forwarding traffic from other neighbors for a limited time using the routes they already have 💅 The restarting router then re-establishes its routing adjacencies, downloads updated routes from all of its neighbors, recomputes new routes, and uses them to replace the older ones. After that, it sends these updated routes to its neighbors and signals that the graceful restart process is complete 🎉
Non-stop forwarding is a requirement for graceful restart 🔥 A router needs it so it can keep forwarding packets while it downloads routing information and recomputes routes. That's how packets don't get dropped. And one of the perks of separating the CE and FE is exactly this: non-stop forwarding even when the CE fails and restarts 👑 Because ForCES supports dynamic changes to CE/FE association, it's also compatible with high availability mechanisms like graceful restart 🫶
ForCES should be able to support a CE graceful restart easily 🌈 When the association is established for the first time, the CE must tell the FEs what to do if the CE fails. If graceful restart isn't supported, the FEs may be told to stop packet processing altogether if the CE fails. If graceful restart is supported, the FEs should be told to cache and hold on to their FE state, including the forwarding tables, across restarts. A timer must be included so that a timeout eventually expires that cached state. Those timers should be settable by the CE ⏰
4.3.2. FE restart 🧚
Using the same example from Figure 5, let's say CE1 is the working CE for now. What happens if one of the FEs, say FE1, leaves the NE temporarily? 👀 FE1 may voluntarily decide to leave the association. Or FE1 may stop working because of an unexpected failure. In the first case, CE1 gets a "leave-association request" from FE1. In the second, CE1 detects FE1's failure some other way. Either way, CE1 must tell the routing protocols about the event. That will most likely trigger a reachability and SPF (Shortest Path First) recalculation, and CE1 will download new FIBs to the other remaining FEs (only FE2 in this example). That recalculation and those FIB updates also get propagated from CE1 to the NE's neighbors that are affected by FE1's connectivity 🌸
When FE1 decides to rejoin, or restarts after a failure, FE1 needs to rediscover its master (CE) 🔍 It can do that in several ways. It may re-enter the pre-association phase and get the information from its FE manager. It may also pull the previous CE information from its cache, as long as it can validate that the information is still fresh. Once FE1 finds its CE, it starts a message exchange with the CE to re-establish the association, as outlined in Figure 9. The one possible difference is that it might be able to skip transporting the complete initial configuration 🦋 Suppose FE1 still has its routing table and other state information from the last association. Instead of resending everything, it may be able to use a more efficient mechanism to re-sync its state with the CE, if the ForCES Protocol supports one. For example, a CRC-32 of the state might give a quick indication of whether the state is in sync with the CE. By comparing its state with the CE first, FE1 sends an information update only when it's needed 💖 The ForCES Protocol may choose to implement similar optimization mechanisms, but it may also choose not to, since this isn't a requirement ✨
5. Applicability to RFC 1812 🎀
[4] Okay bestie, Section 5, requirement #9 says "Any proposed ForCES architecture must explain how that architecture supports all of the router functions as defined in RFC 1812." 💅 RFC 1812 [2] covers loads of important requirements for IPv4 routers, from the link layer all the way up to the application layer. So this section sorts out the relevant bits for building IPv4 routers on the ForCES architecture 🦋 and shows how ForCES handles them by giving guidance on splitting the functions between the forwarding plane and the control plane. ✨
In general, the forwarding plane does the bulk of the per-packet work that has to happen at line speed, while the control plane handles the computationally complex stuff that's typical of control and signaling protocols. ngl though, there's no rigid line for dividing the processing between CEs and FEs, and the ForCES architecture shouldn't limit innovative approaches to control and forwarding plane separation 🌈 As FEs get more processing power, some functions that traditionally live on CEs may move over to FEs for better performance and scalability 🚀 Stuff like part of ICMP or TCP processing, or part of routing protocols, can get offloaded like that. 🧚
Once offloaded onto the forwarding plane, those CE functions, even though they logically belong to the control plane, become part of the FE functions 👑 Just like the other logical functions on FEs, offloaded functions must be expressed as part of the FE model, so CEs can decide how to best take advantage of them when they're present on the FEs. 💖
5.1. General Router Requirements
Routers have at least two or more logical interfaces. 🌸 When CEs and FEs are separated by ForCES within a single NE, some additional interfaces are needed for intra-NE communications, as shown in figure 12. This NE has one CE and two FEs. Each FE has four interfaces: two of them receive and transmit packets to the external world, and the other two are for intra-NE connections. The CE has two logical interfaces, #9 and #10, connected to interfaces #3 and #6 on FE1 and FE2 respectively. Interfaces #4 and #5 are connected for FE1-FE2 communication. So this router NE gives you four external interfaces (#1, 2, 7, and 8). 🍓
---------------------------------
| router NE |
| ----------- ----------- |
| | FE1 | | FE2 | |
| ----------- ----------- |
| 1| 2| 3| 4| 5| 6| 7| 8| |
| | | | | | | | | |
| | | | +----+ | | | |
| | | | | | | |
| | | 9| 10| | | |
| | | -------------- | | |
| | | | CE | | | |
| | | -------------- | | |
| | | | | |
-----+--+----------------+--+----
| | | |
| | | |
Figure 12. A router NE example with four interfaces.IPv4 routers must implement IP to support their packet forwarding function, which is driven by their FIB (Forwarding Information Base). 📦 This Internet layer forwarding (see RFC 1812 [2] Section 5) naturally belongs to FEs in the ForCES architecture. 💅
A router may implement transport layer protocols (like TCP and UDP) that are required to support application layer protocols (see RFC 1812 [2] Section 6). 🎀 One important class of application protocols is routing protocols (see RFC 1812 [2] Section 7). In the ForCES architecture, routing protocols are naturally implemented by CEs. 🧠 Routing protocols need routers to talk to each other, and ForCES supports that because FEs can redirect data packets addressed to routers (i.e., NEs), and CEs can originate packets and have their FEs deliver them. 📬 This communication happens across the Fp reference point inside each router and between neighboring routers' external interfaces, as shown in Figure 11. 🦋
5.2. Link Layer
Since FEs own all the external interfaces for the router, FEs need to conform to the link layer requirements in RFC 1812 [2]. 🔗 Arguably, ARP support may be implemented in either CEs or FEs. As we'll see later, a bunch of behaviors that RFC 1812 mandates fall into this category: they may be performed by the FE, and they may also be performed by the CE. 🤔 A general guideline is needed to keep separated control and forwarding planes interoperable, and the one offered here is that CEs MUST be capable of these kinds of operations, while FEs MAY choose to implement them. ✨ The FE model should indicate its capabilities in this regard, so CEs can decide where these functions get implemented. 💜
Interface parameters, including MTU, IP address, etc., must be configurable by CEs via ForCES. 🛠️ CEs must be able to determine whether a physical interface on an FE is available to send packets or not. 🌈 FEs must also inform CEs of interface status changes (like link up/down) via ForCES. 🚨
5.3. Internet Layer Protocols
Both FEs and CEs must implement the IP protocol and all mandatory extensions, as RFC 1812 specifies. 🌟 CEs should implement IP options like source route and record route, while FEs may choose to implement those as well. ⭐ The timestamp option should be implemented by FEs so they can insert the timestamp most accurately. ⏱️ The FE must interpret the IP options it understands and preserve the rest unchanged for use by CEs. 🫶 Both FEs and CEs might choose to silently discard packets without sending ICMP errors, but those events should be logged and counted. 📝 FEs may report statistics for such events to CEs via ForCES. 📊
When multiple FEs are involved in processing packets, the appearance of a single NE must be strictly maintained. 🎭 For example, Time-To-Live (TTL) must be decremented only once within a single NE. It can always be decremented by the last FE with the egress function. 🏁
FEs must receive and process normally any packets with a broadcast destination address, or a multicast destination address that the router has asked to receive. 📡 When IP multicast is supported in routers, IGMP is implemented in CEs. 🌐 CEs are also required to support ICMP, while it is optional for FEs to support ICMP. 🙈 That option can be communicated to CEs as part of the FE model. So FEs can always rely on CEs to send out ICMP error messages, but FEs also have the option of generating ICMP error messages themselves. 💅
5.4. Internet Layer Forwarding
IP forwarding is implemented by FEs. 🚚 When the routing table is updated at the CEs, ForCES is used to send the new route entries from the CEs to the FEs. Each FE has its own forwarding table and uses it to direct packets to the next hop interface. 🗺️
Upon receiving IP packets, the FE verifies the IP header and processes most of the IP options. 🔍 Some options can't be processed until the routing decision has been made. The routing decision is made after examining the destination IP address. 🧭 If the destination address belongs to the router itself, the packets are filtered and either processed locally or forwarded to the CE, depending on the instructions set up by the CE. 🎀 Otherwise, the FE determines the next hop IP address by looking in its forwarding table, and it also works out which network interface to use to send the packets. 🛣️ Sometimes an FE may need to forward the packets to another FE before they can go out to the next hop. 🔁 Right before packets are forwarded out to the next hop, the FE decrements TTL by 1 and processes any IP options that could not be processed before. ⏳ The FE then performs IP fragmentation if necessary, determines the link layer address (e.g., by ARP), and encapsulates the IP datagram (or each of its fragments) in an appropriate link layer frame, then queues it for output on the selected interface. 📦✨
Other options mentioned in RFC 1812 [2] for IP forwarding may also be implemented at FEs, for example packet filtering. 🧱
FEs typically forward packets destined for the router locally to CEs. 💌 FEs may also forward exceptional packets (packets that FEs don't know how to handle) to CEs. 🆘 CEs are required to handle packets forwarded by FEs, for whatever reason. 🫡 It might be necessary for ForCES to attach some meta-data to the packets to indicate why FEs forwarded them to CEs. 🏷️ Upon receiving packets with meta-data from FEs, CEs can decide to either process the packets themselves, or pass them to upper layer protocols, including routing and management protocols. 🧩 If CEs are going to process the packets themselves, they may choose to discard the packets, or modify and re-send them. 🔄 CEs may also originate new packets and deliver them to FEs for further forwarding. 🚀
Any state change during router operation must also be handled correctly according to RFC 1812. 🛡️ For example, when an FE stops forwarding, the entire NE may continue forwarding packets, but it needs to stop advertising the routes affected by the failed FE. 📢 Not gonna lie, this is the kind of detail that makes or breaks a router 😭
5.5. Transport Layer
The Transport layer is typically implemented at CEs to support higher layer application protocols like routing protocols. 🚛 In practice, this means most CEs implement both the Transmission Control Protocol (TCP) and the User Datagram Protocol (UDP). 💕
Both CEs and FEs need to implement the ForCES Protocol. 🤝 If some layer-4 transport is used to support ForCES, then both CEs and FEs need to implement the L4 transport and ForCES Protocols. 🧦
5.6. Application Layer -- Routing Protocols
Interior and exterior routing protocols are implemented on CEs. 🗺️ The routing packets that CEs originate are forwarded to FEs for delivery. 📮 The results of these protocols (like forwarding table updates) are communicated to FEs via ForCES. 🔔
For performance or scalability reasons, portions of the control plane functions that need faster response may be moved off the CEs and onto the FEs. 🏎️ For example, in OSPF, Hello protocol packets are generated and processed periodically. When that's done at the CEs, inbound Hello packets have to travel from the external interfaces at the FEs to the CEs over the internal CE-FE channel. Outbound Hello packets go the other way, from the CEs to the FEs and out the external interfaces. 🔁 Frequent Hello updates pile heavy processing overhead onto the CEs and can overwhelm the CE-FE channel too. 😩 Since there are typically far more FEs than CEs in a router, the off-loaded Hello packets get processed in a much more distributed and scalable fashion. 🌈 By expressing such off-loaded functions in the FE model, we can ensure interoperability. 🫶 However, the exact description of the off-loaded functionality that corresponds to the off-loaded functions expressed in the FE model is not part of the model itself, and it will need to be worked out as a separate specification. 📜
5.7. Application Layer -- Network Management Protocol 🎀
okay so RFC 1812 [2] says "Routers MUST be manageable by SNMP" 😭 and bestie, that is a whole MUST, so it stays right there in caps. 💅
Once things are associated, most outside management tasks (SNMP included) should go through the CE so the whole box looks like one functional device ✨ That's the vibe. So the recommendation is that an SNMP agent gets implemented on the CEs, and any SNMP messages that land on an FE get redirected to its CE 🦋 The AgentX framework from RFC 2741 ([6]) may be applied here: the CEs play the master agent and process the SNMP protocol messages, while the FEs act as subagents that give access to the MIB objects living on the FEs 🧚 The AgentX messages between the master agent (CE) and the subagent (FE) get encapsulated and carried over ForCES, just like data packets from any other application layer protocol 🌸
6. Summary ✨
okay so this document is the architectural framework for ForCES 💖 It maps out the parts of a ForCES network element: one or more FEs, one or more CEs, one optional FE manager, and one optional CE manager. It also covers how all of these pieces talk to each other and walks through every major reference point 🗺️ Here's the tea: out of all those reference points, only the Fp interface between CEs and FEs is actually inside ForCES's scope 👑 ForCES on its own might not cover every fancy NE configuration, but ForCES over an Fp interface is the big one for making CEs and FEs physically separate and interoperable, which is why it's the first interface that ought to get standardized 🔥 Simple and useful setups can still work with only the CE-FE interface standardized, like a single CE with full-meshed FEs 🍓
7. Acknowledgements 🫶
Joel M. Halpern dropped so many insightful comments and suggestions and pointed out several major issues 🙌 T. Sridhar suggested that the AgentX protocol could be used with SNMP to manage the ForCES network elements 🧠 Susan Hares flagged the graceful restart issue with ForCES 💜 And Russ Housley, Avri Doria, Jamal Hadi Salim, and many others on the ForCES mailing list gave valuable feedback too, so thank you all, you're literally the best 🌈
8. Security Considerations 💀
okay this section is serious, so buckle up 😭
The NE administrator gets to decide the exact security configuration for their specific deployment 🔐 For example, ForCES might be deployed between CEs and FEs that sit inside the same box and connect over a backplane. In that setup, physical security of the box means most attacks, like man-in-the-middle, snooping, and impersonation, aren't possible 🙈 So the ForCES architecture may rely on that physical security to defend against these attacks, and protocol mechanisms may be turned off. BUT, and this is a big but, denial of service attacks via external interfaces (more on this in Section 8.1.8) are still a real threat even in an "all-in-one-box" deployment 💥 That's why rate limiting is still necessary. This is just one example of why it matters to assess the security needs of ForCES-enabled network elements under each deployment scenario 🧐 It should be possible for the administrator to configure the level of security needed for the ForCES Protocol ✨
In general, physically separating two entities usually creates a potentially insecure link between them, so much stricter security measures are required 🚨 For example, Section 4.1 pointed out that authentication becomes necessary between the CE manager and FE manager, between the CE and CE manager, and between the FE and FE manager in some configurations 🔑 The physical separation of the CE and FE also brings serious security requirements for the ForCES Protocol over the Fp interface 😱 This section first looks at the security threats that physical separation of FEs and CEs can introduce, and then gives recommendations and guidelines for securely operating and managing the ForCES Protocol over the Fp interface, based on existing standard security solutions 🛡️
8.1. Analysis of Potential Threats Introduced by ForCES 🕵️
This section is the threat analysis for ForCES, with a focus on the Fp interface 🔍 Each threat gets described in detail, including its effects on the ForCES Protocol entities and/or the NE as a whole, plus the functionality that needs to be in place to defend against it 💅
8.1.1. "Join" or "Remove" Message Flooding on CEs 🌊
Threats: A malicious node could send a stream of false "join NE" or "remove from NE" requests on behalf of a non-existent or unauthorized FE to legitimate CEs at a very rapid rate, creating unnecessary state in the CEs 😤
Effects: If a CE maintains state for non-existent or unauthorized FEs, it may become unavailable for other processing and suffer a denial of service (DoS) attack similar to the TCP SYN DoS 💀 If multiple CEs are used, the unnecessary state information may also be conveyed to multiple CEs via the Fr interface (e.g., from the active CE to the stand-by CE), subjecting multiple CEs to a DoS attack 😭
Requirement: A CE that receives a "join" or "remove" request should not create any state information until it has authenticated the FE endpoint 🔐
8.1.2. Impersonation Attack 🎭
Threats: A malicious node can impersonate a CE or FE and send out false messages 🙈
Effects: The whole NE could be compromised 💔
Requirement: The CE or FE must authenticate the message as having come from an FE or CE on the list of the authorized ForCES elements (provided by the CE or FE Manager in the pre-association phase) before accepting and processing it 🛂
8.1.3. Replay Attack 🔁
Threat: A malicious node could replay an entire message previously sent by an FE or CE entity to get around authentication 😈
Effect: The NE could be compromised 💀
Requirement: A replay protection mechanism needs to be part of the security solution to defend against this attack 🛡️
8.1.4. Attack during Fail Over 🌩️
Threat: A malicious node may exploit the CE fail-over mechanism to take over control of the NE 👀 For example, suppose two CEs, say CE-A and CE-B, are controlling several FEs. CE-A is active and CE-B is stand-by. When CE-A fails, CE-B takes over the active CE position. The FEs already had a trusted relationship with CE-A, but they may not have the same trusted relationship established with CE-B prior to the fail-over. A malicious node can take over as CE-B if such a trusted relationship hasn't been established prior to or during the fail-over 😱
Effect: The NE may be compromised after such an insecure fail-over 💔
Requirement: The level of trust between the stand-by CE and the FEs must be as strong as the one between the active CE and the FEs 💪 The security association between the FEs and the stand-by CE may be established prior to fail-over. If it isn't already in place, that security association must be re-established before the stand-by CE takes over 🔄
8.1.5. Data Integrity 🧾
Threats: A malicious node may inject false messages to a legitimate CE or FE 🤡
Effect: An FE or CE receives the fabricated packet and performs an incorrect or catastrophic operation 💥
Requirement: Protocol messages require integrity protection 🔒
8.1.6. Data Confidentiality 🤫
Threat: When the FE and CE are physically separated, a malicious node may eavesdrop on the messages in transit 👂 Some of the messages are critical to the functioning of the whole network, while others may contain confidential business data. Leaking that information may lead to a compromise even beyond the immediate CE or FE 😰
Effect: Sensitive information might be exposed between the CE and FE 🫣
Requirement: Data confidentiality between the FE and CE must be available for sensitive information 🔐
8.1.7. Sharing security parameters 🔑
Threat: Picture this: several FEs are talking to the same CE and sharing the same authentication keys for the Fp interface. If any FE or CE is compromised, all the other entities are compromised too 😵
Effect: The whole NE is compromised 💔
Recommendation: To avoid this side effect, it's better to configure different security parameters for each FE-CE communication over the Fp interface 🧩
8.1.8. Denial of Service Attack via External Interface 🌪️
Threat: When an FE receives a packet destined for its CE, the FE forwards the packet over the Fp interface. A malicious node can generate a huge message storm, such as routing protocol packets, through the external Fi/f interface, so the FE has to process and forward all of those packets to the CE through the Fp interface 🌊
Effect: The CE encounters resource exhaustion and bandwidth starvation on the Fp interface due to an overwhelming number of packets from FEs 🥵
Requirement: Some sort of rate limiting mechanism MUST be in place at both the FE and CE. The Rate Limiter SHOULD be configured at the FE for each message type received through the Fi/f interface 🚦
Yes, that MUST and SHOULD are doing a lot of work here, so don't skip them 💅
8.2. Security Recommendations for ForCES 🛡️
The requirements document [4] suggested that the ForCES Protocol should support reliability over the Fp interface, but no particular transport protocol has been specified for ForCES yet 🚚 This framework document doesn't intend to specify a particular transport either, so we only provide recommendations and guidelines based on the existing standard security protocols [18] that can work with the common transport candidates suitable for ForCES 🫶
We review two existing security protocol solutions: IPsec (IP Security) [15] and TLS (Transport Layer Security) [14]. TLS works with reliable transports such as TCP or SCTP for unicast, while IPsec can be used with any transport (UDP, TCP, SCTP) and supports both unicast and multicast 🦋 Both TLS and IPsec can potentially be used to satisfy all of the security requirements for the ForCES Protocol ✨ Other approaches that satisfy the requirements can be used as well, but they aren't documented here, including the use of L2 security mechanisms for a given L2 interconnect technology 🌈
When ForCES is deployed between CEs and FEs inside a box or a physically secured room, authentication, confidentiality, and integrity may be provided by the physical security of the box 📦 Thus, the security mechanisms may be turned off, depending on the networking topology and its administration policy 🙃 But it's important to realize that even in a single-box NE, the DoS attacks described in Section 8.1.8 can still be launched through the Fi/f interfaces 😬 So it's important to have the corresponding countermeasures in place, even for single-box deployments 🚨
8.2.1. Using TLS with ForCES ✨
okay so TLS [14] can be used if you're running a reliable unicast transport like TCP or SCTP for ForCES over the Fp interface 🔐 The TLS handshake protocol does the negotiating during association establishment or re-establishment, and it sets up a TLS session between the CE and FE. Once that session is live, the TLS record protocol keeps ForCES messages between the CE and FE locked down 💖
Here's the basic flow, bestie 🦋 Steps 1) through 7) are the security handshake shown in Figure 9. Step 8) covers everything after that, including the rest of the messages past the handshake and the steady-state communication in Figure 10.
- During the Pre-association phase, every FE gets configured with the CEs, both the active CE and the standby CE 🌸
- The FE sets up a TLS connection with the CE (master) and they agree on a cipher suite 🎀
- The FE (slave) grabs the CE certificate, checks the signature, checks the expiration date, and checks whether the cert has been revoked 👀
- The CE (master) grabs the FE certificate and runs the exact same validation as the FE in step 3) 🫶
- If any check in step 3) or step 4) fails, the endpoint must generate an error message and abort 💀
- After successful mutual authentication, a TLS session is established between the CE and FE ✨
- The FE sends a "join NE" message to the CE 🧚
- The FE and CE keep using the TLS session for all further communication 💅
Note that there are different ways to validate a received certificate, and it's giving flexibility 🌈 One option is to configure the FE Manager, the CE Manager, or some other central component as the CA. Then the CE or FE can ask that pre-configured CA whether the certificate has been revoked. Another option is for the CE and FE to directly configure a list of valid certificates during the pre-association phase 🍓
Now for fail-over, which is a whole main character moment 👑 The active CE and standby CE are responsible for syncing ForCES states, including the TLS states, so that state re-establishment during fail-over stays minimal ⭐ Be careful here: the standby CE has to be authenticated the same way as the active CE, either before or during the fail-over. no cap, this part matters 🙈
8.2.2. Using IPsec with ForCES 🔥
IPsec [15] works with any transport protocol, like UDP, SCTP, and TCP, over the Fp interface for ForCES 💜 When you use IPsec, we recommend ESP in transport mode for ForCES, because message confidentiality is required for ForCES 🔐
IPsec can run with manual or automated SA and cryptographic key management, but here's the tea 🍵 IPsec's replay protection isn't available if you go manual. So if replay protection matters to you, automatic key management is recommended. Manual key management might still be enough for some deployments, especially when there aren't many CEs and FEs. Even with manual keys, it's recommended that you change the keys periodically 🗝️
IPsec handles both unicast and multicast transport 🌸 When this document was published, the MSEC working group was actively standardizing protocols for multicast security [17]. Multicast solutions that rely on IPsec should specify how they meet the security requirements in [4] 🦋
Unlike TLS, IPsec gives security services between the CE and FE at the IP level. So the security handshake shown in Figure 9 turns into a "no-op" when manual key management is used 🙃 Here's what happens for ForCES in that case:
- During the Pre-association phase, all the FEs get configured with the CEs (the active CE and standby CE) and the SA parameters, all by hand 🛠️
- The FE sends a "join NE" message to the CE. This message and every one after it gets security services under the manually configured IPsec SA parameters, but replay protection isn't available 💀
It's up to the admin whether multiple FE-CE communications share the same key, but using different keys is recommended 🔑 Same deal for inbound and outbound traffic: different keys are recommended 🫶
If you need automatic key management, IKE [16] can handle it. Other automatic key distribution techniques, like Kerberos, can work too ✨ The key exchange process is the security handshake from Figure 9. Here's how it goes with IKE and IPsec for ForCES. Steps 1) to 6) make up the security handshake in Figure 9 💅
- During the Pre-association phase, all FEs get configured with the CEs (the active CE and standby CE), the IPsec policy, and so on 🎀
- The FE kicks off the IKE process and tries to set up an IPsec SA with the CE (master). The FE (slave) gets the CE certificate as part of the IKE negotiation. The FE checks the signature, checks the expiration date, and checks whether the cert has been revoked 👀
- The CE (master) grabs the FE certificate and runs the same check the FE ran in step 2) 🫶
- If any check in step 2) or step 3) fails, the endpoint must generate an error message and abort 💀
- After successful mutual authentication, the IPsec session is established between the CE and FE 🌈
- The FE sends a "join NE" message to the CE. No SADB entry has been created in the FE yet 🧚
- The FE and CE keep using the IPsec session for further communication 🍓
The FE Manager, CE Manager, or some other central component can act as a CA for validating CE and FE certificates during IKE. Or, during the pre-association phase, the CE and FE can be configured directly with whatever they need, like certificates or passwords, depending on the authentication type the admin wants 🌸
For fail-over, the active CE and standby CE have to sync ForCES states and IPsec states so that state re-establishment stays minimal ⭐ There's also an alternative: the FE can set up a different IPsec SA with each CE during the startup operation itself. That keeps the periodic state transfer across the IPsec layer through the Fr (CE-CE) Interface to a minimum 👑
9. References 📚
9.1. Normative References
- [1] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
- [2] Baker, F., Ed., "Requirements for IP Version 4 Routers", RFC 1812, June 1995.
- [3] Floyd, S., "Congestion Control Principles", BCP 41, RFC 2914, September 2000.
- [4] Khosravi, H. and Anderson, T., Eds., "Requirements for Separation of IP Control and Forwarding", RFC 3654, November 2003.
9.2. Informative References
- [5] Case, J., Mundy, R., Partain, D. and B. Stewart, "Introduction and Applicability Statements for Internet Standard Management Framework", RFC 3410, December 2002.
- [6] Daniele, M., Wijnen, B., Ellison, M. and D. Francisco, "Agent Extensibility (AgentX) Protocol Version 1", RFC 2741, January 2000.
- [7] Chan, K., Seligson, J., Durham, D., Gai, S., McCloghrie, K., Herzog, S., Reichmeyer, F., Yavatkar, R. and A. Smith, "COPS Usage for Policy Provisioning (COPS-PR)", RFC 3084, March 2001.
- [8] Crouch, A. et al., "ForCES Applicability Statement", Work in Progress.
- [9] Anderson, T. and J. Buerkle, "Requirements for the Dynamic Partitioning of Switching Elements", RFC 3532, May 2003.
- [10] Leelanivas, M., Rekhter, Y. and R. Aggarwal, "Graceful Restart Mechanism for Label Distribution Protocol", RFC 3478, February 2003.
- [11] Moy, J., Pillay-Esnault, P. and A. Lindem, "Graceful OSPF Restart", RFC 3623, November 2003.
- [12] Sangli, S. et al., "Graceful Restart Mechanism for BGP", Work in Progress.
- [13] Shand, M. and L. Ginsberg, "Restart Signaling for IS-IS", Work in Progress.
- [14] Dierks, T. and C. Allen, "The TLS Protocol Version 1.0", RFC 2246, January 1999.
- [15] Kent, S. and R. Atkinson, "Security Architecture for the Internet Protocol", RFC 2401, November 1998.
- [16] Harkins, D. and D. Carrel, "The Internet Key Exchange (IKE)", RFC 2409, November 1998.
- [17] Hardjono, T. and Weis, B. "The Multicast Group Security Architecture", RFC 3740, March 2004.
- [18] Bellovin, S., Schiller, J. and C. Kaufman, Eds., "Security Mechanisms for the Internet", RFC 3631, December 2003.
10. Authors' Addresses 🫶
L. Lily Yang
Intel Corp., MS JF3-206,
2111 NE 25th Avenue
Hillsboro, OR 97124, USA
Phone: +1 503 264 8813
EMail: lily.l.yang@intel.com
Ram Dantu
Department of Computer Science,
University of North Texas,
Denton, TX 76203, USA
Phone: +1 940 565 2822
EMail: rdantu@unt.edu
Todd A. Anderson
Intel Corp.
2111 NE 25th Avenue
Hillsboro, OR 97124, USA
Phone: +1 503 712 1760
EMail: todd.a.anderson@intel.com
Ram Gopal
Nokia Research Center
5, Wayside Road,
Burlington, MA 01803, USA
Phone: +1 781 993 3685
EMail: ram.gopal@nokia.com11. Full Copyright Statement ⚖️
Copyright (C) The Internet Society (2004). This document is subject to the rights, licenses and restrictions in BCP 78. Except as set out there, the authors keep all their rights 💜
This document and the info in it are provided on an "AS IS" basis, and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE 🙈
Intellectual Property 🌈
The IETF doesn't take any position on whether the Intellectual Property Rights or other rights that might be claimed for implementing or using the tech in this document are valid or how far they reach, or whether any license under those rights might be available. It also doesn't claim it has made any independent effort to find such rights 🔍 For the procedures around rights in RFC documents, check BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat, any assurances of licenses to be made available, or the results of an attempt to get a general license or permission to use those proprietary rights for implementers or users of this spec can be found in the IETF's online IPR repository at http://www.ietf.org/ipr 📂
The IETF invites anyone interested to tell it about any copyrights, patents, patent applications, or other proprietary rights that might cover technology required to implement this standard. Send that info to the IETF at ietf-ipr@ietf.org 💌
Acknowledgement 🎀
Funding for the RFC Editor function is currently provided by the Internet Society 💖
📦 where this page comes from
- ietf: rfc3746 2004-04
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms