RFC 3821
Fibre Channel Over TCP/IP (FCIP)
RFC 3821: Fibre Channel Over TCP/IP (FCIP) 🦋
Okay bestie, let's start with the title block 🎀 This one is RFC 3821, called Fibre Channel Over TCP/IP (FCIP). It's from the Network Working Group and was written by M. Rajagopal, E. Rodriguez, and R. Weber. The category is Standards Track, and the date is July 2004. The header doesn't list any RFCs that it obsoletes or updates, so there's no drama there 💅
Status of This Memo
This document specifies an Internet standards track protocol for the Internet community, and it asks for discussion and suggestions for improvements 🌸 If you want to know where this protocol stands on the standardization ladder, check the current edition of the "Internet Official Protocol Standards" (STD 1). Distribution of this memo is unlimited, so share it with your whole group chat 💜
Copyright Notice
Copyright (C) The Internet Society (2004) ✨
Abstract
FCIP is the thing that lets little islands of Fibre Channel storage area networks hook up over IP-based networks, so they all turn into one unified storage area network in a single Fibre Channel fabric 🌐 It leans on IP-based network services to handle the connectivity between those SAN islands, whether that's over local area networks, metropolitan area networks, or wide area networks. Main character energy, honestly 👑
Table Of Contents
1. Purpose, Motivation, and Objectives. . . . . . . . . . . . . . 3
2. Relationship to Fibre Channel Standards. . . . . . . . . . . . 4
2.1. Relevant Fibre Channel Standards . . . . . . . . . . . . 4
2.2. This Specification and Fibre Channel Standards . . . . . 5
3. Terminology. . . . . . . . . . . . . . . . . . . . . . . . . . 5
4. Protocol Summary . . . . . . . . . . . . . . . . . . . . . . . 7
5. The FCIP Model . . . . . . . . . . . . . . . . . . . . . . . . 9
5.1. FCIP Protocol Model. . . . . . . . . . . . . . . . . . . 9
5.2. FCIP Link. . . . . . . . . . . . . . . . . . . . . . . . 10
5.3. FC Entity. . . . . . . . . . . . . . . . . . . . . . . . 11
5.4. FCIP Entity. . . . . . . . . . . . . . . . . . . . . . . 12
5.5. FCIP Link Endpoint (FCIP_LEP). . . . . . . . . . . . . . 13
5.6. FCIP Data Engine (FCIP_DE) . . . . . . . . . . . . . . . 14
5.6.1. FCIP Encapsulation of FC Frames. . . . . . . . . 16
5.6.2. FCIP Data Engine Error Detection and Recovery. . 19
6. Checking FC Frame Transit Times in the IP Network. . . . . . . 22
7. The FCIP Special Frame (FSF) . . . . . . . . . . . . . . . . . 23
7.1. FCIP Special Frame Format. . . . . . . . . . . . . . . . 23
7.2. Overview of FSF Usage in Connection Establishment. . . . 26
8. TCP Connection Management. . . . . . . . . . . . . . . . . . . 28
8.1. TCP Connection Establishment . . . . . . . . . . . . . . 28
8.1.1. Connection Establishment Model . . . . . . . . . 28
8.1.2. Creating New TCP Connections . . . . . . . . . . 29
8.1.3. Processing Incoming TCP Connect Requests . . . . 32
8.1.4. Simultaneous Connection Establishment. . . . . . 36
8.2. Closing TCP Connections. . . . . . . . . . . . . . . . . 36
8.3. TCP Connection Parameters. . . . . . . . . . . . . . . . 36
8.3.1. TCP Selective Acknowledgement Option . . . . . . 36
8.3.2. TCP Window Scale Option. . . . . . . . . . . . . 36
8.3.3. Protection Against Sequence Number Wrap. . . . . 37
8.3.4. TCP_NODELAY Option . . . . . . . . . . . . . . . 37
8.4. TCP Connection Considerations. . . . . . . . . . . . . . 37
8.5. Flow Control Mapping between TCP and FC. . . . . . . . . 37
9. Security . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
9.1. Threat Models. . . . . . . . . . . . . . . . . . . . . . 38
9.2. FC Fabric and IP Network Deployment Models . . . . . . . 40
9.3. FCIP Security Components . . . . . . . . . . . . . . . . 40
9.3.1. IPsec ESP Authentication and Confidentiality . . 40
9.3.2. Key Management . . . . . . . . . . . . . . . . . 41
9.3.3. ESP Replay Protection and Rekeying Issues. . . . 43
9.4. Secure FCIP Link Operation . . . . . . . . . . . . . . . 44
9.4.1. FCIP Link Initialization Steps . . . . . . . . . 44
9.4.2. TCP Connection Security Associations (SAs) . . . 44
9.4.3. Handling Data Integrity and Confidentiality
Violations . . . . . . . . . . . . . . . . . . . 45
10. Performance. . . . . . . . . . . . . . . . . . . . . . . . . . 45
10.1. Performance Considerations . . . . . . . . . . . . . . . 45
10.2. IP Quality of Service (QoS) Support. . . . . . . . . . . 46
11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 47
11.1. Normative References . . . . . . . . . . . . . . . . . . 47
11.2. Informative References . . . . . . . . . . . . . . . . . 49
12. Acknowledgments. . . . . . . . . . . . . . . . . . . . . . . . 50
Appendix A Fibre Channel Bit and Byte Numbering Guidance. . . . . 51
B IANA Considerations. . . . . . . . . . . . . . . . . . 51
C FCIP Usage of Addresses and Identifiers. . . . . . . . 52
D Example of synchronization Recovery Algorithm. . . . . 53
E Relationship between FCIP and IP over FC (IPFC). . . . 58
F FC Frame Format. . . . . . . . . . . . . . . . . . . . 59
G FC Encapsulation Format. . . . . . . . . . . . . . . . 61
H FCIP Requirements on an FC Entity. . . . . . . . . . . 63
Editors and Contributors Acknowledgements. . . . . . . . . . . . . 69
Editors and Contributors Addresses . . . . . . . . . . . . . . . . 70
Full Copyright Statement . . . . . . . . . . . . . . . . . . . . . 741. Purpose, Motivation, and Objectives ✨
Warning to Readers Familiar With Fibre Channel: Both Fibre Channel and IETF standards use the same byte transmission order. However, the bit and byte numbering is different. See appendix A for guidance. 🙈
Fibre Channel (FC) is a gigabit or multi-gigabit speed networking technology, and it's mainly used to build Storage Area Networks (SANs) 💖 Section 2 explains how Fibre Channel is standardized and how this specification fits in with those Fibre Channel standards. If you want an overview of Fibre Channel, check out [34].
This specification describes mechanisms that let islands of Fibre Channel SANs get interconnected over IP Networks, so they form one unified SAN in a single Fibre Channel fabric 🦋 The whole point is connecting physically remote FC sites, so you can do remote disk access, tape backup, and live mirroring. Iconic use case, no cap 🔥
Fibre Channel standards chose nominal distances between switch elements that are shorter than what an IP Network can cover. Since Fibre Channel and IP networking technologies are compatible, it just makes sense to use IP networking to stretch the allowable distances between Fibre Channel switch elements 🌈
The big assumption here is that the Fibre Channel traffic travels over the IP Network in a way where the Fibre Channel Fabric and every Fibre Channel device on the Fabric have no idea the IP Network is there 🫶 That means the FC datagrams have to be delivered in time to comply with existing Fibre Channel specifications. FC traffic can span LANs, MANs, and WANs, as long as this assumption holds.
The objectives of this document are to:
- specify the encapsulation and mapping of Fibre Channel (FC) frames employing FC Frame Encapsulation [19].
- apply the mechanism described in 1) to an FC Fabric using an IP network as an interconnect between two or more islands in an FC Fabric.
- address any FC concerns arising from tunneling FC traffic over an IP-based network, including security, data integrity (loss), congestion, and performance. This will be accomplished by utilizing the existing IETF-specified suite of protocols.
- be compatible with the referenced FC standards. While new work may be undertaken in T11 to optimize and enhance FC Fabrics, this specification REQUIRES conformance only to the referenced FC standards.
- be compatible with all applicable IETF standards so that the IP Network used to extend an FC Fabric can be used concurrently for other reasonable purposes.
The objectives of this document do not include using an IP Network as a replacement for the Fibre Channel Arbitrated Loop interconnect. No definition is provided for encapsulating loop primitive signals for transmission over an IP Network 🍓
Conventions used in this document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14, RFC 2119 [1] 💜
2. Relationship to Fibre Channel Standards 🌸
2.1. Relevant Fibre Channel Standards
FC is standardized as a family of American National Standards developed by the T11 technical committee of INCITS (InterNational Committee for Information Technology Standards) 🎀 T11 has specified a bunch of documents describing FC protocols, operations, and services. The T11 documents of interest to readers of this specification include (but are not limited to):
- FC-BB - Fibre Channel Backbone [2]
- FC-BB-2 - Fibre Channel Backbone -2 [3]
- FC-SW-2 - Fibre Channel Switch Fabric -2 [4]
- FC-FS - Fibre Channel Framing and Signaling [5]
FC-BB and FC-BB-2 describe the relationship between an FC Fabric and interconnect technologies that aren't defined by Fibre Channel standards (e.g., ATM and SONET). FC-BB-2 is the Fibre Channel document describing the relationships between FC and TCP/IP, including the FC use of FCIP 👑
FC-SW-2 describes the switch components of an FC Fabric, and FC-FS describes the FC Frame format and basic control features of Fibre Channel 🧚
More info about T11 activities is available on the committee's web site www.t11.org 🌈
2.2. This Specification and Fibre Channel Standards
When you're figuring out how to transport FC Frames over an IP Network, it makes sense to split the standardization work between TCP/IP requirements and Fibre Channel requirements 💅 This specification covers the TCP/IP requirements for transporting FC Frames, and the Fibre Channel documents from section 2.1 cover the Fibre Channel requirements.
This specification only addresses the requirements needed to properly use an IP Network as a conduit for FC Frames. The result is a specification for an FCIP Entity (see section 5.4) ✨
A product that tunnels an FC Fabric through an IP Network MUST combine the FCIP Entity with an FC Entity (see section 5.3) using an implementation specific interface 🫶 The requirements this specification places on an FC Entity to get proper delivery of FC Frames are summarized in appendix H. For more on FC Entities, look at the Fibre Channel standards, and for an example of an FC Entity, see FC-BB-2 [3] 🦋
No attempt is made to define a specific API between an FCIP Entity and an FC Entity. The approach is to specify the required functional interactions between an FCIP Entity and an FC Entity (both of which are required to forward FC frames across an IP Network), but let implementers choose how those interactions get realized 💖
🌷 Part 1 of 14 down! Part 2 picks up with Section 3 (Terminology) 🌷
3. Terminology ✨
okay so this section is basically the FCIP glossary, bestie 📖 every term we need to know lives right here, so let's get into it 💅
FC End Node 💻 - An FC device that uses the connection services provided by the FC Fabric.
FC Entity 🧩 - The Fibre Channel specific functional component that combines with an FCIP Entity to form an interface between an FC Fabric and an IP Network (see section 5.3).
FC Fabric 🕸️ - An entity that interconnects various Nx_Ports (see [5]) attached to it, and it can route FC Frames using only the destination ID information in an FC Frame header (see appendix F). iconic routing skills ngl 👑
FC Fabric Entity 🏗️ - A Fibre Channel specific element containing one or more Interconnect_Ports (see FC-SW-2 [4]) and one or more FC/FCIP Entity pairs. See FC-BB-2 [3] for details about FC Fabric Entities.
FC Frame 📦 - The basic unit of Fibre Channel data transfer (see appendix F). the main character of the whole thing, honestly 🎀
FC Frame Receiver Portal 🚪 - The access point through which an FC Frame and time stamp enter an FCIP Data Engine from the FC Entity.
FC Frame Transmitter Portal 🚪 - The access point through which a reconstituted FC Frame and time stamp leave an FCIP Data Engine to the FC Entity.
FC/FCIP Entity pair 👯 - The combination of one FC Entity and one FCIP Entity.
FCIP Data Engine (FCIP_DE) ⚙️ - The component of an FCIP Entity that handles FC Frame encapsulation, de-encapsulation, and transmission of FCIP Frames through a single TCP Connection (see section 5.6). this one does the heavy lifting 💪
FCIP Entity 🌐 - The entity responsible for the FCIP protocol exchanges on the IP Network. It encompasses FCIP_LEP(s) and the FCIP Control and Services module (see section 5.4).
FCIP Frame 🎁 - An FC Frame plus the FC Frame Encapsulation [19] header, encoded SOF and encoded EOF that contains the FC Frame (see section 5.6.1). it's the FC Frame all dressed up for the trip 💖
FCIP Link 🔗 - One or more TCP Connections that connect one FCIP_LEP to another (see section 5.2).
FCIP Link Endpoint (FCIP_LEP) 📍 - The component of an FCIP Entity that handles a single FCIP Link and contains one or more FCIP_DEs (see section 5.5).
Encapsulated Frame Receiver Portal 📥 - The TCP access point through which an FCIP Frame is received from the IP Network by an FCIP Data Engine.
Encapsulated Frame Transmitter Portal 📤 - The TCP access point through which an FCIP Frame is transmitted to the IP Network by an FCIP Data Engine.
FCIP Special Frame (FSF) ✨ - A specially formatted FC Frame containing information used by the FCIP protocol (see section 7). the VIP of frames, no debate 👑
4. Protocol Summary 🌸
okay this is the highlight reel, so here's how FCIP works in 14 bullet-point vibes 🎬
- The main job of an FCIP Entity is forwarding FC Frames, and it does that using the FC Frame Encapsulation described in [19]. 🎁
- From the IP Network point of view, FCIP Entities are peers, and they talk to each other using TCP/IP. Each FCIP Entity has one or more TCP endpoints in the IP-based network. 🌐
- From the FC Fabric point of view, pairs of FCIP Entities, together with their associated FC Entities, forward FC Frames between FC Fabric elements. The FC End Nodes have no idea the FCIP Link exists. 🤫
- FC Primitive Signals, Primitive Sequences, and Class 1 FC Frames are not transmitted across an FCIP Link, because they cannot be encoded using FC Frame Encapsulation [19]. not them being left out 💀
- The path (route) that an encapsulated FC Frame takes follows the normal routing procedures of the IP Network. 🗺️
- An FCIP Entity MAY contain multiple FCIP Link Endpoints, but each FCIP Link Endpoint (
FCIP_LEP) communicates with exactly one otherFCIP_LEP. one-on-one energy, it's giving loyalty 🫶 - When multiple
FCIP_LEPs with multipleFCIP_DEs are in use, the selection of whichFCIP_DEto use for encapsulating and transmitting a given FC Frame is covered in FC-BB-2 [3]. FCIP Entities do not actively participate in FC Frame routing. they're not the router, they just deliver 📬 - The FCIP Control and Services module MAY use TCP/IP quality of service features (see section 10.2). ✨
- It is necessary to statically or dynamically configure each FCIP entity with the IP addresses and TCP port numbers of the FCIP Entities it is expected to start communicating with. If dynamic discovery of participating FCIP Entities is supported, the function SHALL be performed using the Service Location Protocol (SLPv2) [17]. It is outside the scope of this specification to describe any static configuration method for participating FCIP Entity discovery. Refer to section 8.1.2.2 for a detailed description of dynamic discovery of participating FCIP Entities using SLPv2. 🔍
- Before creating a TCP Connection to a peer FCIP Entity, the FCIP Entity trying to create that connection SHALL statically or dynamically determine the IP address, TCP port, expected FC Fabric Entity World Wide Name, TCP Connection Parameters, and Quality of Service Information. checklist time 📝
- FCIP Entities do not actively participate in discovering FC source and destination identifiers. Discovery of FC addresses (accessible via the FCIP Entity) is handled by techniques and protocols within the FC architecture, as described in FC-FS [5] and FC-SW-2 [4]. not their job, and they know it 💅
- To support IP Network security (see section 9), FCIP Entities MUST:
- implement cryptographically protected authentication and cryptographic data integrity keyed to the authentication process, and
- implement data confidentiality security features. 🔐
- On an individual TCP Connection, this specification relies on TCP/IP to deliver a byte stream in the same order it was sent. 🧵
- This specification assumes the presence of, and requires the use of, TCP and FC data loss and corruption mechanisms. The error detection and recovery features described in this specification complement and support those existing mechanisms. teamwork makes the dream work 🫶
5. The FCIP Model 🎀
5.1. FCIP Protocol Model 🥞
okay so here's how FCIP fits with everything else, shown in a stack picture 🖼️
Figure 1 is the stack, and it's giving layers on layers 🥞
+------------------------+ FCIP Link +------------------------+
| FCIP |===========| FCIP |
+--------+------+--------+ +--------+------+--------+
| FC-2 | | TCP | | TCP | | FC-2 |
+--------+ +--------+ +--------+ +--------+
| FC-1 | | IP | | IP | | FC-1 |
+--------+ +--------+ +--------+ +--------+
| FC-0 | | LINK | | LINK | | FC-0 |
+--------+ +--------+ +--------+ +--------+
| | PHY | | PHY | |
| +--------+ +--------+ |
| | | |
| | IP Network | |
V +--------------------+ V
to Fibre to Fibre
Channel Channel
Fabric Fabric
Key: FC-0 - Fibre Channel Physical Media Layer
FC-1 - Fibre Channel Encode and Decode Layer
FC-2 - Fibre Channel Framing and Flow Control Layer
TCP - Transmission Control Protocol
IP - Internet Protocol
LINK - IP Link Layer
PHY - IP Physical Layer
Figure 1: FCIP Protocol Stack ModelNote that the objective of the FCIP Protocol is to create and maintain one or more FCIP Links to transport data. 📌
5.2. FCIP Link 🔗
The FCIP Link is the basic unit of service that the FCIP Protocol provides to an FC Fabric. As shown in figure 2, an FCIP Link connects two portions of an FC Fabric, using an IP Network as the transport, to form a single FC Fabric. two fabrics, one vibe ✨
/\/\/\/\/\/\ /\/\/\/\/\/\ /\/\/\/\/\/\
\ FC / \ IP / \ FC /
/ Fabric \=========/ Network \=========/ Fabric \
\/\/\/\/\/\/ \/\/\/\/\/\/ \/\/\/\/\/\/
| |
|<--------- FCIP Link -------->|
Figure: 2 FCIP Link ModelAt the points where the ends of the FCIP Link meet portions of the FC Fabric, an FCIP Entity (see section 5.4) combines with an FC Entity, as described in section 5.3, to serve as the interface between FC and IP. 🤝
An FCIP Link SHALL contain at least one TCP Connection and MAY contain more than one TCP Connection. The endpoints of a single TCP Connection are FCIP Data Engines (see section 5.6). The endpoints of a single FCIP Link are FCIP Link Endpoints (see section 5.5). 💖
5.3. FC Entity 🧩
An implementation that tunnels an FC Fabric through an IP Network MUST combine an FC Entity with an FCIP Entity (see section 5.4) to form a complete interface between the FC Fabric and the IP Network, as shown in figure 3. An FC Fabric Entity may contain multiple instances of the FC/FCIP Entity pair shown on either the right-hand or left-hand side of figure 3. 🪞
|<--------- FCIP Link -------->|
| |
+----------+ /\/\/\/\/\/\ +----------+
| FCIP | \ IP / | FCIP |
| Entity |=========/ Network \=========| Entity |
+----------+ \/\/\/\/\/\/ +----------+
| FC | | FC |
| Entity | | Entity |
+----------+ +----------+
| |
/\/\/\/\/\/\ /\/\/\/\/\/\
\ FC / \ FC /
/ Fabric \ / Fabric \
\/\/\/\/\/\/ \/\/\/\/\/\/
Figure 3: Model for Two Connected FC/FCIP Entity PairsIn general, the combination of an FCIP Link and two FC/FCIP Entity pairs is intended to provide a non-Fibre Channel backbone transport between Fibre Channel components. For example, this combination can be used to function as the hard-wire connection between two Fibre Channel switches. the backbone glow-up 🦋
The interface between the FC and FCIP Entities is implementation specific. The functional requirements placed on an FC Entity by this specification are listed in appendix H. More information about FC Entities can be found in the Fibre Channel standards, and an example of an FC Entity can be found in FC-BB-2 [3]. 📚
5.4. FCIP Entity 🏰
Okay so the FCIP Entity model is shown in figure 4, and it's giving whole architecture vibes 💅
.......................................................
: FCIP Entity :
: :
: +-----------+ :
: | FCIP | :
: |Control and|------------------------------------+ :
: | Services | | :
: | Module | | :
: +-----------+ | :
: | +--------------------+ | :
: | +-------+--------------------+|----+ | :
: | |+-----+--------------------+|----+| | :
: | ||+----| FCIP Link Endpoint |----+|| | :
: | ||| +--------------------+ ||| | :
:.............................................|||.....:
| ||| ||| |
| ||| ||| o<--+
| ||| unique TCP ||| | |
| ||| connections-->||| | |
| ||| ||| | |
+----------+ /\/\/\/\/\/\ |
| FC | \ IP / |
| Entity | / Network \ |
+----------+ \/\/\/\/\/\/ |
| |
/\/\/\/\/\/\ +------------------+
\ FC / +->TCP port for
/ Fabric \ incoming
\/\/\/\/\/\/ connectionsFigure 4: FCIP Entity Model
So the FCIP Entity is the one that receives TCP connect requests on behalf of the FCIP_LEPs it manages 🫶 To make that work, it's the only owner of at least one TCP port/IP Address combo used to form TCP Connections. The TCP port can be the FCIP well known port at a given IP Address, which is so iconic 🌸 Also, an FC Fabric to IP Network interface product SHALL give each FC/FCIP Entity pair inside it a unique combo of FC Fabric Entity World Wide Identifier and FC/FCIP Entity Identifier values (see section 7). Unique IDs for every pair, no drama 🆔
An FCIP Entity has an FCIP Control and Services Module that handles FCIP link initialization and FCIP link dissolution. It also gives the FC Entity an interface to key IP Network features 🎀
The interfaces to the IP Network features are implementation specific. However, the following REQUIRED TCP/IP functional support is specified in this document:
- TCP Connections - see section 8
- Security - see section 9
- Performance - see section 10
- Dynamic Discovery - see section 8.1.2.2
Last thing: the FCIP Link Endpoints inside an FCIP Entity are the ones doing the FC Frame encapsulation and transmission features of FCIP 🦋
5.5. FCIP Link Endpoint (FCIP_LEP) 🔗
As shown in figure 5, an FCIP Link Endpoint contains one FCIP Data Engine for each TCP Connection in the FCIP Link. Main character energy for sure ✨
................................................
: FCIP Link Endpoint :
: +------------------+ :
: +-------+------------------+|----+ :
: |+-----+------------------+|----+| :
: ||+----| FCIP Data Engine |----+|| :
: ||| +------------------+ ||| :
:..............................................:
||| |||
+----------+ /\/\/\/\/\/\
| FC | \ IP /
| Entity | / Network \
+----------+ \/\/\/\/\/\/
|
/\/\/\/\/\/\
\ FC /
/ Fabric \
\/\/\/\/\/\/Figure 5: FCIP Link Endpoint Model
Every time a TCP Connection gets formed with a new FC/FCIP Entity pair (including all the actions described in section 8.1), the FCIP Entity SHALL create a new FCIP Link Endpoint containing one FCIP Data Engine. Fresh pair, fresh endpoint 🌈
An FCIP_LEP is a transparent data translation point between an FC Entity and an IP Network. When a pair of FCIP_LEPs talk over one or more TCP Connections, they create an FCIP Link that joins two islands of an FC Fabric and makes them one single FC Fabric 🧚 Two islands becoming one, iconic behaviour honestly 😭
The IP Network carrying those two FCIP_LEPs has no idea what FC payloads it's moving around 🙈 And the FC End Nodes hooked up to the FC Fabric are unaware of the TCP/IP based transport used in the structure of the FC Fabric. Nobody's spilling the tea to anybody, lol 🤫
An FCIP_LEP uses normal TCP based flow control mechanisms to manage its internal resources and match them with the advertised TCP Receiver Window Size (see sections 8.3.2, 8.5). An FCIP_LEP MAY communicate with its local FC Entity counterpart to coordinate flow control 💜
5.6. FCIP Data Engine (FCIP_DE) ⚙️
The model for one of the multiple FCIP_DEs that MAY be present in an FCIP_LEP is shown in figure 6. Okay, this diagram is a lot but we're going to get through it together 🫶
+--------------------------------+
| |
F |-+ +------------------+ +-|
C |p| | Encapsulation | |p| N
-->|1|--->| Engine |--->|2|--> e
E |-+ +------------------+ +-| t
n | | I w
t |-+ +------------------+ +-| P o
i |p| | De-Encapsulation | |p| r
t <--|4|<---| Engine |<---|3|<-- k
y |-+ +------------------+ +-|
| |
+--------------------------------+Figure 6: FCIP Data Engine Model
Data enters and leaves the FCIP_DE through four portals (p1 - p4). The portals don't process or examine the data that passes through them. They're just the named access points where the FCIP_DE meets the external world 🚪 Here are the names:
- p1) FC Frame Receiver Portal - The interface where an FC Frame and time stamp come into an FCIP_DE from the FC Entity.
- p2) Encapsulated Frame Transmitter Portal - The TCP interface where an FCIP Frame gets sent out to the IP Network by an FCIP_DE.
- p3) Encapsulated Frame Receiver Portal - The TCP interface where an FCIP Frame gets received from the IP Network by an FCIP_DE.
- p4) FC Frame Transmitter Portal - The interface where a reconstituted FC Frame and time stamp leave an FCIP_DE for the FC Entity.
The FCIP_DE's work is done by the Encapsulation and De-Encapsulation Engines, and they have two jobs 🔥
- Encapsulating and de-encapsulating FC Frames using the encapsulation format described in FC Frame Encapsulation [19] and in section 5.6.1 of this document.
- Detecting some data transmission errors and doing minimal error recovery as described in section 5.6.2.
Data flows through a pair of IP Network connected FCIP_DEs in seven steps, so let's go bestie 💖
- An FC Frame and time stamp arrives at the FC Frame Receiver Portal and gets passed to the Encapsulation Engine. The FC Frame is assumed to have been processed by the FC Entity according to the applicable FC rules, and the FCIP_DE doesn't validate it. If the FC Entity is in the Unsynchronized state with respect to a time base, as described in the FC Frame Encapsulation [19] specification, the time stamp delivered with the FC Frame SHALL be zero ⏱️
- In the Encapsulation Engine, the encapsulation format described in FC Frame Encapsulation [19] and in section 5.6.1 of this document SHALL be applied to prepare the FC Frame and its time stamp for transmission over the IP Network 📦
- The entire encapsulated FC Frame (a.k.a. the FCIP Frame) SHALL be passed to the Encapsulated Frame Transmitter Portal, where it SHALL be inserted in the TCP byte stream 🚀
- Sending the FCIP Frame over the IP Network follows all the TCP rules of operation. That includes, but isn't limited to, the in-order delivery of bytes in the stream, as specified by TCP [6] 🧵
- The FCIP Frame arrives at the partner FCIP Entity, where it enters the FCIP_DE through the Encapsulated Frame Receiver Portal and gets passed to the De-Encapsulation Engine for processing 📬
- The De-Encapsulation Engine SHALL validate the incoming TCP byte stream as described in section 5.6.2.2, and it SHALL de-encapsulate the FC Frame and associated time stamp according to the encapsulation format described in FC Frame Encapsulation [19] and in section 5.6.1 of this document 🔍
- In the absence of errors, the de-encapsulated FC Frame and time stamp SHALL be passed to the FC Frame Transmitter Portal for delivery to the FC Entity. Error handling is discussed in section 5.6.2.2 🎉
To wrap this up, every FC Frame that arrives at the FC Frame Receiver Portal SHALL be transmitted on the IP Network as described in steps 1 through 4 above. In the absence of errors, data bytes arriving at the Encapsulated Frame Receiver Portal SHALL be de-encapsulated and forwarded to the FC Frame Transmitter Portal as described in steps 5 through 7. And that's the whole data path, so iconic 💅✨
5.6.1. FCIP Encapsulation of FC Frames ✨
Okay bestie, FCIP wraps FC Frames using FC Frame Encapsulation [19] 🎀. Any features of FC Frame Encapsulation that are specific to a protocol SHALL be applied like this for FCIP 💖.
The Protocol# field SHALL contain 1, as listed in the IANA Considerations annex of FC Frame Encapsulation [19] 🦋.
The Protocol Specific field SHALL have the format in figure 7 💅. Quick heads up: the word numbers in figure 7 count from the start of the full FC Frame Encapsulation header, not from the Protocol Specific field 🙈.
W|------------------------------Bit------------------------------|
o| |
r| 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3|
d|0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1|
+---------------------------------------------------------------+
1| replication of encapsulation word 0 |
+---------------+---------------+---------------+---------------+
2| pFlags | Reserved | -pFlags | -Reserved |
+---------------+---------------+---------------+---------------+Figure 7: FCIP Usage of FC Frame Encapsulation Protocol Specific field
Word 1 of the Protocol Specific field SHALL hold an exact copy of word 0 from FC Frame Encapsulation [19] 🫶.
The pFlags (protocol specific flags) field tells you how the FC Encapsulation Header is being used for the protocol, and figure 8 shows the defined pFlags bits 🌸.
|----------------Bit--------------------|
| |
| 0 1 2 3 4 5 6 7 |
+----+-----------------------------+----+
| Ch | Reserved | SF |
+----+-----------------------------+----+Figure 8: pFlags Field Bits
The SF (Special Frame) bit says whether the FCIP Frame is an encapsulated FC Frame or an FSF (FCIP Special Frame, see section 7) 👑. When the FCIP Frame has an encapsulated FC Frame in it, the SF bit SHALL be 0. When the FCIP Frame is an FSF, the SF bit SHALL be 1.
An FSF SHALL only be sent as the very first bytes going each direction on a newly formed TCP Connection, and only one FSF SHALL go each direction at that time (see section 8.1) 😭. After that, every FCIP Frame SHALL have the SF bit set to 0 🌈.
The Ch (Changed) bit says whether an echoed FSF has been intentionally altered (see section 8.1.3) 🧚. The Ch bit SHALL be 0 unless the FSF bit is 1. When the initial TCP Connection FSF is sent, the Ch bit SHALL be 0. If the recipient of a TCP connect request echoes the FSF without any changes, then the Ch bit SHALL continue to be 0. If the recipient of a TCP connect request alters the FSF before echoing it, then the Ch bit SHALL be changed to 1.
The -pFlags field SHALL contain the ones complement of the contents of the pFlags field 🍓.
Table 1 sums up how the pFlags SF and Ch bits get used 💜.
+----+----+------------+--------------------------------------+
| | | Originated | |
| SF | Ch | or Echoed | Validity/Description |
+----+----+------------+--------------------------------------+
| 0 | 0 | n/a | Encapsulated FC Frame |
+----+----+------------+--------------------------------------+
| 0 | 1 | n/a | Always Illegal |
+----+----+------------+--------------------------------------+
| 1 | 0 | Originated | Originated FSF |
+----+----+------------+--------------------------------------+
| 1 | 1 | Originated | Always Illegal |
+----+----+------------+--------------------------------------+
| 1 | 0 | Echoed | Echoed FSF without changes |
+----+----+------------+--------------------------------------+
| 1 | 1 | Echoed | Echoed FSF with changes |
+----+----+------------+--------------------------------------+
| Note 1: Echoed FSFs may contain changes resulting from |
| transmission errors, necessitating the comparison between |
| sent and received FSF bytes by the FSF originator described |
| in section 8.1.2.3. |
| |
| Note 2: Column positions in this table do not reflect the |
| bit positions of the SF and Ch bits in the pFlags field. |
+-------------------------------------------------------------+Table 1: pFlags SF and Ch bit usage summary
The Reserved pFlags bits SHALL be 0 🫧.
The Reserved field (bits 23-16 in word 2): SHALL contain 0.
The -Reserved field (bits 7-0 in word 2): SHALL contain 255 (or 0xFF) 💅.
The CRCV (CRC Valid) Flag SHALL be set to 0 ✨.
The CRC field SHALL be set to 0 🎀.
In FCIP, the SOF and EOF codes listed as Class 2, Class 3, and Class 4 in the FC Frame Encapsulation [19] are legal 🦋.
5.6.2. FCIP Data Engine Error Detection and Recovery 🌸
5.6.2.1. TCP Assistance With Error Detection and Recovery
TCP [6] requires in order delivery, generation of TCP checksums, and checking of TCP checksums 💖. So the byte stream passed from TCP to the FCIP_LEP will be in order and free of errors detectable by the TCP checksum ✨. The FCIP_LEP just relies on TCP to handle all of that 🫶.
5.6.2.2. Errors in FCIP Headers and Discarding FCIP Frames
Bytes delivered through the Encapsulated Frame Receiver Portal that aren't correctly delimited, as defined by the FC Frame Encapsulation [19], count as errors 🙈.
If the Protocol# and Version fields in the FCIP Frame header don't have the values defined for an FCIP Frame, that SHALL be considered an error 😭.
Some errors in the encapsulation can also make the FCIP_DE lose synchronization with the FC Frames in the byte stream coming through the Encapsulated Frame Receiver Portal 💀.
The Frame Length field in the FC Frame Encapsulation header is how the FCIP_DE figures out where the next FC Encapsulated Header lives in the data stream 🧚. The following tests SHALL be performed to verify synchronization with the byte stream entering the Encapsulated Frame Receiver Portal, and synchronization SHALL be considered lost if any of the tests fail 👑:
- Frame Length field validation --
15 < Frame Length < 545; - Comparison of Frame Length field to its ones complement; and
- A valid EOF is found in the word right before the start of the next FCIP header, as indicated by the Frame Length field, tested as follows:
- Bits 24-31 and 16-23 contain identical legal EOF values (the list of legal EOF values is in the FC Frame Encapsulation [19]); and
- Bits 8-15 and 0-7 contain the ones complement of the EOF value found in bits 24-31.
Note: here's where the valid Frame Length range comes from 🍓. The FCIP Frame header is seven words, one word each is needed for the encoded SOF and EOF values, the FC Frame header is six words, and the FC CRC needs one word. That gives a base Frame Length of 16 (7+1+1+6+1) words if there's no FC Payload. Since the FC Payload is optional, any Frame Length value greater than 15 is valid. The maximum FC Payload size is 528 words, so any Frame Length value up to and including 544 (528+16) is valid 💜.
If synchronization is lost, the FC Frame SHALL NOT be forwarded on to the FC Entity, and further recovery SHALL be handled as defined by section 5.6.2.3 😭.
Besides the tests above, the validity and positioning of the following FCIP Frame information SHOULD be used to detect encapsulation errors that may or may not affect synchronization 🌈:
- a) Protocol# ones complement field (1 test);
- b) Version ones complement field (1 test);
- c) Replication of encapsulation word 0 in word 1 (1 test);
- d) Reserved field and its ones complement (2 tests);
- e) Flags field and its ones complement (2 tests);
- f) CRC field is equal to zero (1 test);
- g) SOF fields and ones complement fields (4 tests);
- h) Format and values of FC header (1 test);
- i) CRC of FC Frame (2 tests);
- j) FC Frame Encapsulation header information in the next FCIP Frame (1 test).
At least 3 of the 16 tests listed above SHALL be performed 🦋. If any test that actually gets performed fails, that SHALL indicate an encapsulation error, and the FC Frame SHALL NOT be forwarded on to the FC Entity 🔥. Also, these errors SHOULD be considered carefully, since some of them may be synchronization errors 🙈.
Whenever an FCIP_DE discards bytes delivered through the Encapsulated Frame Receiver Portal, it SHALL cause the FCIP Entity to notify the FC Entity of the condition, and give a suitable description of why the bytes were discarded 💅.
The burden of recovering from discarded data falls on the FC Entity and other parts of the FC Fabric, and it's outside the scope of this specification 💀.
5.6.2.3. Synchronization Failures
If an FCIP_DE figures out it can't find the next FCIP Frame header in the byte stream coming through the Encapsulated Frame Receiver Portal, the FCIP_DE SHALL do one of the following 😭:
- a) close the TCP Connection [6] [7] and notify the FC Entity with the reason for the closure;
- b) recover synchronization by searching the bytes delivered by the Encapsulated Frame Receiver Portal for a valid FCIP Frame header with the correct properties (see section 5.6.2.2), and discarding bytes delivered by the Encapsulated Frame Receiver Portal until a valid FCIP Frame header is found; or
- c) try to recover synchronization as described in b), and if synchronization can't be recovered, close the TCP Connection as described in a), including notifying the FC Entity with the reason for the closure.
If the FCIP_DE tries to recover synchronization, the resynchronization algorithm used SHALL meet the following requirements 🧚:
- a) discard, or identify with an EOFa (see appendix section F.1), those FC Frames and fragments of FC Frames identified before synchronization has been completely verified again. The number of FC Frames not forwarded may vary based on the algorithm used;
- b) return to forwarding FC Frames through the FC Frame Transmitter Portal only after synchronization on the transmitted FCIP Frame stream has been verified; and
- c) close the TCP/IP connection if the algorithm ends without verifying successful synchronization. The probability of failing to synchronize successfully, and the time needed to figure out whether synchronization worked, may vary with the algorithm used.
An example algorithm that meets these requirements can be found in appendix D 👑.
The burden of recovering from discarding FCIP Frames during the optional resynchronization process described in this section falls on the FC Entity and other parts of the FC Fabric, and is outside the scope of this specification 🫶.
6. Checking FC Frame Transit Times in the IP Network ✨
okay bestie, so FC-BB-2 [3] is the spec that explains how to measure IP Network transit time 🧚, and it builds on the requirements from the FC Frame Encapsulation [19] spec. The part I'm obsessed with is why this job goes to the FC Entity. They want the time spent going through the FCIP Entities to count too 💅 that's so thorough it's giving main character energy 🔥
Each FC Frame that enters the FCIP_DE through the FC Frame Receiver Portal SHALL be accompanied by a time stamp value that the FCIP_DE SHALL place in the Time Stamp [integer] and Time Stamp [fraction] fields of the encapsulation header of the FCIP Frame that contains the FC Frame 🎀 If there's no synchronized time stamp value to go with the incoming FC Frame, a value of zero SHALL be used 🙈
Each FC Frame that exits the FCIP_DE through the FC Frame Transmitter Portal SHALL be accompanied by the time stamp value taken from the FCIP Frame that encapsulated the FC Frame 🦋
The FC Entity SHALL use suitable internal clocks and either Fibre Channel services or an SNTP Version 4 server [26] to set up and keep the required synchronized time value ⏰ Then the FC Entity SHALL verify that the FC Entity it's talking to on an FCIP Link is using the same synchronized time source, either Fibre Channel services or the SNTP server 🫶
Note that since the FC Fabric is expected to have a single synchronized time value throughout, relying on the Fibre Channel services means only one synchronized time value is needed for all FCIP_DEs, no matter their connection characteristics 🌈 iykyk, one clock to rule them all 👑
7. The FCIP Special Frame (FSF)
7.1. FCIP Special Frame Format
Figure 9 shows the FSF format 💖
W|------------------------------Bit------------------------------|
o| |
r| 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3|
d|0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1|
+---------------+---------------+---------------+---------------+
0| Protocol# | Version | -Protocol# | -Version |
| (0x01) | (0x01) | (0xFE) | (0xFE) |
+---------------+---------------+---------------+---------------+
1| Protocol# | Version | -Protocol# | -Version |
| (0x01) | (0x01) | (0xFE) | (0xFE) |
+---------------+---------------+---------------+---------------+
2| pFlags | Reserved | -pFlags | -Reserved |
| | (0x00) | | (0xFF) |
+-----------+---+---------------+-----------+---+---------------+
3| Flags | Frame Length | -Flags | -Frame Length |
| (0b000000)| (0b0000010011) | (0b111111)| (0b1111101100) |
+-----------+-------------------+-----------+-------------------+
4| Time Stamp [integer] |
+---------------------------------------------------------------+
5| Time Stamp [fraction] |
+---------------------------------------------------------------+
6| CRC (Reserved in FCIP) |
| (0x00-00-00-00) |
+-------------------------------+-------------------------------+
7| Reserved | -Reserved |
| (0x00-00) | (0xFF-FF) |
+-------------------------------+-------------------------------+
8| |
+----- Source FC Fabric Entity World Wide Name -----+
9| |
+---------------------------------------------------------------+
10| |
+----- Source FC/FCIP Entity Identifier -----+
11| |
+---------------------------------------------------------------+
12| |
+----- Connection Nonce -----+
13| |
+---------------+---------------+-------------------------------+
(Continued)Figure 9: FSF Format (part 1 of 2)
W|------------------------------Bit------------------------------|
o| |
r| 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3|
d|0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1|
| |
| (Concluded) |
+---------------------------------------------------------------+
14| Connection | Reserved | Connection Usage Code |
| Usage Flags | (0x00) | <defined in FC-BB-2> |
+---------------+---------------+-------------------------------+
15| |
+----- Destination FC Fabric Entity World Wide Name -----+
16| |
+---------------------------------------------------------------+
17| K_A_TOV |
+-------------------------------+-------------------------------+
18| Reserved | -Reserved |
| (0x00-00) | (0xFF-FF) |
+-------------------------------+-------------------------------+Figure 9: FSF Format (part 2 of 2)
The FSF SHALL only be sent as the first bytes transmitted in each direction on a newly formed TCP Connection, and only one FSF SHALL be transmitted in each direction 🚦
The contents of the FSF SHALL be as described for encapsulated FC Frames, except for the fields described in this section 📦
All FSFs SHALL have the pFlags SF bit set to 1 (see section 5.6.1) 🌸
The Source FC Fabric Entity World Wide Name field SHALL contain the Fibre Channel Name_Identifier [5] for the FC Fabric entity associated with the FC/FCIP Entity pair that generates (as opposed to echoes) the FSF 🍓 For example, if the FC Fabric entity is a FC Switch, the FC Fabric Entity World Wide Name field SHALL contain the Switch_Name [4]. The Source FC Fabric Entity World Wide Name SHALL be world wide unique 🌍
The Source FC/FCIP Entity Identifier field SHALL contain a unique identifier for the FC/FCIP Entity pair that generates (as opposed to echoes) the FSF 🔑 The value is assigned by the FC Fabric entity whose world wide name appears in the Source FC Fabric Entity World Wide Name field.
Note: The combination of the Source FC Entity World Wide Name and Source FC/FCIP Entity Identifier fields uniquely identifies every FC/FCIP Entity pair in the IP Network 💜
The Connection Nonce field shall contain a 64-bit random number generated to uniquely identify a single TCP connect request. In order to provide sufficient security for the connection nonce, the Randomness Recommendations for Security [9] SHOULD be followed 🔐 security is serious bestie 😭
The Connection Usage Flags field identifies the types of SOF values [19] to be carried on the connection, as shown in figure 10 🎀
|------------------------------Bit------------------------------|
| |
| 0 1 2 3 4 5 6 7 |
+-------+-------+-------+-------+-------------------------------+
| SOFf | SOF?2 | SOF?3 | SOF?4 | Reserved |
+-------+-------+-------+-------+-------------------------------+Figure 10: Connection Usage Flags Field Format
If the SOFf bit is one, then FC Frames containing SOFf are intended to be carried on the connection ✨
If the SOF?2 bit is one, then FC Frames containing SOFi2 and SOFn2 are intended to be carried on the connection ✨
If the SOF?3 bit is one, then FC Frames containing SOFi3 and SOFn3 are intended to be carried on the connection ✨
If the SOF?4 bit is one, then FC Frames containing SOFi4, SOFn4, and SOFc4 are intended to be carried on the connection ✨
All or none of the SOFf, SOF?2, SOF?3, and SOF?4 bits MAY be set to one. If all of the SOFf, SOF?2, SOF?3, and SOF?4 bits are zero, then the types of FC Frames intended to be carried on the connection have no specific relationship to the SOF code 🧩
The FCIP Entity SHALL NOT enforce the SOF usage described by the Connection Usage Flags field, and SHALL only use the contents of the field as described below 🙅♀️
The Connection Usage Code field contains Fibre Channel defined information regarding the intended usage of the connection, as specified in FC-BB-2 [3] 📚
The FCIP Entity SHALL use the contents of the Connection Usage Flags and Connection Usage Code fields to locate appropriate QoS settings in the "shared" database of TCP Connection information (see section 8.1.1) and apply those settings to a newly formed connection 💅
The Destination FC Fabric Entity World Wide Name field MAY contain the Fibre Channel Name_Identifier [5] for the FC Fabric entity associated with the FC/FCIP Entity pair that echoes (as opposed to generates) the Special Frame 🪞
The K_A_TOV field SHALL contain the FC Keep Alive Timeout value to be applied to the new TCP Connection, as specified in FC-BB-2 [3] ⏳
For each new incoming TCP connect request and subsequent FSF received, the FCIP Entity SHALL send the contents of the Source FC Fabric Entity World Wide Name, Source FC/FCIP Identifier, Connection Usage Flags and Connection Usage Code fields to the FC Entity along with the other connection information (e.g., FCIP_LEP and FCIP_DE information) 🎉
7.2. Overview of FSF Usage in Connection Establishment 💖
okay so, whenever a brand-new TCP Connection gets set up, an FCIP Special Frame (FSF) makes one round trip 🔁 from the FCIP Entity that starts the TCP connect operation to the FCIP Entity that receives the TCP connect request, and then back again. 🦋 This FSF usage has three jobs:
- Identification of the FCIP Link endpoints 🫶
- Conveyance of a few critical parameters shared by the FC/FCIP Entity pairs involved in the FCIP Link 🔥
- Configuration discovery (used in place of SLP only when allowed by site security policies) 🧚
The specific format and protocol requirements for this usage of the FSF are found in sections 7.1 and 8.1.2.3. This section is just the overview, so no requirements are stated here, only the vibe ✨
Because FCIP is only a tunnel for a Fibre Channel Fabric, and because the Fabric already has its own complex link setup algorithm that can cover many FCIP link setup needs, it is desirable to minimize the complexity of the FSF usage during TCP Connection setup. 💅 With this in mind, this FSF usage is not a login or parameter negotiation mechanism. A single FSF transits each newly established TCP connection as the first bytes sent in each direction. 🌸
Note: This usage of the FSF cannot be eliminated entirely because a newly created TCP Connection must be associated with the correct FCIP Link before FC Fabric initialization of the connection can commence. 🙈
The first bytes sent from the TCP connect request initiator to the receiver are an FSF identifying both the sender and who the sender thinks is the receiver. 🎀 If the contents of this FSF are correct and acceptable to the receiver, the unchanged FSF is echoed back to the sender. This send/echo process is the only set of actions that allows the TCP Connection to be used to carry FC Fabric traffic. 👑 If the send and unchanged echo process does not occur, the algorithm followed at one or both ends of the TCP Connection results in the closure of the TCP Connection (see section 8.1 for specific algorithm requirements). 💀
Note: Since the FSF is used in such a limited manner, and since it must be echoed without changes before a TCP Connection is allowed to carry user data, no error checking beyond that provided by TCP is deemed necessary. 🫶
As described above, the primary purpose of the FSF usage during TCP Connection setup is identifying the FCIP Link to which the new TCP Connection belongs. 🌈 From these beginnings, it is only a small stretch to imagine using the FSF as a simplified configuration discovery tool, and the mechanics of such a usage are described in section 8.1.
However, using the FSF for configuration discovery lacks the broad range of capabilities provided by SLPv2, and most particularly lacks the security capabilities of SLPv2. 😭 For these reasons, using the FSF for configuration discovery is not appropriate for all environments. So the choice to use the FSF for discovery purposes is a policy choice, to be included in the TCP Connection Establishment "shared" database described in section 8.1.1. 📋
When FSF-based configuration discovery is enabled, the normal TCP Connection setup rules outlined above are modified as follows. 🦋
Normally, the algorithm executed by an FCIP Entity receiving an FSF includes verifying that its own identification information in the arriving FSF is correct, and closing the TCP Connection if it is not. 🙅 This can be viewed as requiring the initiator of a TCP connect request to know in advance the identity of the FCIP Entity that is the target of that request (using SLP, for example). Through the FSF, it effectively says, "I think I'm talking to X." If the party at the other end of the TCP connect request is really Y, then it simply hangs up. 📞💔
FSF-based discovery lets the "I think I'm talking to X" be replaced with "Please tell me who I am talking to?", and this is done by replacing an explicit value in the Destination FC Fabric Entity World Wide Name field with zero. 🧠✨
If the policy at the receiving FCIP Entity allows FSF-based discovery, the zero is replaced with the correct Destination FC Fabric Entity World Wide Name value in the echoed FSF. 🎉 This is still subject to the rules of sending with unchanged echo, so closure of the TCP Connection occurs after the echoed FSF is received by the TCP connect initiator. 🔒
Despite the TCP Connection closure, however, the TCP connect initiator now knows the correct Destination FC Fabric Entity World Wide Name identity of the FCIP Entity at a given IP Address, and a subsequent TCP Connection setup sequence probably will be successful. 🥹💜
The Ch bit in the pFlags field (see section 5.6.1) allows for differentiation between changes in the FSF resulting from transmission errors and changes resulting from intentional acts by the FSF recipient. 🕵️♀️🌟
8. TCP Connection Management 🎀
8.1. TCP Connection Establishment 🚀
8.1.1. Connection Establishment Model
The description of the connection establishment process is a model for the interactions between an FC Entity and an FCIP Entity during TCP Connection establishment. 🧩 The model is written in terms of a "shared" database that the FCIP Entity consults to determine the properties of the TCP Connections to be formed, combined with routine calls to the FC Entity when connections are successfully established. Whether the FC Entity contributes information to the "shared" database is not critical to this model. 🤷♀️ However, the fact that the FCIP Entity MAY consult the database at any time to determine its actions relative to TCP Connection establishment is important. 📌
It is important to remember that this description is only a model for the interactions between an FC Entity and an FCIP Entity. Any implementation that has the same effects on the FC Fabric and IP Network as those described using the model meets the requirements of this specification. 💖 For example, an implementation might replace the "shared" database with a routine interface between the FC and FCIP Entities. 🦋
8.1.2. Creating New TCP Connections
8.1.2.1. Non-Dynamic Creation of New TCP Connections
When an FCIP Entity discovers that a new TCP Connection needs to be established, it SHALL determine the IP Address to which the TCP Connection is to be made, and establish all enabled IP security features for that IP Address as described in section 9. 🔐 Then the FCIP Entity SHALL determine the following information about the new connection in addition to the IP Address:
- The expected Destination FC Fabric Entity World Wide Name of the FC/FCIP Entity pair to which the TCP Connection is being made 🏷️
- TCP Connection Parameters (see section 8.3) ⚙️
- Quality of Service Information (see section 10) 🌈
Based on this information, the FCIP Entity SHALL generate a TCP connect request [6] to the FCIP Well-Known Port of 3225 (or other configuration specific port number) at the specified IP Address. 📨
If the TCP connect request is rejected, the FCIP Entity SHALL act to limit unnecessary repetition of attempts to establish similar connections. 🙅♀️ For example, the FCIP Entity might wait 60 seconds before trying to re-establish the connection. ⏳
If the TCP connect request is accepted, the FCIP Entity SHALL follow the steps described in section 8.1.2.3 to complete the establishment of a new FCIP_DE. 🥳
It is RECOMMENDED that an FCIP Entity not initiate TCP connect requests to another FCIP Entity if incoming TCP connect requests from that FCIP Entity have already been accepted. 🫶✨
8.1.2.2. Dynamic Creation of New TCP Connections
If dynamic discovery of participating FCIP Entities is supported, the function SHALL be performed using the Service Location Protocol (SLPv2) [17] in the manner defined for FCIP usage [20]. 🔍 It's giving discovery, bestie 💅
Upon discovering that dynamic discovery is to be used, the FCIP Entity SHALL enable IP security features for the SLP discovery process as described in [20], and then:
- Determine the one or more FCIP Discovery Domain(s) to be used in the dynamic discovery process; 🗺️
- Establish an SLPv2 Service Agent to advertise the availability of this FCIP Entity to peer FCIP Entities in the identified FCIP Discovery Domain(s); and 📣
- Establish an SLPv2 User Agent to locate service advertisements for peer FCIP Entities in the identified FCIP Discovery Domain(s). 🕵️♀️
For each peer FCIP Entity dynamically discovered through the SLPv2 User Agent, the FCIP Entity SHALL establish all enabled IP security features for the discovered IP Address as described in section 9, and then determine the following information about the new connection:
- The expected Destination FC Fabric Entity World Wide Name of the FC/FCIP Entity pair to which the TCP Connection is being made 🏷️
- TCP Connection Parameters (see section 8.3) ⚙️
- Quality of Service Information (see section 10) 🌈
Based on this information, the FCIP Entity SHALL generate a TCP connect request [6] to the FCIP Well-Known Port of 3225 (or other configuration specific port number) at the IP Address specified by the service advertisement. 📨 If the TCP connect request is rejected, act to limit unnecessary repetition of attempts to establish similar connections. 🙅♀️ If the TCP connect request is accepted, the FCIP Entity SHALL follow the steps described in section 8.1.2.3 to complete the establishment of a new FCIP_DE. 🥳
It is recommended that an FCIP Entity not initiate TCP connect requests to another FCIP Entity if incoming TCP connect requests from that FCIP Entity have already been accepted. 💜
8.1.2.3. Connection Setup After a Successful TCP Connect Request ✨
okay bestie, whether you're doing Non-Dynamic TCP Connection creation (see section 8.1.2.1) or Dynamic TCP Connection creation (see section 8.1.2.2), the steps in this section SHALL be followed to get the TCP Connection setup all the way done 💅🎀
once the TCP connect request gets accepted, the FCIP Entity SHALL send an FCIP Special Frame (FSF, see section 7) as the very first bytes on the newly formed connection, and it has to keep a copy of those bytes for comparing later 🧚✨ every field in the FSF SHALL be filled in the way section 7 describes, and especially these ones:
- the
Source FC Fabric Entity World Wide Namefield SHALL contain the FC Fabric Entity World Wide Name for the FC/FCIP Entity pair that's originating the TCP connect request 👑; - the
Source FC/FCIP Entity Identifierfield SHALL contain a unique identifier that's assigned by the FC Fabric entity whose world wide name shows up in theSource FC Fabric Entity World Wide Namefield 🦋; - the
Connection Noncefield SHALL contain a 64-bit random number that's different from any recently usedConnection Noncevalue. to make the nonce secure enough, the Randomness Recommendations for Security [9] SHOULD be followed 🔐; and - the
Destination FC Fabric Entity World Wide Namefield SHALL contain 0 or the expected FC Fabric Entity World Wide Name for the FC/FCIP Entity pair whose destination is the TCP connect request 🌸.
once the FSF is sent on the new connection, the FCIP Entity SHALL wait for that FSF to be echoed back as the first bytes received on the newly formed connection 🫶.
the FCIP Entity MAY apply a timeout of not less than 90 seconds while it waits for the echoed FSF bytes. if that timeout expires, the FCIP Entity SHALL close the TCP Connection and notify the FC Entity with the reason for the closure 😭.
if the echoed FSF bytes don't exactly match the FSF bytes that were sent (words 7 through 17 inclusive), or if the echoed Destination FC Fabric Entity World Wide Name field contains zero, the FCIP Entity SHALL close the TCP Connection and notify the FC Entity with the reason for the closure 💀.
the FCIP Entity SHALL only do the following steps if the echoed FSF bytes exactly match the FSF bytes sent (words 7 through 17 inclusive) ✨
- Instantiate the appropriate Quality of Service (see section 10) conditions on the newly created TCP Connection,
- if the IP Address and TCP Port that the TCP Connection was made to aren't associated with any other FCIP_LEP, create a new FCIP_LEP for the new FCIP Link,
- create a new FCIP_DE within the newly created FCIP_LEP to service the new TCP Connection, and
- inform the FC Entity of the new FCIP_LEP, FCIP_DE, Destination FC Fabric Entity World Wide Name, Connection Usage Flags, and Connection Usage Code 🎉
8.1.3. Processing Incoming TCP Connect Requests 🧚
the FCIP Entity SHALL listen for new TCP Connection requests [6] on the FCIP Well-Known Port (3225) 👂. an FCIP Entity MAY also accept and establish TCP Connections to a TCP port number other than the FCIP Well-Known Port, as configured by the network administrator in a manner that's outside the scope of this specification 🌈.
the FCIP Entity SHALL figure out the following info about the requested connection:
- whether the "shared" database (see section 8.1.1) allows the requested connection
- whether IP security setup has been performed for the IP security features enabled on the connection (see section 9)
if the requested connection isn't allowed, the FCIP Entity SHALL reject the connect request using appropriate TCP means 🚫. if the requested connection is allowed, the FC Entity SHALL make sure the required IP security features are enabled, and accept the TCP connect request 🔥.
once the TCP connect request has been accepted, the FCIP Entity SHALL wait for the FSF sent by the originator of the TCP connect request (see section 8.1.2) as the first bytes received on the accepted connection 🎀.
the FCIP Entity MAY apply a timeout of no less than 90 seconds while it waits for the FSF bytes. if that timeout expires, the FCIP Entity SHALL close the TCP Connection and notify the FC Entity with the reason for the closure ⏰.
Note: one method for attacking the security of the FCIP Link formation process (detailed in section 9.1) depends on keeping a TCP connect request open without sending an FSF. implementations should keep this in mind when handling TCP connect requests where the FSF isn't sent in a timely manner 🚨🙈
once the FCIP Entity receives the FSF sent by the originator of the TCP connect request, it SHALL inspect the contents of these fields:
Connection Nonce,Destination FC Fabric Entity World Wide Name,Connection Usage Flags, andConnection Usage Code.
if the Connection Nonce field contains a value that's identical to the most recently received Connection Nonce from the same IP Address, the FCIP Entity SHALL close the TCP Connection and notify the FC Entity with the reason for the closure 🙅♀️.
if an FCIP Entity receives a duplicate FSF during the FCIP Link formation process, it SHALL close that TCP Connection and notify the FC Entity with the reason for the closure 👯.
if the Destination FC Fabric Entity World Wide Name contains 0, the FCIP Entity SHALL take one of the following three actions:
- leave the
Destination FC Fabric Entity World Wide Namefield and Ch bit both 0; - change the
Destination FC Fabric Entity World Wide Namefield to match the FC Fabric Entity World Wide Name associated with the FCIP Entity that received the TCP connect request, and change the Ch bit to 1; or - close the TCP Connection without sending any response.
which of these actions you pick depends on how the FCIP Entity is expected to be used 🤔. the FCIP Entity may consult the "shared" database when it's choosing between the above actions.
if:
- a) the
Destination FC Fabric Entity World Wide Namecontains a non-zero value that does not match the FC Fabric Entity World Wide Name associated with the FCIP Entity that received the TCP connect request, or - b) the contents of the
Connection Usage FlagsandConnection Usage Codefields aren't acceptable to the FCIP Entity that received the TCP connect request,
then the FCIP Entity SHALL take one of the following two actions:
- change the contents of the unacceptable fields to correct/acceptable values and set the Ch bit to 1; or
- close the TCP Connection without sending any response.
if the FCIP Entity makes any changes to the content of the FSF, it SHALL also set the Ch bit to 1 🎀.
if any changes have been made to the received FSF during the processing described above, the following steps SHALL be performed:
- the changed FSF SHALL be echoed to the originator of the TCP connect request as the only bytes transmitted on the accepted connection;
- the TCP Connection SHALL be closed (the FC Entity doesn't need to be notified of the TCP Connection closure in this case, because it isn't a sign of an error); and
- all of the additional processing described in this section SHALL be skipped.
the remaining steps in this section SHALL be performed only if the FCIP Entity hasn't changed the contents of the above mentioned fields to correct/acceptable values 💖.
if the Source FC Fabric Entity World Wide Name and Source FC/FCIP Entity Identifier field values in the FSF don't match the Source FC Fabric Entity World Wide Name and Source FC/FCIP Entity Identifier associated with any other FCIP_LEP, the FCIP Entity SHALL:
- echo the unchanged FSF to the originator of the TCP connect request as the first bytes transmitted on the accepted connection;
- instantiate the appropriate Quality of Service (see section 10.2) conditions on the newly created TCP Connection, considering the Connection Usage Flags and Connection Usage Code fields, and "shared" database information (see section 8.1.1) as appropriate,
- create a new FCIP_LEP for the new FCIP Link,
- create a new FCIP_DE within the newly created FCIP_LEP to service the new TCP Connection, and
- inform the FC Entity of the new FCIP_LEP, FCIP_DE, Source FC Fabric Entity World Wide Name, Source FC/FCIP Entity Identifier, Connection Usage Flags, and Connection Usage Code 🦋.
if the Source FC Fabric Entity World Wide Name and Source FC/FCIP Entity Identifier field values in the FCIP Special Frame match the Source FC Fabric Entity World Wide Name and Source FC/FCIP Entity Identifier associated with an existing FCIP_LEP, the FCIP Entity SHALL:
- request that the FC Entity authenticate the source of the TCP connect request (see FC-BB-2 [3]), giving the FC Entity the following info for authentication purposes:
- a) Source FC Fabric Entity World Wide Name,
- b) Source FC/FCIP Entity Identifier, and
- c) Connection Nonce.
the FCIP Entity SHALL NOT use the new TCP Connection for any purpose until the FC Entity authenticates the source of the TCP connect request 🔒. if the FC Entity indicates that the TCP connect request cannot be properly authenticated, the FCIP Entity SHALL close the TCP Connection and skip all of the remaining steps in this section.
the definition of the FC Entity SHALL include an authentication mechanism for use in response to a TCP connect request source that communicates with the partner FC/FCIP Entity pair on an existing FCIP Link. this authentication mechanism should use a previously authenticated TCP Connection in the existing FCIP Link to authenticate the Connection Nonce sent in the new TCP Connection setup process. the FCIP Entity SHALL treat failure of this authentication as an authentication failure for the new TCP Connection setup process 🛡️.
- echo the unchanged FSF to the originator of the TCP connect request as the first bytes transmitted on the accepted connection;
- instantiate the appropriate Quality of Service (see section 10.2) conditions on the newly created TCP Connection, considering the Connection Usage Flags and Connection Usage Code fields, and "shared" database information (see section 8.1.1) as appropriate,
- create a new FCIP_DE within the existing FCIP_LEP to service the new TCP Connection, and
- inform the FC Entity of the FCIP_LEP, Source FC Fabric Entity World Wide Name, Source FC/FCIP Entity Identifier, Connection Usage Flags, Connection Usage Code, and new FCIP_DE 🌸.
ok so here's the tricky bit 🙈. the originator of TCP connect requests uses the IP Address and TCP Port to figure out which TCP Connections belong to which FCIP_LEPs. the recipient of TCP connect requests, on the other hand, uses the Source FC Fabric Entity World Wide Name and Source FC/FCIP Entity Identifier fields from the FSF to figure out which TCP Connection belongs to which FCIP_LEP. because of that, an FCIP Entity that both originates and receives TCP connect requests can't match the FCIP_LEPs linked to originated TCP connect requests with the FCIP_LEPs linked to received TCP connect requests 😭✨
8.1.4. Simultaneous Connection Establishment ✨
okay so what if two FCIP Entities both try to open up at the exact same time? 👀 Then two TCP Connections get formed, and the SF starts at one end on one connection and at the other end on the other. Setup keeps going like we described above on both connections, and those steps end up forming two FCIP Links between the same FCIP Entities. 🫶
and this is NOT an error, bestie 🙅♀️ FC is literally built to handle two roughly equal connections between FC Fabric elements. it's giving "no big deal" energy 💅
deciding to set up pairs of FCIP Links like this is a site policy decision, and it can be covered in the "shared" database described in section 8.1.1 🗂️
8.2. Closing TCP Connections
the FCIP Entity SHALL give the FC Entity a mechanism with acknowledgement so the FC Entity can close an existing TCP Connection at any time 🚪 this is so useful for when a connection is throwing too many errors, etc. 😤
8.3. TCP Connection Parameters
so if you want to manage FCIP_LEP resources and FCIP Link resources efficiently, looking at certain TCP Connection parameters is recommended 🧐
8.3.1. TCP Selective Acknowledgement Option
the Selective Acknowledgement option RFC 2883 [18] lets the receiver acknowledge multiple lost packets in a single ACK, which means faster recovery 🏃♀️ an FCIP Entity MAY negotiate use of TCP SACK and use it for faster recovery from lost packets and holes in TCP sequence number space.
8.3.2. TCP Window Scale Option
the TCP Window Scale option [8] lets the receiver advertise TCP window sizes bigger than the 16-bit limits. this is necessary so data in long fat networks can actually fill the available pipe 🌊 it also means the TCP sender needs buffering that matches the (bandwidth*delay) product of the TCP Connection. an FCIP_LEP uses locally available mechanisms to set a window size that matches the available local buffer resources and the desired throughput. 💖
8.3.3. Protection Against Sequence Number Wrap
main character energy alert: It is RECOMMENDED that FCIP Entities implement protection against wrapped sequence numbers PAWS [8]. it's quite possible that within a single connection, TCP sequence numbers wrap within a timeout window 🙀
8.3.4. TCP_NODELAY Option
FCIP Entities should disable the Nagle Algorithm as described in RFC 1122 [7] section 4.2.3.4. by tradition, this can be done by setting the TCP_NODELAY option to one at the local TCP interface 🎀
8.4. TCP Connection Considerations
when a connection is idle, the TCP "keep alive" option of TCP is normally used to keep it alive 🫧 BUT this timeout is pretty large and might stop you from catching a loss of connectivity early 🥲 so to detect that faster, FC Entities SHOULD implement some form of Fibre Channel connection failure detection (see FC-BB-2 [3]) 🔍
when an FCIP Entity discovers that TCP connectivity has been lost, the FCIP Entity SHALL notify the FC Entity of the failure, including information about the reason for the failure 📣
8.5. Flow Control Mapping between TCP and FC
the FCIP Entity and FC Entity are connected to the IP Network and FC Fabric, respectively, and they have to follow the flow control mechanisms of both TCP and FC. those two work independently of each other, so it's a whole vibe 🌈
this section gives guidelines on how the FCIP Entity can map TCP flow control to status notifications for the FC Entity 🗺️
there are two scenarios where flow control management gets crucial:
- When there is a line speed mismatch between the FC and IP interfaces. Even though it is RECOMMENDED that both the FC and IP interfaces to the FC Entity and FCIP Entity, respectively, be of comparable speeds, it is possible to carry FC traffic over an IP Network that has a different line speed and bit error rate. 🏎️
- When the FC Fabric or IP Network encounters congestion. Even when both the FC Fabric or IP network are of comparable speeds, during the course of operation, the FC Fabric or the IP Network could hit congestion due to transient conditions. 🚦
the FC Entity uses Fibre Channel mechanisms for flow control at the FC Frame Receiver Portal, based on information supplied by the FCIP Entity about flow constraints at the Encapsulated Frame Transmitter Portal. the FCIP Entity uses TCP mechanisms for flow control at the Encapsulated Frame Receiver Portal, based on information supplied by the FC Entity about flow constraints at the FC Frame Transmitter Portal. 🤝
coordinating these flow control mechanisms, one credit based and the other window based, depends on a painstaking design that is outside the scope of this specification 🧵
9. Security 🔐
FCIP uses the IPsec protocol suite to provide data confidentiality and authentication services, and IKE as the key management protocol. this section describes the requirements for the various components of these protocols as used by FCIP, based on FCIP operating environments. more consideration for using IPsec and IKE with the FCIP protocol can be found in [21]. in the event that requirements in [21] conflict with requirements stated in this document, the requirements in this document SHALL prevail. 🛡️
9.1. Threat Models
using a general purpose, wide-area network, such as an IP Network, as a functional replacement for physical cabling brings in some security problems you don't normally see in Fibre Channel Fabrics. FC interconnect cabling is typically physically protected from outside access, but public IP Networks let hostile parties impact the security of the transport infrastructure. 😬
the general effect is that the security of an FC Fabric is only as good as the security of the entire IP Network that carries the FCIP Links used by that FC Fabric. the following broad classes of attacks are possible:
- Unauthorized Fibre Channel elements can gain access to resources through normal Fibre Channel Fabric and processes. Although this is a valid threat, securing the Fibre Channel Fabrics is outside the scope of this document. Securing the IP Network is the issue considered in this specification.
- Unauthorized agents can monitor and manipulate Fibre Channel traffic flowing over physical media used by the IP Network and accessible to the agent. 👀
- TCP Connections may be hijacked and used to instantiate an invalid FCIP Link between two peer FCIP Entities. 🏴☠️
- Valid and invalid FCIP Frames may be injected on the TCP Connections.
- The payload of an FCIP Frame may be altered or transformed. The TCP checksum, FCIP ones complement checks, and FC frame CRC do not protect against this, because all of them can be modified or regenerated by a malicious and determined adversary. 💀
- Unauthorized agents can masquerade as valid FCIP Entities and disturb proper operation of the Fibre Channel Fabric.
- Denial of Service attacks can be mounted by injecting TCP Connection requests and other resource exhaustion operations. 🚫
- An adversary may launch a variety of attacks against the discovery process [17].
- An attacker may exploit the FSF authentication mechanism of the FCIP Link formation process (see section 8.1.3). The attacker could observe the FSF contents sent on an initial connection of an FCIP Link and use the observed nonce, Source FC/FCIP Entity Identifier, and other FSF contents to form an FCIP Link using the attacker's own previously established connection, while resetting/blocking the observed connection. Although the use of timeout for reception of FSF reduces the risk of this attack, such an attack is possible. See section 9.3.1 to protect against this specific attack. 😱
this one is serious, bestie. the attacker can copy the FSF contents off a real connection and reuse them on their own connection, so go read section 9.3.1 for the protection 🔒
the existing IPsec Security Architecture and protocol suite [10] offers protection from these threats. An FCIP Entity MUST implement portions of the IPsec protocol suite as described in this section. 🫶
9.2. FC Fabric and IP Network Deployment Models
in the context of enabling a secure FCIP tunnel between FC SANs, these characteristics of the IP Network deployment are useful to note:
- The FCIP Entities share a peer-to-peer relationship. Therefore, the administration of security policies applies to all FCIP Entities in an equal manner. This differs from a true Client-Server relationship, where there is an inherent difference in how security policies are administered. 🤝
- Policy administration, as well as security deployment and configuration, is constrained to the set of FCIP Entities, which poses less of a requirement on a scalable mechanism. For example, the validation of credentials can be relaxed to the point where deploying a set of pre-shared keys is a viable technique. 🗝️
- TCP Connections and the IP Network are terminated at the FCIP Entity. The granularity of security implementation is at the level of the FCIP tunnel endpoint (or FCIP Entity), unlike other applications where there is user-level termination of TCP Connections. User-level objects are not controllable by or visible to FCIP Entities. All user-level security related to FCIP is the responsibility of the Fibre Channel standards and is outside the scope of this specification. 🧱
- When an FCIP Entity is deployed, its IP addresses will typically be statically assigned. However, support for dynamic IP address assignment, as described in [33], while typically not required, cannot be ruled out. 📍
9.3. FCIP Security Components
okay this part is the big one 👑 FCIP Security compliant implementations MUST implement ESP and the IPsec protocol suite based cryptographic authentication and data integrity [10], as well as confidentiality using the algorithms and transforms described in this section. also, FCIP implementations MUST meet the secure key management requirements of the IPsec protocol suite. 🔑
9.3.1. IPsec ESP Authentication and Confidentiality
FCIP Entities MUST implement IPsec ESP [12] in Tunnel Mode for providing Data Integrity and Confidentiality. FCIP Entities MAY implement IPsec ESP in Transport Mode, if deployment considerations require the use of Transport Mode. When ESP is utilized, per-packet data origin authentication, integrity, and replay protection MUST be used. 🧁
if Confidentiality is not enabled but Data Integrity is enabled, ESP with NULL Encryption [15] MUST be used. 🙈
IPsec ESP for message authentication computes a cryptographic hash over the payload that is protected. While IPsec ESP mandates that compliant implementations support certain algorithms for deriving this hash, FCIP implementations:
- MUST implement HMAC with SHA-1 [11]
- SHOULD implement AES in CBC MAC mode with XCBC extensions [23]
- DES in CBC mode SHOULD NOT be used due to inherent weaknesses 💅
for ESP Confidentiality, FCIP Entities:
- MUST implement 3DES in CBC mode [16]
- SHOULD implement AES in CTR mode [22]
- MUST implement NULL Encryption [15] 🌸
9.3.2. Key Management ✨
okay bestie, key management is where the secrets get handled 🔐 FCIP Entities MUST support IKE [14] for peer authentication, negotiating Security Associations (SA), and Key Management using the IPsec DOI [13]. Manual keying SHALL NOT be used for establishing an SA, because it doesn't give you what you need for rekeying (see section 9.3.3). 🙅♀️ Conformant FCIP implementations MUST support peer authentication using pre-shared keys, and they MAY support peer authentication using digital certificates. Peer authentication using the public key encryption methods from IKE [14] sections 5.2 and 5.3 SHOULD NOT be used. 💅
IKE Phase 1 sets up a secure, MAC-authenticated channel that IKE Phase 2 then uses 🦋 FCIP implementations MUST support IKE Main Mode and SHOULD support Aggressive Mode.
IKE Phase 1 exchanges MUST explicitly carry the Identification Payload fields (IDii and IDir). 🆔 Conformant FCIP implementations MUST use ID_IPV4_ADDR, ID_IPV6_ADDR (if the protocol stack supports IPv6), or ID_FQDN Identification Type values. The ID_USER_FQDN, IP Subnet, IP Address Range, ID_DER_ASN1_DN, and ID_DER_ASN1_GN Identification Type values SHOULD NOT be used. The ID_KEY_ID Identification Type values MUST NOT be used. As described in [13], the port and protocol fields in the Identification Payload MUST be set to zero or UDP port 500. 🌸
FCIP Entities negotiate SA parameters during IKE Phase 2 only, using "Quick Mode" 🎀 For FCIP Entities doing IKE "Quick Mode," there is no requirement for PFS (Perfect Forward Secrecy). FCIP implementations MUST use either ID_IPV4_ADDR or ID_IPV6_ADDR Identification Type values, depending on which version of IP is supported. Other Identification Type values MUST NOT be used.
The number of Phase 2 SAs may be limited, so Phase 2 delete messages may be sent for idle SAs. 🧹 Getting a Phase 2 delete message SHOULD NOT be treated as a reason to tear down an FCIP Link or any of its TCP connections. When there's new activity on that idle link, a new Phase 2 SA MUST be re-established. ✨
For a given pair of FCIP Entities, one IKE Phase 1 negotiation can be used for all Phase 2 negotiations. In other words, every TCP Connection bundled into the single FCIP Link can share the same Phase 1 results 🫶
Rekeying over and over with "Quick Mode" on the same shared secret slowly weakens that secret's cryptographic properties 😬 To prevent that, Phase 1 SHOULD be invoked periodically to create a new set of IKE shared secrets and related security parameters. 🔄
IKE Phase 1 establishment requires the following key distribution and FCIP Entities:
- MUST support pre-shared IKE keys.
- MAY support certificate-based peer authentication using digital signatures.
- SHOULD NOT use peer authentication using the public key encryption methods outlined in sections 5.2 and 5.3 of [14].
When pre-shared keys are used, IKE Main Mode only works when both peers of an FCIP Link use statically assigned IP addresses. 📌 If you try to support dynamically assigned IP addresses with Main Mode, you'd be forced into using group pre-shared keys, and group pre-shared keys together with Main Mode is not recommended, because it exposes the deployed environment to man-in-the-middle attacks. 👀 So if either peer of an FCIP Link uses dynamically assigned addresses, Aggressive Mode SHOULD be used and Main Mode SHOULD NOT be used. 🙈
When Digital Signatures are used, either IKE Main Mode or IKE Aggressive Mode may be used. In every case, access to locally stored secret information (the pre-shared key, or the private key used for digital signing) MUST be suitably restricted, because if secret information gets compromised, the security properties of the IKE/IPsec protocols are nullified 💀 The mechanisms for doing that are outside the scope of this document. Support for IKE Oakley Groups [27] is not required. 🔐
To set up a secure FCIP Link, the two participating FCIP Entities consult a Security Policy Database (SPD). The SPD is described in IPsec [10] Section 4.4.1. 🗂️ FCIP Entities may have more than one interface and IP Address, and an FCIP Link can contain multiple TCP connections whose FCIP endpoint IP Addresses are different. In that case, an IKE Phase 1 SA is established for each FCIP endpoint IP Address pair. Within IKE Phase 1, FCIP implementations must support the ID_IPV4_ADDR, ID_IPV6_ADDR (if the protocol stack supports IPv6), and ID_FQDN Identity Payloads. If FCIP Endpoint addresses are dynamically assigned, it may be helpful to use ID_FQDN, which is why the IP_FQDN Identity Payload MUST be supported. Other identity payloads (ID_USER_FQDN, ID_DER_ASN1_GN, ID_KEY_ID) SHOULD NOT be used. 🧠
When IKE negotiations succeed, both FCIP Entities store the SA parameters in their SA database (SAD). The SAD is described in IPsec [10] Section 4.4.3. 🗄️ The SAD holds the set of active SA entries, and each entry contains the Sequence Counter Overflow, Sequence Number Counter, Anti-replay Window, and the Lifetime of the SA. FCIP Entities SHALL use a default SA Lifetime of one hour and a default Anti-replay window of 32 sequence numbers. ⏰
When a TCP Connection gets established between two FCIP_DEs, two unidirectional SAs are created for that connection, and each SA is identified by a Security Parameter Index (SPI) 🦋 One SA is tied to the incoming traffic flow and the other is tied to the outgoing traffic flow. The FCIP_DEs at each end of the TCP connection MUST keep track of the SPIs for both their incoming and outgoing FCIP Encapsulated Frames. 📇
FCIP Entities MAY provide administrative management of Confidentiality usage. Those management interfaces SHOULD be provided in a secure manner, so an attacker can't subvert the security process by going after the management interface 🛡️
9.3.3. ESP Replay Protection and Rekeying Issues 🔁
FCIP Entities MUST implement Replay Protection against ESP Sequence Number wrap, as described in [14]. On top of that, depending on the cipher algorithm and the number of bits in the cipher block size, the validity of the key can become compromised. ⚠️ In both cases, the SA needs to be re-established.
FCIP Entities MUST use the results of IKE Phase 1 negotiation to kick off an IKE Phase 2 "Quick Mode" exchange and establish new SAs. 🔑
For a smooth SA transition, it is RECOMMENDED that both FCIP Entities refresh the SPI when the sequence number counter reaches 2^31 (aka half the sequence number space) 🌗 It also is RECOMMENDED that the receiver run with multiple SPIs for the same TCP Connection for a period of 2^31 sequence number packets before aging out an SPI. 🕰️
When a new SPI gets created for the outgoing direction, the sending side SHALL start using it for all new FCIP Encapsulated Frames. Frames that are in-flight, or re-sent because of TCP retransmissions (etc.), MAY use either the new SPI or the one being replaced. 🫠
9.4. Secure FCIP Link Operation 🌈
9.4.1. FCIP Link Initialization Steps 🚀
FCIP implementations may allow security mechanisms to be enabled and disabled at the granularity of an FCIP Link. If security is enabled, the following FCIP Link Initialization steps MUST be followed. 📋
When an FCIP Link is initialized, before any FCIP TCP Connections get established, the local SPD is consulted to figure out whether IKE Phase 1 has been completed with the FCIP Entity in the peer FCIP Entity, as identified by the WWN. 🔍
If Phase 1 is already completed, IKE Phase 2 proceeds. Otherwise, IKE Phase 1 MUST be completed before IKE Phase 2 can start. Both IKE Phase 1 and Phase 2 transactions use UDP Port 500. If IKE Phase 1 fails, the FCIP Link initialization terminates and tells the FC entity why it terminated. Otherwise, the FCIP Link initialization moves on to TCP Connection Initialization. ➡️
As described in section 8.1, FCIP Entities exchange an FSF to form an FCIP Link. Using ESP Confidentiality is an effective countermeasure against any perceived security risks of FSF. 🛡️✨
9.4.2. TCP Connection Security Associations (SAs) 🎀
Each TCP connection MUST be protected by an IKE Phase 2 SA. Traffic from one or more TCP connections may flow within each IPsec Phase 2 SA. An IKE Phase 2 SA can protect multiple TCP connections, but all packets of a TCP connection are protected using only one IKE Phase 2 SA. 🧵
If different Quality of Service settings are applied to TCP connections, it is advisable to use a different IPsec SA for each of those connections. Applying a different quality of service to connections handled by the same IPsec SA can cause reordering and push packets outside the replay window 😵 For additional details, see [21].
FCIP implementations need not verify that the IP addresses and port numbers in a packet match any locally stored per-connection values, since that check is left to the IPsec layer. 🙌
An implementation is free to perform several IKE Phase 2 negotiations and cache them in its local SPIs, although entries in that cache can be flushed according to the current SA Lifetime settings. 🧹
9.4.3. Handling Data Integrity and Confidentiality Violations 🚨
When a datagram arrives and the ESP packet fails an integrity check, the receiver MUST drop the datagram, which triggers TCP retransmission. If lots of these datagrams get dropped, a receiving FCIP Entity MAY close the TCP Connection and notify the FC Entity of the reason for the closure. 💔
An implementation SHOULD follow guidelines for auditing all auditable ESP events per IPsec [10] Section 7. 📝
Integrity checks MUST be performed if Confidentiality is enabled. 🔒💖
10.1. Performance Considerations 🦋
Okay so, traditionally the links between FC Fabric components have been giving low latency and high throughput 🔥. FCIP's whole job is to give you that same energy over an IP Network, where low latency and high throughput are not as guaranteed 😭. So it's no surprise that FCIP Entities and their counterpart FC Entities are probably going to be obsessed with using the IP Network optimally 💅.
There are lots of ways to get high throughput and low latency that fit the distances involved in an IP Network ✨. For example, you could build a private IP Network that's just for the FCIP Entities 🏰. The options that are actually in scope for this spec are the ones we talk about here 🫶.
One option for making it more likely that FCIP data streams get low latency and high throughput is the IP QoS techniques in section 10.2 🎀. This can be valuable when it's applied to a single TCP Connection. Depending on how sophisticated the FC Entity is, you can get even more value by having multiple TCP Connections with different QoS characteristics 💖.
There are a bunch of reasons an FC Entity might want multiple TCP Connections inside an FCIP_LEP 🧚. Maybe it wants differentiated services for different TCP data connections between FCIP_LEPs, or maybe it just wants to separately queue different streams of traffic that don't have a common in-order delivery requirement 🌸.
When a new TCP Connection gets created, the FC Entity SHALL specify to the FCIP Entity the QoS characteristics (including but not limited to IP per-hop-behavior) that will be used for the lifetime of that connection 👑. This MAY be achieved by having:
a) only one set of QoS characteristics for all TCP Connections;
b) a default set of QoS characteristics that the FCIP Entity applies in the absence of differing instructions from the FC Entity; or
c) a sophisticated mechanism for exchanging QoS requirements information between the FC Entity and FCIP Entity each time a new TCP Connection is created.
Once established, the QoS characteristics of a TCP Connection SHALL NOT be changed, since this specification provides no mechanism for the FC Entity to control such changes 🚫. The way FCIP gives you different QoS is by setting up a different TCP Connection and its associated FCIP_DEs 🦋.
When FCIP is used with a network with a large (bandwidth*delay) product, it is RECOMMENDED that FCIP_LEPs use the TCP mechanisms (window scaling and wrapped sequence protection) for Long Fat Networks (LFNs) as defined in RFC 1323 [24]. 🍓
10.2. IP Quality of Service (QoS) Support 🎀
Tons of methods for QoS have been devised or proposed 🌈. Here are some of the big ones (and yes, the list can go on, bestie):
- Multi-Protocol Label Switching (MPLS) -- RFC 3031 [32]
- Differentiated Services Architecture (diffserv) -- RFC 2474 [28], RFC 2475 [29], RFC 2597 [30], and RFC 2598 [31] -- and other forms of per-hop-behavior (PHB)
- Integrated Services, RFC 1633 [25]
- IEEE 802.1p
The purpose of this specification is not to pick any particular form of IP QoS 🙅. It's about specifying only the issues that must be addressed in order to maximize interoperability between FCIP equipment that has been manufactured by different vendors 🤝.
It is RECOMMENDED that some form of preferential QoS be used for FCIP traffic to minimize latency and packet drops ✨. No particular form of QoS is recommended 🙈.
If a PHB IP QoS is implemented, it is RECOMMENDED that it interoperate with diffserv (see RFC 2474 [28], RFC 2475 [29], RFC 2597 [30], and RFC 2598 [31]) 🧁.
If no form of preferential QoS is implemented, the DSCP field SHOULD be set to '000000' to avoid negative impacts on other network components and services that may be caused by uncontrolled usage of non-zero values of the DSCP field 💀.
11. References 📚
11.1. Normative References
The references in this section were current as of the time this specification was approved 🗓️. This specification is intended to operate with newer versions of the referenced documents, and looking for newer reference documents is recommended 👀.
- [1] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
- [2] Fibre Channel Backbone (FC-BB), ANSI INCITS.342:2001, December 12, 2001.
- [3] Fibre Channel Backbone -2 (FC-BB-2), ANSI INCITS.372:2003, July 25, 2003.
- [4] Fibre Channel Switch Fabric -2 (FC-SW-2), ANSI INCITS.355:2001, December 12, 2001.
- [5] Fibre Channel Framing and Signaling (FC-FS), ANSI INCITS.373:2003, October 27, 2003.
- [6] Postel, J., "Transmission Control Protocol", STD 7, RFC 793, September 1981.
- [7] Braden, R., "Requirements for Internet Hosts -- Communication Layers", STD 3, RFC 1122, October 1989.
- [8] Jacobson, V., Braden, R. and D. Borman, "TCP Extensions for High Performance", RFC 1323, May 1992.
- [9] Eastlake, D., Crocker, S. and J. Schiller, "Randomness Recommendations for Security", RFC 1750, December 1994.
- [10] Kent, S. and R. Atkinson, "Security Architecture for the Internet Protocol", RFC 2401, November 1998.
- [11] Krawczyk, H., Bellare, M. and R. Canetti, "HMAC: Keyed- Hashing for Message Authentication", RFC 2104, February 1997.
- [12] Kent, S. and R. Atkinson, "IP Encapsulating Security Payload (ESP)", RFC 2406, November 1998.
- [13] Piper, D., "The Internet IP Security Domain of Interpretation of ISAKMP", RFC 2407, November 1998.
- [14] Harkins, D. and D. Carrel, "The Internet Key Exchange (IKE)", RFC 2409, November 1998.
- [15] Glenn, R. and S. Kent, "The NULL Encryption Algorithm and Its Use With IPsec", RFC 2410, November 1998.
- [16] Pereira, R. and R. Adams, "The ESP CBC-Mode Cipher Algorithms", RFC 2451, November 1998.
- [17] Guttman, E., Perkins, C., Veizades, J. and M. Day, "Service Location Protocol, version 2", RFC 2608, July 1999.
- [18] Floyd, S., Mahdavi, J., Mathis, M. and M. Podolsky, "SACK Extension", RFC 2883, July 2000.
- [19] Weber, R., Rajagopal, M., Travostino, F., O'Donnell, M., Monia, C. and M. Merhar, "Fibre Channel (FC) Frame Encapsulation", RFC 3643, December 2003.
- [20] Peterson, D., "Finding Fibre Channel over TCP/IP (FCIP) Entities Using Service Location Protocol version 2 (SLPv2)", RFC 3822, July 2004.
- [21] Aboba, B., Tseng, J., Walker, J., Rangan, V. and F. Travostino, "Securing Block Storage Protocols over IP", RFC 3723, April 2004.
- [22] Frankel, S., Glenn, R. and S. Kelly, "The AES-CBC Cipher Algorithm and Its Use with IPsec", RFC 3602, September 2003.
- [23] Frankel, S. and H. Herbert, "The AES-XCBC-MAC-96 Algorithm and Its Use With IPsec", RFC 3566, September 2003.
11.2. Informative References
- [24] Jacobson, V., Braden, R. and D. Borman, "TCP Extensions for High Performance", RFC 1323, May 1992.
- [25] Braden, R., Clark, D. and S. Shenker, "Integrated Services in the Internet Architecture: an Overview", RFC 1633, June 1994.
- [26] Mills, D., "Simple Network Time Protocol (SNTP) Version 4 for IPv4, IPv6 and OSI", RFC 2030, October 1996.
- [27] Orman, H., "The OAKLEY Key Determination Protocol", RFC 2412, November 1998.
- [28] Nichols, K., Blake, S., Baker, F. and D. Black, "Definition of the Differentiated Services Field (DS Field) in the IPv4 and Ipv6 Headers", RFC 2474, December 1998.
- [29] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z. and W. Weiss, "An Architecture for Differentiated Services", RFC 2475, December 1998.
- [30] Heinanen, J., Baker, F., Weiss, W. and J. Wroclawski, "An Assured Forwarding PHB", RFC 2597, June 1999.
- [31] Jacobson, V., Nichols, K. and K. Poduri, "An Expedited Forwarding PHB Group", RFC 2598, June 1999.
- [32] Rosen, E., Viswanathan, A. and R. Callon, "Multiprotocol Label Switching Architecture", RFC 3031, January, 2001.
- [33] Patel, B., Aboba, B., Kelly, S. and V. Gupta, "Dynamic Host Configuration Protocol (DHCPv4) Configuration of IPsec Tunnel Mode", RFC 3456, January 2003.
- [34] Kembel, R., "The Fibre Channel Consultant: A Comprehensive Introduction", Northwest Learning Associates, 1998.
12. Acknowledgments 🫶
The developers of this specification thank Mr. Jim Nelson for his assistance with FC-FS related issues 🙌.
The developers of this specification express their appreciation to Mr. Mallikarjun Chadalapaka and Mr. David Black for their detailed and helpful reviews 💜.
Appendix A - Fibre Channel Bit and Byte Numbering Guidance 🌈
Both Fibre Channel and IETF standards use the same byte transmission order 🔁. However, the bit and byte numbering is different 🙃.
Fibre Channel bit and byte numbering can be observed if the data structure heading, shown in figure 11, is cut and pasted at the top of figure 7, figure 9, and figure 17 ✂️.
W|------------------------------Bit------------------------------|
o| |
r|3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1 |
d|1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0|Figure 11: Fibre Channel Data Structure Bit and Byte Numbering
Fibre Channel bit numbering for the pFlags field can be observed if the data structure heading, shown in figure 12, is cut and pasted at the top of figure 8 🔍.
|----------------Bit--------------------|
| |
| 31 30 29 28 27 26 25 24 |Figure 12: Fibre Channel pFlags Bit Numbering
Fibre Channel bit numbering for the Connection Usage Flags field can be observed if the data structure heading, shown in figure 13, is cut and pasted at the top of figure 10 🧩.
|------------------------------Bit------------------------------|
| |
| 31 30 29 28 27 26 25 24 |Figure 13: Fibre Channel Connection Usage Flags Bit Numbering
Appendix B - IANA Considerations 🧚
IANA has made the following port assignments to FCIP 🎟️:
fcip-port 3225/tcp FCIPfcip-port 3225/udp FCIP
IANA has changed the authority for these port allocations to reference this RFC 📌.
Use of UDP with FCIP is prohibited even though IANA has allocated a port 🚫.
The FC Frame encapsulation used by this specification employs Protocol# value 1, as described in the IANA Considerations appendix of the FC Frame Encapsulation [19] specification 💖.
Appendix C - FCIP Usage of Addresses and Identifiers ✨
Okay bestie, so here's the big FCIP secret 🤫: FCIP does not use IP Addresses to identify FCIP Entities or FCIP_LEPs 🙈. This is so network address translators can do their thing without FCIP freaking out 💅.
The only time IP Addresses get used for identification is when someone starts a new TCP connect request (see section 8.1.2.3). The destination IP Address of that request answers one question: "Have previous TCP connect requests been made to the same destination FCIP Entity?" 👀 Then, to double-check that guess, the Destination FC Fabric Entity World Wide Name goes in the FCIP Special Frame (FSF). The FCIP Entity that gets the TCP connect request and the FSF checks that value too (see section 8.1.3). Two checks, no drama 🫶.
When an FCIP Entity processes an incoming TCP connect request, it figures out who the source is from the FSF. The FSF goes from the TCP connect requestor to the TCP connect recipient, and its first bytes come right after the TCP connect request. The fields that matter are the Source FC Fabric Entity World Wide Name and the Source FC/FCIP Entity Identifier (see section 8.1.2.3 and section 8.1.3) 🧚.
[3] is where the definitions for each FSF field live, and these are the three you need to know 📖:
- Source FC Fabric Entity World Wide Name,
- Source FC/FCIP Entity Identifier, and
- Destination FC Fabric Entity World Wide Name.
As section 8.1.3 explains, FCIP Entities sort their FCIP_LEPs into two groups 🌸:
- Connections that come from TCP connect requests the FCIP Entity started itself, and
- Connections that come from TCP connect requests the FCIP Entity received.
Inside each group, the same three pieces of info are used to tell each FCIP_LEP apart 💜:
- Source FC Fabric Entity World Wide Name,
- Source FC/FCIP Entity Identifier, and
- Destination FC Fabric Entity World Wide Name.
Main character energy from the identifiers, no cap 👑.
Appendix D - Example of Synchronization Recovery Algorithm 🦋
The contents of this annex are informative. 🌈
Synchronization can be recovered the way section 5.6.2.3 describes 🔁. This annex gives an example of an algorithm that searches the bytes delivered to the Encapsulated Frame Receiver Portal for a valid FCIP Frame header. It's giving detective energy 🕵️♀️.
The big idea is that a valid FCIP data stream has to contain at least one valid header every 2176 bytes, which is the maximum length of an encapsulated FC Frame 📏. Other data patterns can look like valid headers, sure 🙃. But the FC CRC or FCIP Frame validity of those patterns in the data stream will always get interrupted by, or resynchronized with, the real valid FCIP Frame headers. Truth always wins 💅.
Picture figure 14. A bunch of short FCIP Frames, maybe from a trace, are tucked inside bigger FCIP Frames. That happens when a trace file gets moved from one disk to another 💾. The short FCIP Frame headers are labelled SFH, and the long FCIP Frame headers are labelled LFH 🎀.
+-+--+-+----+-+----+-+----+-+-+-+---+-+---
|L| |S| |S| |S| |S| |L| |S|
|F| |F| |F| |F| |F| |F| |F|...
|H| |H| |H| |H| |H| |H| |H|
+-+--+-+----+-+----+-+----+-+-+-+---+-+---
| |
|<---------2176 bytes-------->|Figure 14: Example of resynchronization data stream ✨
Here's the tea ☕. If a resynchronization attempt starts just to the right of an LFH, it'll run into a bunch of SFH FCIP Frames before it realises they don't match the transmitted stream of FCIP Frames 😭. Somewhere within 2176 bytes, plus or minus, it'll hit an SFH whose length doesn't line up with the next SFH. That happens because the LFH lands in the middle of the short FCIP Frame and shoves the next header further along the byte stream 🫠.
Now the important bit 🚨: the resynchronization algorithm cannot forward any prospective FC Frames to the FC Frame Transmitter Portal. Until synchronization is fully established, there's no certainty that anything that looked like an FCIP Frame actually was one 🙈. For example, an SFH could accidentally hold a length that points exactly at the start of an LFH. The LFH would show the correct start of a transmitted FCIP Frame, but that doesn't prove the SFH was a correct FCIP Frame header either. Suspicious vibes only 🧐.
Some data streams just can't be resynchronized by this algorithm 💀. If you run into one of those, the algorithm closes the TCP Connection. Bye bye 👋.
The resynchronization also assumes that security and authentication procedures outside the FCIP Entity are protecting the valid data stream. Those outside procedures keep an intruding data stream full of valid FCIP data from replacing the real one 🛡️. Security is literally the bodyguard here 💪.
Here's one example of how an FCIP_DE might resynchronize with the data stream entering the Encapsulated Frame Receiver Portal 🧚♀️.
1) Search for candidate and strong headers:
The data stream entering the Encapsulated Frame Receiver Portal is searched for 12 bytes in a row that contain the required values for:
- a) Protocol field,
- b) Version field,
- c) ones complement of the Protocol field,
- d) ones complement of the Version field,
- e) replication of encapsulation word 0 in word 1, and
- f) pFlags field and its ones complement.
If a grouping of 12 bytes like that shows up, the FCIP_DE assumes it has found bytes 0-2 of a candidate FCIP encapsulation header 🎯. Every byte up to and including the candidate header byte gets thrown out 🗑️.
If no candidate header turns up after searching a specified number of bytes that's greater than some multiple of 2176 (the maximum length of an FCIP Frame), resynchronization has failed. The TCP/IP connection gets closed 😤.
Word 3 of the candidate header holds the Frame Length and Flags fields, along with their ones complements. If those fields agree with their ones complements, the candidate header counts as a strong candidate header 💪. The Frame Length field then tells the algorithm where the next strong candidate header should be, and processing continues at step 2) 🔜.
2) Use multiple strong candidate headers to locate a verified candidate header:
The Frame Length in one strong candidate header is used to skip incoming bytes until the expected spot for the next strong candidate header shows up. Then the tests from step 1) get run to see if another strong candidate header has been found 🔍.
All skipped bytes, and all bytes in every strong candidate header that got processed, are discarded 🧹.
Strong candidate headers keep getting verified this way for at least 4352 bytes, which is twice the maximum length of an FCIP Frame ⏳. If a verification test fails at any point, processing restarts at step 1 and a retry counter goes up by one 📈. If the retry counter goes over 3 retries, resynchronization has failed. The TCP Connection gets closed, and the FC entity is told why 📣.
Once strong candidate headers have been verified for at least 4352 bytes, the next header identified is a verified candidate header, and processing continues at step 3) 🎉.
Note: If a strong candidate header was part of the data content of an FCIP Frame, the FCIP Frame defined by that or a subsequent strong candidate header will eventually cross an actual header in the byte stream. As a result, it will either identify the actual header as a strong candidate header, or it will lose synchronization again because of the extra 28 bytes in the length, returning to step 1 as described above. 🔄
3) Use multiple strong candidate headers to locate a verified candidate header:
Incoming bytes get inspected and discarded until the next verified candidate header shows up 👀. Inspecting those bytes includes testing for other candidate headers using the criteria from step 1. Each verified candidate header gets tested against the tests listed in section 5.6.2.2, just like normal 📋.
Verified candidate headers keep being located and tested this way for a minimum of 4352 bytes, which is twice the maximum length of an FCIP Frame 🌀. If every verified candidate header encountered is valid, the last verified candidate header is a valid header 🥳. At that point, the FCIP_DE stops discarding bytes and starts normal FCIP de-encapsulation. For the first time since synchronization was lost, FC Frames get delivered through the FC Frame Transmitter Portal according to normal FCIP rules 🚀.
If any verified candidate headers are invalid but still meet all the requirements of a strong candidate header, increment the retry counter and return to step 2) 🔁. If any verified candidate headers are invalid and fail the tests for a strong candidate header, or if inspecting the bytes between verified candidate headers finds any candidate headers, increment the retry counter and return to step 1 😵💫. If the retry counter goes over 4 retries, resynchronization has failed and the TCP/IP connection is closed 💔.
A flowchart for this algorithm is in figure 15 🗺️.
Synchronization is lost
|
_____________v_______________
| |
| Search for candidate header |
+----------->| |
| | Found Not Found |
| | (Strong candidate) |
| |_____________________________|
| | |
| | + --------->close TCP
| _______v_____________________ Connection
| | | and notify
| | Enough strong candidate | the FC Entity
| +---->| headers identified? | with the reason
| | | | for closure
| | | No Yes |
| | | (Verified candidate) |
| | |_____________________________|
|___________________| |
^ | |
| | |
| | _______________________v_____
| | | |
| | | Enough verified candidate |
| | | headers validated? |
| | | |
| | | No Yes |
| | | (Resynchronized) |
| | |_____________________________|
| | | |
| | ______v__________ | Resume
| | | | + ---> Normal
| | | Synchronization | De-encapsulation
| | | Lost? |
| | | |
| | | No Yes |
| | |_________________|
| | | |
| |________| |
|___________________________|Figure 15: Flow diagram of simple synchronization example 🌟
Appendix E - Relationship between FCIP and IP over FC (IPFC) 🦋
This annex is informative only, bestie, so it's here to help you vibe with the bigger picture 💅
IPFC (RFC 2625) describes how IP packets get encapsulated in FC Frames. The whole point is to make IP communication over an FC network easier 🌸
FCIP is a different beast. It describes how FC Frames get encapsulated in TCP segments, and those segments then get wrapped in IP packets so they can travel over an IP network. FCIP doesn't care what kind of FC Frame it's carrying 🤷♀️ So the FC Frame might actually contain an IP packet, the kind described in the IP over FC specification (RFC 2625). When that happens, the data packet ends up looking like this:
- Data Link Header
- IP Header
- TCP Header
- FCIP Header
- FC Header
- IP Header
Note: The two IP headers are not identical. One has info about the final destination, and the other has info about the FCIP Entity. Two different addresses, one packet, lots of drama 🎀
The two documents have different goals. Like we said above, running FCIP means IP gets encapsulated inside IP. That's a bit inefficient, but it shouldn't cause problems for IP communication 🧚 There's one caveat, though. Say a Fibre Channel device is wrapping IP packets in an FC Frame (for example, an IPFC device), and it's talking to a device running IP over a non-FC medium. In that case, a second IPFC device may need to act as a gateway between the two networks. FCIP doesn't specifically address this scenario 🙈
Nothing in either specification stops one device from implementing both FCIP and IP-over-FC (IPFC). Still, that's an implementation-specific choice and it's outside the scope of this document ✨
Appendix F - FC Frame Format 💜
Note: Whenever this section says "character" or "characters," it's talking about 8bit/10bit link encoding. Each 8 bit "character" in a link frame gets encoded as a 10 bit "character" for transmission on the link. These words have nothing to do with ASCII, Unicode, or any other text characters, even though octets from those characters will show up as 8 bit "characters" in this encoding. This usage matches the ANSI T11 standards that specify Fibre Channel, so we're keeping it consistent 🤓
The contents of this annex are informative.
All FC Frames follow a standard format (see FC-FS [5]), kinda like LAN's 802.x protocols. The exact size of each FC Frame changes depending on how big its variable fields are. The variable field ranges from 0 to 2112-bytes, as shown in the FC Frame Format in figure 16. That means the smallest FC Frame is 36 bytes and the biggest is 2148 bytes. Valid FC Frame lengths are always a multiple of four bytes 📏
+------+--------+-----------+----//-------+------+------+
| SOF |Frame |Optional | Frame | CRC | EOF |
| (4B) |Header |Header | Payload | (4B) | (4B) |
| |(24B) |<----------------------->| | |
| | | Data Field = (0-2112B) | | |
+------+--------+-----------+----//-------+------+------+Figure 16: FC Frame Format
SOF and EOF Delimiters
On an FC link, Start-of-Frame (SOF) and End-Of-Frame (EOF) are called Ordered Sets. They're sent as special words built from the 8B/10B comma character (K28.5), followed by three additional 8B/10B data characters, which makes them uniquely identifiable in the data stream 🔥
On an FC link, the SOF delimiter marks the beginning of an FC Frame and gets the receiver ready for it. The SOF carries info about the FC Frame's Class of Service, its position within a sequence, and, in some cases, connection status 💖
The EOF delimiter marks the end of the FC Frame and the final FC Frame of a sequence. It also forces the running disparity to negative. The EOF is used to end the connection in connection-oriented classes of service 🌈
There's a special one called EOFa (End Of Frame - Abort). It's used to terminate a partial FC Frame that resulted from a malfunction in a link facility during transmission. An FCIP Entity acts like a transmission link as far as the rest of the FC Fabric is concerned, so FCIP_DEs may use EOFa in their error recovery procedures 🛠️
So it's really important that the delimiters' info survives the trip across the IP-based network. That way, the receiving FCIP Entity can rebuild the FC Frame in its original SOF and EOF format before forwarding it to its final FC destination on the FC link 🎯
When an FC Frame gets encapsulated and sent over a byte-oriented interface, the SOF and EOF delimiters are represented as sequences of four consecutive bytes. Those bytes carry the equivalent Class of Service and FC Frame termination info as the FC ordered sets 🦋
The representation of SOF and EOF in an encapsulation FC Frame is described in FC Frame Encapsulation [19].
Frame Header
The FC Frame Header is transparent to the FCIP Entity. It's 24 bytes long and has several fields tied to identifying and controlling the payload. Current FC Standards allow up to 3 Optional Header fields [5]:
Network_Header(16-bytes)Association_Header(32-bytes)Device_Header(up to 64-bytes)
Frame Payload
The FC Frame Payload is transparent to the FCIP Entity too 🫶 An FC application level payload is called an Information Unit at the FC-4 Level, and it gets mapped into the FC Frame Payload. A large Information Unit is segmented using a structure made of FC Sequences. A Sequence usually consists of more than one FC Frame. FCIP doesn't keep any state info about how FC Frames relate to each other within an FC Sequence 🧩
CRC
The FC CRC is 4 bytes long. It uses the same 32-bit polynomial used in FDDI and is specified in ANSI X3.139 Fiber Distributed Data Interface. This CRC value is calculated over the entire FC header and the FC payload. It does not include the SOF and EOF delimiters 🔐
Note: When FC Frames get encapsulated into FCIP Frames, the FC Frame CRC is left completely untouched by the FCIP Entity 💅
Appendix G - FC Encapsulation Format 🎀
This annex reproduces the FC Encapsulation Format [19] as it applies to FCIP Frames that encapsulate FC Frames. The info here is not meant to represent the FCIP Special Frame (FSF) described in section 7 ✨
The info in this annex was correct when this specification was approved, and it's informative only 🌸
If anything here differs from the FC Encapsulation Format specification [19], the FC Encapsulation Format specification wins 👑
If anything here differs from the contents of section 5.6.1, then section 5.6.1 wins 👑
Figure 17 applies the requirements from section 5.6.1, along with the FC Encapsulation Frame format, to give a summary of the FC Frame format. Where FCIP needs specific values, those values appear in hexadecimal in parentheses. The detailed requirements for the FCIP usage of the FC Encapsulation Format are in section 5.6.1 📖
W|------------------------------Bit------------------------------|
o| |
r| 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3|
d|0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1|
+---------------+---------------+---------------+---------------+
0| Protocol# | Version | -Protocol# | -Version |
| (0x01) | (0x01) | (0xFE) | (0xFE) |
+---------------+---------------+---------------+---------------+
1| Protocol# | Version | -Protocol# | -Version |
| (0x01) | (0x01) | (0xFE) | (0xFE) |
+---------------+---------------+---------------+---------------+
2| pFlags | Reserved | -pFlags | -Reserved |
| (0x00) | (0x00) | (0xFF) | (0xFF) |
+-----------+---+---------------+-----------+---+---------------+
3| Flags | Frame Length | -Flags | -Frame Length |
| (0x00) | | (0x3F) | |
+-----------+-------------------+-----------+-------------------+
4| Time Stamp [integer] |
+---------------------------------------------------------------+
5| Time Stamp [fraction] |
+---------------------------------------------------------------+
6| CRC (Reserved in FCIP) |
| (0x00-00-00-00) |
+---------------+---------------+---------------+---------------+
7| SOF | SOF | -SOF | -SOF |
+---------------+---------------+---------------+---------------+
8| |
+----- FC Frame content (see appendix F) -----+
| |
+---------------+---------------+---------------+---------------+
n| EOF | EOF | -EOF | -EOF |
+---------------+---------------+---------------+---------------+Figure 17: FCIP Frame Format
The field names are generally descriptive of their contents, and the FC Encapsulation Format specification [19] is referenced for details. Field names with a minus sign in front are ones complement values of the named field 🧠
Note: Figure 17 does not represent the FSF described in section 7.
Appendix H - FCIP Requirements on an FC Entity 👑
The contents of this annex are informative for FCIP, but they might be considered normative on FC-BB-2. Lowkey important stuff 🥹
The capabilities that FCIP needs from an FC Entity include:
- The FC Entity must deliver FC Frames to the correct FCIP Data Engine (in the correct FCIP Link Endpoint).
- Each FC Frame delivered to an FCIP_DE must be accompanied by a time value synchronized with the clock maintained by the FC Entity at the other end of the FCIP Link (see section 6). If a synchronized time value is not available, a value of zero must accompany the FC Frame ⏰
- When FC Frames exit FCIP Data Engine(s) via the FC Frame Transmitter Portal(s), the FC Entity should forward them to the FC Fabric. However, before forwarding an FC Frame, the FC Entity must compute the end-to-end transit time for the FC Frame using the time value supplied by the FCIP_DE (taken from the FCIP header) and a synchronized time value (see section 6). If the end-to-end transit time exceeds the requirements of the FC Fabric, the FC Entity is responsible for discarding the FC Frame 🗑️
- The only delivery ordering guarantee FCIP provides is correctly ordered delivery of FC Frames between a pair of FCIP Data Engines. FCIP expects the FC Entity to implement all other FC Frame delivery ordering requirements 📋
- When a TCP connect request is received and that request would add a new TCP Connection to an existing FCIP_LEP, the FC Entity must authenticate the source of the TCP connect request before use of the new TCP connection is allowed. No skipping this one, bestie 🔒
- The FC Entity may help decide which TCP Connections are allowed, what TCP Connection parameters get used, and how quality of service and security get used. It does this by changing how it interacts with the FCIP Entity, which is modelled as a "shared" database in section 8.1.1. 💅
- The FC Entity may ask the FCIP Entity to perform TCP close requests. 🎀
- The FC Entity may recover from connection failures. 🦋
- The FC Entity must recover from events that the FCIP Entity cannot handle, such as: a. losing synchronization with FCIP Frame headers coming from the Encapsulated Frame Receiver Portal, which requires resetting the TCP Connection; and b. recovering from FCIP Frames that get discarded because of synchronization problems (see section 5.6.2.2 and section 5.6.2.3). 😭
- The FC Entity must work cooperatively with the FCIP Entity to manage flow control problems in either the IP Network or FC Fabric. 🫶
- The FC Entity may test for failed TCP Connections. 🔍
Note that the Fibre Channel standards must be consulted for a complete understanding of the requirements placed on an FC Entity. 📚
Okay, so here's the table breakdown bestie 🌸 Table 2 shows the explicit interactions between the FCIP Entity and the FC Entity, and it's honestly so organized. ✨
+-------------+-----------------+-----------------------------------+
| | | Information/Parameter Passed and |
| | | Direction |
| Reference | +-----------------+-----------------+
| Section | Condition | FCIP Entity---> | <---FC Entity |
+-------------+-----------------+-----------------+-----------------+
| 5.6 | FC Frame ready | | Provide FC |
| FCIP Data | for IP transfer | | Frame and |
| Engine | | | time stamp at |
| | | | FC Frame |
| | | | Receiver Portal |
+-------------+-----------------+-----------------+-----------------+
| WWN = World Wide Name |
+-------------------------------------------------------------------+
| continued |
+-------------------------------------------------------------------+
Table 2: FC/FCIP Entity pair interactions (part 1 of 5)
+-------------+-----------------+-----------------------------------+
| | | Information/Parameter Passed and |
| | | Direction |
| Reference | +-----------------+-----------------+
| Section | Condition | FCIP Entity---> | <---FC Entity |
+-------------+-----------------+-----------------+-----------------+
| continued |
+-------------+-----------------+-----------------+-----------------+
| 5.6 | FCIP Frame | Provide FC | |
| FCIP Data | received from | Frame and | |
| Engine | IP Network | time stamp at | |
| | | FC Frame Trans- | |
| | | mitter Portal | |
+-------------+-----------------+-----------------+-----------------+
| 5.6.2.2 | FCIP_DE | Inform FC | |
| Errors | discards bytes | Entity that | |
| in FCIP | delivered | bytes have been | |
| Headers and | through | discarded with | |
| Discarding | Encapsulated | reason | |
| FCIP Frames | Frame Receiver | | |
| | Portal | | |
+-------------+-----------------+-----------------+-----------------+
| 5.6.2.3 | FCIP Entity | Inform FC | |
| Synchron- | closes TCP | Entity that TCP | |
| ization | Connection due | Connection has | |
| Failures | to synchron- | been closed | |
| | ization failure | with reason | |
| | | for closure | |
+-------------+-----------------+-----------------+-----------------+
| 8.1.2.3 | Receipt of the | Inform FC | |
| Connection | echoed FSF | Entity that TCP | |
| Setup | takes too long | Connection has | |
| Following a | or the FSF | been closed | |
| Successful | contents have | with reason | |
| TCP Connect | changed | for closure | |
| Request | | | |
+-------------+-----------------+-----------------+-----------------+
| WWN = World Wide Name |
+-------------------------------------------------------------------+
| continued |
+-------------------------------------------------------------------+
Table 2: FC/FCIP Entity pair interactions (part 2 of 5)
+-------------+-----------------+-----------------------------------+
| | | Information/Parameter Passed and |
| | | Direction |
| Reference | +-----------------+-----------------+
| Section | Condition | FCIP Entity---> | <---FC Entity |
+-------------+-----------------+-----------------+-----------------+
| continued |
+-------------+-----------------+-----------------+-----------------+
| 8.1.2.1 | New TCP | Inform FC | |
| Non-Dynamic | Connection | Entity of | |
| Creation of | created based | new or existing | |
| a New TCP | on "shared" | FCIP_LEP and | |
| Connections | database | new FCIP_DE | |
| | information | along with | |
| | | Destination FC | |
| | | Fabric Entity | |
| | | WWN, Connection | |
| | | Usage Flags, | |
| | | Connection | |
| | | Usage Code and | |
| | | Connection | |
| | | Nonce | |
+-------------+-----------------+-----------------+-----------------+
| 8.1.2.2 | New TCP | Inform FC | |
| Dynamic | Connection | Entity of | |
| Creation of | created based | new or existing | |
| a New TCP | on SLP service | FCIP_LEP and | |
| Connections | advertisement | new FCIP_DE | |
| | and "shared" | along with | |
| | database | Destination FC | |
| | information | Fabric Entity | |
| | | WWN, Connection | |
| | | Usage Flags, | |
| | | Connection | |
| | | Usage Code and | |
| | | Connection | |
| | | Nonce | |
+-------------+-----------------+-----------------+-----------------+
| WWN = World Wide Name |
+-------------------------------------------------------------------+
| continued |
+-------------------------------------------------------------------+
Table 2: FC/FCIP Entity pair interactions (part 3 of 5)
+-------------+-----------------+-----------------+-----------------+
| continued |
+-------------+-----------------+-----------------+-----------------+
| 8.1.3 | New TCP | Inform FC | |
| Processing | Connection | Entity of | |
| Incoming | created based | new or existing | |
| TCP Connect | on incoming TCP | FCIP_LEP and | |
| Requests | Connect request | new FCIP_DE | |
| | and "shared" | along with | |
| | database | Source FC | |
| | information | Fabric Entity | |
| | | WWN, Source | |
| | | FC/FCIP Entity | |
| | | Identifier, | |
| | | Connection | |
| | | Usage Flags, | |
| | | Connection | |
| | | Usage Code and | |
| | | Connection | |
| | | Nonce | |
+-------------+-----------------+-----------------+-----------------+
| 8.1.3 | TCP Connect | Request FC | Yes or No |
| Processing | Request wants | Entity to | answer about |
| Incoming | to add a new | authenticate | whether the |
| TCP Connect | TCP Connection | the source of | source of the |
| Requests | to an existing | the TCP Connect | TCP Connect |
| | FCIP_LEP | Request | Request can be |
| | | | authenticated |
+-------------+-----------------+-----------------+-----------------+
| 8.1.3 | Receipt of the | Inform FC | |
| Processing | FSF takes too | Entity that TCP | |
| Incoming | long or | Connection has | |
| TCP Connect | duplicate | been closed | |
| Requests | Connection | with reason | |
| | Nonce value | for closure | |
+-------------+-----------------+-----------------+-----------------+
| WWN = World Wide Name |
+-------------------------------------------------------------------+
| continued |
+-------------------------------------------------------------------+ Table 2: FC/FCIP Entity pair interactions (part 4 of 5)
+-------------+-----------------+-----------------------------------+
| | | Information/Parameter Passed and |
| | | Direction |
| Reference | +-----------------+-----------------+
| Section | Condition | FCIP Entity---> | <---FC Entity |
+-------------+-----------------+-----------------+-----------------+
| concluded |
+-------------+-----------------+-----------------+-----------------+
| 8.2 | FC Entity | Acknowledgement | Identification |
| Closing TCP | determines | of TCP | of the FCIP_DE |
| Connections | that a TCP | Connection | whose TCP |
| | Connection | closure | Connection |
| | needs to be | | needs to be |
| | closed | | closed |
+-------------+-----------------+-----------------+-----------------+
| 8.4 | Discovery that | Inform FC | |
| TCP | TCP connectiv- | Entity that TCP | |
| Connection | ity has been | Connection has | |
| Considera- | lost | been closed | |
| tions | | with reason | |
| | | for closure | |
+-------------+-----------------+-----------------+-----------------+
| 9.4.1 | IKE phase 1 | Inform FC | |
| FCIP | failed, result- | Entity that TCP | |
| Link | ing in termin- | Connection can | |
| Initializ- | ation of link | not be opened | |
| ation Steps | initialization | with reason for | |
| | | failure | |
+-------------+-----------------+-----------------+-----------------+
| 9.4.3 | Excessive | Inform FC | |
| Handling | numbers of | Entity that TCP | |
| data | dropped | Connection has | |
| integrity | datagrams | been closed | |
| and confi- | detected and | with reason | |
| dentiality | TCP Connection | for closure | |
| violations | closed | | |
+-------------+-----------------+-----------------+-----------------+
| RFC 3723 | TCP Connection | Inform FC | |
| | closed due to | Entity that TCP | |
| Handling SA | SA parameter | Connection has | |
| parameter | mismatch | been closed | |
| mismatches | problems | with reason | |
| | | for closure | |
+-------------+-----------------+-----------------+-----------------+
| WWN = World Wide Name |
+-------------------------------------------------------------------+
Table 2: FC/FCIP Entity pair interactions (part 5 of 5)Editors and Contributors Acknowledgements 🫶
Okay, the people who made this spec happen! 💖 Murali Rajagopal, Elizabeth Rodriguez, Vi Chau, and Ralph Weber took turns being the editors, one after another. 👑 Raj Bhagwat did a ton of work on the very first basic FCIP concepts, so he's basically the OG. ✨
Venkat Rangan wrote the Security section and is still keeping an eye on security stuff with the ips Working Group and the IETF. 🔐 Andy Helland gave the Performance section a big glow-up and lined it up with TCP/IP QoS concepts. 💅 Dave Peterson brought in the dynamic discovery section and did edits on RFC 3822. 🦋
Anil Rijhsinghani added material for the FCIP MIB and is the one who edits the FCIP MIB document. 📚 Bob Snively contributed error detection and recovery material, including most of the synchronization recovery example annex. 🧚 Lawrence J. Lamers had tons of ideas for keeping FCIP compatible with B_Port devices. 🌸 Milan Merhar contributed several of the FCIP conceptual changes needed to support NATs. 🌈 Don Fraser added material about link failure detection and reporting. 🔥 Bill Krieg restructured the TCP Connection setup sections so they flow more linearly over time and are way easier to read. 🍓
Several T11 leaders backed this effort and advised the editors on coordinating with T11 documents and projects. 🫶 They are: Jim Nelson (Framing and Signaling), Neil Wanamaker (Framing and Signaling), Craig Carlson (Generic Services), Ken Hirata (Switch Fabric), Murali Rajagopal (Backbone), Steve Wilson (Switch Fabric), and Michael O'Donnell (Security Protocols). ⭐
Editors and Contributors Addresses 📬
Here's where everyone can be reached, exactly as listed in the original. 💜
Neil Wanamaker
Akara
10624 Icarus Court
Austin, TX 78726
USA
Phone: +1 512 257 7633
Fax: +1 512 257 7877
EMail: nwanamaker@akara.com
Ralph Weber
ENDL Texas, representing Brocade
Suite 102 PMB 178
18484 Preston Road
Dallas, TX 75252
USA
Phone: +1 214 912 1373
EMail: roweber@ieee.org
Elizabeth G. Rodriguez
Dot Hill Systems Corp.
6305 El Camino Real
Carlsbad, CA 92009
USA
Phone: +1 760 431 4435
EMail: elizabeth.rodriguez@dothill.com
Steve Wilson
Brocade Comm. Systems, Inc.
1745 Technology Drive
San Jose, CA. 95110
USA
Phone: +1 408 333 8128
EMail: swilson@brocade.com
Bob Snively
Brocade Comm. Systems, Inc.
1745 Technology Drive
San Jose, CA 95110
USA
Phone: +1 408 303 8135
EMail: rsnively@brocade.com
David Peterson
Cisco Systems - SRBU
6450 Wedgwood Road
Maple Grove, MN 55311
USA
Phone: +1 763 398 1007
Cell: +1 612 802 3299
EMail: dap@cisco.com
Donald R. Fraser
Hewlett-Packard
301 Rockrimmon Blvd., Bldg. 5
Colorado Springs, CO 80919
USA
Phone: +1 719 548 3272
EMail: Don.Fraser@HP.com
R. Andy Helland
LightSand Communications, Inc.
375 Los Coches Street
Milpitas, CA 95035
USA
Phone: +1 408 404 3119
Fax: +1 408 941 2166
EMail: andyh@lightsand.com
Raj Bhagwat
LightSand Communications, Inc.
24411 Ridge Route Dr.
Suite 135
Laguna Hills, CA 92653
USA
Phone: +1 949 837 1733 x104
EMail: rajb@lightsand.com
Bill Krieg
Lucent Technologies
200 Lucent Lane
Cary, NC 27511
USA
Phone: +1 919 463 4020
Fax: +1 919 463 4041
EMail: bkrieg@lucent.com
Michael E. O'Donnell
McDATA Corporation
310 Interlocken Parkway
Broomfield, CO 80021
USA
Phone: +1 303 460 4142
Fax: +1 303 465 4996
EMail: modonnell@mcdata.com
Anil Rijhsinghani
McDATA Corporation
310 Interlocken Parkway
Broomfield, CO 80021
USA
Phone: +1 508 870 6593
EMail: anil.rijhsinghani@mcdata.com
Milan J. Merhar
43 Nagog Park
Pirus Networks
Acton, MA 01720
USA
Phone: +1 978 206 9124
EMail: Milan@pirus.com
Craig W. Carlson
QLogic Corporation
6321 Bury Drive
Eden Prairie, MN 55346
USA
Phone: +1 952 932 4064
EMail: craig.carlson@qlogic.com
Venkat Rangan
Rhapsody Networks Inc.
3450 W. Warren Ave.
Fremont, CA 94538
USA
Phone: +1 510 743 3018
Fax: +1 510 687 0136
EMail: venkat@rhapsodynetworks.com
Lawrence J. Lamers
SAN Valley Systems, Inc.
6320 San Ignacio Ave.
San Jose, CA 95119-1209
USA
Phone: +1 408 234 0071
EMail: ljlamers@ieee.org
Murali Rajagopal
Broadcom Corporation
16215 Alton Parkway
Irvine,CA 92619
USA
Phone: +1 949 450 8700
EMail: muralir@broadcom.com
Ken Hirata
Vixel Corporation
15245 Alton Parkway, Suite 100
Irvine, CA 92618
USA
Phone: +1 949 788 6368
Fax: +1 949 753 9500
EMail: ken.hirata@vixel.com
Vi Chau
USA
Email: vchau1@cox.netFull Copyright Statement ©
The legal bit, but make it cute. 🎀 Copyright (C) The Internet Society (2004). This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights. 🦋
This document and the information in it are provided on an "AS IS" basis. The contributor, the organization they represent or are sponsored by (if any), the Internet Society, and the Internet Engineering Task Force all disclaim every warranty, express or implied. That includes, but isn't limited to, any warranty that using the information won't infringe any rights, and any implied warranties of merchantability or fitness for a particular purpose. 🙈
Intellectual Property 🌸
The IETF doesn't take a position on whether any Intellectual Property Rights or other rights claimed to cover the technology in this document are valid or in scope. It also doesn't say whether licenses under those rights are available, and it hasn't independently tried to find any such rights. ✨ You can find info about how rights work for RFC documents in BCP 78 and BCP 79. 📖
Copies of IPR disclosures made to the IETF Secretariat, plus any assurances of licenses, or the results of an attempt to get a general license or permission to use those proprietary rights, can be found in the IETF online IPR repository at http://www.ietf.org/ipr. 🌐
The IETF invites anyone to bring to its attention any copyrights, patents, patent applications, or other proprietary rights that might cover technology needed to implement this standard. 🫶 Send that info to the IETF at ietf-ipr@ietf.org. 💌
Acknowledgement 🌈
Funding for the RFC Editor function is currently provided by the Internet Society. 💜
📦 where this page comes from
- ietf: rfc3821 2004-07
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms