RFC 1374
IP and ARP on HIPPI
RFC 1374: IP and ARP on HIPPI ✨
Part 1 of 9 💖
Okay bestie, so this is RFC 1374 🎀 It came from the Network Working Group and was written by J. Renwick and A. Nicholson at Cray Research, Inc. It's dated October 1992, and it's an IAB standards track protocol 👑 The header doesn't list anything it obsoletes or updates, so nothing to untangle there 💅
Status of this Memo 📜
ngl this part is the official fine print, but it matters. This RFC specifies an IAB standards track protocol for the Internet community, and it asks for discussion and suggestions for improvements 💬 For the standardization state and status of this protocol, check the current edition of the "IAB Official Protocol Standards." Distribution of this memo is unlimited 🌈
Abstract 🦋
So the ANSI X3T9.3 committee drafted a proposal for encapsulating IEEE 802.2 LLC PDUs, and by extension IP, on HIPPI. Another X3T9.3 draft describes how HIPPI physical switches work 🔌 X3T9.3 mostly left HIPPI networking issues outside their standards, so this document covers how to use ANSI standard HIPPI hardware and protocols on the Internet, including using HIPPI switches as LANs and talking to other networks 🌸
Table of Contents 🗂️
Introduction 2
Scope 2
Definitions 3
Equipment 4
Protocol 6
Packet Format 6
48 bit Universal LAN MAC addresses 10
I-Field Format 11
Rules For Connections 13
MTU 15
Camp-on 16
Address Resolution 16
ARP and RARP Message Format 17
ARP Procedure 21
ARP Implementation Methods 22
ARP Example 23
Discovery of One's Own Switch Address 25
Path MTU Discovery 27
Channel Data Rate Discovery 27
Performance 29
Sharing the Switch 31
Appendix A -- HIPPI Basics 31
Appendix B -- How to Build a Practical HIPPI LAN 37
References 41
Security Considerations 42
Authors' Addresses 42Introduction 🌸
Let's get into it! The ANSI High-Performance Parallel Interface (HIPPI) is a simplex data channel. When it's set up in pairs, HIPPI can send and receive data simultaneously at nearly 800 megabits per second 🔥 There's also an equally applicable 1600 megabit/second option. Between 1987 and 1991, the ANSI X3T9.3 HIPPI working group drafted four documents that matter for using HIPPI as a network interface 📚 They cover the physical and electrical specification (HIPPI-PH [1]), the framing of a stream of octets (HIPPI-FP [2]), encapsulation of IEEE 802.2 LLC (HIPPI-LE [3]), and how a standard physical layer switch behaves (HIPPI-SC [4]). HIPPI-LE also implies encapsulation of Internet Protocol[5] 🧚
You should be familiar with the ANSI HIPPI documents. Copies are archived at the site nsco.network.com in the directory hippi, and you can grab them via anonymous FTP until they become published standards 💻
HIPPI switches can connect a whole bunch of computers and peripheral equipment for lots of purposes, but the working group stopped short of describing them as Local Area Networks. This memo picks up right where they left off 🦋 The guiding principle is that, except for length and hardware header, Internet datagrams sent on HIPPI SHOULD be identical to the same datagrams sent on a conventional network. Any datagram sent on a conventional 802 network[6] SHOULD be valid on HIPPI too. It's giving consistency 💜
Scope 🎀
This memo describes the HIPPI interface between a host and a crosspoint switch that complies with the HIPPI-SC draft standard. Issues with no impact on host implementations are outside the scope of this memo ✨ Host implementations that comply with this memo are believed to be interoperable on a network made of a single HIPPI-SC switch. They're also interoperable on a simple point-to-point, two-way HIPPI connection with no switch between them.
They may as well be interoperable on more complex networks too, depending on the internals of the switches and how they're interconnected. However, those details are implementation dependent and outside the scope of this memo 🙈 And to the extent that a gateway acts as a host on a HIPPI-SC LAN, its behavior is within the scope of this memo.
Within the scope of this memo are:
- Packet format and header contents, including HIPPI-FP, HIPPI-LE, IEEE 802.2 LLC[7], SNAP and ARP
- I-Field contents
- HIPPI switch address resolution, including self discovery
- Rules for the use of connections.
Outside of the scope are:
- Vendor dependent solutions for multicast or third party ARP
- Network configuration and management
- Host internal optimizations
- The interface between a host and an outboard protocol processor.
Definitions 📖
Conventional Used with respect to networks, this refers to Ethernet, FDDI and 802 LAN types, as distinct from HIPPI-SC LANs.
Destination The HIPPI implementation that receives data from a HIPPI Source.
Node An entity made of one HIPPI Source/Destination pair. It's connected by parallel or serial HIPPI to a HIPPI-SC switch and transmits and receives ARP and IP datagrams. A node can be an Internet host, bridge, router or gateway. This memo uses the term node instead of the usual "host," because a host might be connected to the HIPPI LAN not directly, but through an external adaptor that does some of the protocol processing for the host 🫶
Serial HIPPI An implementation of HIPPI in serial fashion on coaxial cable or optical fiber, informally standardized by implementor's agreement in the Spring of 1991 🌈
Switch Address A value used as the address of a node on a HIPPI-SC network. It's transmitted in the I-field. HIPPI-SC switches may map Switch Addresses to physical port numbers 🗺️
Source The HIPPI implementation that generates data to send to a HIPPI Destination.
Universal LAN Address (ULA) A 48 bit globally unique address, administered by the IEEE and assigned to each node on an Ethernet, FDDI, 802 network or HIPPI-SC LAN 🪪
Equipment 🔌
A HIPPI network can be made of nodes with HIPPI interfaces, HIPPI cables or serial links, HIPPI-SC switches, gateways to other networks and, possibly, proprietary equipment that multicasts or responds to ARP requests on behalf of the real nodes 🧚
Each HIPPI interconnection between a node and a switch SHALL consist of a pair of HIPPI links, one in each direction. 🔁
Here's the bit that's giving main character energy: if a link between a node and the switch is capable of the 1600 Megabit/second data rate option (i.e. Cable B installed for 64 bit wide operation) in either direction, the node's HIPPI-PH implementation SHALL also be capable of 32 bit operation (Cable B data suppressed) and SHALL be able to select or deselect the 1600Mb/s data rate option at the establishment of each new connection 👑
The following figure shows a sample HIPPI switch configuration 🦋
+-----+
| | H 4 |
| +--+--+
| +----+ +----+ +----+ |
| | H1 | | H2 | | H3 | +-++
| +--+ +-++-+ +-++-+ +-++-+ |PP|
+---+H5| || || || ++++
| +--+ || || || ||
| +---++--------++--------++------++----+
| | | +---+
| +----+ | HIPPI-SC +----+ARP|
+---+ G1 +--------+ +----+ |
| | +--------+ Switch | +---+
| +----+ | |
| +---++--------++--------++------++----+
| +--+ || || || ||
+---+H6| || ++++
| +--+ +-++-+ |PP|
| | | +-++
| | G2 | |
| | | +--+--+
| +--+-+ | H 7 |
| | +-----+
|
-----+------------+-------+-----------+-------------+------
| | | |
| | | |
+--+--+ +--+--+ +--+--+ +--+--+
| H 8 | | H 9 | | H10 | | H11 |
+-----+ +-----+ +-----+ +-----+
Legend: ---+---+---+-- = 802 network, Ethernet or FDDI
|| = Paired HIPPI link
H = Host computer
PP = Outboard Protocol Processor
G = Gateway
ARP = ARP Agent
A possible HIPPI configurationA single HIPPI-SC switch is "non-blocking," which means there's always a path available from any Source to any Destination 🛤️ If the network has more than one switch, the path from a Source to a Destination may include a HIPPI link between switches. If that link is shared by more than one Source/Destination pair, you get a "blocking" network. Now one Source might be blocked from reaching a Destination because another Source is using the link they share 😭 Connection strategies can get more complicated on blocking networks than on non-blocking ones.
This memo ignores blocking issues. It assumes the HIPPI LAN is made of one HIPPI-SC switch or, if the network is more complex than that, that it doesn't present any extra problems a node needs to be aware of 🙈
Part 2 drops next, so stay tuned, bestie 💖🌈✨
Protocol 💅
okay so this is where the actual rules live, bestie 🎀 everything from here on is the spec part, so we read it closely 👀
Packet Format ✨
okay so the HIPPI packet format for Internet datagrams SHALL conform to the HIPPI-FP and HIPPI-LE draft standards 💖 the HIPPI-FP D1_Area SHALL contain the HIPPI-LE header. the HIPPI-FP D2_Area, when present, SHALL contain one IEEE 802.2 Type 1 LLC Unnumbered Information (UI) PDU. support for IEEE 802.2 XID, TEST and Type 2 PDUs is not REQUIRED on HIPPI, and Destinations that receive these PDUs MAY either ignore them or respond correctly according to IEEE 802.2 requirements 🦋
the length of a HIPPI packet, including trailing fill, SHALL be a multiple of eight octets as REQUIRED by HIPPI-LE 🌸
+----------+-----------+---------------------+----------- ------+
| | | | IP . . . 0 - 7 |
| HIPPI-FP | HIPPI-LE | IEEE 802.2 LLC/SNAP | octets|
|(8 octets)|(24 octets)| (8 octets) | ARP . . . fill |
+----------+-----------+---------------------+----------- ------+
HIPPI Packet StructureHIPPI-FP Header 🩷
ULP-id(8 bits) SHALL contain4.D1_Data_Set_Present(1 bit) SHALL be set.Start_D2_on_Burst_Boundary(1 bit) SHALL be zero.Reserved(11 bits) SHALL contain zero.D1_Area_Size(8 bits) SHOULD be sent as3. Destinations SHALL accept any value that HIPPI-FP defines as legal: from3to127(32 bit HIPPI) or3to255(64 bit HIPPI).D2_Offset(3 bits) MAY be any value from0to7.D2_Size(32 bits) SHALL contain the number of octets in the IEEE 802.2 LLC Type 1 PDU, or zero if no PDU is present. It SHALL NOT exceed65,288(decimal). This value includes the IEEE 802.2 LLC/SNAP header and the IP datagram. It does not include trailing fill octets. (See "MTU," below.)
the first octet of the IEEE 802.2 LLC PDU (SSAP) SHALL be located at offset "n" of the packet, where:
n = 8 + (D1_Area_Size*8) + D2_Offsetas specified in HIPPI-FP. ✨
HIPPI-LE Header 🦋
FC(3 bits) SHALL contain zero unless otherwise defined by local administration.Double_Wide(1 bit) SHALL contain one if the Destination associated with the sending Source supports 64 bit HIPPI operation. Otherwise it SHALL contain zero.Message_Type(4 bits) contains a code identifying the type of HIPPI-LE PDU. Defined values (binary) are:
0 Data PDU
1 Address Resolution Request PDU (AR_Request)
2 Address Resolution Response PDU (AR_Response)
3 Self Address Resolution Request PDU (AR_S_Request)
4 Self Address Resolution Response PDU (AR_S_Response)
5-11 Reserved by the ANSI X3T9.3 committee
12-15 Locally AssignedDestination_Switch_Addressis a 24-bit field containing the Switch Address of the Destination if known, otherwise zero. If the address comprises less than 24 bits, it SHALL be right justified (occupying the least significant bits) in the field.Destination_Address_Type(4 bits) andSource_Address_Type(4 bits) contain codes identifying the type of addresses in theDestination_Switch_AddressandSource_Switch_Addressfields respectively. Defined values (binary) are:
0 Unspecified
1 HIPPI-SC Source Route (24 bits)
2 HIPPI-SC Address (12 bits)
3-11 Reserved by the ANSI X3T9.3 committee
12-15 Locally AssignedSource_Switch_Addressis a 24-bit field containing the Switch Address of the Source. If the address comprises less than 24 bits, it SHALL be right justified (occupying the least significant bits) in the field.Reserved(16 bits) SHALL contain zero.Destination_IEEE_Address(48 bits) SHALL contain the 48 bit Universal LAN MAC Address of the Destination if known, otherwise zero.LE_Locally_Administered(16 bits) SHALL contain zero unless otherwise defined by local administration.Source_IEEE_Address(48 bits) SHALL contain the 48 bit Universal LAN MAC Address of the Source if known, otherwise zero.
IEEE 802.2 LLC 💜
the IEEE 802.2 LLC Header SHALL begin in the first octet of the HIPPI-FP D2_Area.
SSAP(8 bits) SHALL contain170(decimal).DSAP(8 bits) SHALL contain170(decimal).CTL(8 bits) SHALL contain3(Unnumbered Information).
SNAP 🧚
- Organization Code (24 bits) SHALL be zero.
- EtherType (16 bits) SHALL be set as defined in Assigned Numbers [8] (IP = 2048, ARP = 2054, RARP = 32,821).
here's the big picture of the whole packet, bestie 🍓
31 28 23 21 15 10 7 2 0
+-----+---------+-+-+-----------+---------+-----+---------+-----+
0 | 04 |1|0| Reserved | 03 | 0 |
+---------------+-+-+---------------------+---------------+-----+
1 | (n+8) |
+-----+-+-------+-----------------------------------------------+
2 |[LA] |W|M_Type | Destination_Switch_Address |
+-----+-+-------+-----------------------------------------------+
3 | D_A_T | S_A_T | Source_Switch_Address |
+-------+-------+---------------+-------------------------------+
4 | Reserved | [Destination_IEEE_Address] |
+-------------------------------+ |
5 | |
+-------------------------------+-------------------------------+
6 | [LA] | [Source_IEEE_Address] |
+-------------------------------+ |
7 | |
+===============+===============+===============+===============+
8 | AA | AA | 03 | 00 |
+---------------+---------------+---------------+---------------+
9 | 00 | 00 | [EtherType] |
+---------------+---------------+---------------+---------------+
10 |Message octet 0|Message octet 1|Message octet 2| . . . |
+---------------+---------------+---------------+--- |
| . . .
|
| -------+---------------+---------------+---------------+
| . . . | octet (n-2) | octet (n-1) | FILL |
+---------------+---------------+---------------+---------------+
N-1| FILL | FILL | FILL | FILL |
+---------------+---------------+---------------+---------------+
HIPPI Packet Format Words 0-1: HIPPI-FP Header
Words 2-7: D1 Area (HIPPI-LE Header)
Words 8-9: D2 Area (IEEE 802.2 LLC/SNAP)
Words 10-(N-1): D2 Area (IP or ARP message)
(n) is the number of octets in the IP or ARP message.
+====+ denotes the boundary between D1 and D2 areas.
[LA] fields are zero unless used otherwise locally.
Abbreviations: "W" = Double_Wide field;
"M_Type" = Message_Type field;
"D_A_T" = Destination_Address_Type;
"S_A_T" = Source_Address_Type;
[FILL] octets complete the HIPPI packet to an even
number of 32 bit words. The number of fill octets
is not counted in the data length.IEEE 802.2 Data 🌈
the IEEE 802.2 Data SHALL follow the EtherType field immediately. Fill octets SHALL be used following the Data as necessary to make the number of octets in the packet a multiple of 8. in accordance with HIPPI-FP, the amount of this fill is not included in the D2_Size value in the HIPPI-FP Header 🫶
the order of the octets in the data stream runs from higher numbered to lower numbered data signal (left to right) within the HIPPI word, as specified in HIPPI-FP Clause 7, "Word and byte formats." with the 1600 megabit/second data rate option (64 bit), bits 32 through 63 are on Cable B, so the four octets on Cable B come logically before those on Cable A. within each octet, the most significant bit is the highest numbered signal ⭐
48 bit Universal LAN MAC Addresses 🍓
IEEE Standard 802.1A specifies the Universal LAN MAC Address. the globally unique part of the 48 bit space is administered by the IEEE. Each node on a HIPPI-SC LAN SHOULD be assigned a ULA. Multiple ULAs MAY be used if a node contains more than one IEEE 802.2 LLC protocol entity 💅
the format of the address within its 48 bit HIPPI-LE fields follows IEEE 802.1A canonical bit order and HIPPI-FP bit and byte order:
31 23 15 7 0
+-------------------------------+---------------+---------------+
| (not used for ULA) |ULA octet 0|L|G| ULA octet 1 |
+---------------+---------------+---------------+---------------+
| ULA octet 2 | ULA octet 3 | ULA octet 4 | ULA octet 5 |
+---------------+---------------+---------------+---------------+
Universal LAN MAC Address FormatL(U/L bit) = 1 for Locally administered addresses, 0 for Universal.G(I/G bit) = 1 for Group addresses, 0 for Individual.
the use of ULAs is OPTIONAL, but encouraged 🌸 Although ULAs are not used by HIPPI-SC switches, they are helpful for HIPPI Switch Address resolution, and for distinguishing between multiple logical entities that may exist within one node. They MAY also be used by gateway devices that replace HIPPI hardware headers with the MAC headers of other LANs. Carrying the ULAs in the HIPPI header may simplify these devices, and it may also help if HIPPI is used as an interface to some future HIPPI based LAN that uses ULAs for addressing 🧚
Recommended HIPPI-FP Options 🦋
HIPPI-FP allows some flexibility in building a HIPPI packet, including placement of short bursts, optional fill and offset octets between the D1 and D2 areas, and fill following the D2 data. For efficiency, Sources SHOULD limit the use of these options:
- Send the short burst as the last burst of the packet rather than the first.
- Do not place fill octets between the HIPPI-LE header and the start of the D2_Area.
- Use no more than seven octets after the D2 Data, as needed to make the total packet length a multiple of 8 octets.
One HIPPI-FP option is forbidden: setting the Start_D2_on_Burst_Boundary flag to one. This places no limitation on the formation of packets into a series of bursts. A Source MAY segment the packet in any legal manner according to HIPPI-FP, including forcing the D2_Area to start on a burst boundary. The purpose of the Start_D2_on_Burst_Boundary flag is to help preserve the segmentation of the packet for some device-control protocols that use the first burst boundary to separate command and data areas within the packet. Requiring this flag to be clear means that when a packet arrives at the Destination, its burst boundaries might not be exactly as the Source sent them. This may occur if a HIPPI packet passes over some other medium in the route between HIPPI LANs 💜
Notwithstanding these recommendations, each Destination SHALL accept any well-formed HIPPI packet within the definitions in HIPPI-FP ✨
Note that neither HIPPI-FP nor HIPPI-LE limits the number of fill bytes placed between the end of the IP packet and the end of the HIPPI-PH
I-Field format ✨
Okay so the I-field bits are defined in HIPPI-SC, and they've got a whole set of rules 💅
- Locally Administered (bit 31): SHALL be zero.
- Reserved (bits 30, 29): SHOULD be zero. Destinations SHALL accept any value for these bits.
- Double wide (bit 28): SHALL be set when Source Cable B is connected and the Source wants a 64 bit connection. It SHALL be zero otherwise. 🌈
- Direction (bit 27): SHOULD be sent as zero. However, Destinations SHALL accept either zero or one and interpret the Routing Control field accordingly, per HIPPI-SC.
- Path Selection (bits 26, 25): SHALL be
00,01, or11(binary), at the Source's option.00(source route mode) means the I-field bits 23-00 hold a 24 bit source route.01or11(logical address mode) means bits 23-00 hold 12 bit Source and Destination Addresses. The value11is meaningful when more than one route exists from a Source to a Destination, because it lets the switch pick the route. Using01forces the switch to always use the same route for the same Source/Destination pair. 🦋 - Camp-on (bit 24): MAY be 1 or 0. However, a Source SHALL NOT make consecutive requests without Camp-on to the same Destination while those requests are being rejected. The point of this restriction is to stop a node from dodging the switch's fair share arbitration mechanism by repeating requests at a super high rate. 🙅♀️
If logical address mode is used:
- Source Address (bits 23-12) is not used.
- Destination Address (bits 11-0) SHALL contain the Switch Address of the Destination.
If source route mode is used:
- Routing control (bits 23-00) SHALL contain the route to the Destination.
Note: the outcome of Switch Address Resolution (see "Address Resolution" below) decides whether you use logical address mode or source route mode. If source route mode is used with multiple interconnected switches, different sources may use different addresses to reach the same destination, and multicast-based address resolution may not be possible because a target node may not know the route to itself from a given remote source. Even so, it may be possible to use source route mode if the network is a single switch, or if address resolution is supported by an ARP agent that can deliver correct routes to each node. The nodes themselves need not worry about any of this if they use the addressing mode suggested by the value of the Source_Address_Type field in a HIPPI-LE Address Resolution Response packet. 🧚
Rules For Connections 🎀
These connection management rules, for both Source and Destination, are meant to make sure there's frequent, fair share access to Destinations that lots of Sources are fighting over. If possible, nodes SHOULD transfer data at full HIPPI speeds and hold connections no longer than they need to. ⭐
A source MAY hold a connection for as long as it takes to send 68 HIPPI bursts at whatever speed the two connected nodes can manage together. The number of packets sent in one connection isn't limited, except that the number of bursts across all the packets SHOULD NOT exceed 68. This is not a recommendation to send as many packets as possible per connection, and one packet per connection is totally fine. The whole point of this limit is to give each Source a fair share of a common Destination's bandwidth. Without it, if a Destination is constantly in demand by multiple Sources, the Source that sends the most data per connection wins the biggest share. That's not it, bestie 😤
The limit of 68 bursts isn't absolute. An implementation MAY check the burst count after sending a packet and end the connection if the count is greater than or equal to some threshold. If it does this, the threshold SHOULD be less than 68, depending on the typical packet size, so the 68 burst limit isn't normally exceeded. For example, a Source sending 64K packets would send two per connection (130 bursts) if it checked for 68 at the end of each packet. In this situation the Source is REQUIRED to check for a value small enough that it won't send a second packet in the same connection. 🙈
Destinations SHALL accept all packets that arrive during a connection, and MAY discard those that exceed their buffering capacity. A Destination SHALL NOT abort a connection (deassert CONNECT) just because too many bursts came in. However, a Destination MAY abort a connection whose duration has gone past a time period of the Destination's choosing, as long as the Source gets plenty of time to send its quota of bursts. 🫶
These rules tell the node to do certain things as fast as it can, but there's no absolute measure of compliance. Nodes that can't move data at full HIPPI speeds can still interoperate. The faster the implementation, though, the better the network performs. 🚀
Assuming bursts are flowing at the maximum rate, the biggest factor in network throughput is the connection switching time. That's measured from when the Source deasserts REQUEST at the end of one connection to its first assertion of BURST after the new connection is set up. Implementations SHOULD keep this time as short as possible. As a guideline, assuming parallel HIPPI and a single HIPPI-SC switch, ten microseconds gets you nearly full HIPPI throughput with full-sized packets, and at 60 microseconds the available throughput drops by about 10%. (See "Performance," below.) 💅
All HIPPI electrical signaling SHALL comply with HIPPI-PH. In every case, the following rules go beyond what HIPPI-PH requires. 🔥
Rules for the Source 🌸
- Do not assert REQUEST until a packet is ready to send.
- Transmit bursts as quickly as READYs permit. Except for the required HIPPI Source Wait states, there SHOULD be no delay in the assertion of BURST whenever the Source's READY counter is nonzero.
- Make a best effort to ensure that connection durations do not exceed 68 bursts.
- Deassert REQUEST immediately when no packet is available for immediate transmission, or when the last packet of the connection has been sent.
Rules for the Destination 🍓
- Reject all connections if unable to receive packets. This frees the requesting Source to connect to other Destinations with minimal delay. Being unable to receive packets isn't a temporary thing. It's the state a Destination is in when its network interface isn't initialized. 😬
- A HIPPI node SHOULD be ready to efficiently accept connections and process incoming data packets. This may be best achieved by not asserting connect unless 68 bursts worth of buffers are available, but it may be possible to meet this requirement with fewer buffers. That might come from an a priori agreement between nodes on packet sizes, the speed of the interface moving buffers, or other implementation dependent considerations. 🧠
- Accept a connection immediately when buffers are available. The Destination SHOULD never delay accepting a connection unnecessarily. ⏩
- Once initialized, a Destination MAY reject connection requests only for one of the following reasons:
- The I-field was received with incorrect parity.
- The I-field contents are invalid, e.g. the "W" bit is set when the Destination doesn't support the 1600 megabit data rate option, the "Locally Administered" bit is set, the Source isn't permitted to send to this Destination, etc.
Transient conditions within the Destination, such as temporary buffer shortages, MUST never cause rejected connections. 🙅♀️
- Ignore aborted connection sequences. Sources may time out and give up on connection attempts, so aborted connection sequences are totally normal. 🙃
MTU 💖
Maximum Transmission Unit (MTU) is defined as the length of the IP packet, including the IP header, but not including any overhead below IP. Conventional LANs have MTU sizes set by the physical layer specification. MTUs may be required simply because the chosen medium won't work with larger packets, or they may serve to limit how long a node has to wait for a chance to send a packet. ⏳
HIPPI has no inherent limit on packet size. The HIPPI-FP header has a 32 bit D2_Size field that, while it may limit packets to about 4 gigabytes, imposes no practical limit for networking purposes. Even so, a HIPPI-SC switch used as a LAN needs an MTU so that Destination buffer sizes can be determined. 📦
The MTU for HIPPI-SC LANs is 65280 (decimal) octets. ✨
This value was picked because it lets the IP packet fit in one 64K octet buffer with up to 256 octets of overhead. The overhead is 40 octets right now, which leaves 216 octets of room for expansion. 🌈
HIPPI-FP Header 8 octets
HIPPI-LE Header 24 octets
IEEE 802.2 LLC/SNAP Headers 8 octets
Maximum IP packet size (MTU) 65280 octets
------------
Total 65320 octets (64K - 216)Camp-on 👑
When several Sources contend for a single Destination, the Camp-on feature lets the HIPPI-SC switch arbitrate and make sure every Source gets fair access. (HIPPI-SC doesn't specify how arbitration works.) Without Camp-on, the contending Sources would just have to keep retrying the connection until it was accepted, and the fastest Source would usually win. To guarantee fair share arbitration, Sources are prohibited from making repeated requests to the same Destination without Camp-on in a way that defeats the arbitration. 🏆
There's another big reason to use Camp-on. When a connection without Camp-on gets rejected, the Source can't tell whether the rejection came from the requested Destination or from the switch. It also can't tell the reason, which could be that the Destination is offline or not cabled, or that the I-field was erroneous or had incorrect parity. Sources SHOULD NOT treat a rejection of a request without Camp-on as an error. 🫠
Camp-on prevents rejection due to the temporary busy case. With one exception, rejection of a Camp-on request indicates an error condition, and an error event can be recorded. The exception happens when a 64 bit connection is attempted to a Destination that doesn't have Cable B connected, which results in a reject. That case is covered in "Channel Data Rate Discovery," below. 💜
Address Resolution ✨
okay so the Internet Address Resolution Protocol (ARP) is defined in RFC 826 [9] 💅 Ethernet, FDDI and 802 networks use ARP to find another host's ULA when they already know its Internet address. Reverse ARP [10] goes the other way, letting you find the Internet address when you already know the ULA. ARP can also be used the usual way on HIPPI-SC LANs that have a multicast capability or a third party ARP agent 🦋
HIPPI-LE defines similar lower-level address resolution between ULAs and switches 🌸 It also adds a self-address resolution mechanism that ARP doesn't have, which lets a node find out its own switch address dynamically. main character energy 👑
ARP for finding ULAs is not necessary for a HIPPI-SC LAN to run, but it's the vehicle for discovering HIPPI-SC Switch Addresses, and the LAN literally cannot function without those 😭 So while a node is using ARP to map another node's IP address to its ULA, it's also mapping that ULA to the 12 bit HIPPI Switch Address. From there it builds the I-field value it needs to send messages to that node. This extra layer of hardware addressing uses the address fields in the HIPPI-LE header 🧚
In the following discussion, "requester" and "target" mean the node asking for address resolution and the node whose address it's trying to find, respectively 🎀 In third party ARP (see "ARP Implementation Methods," below), the source of a reply is an ARP agent node, not the target node 🫶
ARP and RARP Message Format ✨
The HIPPI ARP/RARP protocol uses the same packet format as ARP for Ethernet 💖 ARP packets SHALL be transmitted with a hardware type code of 1 (as for Ethernet). Plus, ARP packets SHALL be accepted if they're received with hardware type codes of either 1 or 6 (IEEE 802 networks).
Here's the field checklist, bestie 📋
ar$hrd(16 bits) SHALL contain1.ar$pro(16 bits) SHALL contain the IP protocol code2048(decimal).ar$hln(8 bits) SHALL contain6.ar$pln(8 bits) SHALL contain4.ar$op(16 bits) SHALL contain1for requests and2for responses.ar$sha(48 bits) in requests SHALL contain the requester's ULA. In replies, it SHALL contain the target node's ULA.ar$spa(32 bits) in requests SHALL contain the requester's IP address if known, otherwise zero. In replies, it SHALL contain the target node's IP address.ar$tha(48 bits) in requests SHALL contain the target's ULA if known, otherwise zero. In replies, it SHALL contain the requester's ULA.ar$tpa(32 bits) in requests SHALL contain the target's IP address if known, otherwise zero. In replies, it SHALL contain the requester's IP address.
The six octets of the ULA SHALL be formatted the same way as what's required in the HIPPI-LE header (see "48 bit Universal LAN MAC Addresses" above). The one difference is how the Source ULA lines up with the 32 bit HIPPI word, which is different between ARP and HIPPI-LE. No bit reversal is necessary here, even though FDDI requires it [11] 🌈
31 28 23 21 15 10 7 2 0
+-----+---------+-+-+-----------+---------+-----+---------+-----+
0 | 04 |1|0| 000 | 03 | 0 |
+---------------+-+-+---------------------+---------------+-----+
1 | 36 |
+-----+-+-------+-----------------------+-----------------------+
2 |[LA] |W| 1 | 000 | Target Switch Addr |
+-----+-+-------+-----------------------+-----------------------+
3 | 2 | 2 | 000 |Requester's Switch Addr|
+---------------+---------------+-------+-----------------------+
4 | 00 00 | |
+-------------------------------+ |
5 | Target ULA |
+-------------------------------+-------------------------------+
6 | [LA] | |
+-------------------------------+ |
7 | Requester's ULA |
+===============+===============+===============+===============+
8 | AA | AA | 03 | 00 |
+---------------+---------------+---------------+---------------+
9 | 00 | 00 | EtherType (2054) |
+---------------+---------------+-------------------------------+
10 | hrd (1) | pro (2048) |
+---------------+---------------+-------------------------------+
11 | hln (6) | pln (4) | op (1) |
+---------------+---------------+-------------------------------+
12 | Requester's ULA octets 0 - 3 |
+-------------------------------+-------------------------------+
13 | Requester's ULA octets 4 - 5 | Requester's IP Address upper |
+-------------------------------+-------------------------------+
14 | Requester's IP Address lower | Target ULA octets 0 - 1 |
+-------------------------------+-------------------------------+
15 | Target ULA octets 2 - 5 |
+---------------------------------------------------------------+
16 | Target IP Address |
+---------------------------------------------------------------+
HIPPI ARP/RARP Request (logical address mode)All ARP requests SHALL be sent with the I-field bit 28 set to zero, i.e. requesting a 32 bit connection 🙈
Unless some other convention is locally defined for ARP requests, the I-field Path Selection bits MAY be set to binary 01 or 11 (logical address mode). The Destination Address field gets set to the HIPPI-SC address reserved for traffic that's normally aimed at the IEEE 802.1[12] broadcast address, which HIPPI-SC defines as FE0, hex 🦋
Reply packets SHALL be sent with the I-field Path Selection and Routing Control fields set according to the Source_Address_Type and Source_Switch_Address fields in the request.
In the HIPPI-LE header of ARP/RARP requests and replies, the following fields SHALL be set:
Double-Wide SHOULD be 1 if the HIPPI Destination at the sending node can accept 64 bit HIPPI connections 💜
Message_Type SHALL contain an address resolution type code as defined in HIPPI-LE. It SHALL be set appropriately to the value of the ARP operation code (ar$op) in piggybacked ARP messages:
+-----------------------+-----------------------+
| ARP ar$op | HIPPI-LE Message_Type |
+=======================+=======================+
|ARP Request (1) |AR_Request (1) |
|ARP Reply (2) |AR_Response (2) |
+-----------------------+-----------------------+
|Reverse ARP Request (3)|AR_Request (1) |
|Reverse ARP Reply (4) |AR_Response (2) |
+-----------------------+-----------------------+There is no ARP message corresponding to HIPPI-LE self address discovery, so these packets are sent without ULP data 🫧
Destination_Switch_Address in requests SHALL be the Switch Address of the target node if known, otherwise zero. In replies, it SHALL be the requesting node's Switch Address.
Destination_Address_Type SHALL be 1 if the Destination_Switch_Address is a source route, and 2 if it is a 12 bit address.
Source_Address_Type SHALL be 1 if the Source_Switch_Address is a source route, and 2 if it is a 12 bit address.
Source_Switch_Address in requests SHALL be the Switch Address of the requesting node if known, otherwise zero. In replies, it SHALL be the target node's Switch Address.
Destination_IEEE_Address SHALL be the same as the ar$tha field in the ARP message 💖
Source_IEEE_Address SHALL be the same as the ar$sha field in the ARP message ✨
31 28 23 21 15 10 7 2 0
+-----+---------+-+-+-----------+---------+-----+---------+-----+
0 | 04 |1|0| 000 | 03 | 0 |
+---------------+-+-+---------------------+---------------+-----+
1 | 36 |
+-----+-+-------+-----------------------+-----------------------+
2 |[LA] |W| 2 | 000 |Requester's Switch Addr|
+-----+-+-------+-----------------------+-----------------------+
3 | 2 | 2 | 000 | Target Switch Address |
+---------------+---------------+-------+-----------------------+
4 | 00 00 | |
+-------------------------------+ |
5 | Requester's ULA |
+-------------------------------+-------------------------------+
6 | [LA] | |
+-------------------------------+ |
7 | Target ULA |
+===============+===============+===============+===============+
8 | AA | AA | 03 | 00 |
+---------------+---------------+---------------+---------------+
9 | 00 | 00 | EtherType (2054) |
+---------------+---------------+-------------------------------+
10 | hrd (1) | pro (2048) |
+---------------+---------------+-------------------------------+
11 | hln (6) | pln (4) | op (2) |
+---------------+---------------+-------------------------------+
12 | Target ULA octets 0 - 3 |
+-------------------------------+-------------------------------+
13 | Target ULA octets 4 - 5 | Target IP Address upper |
+-------------------------------+-------------------------------+
14 | Target IP Address lower | Requester's ULA octets 0 - 1 |
+-------------------------------+-------------------------------+
15 | Requester's ULA octets 2 - 5 |
+---------------------------------------------------------------+
16 | Requester's IP Address |
+---------------------------------------------------------------+
HIPPI ARP/RARP Reply (logical address mode)ARP procedure ✨
Okay bestie, so the combined HIPPI-LE/ARP packet holds six addresses 💖, three for the requester and three for the target 🎀:
Requester's IP Address (ARP)
Requester's ULA (ARP and HIPPI-LE)
Requester's Switch Address (HIPPI-LE)
Target's IP Address (ARP)
Target's ULA (ARP and HIPPI-LE)
Target's Switch Address (HIPPI-LE)Internet ARP handles the IP Address and ULA, and HIPPI-LE address resolution handles the ULA and Switch Address. That means the ULA shows up in both halves of the packet 🦋.
A successful ARP fills in tables on every node so they can map remote IP addresses to ULAs and ULAs to Switch Addresses. Then when an app asks for a connection by IP address, the node can find the remote ULA and Switch Address, build a correct HIPPI-LE header, and connect using the right Switch Address in the I-field ✨. Anyone who gets an ARP request or reply can use the info in it to update their tables, even if they're not the target or the requester. Main character energy for every node 👑.
ULAs aren't required with HIPPI 🙈. In both the HIPPI-LE header and the Internet ARP message, the ULA fields SHOULD be set to zero when the ULA isn't known. Address resolution is two separate protocols, HIPPI-LE address resolution and Internet ARP, and neither one can work on its own without ULAs. BUT HIPPI Switch Address resolution CAN work without ULAs if the two protocols are piggybacked and treated as one operation that maps Internet addresses straight to switch addresses 🧚. Apart from the OPTIONAL self-address resolution request (which has no Internet protocol equivalent), HIPPI-LE address resolution and Internet ARP messages SHOULD be sent together as one HIPPI packet 🌸.
If ULAs are in use, the HIPPI-LE address resolution request can be sent without a piggybacked 802.2 LLC PDU, so mapping ULAs to HIPPI Switch Addresses without ARP is possible 💅. Nodes SHALL accept both the piggybacked and the non-piggybacked forms of HIPPI-LE address resolution messages 🫶.
When a recipient gets an address resolution request, it first updates its mapping tables with any new info in the request. Then it checks whether it's the target. If it is, it builds a reply by filling in the unknown target address fields based on the HIPPI-LE message type and the ARP operation code, and it swaps the four source/target address field pairs. After that, it connects back to the requester using the Source Switch Address from the request and sends the reply packet 🔥.
A node is the target of an address resolution request if the request contains either of these:
- the node's ULA in the Destination_IEEE_Address field of a HIPPI-LE AR_Request message
- the node's IP address in the target protocol address field (ar$tpa) of a piggybacked Internet ARP message
Now the tricky bit 💀. If two target fields are known but aren't mapped to each other in the recipient's tables, it MAY do one of three things:
- treat the request as having two targets, and send correct replies for both to the requester.
- assume its own tables are invalid and ignore the request.
- assume one of the "known" target fields is correct and respond as if the other one had been unknown.
Which one is best depends on which fields conflict and what kind of implementation you've got 🤔. Choice 3 is probably the move for ordinary nodes, but third party ARP agents MAY have a reason to pick one of the other two. Future experience MAY shed light on this 🌈.
ARP Implementation Methods 🦋
Okay so the rules for nodes handling address resolution messages depend on how address resolution is set up on the LAN. It's giving "it depends on the vibe" 😭.
In conventional networks, ARP is a distributed function 💜. ARP requests get broadcast, each host MAY update its address mappings with any ARP request or reply it sees, and it responds to ARP requests that contain its own address in the target protocol address field. HIPPI-SC switches are NOT REQUIRED to offer multicast service, although some do 🙈. Even when the switches don't multicast, one or more nodes can act as multicast servers. They receive packets sent to the HIPPI-SC broadcast address and pass them along to every other node, one at a time 🧚. Either way, if multicast exists on a HIPPI-SC LAN, ARP can be a distributed function. In that case each node is REQUIRED to respond correctly to address resolution requests where it's the target 👑.
Third party ARP is the second method, and it doesn't rely on multicast 💅. The switches can map the HIPPI ARP multicast address to a node acting as an ARP agent, which replies to ARP requests on behalf of the real target nodes. Ordinary nodes never receive ARP requests, never generate replies, and never get the chance to update their mapping tables from other nodes' ARP requests the way they usually do on conventional networks 😭. Each node MUST request any address information it needs, but it never has to process ARP info it doesn't need 🫶. Under third party ARP, a node SHOULD NOT receive address resolution requests, and every node that isn't an ARP agent SHOULD ignore the ones it does receive 🙈.
The third option is to skip ARP entirely and build the address mapping tables on each node from info a network administrator already has 🍓. That technique is implementation dependent and outside the scope of this memo. It MAY be helpful in prototype networks without multicast where no ARP agent exists yet ✨. In this case, nodes are NOT REQUIRED to respond to address resolution requests, but they MUST accept any they MAY receive 💖.
ARP Example 🎀
Okay, picture this 📸. A HIPPI-SC switch is set up with two nodes, X and Y, connected to it. Each node has its own unique Switch Address. Both nodes can read the host database (like /etc/hosts), where the network admin set up the network and gave the two nodes IP addresses. There's an ARP agent plugged into a switch port that's mapped to address FE0 (hex). The ARP agent doesn't have any IP, IEEE, or Switch mappings yet 🧚. Both nodes know their own ULAs and Switch Addresses. They want to talk to each other 💕. Each one knows the other's IP address from the host database, but neither knows the other's ULA or Switch Address. Node X goes first 🔥:
- Node X connects to FE0 and sends a piggyback ARP Request asking for Y's addresses:
HIPPI-LE Message_Type is 1, AR_Request
HIPPI-LE Destination_Switch_Address = 0
HIPPI-LE Source_Switch_Address = X's Switch Address
HIPPI-LE Destination_IEEE_Address = 0
HIPPI-LE Source_IEEE_Address = X's ULA
ARP ar$op = 1 (request)
ARP ar$sha = X's ULA
ARP ar$spa = X's IP Address
ARP ar$tha = 0
ARP ar$tpa = Y's IP Address- The ARP agent gets the ARP request and adds an entry for X to its address mapping table. It doesn't know about Y yet, so it doesn't send a reply 🙈.
- Node X waits for a reply. It MAY set a timer to resend the request now and then, but its requests will just get ignored until node Y sends an ARP request 🥲.
- Node Y connects to FE0 and sends an ARP request asking for X's addresses:
HIPPI-LE Message_Type is 1, AR_Request
HIPPI-LE Destination_Switch_Address = 0
HIPPI-LE Source_Switch_Address = Y's Switch Address
HIPPI-LE Destination_IEEE_Address = 0
HIPPI-LE Source_IEEE_Address = Y's ULA
ARP ar$op = 1 (request)
ARP ar$sha = Y's ULA
ARP ar$spa = Y's IP Address
ARP ar$tha = 0
ARP ar$tpa = X's IP Address- The ARP agent gets Y's request and adds an entry for Y to its address mapping table. It knows about the target node, X, so it connects to Y (using the Source_Switch_Address from the request) and sends an ARP Reply 🫶:
HIPPI-LE Message_Type is 2, AR_Reply
HIPPI-LE Destination_Switch_Address = Y's Switch Address
HIPPI-LE Source_Switch_Address = X's Switch Address
HIPPI-LE Destination_IEEE_Address = Y's ULA
HIPPI-LE Source_IEEE_Address = X's ULA
ARP ar$op = 2 (reply)
ARP ar$sha = X's ULA
ARP ar$spa = X's IP Address
ARP ar$tha = Y's ULA
ARP ar$tpa = Y's IP Address- Node Y gets the ARP reply and builds its address mapping table entry for Node X 🎉.
- Node Y connects to node X and sends an IP packet with this info in the HIPPI-LE header 💌:
HIPPI-LE Message_Type is 0, AR_Data
HIPPI-LE Destination_Switch_Address = X's Switch Address
HIPPI-LE Source_Switch_Address = Y's Switch Address
HIPPI-LE Destination_IEEE_Address = X's ULA
HIPPI-LE Source_IEEE_Address = Y's ULA- Node X gets the IP packet. Since the ARP agent now knows about node Y, node X can resend its ARP request (repeating step 1) and get an ARP reply 🥳:
HIPPI-LE Message_Type is 2, AR_Reply
HIPPI-LE Destination_Switch_Address = X's Switch Address
HIPPI-LE Source_Switch_Address = Y's Switch Address
HIPPI-LE Destination_IEEE_Address = X's ULA
HIPPI-LE Source_IEEE_Address = Y's ULA
ARP ar$op = 2 (reply)
ARP ar$sha = Y's ULA
ARP ar$spa = Y's IP Address
ARP ar$tha = X's ULA
ARP ar$tpa = X's IP AddressAnd just like that, address resolution is complete for both nodes 💖✨.
Quick bonus, iykyk 🙈: if the configuration had a multicast facility instead of the ARP agent, the target nodes themselves would have received the requests and replied to them the same way the ARP agent did 🌸.
Discovery of One's Own Switch Address ✨
okay so the ARP example above just assumed every node already knew its own switch address 🙈, which is kinda convenient, not gonna lie. This can be manually configured, by means that are outside the scope of this memo, when each node gets connected to the switch 🔌. If a multicast capability exists, though, the node can discover its own address automatically when it starts up, using a protocol defined in HIPPI-LE 💅.
In the self-address discovery protocol, a node connects to a multicast address and sends a HIPPI-LE message containing its own ULA. It gets a multicast copy of its own message and learns its own switch address from the destination address field of the received I-field 🦋.
HIPPI-LE self address resolution uses the same HIPPI-LE message format described in "ARP and RARP Message Format," above, with the AR_S_Request and AR_S_Response message type codes and no piggybacked ULP data. The HIPPI-LE header contents for the request are:
Message_Type is 3, AR_S_Request
Destination_Address_Type = 0 (undefined)
Destination_Switch_Address = 0 (unknown)
Source_Address_Type = 0 (undefined)
Source_Switch_Address = 0 (unknown)
Destination_IEEE_Address = my ULA
Source_IEEE_Address = my ULAThere is no D2 data. The packet contains only the HIPPI-FP header and D1_Area containing the HIPPI-LE header 📦.
The node that wants to discover its address connects to the multicast address for this purpose (hex FE0 in HIPPI-SC) and transmits the request packet. What happens next depends on the particular network 🤔:
With multicast:
The node receives its own request and can learn its own switch address from the I-field it receives. This is the only time a node should use an address from a received I-field 🎯.
With an ARP agent:
The node may receive an AR_S_Response message with its own ULA in the Destination_IEEE_Address field and its own switch address in the Destination_Switch_Address field. This address may be different from the address contained in the I-field, and should be used instead 💖.
The ARP agent response alternative requires that the agent have prior knowledge of the node's location and ULA through some process not specified by this memo. The node may receive both its request and the agent's response if both an ARP agent and multicast are active. In this case, the address it learns from the I-field is later replaced by the address given by the ARP agent in the response. Agents may assign new addresses to nodes and inform them by sending unsolicited AR_S_Response messages. Any node whose switch address is updated in this way should invalidate the switch addresses it has saved for other nodes, and use ARP to rediscover them 🔄.
If the node reacts correctly to either the multicast request or agent-generated response, it can discover its address without having to know whether or not an ARP agent is active 🧚. The full procedure is:
- Transmit the
AR_S_Request - When a connection arrives, accept it and save the I-field for later analysis.
- Receive the message and look at the HIPPI-LE header. If the message is Message Type
AR_S_Request, analyze the I-field to discover the node's own switch address. HIPPI-SC I-field formats suggest the following:
if bit 25 == 1
Address type is HIPPI-SC logical address.
if bit 27 == 1
take address from bits 23-12
else
take address from bits 11-00
endif
else
Address is unusable (source route)
endifThis is a one-time operation. Once the node knows an address for itself, it should not take any new address from a received I-field 🚫.
- If a message of type
AR_S_Responsearrives and theDestination_IEEE_Addressfield contains the node's own ULA, take the new switch address from theDestination_Switch_Addressfield and its type from theDestination_Address_Typefield. - The node should invalidate its ARP tables when an
AR_S_Responsechanges its own switch address, to force retransmission of ARP requests containing its new address to all the remote nodes with which it communicates 📣.
Path MTU Discovery 🛣️
RFC 1191 [13] describes the method of determining MTU restrictions on an arbitrary network path between two hosts. HIPPI nodes may use this method without modification to discover restrictions on paths between HIPPI-SC LANs and other networks. Gateways between HIPPI-SC LANs and other types of networks should implement RFC 1191 🌉.
Channel Data Rate Discovery ⚡
HIPPI exists in two data rate options (800 megabit/second and 1600 megabit/second). The higher data rate is achieved by making the HIPPI 64 bits parallel instead of 32, using an extra cable containing 32 additional data bits and four parity bits. HIPPI-SC switches can be designed to attach to both. Source and Destination HIPPI implementations can be designed to operate at either rate, selectable at the time a connection is established 🎀.
The "W" bit (bit 28) of the I-field controls the width of the connection through the switch. Sources with both cables A and B attached to the switch may set the "W" bit to request a 1600 megabit/second connection. If the requested destination also has both cables attached, the switch can connect Source to Destination on both cables. If the requested Destination has only Cable A, the switch rejects the request 🙅♀️. Sixty-four bit Sources can connect to 32 bit Destinations by requesting with the "W" bit clear and not using Cable B. Sixty-four bit Destinations must examine the "W" bit in the received I-field and use or ignore Cable B accordingly. Note that both INTERCONNECT signals stay active while a 64 bit HIPPI is used in 32 bit mode 👀.
The following table summarizes the possible combinations, the switch's action for each, and the width of the resulting connection 📊.
Destination
+-------------------+-------------------+
| 32 | 64 |
+----+-----+-------------------+-------------------+
| | W=0 | Accept 32 | Accept 32 |
| 32 +-----+-------------------+-------------------+
| | W=1 | N/A | N/A |
Source +----+-----+-------------------+-------------------+
| | W=0 | Accept 32 | Accept 32 |
| 64 +-----+-------------------+-------------------+
| | W=1 | Reject | Accept 64 |
+----+-----+-------------------+-------------------+
HIPPI Connection CombinationsIf the path between a 64 bit Source and a 64 bit Destination includes more than one switch, and the route between switches uses a link that is only 32 bits wide, the switch rejects 64 bit connection requests as if the Destination did not have 64 bit capability 🚧.
In a mixed LAN of 32 bit and 64 bit HIPPIs, a 64 bit Source needs to know the data rates available at each Destination and on the path to it. This can be known a priori by manual configuration, or it can be discovered dynamically. The only reliable method of discovery is simply to attempt a 64 bit connection with Camp-on. As long as 64 bit connections succeed, the Source knows the Destination and path are double width 💪. If a 64 bit connection is rejected, the Source tries to connect for 32 bits. If the 32 bit connection succeeds, the Source assumes that the Destination or path is not capable of double width operation, and uses only 32 bit requests after that. If the 32 bit request is rejected, the Source assumes that the Destination or path is down and makes no determination of its capability 😬.
The Double_Wide bit in the HIPPI-LE header, if nonzero, gives the node that receives it a hint that the 64 bit connection attempt may be worthwhile when sending on the return path 💡.
Note that Camp-on must be used at least in the 64 bit attempt, because it removes some ambiguity from the meaning of rejects. If the request is made with the "W" bit and no Camp-on, a reject could mean either that the Destination has no Cable B or that it is simply busy, and no conclusion can be drawn as to its status for 64 bit connections. 🫶
Performance ✨
okay bestie, so the HIPPI connection rules are designed to get the most out of the available HIPPI throughput 🚀 while still making sure each Destination is free often enough to receive packets from different Sources 💖. that means both Sources and Destinations have to keep connection setup overhead low if they want high performance 💅. low setup times are easy to hit in hardware 🔥, but it gets messy if software has to run between the first connection request and the start of data transfer 😭. hardware where connection setup and data transfer both come from one software action is literally the dream 😍.
HIPPI connections are controlled by HIPPI Sources 👑. a Destination can't just hang up on its own without risking data loss, so once it accepts a connection it's basically at the Source's mercy 🙈. which means connection strategy optimizations are the Source's job, and several of them are allowed ✨.
if messages show up slower than the available HIPPI throughput and Destinations are rarely busy when a connection gets requested, optimizing doesn't really pay off 🤷♀️. the simplest approach, waiting indefinitely for each connection and sending messages strictly in the order they were queued, can't be beaten in that case 💅. but if some nodes are slow, or network apps can send or receive at a higher aggregate rate than the HIPPI bandwidth, Sources will often run into a busy Destination 😤. then certain host output queuing strategies can help channel utilization 📈. Sources may keep separate output queues for different HIPPI Destinations, and bail on one Destination in favor of another if a connection attempt without Camp-on gets rejected, or if a connection request with Camp-on isn't accepted within a predetermined interval 🏃♀️. this leads to aborted connection sequences (defined in HIPPI-PH: REQUEST is deasserted before any data is sent) 🙃. Destinations must treat these as normal events, maybe counting them but otherwise ignoring them 🫶.
two parts of connection setup time are totally out of the control of both the Source and the Destination 🙅♀️. the first is the time it takes the switch to connect Source to Destination, which is currently less than four microseconds on the largest commercially available (32 port) switch ⚡. the second is the round trip propagation time of the REQUEST and CONNECT signals. that's negligible on a standard 25 meter copper HIPPI cable, but on fiber optic links it adds up to about 10 microseconds per kilometer 🌈. so HIPPI-SC LANs spanning more than a few kilometers will have reduced throughput 💀. limited span networks with buffered gateways or bridges between them may perform better than long serial HIPPI links 🧚.
a Source is required to drop its connection after the transmission of 68 HIPPI bursts 🛑. this number was picked so it can fit one maximum sized packet or a reasonable number of smaller ones 📦.
here's the table with some possibilities, showing the calculated maximum burst and throughput rates in millions (10**6) of bytes per second 🍓:
Maximum HIPPI Throughput Rates
Number Number Hold Burst ------Max throughput MB/sec-------
User of of Time Rate Connection Setup Overhead (usec)
Data Packets Bursts (usec) MB/sec 10 30 60 90 120 150
---- ------- ------ ------ ------ ---- ---- ---- ---- ---- ----
63K 1 64 654 98.7 97.2 94.4 90.4 86.8 83.4 80.3
32K 2 66 665 98.6 97.1 94.3 90.4 86.8 83.5 80.4
16K 4 68 667 98.3 96.8 94.1 90.2 86.6 83.3 80.2
8K 7 63 587 97.8 96.1 93.0 88.7 84.8 81.2 77.8
4K 13 65 551 96.7 95.0 91.7 87.2 83.1 79.4 76.0
2K 22 66 476 94.6 92.7 89.0 84.0 79.6 75.6 72.0
1K 34 68 384 90.8 88.5 84.2 78.5 73.5 75.8 65.3these calculations are based 259 40 ns clock periods to transmit a full burst and 23 clock periods for a short burst. (HIPPI-PH specifies three clock periods of overhead per burst 🤓.) a packet of "n" kilobytes of user data is "n" full bursts plus one short burst equal in length to the number of bytes in the HIPPI, LLC, IP and TCP headers 📏. "Hold Time" is the minimum connection duration needed to send the packets 🕰️. "Burst Rate" is the effective transfer rate for the duration of the connection, not counting connection switching time ⏱️. throughput rates are in megabytes/second, accounting for connection switching times of 10, 30, 60, 90, 120 and 150 microseconds 🎀. these calcs ignore any limit on the rate at which a Source or Destination can process small packets. those limits could lower the available throughput even more if small packets are used 🙈.
Sharing the Switch 🦋
network interconnection is only one thing HIPPI and HIPPI-SC switches can do 🌸. network apps need very frequent short-lived connections, but other apps might like longer term or even permanent connections between a Source and a Destination 💜. since the switch can give each Source or Destination hardware paths that are totally separate from everyone else's, it's totally doable to run LAN interconnects and computer/peripheral stuff on the same switch at the same time 🧚♀️.
switch sharing is no problem when unlike apps never share a HIPPI cable on any path ✨. but if a host has to use a single input or output cable for network traffic as well as other kinds of traffic, or a link between switches has to be shared, you need to be extra careful and make sure every app is compatible with the connection discipline in this memo 😬. apps that hold connections too long on links shared with network traffic can cause loss of network packets or serious degradation of network service 💀.
Appendix A -- HIPPI Basics ✨
okay bestie, this is a little crash course for anyone who isn't fluent in HIPPI yet 🌸
HIPPI-PH describes a parallel copper data channel between a Source and a Destination 🔌 HIPPI only goes one way, so you need two sets for bidirectional flow. Here's a simple point-to-point link between two computer systems:
+----------+ +----------+
| | | |
| +--------+ +--------+ |
| | HIPPI | Cable | HIPPI | |
| | +--------------------->| | |
| | Source | | Dest. | |
| System +--------+ +--------+ System |
| X +--------+ +--------+ Y |
| | HIPPI | Cable | HIPPI | |
| | |<---------------------+ | |
| | Dest. | | Source | |
| +--------+ +--------+ |
| | | |
+----------+ +----------+
A Simple HIPPI Duplex LinkParallel copper cables may be up to 25 meters in length 📏
Heads up: in this doc, every HIPPI connection is assumed to be a paired HIPPI channel 🎀
HIPPI-PH has one optional feature 🦋 It can use a single cable in each direction for a 32 bit parallel channel with a maximum data rate of 800 megabit/second, or two cables for 64 bits and 1600 megabit/second. Cable A carries bits 0-31 and is used in both modes, while Cable B carries bits 32-63 and is used only with the 1600 megabit/second data rate option.
HIPPI Signal Hierarchy
HIPPI has the following hardware signals:
Source to Destination
INTERCONNECT AINTERCONNECT B (64 bit only)CLOCK (25 MHz)REQUESTPACKETBURSTDATA (32 or 64 signals)PARITY (4 or 8 signals)
Destination to Source
INTERCONNECT AINTERCONNECT B (64 bit only)CONNECTREADY
The INTERCONNECT lines carry DC voltages that indicate that the cable is connected and that the remote interface has power 🔋 INTERCONNECT is not used for signaling.
The CLOCK signal is a continuous 25 MHz (40 ns period) square wave ⏱️ All Source-to-Destination signals are synchronized to the clock.
The REQUEST and CONNECT lines are used to establish logical connections. A connection is always initiated by a Source as it asserts REQUEST. At the same time, it puts 32 bits of data on DATA lines 0-31, called the I-field. The Destination samples the DATA lines and can complete a connection by asserting CONNECT. Packets can be transmitted only while both REQUEST and CONNECT are asserted 🤝
A Destination can also reject a connection by asserting CONNECT for only a short interval between 4 and 16 HIPPI clock periods (160-640 nanoseconds) 🙅♀️ The Source knows a connection has been accepted when CONNECT is asserted for more than 16 clocks or it receives a READY pulse.
Either Source or Destination can terminate a connection by deasserting REQUEST or CONNECT, respectively. Normally, connections are terminated by the Source after its last Packet has been sent. A Destination cannot terminate a connection without potential loss of data 😭
+------+-------------------------+------+
| Idle | Connected | Idle | . . .
+------+-------------------------+------+
/ \
/ \
/ \
/ \
/ \
+-------+ +-------+ +-------+ +-------+
|I-field| |Packet | |Packet | |Packet |
+-------+ +-------+ +-------+ +-------+
/ \
/ \
/ \
/ \
/ \
/ \
/ \
+-----+ +-----+ +-----+
|Burst| |Burst|...|Burst|
+-----+ +-----+ +-----+
HIPPI Logical Framing HierarchyThe Source asserts PACKET for the duration of a Packet transmission, deasserting it to indicate the end of a Packet. A sequence of Bursts comprise a Packet. To send a burst, a Source asserts the BURST signal for 256 clock periods, during which it places 256 words of data on the DATA lines. The first or last Burst of a Packet may be less than 256 clock periods, allowing the transmission of any integral number of 32 or 64 bit words in a Packet 🧩
The READY signal is a pulse four or more clock periods long. Each pulse signals the Source that the Destination can receive one Burst. The Destination need not wait for a burst before sending another READY if it has burst buffers available; up to 63 unanswered READYs may be sent, allowing HIPPI to operate at full speed over distances of many kilometers 🚀 If a Source must wait for flow control, it inserts idle periods between Bursts.
+------------------------------------------------+
REQUEST---+ +----
+--------------------------------------------+
CONNECT---------+ +--
+---------------------------------------+
PACKET-------------+ +----
+-+ +-+ +-+ +-+ +-+ +-+ +-+ +-+
READY------------+ +---+ +---+ +---+ +---+ +---+ +---+ +---+ +--
+-------+ +-------+ +-------+ +-----+
BURST--------------+ +-+ +-+ +-+ +--------
DATA------I-field----DATA------DATA------DATA-----DATA----------
HIPPI Signal Timing DiagramSerial HIPPI
There is no ANSI standard for HIPPI other than the parallel copper cable version 🤓 However, an implementors' agreement exists, specifying a serial protocol to extend HIPPI signals on optical fiber or coaxial copper cable. Serial links may be used interchangeably with parallel links to overcome HIPPI distance limitations. They are transparent to the Source and Destination, except for the possibility of longer propagation delays ⏳
I-Field and Switch Control
The REQUEST, CONNECT and I-field features of HIPPI-PH were designed for the control of switches as described in HIPPI-SC 🎛️ A switch is a hub with a number of input and output HIPPI ports. HIPPI Sources are cabled to switch input ports, and switch output ports are cabled to HIPPI Destinations. When a HIPPI Source requests a connection, the switch interprets the I-field to select an output port and electrically connects the HIPPI Source to the HIPPI Destination on that port. Once connected, the switch does not interact with the HIPPIs in any way until REQUEST or CONNECT is deasserted, at which time it breaks the physical connection and deasserts its output signals to both sides. Some existing switch implementations can switch connections in less than one microsecond ⚡ There is the potential for as many simultaneous connections, each transferring data at HIPPI speeds, as there are input or output ports on the switch. A switch offers much greater total throughput capacity than broadcast or ring media 👑
31 28 26 23 11 0
+-+---+-+-+---+-+-----------------------+-----------------------+
|L| |W|D|PS |C| Source Address | Destination Address |
+-+---+-+-+---+-+-----------------------+-----------------------+
HIPPI-SC I-field Format (Logical Address Mode)
L = Locally defined (1 => entire I-field is locally defined)
W = Width (1 => 64 bit connection)
D = Direction (1 => swap Source and Destination Address)
PS = Path Selection (01 => Logical Address Mode)
C = Camp-on (1 => wait until Destination is free)HIPPI-SC defines I-field formats for two different addressing modes. The first, called Source Routing, encodes a string of port numbers in the lower 24 bits. This string specifies a route over a number of switches. A Destination's address may differ from one Source to another if multiple switches are used 🗺️
The second format, called Logical Address Mode, defines two 12 bit fields, Source Address and Destination Address. A Destination's 12 bit Switch Address is the same for all Sources. Switches commonly have address lookup tables to map 12 bit logical addresses to physical ports. This mode is used for networking 🌐
Control fields in the I-field are:
L: The "Locally Defined" bit 🎀 When set, it indicates that the I-field is not in the standard format. The meaning of bits 30-0 is locally defined.W: The Width bit 🦋 When set, it requests a 64 bit connection through the switch. It is meaningless if Cable B is not installed at the Source. If W is set and either the Source or the requested Destination has no Cable B to the switch, the switch rejects the connection. Otherwise, the switch connects both Cable A and Cable B if W is set, or Cable A only if W is clear. This feature is useful if both Source and Destination implementations can selectively disable or enable Cable B on each new connection.D: The Direction bit 🔄 When set, it reverses the sense of the Source Address and Destination Address fields. In other words, D=1 means that the Source Address is in bits 0-11 and the Destination Address is in bits 12-23. This bit was defined to give devices a simple way to route return messages. It is not useful for LAN operations.PS: The Path Selection field determines whether the I-field contains Source Route or Address information, and, in Logical Address mode, whether the switch may select from multiple possible routes to the destination. The value"01"selects Logical Address mode and fixed routes 🛤️C: The Camp-on bit requests the switch not to reject the connection if the selected Destination is busy (connected to another Source), but to wait and make the connection when the Destination is free 🏕️ 💜
Appendix B -- How to Build a Practical HIPPI LAN ✨
okay so this appendix is where they actually show you how to build a HIPPI LAN 🔧 the other section is about what the host sees, but this one is the whole network setup bestie 💅 here's the build we're going with:
- A tree structure with a central HIPPI-SC compliant hub and optional satellite switches
- Each satellite is connected to the hub by just one bidirectional HIPPI link.
- Serial HIPPI or transparent fiber optic HIPPI extender devices may be used in any link.
- Some satellites may be a particular switch product which is not HIPPI-SC compliant.
- Host systems are attached either directly to the hub or to satellites, by single bidirectional links in which both HIPPI cables go to the same numbered switch port.
- A server system is attached to the hub. It provides multicast and ARP services.
- All options of the Internet Draft are supported. Hosts may use any form of address resolution: manual configuration, ARP with multicast, or ARP with a server.
Switch Address Management
okay the address space is giving simple 🎀 Switch addresses use a flat address space. The 12-bit address is split into 6 bits of switch number and 6 bits of port number.
11 5 0
+-----------------------+-----------------------+
| Switch Number | Port Number |
+-----------------------+-----------------------+Logical Address Construction
this part is so clean ✨ Switches can be numbered however you want. A host's address is the number of the switch it's plugged into directly, plus the physical port number on that switch where its input channel connects.
In the singly-connected tree, there is exactly one path between any pair of hosts. Every satellite has to connect straight to the hub, so the longest possible path is three hops and the shortest is one. Each HIPPI-SC compliant switch gets programmed to map every host switch address to the right output port. That's either the port the host is directly attached to, or a port linked to another switch on the way there 🦋
Special Treatment of Nonstandard Switches
okay the drama starts here 🙈 There's one commercially available switch that was built before HIPPI-SC was drafted, so it's not fully compliant. It's in common use, so it's worth making some special provisions so it can work in a HIPPI LAN. This switch only supports the Source Route mode of addressing, with a four bit right shift that can be turned off by a hardware switch on each input port. Addresses can't be mapped. The switch doesn't support the "W," "D," or "PS" fields of the I-field, so it ignores what's in them. Using this switch as a satellite means the hosts plugged directly into it have to deviate a little from normal I-field usage. Hosts on standard switches aren't affected at all 💖
For a destination on a non compliant satellite, the satellite only uses the least significant four bits of the I-field as the address. The address holds the destination's physical port number in its least significant bits, so that's the port that gets selected. Nonstandard switches should have I-field shifting turned off at the input from the hub, so the destination host sees its correct switch address in the I-field when it does self-address discovery. I-field shifting has to be turned on for the satellite on each input port that a host is attached to.
Hosts on nonstandard satellites have to deviate from normal I-field usage when they connect to hosts on another switch. It's suggested that every host implementation has this capability for as long as the nonstandard switches are still around 🫶 The host has to know, through some manual configuration method, that it's connected to a nonstandard switch, and it has to have its "link port" number. That's the number of the port on the satellite that connects to the hub.
The normal I-field format for a 32-bit connection, per the Internet Draft, looks like this:
31 26 23 11 0
+---------+---+-+-----------------------+-----------------------+
|0 0 0 0 0|x 1|C| Unused | Destination Address |
+---------+---+-+-----------------------+-----------------------+The special I-field format looks like this:
31 26 24 15 4 3 0
+---------+---+-+---------------+-----------------------+-------+
|0 0 0 0 0|x 1|C| Unused | Destination Address | Link |
+---------+---+-+---------------+-----------------------+-------+So this I-field gets altered by shifting the lower 24 bits left by four and adding the link port number. Camp-on is optional, and the PS field is set to 01 or 11 (the host's call) as if the switch supported logical address mode. All other I-field bits are set to zero. When the host asks for a connection with this I-field, the switch picks a connection through the link port to the hub, then shifts the lower 24 bits of the I-field right by four bits. The link port number gets thrown away, and the I-field that goes to the hub is a proper HIPPI-SC I-field selecting logical address mode. it's giving a whole glow-up arc ✨
A host on a nonstandard satellite can use the special I-field format for every connection request. But if it's connecting to another host on the same satellite, that takes an unnecessarily long trip through the hub and back 😭 If you want an optimization, the host can be given extra info so it can use the standard I-field format when it talks to another host on the same switch. That info could be a list of the other hosts on the same switch, or the details of how the address is formed, plus the switch number of the local satellite. With that, the host can look at a switch address and figure out whether the destination is on the local switch. This optimization is fairly complicated and may not always be worth it 🧚
Server Algorithm
the server is literally the main character of this section 👑 Different host implementations can take any of three approaches to address resolution:
- Manual configuration, no ARP
- Send ARP requests but expect a server to reply on its behalf
- Send ARP requests and expect to receive them via multicast.
If the network has a server that can do both multicast and ARP service, and that knows the services each host expects, then all of them can coexist on the same net 🌈
The HIPPI-SC compliant switches are programmed to route the HIPPI-SC "broadcast" address FE0 (hex) to the server's port. A human network administrator gives the server the following info to start:
- The list of all addresses eligible to be used by network hosts
- The list of addresses that should not receive multicast messages (a subset of list 1). This is also the list of all hosts that either do manual configuration or expect a server to answer ARP requests.
- The list of addresses of hosts that do manual configuration and do not send ARP requests (a subset of list 2) with the IP address corresponding to each one.
The server keeps an address resolution cache that it sets up from list 3 (the manually configured hosts). It keeps adding to its cache as other hosts send ARP requests 💅
When the server gets a message sent to the broadcast address FE0, it:
- Repeats the message to all addresses in list 1 but not in list 2
- If the message is a HIPPI-LE
AR_Requestwith a piggybacked ARP Request, updates the cache with info about the sender. - If the message is a HIPPI-LE
AR_Requestwith a piggybacked ARP Request, the target system has an entry in the cache, and the target is in list 2, responds to the ARP request.
Server Optimizations
ok these shortcuts are so smart 🧠
- The server could be given a topological map of the hub and satellites, and it could build list 1 from that.
- If all the hosts in list 2 ignore ARP messages as required in the Internet Draft, list 2 may be eliminated and the server can respond to all ARP requests (redundant replies may be sent).
Sharing Switch Hardware With Other Devices
Some host channels and peripheral devices connected to the switches might use protocols other than IP, and they don't take part in the LAN. Since connections in a switch are independent, these apps can share switch hardware with zero effect on LAN operation. To make sure it works:
- The server's lists of addresses should not include addresses for ports that aren't used by LAN links or hosts.
- If non-LAN applications use paths between switches, separate links should be installed for them so they don't use the same inter-switch links the LAN does.
References
[1] ANSI X3.183-1991, High-Performance Parallel Interface - Mechanical, Electrical and Signalling Protocol Specification (HIPPI-PH).
[2] ANSI X3.210-199X, High-Performance Parallel Interface - Framing Protocol (HIPPI-FP).
[3] ANSI X3.218-199X, High-Performance Parallel Interface - Encapsulation of IEEE 802.2 (IEEE Std 802.2) Logical Link Control Protocol Data Units (802.2 Link Encapsulation) (HIPPI-LE).
[4] ANSI X3.222-199X, High-Performance Parallel Interface - Physical Switch Control (HIPPI-SC).
[5] Postel, J., "Internet Protocol", RFC 791, USC/Information Sciences Institute, September 1981.
[6] Postel, J., and Reynolds, J., "A Standard for the Transmission of IP Datagrams over IEEE 802 Networks", RFC 1042, USC/Information Sciences Institute, February 1988.
[7] IEEE, "IEEE Standards for Local Area Networks: Logical Link Control", IEEE, New York, New York, 1985.
[8] Reynolds, J.K., and Postel, J., "Assigned Numbers", RFC 1340, USC/Information Sciences Institute, July 1992.
[9] Plummer, D., "An Ethernet Address Resolution Protocol - or - Converting Network Protocol Addresses to 48.bit Ethernet Address for Transmission on Ethernet Hardware", RFC 826, MIT, November 1982.
[10] Finlayson, R., Mann, T., Mogul, J., and Theimer, M., "A Reverse Address Resolution Protocol", RFC 903, Stanford University, June 1984.
[11] Katz, D., "A Proposed Standard for the Transmission of IP Datagrams over FDDI Networks", RFC 1188, Merit/NSFNET, October, 1990.
[12] IEEE, "Draft Standard P802.1A--Overview and Architecture", 1989.
[13] Mogul, J.C., and Deering, S.E., "Path MTU discovery", RFC 1191, Stanford University, November, 1990.
Security Considerations
okay this is the one section we can't skim 🚨 Security issues are not discussed in this memo.
Authors' Addresses
John K. Renwick Cray Research, Inc. 655F Lone Oak Drive Eagan, MN 55121
Phone: (612) 683-5573
Mailing List: (none) EMail: jkr@CRAY.COM
Andy Nicholson Cray Research, Inc. 655F Lone Oak Drive Eagan, MN 55121
Phone: (612) 683-5473
Mailing List: (none) EMail: droid@CRAY.COM
📦 where this page comes from
- ietf: rfc1374 1992-10
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms