RFC 635

Assessment of ARPANET protocols

๐Ÿ“… published April 1974, from ietf

An Assessment of ARPANET Protocols ๐Ÿ’–

Okay bestie, this is RFC 635 ๐Ÿ“œ, and it's a Network Working Group document by Vinton G. Cerf from Stanford University (NIC 30489). It's dated May 1974. The header doesn't list any category, and it doesn't say this RFC obsoletes or updates anything. It's basically a vintage design review of how the ARPANET talks to itself, and it is iconic โœจ.

ABSTRACT ๐Ÿฆ‹

This paper explains why the ARPANET communication protocols were worth redesigning, both in theory and in practice ๐Ÿ”ฅ. It covers multipacket messages, Host retransmission, duplicate detection, sequencing, and acknowledgment. It also proposes simplifications to the IMP/IMP protocol, assuming new Host level protocols get adopted. You'll probably need to already know the current protocol designs, since a lot of the arguments point at specific details in them ๐Ÿค“.

Introduction. ๐ŸŒธ

Okay so the ARPANET (the Advanced Research Project Agency resource sharing computer network) [6] is basically a story about protocols being studied, built, and put into use ๐Ÿงš. Early on, the team came up with a bunch of big ideas. There's protocol layering, which means separating the different levels of network transmission into their own functional jobs. There's the notion of bilateral rendezvous for setting up Host-to-Host connections [l,2]. And there's the Network Virtual Terminal, which helped specify a Terminal-to-Host protocol [3,14]. Those were all major moments ๐Ÿ’….

The design teams were often working without a map. Sometimes they needed agreements nobody had ever thought about before, like common protocols so that different operating systems and hardware could talk to each other ๐Ÿซถ. Looking back, the fact that it came together at all is kind of astonishing, and a lot of credit goes to the people who committed themselves to putting the ARPANET together ๐Ÿ‘‘.

Over the five years since the ARPANET started, we've learned a LOT about how the protocols in use actually design and behave ๐Ÿ˜ญ. The Imp-to-Imp protocol [4] has been revised continuously, and the HOST/IMP interface specification [5] has been changed a little. Looking back with what we now know, it makes sense to rethink some parts of the designs and implementations currently in use ๐Ÿค”. On top of that, national computer network projects are popping up all over the world, which makes it clear that countries need to cooperate on communication protocol design so international connections can happen ๐ŸŒˆ.

This paper is about why the HOST-to-HOST, IMP-to-IMP, and HOST/IMP communication protocols should be redesigned in the ARPANET ๐Ÿ’œ. It also includes analyses of the theoretical throughput and delay you can get from existing protocols, plus a look at some of their weaknesses ๐Ÿ“.

The basic conclusions are:

  • a) Multipacket message facilities can be eliminated without losing potential throughput, and that also simplifies IMP software. [8] ๐Ÿง 
  • b) If the destination IMP orders the messages it delivers to a destination HOST, that can lead to a lockup condition. It's similar to the reassembly lockup from an earlier version of the IMP protocol with multipacket message reassembly [7] ๐Ÿ’€. Hosts have to order arriving messages anyway, so the IMP doesn't need to do it ๐Ÿ™ˆ.
  • c) The HOST/IMP protocol could be changed to allow arbitrarily long messages to be sent from HOST to IMP, as long as the destination IMP doesn't have to reassemble or re-order the arriving packets before handing them to the HOST โœจ.
  • d) Host level retransmission, positive end-to-end acknowledgments, error detection, duplicate detection, and message ordering can eliminate the need for many of the IMP/IMP protocol's features. It can also replace the Request for next Message (RFNM) facility in the present HOST/IMP protocol ๐Ÿฆ‹.
  • e) The flow control mechanism in the current HOST-HOST protocol can lose synchronization because of message loss or duplication ๐Ÿ˜ฌ.
  • f) Host level connections should be full duplex ๐Ÿ”ฅ.
  • g) You don't need a separate HOST/HOST control connection if control information rides in the header of each Host transmission ๐ŸŽ€.

Throughput Considerations. ๐Ÿ“

Here's the tea ๐Ÿซ–: the IMP subnet can deliver up to 80 kb/sec between pairs of Hosts^, but basically no application using Host software has ever hit that number ๐Ÿ’€. In 1971, an experiment between Tinker and McClellan Air Force Bases reached burst rates as high as 40 kb/sec. It did that with a non-standard Host/Host protocol, which sent data over multiple logical connections and used Host level re-assembly and acknowledgement for reliable, ordered transmission^^ ๐Ÿ‘‘.

The analysis that follows shows that the current Host/Host protocol can't offer more than 40 kb/sec on a single connection, because of its message format overhead ๐Ÿ˜ญ. It also shows that this number drops hyperbolically if the communicating Hosts are separated by several IMPs ๐Ÿ“‰.

The main reason Host/Host throughput depends on distance (hop count) is the "message-at-a-time" Host/Host protocol ๐ŸŽ€. It means that on a given connection between processes in the Hosts, only one message ranging from 0-8063 bits of data can be outstanding at any moment ๐Ÿงโ€โ™€๏ธ. When the Host/Host protocol was first designed, the IMPs gave you up to 256 simplex logical links between pairs of Hosts ๐ŸŒธ. If a message went over a link (and there was a one-to-one relationship between a link and a connection), that link stayed blocked until the destination IMP sent a RFNM saying the message had been delivered to the Host. Of course, there was a time-out in case the RFNM never showed up โฐ.

^ Unpublished measurement experiments at UCLA run by R. Kahn and V. Cerf confirmed this. ^^ Unpublished measurement data obtained by V. Cerf at the ARPA Network Measurement Center, UCLA.

The IMP protocol has changed a lot since then. It now allows up to n messages^ to be outstanding between pairs of IMPs, no matter which links are used or which Hosts are communicating ๐Ÿ”ฅ.

That last bit means there can be some interference among Hosts connected to the same IMP, if those Hosts are talking to the same destination IMP ๐Ÿ™ˆ.

The Host/Host protocol hasn't been updated to take advantage of multiple outstanding messages, so it can't reach the maximum possible throughput ๐Ÿ˜ญ. Figure 1 shows how the timing of a multipacket message looks as it passes through several IMPs from source to destination โœจ.

        ------------------------------------
IMP(0)  | pkt(0) | pkt(1) | ... | pkt(m-1) |
        ------------------------------------
        |        | ------------------------------------
IMP(1)  |        | | pkt(0) | pkt(1) | ... | pkt(m-1) |
        |        | ------------------------------------
        |        | |                    :
        |        | |                    :
        |        | |        ------------------------------------
IMP(h-1)|        | |        | pkt(0) | pkt(1) | ... | pkt(m-1) |
        |<------>| |<-      ------------------------------------
            \     \
             \     `---> propagation delay from IMPO to IMPl
              `-->  packet transmission delay

Packet handling by h IMPs for an m-packet message

                    Figure 1
^ currently four, this limit being imposed by IMP buffer space.

Figure 1 is pretty naive in a few ways ๐Ÿ™ƒ. First, it doesn't show any interfering traffic, and it doesn't show packets getting out of order or taking alternate paths. Second, it assumes every packet is the same maximum size. The figure also leaves out the transmission delay to and from the Hosts. So the results of the analysis will be a bit optimistic ๐ŸŒˆ.

The logical connection between Hosts is only busy for m packet times out of every h+m-1 packet times ๐Ÿฆ‹. The source IMP is busy for m packet transmission times while it sends the message to a neighboring IMP. The last bit of the first packet reaches the destination IMP after h packet transmission times (not counting propagation delay), and the remaining m-1 packets finish arriving after m-1 packet transmission times. The source Host can send another message only after it gets a RFNM from the destination IMP. That RFNM is actually sent after the message has been reassembled, the first packet has been delivered, and the destination IMP has enough free buffer space for another maximum length multipacket message^^^. So a new transmission can't happen until at least h+m-1 packet times later, which means the fraction of busy time is just m/(h+m-1) ๐Ÿ’….

The actual bandwidth between IMPs drops from 50 kb^^^^ to 40 kb, because of the overhead bits needed for Host/Host and IMP/IMP control ๐Ÿ˜ฌ. IMPs also send periodic routing messages to all their neighbors (every .640 seconds)^^^^^, and those eat up even more bandwidth ๐Ÿ“‰. We can estimate the nominal fraction of the 50 kb/sec bandwidth available from source to destination IMP, then multiply that by the fractional busy time per connection. That gives an optimistic upper bound on maximum throughput per connection ๐ŸŽฏ.

^^^ If no space is available after 1 second, the RFNM is sent anyway. ^^^^ Some IMPs have 230 kb/sec lines, or 9.6 kb/sec, but most have 50 kb/sec. ^^^^^ This interval depends on line speed and load, and it may be as low as 128 ms.

Analysis of Expected Throughput Bounds. ๐Ÿงฎ

Let T be the number of bits of text to be transmitted by a Host whose natural word length is W bits ๐Ÿ’–. The Host/Host message format has a 32 bit leader, then a 40 bit prefix, then the text to be sent. We'll assume a sending Host transmits an integral number of its words, including the 72 bits that come before the message text ๐ŸŽ€. The Host/IMP interface also appends a one bit to each message, followed by as many zeroes as needed to make the whole thing an integral number of 16 bit IMP words (the IMP is a Honeywell 316 or 516 computer) ๐Ÿ–ฅ๏ธ.

The total number of bits in a Host message whose text contains T bits is given by equation 1 โœจ.

          M(T,W) = B1 (T) + B2 (T,W) + B3(T,W)                   (1)
          where B1(T)     = T + 72

                B2(T,W)   = - B1(T) mod W

                B3(T,W) = 1 + (-B1(T) - B2(T,W) - 1) mod 16

B1(T) is the number of bits in the Host message, counting the leader, prefix, and text. B2(T,W) is the number of bits needed to make B1(T) an integral number of Host words. B3(T,W) is the number of bits needed to make the total an integral number of 16 bit IMP words ๐Ÿงฉ.

The M(T,W) bits are turned into packets like this. The 32 bit leader is removed, and the remaining words are split into packets with no more than 1008 bits of data each. Each packet gets a 96 bit header, which includes the data from the 32 bit leader ๐ŸŽ. When these packets go to a neighboring IMP, they are enclosed

Part 1 ends here, mid-thought, because the packet enclosing details continue in Part 2 ๐Ÿฆ‹.

okay so each packet then gets tucked into a line control envelope ๐ŸŽ€ made of 48 bits of control octets plus a 24 bit cyclic checksum โœจ and we can work out how many bits all the packets need like this:

P(T,W) =( (M(T,W)-33)/1008 + 1 ) x 168 + M(T,W) - 32       (2)

the line transmission efficiency when sending T bits of Host text is:

LTE(T,W) = T/P(T,W)                                    (3)

and the expected fraction of time a logical link that is H hops long can be busy carrying a Host message of T text bits is:

                            P(T,W)
        EBF(T,W,H) = _____________________________
                     H*min[P(T,W) , 1176]+ max [P(T,W)-1176 ,0]          (4)

๐Ÿฆ‹ EBF(T,W,H) is a refinement of the fraction computed earlier, m/(m+h-1), and it's giving precision. The numerator is just the number of bits that must be transmitted from the source IMP. The denominator uses the min and max functions to handle the case where a message is less than a full single packet in length. In any case it takes H hops to deliver the first packet, and the remaining bits follow that packet until the entire message has arrived at the destination IMP. (Note that DLE may be doubled on the line, so this calculation assumes 'DLE' is never sent as data ๐ŸŒธ)

the routing messages need 1024 bits of text and 136 bits of packet header and line control, and every IMP sends them to all its adjacent neighbors every .640 seconds ๐Ÿ“ so the bandwidth for routing messages is (1160)/.640 = 1.8 kb/sec. no cap, that's a lot of routing chatter lol.

so the bandwidth you can expect for Host messages with T text bits, sent over H hops, is equation (5) ๐Ÿ‘‘

B(T,W,H) = EBF(T,W,H) x LTE(T,W) x (50-1.8) kb/sec           (5)

B(T,W,H) ignores a bunch of complicating factors though ๐Ÿ™ˆ

[Page 7]

RFC 635 An Assessment of ARPANET Protocols May 1974

  • a) delay for sending RFNM and implicit space reservation for multipacket messages to the source IMP.
  • b) propagation delays between Host/IMP and IMP/IMP.
  • c) queueing delays at intermediate IMPs.
  • d) retransmission delays.

still, B(T,W,H) is an optimistic estimate of the bandwidth you can expect using the current ARPANET Host/Host protocol. it's the best case bestie ๐Ÿ’œ

there is an implicit assumption that the packets of a multipacket message are not sent over alternate routes (for example, two 50 kb/sec paths). alternate routing in the IMP subnet is used to avoid congested areas, not to improve bandwidth, so this assumption is probably valid for the low traffic densities currently found in the ARPANET โœจ

B(T,W,H) has been plotted in figure 2 for a 32 bit Host (W=32) and a range of message text lengths and hops. as you can see, the effect of single message at a time transmission on a single logical connection is super marked for longer and longer hops ๐Ÿ˜ญ the curves would be even lower on a satellite channel, because of the long propagation delay (1/4 second up and down) for both the message and the returning RFNM.

[Page 8]

RFC 635 An Assessment of ARPANET Protocols May 1974

42 ||
   ||
40 |||
   |||
38 |||8
   1||78
36  12|78
    12||78
34  123|678
    123456 78
32  1 23456  78
    1 2 3456   78
30  1 2 3 45 6  7 8
    1  2 3 45  6  7 8
28  1  2  3 4 5  6  7 8
    1  2   3 4  5  6  7 8
26  1  2    3 4   5  6  7 8
     1  2    3  4   5  6  7 8
24   1   2    3  4    5  6  7  8
     1    2    3  4     5  6  7  8
22   1     2    3   4     5  6  7   8
     1      2     3   4     5   6  7   8
20    1      2      3   4      5   6  7    8
       1       2      3   4       5   6   7      8  (8 Packets, 7992 bits)
18     1        2       3   4         5   6      7  (7 Packets, 7000 bits)
        1         2       3    4         5       6  (6 Packets, 5976 bits)
16       1         2        3     4         5
          1         2           3     4          5  (5 Packets, 4984 bits)
14         1          2            3       4
            1           2             3          4  (4 Packets, 3960 bits)
12            1           2                3
                1           2                    3  (3 Packets, 2968 bits)
10                1             2
                    1                 2
8                                                2  (2 Packets, 1944 bits)
                            1
6
                                      1
4                                                1  (1 Packet, 952 bits)

2
   1    2    3    4    5    6    7    8    9    10

               Number of Hops

Single Link Source to Destination Host Throughput (32 bit word Host)
                          Figure 2

[Page 9]

RFC 635 An Assessment of ARPANET Protocols May 1974

The Multipacket Message Issue. ๐Ÿ”ฅ

the original IMP system only allowed one message at a time on a single link, so some means was needed to get higher bandwidth than single packet messages could provide. this was achieved, to some extent, by allowing up to eight packets in a single message ๐Ÿ’–

it was soon discovered, though, that a Host sending multipackets on separate logical links could cause a lockup condition at the destination. this was first described by R. Kahn and W. Crowther [7].^ basically, there might not be enough space at the destination to reassemble all the multipackets in transit on several links. the condition was self-sustaining if the Host kept transmitting, although the destination could discard unassembled multipackets after a time-out. the condition either backed up into the rest of the network, or at best caused loss of messages in the network ๐Ÿ˜ฌ

the solution to the multipacket reassembly lockup problem that was eventually implemented required the source IMP to reserve reassembly buffer space at the destination IMP before transmitting the multipacket ๐Ÿซถ

actually, this problem is just a case of a more general problem, which can be caused by the destination IMP sequencing of messages delivered to the Host.

Ordering of Messages. ๐Ÿงš

the IMP system guarantees that messages will be delivered to a destination Host in the same order that they left a source Host. this service can cause a lockup similar to reassembly lockup if enough messages are in transit to the destination IMP. single packet messages are sent without prior reservation to the destination, and if space is available for them, a RFNM is returned to the source IMP. if no room is available, an implicit reservation request is queued at the destination IMP. when space becomes available, an allocation message is sent to the source IMP, which retransmits the single packet message. the source IMP keeps a copy of the single packet message for retransmission until it either receives a RFNM from the first copy transmitted, or an allocate message indicating there is now room available for a second copy to be accepted.^

^ Kahn actually knew in 1967 that the condition could occur, but was unable to convince his colleagues until he actually locked up the network by using a message generator to flood the network in March, 1970.

[Page 10]

RFC 635 An Assessment of ARPANET Protocols May 1974

this scheme can fail if either a given Host has too many messages in transit, or if many Hosts, served by different IMPs, have too many messages in transit for the same destination. that's because the destination IMP will accept packets that arrive out of order and buffer them until they can be re-ordered for transmission to the destination Host ๐ŸŒˆ

presently, a source IMP only permits up to four messages (regardless of length) to be in transit for a given destination at a time. this essentially reduces the probability of a lockup, but it is not zero, since enough messages may be outstanding from different IMPs for the same destination to cause a lockup ๐Ÿ’…

such lockups are also protected against, by timing out undelivered messages at the destination and discarding them. the timeout is on the order of tens of seconds. even though the IMP subnet can recover from such conditions, it is apparent that Hosts must be prepared to retransmit messages occasionally, to recover from message loss caused by deliberate discarding of messages at the destination, or by catastrophic failures in which an IMP loses all its packets upon crashing ๐Ÿ’€

^ R. Kahn, L. Kleinrock, and H. Opderbeck point out that IMPs do not accept out-of-order packets, but do send allocates back for them. If room is also allocated for unreceived but anticipated in-order packets, no lockup will occur. If this step is omitted, then the implementation may fail.

[Page 11]

RFC 635 An Assessment of ARPANET Protocols May 1974

Host Retransmission, Sequencing, and Duplicate Detection. ๐ŸŽ€

the Host/Host protocol does not provide for retransmission. if it did, though, it would require the destination Host to detect duplicate transmissions and also verify the sequencing of arriving messages, since the destination IMP cannot, in the current scheme, detect that a Host has sent a duplicate message ๐Ÿค”

if you follow this line of reasoning, it becomes evident that sequencing of messages by the destination IMP is redundant and could be eliminated. furthermore, with ordering eliminated, multipacket messages could also be eliminated, as long as Hosts were permitted to transmit a sufficient number of single packet messages to achieve maximum potential bandwidth ๐Ÿฆ‹

along with Host retransmission, it is necessary to introduce some kind of end-to-end positive acknowledgment. the RFNM is currently sent by the destination IMP to the source Host and is taken to mean that a message has been successfully delivered to the destination Host (for multipacket messages, the RFNM is sent after the first packet has been delivered). it seems sensible to arrange a Host level acknowledgment that confirms delivery. in that case, the RFNM could also be eliminated โœจ

one might use RFNMs optionally as a debugging tool, turned off and on at will ๐Ÿ› ๏ธ

statistics taken from the ARPANET indicate that Host retransmission would rarely be required on account of message loss, but this is partly because of the retransmission and reservation facilities in the current IMP system ๐Ÿ“Š

[Page 12]

RFC 635 An Assessment of ARPANET Protocols May 1974

Flow Control. ๐ŸŒธ

if all end-to-end retransmission, duplicate detection, and sequencing are performed by Hosts, then it is essential that the source and destination Hosts agree on a maximum number of packets (or bits, octets, etc.) that can be outstanding at one time. otherwise, the destination Host may experience lockup problems similar to those now found in the destination IMP ๐Ÿ˜ฑ

the current Host/Host flow control scheme has several weaknesses, bestie.

first, it requires that special control messages be sent on a control connection that is distinct from the connection on which data is transmitted ๐Ÿ™„

second, it is an incremental scheme in which the destination Host allocates a certain number of bits and messages that may be sent by the source. both source and destination Hosts decrement these counts as messages are sent and received. to keep throughput up, the destination must periodically send allocations to the source to replenish its available buffer space. destinations with a small amount of buffer space (e.g., Terminal IMPs or TIPs) must do this fairly frequently, and so they generate considerable control traffic ๐Ÿฅน third, the loss of an allocation or the duplication of

...and that can totally cause a loss of synchrony between source and destination ๐Ÿ’€.

In an earlier paper [9], the author and R. Kahn pitched a sturdier flow control scheme that borrowed ideas from the French CYCLADES network [10] ๐Ÿฆ‹. Basically, the receiver hands out a window covering the span of sequence numbers the sender is allowed to transmit. The receiver's acknowledgments say what the largest sequence number received so far is (implicitly acknowledging everything before it) and also show the current width of the window โœจ. So the sender instantly knows which sequence numbers it can send next ๐Ÿ‘‘. The scheme also handles duplicate detection and reordering of messages ๐Ÿ’….

Acknowledgment and flow control info is sent "piggy-back" along with data flowing the other way on a full duplex logical connection, so you don't need a separate control connection just for that ๐Ÿซถ. For example, a message goes out with sequence number M and length L in octets. The receiver responds by acknowledging sequence number M+L and window size W. The sequence number of each message is the sequence number of the previous message plus that message's length in octets ๐ŸŽ€.

The receiver can vary the size of W without any serious adverse effect, and it can survive receiving duplicates or losing messages, thanks to the retransmission and duplicate detection the scheme permits ๐Ÿงš.

The sender is not permitted to transmit a message whose sequence number would exceed the sum of the last sequence number acknowledged plus the current window size, W, modulo the maximum sequence number plus one ๐Ÿ™ˆ.

Arbitrary Message Lengths. ๐ŸŒŠ

Until now, it has been implicit that multipacket messages are unnecessary for maintaining high throughput, as long as sufficient packets can be sent to fill the delay pipeline from source to destination Host. If the IMP system were programmed with knowledge of the Host/Host protocol so that it could create a properly formatted Host/Host header for each packet it transmits, given the initial header of an arbitrarily long message, then packets could be delivered out of order to the destination Host, so long as each one correctly identified the range of sequence numbers it contained. Since each octet of a message has an implicit sequence number, this would not be difficult to compute ๐Ÿงฎ.

An idea similar to this is found in the Very Distant Host Reliable Transmission Package: [appendix F, 5] in the current ARPANET, except in this case, a Host must know about IMP packet format ๐Ÿค”. It is debatable whether this would be a good idea, since changes in Host/Host protocol would require changes in IMP programming ๐Ÿ˜ฌ. But if it were implemented, then Hosts could send arbitrarily long messages. The destination Host would receive a collection of single packet messages, which it would then sequence as if they had been sent that way by the source Host in the first place ๐Ÿ”ฅ.

Simplex versus Full-Duplex Logical Connections ๐Ÿ’œ

The present Host/Host protocol implements simplex connections. The usage over the last five years seems to indicate that most often, two simplex connections are set up to act as a full duplex connection.

If Host level acknowledgments and flow control are implemented, then it is natural for them to be carried in the reverse direction of a full duplex logical connection. Furthermore, terminal to Host connections are necessarily full-duplex to allow for data to move in both directions ๐Ÿ–ฅ๏ธ.

Finally, by embedding control in the headers of returning traffic on the full duplex connection, the need for a separate control connection could be eliminated ๐ŸŽ€.

Connection Set-up. ๐Ÿ“ž

The current Host/Host protocol uses control messages sent on a special control connection to establish new connections. The procedure is called the Initial Connection Protocol or ICP [11]. The protocol is symmetric and requires that information be exchanged by both Hosts as to the names of the sockets at either end of the connection. This exchange precedes any flow of data. Other control messages are exchanged which determine the buffer space available at the receiver ๐Ÿงบ.

A proposal by D. Walden [12] suggests that this is largely unnecessary, as long as both sides can simultaneously send data identifying the source and destination sockets (Walden calls them Ports) along with the text of the messages ๐Ÿฆ‹.

A post office analogy is useful to describe what is intended ๐Ÿ’Œ. The source Host writes a letter and encloses it in an envelope addressed to the destination port, with a return port address. Either the destination port is willing to receive or it is not (e.g., it may not even be known to the destination Host). In the former case, the letter is acknowledged in the usual fashion. In the latter case, the letter is not acknowledged (port unprepared to receive), or it is rejected ("address unknown").

Since port addresses may be dynamically assigned to processes in a destination Host, it will probably be necessary to include a formal control exchange which indicates to the sender that a receive port is being closed, and the sender would be expected to acknowledge this. Similarly, the sender may end a transmission with the indication that the send port is being closed and the receiver would similarly acknowledge. Since Hosts do the sequencing, there can be no confusion as to when the closure is to take place. The rejection of an initial transmission can be made to look like the closure of the destination port so that the number of distinct control messages can be kept to a minimum. This method is similar to the one currently used in the ARPANET, but could be carried out via control bits in the Host level messages and thus eliminate the need for a special control connection ๐Ÿ™Œ.

Summary. ๐Ÿ“

Arguments have been presented in this paper which show that multipacket reassembly is not the best vehicle for achieving high throughput from Host to Host ๐Ÿ˜ค. The elimination of IMP reassembly as well as message sequencing by the destination IMP can permit considerable simplification of the IMP protocols, while simultaneously placing the burden of buffering, duplicate detection, and sequencing of messages on the Hosts which have the buffer space for this purpose ๐Ÿ’–.

Arbitrarily long messages could be sent by Hosts, at the expense of IMP knowledge of Host protocol. Eliminating the ordering requirement at the destination IMP also eliminates serious potential lockup conditions ๐Ÿ”’.

Host level positive acknowledgments can eliminate the erroneous use of the RFNM for this purpose, and permit a more robust protocol which need not depend upon perfect performance without message loss by the IMP subnet ๐ŸŒˆ.

Full duplex logical connections between ports in Hosts are more natural than the simplex connections presently used, and facilitate the elimination of the special control connection required in the current Host protocol ๐Ÿ‘‘.

Unresolved Problems and Issues. ๐Ÿšฆ

Even if a source and destination Host have adequate buffer space to permit a large number of messages (or packets, or octets) to be outstanding between them, the IMP subnet must have a way of combating congestion which may result from too rapid influx of data from a source Host, or from momentary congestion owing to the confluence of excessive traffic heading in the same direction, possibly to the same destination. Alternate routing strategies can help, but not completely solve the problem. One possibility is to insist that source Hosts monitor actual throughput achieved over the last few seconds (milliseconds?) and adjust output rate accordingly. Destination Hosts can monitor this throughput as well, and adjust the receive buffer space they allocate to the sender to reduce unnecessary retransmissions. The IMPs can simply discard traffic which cannot be buffered, knowing that the source will retransmit. IMPs which discard packets to eliminate congestion could even send short warning messages to source or destination (or both) to stimulate adjustment. This is a very sticky problem and involves issues such as payment by Hosts for retransmission ๐Ÿ’ธ.

Most strategies in use today involve limiting, a priori, the amount of data which a source Host is allowed to send (e.g., isarithmic network proposed by Davies [13]; maximum of n messages allowed by ARPANET IMPs). Measurement of throughput achieved by source and destination Hosts may be a good strategy in any case since it serves as a measure of quality of service provided by the packet switching network ๐Ÿ“Š.

In the ARPANET, the TELNET protocol [14] for terminal to Host communication has needed some way of signalling the Host in which the serving process resides that any accumulated data should be discarded up to the point of the "interrupt signal." This facility permits a remote user to abort or recapture control from an uncooperative serving process which has stopped accepting data. The current scheme involves the use of a special interrupt signal sent on the control connection, but there is a problem of synchronizing the interrupt request with the data in the pipeline. This signal could be carried in the control field of a Host message and would participate in the sequence numbering of the data, thus providing for synchronization. Since the Host operating system would process the message header before passing the data to the receiving port, the interrupt could bypass processing by the receiving process and thus provide the desired interrupt-like effect ๐Ÿšจ.

There are undoubtedly many other problems and issues which could not be mentioned in the scope of this paper, and the author would be pleased if these and the preceding commentary will stimulate discussion and thus further the general understanding of protocol requirements for distributed computer networks ๐Ÿง .

Acknowledgments: ๐Ÿซถ

Throughput and delay analysis: some of the basic ideas in this analysis are due to J. McQuillan (Bolt, Beranek, and Newman). Single packet re-ordering lockup: first called to the author's attention by P. Higginson (University of London). Host-Host Protocol Design: developed largely under the auspices of the International Network Working Group (IFIP TC6.1), and the author especially acknowledges contributions by R. Kahn, R. Metcalfe, L. Pouzin and H. Zimmerman, as well as S. Crocker, A. McKenzie, and R. Scantlebury. Numerous omissions and misstatements were detected by R. Kahn, L. Kleinrock and H. Opderbeck.

The author is grateful for the support of the Defense Advanced Research Projects Agency under contract DAHC-15-73C-0435.

References ๐Ÿ“š

  1. McKenzie, A. "HOST/HOST Protocol for the ARPA Network," Current Network Protocols, Network Information Center, Stanford Research Institute, Menlo Park, California, January 1972 (NIC8246).
  2. Carr, S. and S. Crocker, V. Cerf, "HOST-HOST Communication Protocol in the ARPA Network, AFIPS 1970, SJCC Proceedings, Vol. 36, Atlantic City, AFIPS Press, New Jersey, pp. 589-597 [somewhat out of date].
  3. Crocker, S. D., and J. Heafner, R. Metcalfe, J. Postel, "Function-Oriented Protocols for the ARPA Computer Network," AFIPS 1972 SJCC Proceedings, Vol. 40, Atlantic City. AFIPS Press, New Jersey, pp. 271-279. ๐Ÿ’–
  4. Heart, F. E., and R. E. Kahn, et al, "The Interface Message Processor for the ARPA Computer Network, AFIPS 1970 SJCC Proceedings, Vol. 36, Atlantic City, AFIPS Press, New Jersey, pp. 551-567. โœจ
  5. Bolt, Beranek and Newman, Inc., "Specification for the Interconnection of a Host and an IMP," BBN Report 1822, April 1973 (Revised). ๐ŸŽ€
  6. Roberts, L. and B. Wessler, "Computer Network Development to Achieve Resource Sharing," AFIPS 1970, SJCC Proceedings, Vol. 36, Atlantic City, AFIPS Press, New Jersey, pp. 543-549. ๐Ÿฆ‹
  7. Kahn, R. E. and W. Crowther, "Flow Control in a Resource Sharing Computer Network," Second Symposium on Problems in the Optimization of Data Communications Systems, Palo Alto, October 1971, p. 108-116 [also IEEE Transactions on Communication, June 1972]. ๐ŸŒธ
  8. Kahn, R. E., "Resource Sharing Communication Networks," IEEE Proceedings, Nov. 1972. ๐Ÿงš
  9. Cerf, V. G. and R. E. Kahn, "A Protocol for Packet Network Inter-communication," IEEE Transactions on Communication, vol. COM-22 No. 5, May 1974 p 637-641. ๐Ÿ‘‘

Page break, obviously, because the text turned the page:

                               [Page 20]

RFC  635           An Assessment of ARPANET Protocols              May 1974
  1. Pouzin, L., "Presentation and Major Design Aspects of the CYCLADES Computer Network," Data Networks: Analysis and Design, Third Data Communications Symposium, St. Petersburg, Florida, November 1973, pp. 80-87. ๐ŸŒˆ
  2. Postel, J., "Official Initial Connection Protocol," Current Network Protocols, Network Information Center, Stanford Research Institute, Menlo Park, California, January 1972 (NIC 7101). ๐Ÿ’…
  3. Walden, D., 'A System for Interprocess Communication in a Resource Sharing Computer Network, Communications of the ACM, 15, 4, April 1972, pp. 221-230 (NIC 9926). ๐Ÿซถ
  4. Davies, D., "Communication Networks to Serve Rapid Response Computers," Information Processing 68, Proceedings of IFIP Congress 1968, Vol. 2, Edinburgh, Scotland, 1968, North-Holland Publishing Co., Amsterdam, 1969, p. 650-658. ๐Ÿ“
  5. McKenzie, A. "TELNET Protocol Specification," Current Network Protocols, Network Information Center, Stanford Research Institute, Menlo Park, California, August 1973 (NIC 18639, NIC 18638 - Revisions). ๐Ÿ’œ

Okay so the bottom of the doc is the archive's own little housekeeping note, and it's lowkey iconic. ๐Ÿ™ˆ Here's the machine-readable header exactly as it was stamped:

       [ This RFC was put into machine readable form for entry ]
       [ into the online RFC archives by Roger D. Moore, with  ]
       [ assistance from William M. Stewart on Figures 1 and 2,]
       [ 11/2006 ]

Notes by Roger D. Moore regarding copy and formatting changes, bestie. He was SO thorough. ๐Ÿ˜ญ๐Ÿซถ

  • A] Page numbers zero and one added to text.
  • B] Paragraph indent in pages 1-3 is five; it is four spaces in pp 4++.
  • C] All pen marked underlining from paper copy has been discarded.
  • D] Footnotes: The original text used a sequence of superscript asterisks to mark a footnote. I have replaced all of these with the character ^ which does not otherwise appear in the document. I have used a sequence of underscore characters to separate text and notes at bottom of pages 3 4 5 10 11.
  • E] Formulae: On page six I have replaced symbols of the form "B subscript digit" with "Bdigit".
  • F] Forumla [2] page seven: I have rewritten this to eliminate horizontal line (division symbol).
  • G] Quarter symbol: page eight (last line) had a special symbol which I have replaced with 1/4.
  • H] Marginal notes: I have preserved formulae notes from page six. Others have been discarded.
  • I] Page numbers: I have left two blank lines after page header. Page footer is incomplete but has the form [Page 9] or [Page 99]. RFCs in the archive have some special character between the footer and header (ASCII formfeed?). I was unable to enter this character.
  • J] Reference 9: I have included page numbers since this paper is now published.
  • K] Figure 2 on page nine originally contained continuous curves in heavy pencil, redrawn in ASCII.

And that's a wrap on RFC 635, bestie!! ๐ŸŒˆโœจ๐Ÿ‘‘ The ARPANET crew really said "we're building the internet, no cap" and then wrote everything down so well that we can still read it today. Iconic. ๐Ÿ’…๐Ÿฆ‹

๐Ÿ“ฆ where this page comes from