RFC 647

Proposed protocol for connecting host computers to ARPA-like networks via front end processors

📅 published November 1974, from ietf

RFC 647 🎀 Title Block Tea

okay bestie, this is RFC 647 from the Network Working Group, written by M. A. Padlipsky (Michael Padlipsky) of MITRE-TIP and dated November 1974. Its NIC number is 31117 🌸 The title is A PROPOSED PROTOCOL FOR CONNECTING HOST COMPUTERS TO ARPA-LIKE NETWORK VIA DIRECTLY-CONNECTED FRONT END PROCESSORS ✨

Michael got advice from Jon Postel, Jim White (SRI-ARC) and Virginia Strazisar (MITRE), and general agreement from a whole squad: Tom Boynton (ISI), Frank Brignoli (NSRDC), John Day (CAC), Bob Fink (LBL), Carl Ellison (UTAH), Eric Harslem (RAND), Wayne Hataway (AMES-67), John McConnell (I4-TENEX), Ari Ollikainen (UCLA), Dave Retz (SCRL), Al Rosenfeld (CASE), Stan Taylor (BRL) and Doug Wells (MIT-MULTICS) 💜

The header doesn't name a category or status, and it doesn't say it obsoletes or updates anything.

The affiliations are only there to show who was involved with ARPANET. They don't mean those organisations approved the positions here 🙈 And the bit that matters most: No -- repeat NO -- general agreement should be read as covering Appendix 2, because that appendix is purely one person's viewpoint 💀

INTRODUCTION ✨

okay so, this proposal was written in the fall of 1974, and by then the ARPA network was SO accepted that a bunch of organisations were planning to hook their general purpose computers straight up to the ARPANET, or to link their systems using "ARPANET technology" 🌈 The authors had been in touch with the Air Force systems command, the Naval Ship Research and Development Center, the Defense Communications Agency ("PWIN" -- the Prototype World-Wide Military command and Control system Intercomputer Network), ARPA (The National Software Works), the AEC, and other government agencies 👑

The big thing these networks have in common is that they include systems with no counterparts on the current ARPANET. So the hardware "special interfaces" (between the host and the network Interface Message Processor) and, even more important, the Network Control Programs can't just be copied from working versions 😭 The systems in question include CDO 6600's, XDS Sigma 9's, Univac 494's, 1107's, 1108's, 1110's, and IBM 370's running operating systems with no current ARPANET counterparts 🍓

It's also widely accepted that designing and implementing an NCP for a "new" system is a major undertaking. So an immediate concern for these networks is using as much off-the-shelf hardware and software as is practicable 🧚 This paper looks at two such approaches. One is popularly assumed to be the way to go right now, and the other is one the authors feel is superior to the more widely known option 💅

FRONT-ENDING 🦋

so in the broader network community, the consensus is SO strong that front-ending is desirable that this topic barely needs discussing 🫶 Basically, a small machine (a PDP-11 is widely thought to be the most suitable) sits between the IMP and the host, which shields the host from the complexities of the NCP 💜

The perks are pretty obvious. It's more economical to develop a single NCP. "Outward" (User Telnet) network access is also provided by the front end acting as a mini-Host. There's potential for file manipulations on the mini-Host 🔥 Two operating systems are in advanced stages of development on the ARPANET for PDP-11's, and they'll clearly work well as bases for network front ends. So the hardware and software are copiable ✨ If we picture a model along these lines:

Host *** Front End --- IMP --- Network

then everything to the right of the asterisks can almost be taken as given 🌸

(Caveat: Note the "almost" well in last sentence neither ANTS nor ELF -- the two systems alluded to above -- is a completely finished product in the estimation of either their respective developers or of the knowledgeable ARPANET workers who have contributed to this report. Both are capable of being brought to fruition, though, and in a reasonable amount of time. We will assume ELF as the actual front-end system here for two reasons: apparent consensus, and current activity level of the development team. However, we have no reason to believe that readers who prefer ANTS would encounter substantive difficulties in implementing our proposal on it.)
(Explanatory notes: ANTS is an acronym for ARPA Network Terminal Support system; it was developed at the Center for Advanced Computation (CAC), University of Illinois. ELF is not an acronym (It is said to be German for "eleven"); it was designed at the Speech Communications Research Lab (SCRL), Santa Barbara, California.)

THE RIGID FRONT-END ALTERNATIVE 💅

okay so, going back to the model above, the popular view of those asterisks is to have the front-end system simulate a well known device for each Host. That's typically a remote job entry station along the lines of the 200UT on the CDC 6600 🎀 The upside is that it effectively needs no software changes on the Host system 🙈

The authors call this approach "rigid" because of one big implication: the Host system is limited to handling data to and from the network only in ways its system already supports. (For example, if you simulate a card reader, your data will necessarily be treated as batch input. If you simulate a terminal, it'll necessarily be treated as time-sharing input 😭)

Now, somebody might argue that Host software changes are only being avoided so you can "get on the air" quickly, and that those changes could come later to allow unconstrained channelling of network data within the Host. The authors say this reasoning can be refuted if it's shown that an alternative exists which is essentially as quick to implement and doesn't need the waste motion of building known-device simulation hardware and software for each new Host, only to eventually skip the simulation in the Host anyway 💜

The major advantage claimed for the rigid front-end approach, apart from quickness to implement, would be embarrassing if it were true. The possibility exists that the "new" Host's operating systems or system programming staffs are so intractable that avoiding Host software changes is a necessity rather than a desire. The authors certainly hope neither is the case and have no reason to believe it is, but they acknowledge that such possibilities exist as meta-issues to this report 🌈

DISAVANTAGES OF THE RIGID FRONT-END ALTERNATIVE ✨

okay so the rigidity argument from above deserves a deeper look 🫶 The biggest problem with only talking to the Host in fixed ways is that you lose functionality 😭 Sure, "Telnet" and "RJE" stuff can be done by simulating a known device (we have deep reservations about file transfer though 💀), but there's way more going on in practice and in theory than just the Host's time-sharing and batch monitors. Teleconferencing is an obvious example, and graphics is another. Neither one fits neatly into the setting an operating system is likely to assume for a Telnet or RJE connection 🎀

It gets more iconic too. The ARPANET is just starting to develop a view of "process-to-process" protocols, where cooperating programs on different kinds of systems talk over network sockets, which is a real use of sockets as interprocess communication 🦋 Picturing a (simulated) line printer as an abstract "port" is hard without a lot of contortion of the existing operating system 🙈 To sum up this whole pile of objections: a simulation of a known device might be cheaper than a large enough number of phone calls, but it's not networking 💅

For that matter, it's not even clear the goal of no Host software changes can be met 😬 One ARPANET system used a PDP-15 as a front end to a PDP-10. One of the authors tried to log in over the net and got asked what type of terminal he was on, because the front end had been attached at the wrong point in the PDP-10's terminal handling code 💀 He's a battle-scarred Telnet veteran, so he gave good answers for describing a "Network Virtual Terminal" 🧚 Unfortunately the NVT had no counterpart among the Host's normal local terminals, and when he tried Telnet control functions like "don't echo, I'm at a physically half-duplex terminal," things got really confused 😭 He later ended up near that Host and spent an afternoon explaining the philosophy and importance of the NVT to the Telnet protocol. The site people were super appreciative and cooperative 🫶 We haven't checked, but we assume the site is probably usable from the ARPANET now 🌸

The real point is this: operating systems tend to make lots of assumptions about their environments, often without even realizing it 🔥 That's especially true for terminal types, and there's no guarantee that the systems involved could "do the right thing" if they were front-ended by simulating a known device. The only way around that would be a simulation so painstaking that all you'd get is an expensive way of adding an RJE station, period 🙃

Less abstract stuff matters too 💖 A minicomputer, even with "third-generation" software, is not as free and convenient to program in as a full-scale Host, so writing these simulations won't be trivial software engineering 😤 And if front-end experts write the simulation software, they'll go through repeated start-up transients while they learn what the various Hosts expect from them 🍓 If personnel from those Hosts are kept out of actively attaching to the network, there will be natural (and understandable) resentment about the "intrusion" the network will seem to be. Systems programmers have territorial feelings too, let's be real 👑

On a more practical level, needing to simulate more than one known device, or even just how complex a single simulation is, could push you toward a bigger PDP-11 configuration than would otherwise make sense 💜 There are other reasons to make every front-end processor as big as possible, but dollars do matter 💸 Numbers are a problem on the speed side too. Line speed for known-device simulations can be pretty low. The 200UT, for example, sits on a 4800 baud line, which is a rough mismatch with a 50,000 baud communication subnet 🙈 (Sure, there's the 40,800 baud line into the 6600, but it's not expected to carry interactive devices, so the existing software won't send the data to the "right place"... 😭) And no experienced ARPANET protocol designer would ignore that there will probably need to be a flow control discipline between the Host and the front-end processor anyway. That makes the no-change-to-Host goal look pretty dubious 🥲

Okay, after all that it might be a little harsh to point out one more layer of difficulty, but we feel strongly that it needs addressing 💀 The question has to be asked: who will do the front-end implementations? This kind of work is barely within the scope of CAC or SCRL 😅 As Appendix 2 will argue, it's super important that whoever does the job already has ARPANET expertise. We don't know of a single case where "outsiders" successfully came aboard without becoming "insiders" along the way, and that's neither easy nor cost effective 🦋

So yeah, it's at least reasonable to look at an alternative to the rigid front-end approach. No matter how much weight you give any one of these disadvantages, taken together they suggest that the known-device simulation tactic is not a panacea 🌈

THE FLEXIBLE FRONT-END ALTERNATIVE 🦋

Our alternative is built on a principle that's actually been around since at least a month before the ARPANET started running User and Server Telnets on a regular basis 🎀 The idea is that it would be lovely to offload as much of the NCP from the Host as possible, because Hosts are supposed to have better things to do with their CPU cycles than deal with control messages from other Hosts 💅 Especially when 90% of those control messages are just ALL(ocate) commands 🙄 That led to the insight that all a Host "really" has to do is associate sockets with processes, and pass data along socket connections ✨

The flexible front-end approach is basically an update of those 1971 ideas, and it goes like this 🌸

  • Drop the hard-and-fast goal of NO changes to Host software, in favor of the more realistic goal of MINIMAL changes to the Host 🫶
  • Attach the front-end processor to any convenient high-speed "channel" ( / "port" / "multiplexer" / "line" / "cable") 🔌
  • Let the front-end processor handle the NCP 🧚
  • Define an extremely compact protocol for the Host and front-end to follow, called the H-FP 🍓
  • Let the Host's H-FP module distribute the data appropriately within its operating system, because the H-FP makes it clear where the data should go 💖

And if you have to ram the data into the teletype buffers, it's still cleaner than trying to do interprocess communication over a card reader 💀 (The H-FP is described in less bald terms in Appendix 1 📖)

I know that might sound pretty uncompromising, and honestly kind of cryptic 🙈 But between the advantages it brings and the more comprehensive description that follows, we genuinely feel it's a superior basis for solving the big problem: how best to attach "new" Hosts to an ARP-like net 👑 🌈

ADVANTAGES OF THE FLEXIBLE FRONT-END ALTERNATIVE 💅✨

Okay bestie, the big win here is flexibility 🦋. Sure, you could build minimal versions on some Hosts, but even the most minimal one is still just as powerful as the rigid front-end approach. And when people want more features, they're super easy to add. 🎀 Why? Because programs on the Host can "get their hands on" data from the net (and send data back out) in a totally natural way. The rigid approach only lets you do stuff with data the way a card reader would hand it over. With the flexible approach, the Host is genuinely doing networking. 🌸 A major "real" ARPANET server site was so into it that they wanted to implement the H-FP after a five minutes' worth of blackboard explaining with two of the authors. Iconic, honestly 😭🔥

Another perk is that the people at the new sites get to be part of the process as they join the network. 🫶 This matters emotionally too. If known-device simulation were used, the network could feel like an alien intrusion pushed onto the site's systems folks. It's also technically better to have per-site coding done by "experts," because if the per-site tailoring happened only in the mini, it wouldn't be as good. 💜 Remember the PDP-15 to PDP-10 attempt we talked about earlier? That one kinda counts as front-ending done in ignorance of both the Host's operating system conventions and the ARPANET conventions 💀 Not only should that be avoided by bringing in experts on the target operating systems while attaching to the network, there are practical reasons too: we estimate that adding a minimal Host-Front End Protocol routine to a given operating system would take no longer than the same few man months it would take to add a new known-device simulation package to the mini. So we also see scheduling advantages, on top of the more abstract ones we already mentioned. 🍓 Plus, it should be a friendlier environment to program in on the Host than in the mini. (To be clear, this isn't saying the ELF isn't a good environment to program in. It's just the "obvious" point that if the big systems didn't have convenient programming environments, we wouldn't have them 🙈)

We also need to look more closely at flow control, which we brushed on earlier 👀. The known-device simulation approach seems to assume the mini will handle this, and that the simulation will know whatever flow control discipline the host and the simulated physical device follow. But the one device "everybody knows" (the CDC 200UT) runs on a 4800 bit-per-second line, while the IMP subnetwork runs on 50,000 bps lines. So the mismatch needs attention, especially since only one process on the Host is usually tied to a known device, but the network is actually sending data for many processes. ⚡ Our approach lets you impose a very direct, simple flow control discipline without getting tangled up in per-Host idiosyncrasies. (More elaborate, potentially more efficient, flow control disciplines are also an option.) So we can just pick the best line speed available on a particular Host and attach to it. Easy 💖

Here's another practical bonus 🧚: the min's H-FP module can be shipped along with its operating system by the same network "insiders" who are shipping the operating system itself. That means a critical task doesn't have to suffer the perils of subcontracting. Honestly, this approach is also way more compatible with subcontracting than the other one. Since the PDP-11 is almost always the same, the per-cost software can be built by network "insiders" working alongside site personnel, either through commercial consulting contracts or even from inside the ARPANET community. (That last option exists because system programmers, even though they resent "invasions," tend to love getting inside new and different systems, if only to feel superior to them compared with their own 😌)

So the strengths of the flexible front-end approach show up exactly where the rigid front-end approach is weak 💅. And maybe the most important thing of all is that it "makes sense" to almost every experienced member of the ARPANET community it's been discussed with. 👑 So, we might reason, if the ARPANET is desirable, it's desirable because of the efforts of the people who made it work. And if they've picked up insights into networking in general along the way, their opinions deserve special attention 🫶

RECOMMENDATIONS 📝🎀

The protocol specified in Appendix 1 is felt to be around 90% complete 🌈 We know we haven't specified all the codes that will be needed to describe the conditions the Host and Front-End must tell each other about, for example. But we think that, overall, the protocol "Woks" ✨ We're ready to talk it through with the decision makers at the various interested organizations, and honestly, to keep debating it with our technical peers too. 🦋 At this stage, though, the dominant makers should pause the apparent stampede toward the rigid front-end approach 🛑 and evaluate the flexible front-end alternative with the arguments above and the protocol specification below in mind 💜

APPENDIX 1. THE HOST-FRONT END PROTOCOL 🧚✨

ASSUMPTIONS 🎀

The physical connection of the front end (FE) to the Host is assumed to be made over the "best" port (or channel, line, etc.) available on the Host 🔌 Here "best" covers both line speed and the quality of software available to physically manage the line. The choice should be made by site personnel. 🫶 Hardware interfacing is assumed to be pretty straightforward. At the very least, it's no more complex for the H-FP than it is for known-device simulation. The connection is also assumed to be closely coupled enough that a simple, explicit acknowledgment H-FP command will offer satisfactory flow control. Basically, distances are assumed to be short and bit rates high, so we're making the same assumptions as for Local IMP-Host interfaces: error checking and flow control aren't first-order problems 😌

On the software side, buffering is assumed to be adequate in the Host to accept at least a full (8096 bit) IMP-IMP message. The FE could probably work around this constraint if it absolutely had to, though 🙈 Given only a minimal H-FP module in the Host, the FE will allow the same level of Telnet and RJE functioning as known-device simulation would. Here's how that works: the FE will always shield the Host from the NCP commands and the simplex sockets they deal with. Instead, the Host deals with a repertoire of just five H-FP commands and talks over duplex data streams, with managing the Network sockets left up to the FE 💖 (The commands are described below. We're going to keep going with the assumptions for now, but some readers might want to study the commands first before continuing with the rest of this section.)

For Telnet, the present assumption is that the FE will normally refuse all "negotiated options" and strip all Telnet control codes from the data it passes to the Host. The exception is when the Host orders it to pass an unaltered Telnet stream. We know a more sophisticated treatment might come out of later analysis, but for now that's the plan 🍓 On a pre-installation basis, the FE will also map from Telnet ASCII to the Host's desired character set. And Telnet "interrupt process" controls get handled by an H-FP command, which we'll talk about below ⚡

For RJE, the ARPANET RJE Protocol is only known to have been implemented on one Host in the ARPANET, and it's generally considered too cumbersome anyway 😩 So the standard socket for RJE will be reserved for future use. Instead, a special designator will tell the Host that input on the given connection should be treated as data in the format and job control language of its own "batch" system. Again, character set mapping will be available on a per-installation basis 🌸

File transfer needs a further assumption about Host software, though 📂 The FE can't be expected to manipulate the Host's file system, so if the Host wants to take part in file transfer activities, its H-FP module must be able to access the Host's file system for both sending and receiving files. Once again, the FE will shield the Host from the details of the underlying protocols to a large extent. But the Host must be able to handle FTP "stor" and "retr" commands, which will be passed over the (single) connection opened between the FE and the Host for file transfer 🎀 (FTP "user" and "pass" commands might also be desirable. Like with Telnet, the FE will manage the various Network sockets involved so the Host only has to operate on the H-FP connection, and it will again optionally do character set mapping. Also, Hosts may refuse to open FTP connections until and unless they choose to, and that has no impact on the FE 🙌)

So, in short: the Host's H-FP module will interpret the commands of the protocol 🧠 It will send Telnet data to and from the right points in its operating system where terminal I/O is expected, and it'll distribute RJE data in a similar way. When it's able to, it'll handle FTP as sketched above and explained more below. It will also, on a when-desired basis, support calls from its system's user processes for unspecified-purpose I/O on ARPANET sockets. That lets stuff like teleconferencing and other process exploitations of the Net happen 🌈 Our overriding assumption is that the initial H-FP module for a given Host (which doesn't need FTP or unspecified socket capability) won't be noticeably harder to implement than a known-device simulation. It should also be extensible to more interesting uses of the network than the alternative sketched here. We'll come back to that after the H-FP commands are described 💜

FORMAT OF THE COMMANDS ✨

okay so all the chat between the FE and the Host happens through H-FP commands 💌 Each field is made of one or more "bytes," and a byte is a per-installation parameter that can be 8, 9, 12, 16, 18, 32, 36, 48, 60 or 64 bits wide, whichever is easiest for the Host's H-FP module implementers to code 🤔 6-bit bytes are a no because they don't leave enough room for all the values expected in certain code fields. A 6-bit machine can still use 12-bit H-FP bytes and keep using its natural byte-oriented instructions 💅

Values are right-justified within their byte widths, which might span several bytes. ngl the list of sizes isn't meant to be exhaustive, and it's probably way too long, since 8, 9 and 12 are likely the only "reasonable" ones in real life. But if a machine works better with whole words than fractions, the FE can totally make life easier for it 🫶

Commands get names for documentation, but the value in the first byte of each command is the binary form of the number shown before its name in the next section. So the command field is one byte wide 🔢

COMMANDS

fyi, all commands can be sent by either the FE or the Host 🔁

1. BEGIN INDEX HOST SOCKET TRANSLATION-TYPE CONNECTION-TYPE

BEGIN is how a "connection" gets set up between the Host and the FE 🦋 No matter how it's represented internally, the two-way data stream is referred to by its INDEX value. So the FE sends input for a Telnet connection "on" a given INDEX, even though it's really juggling two "sockets" to talk to the Network. Main character energy 👑

a) INDEX is a two-byte field. The Host and the FE can each pick arbitrary values when opening a connection with BEGIN. H-FP implementations will probably just add 1 to INDEX whenever they need a new connection. The value 0 is reserved for the "global" connection between the Host and the FE, so whenever either machine comes up, the very first thing it does is send a BEGIN for INDEX=0. END and ACKNOWLEDGE follow this same convention too. There's no reason MESSAGE couldn't also, if we want to extend the FE's functions later, but for now that's just a potential extension 🌸 For INDEX 0 BEGINs, all other fields should be set to 0.

b) HOST is a two-byte field that gives the Host number for the socket in the next field. On FE to Host BEGINs it's purely informational. On Host to FE BEGINs, though, it's necessary so the FE can figure out which foreign Host to talk to at the NCP level 🧚

c) SOCKET is a four-byte field, and it's where things get specific 🎀

  • SOCKET=1 means a Telnet connection is to be established.
  • SOCKET=3 means an FTP connection is to be established.
  • SOCKET=5 means an ARPANET RJE Protocol connection is to be established (no known current utility, sadly 🥲).
  • SOCKET=77 means a Host-specific connection is to be established for RJE/batch.

All other values are for connections with unspecified purposes, which get opened at the NCP level according to the CONNECTION-TYPE field. Sockets 1, 3, 5 and 77 are the "known about" ones, so the FE special-cases them 💖

d) TRANSLATION-TYPE is a one-byte field. From FE to Host, it's informational. From Host to FE, it specifies a character set mapping if one is wanted, or describes the data that'll be sent over the connection 🍓

  • 0 is a request, or specifies ASCII data.
  • 1 is binary data. This value won't be sent from FE to Host under current assumptions, and word size is a per-installation parameter.
  • 2 is a mapping of ASCII to or from the local character set.

Other types will be defined if needs are identified ✨

e) CONNECTION-TYPE is a one-byte field. For FE to Host BEGINs it's informational. For Host to FE BEGINs, it tells the FE which kind of NCP connection discipline to follow 🔥

  • 1 requests a duplex connection, meaning the ARPANET's Initial Connection Protocol should be used.
  • 2 requests a simplex connection, meaning the appropriate ARPANET "request for connection" Host-Host Protocol command should be used for the gender of the socket at hand.

This extended use of H-FP gets really interesting when (and if) user-level programs on the Host start using the Network 👀 Also, the FE will open 8-bit connections at the Network level unless told otherwise.

2. ACKNOLEDGE INDEX CODE

ACKNOWLEDGE is the multi-purpose one 💅 It must be sent in response to every command from the other machine (ACKNOWLEDGEs themselves don't count, obviously). It's mainly used to say whether the command just received on INDEX succeeded or failed. This also means each MESSAGE on a given INDEX must be ACKNOWLEDGEd before the next one can be sent 🫶

a) INDEX is as above.

b) CODE is a two-byte field 🔢

  • 0 means success, or acceptance of the most recent command received for INDEX.
  • 1 means failure, or rejection, of the most recent command. For example, a MESSAGE might fail because buffering was unavailable, so the other machine has to retransmit. Or a BEGIN might fail because the indicated protocol or socket can't be serviced 😭
  • 3 means an invalid or inactive INDEX was used.
  • 4 means (HOST to FE) that no mapping is to be performed on the connection that was just opened.

Other values, for things like "foreign Host down," "undefined type requested" and the like, will be assigned as they're identified 🌈

3. MESSAGE INDEX COUNT PAD TEXT

MESSAGE is the command used to send data 📨

a) INDEX is as above.

b) COUNT is a two-byte field that gives the number of bits of data in the TEXT field.

c) PAD is a 1-to-n-byte field. Its width is a per-installation parameter, used to let the TEXT field start on a word boundary if the local H-FP implementers want that. This is not only a kindness 🥺 It's also a placeholder in case we decide to go with a flow control mechanism that uses sequence numbers.

d) TEXT is a field where byte structure is just coincidental. It's made up of COUNT bits of data to be sent to the process implicitly associated with INDEX by a BEGIN command that hasn't been ENDed yet 💜

4. INTERRUPT INDEX

INTERRUPT is the "quit button" command 🛑 When it's sent from the FE to the Host, it means an FCP interrupt command (INS or INR) has been received for the process associated with INDEX. The Host should interrupt the associated process in whatever way is "normal" for it. The most common use of NCP is in Telnet, where this is the functional equivalent of hitting a terminal's ATTN, INT or BREAK key, or typing "control-c" on certain character-at-a-time systems. Basically, it's a "quit button" push. The FE takes care of the associated Telnet control code in the input stream, so you don't have to worry about that 🙈

When it's sent from the Host to the FE (in process-to-process applications), it means an appropriate NCP interrupt should be sent, depending on the gender of the socket associated with INDEX.

5. END INDEX CODE

END is how a connection gets terminated 🌙 It can be sent because one system or the other is about to go down, because the FE received an NCP "CLS" command, because the destination system or IMP has gone down, or at the behest of a Host user process.

a) INDEX is as above. If INDEX=0, the END refers to the "global" connection between the Host and the FE. In that case, the high-order bit of CODE is set to 1, and the low-order bits give the number of minutes until shutdown, if that info is available. This is included because the associated IMP often tells the FE about this kind of situation ⏰

b) CODE is a two-byte field 💀

  • 1 is the general "close" case, whether it's received or ordered.
  • 2 means the foreign system has gone down.
  • 3 means the foreign IMP has gone down.
  • 4 means the local IMP has gone down.

Other values will be assigned as they're identified ✨

EXTENSIBILITY

Simplicity and compactness are major goals of this protocol, so the small set of commands we just went through is "all there is" 🎀 Recall that we're specifically leaving out error and flow control, since adding those could turn H-FP into another Host-Host Protocol 🙅 If error and flow control turn out to be desirable in practice, we've already thought of a suitable mechanism that fits within the H-FP framework. They just aren't germane to the present context. The main goal is to specify a protocol that makes minimal initial implementations in the Hosts possible, on about the same timeline that known-device simulations would have needed. But it also offers way more flexibility in using the network than known-device simulation would have 🦋

The astute reader will have noticed that most of the commands were specified with the future in mind 👀 The same protocol that lets the Host and the FE talk can easily let user processes on the Host use the Network. So we've tried to encourage this by giving the FE's H-FP module all the necessary hooks and handholds through the broad definitions of the commands. A Host's H-FP module can provide a trivial interface for user programs through a few entry points (open, read, write and close seem to be the minimal set), and still give the user program lots of flexibility in how it uses the net 💖

For example, a "User" FTP program could be made straightforwardly for a Host that didn't choose to field BEGINs on socket 3 (which "Server" FTP capability needs). Files could still be pulled to that Host even if they couldn't be pushed to it. The FE will be required to recognize and special-case BEGINs on socket 3, but that's a small price to pay 🫶

So if the H-FP command repertoire seems a bit more complex than it needs to be, remember that not every Host has to cope with all of it, and any given Host can take advantage of more functions as it wants 🌈 (Although it's not really within the present scope, we're totally willing to invent per-Host H-FP to user program interfaces on request 🧚)

FTP ✨

okay so, to really zoom in on file transfer 📁, only a file system can manage its files. ngl that's basically a tautology and it's hard to argue with 💅. So even though the FE can shield the Host from a ton of FTP mechanism that isn't about moving files 🫶, the Host's operating system still has to be the one that places or extracts a given file, even though it "has" the file's name available to it. There's no in-principle reason the H-FP module on the Host can't call an appropriate routine when it gets a BEGIN on socket 3 🧚. (The FE handles all the type and mode negotiations, passes the stor or retr line along, and is ready to transmit or receive on the right socket. But "somebody" in the Host has to receive or transmit the MESSAGE to or from the right place 😭.) If that's hard on a particular Host, its H-FP module can just negatively ACKNOWLEDGE any BEGINs for socket 3 🙈. The real tea is that the H-FP still allows in principle for User FTP, as explained above, even so, and the simulation of a known device offers neither (User nor Server FTP) function 💀.

(Files could, of course, be transferred into the FE and then somehow gotten into the Host "later", maybe by faxing up a batch job 📠. But that route needs either an awful lot of buffering in the mini or a very sophisticated file system there, or both. It also needs an awful lot of per-Host information in each FE, or maybe human intervention. We're not saying it can't be done... eventually. But it's not going to be clean, or quick, or easy, or cheap 🫠.)

SUMMATION

Several important themes have unavoidably been dealt with piecemeal in the foreign attempt to specify the H-FP in the abstract. To gather the threads together, it might be useful to consider the various ways the protocol can be employed, in the context of their ARPANET counterparts 🧵.

A. "SERVER" FUNCTIONS

There are, in essence, three levels on which a Host can use the H-FP to fulfill ARPANET "Server" functions:

  1. For Hosts which choose to take FULL advantage of the flexibility of the H-FP, all "fourth level" (user process to user process) protocols can be managed by the Host. The FE will perform NCP (Host-Host protocol) and IMP-Host protocol functions (the associated IMP will, of course, perform IMP-IMP protocol functions). This shields the Host from the necessity of implementing a full-blown NCP, with the attendant complexity of being aware of the 11 to 14 "states" of a socket, flow control, retransmission, and the like. It also shields it from the IMP-Host protocol, with the attendant complexity of mapping "links" to/from "sockets", dealing with message types forming and parsing "leaders", and the like 🌸. This mode of use is effected by giving the "no mapping" code when the Host acknowledges a BEGIN on socket 1 and 3 (and by simply accepting BEGINs on all other sockets).
  2. For Hosts which choose to take PARTIAL advantage of the flexibility of the H-FP, many aspects of the fourth level protocols (in particular Telnet and FTP) can be managed by the FE on the Host's behalf. This works by making assumptions about which Telnet and/or FTP "commands" are to be permitted, and only referring search matter as the association of data which processes and/or file names to the Host. (Note that the CODE field of the ACKNOWLEDGE command furnishes the mechanism for conveying such error information as "file not found" from the Host to the FE, which in turn will send out appropriate FTP error messages 💌.) This mode of use is effected by simply accepting (with code 0) BEGINs on sockets 1 and/or 3, and doing as one chooses for all other sockets. That is, fourth level shielding is anticipated to be commonplace, and is the FE's default case ✨.
  3. For Hosts which choose to take NO advantage of the flexibility of the H-FP, the "private" RJE/batch connection type will still provide for the desirable functions of load sharing and transferring files, even though other fourth level protocols were to be rejected by a given Host (by refusing BEGINs on all sockets other than 77). Even in this most restricted case, the ability to upgrade to either of the broader base is additively implicit in the H-FP, with no changes required to the FE's own H-FP module. Whereas it would entail considerable alteration of the Host's operating system had the first step been a known-device simulation 🦋.

B. "USER" FUNCTIONS

  1. On the "User" side, a Host could again elect to handle such fourth level protocols as Telnet and FTP itself. However, particularly in the Telnet case, there is no real need for this. A User Telnet "comes with" the FE, and it is unnecessary to burden the Host with such use unless so many of its local terminals are hardwired that it would be expensive to access the FE directly. (Note that for a User FTP, the Host's H-FP module would, as discussed above, in all likelihood require a user program callable interface 🎀.)
  2. On a less ambitious level, the FE could be induced to perform the same shielding it offers the Server FTP (cf. case A2, above), given an "FTP mapping" TRANSLATION-TYPE on the BEGIN command, or the previously suggested special casting by the FE on socket 3 💜.
  3. Finally, "User" functions could be completely finessed, as per case A3.

C. PROCESS TO PROCESS FUNCTIONS

Irrespective of the positions taken in A and B, given only a user program callable interface to the Host's H-FP module, all other fourth level protocols which might evolve (or, simply, general use of sockets as interprocess communication ports) can be achieved directly. Again, this would fundamentally be an "add-on" to the system, not an alteration of existing software. 👑

APPENDIX 2 - SOME NOTES ON IMPLEMENTERS ✨

INTRODUCTORY DISCLAIMER

okay, bestie, this appendix is strictly the personal views of one of the authors. I (now that I can admit to being Mike Padlipsky) have not even let the other authors agree with the views expressed here, much less disagree with them 🙈. These are insights I gained the hard way during nearly four years of involvement with the ARPANET, and I feel they need saying, regardless of the polite fiction of refraining from finger pointing 💅. Please note at the outset, though, that I'm motivated not by vindictiveness, nor even by righteous indignation 😤. I'm motivated by a desire to present some history in the hope that the reader will not be condemned to repeat it 🌈.

Also, even though it makes the prose more convoluted than it might otherwise have been, the convention will be observed of "naming no names". I am not, I repeat, out to get these guys. I'm merely trying to get away from them and their like in the future 🫶. (The reader can stop here with no loss to the main argument of the paper 💀.)

SEVERAL HORROR STORIES FROM THE WONDERFUL WORLD OF NETWORKING 😱🎀

Okay bestie, first up is the PDP 15/PDP 10 front ending effort 💀. I worked on both the "old" (1971) and the "new" (1973) Telnet Protocols, so I feel a little bit of shame 🙈 that they weren't clear enough. The power of the Network Virtual Terminal / common intermediate representation approach was easy to miss, and it was missed, by system programmers working pretty much in a vacuum with no real contact with knowledgeable ARPANET people. That said, and I really do mean it, we did a good enough job for average-plus system types to cope 💅 (the fact that loads of Hosts are on the Net is proof). BUT average-minus system types exist too, and we have to deal with them. So if we don't make a real effort to "idiot proof" our protocols, we can expect more repeats of the sad state the site under discussion was in before it found us 😭. (And, sadly, support for the "real" ARPANET has slipped so far that the massive effort needed to over-explain ourselves probably couldn't get off the ground in today's climate. More on this later. 🫠)

Horror Story 2 is potentially way worse than a mere "philosophical" muddle over bringing one site aboard 😬. It's an attempt by one of the Armed Services to network a large number of large machines using the ARPANET as a model. The software was done by a house with no known ARPANET expertise. The communications subnet and the hardware interfacing to the Hosts got subcontracted to a well-known hardware manufacturer, also with no known ARPANET expertise. (Random side note, but it's so startling I can't not mention it: the "system architect" for the target network is another well-known hardware manufacturer (!), with, of course, no known ARPANET expertise 🤡.) To make a long, ongoing story short: the "real" ARPANET system whose hardware matches the machines being netted here most closely (even though it's benchmarked at a lower "mips" (million instructions per second) rate than the target net's machines) can transfer files at rates over 28,000 bits per second, using the rather clunky full ARPANET FTP. It does this from a small configuration development machine to a lightly loaded (but still running over 20 users) service machine one Network "hop" away. The new system, meanwhile, achieves rates I'm apparently not allowed to quantify, and they're considerably lower, even though only one process is running on each machine (also one "hop" away), and the file transfer protocol isn't nearly as general as the ARPANET one 😤. Given a year or two, this can presumably be fixed. For now it's fair, if slightly fanciful, to say that if the Japanese had only this level of technology transfer, they'd still be trying to balance their trade with those cute little parasols on matchsticks 🎋🇯🇵.

So what went wrong in Horror Story 2? I'd argue the choice of subcontractors came from a misunderstanding of how technically sophisticated the ARPANET is, and what was (is?) needed was a subcontract to a knowledgeable ARPANET source. (I don't mean the usual profit-making place, though I guess I trust them for the subnet.) I also don't mean any particular place on the Net. Maybe what's needed is a meta-place made out of the whole Net 🌐. More on this, too, later. The real point is that the model was basically ignored by the supposed model-followers, and, demonstrably, it shouldn't have been 💔.

Case three should clear up any impression you might be forming that I'm some kind of hardcore ARPANET chauvinist 🙃. Even "insiders" have blown it. This is actually a dual case, with two unsuccessful attempts to furnish terminal support mini-Hosts for the Net. In one case the choice of machine was faulty. Even with an extra core memory field retrofitted, the buffers couldn't support reasonable data rates without adding a lot of unnecessary Host overhead from processing too many Host-Host Allocation commands. There also wasn't enough room in the mini for more than a rudimentary command language. Now, these were knowledgeable, reasonably well managed "insiders", but contractually they weren't in a position to listen to the technical instincts of several of their own people, or of many of their colleagues throughout the Network Working Group, who could see they'd been painted into a corner 😩.

In the second sub-case, the hardware and contractual obligations seem to have been right, but a badly considered choice of implementation language and inadequate management have kept the project from being fully finished (some two years after it started) 🐢. Again, the NWG saw it coming, since we'd tried to warn them early about the language issue. (On the management level, we could only sympathize, and in some cases empathize. But it's at least a tenable position that the ARPANET as a whole happened despite management, not because of it.) (I guess I'm an ace system programmer chauvinist 💅.)

The final case I'll mention is another military effort 🎖️.

This one I'm not even sure I'm supposed to know about, let alone talk about 🤐. But I can say it involves a subcontractor trying to attach several special purpose machines to a major ARPANET server using an internally invented set of machines and protocols. My information suggests that when they were asked why they didn't follow the obviously sensible route of using ARPANET technology (which already exists on the target server, by the way), the subcontractors basically said they hadn't felt like it 💀. They also still haven't gotten their approach to work, and it's been something like a couple of years they've been trying 😭.

Then there's the fad to simulate RJE terminals... but calling that Horror Story 5 would be begging the question, so that's for another time 👀.

SOME MORALS 🧚

Rather than dig up any more dirty linen 🧺, let's pause and find the lessons. First off, it's kind of obvious that for any technical project, the "right" technicians have to be found and given the power to do it. Despite all the over-sell about the "power of computers," they still absolutely need intelligent, competent programming, which in turn needs intelligent, competent programmers 🧠. And, at the risk of gilding the ragweed 🌿, not every self-professed programmer is intelligent and/or competent. Sorry, not sorry.

Second, and this is the more interesting one, the ARPANET has somehow attracted or created an "in-group" of extremely good system types. Through some sort of natural selection process, they've learned to work well together despite the huge handicap of the heterogeneity of our various "nome" systems' assumptions 🦋. We've not only developed a common tongue, some of us even like each other 🫶. (Fun fact: Appendix 1 was specified on a Wednesday afternoon and a little bit of a Thursday morning. Jon and Jim and I had been there before 📅.) It seems pretty clear to me that the organizations this report is meant for should tap into the expertise that exists in the NWG. We've got a reasonable track record, especially compared to others who have attempted networking 🏆. Lots of us also feel strongly that we didn't get a chance to finish the job on the ARPANET, and we'd love the chance to "do it right," especially given the errors that have been committed in our name 😤. (This matters a lot because the old gang is starting to scatter. As for me, I expect this will be my last RFC. Well, at least I tried to make the most of it 🥲.) The ARPANET is no more a finished product than ANTS or ELF, but all of them could and should be 🌸.

Finally, a pretty trite moral has to be drawn: technical competence is extremely hard to assess a priori. (I'm super fond of saying "Don't ask me what I'm going to say, I haven't said it yet" myself 🙈.) But "track records" ARE important, and competence CAN be demonstrated, to a suitable jury of technical peers ⚖️. So be wary of plausible-sounding subcontractors who tell you "It's easy." In our field, and especially when you're getting all those strange machines, developed by people who mostly didn't talk to each other, to "talk" to each other, it's NOT easy 🚫. I'm willing to claim it'll be easier letting some NWG types do it with the H-FP approach, but it might never be really easy. Here "never" means the next 10 years or so, until "real" networking comes off the shelf with the operating system (which itself barely comes off the shelf today) 🛒. But don't get me started on The Manufacturers 💀.

BEYOND THE PAIN PRINCIPLE 💖

So it's not easy. It's also not impossible 🌈. Honestly, the time looks ripe right now to avoid generating a whole new generation of horror stories, by getting decision makers to pay attention to technical realities and "doing things right" this time around 🔥. Having taken this opportunity to say some things I think are important, I have to stand ready in good conscience to defend the claims I've made about error in some camps and correctness in what I might loosely call "our" camp. And I do stand by them, with a right good will 💜. If any reader wants more backing detail, or just wants to see if I rant like this outside RFCs (or to have a go at my explanation of the common intermediate representation principle), well, I'm still in the ARPANET Directory, even though the phone number's different (try 703-790-6375) ☎️. The mailbox is still accurate. (Even though there's no "ARPANET mail protocol," it's marvelous how stopgaps endure 📬✨.)

[This RFC was put into machine readable form for entry] [into the online RFC by Helene Morin, Viagenie,12/1999]

📦 where this page comes from