RFC 310

Another Look At Data And File Transfer Protocols ✨🎀

Network Working Group 🌸 this is an RFC (a Request for Comments, the docs that define how the Internet's protocols work). Here are the deets:

• Author: A. Bhushan, MIT-MAC 💖
• NIC number: 9261
• RFC number: 310
• Date: April 3, 1972 📅
• Category/status: not stated in the document
• Obsoletes/Updates: the document doesn't list anything it obsoletes or updates 🙈

---

Okay so this RFC is basically Bhushan saying, "our team has been building DTP and FTP on ARPANET and we noticed some stuff could be way simpler, so let's talk about it before the Data and File Transfer Workshop." DTP is the Data Transfer Protocol and FTP is the File Transfer Protocol, and this paper is a big brainstorm to make them more useful and easier to implement. It's giving "let's be better, bestie" 💅✨

Experience to Date

Okay so before anyone got fancy, people were already moving data and files around ARPANET with all kinds of ad hoc hacks 🛠️ The most versatile one was the TENEX "CPYNET" system, which used an ad hoc or interim file transfer protocol that Ray Tomlinson and others at BBN built to move files between TENEX systems. [Private Communication with Bill Crowther, BBN.]

Here's how CPYNET works, and honestly it's kind of iconic 🧚

• The using process connects to server socket 7 through the Initial Connection Protocol (ICP), which gives a full-duplex connection with an 8-bit byte size.
• Control info gets sent from the using process to the serving process: user name, password, command (read, write, or append), file name, and byte size for the data connection.
• Then the original connection gets closed and a new full-duplex connection is opened on the same socket numbers, possibly with a different byte size. 🔄
• The file goes over that new connection, and the end-of-file is just closing the connection. (That's basically like DTP's "indefinite bit-stream" mode, so it's giving familiar vibes 🫶)

CPYNET has been used a LOT for moving TENEX system files 📁 Since the data doesn't get reformatted and you can pick the optimum byte size, it's super efficient. The PDP-10 (and there are a lot of them on ARPANET) works best with a 36-bit byte size, because it cuts down on packing and unpacking and boosts effective I/O speed. The bit rate is 36 times the I/O word transfer rate instead of 8 times, which is a whole glow-up 💜 Closing and reopening connections adds overhead, but in TENEX that's tiny compared to the mess you get from using the wrong byte size.

Other sites got data and file transfer working by tweaking the user TELNET so it could move ASCII files as terminal I/O streams, staying inside the TELNET protocol's rules. 💻 One example is the "send.file" and "script" features in the MIT-DMCG user-TELNET.

• send.file lets a PDP-10 (DMCG) user send local ASCII files to a receiving process, like an editor on the remote host, over a TELNET connection. It takes a variable buffer size for transmission and measures the transfer rate of information bits. 📊
• script lets a user receive an ASCII file from a remote host by basically printing it out, with the terminal output stream going into a local file. 🖨️

Early experience with send.file showed an almost linear relationship between buffer size and transmission rate (inverse to processing cost), until you hit the limits from allocates, NCP sending buffers, the IMP message size, or how fast the receiving process is. 🚀 The team also found that TELNET processes where the receiver "looks" at each character are slow and expensive, lowkey a huge vibe check 😭 Most TELNET receiving processes ran at about 200 to 2,000 bits per second. NCP-to-NCP transmission is an order of magnitude higher, at 2,000 to 20,000 bits per second. 🔥

A variation that skips the character-by-character TELNET processing is sending files over auxiliary connections (other than the TELNET ones), with or without DTP. The team was still collecting data on response times, connect times, and transfer speeds for different transfer and buffering strategies. 📈

TIP Capabilities and TIP Users

It looks like TIPs won't support DTP in its present form 💀 The fancier TIPs with magnetic tape units will support DTP block mode (descriptors and counts), though. [Private Communication with Bill Crowther, BBN.] It's inconvenient, at the very least, for a TIP user to use DTP-based services like remote job service, file transfer, mail, and the Datacomputer. The TIP philosophy is that "the computational load and storage should be in the hosts or in the terminals and not in the terminal processor." (See ref 4.) So to match that vibe, the protocols should be simple and easy to use from the user's side. 🎀

Ideally, TIP users want to connect (using the initial connection protocol) to the advertised service socket, including logger socket1, on the remote host, and then type their commands in a uniform, easy format. ⌨️ Letting ASCII into DTP would make that way easier. (An alternate approach is extending TELNET to include DTP modes, especially the indefinite bit-stream mode.) Another step would be using printable ASCII strings instead of numeric codes for commands and arguments in user-level protocols. Standard file system commands with uniform interpretation and format would eventually lead to a Network Virtual File System, kind of like the Network Virtual Terminal defined in the TELNET protocol. 🗂️✨

The transparent mode in DTP was added specifically so TIPs could use it conveniently. But since TIPs won't support transparent mode, it makes sense to drop it. That would give a simplier DTP that allows transfer in Block mode and the indefinite bit-stream mode. (The suggested default, which would be acceptable to everyone including the TIPs, involves no overhead.) After that, we could make optional, or get rid of, the now-mandatory modes-available handshake. The using process could say whether it also accepts block mode for transfer, either through a modes-available transaction or an argument in the command string. The server should accept input in DTP mode as well as ASCII. These changes would make talking to TIPs a lot easier 🥰

TIP users who don't have a mediating user-FTP process and a file system on their TIP would probably want to move files to or from input devices and output devices like a line printer, card reader or punch, or magnetic tape. 🖨️🃏 These devices "listen" on specific "ports" or sockets on a TIP. So it would be nice to modify FTP so it can send data to a specified socket in a specified mode and type. Then TIP users could get a listing of their files on a high-speed line printer, load their files from a card reader, and keep backups on cards or magnetic tapes. 💜

Datacomputer Requirements

We've been in a continuing conversation with CCA people (Dick Winter in particular) about CCA's plans for data and file transfer on the Datacomputer, plus their specific requirements. Dick Winter is going to talk about this at the Data and File Transfer Workshop. This section is an attempt to sum up the main points of that discussion and what they mean for data and file transfer. 📝

First, CCA seems pretty flexible right now about how the Datacomputer gets used. Even the Datalanguage (see ref 5) can change. 🙌 However, CCA would like some changes to the current file transfer protocol and how it's envisioned to be used.

Ideally, CCA wants a single full-duplex connection for transferring all control information, which is in Datalanguage. That info is generated by a process, which could be a user at a console or a user program. Being able to mix data and control information together would be a definite advantage. The Datacomputer would probably support DTP and allow use of TELNET-ASCII. 🧠

Data may alternatively be sent to or received from a separate user-defined port (which may be a socket). It would be advantageous if a user could tell the Datacomputer to transfer data to or from a file on a remote system via FTP (assuming there's a server-FTP on that remote system). CCA isn't committed to this idea yet, but they're thinking about it. 🤔

In CCA's view, the Datacomputer is a data management facility with Datalanguage as its command language. From the viewpoint of the Datacomputer as an FTP server, FTP commands would be a subset of Datalanguage. So it's desirable that FTP commands be printable ASCII strings instead of numeric codes. ✨

Remote Job Service Requirements

At first, two separate protocols were proposed for Remote Job Service (RJS). One was the NETRJS protocol (see ref 6) for remote job service from large Hosts, and the other was the NETRJT Protocol (see ref 7) for remote job service from TIPs (and other mini-Hosts). The current thinking is to move toward a single RJS with "as much overlap as possible between the methods of dealing with these two user populations." (See ref 8.) Maybe putting ASCII into DTP would make that feasible. 💅

The existing DTP and FTP proposals have been considered somewhat less than optimal for RJS needs. People have pointed out specific drawbacks of DTP and FTP in how they handle data structures and data types. Most of these problems seem pretty easy to fix. It would mean making Network ASCII the default data type (acceptable to all hosts), and giving FTP a way to propose and reject alternative data types and data structures. 🫶

Another FTP weakness, which also affects other applications, is error recovery. Right now there's no way to "restart" transmission if an element in the transmission path fails. One suggested solution used a sequence number (see ref 9), and there are a number of other solutions out there too. These are discussed later in the section 'FTP Reconsidered'. 🌈

DTP Reconsidered 💖

Okay bestie, so DTP was supposed to be the uniform way to split information into its logical structure: records, files, and control. It also came with some basic error control 🎀. Evaluating DTP and its modes comes down to four things: speed (real-time), efficiency (processing cost), reliability (error control and recovery), and how easy it is to use ✨.

Here's the tea 🫖. Unless DTP got a serious glow-up, the TIP and other mini-Host users were gonna struggle to use services built on DTP 😭. Letting ASCII happen inside DTP and using defaults instead of the "modes available" handshake would help a lot, but it would also compromise the DTP error control function 💀. Whether error control belongs at the DTP level or a higher level still needs more discussion 🦋.

DTP, as it stands, doesn't have enough error control and recovery procedures, no cap 🙈. To make it more useful, it could either get simplified (at least from the user's point of view), or get extended with better error control, built-in error recovery, and maybe handling of data types and data structures 👑.

In the simplified version, DTP would just be a format procedure. Data could go as ASCII (no format), with an escape to an 8-bit transparent (indefinite bit-stream) mode, or in data blocks (descriptor and count mode) 🌸. The higher-level protocol would pick the mode and handle all error control, error recovery, and aborts 🧚.

Lots of people have questioned block mode. It's giving big overhead for what is basically just marking end-of-file and separating data from control info 😩. The alternative is to use separate connections for control and data, and/or close and reopen connections. That brings its own overhead, but the upside is you can pick the byte size for the connection to optimize the transfer 🍓.

The real drawback of present DTP is that it's built around 8-bit bytes 💅. A better strategy might be to let the sender and receiver agree on a transfer mode. That mode would decide the byte size for the connection, the data type, and any data structure information ✨. It could get chosen at the DTP level or at the using protocol level 🌈.

FTP Reconsidered 🦋

The aspiration for FTP was to make file management and file transfer easier in the ARPANET Virtual File System 📂. Its success should be judged by how much it gets used, how convenient and efficient it is to use, and how well it works for other applications like Datacomputer, RJS, and Mail 💜.

Wide adoption would be possible if a user could talk to an FTP server directly, without a mediating DTP/FTP-User process 🙌. That would mean commands become ASCII strings instead of numeric codes, and ASCII characters become acceptable input. It would really boost acceptance 🔥, but the server implementation would get more complex 😅. On the flip side, a combined implementation would be simpler, because the mediating FTP-user process (if you even use one) would be simplified too 🫶.

Efficiency is a huge deal for how useful FTP is 🌟. File transfer can get super expensive in CPU time and slow in real-time if the wrong transfer strategy is used, like a bad byte size 🙈. Every attempt should be made to optimize data transfer 💖. One good strategy could be to send files over a separate connection, or to close and reopen connections (maybe with a different byte size) ✨. The catch with using a connection close to mark end-of-file is that you can't tell if the connection closed because end-of-file was reached or because something failed or errored out 😬. Maybe "NCP interrupts" could be added on top of a "close" to signal a definite end-of-file condition 🎀.

Another drawback: current FTP has no restart procedure 💀. One proposal used the sequence numbers from DTP block mode. My take is that restart might work better with a command in FTP that lets the user specify the place in the file to begin retransmission 🍓. One possible option is the "SPF" command from the UCSB Simple-Minded File System (see ref 10) 🧚. Another idea is optional arguments for the retrieve and store commands, so you can selectively retrieve and replace things by bits, characters, words, lines, pages, or segments 🌈.

One more useful addition would be a protocol procedure between user and server to agree on data type, data structure, and mode for a file transfer 👑. That way the user and server can land on the best transfer strategy that works for both of them 💅.

Concluding Remarks 🎀

So that's our rundown of the major problem areas in the current DTP and FTP specifications ✨. We hope this sparks some good thinking 🧠💜, so we can end up with revised DTP and FTP specs that meet all the different needs in an elegant way 🦋🌸.

REFERENCES 📚

1. The Data Transfer Protocol, Bhushan, et al, NWG/RFC #264, NIC #7212.
2. The File Transfer Protocol, Bhushan, et al, NWG/RFC #265, NIC #7213.
3. Data and File Transfer Workshop Announcement, A. Bhushan, NWG/RFC #309, NIC #9260.
4. The Terminal IMP for the ARPA Compuer Network, Ornstein, et al, SJCC, 1972, NIC #8218.
5. Datalanguage, Computer Operation of America, Datacomputer Project, Working Paper No.3, October 29, 1971, NIC #8208.
6. Interim NETRJS Specifications, R. T. Braden, NWG/RFC #189, NIC #7133.
7. NETRJT - - Remote Job Service Protocol for TIPs, R. T. Braden, NWG/RFC #283, NIC #8165.
8. RJS Protocol Meeting Notes, 25 February 1972, A. McKenzie (limited distribution).
9. A Suggested Addition to File Transfer Protocol, A. McKenzie, NWG/RFC #281, NIC #8163.
10. Network Specifications for UCSB's Simple-Minded Files System, James E. White, NWG/RFC #122, NIC #5834

[This RFC was put into machine readable form for entry] [into the online RFC archives by H�l�ne Morin, Viag�nie 10/99]