RFC 683

RFC 683 🎀 NIC 32251

FTPSRV - TENEX FTP EXTENSIONS FOR PAGED FILES ✨ by R. Clements of BBN, dated 3 April 75 📅. This one doesn't list a category, and it doesn't obsolete or update anything, so it's a standalone proposal 💅.

---

1. Introduction ✨

okay so there's a long-known need to send TENEX paged files over the net with FTP, and the TENEX FTP implementation got a glow-up to make it happen 🦋. It's an extension of the "OLD" protocol (RFC 354) and was built after useful chats with Postel, Neigus, et al. Not saying they agreed it's correct, and honestly? I don't feel it's correct either 🙈. A "correct" version will get negotiated and implemented in the "NEW" protocol (RFC 542), if funding ever shows up for that task 💀.

2. The Problem(s) 🔥

This extension tackles two separate problems: network reliability, and the TENEX disk file format being incompatible with FTP 😭. In my opinion, a checksummed, block-sequence-numbered transmission mode is seriously needed. It should also allow data compression ✨.

It's also necessary to handle paged, holey TENEX files. That one is seriously needed for NLS and is the whole motivation for this extension 💖.

The network problem needs a new MODE command, if done correctly, and probably two MODEs so you can get compression in addition to checksumming 🧚. Honestly, that's the tip of an iceberg that grows as 2**N for more sorts of modes, so maybe some mode combination system needs to be dreamed up 🌈. Cf the AN, AT, AC, EN, ET, EC TYPEs. Also, you should be able to use MODE B and MODE C together (NEW protocol) to get both compression and restart facilities if you want 🫶.

The second problem, TENEX files, are probably a new kind of STRUcture. However, it should be possible to send a paper tape to a disk file, or the other way around, with the transfer looking like a paged file. So maybe we're dealing with a data representation TYPE instead. This argument is a bit strained, though, so a paged STRUcture is quite likely correct. I'll be real, I feel very unsure about what is a MODE, what is a TYPE and what is a STRUcture 🙈.

3. The (Incorrect) choices made 🙃

Having decided that new MODEs and STRUctures were needed, I instead implemented the whole thing as a single new TYPE. I know, the header literally says "incorrect" 😭. But after all, checksumming the data on the network (MODE) and representing the data in the processing system as a checksummed TYPE are really just a matter of where you draw the imaginary line between the net and the data. Also, one new TYPE command meant way less surgery on the FTP user and server programs. Pragmatic, no cap 💅.

4. Implementation details 🧚

The new TYPE is called "XTP". I'm proposing this as a standard for all the Key Letter class of FTP commands. The "X" stands for "experimental", agreed on between cooperating sites. The letter after the "X" is signed out from the protocol deity by an implementor for a given system, and here "T" is for TENEX. Any letter(s) after that distinguish among possibly multiple private values of the FTP command. Here "P" is the "Paged" type 🦋.

TYPE XTP is only implemented for STRU F, BYTE 36, and MODE S. Very specific, very intentional 🎀.

Info of TYPE XTP gets transferred in chunks (I intentionally avoid the words RECORD and BLOCK 🙈), and each chunk is a header plus some data. The data in a chunk may be part of the data portion of the file being transferred, or it may be the FDB (File Descriptor Block) associated with the file.

5. Diversion: the TENEX Disk File 📚

For anyone not familiar with the TENEX file system, here's a brief dissertation so the rest of this makes sense 💜.

A TENEX disk file has four things: a pathname, a page table, a (possibly empty) set of pages, and a set of attributes.

The pathname is specified in the RETR or STOR verb. It includes the directory name, file name, file name extension, and version number.

The page table holds up to 2**18 entries. Each entry may be EMPTY, or may point to a page. If it isn't empty, there are also some page-specific access bits, and not all pages of a file need the same access protection 🔐.

A page is a contiguous set of 512 words of 36 bits each 🍓.

The attributes of the file, in the FDB, contain things like creation time, write time, read time, writer's byte-size, end of file pointer, count of reads and writes, backup system tape numbers, etc.

NOTE: There is NO requirement that pages in the page table be contiguous. There may be empty page table slots between occupied ones. Also, the end of file pointer is simply a number. There is no requirement that it actually point at the "last" datum in the file. Ordinary sequential I/O calls in TENEX will leave the end of file pointer after the last datum written, but other operations may leave it somewhere else if a particular programming system requires that.

Both of these special cases, "holey" files and end-of-file pointers not at the end of the file, show up with NLS data files. Those files were the motivation for the new TYPE 💖.

6. Meanwhile, back at the implementation,... 🧚

Each chunk of information has a header. The first byte, which is also the first word (since TYPE XTP is only implemented for BYTE 36), is a small number, currently 6. That's the number of following words that are still in the header. Next come those six words, and then come some data words ✨.

The six header words are:

• Word 1: a checksum. This is a one's complement sum (magnitude and end-around carry) of the six header words and the following data words, but not the leading "6" itself. The sum of all words, including the checksum, must come out + or - zero 🔢.
• Word 2: a sequence number. The first chunk is number 1, the second is number 2, and so on 🔁.
• Word 3: NDW, the number of data words in this chunk, following the header. So the total length of the chunk is 1 (the word containing NHEAD) + NHEAD + NDW. The checksum checks all but the first of these 🧮.
• Word 4: page number. If the data is a disk file page, this is that page's number in the file's page map. Empty pages (holes) in the file are simply not sent. A hole is NOT the same as a page of zeroes 🕳️.
• Word 5: ACCESS. The access bits associated with the page in the file's page map. This full word goes into AC2 of an SPACS by the program reading from net to disk 🔐.
• Word 6: TYPE. A code for what kind of chunk this is. Currently only type 0 (a data page) and type -3 (an FDB) get sent 🏷️.

After the header come NDW data words. NDW is currently either 1000 octal for a data page or 25 octal for an FDB. Trailing zeroes in a disk file page will soon be discarded, which makes NDW less than 1000 in that case. The receiving parts of the FTP server and user will accept these shortened pages. The sender doesn't happen to send them that way yet 🙈.

Verification is performed so that an error is reported if either:

• The checksum fails,
• The sequence number is not correct,
• NDW is unreasonable for the given chunk type, or
• The network file ends at some point other than immediately following the data portion of an FDB chunk.

7. Closing comments 👑

This FTP server and user are in operation on all the BBN systems and at some other sites. The user has spread more widely, since fewer sites have made local modifications to the user process 🌸.

I believe checksumming and sequencing should be addressed for the "NEW" protocol 💅. I hope the dissertation on TENEX files has been useful to people on other systems. It may explain my lack of comprehension of the "record" concept, for example. A TENEX file is just a bunch of words pointed to by a page table. If those words contain CRLF's, fine, but that doesn't mean "record" to TENEX 🙃.

Net data transfers are basically like the layers of an onion 🧅. Some characters are packaged into a line, and some lines are packaged into a file. The file is broken into other manageable units for transmission, and those units have compression applied to them. The units may be flagged by restart markers (has anyone actually done that? 💀). The compressed units may be checksummed, sequence numbered, date-and-time stamped, and flagged special delivery. On the other end, the process is reversed 🌈.

Perhaps MODE, TYPE, and STRU don't really describe the situation adequately. This RFC was written to let implementors interface with the new FTP server at TENEX sites that install it. It's also really a request for comments on some of these other issues 🦋✨.