RFC 662

PERFORMANCE IMPROVEMENT IN ARPANET FILE TRANSFERS FROM MULTICS ✨

Okay bestie, here's the header info first 🎀

Network Working Group, RFC 662, by Raj Kanodia of Project MAC, MIT, dated Nov. 1974 (NIC #31386). The header doesn't list a category, and it doesn't obsolete or update anything. Clean slate, no drama 💅

With Multics getting popular in the ARPA Network community, people keep needing to move big piles of data from Multics to other systems. But Multics is kinda slow at this, and everyone agrees it needs to get better. Even inside Project MAC, folks are constantly shipping files back and forth between Multics and other Project MAC machines 📦. Then the Cambridge Information Systems Laboratory (CISL) development Multics system got hooked up to the ARPA Network, and suddenly everyone in the CSR (Computer Systems Research) and CISL groups wanted to use the Multics file transfer facility to send Multics system tapes (MST) from one Multics to the other. The problem? At the transfer rates we can actually get right now, one typical MST (about 12 million bits) takes roughly half an hour 😭. The file transfer system is seriously held back by its low bandwidth.

PROBLEM 💀

One reason it's so slow is the output buffering strategy in the Multics IMP DIM (IMP Device Interface Manager). Hoping for a big boost, the team recently changed the IMP DIM buffering strategy. To check whether that change actually helped, they ran an experiment between the service machine (MIT Multics) and the development machine (CISL Multics). This memo writes up the experiment and what came out of it 🔬.

For reasons that are only of historical interest, the output buffers in the IMP DIM have been super tiny. Each buffer can only hold up to 940 bits 🥲. Add the 72 bit long header and the whole message maxes out at 1012 bits, which in Network terms is a single packet message. Because of that buffer limit, Multics can only send single packet messages, even though the network can take messages up to 8096 bit long. The result is extra setup overhead every time you send a big pile of bits 🙄.

There's also an old version of the IMP-to-Host protocol that says a host MUST NOT... well, okay, it says a host may not send another message on a network connection until a Request-for-Next-Message (RFNM) has come back for the previous one. That restriction has been relaxed now, but the protocol still doesn't say how to recover from transmission errors when more than one RFNM is pending on the same connection. If you only send one message at a time, you at least have some hope of recovering from an error. Being reliability-minded, Multics keeps following the old rule 🫶. The catch is that waiting around for each message slows down successive messages on the same link, so the overall bandwidth per connection goes down 📉.

SOLUTION 🔥

One way to boost file transfer bandwidth is kinda obvious: make the IMP DIM output buffers bigger so you can send way larger messages. That means fewer RFNMs and fewer delays, and better bandwidth ✨. The plan rests on two assumptions. First, the delay from RFNMs doesn't grow with message size. Experience on the network backs that up 🙌. Second, the actual time to send a message grows, at best, linearly with message size. That second one doesn't hold on a heavily loaded network, but for what we're doing, it's close enough to treat as true 💅.

Bigger messages have another perk too. For the same amount of data, you send fewer messages, read fewer RFNMs, and spend less time in the channel driver programs. The channel sits idle for some of that time, so less idle time means better channel bandwidth 🦋.

Footnote 1: In one situation it may not be possible to transmit large messages. The number of messages and bits that a host can transmit on a particular connection must not exceed the message and bit allocation provided by the receiving host. If a receiving host gives very small bit allocation, then the sending host can not transmit very large messages. Since Multics always gives out very large allocations, there should be no problem in Multics to Multics file transfers. 💖

Footnote 2: It should be noted that the size of RFNM is fixed and does not change with the size of message. 🌸

EXPERIMENT 🧪

They changed the IMP DIM to have two kinds of output buffers 👑. The first kind is short and can only hold single packet messages. The second kind is large and holds the biggest message the network allows (naturally 🙄). If the number of bits to send is small, which is mostly interactive users, the small buffer gets picked. If it's big, like file transfers, the large buffer gets picked 🎀.

The new IMP DIM ran on an experimental Multics system on the CISL development machine. The MIT service machine still had the old version. Large files went from the development system to the service system, and they measured the transfer rate in bits per second of real time ⏱️. To estimate the gain, they also sent files from the service system to the development system and measured the old rate. Both runs used the same file, about 2.5 million bits long. The BYTE-SIZE and MODE parameters of the ARPANET File Transfer protocol were set to 36 and "image" mode. That means no character conversion happened during the transfer 🧚. Here are the results:

                        Service to Development    Development to Service
                        (old system)              (new system)

average file-
transfer rate           6,837                     27,578
(bits per second)

27,578 vs 6,837. Iconic glow-up 🌈✨

For completeness, here's the environment. The service system was lightly loaded, with a system load between 30.0 and 35.0 🌿. Its configuration was 2 cpu's and 384 K primary memory. The development machine had 1 cpu and 256 K of primary memory. Its load was limited to the file transfer processes and the initializer process 💜. The MIT Multics connects to IMP number 44 (port 0) with a pretty short cable, about 100 feet long. The CISL Multics connects to IMP number 6 (port 0) with a cable about 1500 feet long. Both IMPs sit close together, roughly 2000 feet apart, and are linked by a 50 kilobits per second line 🔗. The results above show a big performance gain with the new IMP DIM 📈💅.

Since the Multics development community really wants to use file transfer to send Multics System Tapes between the two systems, here's an estimate for the current MST. MST 23.4-0A has 334,231 words. Multiply that by 36 (the word length in bits) and you get roughly 12.5 million bits total. At a file transfer rate of 27,500 bits per second, sending system 23.4-0A takes about 7 minutes and 30 seconds 🎉. Compare that to the half hour from before. Bestie, that's a whole different life 😭🍓.

In the experiment above, only one file moved between the two systems at a time. So they ran another one to see how things look when multiple transfers happen at once 🌟. They started with two simultaneous file transfers from the development system to the service system, then went up to three. Each transfer was kicked off by a different process logged in from a separate console. Since they had to start these by hand, perfect simultaneity wasn't possible, so the total bandwidth numbers are approximate 🙈. With two simultaneous transfers, the total bit rate was about 40,000 bits per second, and each transfer got about 20,000 bits per second. With three simultaneous transfers, the total stayed around 40,000 bits per second, and each individual transfer got about 13,500 bits per second 🫠💜.