RFC 150

RFC 150: The Use of IPC Facilities 💖

okay bestie, this is a 1971 working paper and it's giving "early internet dreamers" 🦋

• Author: Richard Bl. Kalin, MIT Lincoln Laboratory 🎀
• Working group: Network Working Group 🌸
• Request for Comments: #150 (also filed as NIC 6754) ✨
• Date: 5 May 1971 📅
• Status: A WORKING PAPER, not a finished standard 🧁
• Obsoletes / Updates: the document doesn't list any RFCs it obsoletes or updates 🫶

⚠️ Heads up before we start: this material has not been reviewed for public release and is only meant for use within the ARPA network. It shouldn't be quoted or cited in any publication that isn't related to the ARPA network. 🙈

---

INTRODUCTION ✨

Okay so the authors have a big hunch 🧠. They think the goals of interprocess communication are way more complicated than people realize, and until that complexity is properly understood, we're not gonna get any satisfying implementations 😭. An IPC design has to do more than just shuffle bits between independent user programs. A bunch of other constraints have to be met too 💅.

Those constraints come from how the facility will actually get used 🎯. Any design that can handle that usage pattern counts as satisfactory, and if it does it efficiently, it gets called well designed. And honestly? It's basically unimaginable that a huge design effort done without fully understanding the usage will ever be well designed, or even satisfactorily designed. Not me going "this is so obvious" 💀.

This paper is about walking through the kinds of usage an IPC facility will face 🗺️. The hope is that this talk will clarify what everyone's chasing and give us a benchmark for judging different implementation strategies. The task is harder than it sounds, so don't underestimate it 🥲. The only experience we have to work with comes from very primitive, overly constrained IPC implementations. Stretching that tiny usage environment out to more general ideas has no real experience behind it yet. So any speculation like that is super open to oversight and misguided perspective ⚠️.

While putting this paper together, the authors had to imagine what a process might want from an IPC facility 🤔. The areas they came up with are:

1. the exchange of bit encoded information via channels 📡
2. the establishment, deletion, and reassignment of these channels 🔁
3. the ability to debug errors and suspected errors 🐛
4. the potential to improve running efficiency 🚀
5. the capacity to avoid storage allocation deadlocks 🔒
6. the aided recovery from transmission errors 🩹

This list is known to be incomplete. Some areas, such as understood to be intelligently discussed. In other cases, omissions should be blamed on simple oversight 🙈. (The original got a little garbled right there, sis.)

Because of all these obvious problems, nobody should treat a document like this as authoritative or final 📜. That's why this paper is being kept as a computer based textfile, so it can keep getting edited and rereleased whenever new insights show up 🔄. The big dream is for it to stay an accurate, up to date statement of the goals any group of IPC implementers can aspire to, and a guidebook for the problems they'll have to face 🌈.

For a few reasons, the paper will not try to design solutions to the problems it raises 🙅‍♀️. First, real solutions have to be machine dependent. A so called "general solution" would be too clumsy and inefficient to be worth anything. Second, we're taking the user's point of view. The user doesn't need to know how their demands get carried out, as long as they get accomplished 💖. Third, we're trying to stay aloof from the eccentricities of present day machine organization 🧚.

We're admittedly walking a fuzzy line when it comes to staying implementation independent 🫠. Our description of usage patterns assumes both a system/process software organization and a computing milieu that can support it. That doesn't seem to change the generality of the document in a big way, but you still have to be careful when picking host machines 👑.

---

In the rest of this paper, we try to describe the kinds of usage an IPC facility should expect 📝. It's organized into titled sections, and each one covers some aspect of the expected usage 🎀.

ABILITY TO USE VARIOUS SOURCES OF WAKEUP INFORMATION 🔔

Most processes have preferences for certain amounts of input data 💅. That preference shows up in how much computing time can be spent on a given amount of input. For example, a character translation routine might prefer eight bit quantities of input. With seven bits or less, no processing is possible, but once a full character is available, a whole translation cycle can kick off ✨. The preference doesn't depend on what the routine actually does. You could write equivalent routines that accept one bit at a time. Other examples: a command interpreter might need a complete line of input, and a linking loader might need a complete file 📂.

Every executive system has to decide when there's enough input for a given routine to run efficiently 🧐. That's a huge decision, not one to take lightly. Waking up a dormant process comes with serious overhead. Room in core storage has to be found, the program has to be swapped into memory, tables have to be updated, active registers have to be exchanged, and then the whole procedure has to happen in reverse once the process finishes 😩. So waking a process when there isn't enough input for it is inefficient and wasteful. The computing it could do just doesn't justify the overhead it costs 💸.

There's no general solution for figuring out the best time to issue a wakeup ahead of time 🔮. It hinges on the relationship between waiting costs and running costs. Trying to make reasonable predictions means balancing accuracy against overhead. And the more system code that has to run, the more overhead you rack up, and the less the final prediction is worth 🙃.

Even without a general solution, the scheduler can get help in specific cases 🆘. A common one is having the receiving process say how many bits it's expecting. So a process can tell the supervisor/scheduler that it needs fifty bits of input from some source before it can keep going. Then it can go into the shade, and it'll get awakened once the required input shows up 😴✨.

When input lengths are known in advance, this works really well 🥰. Otherwise, things get messy 😬. If the input size is unknown, a prediction that's too short leads to the inefficiency of premature scheduling, and a prediction that's too long can cause input deadlocks 🔐.

In those situations, it's common for the process to give the scheduler a simple criterion for checking whether there's enough input 🧾. A classic example is a carriage return showing up in the teletype input stream. That criterion gets tested either by system routines or by a low overhead, user supplied routine. (And that routine needs its own criterion for when to wake up, which is a whole thing 🌀.) Once the criterion is met, the main routine gets awakened and processing carries on 🎉.

Sometimes the system criteria and the user criteria team up 🤝. A user might set a maximum character count that matches the available buffer size, while the system watches for line terminators. Wakeups to the user process can come from either source 🔔. Other times, the system takes over and preempts the user's criterion 👑. For example, if a process is trying to put a single character into a full buffer and gets forced into the shade, it usually won't get awakened again until the buffer is almost empty 🫧. Even though the user only wanted room for one character, the system imposed a much stronger criterion: room for N characters. That's based on the assumption that more calls to output characters are coming. So the program ends up outputting in bursts. Instead of going into the shade and getting awakened N times, each time there's only room for one character, it gets awakened once and sends N characters at once. Program efficiency gets a nice boost 📈.

A third source of criteria for deciding when to wake the user process is the device or routine producing the input data 🏭. Hardware designers use this one a lot. Lots of computer peripherals can send an end of record indication. For variable length uninterpreted records, that's an absolute necessity 🚨. For fixed length records, it's a handy double check ✅. The same kind of feature should exist in interprocess communication too 🌟. If the routine generating the data knows how much the receiving routine will need, that information should be made available for scheduling purposes. Doing that requires a standard way to mark logical boundaries in the stream of data moving through a communication channel 🚩. The scheduler/supervisor on the receiving host has to be able to recognize those markers so wakeups can be issued whenever they're wanted. And to make input interpretation easier, the receiving process should also be able to see where the markers are placed 👀.

The data sitting between two boundaries is what we'll call a logical message 💌. It's a natural unit of information exchange, and the markers travel with the data through the channel. The extra info the markers give about data flow unlocks a bunch of useful stuff 🔓. Here's an example: say two processes always exchange 100 bit logical messages. If the receiving process can figure out the length of each logical message that arrives, you get a simple error detection facility for free 🕵️‍♀️. If a 99 bit message shows up, it's obvious that a bit got dropped somewhere 🫣.

It's worth noting that the receiving process can't always work out where the boundary markers sit in the input stream 🤷‍♀️. There's no reason the info implied by marker position has to also be in the coded input data. Even when there's coding redundancy, pulling boundary information out of the rest of the input can be harder than just using the markers directly 🧩.

ABILITY TO SEND PARTS OF LOGICAL MESSAGES 📦

Any IPC facility where user storage is at all constrained can't force a process to send a whole logical message at once 🙅‍♀️. The logical message concept only exists to make it easier to issue wakeups to the receiving process. What's a convenient input chunk for the receiver might not be a convenient output chunk for the sender 🎁. Picture a process running on a small machine sending messages to a process on a big task-multiplexed machine. For efficiency, the receiving process wants big chunks of input at a time 🥹. Meanwhile, the sending process's address space can't hold much data. So the only answer is to let the sender dump its logical message in parts, and have the last part say that it's the end of the message 🏁.

ABILITY TO RECEIVE A LOGICAL MESSAGE IN PARTS 🧺

Flip that around, and the receiving process might not have enough buffer space to take an entire message all at once 😵. In that case, it should be possible to choose to accept the message in parts 🌷. This is also needed for messages that are too long for the system to buffer. If the receiving process doesn't accept part of the message, the transmission can never be completed 🚫. This approach also helps clear out very long messages that show up by mistake in the input stream 🧹.

ABILITY TO FIND OUT IF A MESSAGE CAN BE SENT 📬

If nothing stops it, a routine can easily generate messages faster than they can be consumed 😳. Since any given amount of buffer capacity can

✨ cliffhanger alert!! the rest of the story drops in part 2 🫶🦋

be quickly exhausted, so there has to be a way for the system to limit how fast a process can crank out messages 💅. That means a process trying to send might sometimes get stopped because there's no buffer free, and that's a whole mood 😭. If the process is about to get put in the shade, the pause should not hit it out of nowhere 🙈. There should be a way for the process to find out ahead of time whether its message can be sent 🎀. A program has better things to do than sit around waiting for a buffer to free up, bestie 🧚.

ABILITY TO GET A GUARANTEE OF OUTPUT BUFFER SPACE 🎀

A process should be able to get a guarantee from the system that a message of a certain size can be sent. That way it knows, before it does a whole computation, that the results can actually get output successfully 🙌. This allocation sticks around until it gets used up or until another allocation request changes it ✨.

Okay, this one is going to make system programmers SO mad 💀. From their side, the empty buffer space that's been preallocated is basically being wasted, because even though it holds no messages, nobody else can use it 😤. The user process, though, is like, 'empty' does not mean 'wasteful' and 'full' does not mean 'efficient' 💅. A process needs empty output buffers just as much as it needs full input ones. Both are resources it needs to keep processing, periodt 🫶.

ABILITY TO FIND OUT ABOUT OUTSTANDING MESSAGES 🦋

A process sending messages over a channel should be able to find out how many of those messages haven't been accepted yet by the user process at the far end, and whether or not that number can go down. Ideally it should also be able to figure out how many bits are left in any partially accepted message 🤓. But the overhead needed to do this on conventional systems might be too much to be tolerated 😬.

The count is useful both in the moment and for post mortems 🔍. If you're debugging a remote process, it lets you present inputs on normally concurrent channels one at a time, in any order you want 🧩. Like, you could check that the same results come out no matter what order the inputs arrive in 🙈.

If a receiving process fails, this mechanism lets the sending process figure out the last input that got accepted before the process died 💀. Even between processes that are working fine, it gives the transmitting process a user managed way to slow down and let the receiver catch up 🐢. By pinpointing bottlenecks, it can also help detect design mismatches 🔥.

Unless the channel has no outstanding messages or is dead, there's a chance the receiving process will accept another message at the same time as the request 😭. Because of that, the count returned can not be assumed to be exact. It has to be treated as only an upper bound 📏.

ABILITY TO GET WAKEUPS WHEN MESSAGES ARE ACCEPTED 🌸

Along with the above, it should be possible for a user process to get alerted when the number of messages it has sent over a particular channel, and that haven't been accepted at the far end, drops below a specified threshold 🔔. So if a process finds out twenty messages are still outstanding, it can choose to get put in the shade until that number has dropped to less than five 🌙. Doing that lets the process run in 'burst mode' 💥. Instead of being swapped in and out of core fifteen times, each time only getting to send one message, it gets loaded once and sends fifteen messages 🚀. There's no penalty for this because the bottleneck on throughput is at the receiving process anyway 🤷‍♀️. If the local process's swapping costs are significant, there might be a big economic advantage to this mode 💰.

If the remote process dies or issues a channel 'close', the count of undelivered messages gets frozen 🧊. If the receiving process is expecting this type of wakeup, it should get one at that point, even though the count hasn't hit the desired threshold 🚨. That way the process knows it's time to do a postmortem on the channel 🕵️‍♀️.

---

ABILITY TO LEARN ABOUT MESSAGES QUEUED FOR INPUT 📬

A process should be able to learn the status of the Nth logical message queued on a given input channel 👀. At the very least it should be able to tell whether it's available, whether or not it's complete, how long it is, and what it contains 📦.

This lets a program make general preparations before accepting a message, which is lowkey amazing 🧠. It also gives an escape from the awkward position of having accepted input and then not being able to dispose of it 😵. For example, if it's known that processing the message will result in two more messages being sent, then it's super helpful to get guarantees that the output can be generated before the input is accepted ✅.

Say one end of a channel gets moved from one process to another, like moving a teletype connection between a user program and a debugging program 🖥️. This ability to scan ahead in the input stream lets a process check whether the pending input is really meant for it 🕵️. If it is, the input gets accepted normally. Otherwise, the end of the channel has to be moved to another receiving process first 🔁.

Accepting input should be treated as a grave responsibility 🫡, and it shouldn't be done unless there's reasonable assurance that the input can be processed 🧐. One of the first rules of asynchronous system design is to detect errors as soon as possible 🚨. If errors get propagated, the tangled results can be hopeless to unravel 💀.

ABILITY TO LEARN HOW MANY MESSAGES ARE WAITING 🔢

A process should be able to determine how many messages are still left to be processed on a given input channel 📊. Two uses come to mind right away 💭. If there's pending input on several channels, a process should be able to set its preference 👑. One option is to take input first from the channel with the most messages queued. That might increase throughput, since freeing message buffers can let the remote transmitting process run 🏃‍♀️. Another possibility is that the receiving process has some control over the sending process, so if it sees a backlog on its inputs, it could tell that process to slow down 🐌.

Assuming the remote process can still send messages, the number of inputs reported is only a lower bound, since new inputs may get added concurrently 📈. If the foreign process has died or otherwise closed the connection, then the bound can be made exact 🎯. The local process should be able to learn when it's exact 🙌.

GUARANTEE THAT INPUT WILL STAY AVAILABLE 🛡️

This requirement says that if a process has been told it can receive N messages on a given channel, then those messages really are available and buffered within the host machine 💯. If they've been promised to a user process, messages should not mysteriously become unavailable 👻. An example of how this might happen is illustrated in RFC60 📖. There, during a panic for buffer space, messages get destroyed and reported as being in error. They're later recovered from backup copies held in the foreign host 🙏.

ABILITY TO RECEIVE A WAKEUP WHEN INPUTS ARRIVE 🔔

A process should be able to enable a wakeup when the number of messages queued on an input channel goes over a specified value, or reaches its maximum value 🎉. This lets a program process input in burst mode and save on swapping costs 💸. It also lets inputs get combined in a simple way. For example, if two inputs are needed before anything can happen, the appropriate interrupt can be generated easily ✨.

The same interrupt should be generated if the maximum number of inputs has been received 🛑. Two cases are distinguished here. Either the foreign process has closed the channel and so isn't sending any more messages, or the system won't allocate more buffers until some input is accepted 🙅‍♀️. That way the process can be told there's no point waiting for the condition it's anticipating 🫠.

ABILITY TO SPECIFY SPECIAL WAKEUPS ⭐

A process trying to run efficiently should be able to specify arbitrarily complicated wakeup conditions 🧠. This gives the user managed way of cutting down on premature wakeups 🎯. Perhaps the easiest way to provide this generality is to let the main process designate a small, low overhead, interrupt driven routine. That routine checks for the desired conditions and issues a wakeup to the main process whenever they're met 🧚.

ABILITY TO MEASURE CHANNEL CAPACITY 📏

There's been a lot of discussion about measuring a data stream, and in the heat of committee, a lot of confusion got generated 😵‍💫. Our feeling is that, within the present domain of discussion, there's no single measure of a message channel's capacity 🙅‍♀️. Two completely orthogonal concepts must be measured -- 1) the number of messages buffered and 2) the number of bits of encoded data represented 📐. The system overhead for each is very implementation dependent, so no general equation can express the measure of one in terms of the other 🧮. By making an arbitrary assumption (eg. a message boundary equals 100 bits of buffer), a system runs the risk of excluding new nodes that can not meet the old criterion 😬.

ABILITY TO FIND OUT MAXIMUM CHANNEL CAPACITY 🔍

There should be a system call that lets a user process learn the maximum current capacity of any given channel 📞. It should be reported as a pair of numbers: the maximum bit count and the maximum message count 🔢.

ABILITY TO CONSTRAIN CHANNEL CAPACITY 🚧

A process using a channel should be able to set new bounds on that channel's capacity 🎚️. If possible, the system should try to meet this bound 🫡. The actual bounds imposed must meet the constraints of at least four different sources -- local and remote user process, local and remote system 🌐. Setting an arbitrarily high bound doesn't guarantee it can be met 🙃. Likewise, a low bound can not always be met until buffered messages are consumed 🍽️.

So a receiving process that sets the current message bound to zero effectively disables the transmission of new messages 🚫. That means, without the cooperation of the transmitting process, message generation can be temporarily disabled while outstanding message buffers get flushed 🧹. Later the message allocation can be raised back to its original limit, and transmission can resume 🔄.

ABILITY TO CLOSE A CHANNEL AT ANY TIME 🚪

A process should be able to shut down a channel at any time 🔒. If the process has died, the system should be able to close all open channels for it ☠️. For channels the process was receiving data on, pending input should be thrown away, and indications should go back to the transmitting system marking the channel as dead and identifying the last data item accepted 📨. That identification is in terms of the number of logical messages discarded and the number of bits left in the oldest message 🧾.

If a process closes a channel it had been sending over, buffered output should still be sent normally 📤. Along with it, an indication should be sent that this is all of the input that will ever appear 🏁.

ABILITY TO GIVE AWAY CHANNEL PRIVILEGES 🎁

The right to perform any of the operations discussed here is normally reserved by the process that set up the channel 👑. At times, that process may want to transfer some of its delegated power to another process 🤝, especially in an environment where one process might spawn another and resources have to be passed to the newly created process 🧚.

Schemes for this reassignment can get arbitrarily complicated 🌀. For example, you could assign each aspect of usage individually, then separately assign the various rights of reassignment 🤯. Fortunately it isn't always necessary to get so elaborate, and it's expected that in most cases the following simple

So the simple version is basically the whole game 💅 the process that set up the channel is the only one allowed to close it 🔒 If that process dies while the channel is still open, the system closes it automatically 💀 Everything else about how the channel gets used stays outside system control 🦋 The channel is known by its name, and any process that's been handed that name can use it 💖 Making sure only one process is actually using it at a time is up to user-level coordination 🫶

ABILITY TO INITIATE CHANNEL CREATION ✨

Okay, for most cases, setting up a channel is honestly super simple 🎀 A process tells its local system that it's listening on a specified channel 👂 It gets connected to the first remote process that 'dials' the right number 📞 Figuring out who's calling only happens after the channel is up 🙈 If it's a wrong number, the channel can be closed and listening can start back up 🔁 And callers trying to reach channels that are already set up get a 'busy signal' 📵

To 'dial' a remote process, a process has to specify a channel it's listening on, plus a remote number 🧚 Then the system tries to set up the connection ⚡ While that's happening, the channel is 'busy' 🚧

Now for processes that would rather skip the drama of identifying remote callers 🙄, there's an extra feature 🌸 By specifying both the local and remote channel identifiers, a process hands the system the job of screening callers for proper identification 👑 The connection only gets accepted from the caller that was specified 💅

---

ABILITY TO REPORT TRANSMISSION ERRORS 🚨

Say a process has prescanned an input message and decides it has some sort of transmission error 👀 It should be able to reject that message 🙅‍♀️ After that, the system should kick off whatever internal error recovery mechanism it may have built in 🛠️ no cap, that's the whole safety net 🫶

---

POSTSCRIPT 🌈

The author would love to hear any comments, questions, or corrections about this document 💌 Even the most informal note or a phone call is appreciated 📱 Especially interesting would be opinions on how useful this discussion was, and whether there should be more papers covering other basic questions of computer networking 🤔 If the consensus leans yes, then others are encouraged to write working papers on flow control, error handling, process ownership, accounting, resource control, and the like 💜 literally so many good topics 😭

RBK/TX2

[ This RFC was put into machine readable form for entry ]
[ into the online RFC archives by Michael Baudisch 9/97 ]