RFC 492

RFC 492: Response to RFC 467 🎀

Network Working Group · Request for Comments: 492 · NIC: 15357 · E. Meyer, MIT-Multics · 18 April 1973

okay bestie, so this one is a response to RFC 467 💅 The header doesn't list a category, and it doesn't say this obsoletes or updates anything. It's just E. Meyer from MIT-Multics answering a proposal from Jerry Burchfiel and Ray Tomlinson at Bolt, Beranek, and Newman, Inc. 🦋

Those two wrote up a Network Request for Comments (#467) that tries to fix two problems people kept running into on the Network. Meyer is going to explain both problems, go over the proposed fixes, and drop some comments and alternative ideas. no cap, it's giving peer review energy ✨

---

BACKGROUND ✨

okay so first, how a connection even starts 🧚 The Host-Host protocol says that one host sends a Request for Connection, and the other host has to say yes. If the socket ("port") it's aiming for at the target host is already in use, the target host says no 🙅‍♀️ Once it's connected, data can flow, but it's controlled by data allocation messages sent by the host on the read end. The host on the write side is only allowed to send as much data as the read side has permitted. Once it runs out of allocation, it has to wait for a brand new data allocation control message before it can send anything else 💜

Now the first problem, which is honestly so annoying 😭 Messages get lost somewhere along the transmission path, and it happens with a low but regular frequency. If an allocate control message for an open connection gets lost, the whole connection can freeze permanently. The send side thinks it's used up its allocation, so it just sits there holding its data, waiting for a new allocation from the read side. But the read side already sent that allocation, and it got lost. So the read side thinks the send side is good to go and just sits there waiting for data to show up. This is called the "lost allocate" phenomenon 🥀

Similar symptoms can show up if a data message gets lost and the send side uses up its allocation before the read side gives a new one. The send side waits for a new allocation, but the read side never got one of the data messages, so it thinks there's still allocation left. Either way, you end up with a permanently blocked connection 💀 It happens often enough to be super frustrating for people who hook up typewriters to foreign hosts through the Network, and the only fix right now is to disconnect and make a whole new connection 😤

RFC 467 proposes a pair of allocation-resetting control messages to handle this, one for the send side (RCS) and one for the read side (RCR) 🎀 Whenever it wants, either side can start the allocation-resetting sequence. It sets its own allocation counter to zero and sends an RCS or RCR control message to the other side. The receiving host sets its own allocation counter for that connection to zero, then replies with an RCR or RCS. Now both sides' allocations are synced up (they're both zero), and data can start moving again once the receive side sends a new allocation 🌸 This is meant to be triggered whenever either side thinks the connection has been quiet for a suspiciously long time 👀

The actual spec for this pair in RFC 467 is more complicated, though. The pipeline between the two sides has to be empty of data messages before the send side is allowed to send an RCS control message 🙈

The second problem is the "half-closed" phenomenon 🧊 It happens when the host on one end of an open connection crashes and wipes its tables when it comes back up, but the host on the other end never notices anything happened. (A similar thing happens when the Network path temporarily fails between the two hosts, but only one host notices and closes the connection.) If the host that crashed tries to reconnect, the other host refuses, because the socket the request is aimed at already looks like it's part of an open connection. Because of how the terminal support software works on some systems, users at some consoles can't reconnect to the distant system they were using when their local system crashed 😩 This can go on forever, until the system that still thinks the old connection is open resets its internal state.

RFC 467 fixes this with one simple principle: the side that still has the open connection should be able to notice an inconsistency whenever either side communicates about that connection. When it spots one, it's supposed to silently close the connection (ignoring normal protocol) and get ready to handle connection requests for the port it was connected on before ✨

There are two ways this "half-closed" inconsistency gets caught 🔍 The first is when the connected side sends a message over a write connection. The side that lost the connection gets this as a data message that doesn't match any open connection, so it replies with an Error Report control message. When the connected side gets that, it realizes the connection doesn't actually exist anymore and deletes it from its own tables 🫶 The second is when the host that lost the connection sends a connection request to the other host using the same sockets as the previous connection. The receiving host sees the inconsistency right away, because the local socket is already connected, and it's connected to that exact same foreign socket. It quietly deletes its record of the old connection, which frees up the local socket, and then answers the connection request normally 🌈

COMMENTS AND ALTERNATIVE PROPOSALS 💅🎀

Okay so the Project MAC Computer Systems Research Division is opposing both protocol change proposals in this RFC 💀. They have moderate opposition to the half-closed connection proposal because it doesn't consider every part of the problem and it makes the protocol even more complicated 🙃. But they have very strong opposition to the allocation resynchronization proposal, because it fixes a symptom instead of the disease, and it can hide a potentially very serious network problem from being diagnosed 😭.

RFC 467 proposes two new control messages, Reset Connection by Sender (RCS) and Reset Connection by Receiver (RCR). Their only job is to resynchronize the allocation counters at both ends of a connection. The goal is to fix the "lost allocate" thing, where allocate (ALL) control messages get lost in transit so the sending side can't keep sending data. If it were really a lost allocate problem, this would be a viable solution ✨. But the author feels it's actually a "lost message" problem, where messages of all kinds are getting lost in transit, and that is way more serious 🚨. ALL messages might be super frequent with some hosts, so they might be the ones that get lost the most, but if messages are really being lost in the network, data messages could be getting lost too, and that would look similar 🤔. A lost message in a Telnet connection can be spotted and fixed by the human user, but an undetected lost message from the middle of a transmitted file can cause disastrous consequences 💀. The invalid file might never get corrected, especially if it's detected at all. Because this "solution" papers over the immediate problem and pushes it to a point far away in both space and time, where it shows up as an incomprehensible disaster, it should be strongly opposed 🙅‍♀️.

The real problem seems to be the random, undetected loss of messages somewhere in the transmission path 👀. A true fix is either a) to eliminate the cause of the undetected message loss, or b) to move to a new protocol designed to cope with an unreliable physical transmission path. Both of those are kind of far away 😩. So there's a proposed interim fix that modifies the existing GVB and RET commands, and it also simplifies them a bit 🦋. Here's how it works.

A receiving host may, at an arbitrary time, issue a Give-Back allocation (GVB) control message for a connection 🌸.

          8       8        8        8
      +-------+-------+--------+--------+
      |  GVB  | link  | f =255 | f =255 |
      |       |       |  m     |  b     |
      +-------+-------+--------+--------+

The format of this GVB message is the same as the one currently defined, except that the fraction fields f(m) and f(b) are required to be all 1s. This is designed to give a measure of upward compatibility 🧚. A host running the modified protocol will ignore the fraction fields, but under the current protocol this message means "return everything" 👑. When a sending host receives a GVB control message, it immediately stops transmitting on the specified link. Once the RFNM from the last message it sent is received (which means the pipeline is empty), the sending host issues a Return Allocation (RET) control message, returning the remaining allocation 🍓.

           8      8        16         32
       +------+------+-----------+-----------+
       | RET  | link | msg space | bit space |
       +------+------+-----------+-----------+

The modified RET command has the same format as the one currently defined 💖. There are two differences. It can't be sent until data transmission has stopped and the last RFNM has been received. And it must return all remaining allocation for the send link, meaning the allocation counters are set to zero 🫶.

When the host on the read side of the connection receives the RET message, the allocation counters at the send side are zero and the pipeline is empty ✨. So, if no error has happened during the connection, the allocation returned in the RET message should match the allocation in the counters on the read side. If it does, the read side can go ahead and send a new allocation, totally secure in the knowledge that no message has been lost 💅. If the two sets of values don't agree, some error in the transmitted data may have happened 😬. What to do then is a local host option. Some hosts might choose to close the connection, while others might choose to resume transmission by sending a new allocation to the sending side 🌈. The author feels that, as a minimum, a host should send a message about the error both to the user and to some human at the host who's responsible for monitoring network performance 🔥.

This modified control message pair can do its originally intended job, and it can also detect errors and resynchronize allocations (if desired) when the receiving side starts it 👑. The author feels that the scheme's inability to start allocation checking from either side is only a minor disadvantage, and that it's more than made up for by its positive features 🦋. This scheme gives a positive indication that an error has occurred (the proposed RCS/RCR method hides errors!), and this small change to the protocol may only mean a correspondingly small change to NCP's 🌸.

The author has negative feelings about the solution to the "half-closed" problem proposed in RFC 467 🙈. Putting extra burden on the RTS and STR commands not only complicates the protocol unnecessarily, but in some sense can make operation less fail-safe and problems more obscure 😵‍💫. The main objection is about what to do when control messages arrive that conflict with the current state of the receiving NCP ⚠️. This proposal suggests that an NCP receiving an STR or RTS for a socket it believes is connected should assume something about the state of the foreign NCP (that the foreign NCP has closed the connection), and automatically change its own state to match that assumed state (closing the connection at its end). That might work fine if the assumption is correct and the implementations are bug-free 🤞. But the following situations could cause problems that might be really hard to diagnose 😭:

1. the foreign NCP has a bug that makes it send an RTS or STR for a connected socket,
2. the foreign NCP decides to read the queuing option of the current protocol as allowing RFC's to be sent for sockets that are already connected, or
3. the local NCP has a bug that makes it wrongly treat RFC's coming from a different host, or from the same foreign host but about a different foreign socket, as belonging to the open connection attached to the target socket 🫠.

A second objection is that this proposal doesn't cover all possibilities 🧐. Two likely ones are these. First, another socket (from any host) tries to connect to the socket involved in the dead connection. Second, the host that lost a connection attached to one of its read sockets makes another connection with different sockets, but uses the same link number that implemented the previous connection. The second case can be handled with extra complications added to the protocol. But the first case looks exactly the same as a situation where an RFC is issued for a genuinely already-connected socket, so it can't be handled using this approach 😤.

The author believes that a more rigorous use of the existing Reset Host (RST) control message would get rid of most of the causes of the "half-closed" phenomenon 💜. For example, one of the hosts involved in a connection goes down without sending an RST when it comes back up, or the network between the two hosts partitions and only one host notices. If it turned out to be necessary, a pair of Reset Link control commands could be added to the protocol to reset an individual link and handle instances of the "half-closed" phenomenon caused by other reasons 🎀.

The author would also like to set down a few principles that they think are at least peripherally related to easing the "half-closed" phenomenon ✨. None of these is explicitly stated in the current Host-Host protocol document, but the author believes that spelling them out would help reduce the confusion caused by network and host failures 🧚.

1. An NCP that receives an Imp-to-Host message type 7 (Host Dead) about a host should consider all connections or connection attempts with that host as dead, and should purge them from its tables 💀.
2. When an NCP notes a foreign host as dead (by receiving a "Host Dead" Imp-to-Host message) and then receives any message from that host other than a Reset Host (RST) control message, it should delete the message and respond with an RST. It should respond normally to a received RST 🔁.
3. Two hosts must exchange the RST - RRP reset control message pair before any other form of communication 🤝. An NCP that wants to start communicating with a foreign host must first send an RST, if that host pair has not been reset since the local NCP came up or since it noted the foreign NCP as down. Note that this doesn't require an NCP to send resets to all other hosts every time it comes up 💖.
4. An NCP that receives an Imp-to-Host message type 9 (Incomplete Transmission) about a write link implementing an open connection may, at its option, try several times to retransmit the last message until an RFNM is received or the NCP gives up 🔄. However, unless the message is eventually transmitted successfully to the foreign host, the NCP must abort the connection by sending out a CLS control message 🚫. Successful retransmission depends on the retransmitting host waiting for an RFNM on a data link before sending a subsequent message, and on all hosts being able to discard messages that aren't completely received 🫶.
5. An NCP that receives a message from a foreign host that seems inconsistent with its current state should take no action to modify that state 🙅‍♀️. Instead, it should send an ERR error control message specifying the type of inconsistency, and discard the inconsistent message. An NCP receiving an ERR message should log it for human inspection, and is then allowed to silently modify its internal state or send out control messages to remove the inconsistency 👑. (This extends the proposal in RFC 467 that an NCP should delete a connection when it receives an ERR message specifying that the link involved is unknown.) 🌈

---

[This RFC was put into machine readable form for entry] [into the online RFC archives by Helene Morin, Via Genie,12/1999]