RFC 671

RFC 671: A Note on Reconnection Protocol 🎀

okay bestie, here's the title block so you know who wrote this and when 💅

• Author: Richard Schantz, BBN-TENEX ✨
• RFC number: 671 (NIC # 31439) 🦋
• Series: Network Working Group 🧚
• Date: December 6, 1974 🌸
• Category, status, obsoletes, updates: none of these are listed in the header, so there's nothing to report here 🙈

---

Introduction ✨

okay so this note is basically a diary entry from the team that actually built a modified, experimental version of the Telnet reconnection protocol option inside the Resource Sharing Executive (RSEXEC) 💖 they're sharing what it was like to build it.

the big idea: reconnection is how you take a setup where your process is sitting in the middle, with connections to two correspondent processes, and turn it into one where those two talk directly. once that works, your process is no longer in the communication path between them. main character energy, you literally step out of the chat 💅🫶

okay so why does any of this matter? resource sharing networks and distributed computing are going to get more and more specialized, so a "job" is basically a changing mix of specialized modules hooked together. connections are the glue holding the job together, and reconnection lets that bonding structure keep changing on the fly 🔥 (for the full tea on why reconnection is useful, see RFC 426 🌈)

this doc talks about reconnection the way it's currently defined on the ARPANET, as a Telnet protocol option 🌸 the first section is the modified reconnection protocol itself, the second covers general network implementation details, and the last section covers the TENEX/RSEXEC implementation.

heads up: you're expected to already know the new ARPANET Telnet protocol from RFC 495 🧚 no pressure, but it helps lol.

---

I. PROTOCOL for RECONNECTING TELNET COMMUNICATION PATHS 💜

Starting the reconnection

okay so to start, a process reconnects two of its Telnet connections by sending (or asking its "system" to send) the <IAC><DO><RECONNECT> Telnet command sequence over each of the two send connections 🦋 the initiating process is trying to get the two Telnet objects connected directly, so it can bounce out of the middle. iconic 👑

Yes, no, or both

the initiator then waits for positive answers to both requests before going any further ⏳ a request can be accepted with <IAC><WILL><RECONNECT> or rejected with <IAC><WONT><RECONNECT> 🙈

• if both get rejected, normal communication can resume right away 💖
• if only one gets rejected, the process that said yes has to be told, so the initiator sends <IAC><DONT><RECONNECT> in response to its acceptance reply 😭

Picking the roles

once both answers are positive, the initiator picks one object to be passive and the other to be active 🎀 the passive one listens on a set of sockets, and the active one sends Request for Connections (RFCs) for those sockets. assigning roles like this is how they try to lower the chance of synchronization problems ✨

The passive subnegotiation

next up, the initiator starts a subnegotiation with the passive process 🧚 the message looks like this:

<IAC> <SB> <RECONNECT> <PASSIVE> <NEWHOST> <NEWSOCKET1> <NEWSOCKET2> <NEWSOCKET3> <NEWSOCKET4> <IAC> <SE>

the <PASSIVE> parameter tells the receiver to listen for RFCs coming from the socket pair made of <NEWHOST> and <NEWSOCKET1-4> 🦋 NEWHOST is one 8-bit byte that names the host where the active process (the one you're reconnecting to) lives. NEWSOCKET1-4 are four 8-bit bytes that together give the 32-bit send socket number of the Telnet pair from the active process 🔥 the <IAC><SE> at the end closes off the subnegotiation parameters. then the initiator waits for the passive process to reply 💅 there are two legal replies:

1. <IAC><WONT>(RECONNECT>
   • Meaning: the passive process has decided not to finish the reconnection, even though it said yes at first 😭 this could happen because of weird parameters during the subnegotiation (like if it won't connect to NEWHOST), or some kind of error on the passive host 💀
2. <IAC><SE>
   • Meaning: positive acknowledgement of the subnegotiation ✨ the passive process accepted the reconnection parameters and is going ahead with the reconnection 🎀

okay so if the reply was <WONT><RECONNECT>, the initiator is obliged to send <IAC><DONT><RECONNECT> to the active participant to cancel the pending reconnection request 🙈 a confirming <IAC><WONT><RECONNECT> should follow.

When the passive side says yes

the <IAC><SE> reply means the passive participant has started shutting down its connection and will listen on the right sockets 💖 now the initiator can close its connections to the passive participant and hand the parameters over to the active participant 🫶 it can do this knowing it has done everything it can to make sure the passive process is listening before the active process sends its RFCs ✨ if these steps aren't coordinated, the reconnection might fail, for example if the passive host doesn't queue RFCs that show up before it's ready 😭 also, the active participant should be persistent as part of the protocol, because synchronization is kind of unpredictable 🌈

Handing the parameters to the active side

the parameter list for the active participant looks like this:

<IAC> <SB> <RECONNECT> <ACTIVE> <NEWHOST> <NEWSOCKET1> <NEWSOCKET2> <NEWSOCKET3> <NEWSOCKET4> <IAC> <SE>

the <ACTIVE> parameter tells the receiver to send RFCs to the socket pair made of <NEWHOST><NEWSOCKET1-4> 🦋 the initiator waits for another reply, and these are the legal ones:

1. <IAC><WONT><RECONNECT>
   • Meaning: the process won't finish the reconnection (for example, it couldn't parse the parameter string) 😭
   • Possible action of initiator: try to re-establish communication with the passive participant by sending RFCs for the sockets the passive participant is listening on 🔥 this works if the listener is willing to accept connections from either the host/socket in the reconnect parameters or the host/socket of the former connection ✨ if it manages to bring the connection back, the initiator could send <IAC><DONT><RECONNECT> to confirm that the reconnection has been aborted 💅
2. <IAC><SE>
   • Meaning: positive confirmation of the reconnection subnegotiation 🎀 the active participant is saying it will close its connections to the initiator and send the needed RFCs to connect to the passive participant 👑
   • What the initiator can do: close its connections to the active participant, which takes it out of the communication path between the two objects of the reconnection 🫶🌸

okay the whole point is that the initiator gets to bow out gracefully once both sides agree, so that they can talk to each other directly. slay 💖✨

DEFAULT CONDITIONS and RACES 💖

okay so the default for this option is the same as for most other Telnet options: DONT and WONT 🌸. An initiator uses the <DONT><RECONNECT> sequence to get back to the default state, and a participant uses <WONT><RECONNECT> to stay in (or return to) the default state 💅. The reconnection state is only transient, bestie ✨. Once everyone has accepted, it only lasts until the reconnection is finished. After that, with no more back and forth between the parties, the new connection is in the default state and the negotiated reconnection is totally forgotten 🫶

Reconnection is an option that affects the entire Telnet connection, so the asynchronous option processing that other options (like echo) rely on just doesn't apply here 🙈. That means a race condition happens when two <IAC><DO><RECONNECT> requests cross each other in the network 😭. RFC 426 already came up with a fix for this, and what we have here is a modified version of that protocol extension 💜. The big change is about keeping a process's right to deny someone else's reconnection attempt, even while that process has its own reconnection request still pending 👑.

A process detects the race when it gets a <DO><RECONNECT> while it's still waiting for a reply to a <DO><RECONNECT> it sent earlier on the same Telnet connection 🔥. (Both ends of the connection can detect this.) To sort it out, both ends run the same function to figure out which reconnection request takes precedence ✨. The function compares two numbers, made by concatenating the host address (that becomes the high order 8 bits) with the receive socket number (that becomes the low order 32 bits) at each end of the Telnet connection 🧚. The process that owns the receive socket with the larger number gets its reconnection attempt to go first. So say there's a Telnet connection between host A local sockets X,X+1 and host B local sockets Y,Y+1. If <A><X> is greater than <B><Y>, then the reconnect request from <A><X> has to be completed (or aborted) before the one from <B><Y> can even be considered 🦋.

Here's how it works in practice: the process with the higher <host><socket> number replies to the other process's reconnect request with <IAC><WONT><RECONNECT>. This cancels (temporarily) the attempt coming from the lower numbered <host><socket> 💅. Since the higher one's request goes first, the lower process can reply as if it never issued a reconnection request of its own. It can send <IAC><WILL><RECONNECT> to accept the attempt, or <IAC><WONT><RECONNECT> to refuse it 🍓. Still, the lower process should keep in mind that the rejection it got was forced by the protocol rules, so it might not match what the other process actually wants 🙈. It should also remember that its reconnection request can be sent again once the first reconnection activity is done. This is a case where an option change request can be re-sent after being denied, even though nothing about the state has changed 🌈.

ASIDE:

Okay, real talk, the reconnection feature is kind of held back because it's specified as an option for Telnet (basically terminal-style) connections, instead of being part of a host-host protocol that could be used on general connections 😭. For one thing, it's questionable whether most systems will let a user task keep more than one Telnet connection open 💀. If they don't, a process on that kind of system can't easily start a reconnection request at all.

There are also some indirect perks to putting reconnection in a host-host protocol 🌸. It could make some of the timing problems in setting up the new connection easier to deal with. An NCP would know when a reconnection was happening, so it wouldn't have to rush to reuse an RFC for a socket that's still in use (meaning the connection is still open) but is part of the reconnection ✨. Since it's talking directly to another NCP, it can expect the "reconnect go ahead" to show up pretty soon, unless the system crashes. Also, the info needed to finish the reconnection subnegotiation is already available at the NCP level, while the Telnet service routine has to keep its own duplicate copy whenever reconnection might happen 🧚.

Last but not least, the whole idea of reconnection is described in terms of host-host protocol entities 💜. Putting it at a higher level without enough support at the host-host level creates an artificial and rigid limit on what kinds of communication paths can be part of a reconnection 👑. Since host-host protocol is the base for the function-oriented levels above it, redirecting communication paths really fits better with the smooth, uninterrupted world of OPENing and CLOSEing connections, not the world of "open an 8 bit ASCII path with the conventions that ..." 🫠

---

II. IMPLEMENTATION DETAILS ✨

1. Okay so a process starting a reconnection picks one of the object processes to be passive (meaning it listens for RFCs) and the other to be active (meaning it sends RFCs) 🦋. The reconnection protocol doesn't say how those roles get assigned, so the process gets to pick. Still, info about the types of participants can point toward a role choice that helps the reconnection actually finish, and ignoring that info could end up cancelling the attempt 😭. Certain kinds of hosts (like space-limited TIPs) might be better at the active role, since they don't have to go through checking who's sending. The passive process should do that verification. Other hosts (like one whose NCP won't let an arbitrary process listen on a socket) might be better at the active role 🌸. As more systems add the reconnection option, we'll learn what each type of system prefers, and more definite rules might show up 💅.
2. To dodge a possible deadlock, the active (passive) process has to send (listen for) RFCs for both the send and receive connections at the same time, and those will form the new Telnet connection 💖. The reconnection protocol doesn't set an order for establishing the connections, so it matters that passive processes listen in parallel on both the potential send and receive sockets, and that active processes send RFCs in parallel for both the potential send and receive sockets 🔥.
3. There are two levels of error recovery in reconnection 🛠️. The first level handles cases where network or system delays knock the attempt to set up the new connection out of sync (for example, the RFC shows up at the passive host before the passive process is listening), or cause system timeouts 😭. When that happens, the sockets and connections should go back to a state where the failed operation can be automatically retried 💜. The second level kicks in when all those attempts to reach the active (passive) process fail. How long those attempts keep going can depend on things like the recovery procedures that are available, and whether a human user is waiting for the result ⏳. At this point recovery is hard, since the connections with the initiating process are already broken 💀. If second level recovery is needed, attempts should be started to connect to some reasonable alternative (maybe a local one, or maybe trying to connect back to the original source of the reconnection), which shows that the reconnection has completely failed 🌈.
4. A useful addition to the reconnection mechanism would be a standard way to get back in touch with the reconnection initiator when a task terminates (including when it can't complete the reconnection) 🧚. Honestly, that's the kind of thing that would make this whole feature way more iconic 👑✨

III. TENEX RELATED DETAILS ✨

okay so bestie, our experiments ran in a TIP user setting, where the user is on a TIP and talks to a TIPSER/RSEXEC 💖 The TIPSER/RSEXEC first authenticates the TIP user, then acts as the command interpreter. One of the commands it offers is TELCONN (TELnet CONNect), which lets you connect to other sites for service 🦋 Running TELCONN makes the TIPSER/RSEXEC try to reconnect the "TIP" directly to the host you asked for. Usually that's a logger process on the host. Once the reconnection finishes, the TIP is wired straight to the new job, and the TIPSER/RSEXEC is completely out of the communication path 👑 To avoid having to program the TIP itself, we used a TENEX process to simulate it 🧚

Certain TENEX features made it hard to set up the interaction we wanted between the TENEX jobs in the reconnection experiment 😭 We're writing them down here because other systems might run into the same problems 🫶

1. The TENEX Telnet interface comes through the ATPTY system call, which turns a pair of unused network connections into a Telnet connection pair 🌸 But you lose some control along with it. A program can no longer decide when data gets sent (you lose the MTOPR system call that forces buffered data out), and it can't find out the remote host/socket for the network connection (the GDSTS system call can't do that anymore) 💀 In a highly interactive mode, like option negotiation, short messages sitting in system buffers can cause a deadlock. A process has to be able to override the buffering strategy at the end of a logical message. Without a mechanism like MTOPR, the connection has to be opened in non-buffered mode, which is wasteful most of the time 🙈 Likewise, since you can't get the remote host/socket names, the program has to remember them for the whole life of the connection in case it needs them. (The operating system already keeps this info anyway, but we need to reach it when we want to reconnect the Telnet connection that linked the "TIP" to the TIPSER/RSEXEC 🍓)
2. TENEX has no facility for starting or answering Telnet options that the Telnet server doesn't recognize 🎀 An interface between a user program and the option negotiation mechanism would be super useful for testing new options and for implementing private ones ✨ You can work around this by switching the connection to binary transmission and reception. That only works if both ends are user processes that understand binary transmission. If a user process tried negotiating with a system Telnet server that's obeying the binary transmission option, the doubling of IACs needed for binary output would make the system Telnet misread the request 😭
3. Switching to binary transmission takes two option negotiations 💅 Data can still move during that window, but what actually gets transferred depends on where the negotiation is at that moment. For example, the IAC character may or may not get doubled depending on the state. As far as we can tell there's no way to tell the process that the option was accepted or rejected and that everything after is in the new mode (binary) 🌈 One idea is to suspend the process for the length of the option negotiation, including the timeout, which would clear up that uncertain period during the mode switch. In TENEX, you could pair this with pseudo-interrupts to flag an option negotiation failure for certain critical user-initiated options 🔥
4. At peak load, RFCs sent by the operating system (NCP) in response to program OPENF system call requests kept timing out 😩 The passive process listening for those RFCs didn't get rescheduled fast enough to answer them (accept or reject) before the system timed them out. It got confusing because the two connections that were supposed to form the full-duplex path were opened differently: one OPENF asked for immediate return, while the other waited for the RFC to complete 🙃 If both requests timed out, the two connections were in different states, so the retry mechanism had to handle each one differently. The "immediate return" connection had to be closed via CLOSF, but the other one didn't need that. That feels like an unnecessary complication 🥺 Also, frequent timeouts under heavy load may mean the RFC timeout interval is too short ⭐
5. The TENEX user interface to the network has no concept of logical messages when more than one process (fork) shares a network connection 💜 Telnet option negotiation sequences are strings that have to be sent in order, with no characters from anywhere else in between, or they lose their meaning 😤 Even when a process executes a TENEX "string out" (SOUT) operation, which implies the characters of the string are logically related, there's no guarantee that other processes sending over the same connection won't interfere 💔 Multi-process setups for managing network connections are super common. One process typically handles user output to the network, while another reads from the network and replies as the protocol requires to certain input 🦋 These processes have to synchronize on every output (and input) to keep their messages logically intact. Honestly, this synchronization feels like it belongs in the system routines that manage network connections and handle string I/O, not in user programs 👑

---

          [ This RFC was put into machine readable form for entry ]
          [ into the online RFC archives by Alex McKenzie with    ]
          [ support from BBN Corp. and its successors.     7/2000 ]