RFC 1845

RFC 1845 💖 is an Experimental doc from the Network Working Group, dated September 1995. Its authors are D. Crocker (Brandenburg Consulting), N. Freed (Innosoft International, Inc.), and A. Cargille, the WG Chair. The title is SMTP Service Extension for Checkpoint/Restart ✨ and the header doesn't list anything it obsoletes or updates 🌸

Status of this Memo 📜

This memo defines an Experimental Protocol for the Internet community. This memo does not specify an Internet standard of any kind. Discussion and suggestions for improvement are requested. Distribution of this memo is unlimited.

Abstract 🦋

This memo defines an extension to the SMTP service that lets an interrupted SMTP transaction restart later, without having to repeat all the commands and message content sent before the interruption. 🔁

1. Introduction ✨

Okay bestie, SMTP is super widely deployed and honestly very robust 💅 But people have asked for extensions, because when you're sending really big messages over shaky connections, the sender can burn through a ton of resources retrying the whole message from scratch over and over 😭 The original SMTP spec [1] has no way to pick up a half-finished transaction once the underlying TCP connection drops and gets re-established.

So this memo gives the client a way to uniquely identify a particular SMTP transaction 🪪 The server then stores that identifier along with everything it receives as the transaction goes on. If it gets interrupted during the data transfer phase, the client can open a new SMTP session later and ask the server to pick up from where it lost the connection. The server rebuilds the transaction context and tells the client where to resume 🎀 If that works for the client, it carries on from that exact point.

This extension can also help with the classic timeout problem, where a client times out while waiting for the server to confirm the message was accepted. BUT, and this is important, using this extension is not a replacement for properly setting your timeout parameters 🙅‍♀️

2. Framework for the Checkpointing Extension 🧩

The checkpointing extension is laid out like this:

• (1) the name of the SMTP service extension defined here is checkpointing;
• (2) the EHLO keyword value associated with the extension is CHECKPOINT;
• (3) no parameter is used with the CHECKPOINT EHLO keyword;
• (4) one optional parameter using the keyword TRANSID is added to the MAIL FROM command. The value associated with this parameter, coupled with the name of the client taken from the EHLO command, forms a globally unique value that identifies this particular transaction and keeps it distinct from all others. This value is case-sensitive. The syntax of the value is as follows, using the ABNF notation of [2]:

transid-value  ::= "<" transid-spec ">"
                   ; transid-value may not be longer than
                   ; 80 characters
transid-spec   ::= transid-local "@" transid-domain
transid-domain ::= transid-token
transid-local  ::= transid-token
transid-token  ::= transid-atom *("." transid-atom)
transid-atom   ::= 1*<any (ASCII) CHAR except SPACE,
                      CTLs, tspecials, or ".">

NOTE: tspecials is defined in [3]. The TRANSID is likely to be different from the RFC822 message id, since it must uniquely identify the particular copy of the message being sent over this SMTP link. However, the syntax of transid-value is designed so that any TRANSID is both a legal RFC822 msg-id as well as being a legal esmtp-value [4].

• (5) the maximum length of a MAIL FROM command line is increased by 88 characters by the possible addition of the TRANSID keyword and value;
• (6) no additional SMTP verbs are defined by this extension; and,
• (7) the next section specifies how support for the extension affects the behavior of a server and client SMTP.

3. The checkpointing service extension 🌈

When a client SMTP wants to use checkpointing so it doesn't have to resend all the message data from the top if the session gets interrupted, it first issues the EHLO command to the server SMTP. If the server SMTP responds with code 250 to the EHLO command, and the response includes the EHLO keyword value CHECKPOINT, then the server SMTP is saying it supports SMTP checkpointing and will honor requests to restart interrupted SMTP transactions 🙌

The extended MAIL command is issued by a client SMTP when it wants to turn on server checkpointing. The syntax of this command is identical to the MAIL command in [1], except that a TRANSID parameter must appear after the address.

The complete syntax of this extended command is defined in [4], with the esmtp-keyword TRANSID and transid-value parameter as previously defined.

The value associated with the TRANSID parameter must be an identifier that uniquely identifies this particular SMTP transaction. Only one TRANSID parameter may be used in a single MAIL command. Care must be used when constructing TRANSID values so they are both unique and able to be used to re-identify interrupted transactions 🧠

The TRANSID is structured to guarantee global uniqueness without any additional registry. The transid-domain part should be a valid domain name that uniquely identifies the SMTP client. This is usually the same as the domain name given with the EHLO command, but not always. The EHLO domain name identifies the specific host the SMTP connection came from, whereas the transid-domain may refer to a group of hosts that together make up a multi-homed SMTP client. The transid-local part should be an identifier that tells this SMTP transaction apart from any other coming from this SMTP client.

Even though the TRANSID has structure, the server should treat the value as an opaque, case-sensitive string 🫶

Note that the contents of the RFC822 message-id header are typically NOT appropriate for use as the TRANSID parameter value. Those identifiers may be attached to multiple copies of the same message (for example, when it gets split up and sent down different network paths), and so to multiple distinct SMTP transactions 👀

A server that supports the checkpointing extension will keep the transaction identifier, along with the most recent state of the transaction, in non-volatile storage. This information should be deleted only when the transaction is known to be complete from the client's point of view. Completion is only assured when the client explicitly aborts the transaction, starts a new transaction, or asks for the connection to be closed with a QUIT command.

If the transaction gets interrupted before it completes, this preserved state sticks around for however long the server administrator's operational policies say. It is recommended that transaction state information be preserved for at least 48 hours, although no specific time is required ⏳

When a client notices that a transaction has been interrupted, it then must wait for some period before reconnecting. That period must be long enough for server connections to time out and for the transaction state tied to those connections to be released for use by a new connection. The Internet Host Requirements [5] also put restrictions on how quickly reconnection attempts can be made (section 5.3.1.1).

Once that necessary period has passed, the client first checks DNS as described in [6] and works out the set of acceptable IP addresses the message can be transferred to. If the IP address used to connect to the original server is still on that list, it should be tried first, since that server is the most likely to be able to restart the transaction. If that connection attempt fails, the client must then follow [6] to try all the remaining IP addresses and restart the transaction there. If the attempt to restart fails on one of the other servers, the client is required to retransmit the transaction in its entirety at that point. Waiting for a server with an interrupted transaction state to come back online is not acceptable 😤

Note: Multi-homed SMTP servers do exist, which means it is entirely possible for a transaction to restart on a different server host. 🏠

Once the connection is made, the client issues the same MAIL command with exactly the same transaction identifier. If the transaction was interrupted during or at the end of the transfer of actual message data, the server first rebuilds its context to a point as close as possible to the point of interruption, and then responds with the status message:

355 octet-offset is the transaction offset

The actual status text can vary. However, the octet-offset field is required to be the first thing on the first line of the reply, it must be separated from any following text by linear whitespace, and it is structured as follows:

octet-offset ::= 1*DIGIT

The octet-offset is an offset, counting from zero, to the particular octet in the actual message data that the server expects to see next. (This is also a count of how many octets the server has received and stored successfully.) This offset does NOT account for envelope data, i.e., MAIL FROM and RCPT TO commands. A value of 0 means the client needs to start sending the message from the very beginning, a value of 1 means the client should skip one octet, and so on 🔢

The SMTP canonical format for messages is used when this offset is computed. Any octets added by any SMTP data-stuffing algorithm do not count as part of this offset. For data transferred with the DATA command, the offset must also line up with the beginning of a line 📏

Once this context is rebuilt, the client issues another data transfer command (for example, DATA) and sends the remaining message data. Once that data is terminated, the transaction completes in the normal fashion and the server deletes the transaction context from non-volatile storage 🧹

Note that the semantics of the octet-offset immediately suggest a particularly simple implementation strategy: the client retransmits the message data as it normally would, but suppresses output of the first octet-offset octets of material. The semantics used here were intentionally designed to make that implementation possible, but care must be taken to make sure such an implementation strategy does not impose a significant performance penalty on the client 💭

5. Usage Example 🎬

The following dialogue shows how the checkpointing service extension is used:

S: <wait for connection on TCP port 25>
C: <open connection to server>
S: 220 dbc.mtview.ca.us SMTP service ready
C: EHLO ymir.claremont.edu
S: 250-dbc.mtview.ca.us says hello
S: 250 CHECKPOINT
C: MAIL FROM:<ned@ymir.claremont.edu> TRANSID=<12345@claremont.edu>

S: 250 <ned@ymir.claremont.edu>... Sender and TRANSID ok
C: RCPT TO:<mrose@dbc.mtview.ca.us>
S: 250 <mrose@dbc.mtview.ca.us>... Recipient ok
C: DATA
S: 354 Send checkpointed message, ending in CRLF.CRLF
<some amount of message data transmitted>
<session is interrupted and TCP connection is broken>

Okay so the sender gets the green light for the sender and TRANSID, then the recipient is ok too, then DATA 💅 The server says send it, and the message starts flowing. Then the connection just dies 💀 (the session gets interrupted and the TCP connection is broken), so the message is only partly sent.

Later on, a brand new connection gets set up 🧚 Same server, same energy. The client says EHLO again, and this time the server's reply includes CHECKPOINT ✨

S: <wait for connection on TCP port 25>
C: <open connection to server>
S: 220 dbc.mtview.ca.us SMTP service ready
C: EHLO ymir.claremont.edu
S: 250-dbc.mtview.ca.us says hello
S: 250 CHECKPOINT
C: MAIL FROM:<ned@ymir.claremont.edu> TRANSID=<12345@claremont.edu>
S: 355 6135 is the transaction offset
C: DATA
S: 354 Send previously checkpointed message starting at octet 6135
C: <message data minus first 6135 octets sent>
C: .
S: 250 OK
C: QUIT
S: 221 Goodbye

The same TRANSID comes back in MAIL FROM, and the server says the transaction offset is 6135 🦋 So the client sends the rest of the message, meaning everything minus the first 6135 octets, and the server wraps it up with a 250 OK. Bestie, that's a whole glow-up from a broken connection 🌈

---

6. Security Considerations ✨

Okay, serious moment 🫶 This RFC doesn't actually discuss security issues. It's not believed to raise any security issues that aren't already endemic in electronic mail and present in fully conforming implementations of [1]. So the vibe is: the risks were already there in email, and this RFC isn't adding new ones 💀

7. References 📚

[1] Postel, J., "Simple Mail Transfer Protocol", STD 10, RFC 821, USC/Information Sciences Institute, August 1982.

[2] Crocker, D., "Standard for the Format of ARPA Internet Text Messages", STD 11, RFC 822, UDEL, August 1982.

[3] Borenstein, N., and N. Freed, "Multipurpose Internet Mail Extensions", RFC 1521, Bellcore, Innosoft, September 1993.

[4] Rose, M., Stefferud, E., Crocker, D., Klensin, J., and N. Freed, "SMTP Service Extensions", RFC 1651, Dover Beach Consulting, Inc., Network Management Associates, Inc., Silicon Graphics, Inc., MCI, Innosoft, July 1994.

[5] Braden, R., Editor, "Requirements for Internet Hosts - Application and Support", STD 3, RFC 1123, USC/Information Sciences Institute, October 1989.

[6] Partridge, C., "Mail Routing and the Domain System", STD 14, RFC 974, BBN, January 1986.

8. Authors' Addresses 💌

Dave Crocker
Brandenburg Consulting
675 Spruce Dr.
Sunnyvale, CA 94086 USA
USA

Phone: +1 408 246 8253
Fax: +1 408 249 6205
EMail: dcrocker@mordor.stanford.edu


Ned Freed
Innosoft International, Inc.
1050 East Garvey Avenue South
West Covina, CA 91790
USA

Phone: +1 818 919 3600
Fax: +1 818 919 3614
EMail: ned@innosoft.com