RFC 1047

Okay bestie, let's get into RFC 1047 🎀

Request for Comments: 1047 🦋 was written by Craig Partridge of CIC at BBN Labs, dated February 1988. The title is DUPLICATE MESSAGES AND SMTP 💜. The header doesn't list a category or say what it obsoletes or updates, so I'm not making any of that up 🙈.

Status of This Memo ✨

Okay so this memo looks at a synchronization problem in the Simple Mail Transfer Protocol (SMTP) 📬, the kind that can make one message get delivered multiple times 😭. It also suggests a way to avoid it 🫶. You need a nodding familiarity with the SMTP spec, RFC-821, to follow along (it's required, no skipping 💅). Distribution of this memo is unlimited 🌈.

Introduction 🌸

So over the past few years, the staff of the CSNET Coordination and Information Center (CIC) keeps getting asked to figure out why a single mail message is showing up for its recipients more than once 👀. While tracing this, they found that a lot of the duplicates come from a synchronization problem in SMTP, and that's lowkey the main villain here 💀. There's a moment during delivery where the receiving mailer knows it has accepted the message, but the sending mailer still isn't sure it was reliably delivered 🫠. If the SMTP conversation breaks right there, the sending mailer is forced to re-deliver the message, even though the receiver already got it and delivered it 😤.

Description of the Problem 🔥

The sync problem happens right at the end of delivering a message 🎯. Once the sending mailer finishes sending the text, it is required to send a line with a single dot or period (.) 📝. When the receiving mailer gets that final dot, it is expected to do its final message processing, and then it either confirms receipt with a 250 reply or rejects the message with any one of several error codes 🙅‍♀️.

Here's the gap, and it's giving trouble 👀. From the moment the receiving mailer decides it will accept the message until the sending mailer gets the 250 reply, the message is active at both ends 😱. Until the sending mailer gets that 250, it must assume the message was not delivered. After the receiving mailer has decided to accept, it must assume the message has been delivered to it. If the communications link fails inside this gap, the message gets duplicated, because both mailers now have active copies they'll both try to deliver 💔.

It might be hard to believe this is behind so many duplicates 🙃. Intuitively you'd think the time between the final dot and that accepting 250 is super small, right? In practice it's often quite long, long enough that timeouts by the sending mailer (or network failures) are pretty common 🥲. The author's observations suggest this sync problem may be the second leading cause of duplicate messages on the Internet, second only to mail loops 🔁.

Plenty of mailers take their sweet time responding to the final dot because they're doing sophisticated processing to confirm they can deliver the message 🧐. For example, they might expand an entire mailing list to confirm it can reach every addressee, or try to physically deposit the message into the mailboxes of local users, all before confirming receipt of the final dot 📬. Honestly these practices aren't unreasonable, but they often stretch the gap out for several minutes 😩. That makes it way more likely that the sending mailer times out, or the network fails, before the accepting 250 gets sent 💀.

Avoiding Synchronization Problems 🧚

The best way to dodge this sync problem is to minimize the length of the gap 💖. In other words, receiving mailers should acknowledge the final dot as soon as possible and do the more complex processing of the message later 🏃‍♀️.

RFC-821 (on page 22) says that unless the receiving mailer is completely unable to process a message, it should accept the message and report any processing errors in a separate message or messages sent back to the originator 📨. So receiving mailers should be able to acknowledge the final dot as soon as the message has been safely put in a non-volatile (e.g., disk) queue for further processing 💾. Fast acceptance doesn't violate RFC-821, so you're totally allowed to do it 🙌.

Some mailers can be configured to do more or less processing when they get the final dot ⚙️. In those situations, the mailer should always be configured to do less processing 🫶.

Last but not least, some mailers only give remote mailers a minute or two to acknowledge the final dot before timing out and trying again ⏱️. Given the increasing round-trip times on the Internet, and the fact that some processing after the final dot is required, the timeout for a reply to the final dot should probably be at least 5 minutes 🕐. A 10-minute timeout would not be unreasonable at all 👑✨.