RFC 2197

SMTP Service Extension for Command Pipelining

πŸ“… published September 1997, from ietf

RFC 2197: SMTP Service Extension for Command Pipelining πŸ“¬πŸŽ€

Okay bestie, come sit with me, because this one is a whole vibe πŸ’… RFC 2197 came out of the Network Working Group in September 1997. N. Freed from Innosoft wrote it, and it's a Standards Track document. It obsoletes RFC 1854, so this is the newer, glow-up version of that earlier spec ✨ It doesn't say it updates anything else.


Status of This Memo πŸ“œ

Okay so this document is an Internet standards track protocol for the Internet community, and it's literally asking people for discussion and suggestions for improvements 🫢 If you want the official standardization state and status of this protocol, check the current edition of the "Internet Official Protocol Standards" (STD 1). And the distribution of this memo is unlimited, so share it with everyone 🌈


1. Abstract ✨

This memo defines an extension to the SMTP service that lets a server say how well it can accept multiple commands in a single TCP send operation. Sending several commands in one TCP send operation can seriously boost SMTP performance πŸš€

Also, this document is an updated version of RFC 1854 [3]. The only changes are textual and editorial. The protocol itself hasn't changed at all, so no stress πŸ’–


2. Introduction 🌸

Okay, SMTP is already widely and robustly deployed, but some extensions could still be super useful 🧚 Lots of parts of the Internet use high latency network links, and SMTP's one command-one response structure really suffers on those links. The time spent waiting for each individual command's response (turnaround time) often ends up being the biggest part of total connection time. Painful 😭

In a perfect world, you could just ship SMTP client software that does command pipelining, which means batching up multiple commands into single TCP send operations. The catch is that the original SMTP specification [1] never explicitly said that SMTP servers must support this. So a bunch of Internet SMTP servers can't handle pipelining properly. The known flaws in deployed servers are:

  1. Connection handoff and buffer flushes in the middle of the SMTP dialogue. Making a server process for each incoming SMTP connection is a useful, obvious, and harmless implementation technique πŸ‘ But some SMTP servers wait to fork the process and hand off the connection until some middle point in the SMTP dialogue. When that happens, material read from the TCP connection and stored in process buffers can get lost πŸ’€
  2. Flushing the TCP input buffer when an SMTP command fails. SMTP commands fail a lot, but there's no reason to flush the TCP input buffer when it happens. Some SMTP servers still do it anyway πŸ™ˆ
  3. Improper processing and promulgation of SMTP command failures. For example, some SMTP servers refuse to accept a DATA command if the last RCPT TO command fails, ignoring whether the prior RCPT TO command results succeeded or failed. Other servers accept a DATA command even when all the previous RCPT TO commands have failed. A client that uses pipelining can accommodate this kind of behavior, but it makes the client way more complicated than it needs to be 😀

This memo uses the mechanism described in [2] to define an extension where an SMTP server can declare that it can handle pipelined commands. The SMTP client checks for that declaration and only uses pipelining when the server says it's capable πŸŽ€


2.1. Requirements notation πŸ“

This document occasionally uses terms that appear in capital letters. When the terms "MUST", "SHOULD", "MUST NOT", "SHOULD NOT", and "MAY" appear capitalized, they are being used to indicate particular requirements of this specification. You can read more about what these terms mean in RFC 2119 [4] 🧠


3. Framework for the Command Pipelining Extension πŸ¦‹

The Command Pipelining extension is defined like this:

  1. the name of the SMTP service extension is Pipelining;
  2. the EHLO keyword value associated with the extension is PIPELINING;
  3. no parameter is used with the PIPELINING EHLO keyword;
  4. no additional parameters are added to either the MAIL FROM or RCPT TO commands.
  5. no additional SMTP verbs are defined by this extension; and,
  6. the next section specifies how support for the extension affects the behavior of a server and client SMTP.

4. The Pipelining Service Extension πŸ‘‘

So when a client SMTP wants to use command pipelining, it first sends the EHLO command to the server SMTP. If the server SMTP replies with code 250 to the EHLO command, and the response includes the EHLO keyword value PIPELINING, then the server SMTP has said it can handle SMTP command pipelining. Main character energy πŸ’…


4.1. Client use of pipelining πŸš€

Once the client SMTP has confirmed that the pipelining extension is supported, it may then choose to send groups of SMTP commands in batches, without waiting for a response to each individual command. The commands RSET, MAIL FROM, SEND FROM, SOML FROM, SAML FROM, and RCPT TO can all show up anywhere in a pipelined command group. The EHLO, DATA, VRFY, EXPN, TURN, QUIT, and NOOP commands can only be the last command in a group, since their success or failure changes state, and the client SMTP has to deal with that change. (NOOP is in this group so it can work as a synchronization point 🫢)

Additional commands added by other SMTP extensions may only appear as the last command in a group, unless the extensions that define those commands say otherwise πŸ“Œ

The actual transfer of message content is explicitly allowed to be the first "command" in a group. So an RSET/MAIL FROM sequence that starts a new message transaction can go in the same group as the final transfer of the headers and body of the previous message. Efficient πŸ”₯

Client SMTP implementations that use pipelining MUST check ALL statuses associated with each command in a group. Here's why this matters πŸ‘€ Say none of the RCPT TO recipient addresses were accepted. The client must still check the response to the DATA command. It can't assume DATA will be rejected just because none of the RCPT TO commands worked. If the DATA command was properly rejected, the client SMTP can just issue RSET. But if the DATA command was accepted, the client SMTP should send a single dot.

Command statuses MUST be coordinated with responses by counting each separate response and matching that count against the number of commands known to have been issued. Multiline responses MUST be supported. Matching on the basis of either the error code value or the associated text is expressly forbidden πŸ™…β€β™€οΈ

Client SMTP implementations MAY choose to operate in a nonblocking fashion, processing server responses right away on receipt, even if there's still data waiting to be sent from the client's previous TCP send operation. If nonblocking operation isn't supported, though, client SMTP implementations MUST also check the TCP window size and make sure that each group of commands fits entirely within the window. The window size is usually, but not always, 4K octets. Skipping this check can lead to deadlock conditions, and nobody wants that 😭

Clients MUST NOT confuse responses to multiple commands with multiline responses. Each command gets one or more lines of response, and the last line doesn't have a dash between the response code and the response string. Don't mix these up, bestie 🫢

5. Examples ✨

Okay so, let's look at a normal SMTP chat without pipelining first. Watch how much the client just sits there waiting πŸ™ˆ

S: <wait for open connection>
C: <open connection to server>
S: 220 innosoft.com SMTP service ready
C: HELO dbc.mtview.ca.us
S: 250 innosoft.com
C: MAIL FROM:<mrose@dbc.mtview.ca.us>
S: 250 sender <mrose@dbc.mtview.ca.us> OK
C: RCPT TO:<ned@innosoft.com>
S: 250 recipient <ned@innosoft.com> OK
C: RCPT TO:<dan@innosoft.com>
S: 250 recipient <dan@innosoft.com> OK
C: RCPT TO:<kvc@innosoft.com>
S: 250 recipient <kvc@innosoft.com> OK
C: DATA
S: 354 enter mail, end with line containing only "."
 ...
C: .
S: 250 message sent
C: QUIT
S: 221 goodbye

The client waits for a server response a total of 9 times in this simple example. That's a lot of waiting, bestie 😭

Now here's the pipelined version. Same energy, way more efficient πŸ”₯

S: <wait for open connection>
C: <open connection to server>
S: 220 innosoft.com SMTP service ready
C: EHLO dbc.mtview.ca.us
S: 250-innosoft.com
S: 250 PIPELINING
C: MAIL FROM:<mrose@dbc.mtview.ca.us>
C: RCPT TO:<ned@innosoft.com>
C: RCPT TO:<dan@innosoft.com>
C: RCPT TO:<kvc@innosoft.com>
C: DATA
S: 250 sender <mrose@dbc.mtview.ca.us> OK
S: 250 recipient <ned@innosoft.com> OK
S: 250 recipient <dan@innosoft.com> OK
S: 250 recipient <kvc@innosoft.com> OK
S: 354 enter mail, end with line containing only "."
 ...
C: .
C: QUIT
S: 250 message sent
S: 221 goodbye

The total number of turnarounds has been reduced from 9 to 4. Literally iconic πŸ’…βœ¨

Next up, one possible way things can go when pipelining is used and every recipient gets rejected. Brace yourself πŸ’€

S: <wait for open connection>
C: <open connection to server>
S: 220 innosoft.com SMTP service ready
C: EHLO dbc.mtview.ca.us
S: 250-innosoft.com
S: 250 PIPELINING
C: MAIL FROM:<mrose@dbc.mtview.ca.us>
C: RCPT TO:<nsb@thumper.bellcore.com>
C: RCPT TO:<galvin@tis.com>
C: DATA
S: 250 sender <mrose@dbc.mtview.ca.us> OK
S: 550 remote mail to <nsb@thumper.bellore.com> not allowed
S: 550 remote mail to <galvin@tis.com> not allowed
S: 554 no valid recipients given
C: QUIT
S: 221 goodbye

The client SMTP waits for the server 4 times here as well. Not the best vibe, but at least it's consistent 🫢

Now, if the server SMTP does not check for at least one valid recipient before accepting the DATA command, this is the dialogue that would happen. Plot twist πŸ‘€

S: <wait for open connection>
C: <open connection to server>
S: 220 innosoft.com SMTP service ready
C: EHLO dbc.mtview.ca.us
S: 250-innosoft.com
S: 250 PIPELINING
C: MAIL FROM:<mrose@dbc.mtview.ca.us>
C: RCPT TO:<nsb@thumper.bellcore.com>
C: RCPT TO:<galvin@tis.com>
C: DATA
S: 250 sender <mrose@dbc.mtview.ca.us> OK
S: 550 remote mail to <nsb@thumper.bellore.com> not allowed
S: 550 remote mail to <galvin@tis.com> not allowed
S: 354 enter mail, end with line containing only "."
C: .
C: QUIT
S: 554 no valid recipients
S: 221 goodbye

6. Security Considerations πŸ”

Okay, serious moment, no cap. This document doesn't discuss security issues, and it's not believed to raise any security issues that aren't already endemic to electronic mail and present in fully conforming implementations of [1]. So the security drama lives in email itself, not in this extension πŸ’€


7. Acknowledgements 🌸

This document is based on the SMTP service extension model presented in RFC 1425. Marshall Rose's description of SMTP command pipelining in his book "The Internet Message" also served as a source of inspiration for this extension. Big thank-yous, the people who came before us really did the work 🫢


8. References πŸ“š

  • [1] Postel, J., "Simple Mail Transfer Protocol", STD 10, RFC 821, August 1982.
  • [2] Klensin, J., Freed, N., Rose, M., Stefferud, E., and D. Crocker, "SMTP Service Extensions", RFC 1869, November 1995.
  • [3] Freed, N., "SMTP Service Extension for Command Pipelining", RFC 1854, October 1995.
  • [4] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", RFC 2119, March 1997.

9. Author's Address πŸ’Œ

Ned Freed
Innosoft International, Inc.
1050 Lakes Drive
West Covina, CA 91790
USA

Phone: +1 626 919 3600
Fax: +1 626 919 3614
EMail: ned.freed@innosoft.com

This document is a product of work done by the Internet Engineering Task Force Working Group on Messaging Extensions, Alan Cargille, chair. That's the whole squad behind it, periodt πŸ¦‹πŸ’œ

πŸ“¦ where this page comes from