RFC 6729

Indicating Email Handling States in Trace Fields

📅 published September 2012, from ietf

RFC 6729: Indicating Email Handling States in Trace Fields 💌

Okay bestie, this one is all about giving email's paper trail a little label for when a message gets pulled aside for special handling 🫶 It's an IETF Standards Track document from September 2012, written by D. Crocker (Brandenburg InternetWorking) and M. Kucherawy (Cloudmark, Inc.), ISSN 2070-1721. The title block doesn't list anything it obsoletes or updates 📎

Status of This Memo 📮

This is an Internet Standards Track document, which means it's official IETF stuff 👑 It's a product of the Internet Engineering Task Force (IETF), it represents the consensus of the IETF community, it got public review, and the Internet Engineering Steering Group (IESG) approved it for publication. For more on Internet Standards, check Section 2 of RFC 5741.

Info about the current status, any errata, and how to give feedback lives at http://www.rfc-editor.org/info/rfc6729 🔗

Copyright (c) 2012 IETF Trust and the persons identified as the document authors. All rights reserved 💜

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication. Please actually read those, because they explain your rights and restrictions for this document 🙈 Code Components pulled out of this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they come without warranty as described in the Simplified BSD License.

Table of Contents 🗺️

1. Introduction ....................................................2
2. Key Words .......................................................3
3. Trace Clause ....................................................3
4. Discussion ......................................................6
5. Granularity .....................................................6
6. IANA Considerations .............................................7
   6.1. MAIL Parameters Additional-registered-clauses
        Sub-Registry ...............................................7
   6.2. MAIL Parameters Registered-states Sub-Registry .............7
7. Security Considerations .........................................9
8. References .....................................................10
   8.1. Normative References ......................................10
   8.2. Informative References ....................................10
Appendix A.  Trace Field Examples .................................11
  A.1.  Typical Delivery without Obvious Extra Handling ...........11
  A.2.  Delivery with Moderation ..................................11
Appendix B.  Acknowledgements .....................................12

1. Introduction ✨

Okay so [SMTP] defines what email trace fields look like, which is usually the "Received" header field 📬 These are basically an audit trail of the path a message takes from origin to destination, and one gets added every time a message hops to a new host 🦋

Section 3.7.2 of that document says that "the most important use of Received: lines is for debugging mail faults [...]" 🐛

Sometimes there are big time gaps between trace fields. That might be a transient communication issue, but it could also be a policy decision, special processing of the message content, authorization of some identity on the message, or a handoff between major software components 👀 Common examples are message quarantines (filters that hold a message for further evaluation, or hold it until a human operator does something), pending content analysis, or mailing list servers that enforce moderation rules (the list owner has to act on mail from authors who aren't subscribed) 🙅‍♀️

This document registers a new optional clause for trace fields. It says a message entered one of these special processing queues or states for some period of time 💅 That lets anyone inspecting the trace info tell whether a gap was caused by extra processing or by transient technical difficulties. Main character clarity 👑

2. Key Words 🔑

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [KEYWORDS] 🌸

3. Trace Clause 🎀

This spec defines a clause called "state" 🧚 It MAY be used when creating a Received header field (see Section 4.4 of [SMTP]) to show what kind of extra handling got imposed on relaying a message toward its recipient(s). It's followed by a single keyword that gives the details 💖 A Mail Transfer Agent (MTA), or other handling agent, that decides a message has entered some state other than normal queuing for relaying or delivery MAY generate a trace field with one of these clauses in it. So basically, having this clause on a trace field means the message entered that state, and a later trace field would show it leaving that state ✨

An MTA implementing this specification SHOULD add a Received field as described whenever:

a. It determines that a special handling condition will occur and puts the message into that condition; or

b. It determines that no special handling is required and gets the message ready to relay to the next handling agent.

An MTA need not add a Received field for preparing a normal handoff to the next handling agent if it has already added a Received field for some other reason. Trace data added by the next handling agent will imply the message has exited the special handling condition 🦋

If a single MTA runs a message through multiple special handling conditions, it MAY add a Received field for each distinct condition 🌈

Here's an example. Say a message gets injected into MTA-1, travels to MTA-3 via MTA-2, and then MTA-3 does the final delivery 📦

  1. MTA-1 adds a typical Received field and relays the message to MTA-2.
  2. MTA-2 determines that some atypical handling will happen, which will cause a handling change outside the usual message flow. It adds a Received field using the extension specified here.
  3. Once that atypical handling is done, MTA-2 relays the message to MTA-3.
  4. MTA-3 adds a typical Received field and carries out final delivery of the message.

Using this mechanism properly does NOT mean attaching metadata to the message, like categorizing it (for example, "is spam" or "was 8-bit, converted to 7-bit") 🙅‍♀️ Processing agents also can't reliably use this mechanism to learn anything about the message content, since there's no guarantee that every agent in the handling chain made the annotations needed to reach correct conclusions. The only purpose here is to let someone find the point(s) in the chain of custody where the message got handled outside normal routing and queuing 🕵️‍♀️

These state keywords are defined in this document. Extensions may define other registered keywords (see Section 6.2) 💅

  • auth: The message entered a queue pending authentication of some identifier in the message.
  • content: The message entered a queue pending content analysis, such as scanning for spam or viruses.
  • convert: The message entered a queue pending content conversion.
  • moderation: The message entered a hold pending mailing list moderator action.
  • normal: The message is not in an administrative hold and is queued for or is being handed off to the next handling agent (which may be local delivery). This is the default interpretation when no "state" clause is present.
  • other: The message entered a hold or queue for reasons not covered by other keywords in this list and not for transient technology issues.
  • outbound: The message entered a queue for outbound relaying. This is typically the last case added for a single host, and the next Received header field is expected to be added by some other host.
  • quarantine: The message entered a hold in an isolation queue pending operator action for local policy reasons.
  • timed: The message entered a hold in order to meet a requested delivery window, such as the one defined in [FUTURERELEASE].

In Section 6, the "state" clause gets added to the Additional-Registered-Clauses IANA sub-registry. The ABNF for this clause is:

State = CFWS "state" FWS queue-state-keyword [ "/" value ]

queue-state-keyword = ( reg-state-keyword / unreg-state-keyword )

reg-state-keyword = ( "auth" / "content" / "convert" /
                      "moderation" / "normal" / "other" /
                      "outbound" / "quarantine" / "timed" /
                      additional-state-keyword )

additional-state-keyword = token
                         ; MUST be registered; see
                         ; "IANA Considerations" below

value = token

unreg-state-keyword = token

"FWS" and "CFWS" are defined in [MAIL]. "token" is defined in [MIME] 🧩

A transfer agent using this extension MAY also include header field comments to give extra info 💬

The "value" is there for extra labels that explain the state transition. Examples could include:

  • convert/unicode2ascii
  • moderation/not-subscribed
  • quarantine/spam

4. Discussion ✨

Okay so handling agents aren't expected to implement or support all of these 💅. Recording trace info for every single state above could make the message header inordinately large 😭, and nobody wants that. So instead, an agent is encouraged to drop a state annotation when a message enters a handling queue where something significant happens, or where significant extra processing or delay is possible 🫶. It's especially important when a handoff has happened between two different, independent agents 🔥.

Let's say an MTA receives a message, does message authentication, scans for viruses and spam, and then puts it in an outbound queue. It could add four Received header fields, one for each of those states. But if they're all done as part of one system process in a single pass, that would be unusual (and extremely verbose) 💀. This method SHOULD NOT be applied except when doing detailed analysis of a single component to identify performance issues with those steps 🙈.

Instead, an agent that wants to make a state annotation SHOULD add only a single Received header field that includes that annotation ✨. That one field tells us (a) when the agent finished handling the message, through the date portion of the field, and (b) the final disposition of that message relative to that agent. For example, say an MTA receives a message, runs a bunch of checks on it, and then immediately hands it off to a Mailing List Manager (MLM). That MTA would only record a "normal" state, assuming the message passes those checks. Then the MLM evaluates the message and records its own state once it decides what happens next with it 🦋.

5. Granularity 🌸

How granular you get, and therefore how verbose the trace gets with this extra trace clause, is likely to vary depending on the situation 🤷‍♀️. Usually this clause will only show up for "unusual" transitions, like when a message needs extra scrutiny or other processing, or needs to be quarantined 🔒.

Somewhat greater granularity might also include transitions of administrative responsibility, like moving between a Mail Transfer Agent (MTA) operator and a Mailing List Manager (MLM) operator 👑. It could go even further by noting some transitions that only matter when other transitions have happened. For example, you could note entry to the outbound queue only when the message comes from an "interesting" source, like an MLM. That's because an MLM can make big changes to the message or the delivery delay, so it's useful to know when its processing finished, separate from the later processing by the originating MTA ⏳. If you need really fine-grained trace info, fields could be created to note all of these "significant" network architecture transitions 🧚.

Here's the thing though 👀: when you pick higher levels of granularity, remember that MTAs might count the Received header fields on a message when they're trying to decide whether a message routing loop is in effect. A message with a ton of these could be wrongly judged to be in a delivery loop, and then get removed from the mail stream 😱. See Section 6.3 of [SMTP] for further discussion.

6. IANA Considerations 🎀

6.1. MAIL Parameters Additional-registered-clauses Sub-Registry

This document adds the following entry to the "Additional-registered-clauses" sub-registry of the "MAIL Parameters" registry, which was created by [SMTP] 🦋:

  • Clause name: state
  • Description: Indicates entry into a special queue state
  • Syntax Summary: state <state-name>
  • Reference: [RFC6729]

6.2. MAIL Parameters Registered-states Sub-Registry

The "MAIL Parameters" registry at IANA has been updated with a new "Registered-states" sub-registry, which holds the valid state keywords for use with this specification 💜. Updates to this registry are governed by the First Come, First Served rules of [IANA] for new registrations. Changes to the status of existing entries are limited to the original registrant or IESG approval.

Discussion of all registry updates is encouraged on one or more IETF mailing lists that usually cover email-related subjects, before the change gets approved. It's a way of documenting the work 📝. The ietf-smtp@ietf.org list is suggested.

Note that only registrations of queue state keywords are permitted 🚫. The registry is not for specifying secondary information, meaning the "value" part of the ABNF in Section 3.

Registrations are supposed to include the following entries:

  • Name: The name of the state keyword being defined or updated. It has to conform to the ABNF shown in Section 3.
  • Description: A brief description of what the keyword means.
  • Specification: The specification document that defines the queue state being registered. If no stable reference exists, you give a more detailed explanation of the queue state than what's in the "Description", enough to allow interoperability 🤝.
  • Use: One of current (the state keyword is in current use), deprecated (the state keyword is still in use but not recommended for new implementations), or historic (the state keyword is no longer in substantial current use).

The initial registration set looks like this 💅:

+------------+------------------------+-----------------+---------+
| Name       | Description            | Specification   | Use     |
+------------+------------------------+-----------------+---------+
| auth       | Held for message       |    [RFC6729]    | current |
|            | authentication         |                 |         |
+------------+------------------------+-----------------+---------+
| content    | Held for message       |    [RFC6729]    | current |
|            | content analysis       |                 |         |
+------------+------------------------+-----------------+---------+
| convert    | Held for message       |    [RFC6729]    | current |
|            | content conversion     |                 |         |
+------------+------------------------+-----------------+---------+
| moderation | Held for list          |    [RFC6729]    | current |
|            | moderation             |                 |         |
+------------+------------------------+-----------------+---------+
| normal     | Message is not being   |    [RFC6729]    | current |
|            | held other than to     |                 |         |
|            | accommodate typical    |                 |         |
|            | relaying handling      |                 |         |
+------------+------------------------+-----------------+---------+
| other      | Held for causes not    |    [RFC6729]    | current |
|            | covered by other       |                 |         |
|            | registered state       |                 |         |
|            | keywords               |                 |         |
+------------+------------------------+-----------------+---------+
| outbound   | Message placed in      |    [RFC6729]    | current |
|            | outbound queue         |                 |         |
+------------+------------------------+-----------------+---------+
| quarantine | Held for operator      |    [RFC6729]    | current |
|            | action due to content  |                 |         |
|            | analysis or local      |                 |         |
|            | policy                 |                 |         |
+------------+------------------------+-----------------+---------+
| timed      | Held to accommodate a  |    [RFC6729]    | current |
|            | specific requested     |                 |         |
|            | delivery window        |                 |         |
+------------+------------------------+-----------------+---------+

7. Security Considerations 🔐

⚠️ Heads up, this one is serious. The use of this trace information can reveal hints about the local policy that was in effect at the time the message was handled. Anyone reading the trace could learn how your system was operating, so treat it like the sensitive info it is 🙀.

Further discussion about trace field security can be found in Section 7.6 of [SMTP].

8. References 📚

8.1. Normative References

  • [IANA] Narten, T. and H. Alvestrand, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 5226, May 2008.
  • [KEYWORDS] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
  • [MAIL] Resnick, P., Ed., "Internet Message Format", RFC 5322, October 2008.
  • [MIME] Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies", RFC 2045, November 1996.
  • [SMTP] Klensin, J., "Simple Mail Transfer Protocol", RFC 5321, October 2008.

8.2. Informative References

  • [FUTURERELEASE] White, G. and G. Vaudreuil, "SMTP Submission Service Extension for Future Message Release", RFC 4865, May 2007.

Appendix A. Trace Field Examples 🧁

This section has a sample of the new trace field clause in action 👀.

A.1. Typical Delivery without Obvious Extra Handling

Typical message delivery:

     Received: from newyork.example.com
               (newyork.example.com [192.0.2.250])
           by mail-router.example.net (8.11.6/8.11.6)
               with ESMTP id i7PK0sH7021929
               for <recipient@example.net>;
           Fri, Feb 15 2002 17:19:22 -0800
     Received: from internal.example.com
               (internal.example.com [192.168.0.1])
           by newyork.example.com (8.11.6/8.11.6)
               with ESMTP id i9MKZCRd064134
               for <recipient@example.net>;
           Fri, Feb 15 2002 17:19:08 -0800

Example 1: Typical message delivery with no real extra handling. Only the Received header fields are shown 💖.

A.2. Delivery with Moderation

Message delivery after moderation:

     Received: from newyork.example.com
               (newyork.example.com [192.0.2.250])
           by mail-router.example.net (8.11.6/8.11.6)
               with ESMTP id i7PK0sH7021929
               for <recipient@example.net>;
           Fri, Feb 15 2002 18:33:29 -0800
     Received: from internal.example.com
               (internal.example.com [192.168.0.1])
           by newyork.example.com (8.11.6/8.11.6)
               with ESMTP id i9MKZCRd064134
               for <secret-list@example.com>
               state moderation (sender not subscribed);
           Fri, Feb 15 2002 17:19:08 -0800

Example 2: Message held for moderation. Only the Received header fields are shown 🙈.

In this one, the message goes from internal.example.com to newyork.example.com, and it's meant for a mailing list hosted on the latter. For list admin reasons, the message gets held there for moderation. It's finally released over an hour later and passed on to the next host ⏰. The comment after the state expression tells you the actual reason for the administrative hold 💅.

Appendix B. Acknowledgements 🫶

The authors want to thank the following people for their reviews and constructive criticism of this proposal 💜: Tony Finch, Ned Freed, Carl S. Gutenkunst, John Levine, Bill McQuillan, S. Moonesamy, Alexey Melnikov, Robert A. Rosenberg, Hector Santos, Rolf Sonneveld, and Mykyta Yevstifeyev 🌈.

Authors' Addresses 🎀

okay so the last bit is where you find the people who wrote this thing 🫶 bestie, these are the contacts for both authors, copied exactly as they appear in the original 💅

D. Crocker
Brandenburg InternetWorking
675 Spruce Dr.
Sunnyvale  94086
USA

Phone: +1.408.246.8253
EMail: dcrocker@bbiw.net
URI:   http://bbiw.net


Murray S. Kucherawy
Cloudmark, Inc.
128 King St., 2nd Floor
San Francisco, CA  94107
USA

EMail: superuser@gmail.com
📦 where this page comes from