RFC 6783
Mailing Lists and Non-ASCII Addresses
RFC 6783: Mailing Lists and Non-ASCII Addresses ✨💌
The tea 🍵: RFC 6783 is an Informational doc from the IETF, dated November 2012. It was written by J. Levine (Taughannock Networks) and R. Gellens (Qualcomm Incorporated), and its ISSN is 2070-1721. It obsoletes RFC 5983, and it doesn't update anything. It's the RFC number 6783, bestie 🎀
Abstract ✨
okay so this doc is all about what happens to mailing lists once non-ASCII UTF-8 email addresses show up 🌈 It walks through some possible scenarios for lists that have a mix of non-ASCII addresses and traditional ASCII ones. What it does NOT do is define protocol changes or give implementation or deployment advice, so don't go looking for that here 🙈
Status of This Memo
this one is not an Internet Standards Track spec. It's published purely for informational purposes 📚
it's still a real product of the IETF though, and it represents the consensus of the IETF community. It went through public review and got approved for publication by the Internet Engineering Steering Group (IESG). Not every IESG-approved doc is a candidate for any level of Internet Standard, though, so check Section 2 of RFC 5741 if you want the full story 💅
for the latest status, errata, or a way to send feedback, head to http://www.rfc-editor.org/info/rfc6783
Copyright Notice
Copyright (c) 2012 IETF Trust and the persons identified as the document authors. All rights reserved. 🔒
this doc is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as they were in effect on the publication date. Please actually read them, because they spell out your rights and restrictions for this document. Any code components pulled out of this doc must include the Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come without warranty, as the Simplified BSD License says 🫶
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.1. Mailing List Header Additions and Modifications . . . . . . 3
1.2. Non-ASCII Email Addresses . . . . . . . . . . . . . . . . . 3
2. Scenarios Involving Mailing Lists . . . . . . . . . . . . . . . 4
2.1. Fully SMTPUTF8 Lists . . . . . . . . . . . . . . . . . . . 4
2.2. Mixed SMTPUTF8 and ASCII Lists . . . . . . . . . . . . . . 5
2.3. SMTP Issues . . . . . . . . . . . . . . . . . . . . . . . . 5
3. List Headers . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.1. SMTPUTF8 List Headers . . . . . . . . . . . . . . . . . . . 6
3.2. Downgrading List Headers . . . . . . . . . . . . . . . . . 7
3.3. Subscribers' Addresses in Downgraded Headers . . . . . . . 8
4. Security Considerations . . . . . . . . . . . . . . . . . . . . 8
5. References . . . . . . . . . . . . . . . . . . . . . . . . . . 8
5.1. Normative References . . . . . . . . . . . . . . . . . . . 8
5.2. Informative References . . . . . . . . . . . . . . . . . . 91. Introduction ✨
okay so, this section sets the scene 🎬 It covers considerations for mailing lists now that non-ASCII UTF-8 email addresses exist. The usage of those addresses is described in [RFC6530] 💖
mailing lists are a huge deal for email and for collaborating, so this matters. Introducing internationalized email addresses affects lists in three main areas: (1) transport, meaning receiving and sending messages; (2) message headers of received and retransmitted messages; and (3) mailing list operational policies 🧚
here's how a list works. A mailing list spreads one message out to many recipients when the originator sends it to a single address. An agent at that address (usually software, not a person) gets the message and then redistributes it to a list of recipients 📬
the list agent usually sets the envelope return address (henceforth called the "bounce address") of the redistributed message to a different address than the one on the original message. Why does that matter? A different bounce address sends error and other automatic messages to an error-handling address tied to the list. Those go to the list agent, which can often do something useful with them 🙌 If they went to the original sender instead, that person usually doesn't control the list, so they can't do anything about them. Not me being dramatic, just facts 💅
in most cases, the list agent sends a received message to its subscribers as a brand new message. Conceptually, it uses message submission [RFC6409], the same way the original sender did 🌸 There's one exception this doc does NOT address: when the list isn't run by a separate agent that receives and redistributes messages in separate transactions, and instead an expansion step inside a single SMTP transaction turns one local address into multiple local or non-local addresses.
1.1. Mailing List Header Additions and Modifications
some list agents change message header fields, and others don't 🤷♀️ There's a bunch of standardized list-related header fields, and lots of lists add one or more of them.
Separately from those, some lists alter or add header fields to control where replies go, even though this has a history of interoperability problems. Those lists usually add or replace the "Reply-To" field, and some add or replace the "Sender" field too. Some also change other fields, including "From" 😭
Among the list-specific header fields are the ones specified in RFCs 2369 [RFC2369] and 2919 [RFC2919]. For more info, see Section 3 🎀
1.2. Non-ASCII Email Addresses
the mail transport protocol is the same for regular recipients and mailing list recipients. But list agents have special considerations with non-ASCII email addresses, because they retransmit messages that other agents composed to potentially loads of recipients 🦋
there are considerations for non-ASCII addresses in the envelope AND in the header fields of redistributed messages. The big one: a message with non-ASCII addresses in its headers or envelope cannot be sent to non-SMTPUTF8 recipients 🚫
with mailing lists, there are two kinds of considerations. The first are the purely technical ones: message handling, error cases, and that sort of thing. The second come from the fact that humans are the ones using the lists to communicate 🫠 For the technical side, a list agent might choose to reject all messages from non-ASCII addresses if it isn't ready to handle SMTPUTF8 mail.
For the human side: a user who sends to a list is often unaware of who's on it. They usually don't know whether the members are SMTPUTF8 users, and often neither the original sender nor the recipients personally know each other 👀 So the fixes that work fine for one-to-one chats might not be appropriate for lists. Say a one-to-one message is undeliverable: you can usually just text, DM, or otherwise reach out to get a working address. With a list, the users may have no recourse at all. And of course, the original sender usually has no idea which list members got the message, or whether it failed for some of them 💀
conceptually, a list's internationalization breaks down into three capabilities:
- Does the list have a non-ASCII submission address?
- Does the list agent accept subscriptions for addresses that contain non-ASCII characters?
- Does the list agent accept messages that require SMTPUTF8 capabilities?
if a list has subscribers with ASCII addresses, those subscribers might or might not be able to accept SMTPUTF8 messages ✨
2. Scenarios Involving Mailing Lists 🗺️
okay, this part is the scenarios. Generally (and only within the scope of this doc), an original message goes to a mailing list as a completely separate and independent transaction from the list agent sending the retransmitted message to one or more recipients 💅 In both cases, the message might be addressed only to the list address, or it might also have recipients beyond the list.
The list agent might send the retransmitted message to each recipient in a separate message submission transaction, or it might bundle multiple recipients into one transaction. Often list agents are built to work WITH a message submission server rather than including that functionality themselves. So the list hands one copy of the retransmitted message to a single submission server, and that server decides which recipients go in which transaction 🧚
2.1. Fully SMTPUTF8 Lists
some lists want to be fully SMTPUTF8, meaning all subscribers are expected to be able to receive SMTPUTF8 mail 💜 For list hygiene, a list like that would probably want to block subscriptions from addresses that can't receive SMTPUTF8 mail.
if a putative subscriber has a non-ASCII address, it must be able to receive SMTPUTF8 mail. But there's no way to tell whether a subscriber with an ASCII address can receive SMTPUTF8 mail, unless you send an SMTPUTF8 probe or confirmation message and somehow find out whether it was delivered. For example, that could happen if the user clicked a link in the confirmation message 🔍
2.2. Mixed SMTPUTF8 and ASCII Lists
other lists want to handle a mix of SMTPUTF8 and ASCII subscribers. That could be a transitional thing while subscribers upgrade to SMTPUTF8-capable mail software, or it could be an ongoing feature 🌈 It's not possible in general to downgrade SMTPUTF8 mail to ASCII mail. But list software might split the recipients into two sets (SMTPUTF8 and ASCII) and create a downgraded version of the SMTPUTF8 list messages for the ASCII recipients. See Sections 3.2 and 3.3 🎀
to figure out which set an address belongs in, list software might make the conservative assumption that ASCII addresses get ASCII messages. It might try probing the address with an SMTPUTF8 test message, or it might let the subscriber pick the message format manually. That last one is kind of like how some lists already let people choose plain text versus HTML mail, or individual messages versus a daily digest 🍓
to decide whether a message needs downgrading for ASCII recipients, list software might assume that any message received over an SMTPUTF8 SMTP session is an SMTPUTF8 message. Or it might examine the headers and body to see whether they need SMTPUTF8 treatment. Depending on the interface between the list software and the Mail Transfer Agent (MTA) and Mail Delivery Agent (MDA) that handle incoming messages, it may not be able to tell what type of session an incoming message came in on 🫶
2.3. SMTP Issues 💌
okay so mailing list software almost always swaps out the envelope addresses on every single message 💅 The bounce address gets set to something that sends bounces back to the list agent, and the recipient addresses get set to the subscribers. For some lists, every message gets the same bounce address. For others, the list software might make a bounce address per recipient, or even a unique one per message per recipient. That's the bounce management technique called Variable Envelope Return Paths or VERP [VERP], and it's giving very organized energy 🧚
The bounce address usually includes the name of the list, so a list with a non-ASCII name ends up with a non-ASCII bounce address 😭 The paths bounces might take are kind of a mystery, so list software might go with an ASCII bounce address instead, which makes it more likely the bounce actually makes it home to the list agent 🏠 Same deal with VERP: each subscriber's VERP address usually has a version of their address baked in, so a non-ASCII subscriber address means a non-ASCII VERP bounce address. For that same reason, list software might use ASCII bounce addresses that encode the recipient's identity some other way 🫶
3. List Headers 📋
List agents usually slap list-specific headers onto each message before they resend it to the list recipients 🎀
3.1. SMTPUTF8 List Headers 🌸
The list headers in RFCs 2369 [RFC2369] and 2919 [RFC2919] were all written before SMTPUTF8 mail existed, so their definitions don't say anything about where non-ASCII characters might show up. Some examples:
List-Id: List Header Mailing List
<list-header.example.com>
List-Help:
<mailto:list@example.com?subject=help>
List-Unsubscribe:
<mailto:list@example.com?subject=unsubscribe>
List-Subscribe:
<mailto:list@example.com?subject=subscribe>
List-Post:
<mailto:list@example.com>
List-Owner:
<mailto:listmom@example.com> (Contact Person for Help)
List-Archive:
<mailto:archive@example.com?subject=index%20list>As [RFC2369] puts it, "[t]he contents of the list header fields mostly consist of angle-bracket ('<', '>') enclosed URLs, with internal whitespace being ignored". [RFC2919] says "[t]he list identifier will, in most cases, appear like a host name in a domain of the list owner". Since these headers were made for ASCII mail, they only allow ASCII text, URLs included 🙃
The most commonly used URI schemes in List-* headers are http and mailto [RFC6068]. https and ftp show up sometimes too, and in principle any valid URI can go in there 🌈
Even if a scheme allows an internationalized form, it should use a pure ASCII form of the URI described in [RFC3986]. Future work might extend these header fields, or define replacements that directly support unencoded non-ASCII outside the ASCII repertoire in these and other header fields. Until that happens, non-ASCII characters can only be included by encoding them as ASCII 💀
Here's the tea 🍵 [RFC3986] says to use a pair of hex digits preceded by a percent sign. But percent signs were already used informally in mail addresses for source routing 🙈 Few mail systems still permit source routing, but a lot of mail software still forbids or escapes the characters that used to mean source routing. That can cause messy interactions with percent-encoded URIs, or any URI that includes one of those characters. If a program reading a mailto: URI knew the Mail User Agent (MUA) in use could handle non-ASCII data, it could pass the URI along unencoded and skip the misread percent-sign problem. But right now there's no standard, or even informal, way for MUAs to signal SMTPUTF8 capabilities. Also, whether internationalized domain names should be percent-encoded or appear in A-label form [RFC5890] depends on the context they show up in 🧐
The List-ID header field uniquely identifies a list 🆔 The whole point is that its value stays constant, even if the machine or system hosting the list changes. It's used all the time in filters and tests, like client-side filters, Sieve filters [RFC5228], and so forth. If List-ID were extended to allow non-ASCII text, filters and tests might not properly compare encoded and unencoded versions of the same non-ASCII value, and that's a whole mess 😩 Beyond comparisons, it's generally desirable for this field to contain something meaningful that users can actually type. ASCII encodings of non-ASCII characters are unlikely to be meaningful to users or easy for them to type accurately, though 💔
3.2. Downgrading List Headers 🔽
If list software prepares a downgraded version of an SMTPUTF8 message, all the List-* headers must be downgraded. In particular, if a List-* header contains a non-ASCII mailto (even encoded in ASCII), it may be advisable to edit the header to remove the non-ASCII address, or replace it with an equivalent ASCII address if the list software knows one. Otherwise, a client might run into trouble if the decoded mailto ends up as a non-ASCII address. If a header that contains a mailto URL gets downgraded by percent encoding, some mail software may misinterpret the percent signs as attempted source routing 🙅♀️
When downgrading list headers, it may not be possible to produce a downgraded version that's satisfactorily equivalent to the original. In particular, if a non-ASCII List-ID gets downgraded to an ASCII version, software and humans at recipient systems will typically not be able to tell that both refer to the same list 🫠
If lists permit mail with multiple MIME parts, some MIME headers in SMTPUTF8 messages may include non-ASCII characters in file names and other descriptive text strings. Downgrading these strings may lose the sense of the names, break references from other MIME parts (such as HTML IMG references to embedded images), and otherwise damage the mail 🖼️💥
3.3. Subscribers' Addresses in Downgraded Headers 📭
List software typically leaves the original submitter's address in the From: line. That way recipients can tell who wrote the message and choose whether to reply to the list or directly to the submitter 💁 If a submitter has a non-ASCII address, there's no way to downgrade the From: header and preserve the address so ASCII recipients can respond to it, since non-SMTPUTF8 mail systems can't send mail to non-ASCII addresses 🚫
Possible work-arounds (none implemented that we know of) might include:
- letting subscribers with non-ASCII addresses register an alternate ASCII address with the list software,
- having the list software itself create ASCII forwarding addresses, or
- just putting the list's address in the From: line and losing the ability to respond directly to the submitter 🤷
4. Security Considerations 🔐
None beyond what mailing list agents do now.
5. References 📚
5.1. Normative References
[RFC3986] Berners-Lee, T., Fielding, R., and L. Masinter, "Uniform
Resource Identifier (URI): Generic Syntax", STD 66,
RFC 3986, January 2005.
[RFC6068] Duerst, M., Masinter, L., and J. Zawinski, "The 'mailto'
URI Scheme", RFC 6068, October 2010.
[RFC6409] Gellens, R. and J. Klensin, "Message Submission for Mail",
STD 72, RFC 6409, November 2011.
[RFC6530] Klensin, J. and Y. Ko, "Overview and Framework for
Internationalized Email", RFC 6530, February 2012.5.2. Informative References
[RFC2369] Neufeld, G. and J. Baer, "The Use of URLs as Meta-Syntax
for Core Mail List Commands and their Transport through
Message Header Fields", RFC 2369, July 1998.
[RFC2919] Chandhok, R. and G. Wenger, "List-Id: A Structured Field
and Namespace for the Identification of Mailing Lists",
RFC 2919, March 2001.
[RFC5228] Guenther, P. and T. Showalter, "Sieve: An Email Filtering
Language", RFC 5228, January 2008.
[RFC5890] Klensin, J., "Internationalized Domain Names for
Applications (IDNA): Definitions and Document Framework",
RFC 5890, August 2010.
[VERP] Bernstein, D., "Variable Envelope Return Paths",
February 1997, <http://cr.yp.to/proto/verp.txt>.Authors' Addresses 🦋
John Levine
Taughannock Networks
PO Box 727
Trumansburg, NY 14886
US
Phone: +1 831 480 2300
EMail: standards@taugh.com
URI: http://jl.ly
Randall Gellens
Qualcomm Incorporated
5775 Morehouse Drive
San Diego, CA 92121
US
EMail: rg+ietf@qti.qualcomm.com📦 where this page comes from
- ietf: rfc6783 2012-11
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms