RFC 6530

Overview and Framework for Internationalized Email

📅 published February 2012, from ietf

Overview and Framework for Internationalized Email 💌

RFC 6530 🎀 is the one where J. Klensin and Y. Ko wrote the overview for internationalized email. It's a Standards Track document, dated February 2012, with ISSN 2070-1721. It obsoletes RFC 4952, RFC 5504, and RFC 5825 💅, and it's a replacement for RFC 4952 in particular. It doesn't update anything. So if you're looking at old drafts of these, this is the one that counts 🌸.

Abstract

Okay so, using email all over the world means people need to be able to use close variations on their own names as mailbox names in their addresses, written correctly in their own languages and scripts 🌍✨. That's the goal, subject to some other constraints. This document introduces a whole series of specs that define the mechanisms and protocol extensions needed to fully support internationalized email addresses 🦋. The big pieces are an SMTP extension and an extension of email header syntax so it can handle UTF-8 data 💜. It also talks through key assumptions and issues you'll run into when deploying fully internationalized email, which is honestly very important 🙈. This document replaces RFC 4952 because it picks up extra issues that showed up after that one came out 💖.

Status of This Memo

This is an Internet Standards Track document 🔥.

It's a product of the Internet Engineering Task Force (IETF) and represents the consensus of the IETF community 🫶. It got public review and was approved for publication by the Internet Engineering Steering Group (IESG) 👑. If you want more on Internet Standards, check out Section 2 of RFC 5741 🧚.

For the current status, any errata, and how to give feedback, go to http://www.rfc-editor.org/info/rfc6530 🍓.

Copyright (c) 2012 IETF Trust and the persons identified as the document authors. All rights reserved 💅.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents, which you can find at http://trustee.ietf.org/license-info, as they stand on the date this document was published. Please read these carefully, because they spell out your rights and restrictions for this document 🙈. Code Components pulled out of this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they come without warranty as described in the Simplified BSD License 🌈.

This document may include material from IETF Documents or IETF Contributions that were published or made publicly available before November 10, 2008 📜. The people who control the copyright in some of that material may not have given the IETF Trust permission to allow modifications outside the IETF Standards Process. So unless you get an adequate license from those copyright holders, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be made outside the IETF Standards Process 😭. The one exceptions are formatting it for publication as an RFC or translating it into languages other than English 🫶.

Table of Contents

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  4
   2.  Role of This Specification . . . . . . . . . . . . . . . . . .  4
   3.  Problem Statement  . . . . . . . . . . . . . . . . . . . . . .  5
   4.  Terminology  . . . . . . . . . . . . . . . . . . . . . . . . .  6
     4.1.  Mail User and Mail Transfer Agents . . . . . . . . . . . .  6
     4.2.  Address Character Sets . . . . . . . . . . . . . . . . . .  7
     4.3.  User Types . . . . . . . . . . . . . . . . . . . . . . . .  7
     4.4.  Messages . . . . . . . . . . . . . . . . . . . . . . . . .  8
     4.5.  Mailing Lists  . . . . . . . . . . . . . . . . . . . . . .  8
     4.6.  Conventional Message and Internationalized Message . . . .  8
     4.7.  Undeliverable Messages, Notification, and Delivery
           Receipts . . . . . . . . . . . . . . . . . . . . . . . . .  8
   5.  Overview of the Approach and Document Plan . . . . . . . . . .  9
   6.  Review of Experimental Results . . . . . . . . . . . . . . . .  9
   7.  Overview of Protocol Extensions and Changes  . . . . . . . . . 10
     7.1.  SMTP Extension for Internationalized Email Address . . . . 10
     7.2.  Transmission of Email Header Fields in UTF-8 Encoding  . . 11
     7.3.  SMTP Service Extension for DSNs  . . . . . . . . . . . . . 12
   8.  Downgrading before and after SMTP Transactions . . . . . . . . 12
     8.1.  Downgrading before or during Message Submission  . . . . . 13
     8.2.  Downgrading or Other Processing after Final SMTP
           Delivery . . . . . . . . . . . . . . . . . . . . . . . . . 14
   9.  Downgrading in Transit . . . . . . . . . . . . . . . . . . . . 15
   10. User Interface and Configuration Issues  . . . . . . . . . . . 15
     10.1. Choices of Mailbox Names and Unicode Normalization . . . . 15
   11. Additional Issues  . . . . . . . . . . . . . . . . . . . . . . 17
     11.1. Impact on URIs and IRIs  . . . . . . . . . . . . . . . . . 17
     11.2. Use of Email Addresses as Identifiers  . . . . . . . . . . 17
     11.3. Encoded Words, Signed Messages, and Downgrading  . . . . . 18
     11.4. Other Uses of Local Parts  . . . . . . . . . . . . . . . . 18
     11.5. Non-Standard Encapsulation Formats . . . . . . . . . . . . 19
   12. Key Changes from the Experimental Protocols and Framework  . . 19
   13. Security Considerations  . . . . . . . . . . . . . . . . . . . 19
   14. Acknowledgments  . . . . . . . . . . . . . . . . . . . . . . . 21
   15. References . . . . . . . . . . . . . . . . . . . . . . . . . . 21
     15.1. Normative References . . . . . . . . . . . . . . . . . . . 21
     15.2. Informative References . . . . . . . . . . . . . . . . . . 22

1. Introduction ✨

Bestie, to use internationalized email addresses, you have to internationalize both the domain part and the local part 🫶. The domain part is already internationalized [RFC5890], but the local part is not 💀. Without the extensions in this document, the mailbox name is limited to a subset of 7-bit ASCII [RFC5321] 😭. MIME [RFC2045] lets non-ASCII data travel around, but it doesn't give you a way to do internationalized email addresses 🙈.

RFC 2047 [RFC2047] is where MIME gets a coding mechanism for some specific message header fields so they can hold non-ASCII data 🌸. But it still doesn't let you use email addresses that have non-ASCII characters in them 💅. Without the extensions defined here, or some equivalent set, the only way to get non-ASCII characters into any part of an email address is to RFC 2047 code them into what RFC 5322 [RFC5322] calls the "display name" of the relevant header fields 🦋. That's the name phrase (or other terms elsewhere). The catch is that information hidden in the display name is invisible in the message envelope, and for lots of purposes it isn't part of the address at all 😭. Not me thinking this is a giant workaround with a glow-up needed 💅.

This document replaces RFC 4952 [RFC4952] and picks up extra issues, shared terminology, and some architectural changes that showed up after that one was published ✨. It obsoletes that document 💖. The experimental descriptions of in-transit downgrading [RFC5504] [RFC5825] are now irrelevant and no longer needed, thanks to the changes in Section 12 🔥. The RFC Editor is asked to move all three of those documents to Historic 🧚.

The pronouns "he" and "she" are used interchangeably to mean a human of indeterminate gender 🌈.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14, RFC 2119 [RFC2119] 🍓.

2. Role of This Specification 👑

This document gives the overview and framework for the next stage of email internationalization 💜. This new stage needs more than just internationalized addresses and header fields 🌍. It also needs associated transport and delivery models 🦋.

A prior version of this specification, RFC 4952 [RFC4952], also introduced a series of experimental protocols [RFC5335] [RFC5336] [RFC5337] [RFC5504] [RFC5721] [RFC5738] [RFC5825] ✨. This revised version gives overview and conceptual info for the Standards Track successors of a subset of those protocols 💖. You can find the details of the documents and how they relate in Section 5, and what was learned from the experimental protocols and their implementations is in Section 6 🎀.

Taken together, these specifications give you the details for how to implement and support internationalized email 🫶. This document explains how the different pieces of email internationalization fit together, and how the main specifications for message transport, header formats, and handling relate to each other 🔥.

This document, and the others in the collection described above, assume you already have a reasonable familiarity with the basic Internet electronic mail specifications and terminology [RFC5321] [RFC5322] 📜, plus the MIME [RFC2045] and 8BITMIME [RFC6152] ones 🌸. A general familiarity with the terminology and functions of IDNA [RFC5890] [RFC5891] [RFC5892] [RFC5893] [RFC5894] is also assumed 🧚. It's not strictly required to implement this specification, but it helps a lot 💅.

3. Problem Statement ✨

okay so bestie, let's talk about the real tea here 🫖💅. Internationalizing Domain Names in Applications (IDNA) [RFC5890] lets us have internationalized domain names, but honestly? Most people still can't use them yet 😭. One big reason is that we don't have fully internationalized naming schemes yet. Domain names are just ONE of the many names and identifiers that need to be internationalized, and until the rest catch up, internationalized domain names alone are giving very little value 🥲.

Email addresses are the perfect example of why just fixing the domain name isn't enough 📧🌸. As most people have figured out from experience, users strongly prefer email addresses that look like names or initials 🎀 over seemingly meaningless strings of letters or numbers 💀. Unless the whole email address can use familiar characters and formats, users will feel like email is culturally unfriendly, and that's not it 🙅‍♀️. If the names and initials in email addresses can be written in people's own native languages and writing systems, the Internet will feel way more natural 🌈, especially for people whose native language isn't written in a subset of a Roman-derived script 🫶.

Here's the thing: internationalizing email addresses is NOT just swapping out the SMTP envelope 🙄, or changing the "From:", "To:", and "Cc:" header fields, or letting upgraded Mail User Agents (MUAs) decode some special coding and then show local characters. To feel actually usable, addresses have to be internationalized and handled consistently in EVERY context where they show up 💖. This is a huge requirement with far-reaching implications, bestie 😤:

  • Collections of patches and workarounds are not adequate 🩹. Even if they were, a workaround-based approach could leave us with a bunch of implementations, each with a different mix of patches and workarounds, and then users are super confused about what's actually usable and supported 😵‍💫.
  • So instead, we need to build a fully internationalized email environment 👑, focused on letting people who share a language and writing system communicate efficiently ✨.
  • That means changing the mail header environment so the header fields that make sense to internationalize can use the full range of Unicode characters 🦋.
  • It also means an SMTP extension that allows UTF-8 [RFC3629] [RFC5198] mail addressing and delivery of those extended header fields 💌.
  • We also need support for internationalization of delivery and service notifications [RFC3461] [RFC3464] 🔔.
  • And (finally!) we need the 8BITMIME SMTP extension [RFC6152] to be required, so all of this can travel through the mail system without running into the limitation that header fields don't have content-transfer-encodings 🙈.

4. Terminology 📚

okay so this doc assumes you already know the protocols and terms of the core email standards, which are written up in RFC 5321 [RFC5321] and RFC 5322 [RFC5322] 🤓.

4.1. Mail User and Mail Transfer Agents 🧚

Lots of this document depends on the ideas of "Mail Transfer Agent" ("MTA") and "Mail User Agent" ("MUA") 💜. BUT, it's important to know that those terms and ideas came AFTER the Internet's email architecture was designed and the "protocols on the wire" principle was applied to it 🕰️. That email architecture, as it evolved, and that "on the wire" principle have kept anyone from making strong, standardized distinctions about how MTAs and MUAs interact on a given origin or destination host, or even whether they're separate things at all 🤷‍♀️.

That said, this document uses the term "final delivery MTA" the same way RFC 5321 uses "delivery system" or "final delivery system" 🎯. This is the SMTP server that controls the format of the local parts of addresses and is allowed to look inside them and interpret them 🔍. It receives messages from the network for delivery to mailboxes or for other local processing, including any forwarding or aliasing that changes envelope addresses, rather than relaying 📬. From the network's point of view, any local delivery stuff, like saving to a message store, handing off to specific delivery programs or agents, and mechanisms for retrieving messages, all sits "behind" the final delivery MTA, so none of it is part of the SMTP transport or delivery process 🫠.

4.2. Address Character Sets 🔤

In this document, an address is "all-ASCII" (or just an "ASCII address") if every character in it is in the ASCII character repertoire [ASCII] 🇺🇸. An address is "non-ASCII" (or an "i18n-address") if ANY character isn't in that repertoire 🌏. These addresses MAY be restricted in other ways, but those restrictions don't matter for this definition 🙅‍♀️. The term "all-ASCII" also gets applied to other protocol elements when the distinction matters, with "non-ASCII" or "internationalized" as the opposite 🔄.

The umbrella term for the email address internationalization specified by this document and its companion documents is "SMTPUTF8" 🦋. So an address this spec permits is called a "SMTPUTF8 (compliant) address" 💅.

Please note that, based on these definitions, the set of all "all-ASCII" addresses and the set of all "non-ASCII" addresses are mutually exclusive 🙅‍♀️. The set of all addresses permitted when SMTPUTF8 appears is the union of those two sets 💖.

4.3. User Types 👯‍♀️

An "ASCII user" (i) exclusively uses email addresses that contain only ASCII characters, AND (ii) cannot generate recipient addresses that contain non-ASCII characters 🙃.

An "internationalized email user" has one or more non-ASCII email addresses, OR is able to generate recipient addresses with non-ASCII characters 🌍. Such a user may have ASCII addresses too. If the user has more than one email account and corresponding address, or more than one alias for the same address, they have some method to choose which address to use on outgoing email 🎛️. Note that under this definition, you can't tell from an ASCII address whether its owner is an internationalized email user 🕵️‍♀️. (A non-ASCII address implies a belief that the owner of that address is an internationalized email user 💭.) There's no such thing as an "internationalized email user message"; the term only applies to users and their agents and capabilities 🙅‍♀️. Also, using non-ASCII, and so presumably internationalized, message content is an integral part of the MIME specifications [RFC2045] and does NOT require these extensions (though it's compatible with them) 🎀.

4.4. Messages 💌

A "message" is sent from one user (the sender), using a particular email address, to one or more other recipient email addresses (often just called "users" or "recipient users") 💫.

4.5. Mailing Lists 📯

A "mailing list" is a mechanism where a message can be distributed to multiple recipients by sending it to ONE recipient address 📨. An agent at that single address (typically not a human being 🤖) then redistributes the message to the target recipients. This agent sets the envelope return address of the redistributed message to a different address from that of the original single recipient message 🔁. Using a different envelope return address (reverse-path) makes errors, and other automatically generated messages, go to an error-handling address 🚨.

Special provisions for managing mailing lists that might contain non-ASCII addresses are discussed in a document specific to that topic [RFC5983] and its expected successor [RFC5983bis-MailingList] 📖.

4.6. Conventional Message and Internationalized Message 🧾

o A conventional message is one that does not use any extension defined in the SMTP extension document [RFC6531] or in the UTF8header document [RFC6532] in this set of specifications, and is strictly conformant to RFC 5322 [RFC5322].
o An internationalized message is a message utilizing one or more of the extensions defined in this set of specifications, so that it is no longer conformant to the traditional specification of an email message or its transport.

4.7. Undeliverable Messages, Notification, and Delivery Receipts 📭

As RFC 5321 specifies, a message that can't be delivered for some reason is expected to result in notification to the sender 📢. This can happen in one of two ways. The first, usually called "Rejection", happens when an SMTP server returns a reply code indicating a fatal error (a "5yz" code) or keeps returning a temporary failure error (a "4yz" code) 🚫. The other involves accepting the message during SMTP processing and then generating a message to the sender, usually known as a "Non-delivery Notification" or "NDN" 📩. Current practice often favors rejection over NDNs, because generating NDNs is less likely to get used as a spamming technique 🛡️. The latter, NDN, case is unavoidable if an intermediate MTA accepts a message that the next-hop server then rejects 😬.

A sender MAY also explicitly request message receipts [RFC3461] 🧾, which raise the same issues for these internationalization extensions as NDNs do 🔥.

5. Overview of the Approach and Document Plan 🗺️

This set of specifications changes both SMTP and the character encoding of email message headers so non-ASCII characters can be represented directly 🌸. Each important piece of the work is described in its own document 📚. The document set, whose members are described below, also contains Informational documents whose purpose is to provide implementation suggestions and guidance for the protocols 💡.

Besides this document, the following documents make up this specification and give advice and context for it 🫶.

  • SMTP extension. The SMTP extension document [RFC6531] provides an SMTP extension (as provided for in RFC 5321) for internationalized addresses 💌.
  • Email message headers in UTF-8. The email message header document [RFC6532] essentially updates RFC 5322 so some information in email message headers can be written directly with Unicode characters encoded in UTF-8, when the SMTP extension above is used ✨. This document, possibly with one or more supplemental ones, will also need to address the interactions with MIME, including relationships between SMTPUTF8 and internal MIME headers and content types 🎀.
  • Extensions to delivery status and notification handling to adapt to internationalized addresses [RFC6533] 🔔.
  • Forthcoming documents will specify extensions to the IMAP protocol [RFC3501] to support internationalized message headers [RFC5738bis-IMAP], parallel extensions to the POP protocol [RFC5721] [RFC5721bis-POP3], and some common properties of the two [POPIMAP-downgrade] 🧚.

6. Review of Experimental Results ✨

okay so the big difference between this new set of protocols and the experimental set before them (the one with [RFC5335] [RFC5336] [RFC5337] [RFC5504] [RFC5721] [RFC5738] [RFC5825]) is the in-transit downgrading. The older group had a mechanism for it (explained in detail in RFC 5504) that essentially required every non-ASCII address to come with an all-ASCII equivalent 🙈. That raised security concerns, because pairing addresses that couldn't be authenticated is sus. It was also the first incompatible change to Internet mail addressing in many years, so people worried the new address forms might "leak" into legacy email software and cause chaos 💀. After working with their experimental predecessors, the working group decided the perks of in-transit downgrading would be big enough to outweigh those worries, as long as it was doable in practice.

Plot twist: it wasn't. The initial implementations ran into interoperability problems 😭. Before starting the work that led to this set, the WG concluded that the combined requirements and long-term implications of that earlier model were too complicated to be satisfying, so they decided to move forward without it 💅.

The other big change to the protocols themselves is that the SMTPUTF8 keyword is now required as an SMTP client announcement when the extension is needed. In the experimental version, the server announcing that an extended envelope and/or content was permitted was all that was necessary 🌸.

7. Overview of Protocol Extensions and Changes ✨

7.1. SMTP Extension for Internationalized Email Address

Here's the new SMTP extension, SMTPUTF8, in a nutshell 🦋:

  • It permits the use of UTF-8 strings in email addresses, in both local parts and domain names 🌈.
  • It permits the selective use of UTF-8 strings in email message headers (see Section 7.2) ✨.
  • It requires that the server advertise the 8BITMIME extension [RFC6152] and that the client support 8-bit transmission, so header information can be sent without using a special content-transfer-encoding 🎀.

Some general principles shaped the decisions behind all of this 🧚:

  1. Email addresses pass through subsystems (like a user interface) that may do charset conversions or other encoding changes. When the local part of the address includes characters outside the ASCII character repertoire, using ASCII-compatible encoding (ACE) [RFC3492] [RFC5890] in the domain part is discouraged, so characters stay consistent all the way through the address 💖.
  2. An SMTP relay MUST:
    • Either recognize the format explicitly, agreeing to do so via an ESMTP option, or
    • Reject the message or, if necessary, return a non-delivery notification message, so the sender can make another plan 🫶.
  3. If the message can't be forwarded because the next-hop system can't accept the extension, it MUST be rejected or a non-delivery message MUST be generated and sent 💀.
  4. For the sake of interoperability, charsets other than UTF-8 are prohibited in mail addresses and message headers sent over the Internet. There's no practical way to properly identify multiple charsets with an extension like this without adding a ton of complexity 🙅‍♀️.

Conformance to this group of standards for email transport and delivery requires implementing the SMTP extension specification and the UTF-8 header specification ✨. If the system implements IMAP or POP, it MUST conform to the internationalized IMAP [RFC5738bis-IMAP] or POP [RFC5721bis-POP3] specifications, respectively 💜.

7.2. Transmission of Email Header Fields in UTF-8 Encoding

Email addresses and domain names show up all over the place in MUAs and in what users see on screen 👀. Think of the conventional From:, To:, and Cc: header fields, plus Message-ID: and In-Reply-To:, which normally contain domain names (though they might be a special case), and then the message bodies too. Each of these needs an internationalization check 🌸. Users expect to see mailbox and domain names in their own characters, and they expect to see them consistently. If non-obvious encodings (like protocol-specific ACE variants) get used, people will inevitably run into them every now and then, see them instead of "native" characters, and find that super discomfiting or astonishing 😭. If different encodings are used for transport and for message bodies, users are especially likely to be surprised, which is the long-standing "things leak" principle at work 🙈. The only practical way to avoid these sources of discomfort, now and later, is to make the transport encodings as similar as possible to the encodings used in message headers and message bodies ✨.

When email local parts are internationalized, they SHOULD come with arrangements for message headers to be in the fully internationalized form 🦋. That form SHOULD use UTF-8, not ASCII, as the base character set for the contents of header fields (protocol elements like the header field names themselves don't change and stay entirely in ASCII). For transition purposes and compatibility with legacy systems, this can be done by extending the traditional MIME encoding models for non-ASCII characters in headers [RFC2045] [RFC2231]. Even so, these should be based on UTF-8 rather than other encodings, whenever at all possible [RFC6055] 💅. Still, the goal is fully internationalized message headers, as discussed in [RFC6532], not an extended and painful transition 🎀.

7.3. SMTP Service Extension for DSNs

The existing Delivery Status Notifications (DSNs) specification [RFC3461] is a Draft Standard, and it's limited to ASCII text in the machine-readable parts of the protocol 🤓. "International Delivery and Disposition Notifications" [RFC6533] adds a new address type for international email addresses, so an original recipient address with non-ASCII characters can be preserved correctly even after downgrading 🌈. If an SMTP server advertises both the SMTPUTF8 and the DSN extension, that server MUST implement internationalized DSNs, including support for the ORCPT parameter specified in RFC 3461 [RFC3461] 👑.

8. Downgrading before and after SMTP Transactions ✨

This is a big one, bestie 😭. The key question is how to handle interactions between systems that support non-ASCII addresses and legacy systems that expect ASCII. Sending from ASCII-only systems to systems that can handle internationalized forms is totally fine, because the ASCII forms are just a proper subset 🫶. But when systems that support these extensions send mail, they MAY include non-ASCII addresses for senders, receivers, or both, and they might also include non-ASCII header information other than addresses 🦋. If the first-hop system (that is, the SMTP server that the submission server reaches as an SMTP client) doesn't support the extension, message-originating systems SHOULD be ready either to send conventional envelopes and message headers, or to return the message to the originating user so it can be manually downgraded to the traditional form, possibly using encoded words [RFC2047] in the message headers 💅. Of course, these transformations mean the originating user or system needs ASCII-only addresses available for all senders and recipients. How those addresses are found or identified is outside the scope of these specifications, and so are decisions about originating-system design, like whether the user, the originating MUA, or the submission server makes any required transformations 🧚.

A somewhat more complicated situation comes up when the first-hop system supports these extensions but some later server in the SMTP transmission chain doesn't 🙃. It's important to know that most cases like this, with forward-pointing addresses, are the result of configuration errors. Especially if it hosts non-ASCII addresses, a final delivery MTA that accepts these extensions SHOULD NOT be configured with lower-preference MX hosts that don't 😭. When the only non-ASCII address being transmitted is backward-pointing (for example, in an SMTP MAIL command), recipient configuration can't help in general 💀. On the other hand, alternate, all-ASCII addresses for senders are the ones most likely to be authoritatively known by the submission environment or by the sender herself 🌸. So if an intermediate SMTP relay that requires these extensions discovers that the next system in the chain doesn't support them, it won't have much choice but to reject or return the message 🙈.

As discussed above, downgrading to an ASCII-only form may happen before or during initial message submission ✨. It might also happen after delivery to the final delivery MTA, to accommodate message stores, IMAP or POP servers, or clients that have different capabilities than the delivery MTA 💖. These cases are discussed in the subsections below 🎀.

8.1. Downgrading before or during Message Submission 🎀

okay so the IETF has traditionally avoided spelling out exactly how MUAs behave, because they want maximum flexibility for user interfaces 💅 The SMTP standard [RFC5321], Section 6.4, gives MUAs and submission servers wide latitude on what a user can hand over, as long as the result conforms with "on the wire" standards once it's injected into the public Internet. In that spirit, the rest of Section 8 is general guidance, not normative requirements 🦋

messages that need these extensions will sometimes get transferred to a system that doesn't support them 😭 the most common cases will probably combine ASCII-only forward-pointing addresses with a non-ASCII backward-pointing one. until these extensions are universally implemented across the Internet email environment, senders who prefer non-ASCII addresses (or raw UTF-8 characters in header fields), even when their recipients use and expect all-ASCII ones, will need to be extra careful about the error conditions that can pop up 🙈 the risks are especially big in environments where non-delivery messages (or other indications from submission servers) get routinely dropped or ignored 💀

perhaps obviously, the most convenient time to find an ASCII address that matches an internationalized address is at the originating MUA or closely associated systems ✨ that can happen either before the message is sent or after the internationalized form gets rejected. it's also the most convenient time to convert a message from internationalized form into conventional ASCII form, or to generate a non-delivery message to the sender if either is needed. at that point the user has a full range of choices, including changing backward-pointing addresses, contacting the intended recipient out of band for an alternate address, consulting appropriate directories, arranging translation of both addresses and message content into a different language, and so on 👑 it's natural to think of downgrading as something that should ideally be fully automated, but we should not underestimate what a user of at least moderate intelligence can do when she wants to talk to another such user 🫶

in this context, you can easily imagine modifications to message submission servers (as described in RFC 6409 [RFC6409]) so that they perform downgrading operations, or maybe even upgrading ones 🧚 those operations would let a server receive messages with one or more of the internationalization extensions discussed here, then adapt the outgoing message as needed to respond to the delivery or next-hop environment it runs into. main character energy 🌈

8.2. Downgrading or Other Processing after Final SMTP Delivery

when an email message is received by a final delivery MTA, it's usually stored in some form 📦 then it gets retrieved either by software that reads the stored form directly, or by client software through email retrieval mechanisms such as POP or IMAP.

the SMTP extension described in Section 7.1 only protects things in transport 🦋 it doesn't stop MUAs and email retrieval mechanisms that haven't been upgraded to understand internationalized addresses and UTF-8 message headers from accessing stored internationalized emails. not a protection for storage, bestie 🙈

since the final delivery MTA (or, more precisely, its corresponding mail storage agent) can't safely assume that every agent accessing email storage will be able to handle these extensions, it MAY downgrade internationalized emails, specially identify messages that use these extensions, or both 💅 if either or both of those actions are taken, the final delivery MTA SHOULD include a mechanism to preserve or recover the original internationalized forms without information loss. preserving that info is necessary to support access by SMTPUTF8-aware agents ✨

9. Downgrading in Transit

the base SMTP spec (Section 2.3.11 of RFC 5321 [RFC5321]) says that "due to a long history of problems when intermediate hosts have attempted to optimize transport by modifying them, the local-part MUST be interpreted and assigned semantics only by the host specified in the domain part of the address". this is not a new rule 🙄 equivalent statements showed up in specifications back in 2001 [RFC2821] and even in 1989 [RFC1123]. it's been around forever, iykyk 🍓

so, a downgrade mechanism that transforms the local part of an email address can't be used in transit 💀 it can only be applied at the endpoints, specifically by the MUA or submission server, or by the final delivery MTA 👑

one reason for this rule is legacy email systems that embed mail routing info in the local part of the address. transforming the address destroys that routing info, and no server other than the final delivery server can know whether the local part of user%foo@example.com is a route ("user" is reached via "foo") or just a plain local address 🧩 wild, right? 😭

10. User Interface and Configuration Issues

internationalizing addresses and message headers, especially when you mix in the variations on character coding that come with Unicode, can make careful choices of addresses and careful configuration of servers and DNS records even more important than they already are for traditional Internet email 💖 it's likely that, as experience with these protocols grows, it'll be desirable to produce one or more additional documents with guidance for configuration and interfaces. a document discussing issues with MUAs, especially around downgrading, is expected to be developed 🌸 the subsections below cover some other issues.

10.1. Choices of Mailbox Names and Unicode Normalization

it's been a long-running thing that the email syntax permits choices about mailbox names that are unwise in practice, if you actually want the mailboxes to be reachable by a broad range of senders 🙈 the most cited examples involve case-sensitivity and tricky quoting of embedded characters in mailbox local parts. these deliberately unusual constructions are permitted by the protocols, and servers are expected to support them. they can be valuable in special cases, but using them is almost always bad practice unless the goal is some kind of security by obscurity 🕵️‍♀️

without these extensions, SMTP clients and servers are limited to the addresses permitted by RFC 5321 🎀 the local parts of those addresses MAY be made up of any ASCII characters except the control characters that RFC 5321 prohibits, although some of them MUST be quoted as specified there. it's notable for internationalization that some systems have a long history of using overstruck ASCII (a character, a backspace, and another character) inside a quoted string to approximate non-ASCII characters 🧐 this was permitted by RFC 821 [RFC0821] but is prohibited by RFC 5321, because it requires a backspace (a prohibited C0 control). since RFC 5321 (and its predecessor, RFC 2821) prohibit that character in ASCII mailbox names, and it's even more problematic for canonicalization and normalization in non-ASCII strings, backspace MUST NOT appear in SMTPUTF8 mailbox names 🚫

for mailbox names that contain non-ASCII characters in the local part, the domain part, or both, special attention MUST be paid to Unicode normalization [Unicode-UAX15] ✨ part of the reason is that Unicode strings may get normalized by other processes, completely independent of what a mail protocol says (this is exactly analogous to what can happen with quoting and dequoting in traditional addresses). so here are some principles, offered as advice to anyone picking names for mailboxes 💅

  • in general, it's wise to support addresses in Normalized form, using at least Normalization Form NFC. unless NFKC would map characters together that the folks responsible for the destination mail server would prefer to keep distinguishable, supporting the NFKC-conformant form would give even more predictable behavior for the typical user 🦋
  • it'll usually be wise to support other forms of the same local-part string, either as aliases or by normalizing strings that reach the delivery server. the sender should not be depended upon to send the strings in normalized form 🫶
  • put more specifically, the protocol's rules for local-part strings basically say:
    • unnormalized strings are valid, but they're sufficiently bad practice that they may not work reliably on a global basis. servers should not depend on clients to send normalized forms, but should be aware that procedures on client machines outside the control of the MUA may cause normalized strings to be sent regardless of user intent 😭
    • C0 (and presumably C1) controls (see The Unicode Standard [Unicode]) are prohibited: the first in RFC 5321, and the second by an obvious extension from it [RFC5198] 🚫
    • other kinds of punctuation, spaces, etc. are risky practice. maybe they'll work, and SMTP receiver code is required to handle them without severe errors (even if those strings aren't accepted in addresses to be delivered on that server). but creating dependencies on them in the mailbox names you choose is usually bad practice and may lead to interoperability problems 💀

11. Additional Issues

this section covers issues that aren't covered, or aren't covered comprehensively, by this set of specifications. they'll need ongoing review as part of deploying email address and header internationalization 🔥

11.1. Impact on URIs and IRIs

the mailto: schema [RFC6068], and the discussion of it in the Internationalized Resource Identifier (IRI) specification [RFC3987], may need to be modified once this work is completed and standardized 🌸

11.2. Use of Email Addresses as Identifiers

there are a bunch of places in current Internet usage where email addresses act as identifiers for individuals 👀 that includes identifiers for Web servers supporting some electronic commerce sites, and some X.509 certificates [RFC5280]. these documents don't address those uses, but it's reasonable to expect some difficulties when internationalized addresses are first used in those contexts. many of those contexts can't even handle the full range of addresses permitted today, so buckle up 🎢💜

11.3. Encoded Words, Signed Messages, and Downgrading

Okay bestie, email is seriously the most persistent thing ever 💌 MUAs are expected to handle messages that were sent decades ago, not just ones that landed seconds ago 🕰️ So MUAs and mail filtering software, like the one specified in Sieve [RFC5228], will need to keep accepting and decoding header fields that use the "encoded word" mechanism [RFC2047] to handle non-ASCII characters in some header fields 🎀. Extensions to both POP3 [RFC1939] and IMAP [RFC3501] have been defined that automatically upgrade messages carrying non-ASCII information in encoded form, including RFC 2047 decoding, by the POP3 [RFC5721bis-POP3] or IMAP [RFC5738bis-IMAP] server 🙌. BUT there are message structures and MIME content-types where that just can't happen, or where the change would have unacceptable side effects 🙅‍♀️

For example, message parts that are cryptographically signed, using e.g., S/MIME [RFC5751] or Pretty Good Privacy (PGP) [RFC3156], cannot be upgraded from the RFC 2047 form to normal UTF-8 characters without breaking the signature 💔. Similarly, message parts that are encrypted may contain header fields that use the RFC 2047 encoding once they're decrypted, and those messages can't be "fully" upgraded without access to the cryptographic keys 🔐.

Similar issues may come up if messages get signed and then downgraded, e.g., as discussed in Section 8.1, and then someone tries to upgrade them back to the original form and verify the signatures 😵‍💫. Even the super subtle changes that come from downgrading and then upgrading again can be enough to invalidate the signatures if they affect either the primary or MIME body part headers. When signatures are present, downgrading must be performed with extreme care if at all 🫣

11.4. Other Uses of Local Parts

Local parts are sometimes used to build domain labels 🏗️ For example, the local part "user" in the address user@domain.example could be turned into a host name user.domain.example, with its Web space at <http://user.domain.example> and the catch-all addresses any.thing.goes@user.domain.example 🌐

Schemes like this are obviously limited by, among other things, the SMTP rules for domain names, and they won't work for other local parts without further restrictions 🚧. Whether those limitations matter for these specifications is an open question 🤔 It might just be another case of the really wide flexibility given to delivery MTAs when it comes to deciding which mailbox names they accept and how they interpret them ✨

11.5. Non-Standard Encapsulation Formats

Some applications use formats that look like the application/mbox format [RFC4155], instead of the message/digest form defined in RFC 2046, Section 5.1.5 [RFC2046], to transfer multiple messages as single units 📦. As long as those applications assume that every stored message uses the message/rfc822 format described in RFC 2046, Section 5.2.1 [RFC2046] with ASCII message headers, they aren't ready for the extensions in this series of documents 😬. Special measures may be needed to detect and process them properly 🛠️

12. Key Changes from the Experimental Protocols and Framework

Okay, history time 📜 The original framework for internationalized email addresses and headers was described in RFC 4952 and a follow-up set of experimental protocol documents 🧪. Those relationships are explained in Section 3 🔗. The big architectural difference between the experimental specs and this newer set is that the earlier ones supported in-transit downgrading 🚚. Those mechanisms included syntax and functions for passing alternate, all-ASCII addresses alongside the non-ASCII ones, plus special headers to indicate the downgraded status of messages 🏷️. All of that got cut after experiments showed it was more complex and less necessary than people had assumed 💅. More detail is in Sections 6 and 9 📖

13. Security Considerations

Okay, this part is serious, so let's get into it 🚨 Any expansion of permitted characters and encoding forms in email addresses raises some risks 😰 There have been discussions about so called "IDN-spoofing" or "IDN homograph attacks" 🎭. These attacks let an attacker (or "phisher") spoof the domain or URLs of businesses or other entities 🪤. The same kind of attack is also possible on the local part of internationalized email addresses 💀. It should be noted that the proposed fix of forcing all displayed elements into normalized lowercase works for domain names in URLs, but not for email local parts, since those are case sensitive 🙈

Email addresses are often copied off business cards and paper notes, so they're subject to problems from confusable characters (see [RFC4690]) 📇. These problems are somewhat reduced if the domain associated with the mailbox is unambiguous and supports a relatively small number of mailboxes whose names follow local system conventions 🧐. They increase with very large mail systems where users can freely pick their own addresses 📈

The internationalization of email addresses and message headers must not leave the Internet less secure than it is without the required extensions 🔒. The requirements and mechanisms in this set of specifications do not, in general, raise any new security issues ✅

They do require a review of issues associated with confusable characters, which is being explored thoroughly elsewhere (see, e.g., RFC 4690 [RFC4690]) 👀. They may also raise some issues with UTF-8 normalization, discussed in RFC 3629 [RFC3629], and other transformations 🔄. Normalization and other issues tied to transformations and standard forms are also part of work described elsewhere [RFC5198] [RFC5893] [RFC6055] 📚

Some issues specifically related to internationalized addresses and message headers are covered in more detail in the other documents in this set 📑. But, in particular, caution should be taken that any "downgrading" mechanism, or use of downgraded addresses, does not inappropriately assume authenticated bindings between the internationalized and ASCII addresses 🚩. This potential problem can be eased somewhat by enforcing the expectation that most or all such transformations will be performed before final delivery, by systems presumed to be under the administrative control of the sending user (as opposed to being performed in transit by entities outside that sending user's administrative control) 🏠

The new UTF-8 header and message formats might also raise, or aggravate, another known issue 😤. If the model creates new forms of "invalid" or "malformed" messages, then a new email attack shows up: in an effort to be robust, some or most agents will accept such messages and interpret them as if they were well-formed 🤯. If a filter interprets such a message differently than the MUA used by the recipient, it may be possible to create a message that looks acceptable under the filter's interpretation but should be rejected under the interpretation given by that MUA 🥷. Attacks like this have already happened with existing messages and encoding layers, e.g., invalid MIME syntax, invalid HTML markup, and invalid coding of particular image types 🫠

Email addresses are also used in lots of contexts besides sending mail, such as identifiers in various circumstances (see Section 11.2) 🪪. Each of those contexts will need to be evaluated, one by one, to figure out whether non-ASCII forms are appropriate there and what particular issues they raise 🔍

This work will clearly affect any systems or mechanisms that depend on digital signatures or similar integrity protection for email message headers (see also the discussion in Section 11.3) ✍️. Lots of conventional uses of PGP and S/MIME aren't affected, since they sign body parts but not message headers 🙌. On the other hand, the developing work on DomainKeys Identified Mail (DKIM) [RFC5863] will eventually need to consider this work, and vice versa 🔁. This specification doesn't address or solve the issues raised by DKIM and other signed header mechanisms, but those issues will have to be coordinated and resolved eventually if the two sets of protocols are going to coexist 🤝. Also, to the degree that email addresses appear in PKI (Public Key Infrastructure) certificates [RFC5280], standards about those certificates will need upgrading to handle these internationalized addresses 📜. Those upgrades will need to address questions of spoofing by look-alikes of the addresses themselves 👯

14. Acknowledgments

This document is an update to, and derived from, RFC 4952 💖. It would have been impossible without the work and contributions acknowledged in it 🫶. This one benefited hugely from discussions in the IETF EAI working group and elsewhere after RFC 4952 was published, especially talks about the experimental versions of other documents in the internationalized email collection, plus RFC errata on RFC 4952 itself 🌸

Special thanks go to Ernie Dainow for careful reviews and suggested text in this version, and to several IESG members for a careful review and specific suggestions 🙇‍♀️✨

15. References

15.1. Normative References

  • [ASCII] American National Standards Institute (formerly United States of America Standards Institute), "USA Code for Information Interchange", ANSI X3.4-1968, 1968. 📟

ANSI X3.4-1968 has been replaced by newer versions with slight modifications, but the 1968 version remains definitive for the Internet.

  • [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. 🔑
  • [RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO 10646", STD 63, RFC 3629, November 2003. 🔤
  • [RFC5321] Klensin, J., "Simple Mail Transfer Protocol", RFC 5321, October 2008. 📮
  • [RFC5322] Resnick, P., Ed., "Internet Message Format", RFC 5322, October 2008. ✉️
  • [RFC5890] Klensin, J., "Internationalized Domain Names for Applications (IDNA): Definitions and Document Framework", RFC 5890, August 2010. 🌍
  • [RFC6152] Klensin, J., Freed, N., Rose, M., and D. Crocker, "SMTP Service Extension for 8-bit MIME Transport", STD 71, RFC 6152, March 2011. 💌
  • [RFC6531] Yao, J. and W. Mao, "SMTP Extension for Internationalized Email Address", RFC 6531, February 2012. 🎀
  • [RFC6532] Yang, A., Steele, S., and N. Freed, "Internationalized Email Headers", RFC 6532, February 2012. 🦋
  • [RFC6533] Hansen, T., Newman, C., and A. Melnikov, "Internationalized Delivery Status and Disposition Notifications", RFC 6533, February 2012. 📬

15.2. Informative References ✨

okay bestie, this is the reading list 📚💅 these are the "good to know" docs. the spec leans on them for background but doesn't make you read them to implement it, iykyk 🫶

  • [POPIMAP-downgrade] Fujiwara, K., "Post-delivery Message Downgrading for Internationalized Email Messages", Work in Progress, October 2011.
  • [RFC0821] Postel, J., "Simple Mail Transfer Protocol", STD 10, RFC 821, August 1982.
  • [RFC1123] Braden, R., "Requirements for Internet Hosts - Application and Support", STD 3, RFC 1123, October 1989.
  • [RFC1939] Myers, J. and M. Rose, "Post Office Protocol - Version 3", STD 53, RFC 1939, May 1996.
  • [RFC2045] Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies", RFC 2045, November 1996.
  • [RFC2046] Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types", RFC 2046, November 1996.
  • [RFC2047] Moore, K., "MIME (Multipurpose Internet Mail Extensions) Part Three: Message Header Extensions for Non-ASCII Text", RFC 2047, November 1996.
  • [RFC2231] Freed, N. and K. Moore, "MIME Parameter Value and Encoded Word Extensions: Character Sets, Languages, and Continuations", RFC 2231, November 1997.
  • [RFC2821] Klensin, J., "Simple Mail Transfer Protocol", RFC 2821, April 2001.
  • [RFC3156] Elkins, M., Del Torto, D., Levien, R., and T. Roessler, "MIME Security with OpenPGP", RFC 3156, August 2001.
  • [RFC3461] Moore, K., "Simple Mail Transfer Protocol (SMTP) Service Extension for Delivery Status Notifications (DSNs)", RFC 3461, January 2003.
  • [RFC3464] Moore, K. and G. Vaudreuil, "An Extensible Message Format for Delivery Status Notifications", RFC 3464, January 2003.
  • [RFC3492] Costello, A., "Punycode: A Bootstring encoding of Unicode for Internationalized Domain Names in Applications (IDNA)", RFC 3492, March 2003.
  • [RFC3501] Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1", RFC 3501, March 2003.
  • [RFC3987] Duerst, M. and M. Suignard, "Internationalized Resource Identifiers (IRIs)", RFC 3987, January 2005.
  • [RFC4155] Hall, E., "The application/mbox Media Type", RFC 4155, September 2005.
  • [RFC4690] Klensin, J., Faltstrom, P., Karp, C., and IAB, "Review and Recommendations for Internationalized Domain Names (IDNs)", RFC 4690, September 2006.
  • [RFC4952] Klensin, J. and Y. Ko, "Overview and Framework for Internationalized Email", RFC 4952, July 2007.
  • [RFC5198] Klensin, J. and M. Padlipsky, "Unicode Format for Network Interchange", RFC 5198, March 2008.
  • [RFC5228] Guenther, P. and T. Showalter, "Sieve: An Email Filtering Language", RFC 5228, January 2008.
  • [RFC5280] Cooper, D., Santesson, S., Farrell, S., Boeyen, S., Housley, R., and W. Polk, "Internet X.509 Public Key Infrastructure Certificate and Certificate Revocation List (CRL) Profile", RFC 5280, May 2008.
  • [RFC5335] Yang, A., "Internationalized Email Headers", RFC 5335, September 2008.
  • [RFC5336] Yao, J. and W. Mao, "SMTP Extension for Internationalized Email Addresses", RFC 5336, September 2008.
  • [RFC5337] Newman, C. and A. Melnikov, "Internationalized Delivery Status and Disposition Notifications", RFC 5337, September 2008.
  • [RFC5504] Fujiwara, K. and Y. Yoneya, "Downgrading Mechanism for Email Address Internationalization", RFC 5504, March 2009.
  • [RFC5721] Gellens, R. and C. Newman, "POP3 Support for UTF-8", RFC 5721, February 2010.
  • [RFC5721bis-POP3] Gellens, R., Newman, C., Yao, J., and K. Fujiwara, "POP3 Support for UTF-8", Work in Progress, November 2011.
  • [RFC5738] Resnick, P. and C. Newman, "IMAP Support for UTF-8", RFC 5738, March 2010.
  • [RFC5738bis-IMAP] Resnick, P., Ed., Newman, C., Ed., and S. Shen, Ed., "IMAP Support for UTF-8", Work in Progress, December 2011.
  • [RFC5751] Ramsdell, B. and S. Turner, "Secure/Multipurpose Internet Mail Extensions (S/MIME) Version 3.2 Message Specification", RFC 5751, January 2010.
  • [RFC5825] Fujiwara, K. and B. Leiba, "Displaying Downgraded Messages for Email Address Internationalization", RFC 5825, April 2010.
  • [RFC5863] Hansen, T., Siegel, E., Hallam-Baker, P., and D. Crocker, "DomainKeys Identified Mail (DKIM) Development, Deployment, and Operations", RFC 5863, May 2010.
  • [RFC5891] Klensin, J., "Internationalized Domain Names in Applications (IDNA): Protocol", RFC 5891, August 2010.
  • [RFC5892] Faltstrom, P., "The Unicode Code Points and Internationalized Domain Names for Applications (IDNA)", RFC 5892, August 2010.
  • [RFC5893] Alvestrand, H. and C. Karp, "Right-to-Left Scripts for Internationalized Domain Names for Applications (IDNA)", RFC 5893, August 2010.
  • [RFC5894] Klensin, J., "Internationalized Domain Names for Applications (IDNA): Background, Explanation, and Rationale", RFC 5894, August 2010.
  • [RFC5983] Gellens, R., "Mailing Lists and Internationalized Email Addresses", RFC 5983, October 2010.
  • [RFC5983bis-MailingList] Levine, J. and R. Gellens, "Mailing Lists and UTF-8 Addresses", Work in Progress, December 2011.
  • [RFC6055] Thaler, D., Klensin, J., and S. Cheshire, "IAB Thoughts on Encodings for Internationalized Domain Names", RFC 6055, February 2011.
  • [RFC6068] Duerst, M., Masinter, L., and J. Zawinski, "The 'mailto' URI Scheme", RFC 6068, October 2010.
  • [RFC6409] Gellens, R. and J. Klensin, "Message Submission for Mail", STD 72, RFC 6409, November 2011.
  • [Unicode] The Unicode Consortium. The Unicode Standard, Version 6.0.0, defined by:, "The Unicode Standard, Version 6.0.0", (Mountain View, CA: The Unicode Consortium, 2011. ISBN 978-1-936213-01-6)., <http://www.unicode.org/versions/Unicode6.0.0/>.
  • [Unicode-UAX15] The Unicode Consortium, "Unicode Standard Annex #15: Unicode Normalization Forms", September 2010, <http://www.unicode.org/reports/tr15/>.

Authors' Addresses 🎀

meet the authors, the humans who wrote this whole thing 🌸🧚

John C KLENSIN
1770 Massachusetts Ave, #322
Cambridge, MA  02140
USA

Phone: +1 617 491 5735
EMail: john-ietf@jck.com


YangWoo KO
112-202 Malgeunachim APT. Nae-dong
Seo-gu, Daejeon  302-981
Republic of Korea

EMail: yangwooko@gmail.com
📦 where this page comes from