RFC 6532

Internationalized Email Headers

📅 published February 2012, from ietf

Internationalized Email Headers 💌

okay bestie, this is RFC 6532, Internationalized Email Headers 🎀 It came out in February 2012 and was written by A. Yang (TWNIC), S. Steele (Microsoft), and N. Freed (Oracle). It's a Standards Track document, so it's a real Internet standard, not just vibes 💅 It obsoletes RFC 5335 and updates RFC 2045. Its ISSN is 2070-1721 ✨

Abstract 🦋

Internet mail started out limited to 7-bit ASCII 🙈 MIME later allowed 8-bit character sets in body parts and added encoded-words, so other character sets could show up in some header field values. But full internationalization needs more. We want Unicode, including characters outside ASCII, in mail addresses and in header fields like "From:", "To:", and "Subject:", without making everyone deal with messy encoded-word constructs 😭 This document adds that to the Internet Message Format and to MIME, so Unicode can go in mail addresses and most header field content 🌈

This specification updates Section 6.4 of RFC 2045 to get rid of the rule that stops non-identity content-transfer-encodings on subtypes of "message/" 🔥

Status of This Memo 📜

This is an Internet Standards Track document 👑

This document is a product of the Internet Engineering Task Force (IETF) 💜 It represents the consensus of the IETF community. It got public review and was approved for publication by the Internet Engineering Steering Group (IESG). If you want more on Internet Standards, check Section 2 of RFC 5741 🧚

Info about the current status of this document, any errata, and how to give feedback is at http://www.rfc-editor.org/info/rfc6532 🍓

Copyright (c) 2012 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as they stand on the date this document was published. Please read these carefully, because they explain your rights and restrictions for this document. Code Components pulled out of this document must include Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come without warranty, as described in the Simplified BSD License ⚠️

Table of Contents 🗂️

1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
2.  Terminology Used in This Specification . . . . . . . . . . . .  3
3.  Changes to Message Header Fields . . . . . . . . . . . . . . .  4
  3.1.  UTF-8 Syntax and Normalization . . . . . . . . . . . . . .  4
  3.2.  Syntax Extensions to RFC 5322  . . . . . . . . . . . . . .  5
  3.3.  Use of 8-bit UTF-8 in Message-IDs  . . . . . . . . . . . .  5
  3.4.  Effects on Line Length Limits  . . . . . . . . . . . . . .  5
  3.5.  Changes to MIME Message Type Encoding Restrictions . . . .  6
  3.6.  Use of MIME Encoded-Words  . . . . . . . . . . . . . . . .  6
  3.7.  The message/global Media Type  . . . . . . . . . . . . . .  7
4.  Security Considerations  . . . . . . . . . . . . . . . . . . .  8
5.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . .  9
6.  Acknowledgements . . . . . . . . . . . . . . . . . . . . . . .  9
7.  References . . . . . . . . . . . . . . . . . . . . . . . . . . 10
  7.1.  Normative References . . . . . . . . . . . . . . . . . . . 10
  7.2.  Informative References . . . . . . . . . . . . . . . . . . 10

1. Introduction ✨

Internet mail splits a message from its transport, and it also splits a message into a header and a body [RFC5322] 💌 Header field values hold lots of strings meant to be seen by users. Those strings were originally limited to [ASCII] in 7-bit form 🙈 MIME [RFC2045] [RFC2046] [RFC2047] lets you use more character sets, but only for body part data and for special encoded-word constructs, and those were only allowed in a few spots in header field values 😭

Globalization of the Internet needs the much bigger set of characters that Unicode [RFC5198] offers, in both mail addresses and most header field values 🌍 Encoding schemes like encoded-words are also complicated, which adds inefficiencies and a lot of room for processing errors 💀 And native support for the UTF-8 charset is now on most systems, so it's really desirable for Internet mail to support UTF-8 [RFC3629] directly 🫶

This document adds to the Internet Message Format [RFC5322] and to MIME so you can use UTF-8 directly, not just ASCII, in header field values, mail addresses included 🦋 It also defines a new media type, message/global, for messages that use this extended format 🧚 This specification removes the MIME restriction on non-identity content-transfer-encodings for any subtype of the message top-level type, so message/global parts can be sent safely across existing mail infrastructure 🎀

This spec is based on native, end-to-end support for UTF-8, and it depends on an "8-bit-clean" environment that the transport system guarantees 🌈 Carrying mail across legacy 7-bit infrastructure, or processing it in 7-bit receivers, needs extra mechanisms that these specifications don't provide 🙈

This specification is a revision of and replacement for [RFC5335] ✨ Section 6 of [RFC6530] describes how the approach changed between this spec and the previous version 💅

2. Terminology Used in This Specification 🔥

A plain ASCII string is fully compatible with [RFC5321] and [RFC5322] 💖 In this document, non-ASCII strings are UTF-8 strings when they appear in header field values that contain at least one <UTF8-non-ascii> (see Section 3.1) 🌸

Unless noted otherwise, all terms used here are defined in [RFC5321], [RFC5322], [RFC6530], or [RFC6531] 🧚

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119] 👑

The term "8-bit" means octets are present in the data with values above 0x7F 🍓

3. Changes to Message Header Fields 💜

To allow non-ASCII Unicode characters in field values, the header definition in [RFC5322] gets extended to support the new format ✨ The following sections spell out the changes needed to RFC 5322's ABNF 🎀

Any syntax rules not mentioned below stay defined as they are in [RFC5322] 🫶

Note that this protocol does not change the rules in RFC 5322 for defining header field names 🙈 The bodies of header fields can contain Unicode characters, but the header field names themselves must consist of ASCII characters only 😭

Also, messages in this format require the use of the SMTPUTF8 extension [RFC6531] to be transferred via SMTP 📬

3.1. UTF-8 Syntax and Normalization 🌈

UTF-8 characters can be defined in terms of octets using the following ABNF [RFC5234], taken from [RFC3629]:

UTF8-non-ascii  =   UTF8-2 / UTF8-3 / UTF8-4

UTF8-2          =   <Defined in Section 4 of RFC3629>

UTF8-3          =   <Defined in Section 4 of RFC3629>

UTF8-4          =   <Defined in Section 4 of RFC3629>

See [RFC5198] for a discussion of Unicode normalization. Normalization form NFC [UNF] SHOULD be used 💅 Okay, and this part is so real: if you're doing internationalization properly, one of the most-cited goals is letting people spell their names correctly 🥺 Since a lot of mailbox local parts reflect personal names, that principle applies to mailboxes too 💖 The NFKC normalization form [UNF] SHOULD NOT be used, because it may lose information needed to spell some names correctly in some unusual circumstances 🙈

3.2. Syntax Extensions to RFC 5322 🔥

The following rules extend the ABNF syntax defined in [RFC5322] and [RFC5234] so they can carry UTF-8 content 🦋

VCHAR   =/  UTF8-non-ascii

ctext   =/  UTF8-non-ascii

atext   =/  UTF8-non-ascii

qtext   =/  UTF8-non-ascii

text    =/  UTF8-non-ascii
             ; note that this upgrades the body to UTF-8

dtext   =/  UTF8-non-ascii

Because of those changes, these constructs now allow UTF-8:

  1. Unstructured text, like in header fields such as "Subject:" or "Content-description:".
  2. Any construct that uses atoms, including but not limited to the local parts of addresses and Message-IDs. That covers addresses in the "for" clauses of "Received:" header fields 🧚
  3. Quoted strings 🎀
  4. Domains 🌸

Header field names are not on this list. They're still restricted to ASCII 🙅‍♀️

3.3. Use of 8-bit UTF-8 in Message-IDs 🍓

Implementers of Message-ID generation algorithms MAY prefer to restrain their output to ASCII 🫶 That has some advantages, like when building "In-reply-to:" and "References:" header fields in mailing-list threads where some senders use internationalized addresses and others don't ✨

3.4. Effects on Line Length Limits 📏

Section 2.1.1 of [RFC5322] limits lines to 998 characters and recommends that lines be restricted to only 78 characters 💖 This specification changes the former limit to 998 octets. (Note that in ASCII, octets and characters are effectively the same, but that's not true in UTF-8 🙈) The 78-character limit stays defined in terms of characters, not octets, since it's meant to address display width issues, not line-length issues 💅

3.5. Changes to MIME Message Type Encoding Restrictions 🎀

This specification updates Section 6.4 of [RFC2045] 🌈 [RFC2045] prohibits applying a content-transfer-encoding to any subtypes of "message/". This specification relaxes that rule. It allows newly defined MIME types to permit content-transfer-encoding, and it allows content-transfer-encoding for message/global (see Section 3.7) 🦋

Background: normally, transfer of message/global will happen over 8-bit-clean channels, and body parts will have "identity" encodings, meaning no decoding is necessary 🧚

But when a message containing a message/global gets downgraded from 8-bit to 7-bit as described in [RFC6152], an encoding might have to be applied to the message 😭 If the message travels back and forth between a 7-bit environment and an environment implementing these extensions, multiple levels of encoding may happen. This is expected to be rarely seen in practice, and the complexity of other ways to handle the issue is thought to be larger than the complexity of allowing nested encodings where necessary 💜

3.6. Use of MIME Encoded-Words 💅

okay so the MIME encoded-words thing [RFC2047] lets you stick non-ASCII text in headers, but only in a few of the spots this extension already covers 🙄. Plus encoded-words are kinda messy because they let you use basically any charset, so it's giving unnecessary complexity 💀. So bestie, encoded-words SHOULD NOT be used when you're generating header fields for messages using this extension 🫶. Agents MAY, when they're pulling material in from another message, convert encoded-word use into direct UTF-8 ✨.

Heads up though: when you decode an encoded-word and swap it for its UTF-8 version, the result might be syntactically invalid 😭. Processors that choose to decode encoded-words MUST NOT generate syntactically invalid fields 🔥. No cap, this one is important.

3.7. The message/global Media Type 🦋

Internationalized messages in this format MUST only be transmitted as authorized by [RFC6531] or within a non-SMTP environment that supports these messages 🌸. A message counts as a "message/global message" if:

  • it contains 8-bit UTF-8 header values as specified in this document, or
  • it contains 8-bit UTF-8 values in the header fields of body parts.

Otherwise the content of a message/global part is identical to a message/rfc822 part, which is lowkey iconic 👑.

If an object of this type is sent to a 7-bit-only system, it MUST have an appropriate content-transfer-encoding applied 🙈. (Fun fact: a MIME-compliant system that doesn't recognize message/global is supposed to treat it as "application/octet-stream" as described in Section 5.2.4 of [RFC2046] 🧚.)

The registration goes like this 🎀:

Type name:  message

Subtype name:  global

Required parameters:  none

Optional parameters:  none

Encoding considerations:  Any content-transfer-encoding is permitted.
   The 8-bit or binary content-transfer-encodings are recommended
   where permitted.

Security considerations:  See Section 4.

Interoperability considerations:  This media type provides
   functionality similar to the message/rfc822 content type for email
   messages with internationalized email headers.  When there is a
   need to embed or return such content in another message, there is
   generally an option to use this media type and leave the content
   unchanged or down-convert the content to message/rfc822.  Each of
   these choices will interoperate with the installed base, but with
   different properties.  Systems unaware of internationalized
   headers will typically treat a message/global body part as an
   unknown attachment, while they will understand the structure of a
   message/rfc822.  However, systems that understand message/global
   will provide functionality superior to the result of a down-
   conversion to message/rfc822.  The most interoperable choice
   depends on the deployed software.

Published specification:  RFC 6532

Applications that use this media type:  SMTP servers and email
   clients that support multipart/report generation or parsing.
   Email clients that forward messages with internationalized headers
   as attachments.

Additional information:

   Magic number(s):  none

   File extension(s):  The extension ".u8msg" is suggested.

   Macintosh file type code(s):  A uniform type identifier (UTI) of
      "public.utf8-email-message" is suggested.  This conforms to
      "public.message" and "public.composite-content", but does not
      necessarily conform to "public.utf8-plain-text".

Person & email address to contact for further information:  See the
   Authors' Addresses section of this document.

Intended usage:  COMMON

Restrictions on usage:  This is a structured media type that embeds
   other MIME media types.  An 8-bit or binary content-transfer-
   encoding SHOULD be used unless this media type is sent over a
   7-bit-only transport.

Author:  See the Authors' Addresses section of this document.

Change controller:  IETF Standards Process

4. Security Considerations 🔐

Because UTF-8 often needs several octets to encode a single character, internationalization can make header field values (in general) and mail addresses (in particular) longer 📏. As specified in [RFC5322], each line of characters MUST be no more than 998 octets, excluding the CRLF 🚫. On the other hand, MDA (Mail Delivery Agent) processes that parse, store, or handle email addresses or local parts MUST take extra care not to overflow buffers, truncate addresses, or exceed storage allotments 😱. Also, when comparing, they must take care to use the entire lengths of the addresses 👀.

Be careful, bestie: there are lots of ways to use UTF-8 to represent something equivalent or similar to a particular displayed character or group of characters. See the security considerations in [RFC3629] for the details on the problems this can cause 💀. The normalization process described in Section 3.1 is recommended to minimize these issues 🫶.

The security impact of UTF-8 headers on email signature systems such as Domain Keys Identified Mail (DKIM), S/MIME, and OpenPGP is discussed in Section 14 of [RFC6530] 🔏.

If a user has a non-ASCII mailbox address and an ASCII mailbox address, a digital certificate that identifies that user might have both addresses in the identity 🪪. Having multiple email addresses as identities in a single certificate is already supported in PKIX (Public Key Infrastructure using X.509) [RFC5280] and OpenPGP [RFC3156], but there may be user-interface issues when UTF-8 gets introduced into addresses in this context 🎭.

5. IANA Considerations 🗂️

IANA has updated the registration of the message/global MIME type using the registration form contained in Section 3.7 📝.

6. Acknowledgements 🙏

This document incorporates many ideas first described in a draft document by Paul Hoffman, although many details have changed from that earlier work 💜. The authors especially thank Jeff Yeh for his efforts and contributions on editing previous versions 🌷.

Most of the content of this document was provided by John C Klensin and Dave Crocker 👑. Significant comments and suggestions were received from Martin Duerst, Julien Elie, Arnt Gulbrandsen, Kristin Hubner, Kari Hurtta, Yangwoo Ko, Charles H. Lindsey, Alexey Melnikov, Chris Newman, Pete Resnick, Yoshiro Yoneya, and additional members of the Joint Engineering Team (JET), and were incorporated into the document 🌈. The authors wish to sincerely thank them all for their contributions 🥹.

7. References 📚

7.1. Normative References

[ASCII]    "Coded Character Set -- 7-bit American Standard Code for
           Information Interchange", ANSI X3.4, 1986.

[RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
           Requirement Levels", BCP 14, RFC 2119, March 1997.

[RFC3629]  Yergeau, F., "UTF-8, a transformation format of ISO
           10646", STD 63, RFC 3629, November 2003.

[RFC5198]  Klensin, J. and M. Padlipsky, "Unicode Format for Network
           Interchange", RFC 5198, March 2008.

[RFC5234]  Crocker, D. and P. Overell, "Augmented BNF for Syntax
           Specifications: ABNF", STD 68, RFC 5234, January 2008.

[RFC5321]  Klensin, J., "Simple Mail Transfer Protocol", RFC 5321,
           October 2008.

[RFC5322]  Resnick, P., Ed., "Internet Message Format", RFC 5322,
           October 2008.

[RFC6530]  Klensin, J. and Y. Ko, "Overview and Framework for
           Internationalized Email", RFC 6530, February 2012.

[RFC6531]  Yao, J. and W. Mao, "SMTP Extension for Internationalized
           Email", RFC 6531, February 2012.

[UNF]      Davis, M. and K. Whistler, "Unicode Standard Annex #15:
           Unicode Normalization Forms", September 2010,
           <http://www.unicode.org/reports/tr15/>.

7.2. Informative References

[RFC2045]  Freed, N. and N. Borenstein, "Multipurpose Internet Mail
           Extensions (MIME) Part One: Format of Internet Message
           Bodies", RFC 2045, November 1996.

[RFC2046]  Freed, N. and N. Borenstein, "Multipurpose Internet Mail
           Extensions (MIME) Part Two: Media Types", RFC 2046,
           November 1996.

[RFC2047]  Moore, K., "MIME (Multipurpose Internet Mail Extensions)
           Part Three: Message Header Extensions for Non-ASCII Text",
           RFC 2047, November 1996.

[RFC3156]  Elkins, M., Del Torto, D., Levien, R., and T. Roessler,
           "MIME Security with OpenPGP", RFC 3156, August 2001.

[RFC5280]  Cooper, D., Santesson, S., Farrell, S., Boeyen, S.,
           Housley, R., and W. Polk, "Internet X.509 Public Key
           Infrastructure Certificate and Certificate Revocation List
           (CRL) Profile", RFC 5280, May 2008.

[RFC5335]  Yang, A., "Internationalized Email Headers", RFC 5335,
           September 2008.

[RFC6152]  Klensin, J., Freed, N., Rose, M., and D. Crocker, "SMTP
           Service Extension for 8-bit MIME Transport", STD 71,
           RFC 6152, March 2011.

Authors' Addresses 🎀

Abel Yang
TWNIC
4F-2, No. 9, Sec 2, Roosevelt Rd.
Taipei  100
Taiwan

Phone: +886 2 23411313 ext 505
EMail: abelyang@twnic.net.tw


Shawn Steele
Microsoft

EMail: Shawn.Steele@microsoft.com


Ned Freed
Oracle
800 Royal Oaks
Monrovia, CA  91016-6347
USA

EMail: ned+ietf@mrochek.com
📦 where this page comes from