RFC 5335
Internationalized Email Headers
Internationalized Email Headers ๐๐ฆ
Okay bestie, this is RFC 5335 ๐ It's called Internationalized Email Headers, written by Y. Abel, Ed. from TWNIC, and it dropped in September 2008. It's Experimental, so it's an experiment, not an Internet standard. It updates RFC 2045 and RFC 2822 (the old email format rules), and it doesn't list anything it obsoletes. The Network Working Group published it.
Status of This Memo โจ
This memo defines an Experimental Protocol for the Internet community, and it does not specify an Internet standard of any kind. The authors want feedback and suggestions for improvement, and anyone can distribute it freely. ๐
Abstract ๐ธ
Full internationalization of email is a big ask. You need to send non-ASCII content, encode certain info in specific header fields, and use non-ASCII characters in envelope addresses. You also need to be able to write those addresses, and the info based on them, right inside the mail header fields. This document describes an experimental version of Internet mail that lets Unicode, encoded in UTF-8, replace ASCII as the base form for email header fields. It only gets sent across the wire if an SMTP extension authorizes it, which the associated spec explains. This document also updates Section 6.4 of RFC 2045 so everything lines up with these requirements. ๐ฆ
Table of Contents ๐
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1. Role of This Specification . . . . . . . . . . . . . . . . 3
1.2. Relation to Other Standards . . . . . . . . . . . . . . . 3
2. Background and History . . . . . . . . . . . . . . . . . . . . 3
3. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 4
4. Changes on Message Header Fields . . . . . . . . . . . . . . . 5
4.1. UTF-8 Syntax and Normalization . . . . . . . . . . . . . . 5
4.2. Changes on MIME Headers . . . . . . . . . . . . . . . . . 6
4.3. Syntax Extensions to RFC 2822 . . . . . . . . . . . . . . 6
4.4. Change on addr-spec Syntax . . . . . . . . . . . . . . . . 8
4.5. Trace Field Syntax . . . . . . . . . . . . . . . . . . . . 9
4.6. message/global . . . . . . . . . . . . . . . . . . . . . . 9
5. Security Considerations . . . . . . . . . . . . . . . . . . . 11
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 12
7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 12
8. References . . . . . . . . . . . . . . . . . . . . . . . . . . 12
8.1. Normative References . . . . . . . . . . . . . . . . . . . 12
8.2. Informative References . . . . . . . . . . . . . . . . . . 13
1. Introduction ๐ง
1.1. Role of This Specification ๐
Full internationalization of email needs three things:
- The ability to send non-ASCII content, which the basic MIME spec covers in [RFC2045], [RFC2046].
- The ability to use international characters in envelope addresses. That's discussed in [RFC4952] and specified in [RFC5336].
- The ability to express those addresses, plus the info related to them and based on them, in mail header fields. That's what this document defines.
This document describes an experimental version of Internet mail that lets Unicode encoded in UTF-8 [RFC3629] replace ASCII as the base form for email header fields. It only gets sent if the SMTP extension in [RFC5336] authorizes it, or if some other transport mechanism that can handle it is in play. ๐
1.2. Relation to Other Standards ๐
This document updates Section 6.4 of RFC 2045. It drops the blanket ban on putting a content-transfer-encoding on every subtype of message/. Instead, a composite subtype MAY say whether a content-transfer-encoding can be used for that subtype, and the default is "cannot be used." ๐
It also updates [RFC2822] and MIME ([RFC2045]). Because this is an Experimental spec updating Standards-Track specs, anyone taking part in the experiment has to treat those standards as updated. โจ
Allowing a content-transfer-encoding on message subtypes isn't just for transmissions authorized by the SMTP extension in [RFC5336]. Also, message/global allows a content-transfer-encoding. ๐
2. Background and History ๐ท
Mailbox names are often human names. Lots of people around the world have names that can't be written with only ASCII characters, and they want their mailbox names to look more or less like their real names. ๐ซถ
These users also tend to want non-ASCII text in their display names and in email subjects, both incoming and outgoing. This protocol uses UTF-8 as the encoding for email header field bodies. ๐ฆ
The traditional email message format [RFC2822] only allows ASCII characters in header fields. That stops people from having email addresses with non-ASCII characters. It also forces non-ASCII text in common names, comments, and free text (like the Subject: field) to be encoded, as the MIME format [RFC2047] requires. This specification changes the email message format to match the SMTP transport change in the associated documents [RFC4952] and [RFC5336]. It allows non-ASCII characters in most email header fields. These changes affect SMTP clients, SMTP servers, mail user agents (MUAs), list expanders, gateways to other media, and every other process that parses or handles email messages. ๐ฅ
As specified in [RFC5336], an SMTP protocol extension called "UTF8SMTP" stops messages with UTF-8 header fields from being sent to systems that can't handle them. ๐ก๏ธ
Using this SMTP extension helps keep those messages out of message stores that might misread, display wrong, or mangle them. Keep in mind that using an ESMTP extension doesn't stop email messages with UTF-8 header fields from going to other systems that use the email format and may not be upgraded, like unextended POP and IMAP servers. Changes to those protocols so they can handle UTF-8 header fields are covered in [EAI-POP] and [IMAP-UTF8] . ๐ฌ
The goal of this protocol is to allow UTF-8 in email header fields. How to handle messages with UTF-8 header fields that have to be delivered to systems not yet upgraded for this is discussed in [DOWNGRADE]. ๐ธ
3. Terminology ๐
A plain ASCII string is also a valid UTF-8 string, see [RFC3629]. In this document, ordinary ASCII characters count as UTF-8 characters when they're in headers that contain <utf8-xtra-char>s. ๐
Unless noted otherwise, all the terms used here are defined in [RFC2821], [RFC2822], [RFC4952], or [RFC5336]. ๐
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. ๐
4. Changes on Message Header Fields ๐
SMTP clients can send header fields in UTF-8 format, as long as the UTF8SMTP extension is advertised by the SMTP server or some other transport mechanism permits it. ๐ซง
This protocol does NOT change the [RFC2822] rules for header field names. The bodies of header fields may contain UTF-8 characters, but the header field names themselves must contain only ASCII characters. ๐
To allow UTF-8 characters in field values, the header definition in [RFC2822] must be extended to support the new format. The ABNF below is defined to replace those definitions in [RFC2822]. โจ
The syntax rules this section doesn't cover stay as defined in [RFC2822]. ๐
4.1. UTF-8 Syntax and Normalization ๐
UTF-8 characters can be defined in terms of octets using the ABNF below [RFC5234], taken from [RFC3629]:
UTF8-xtra-char = UTF8-2 / UTF8-3 / UTF8-4
UTF8-2 = %xC2-DF UTF8-tail
UTF8-3 = %xE0 %xA0-BF UTF8-tail /
%xE1-EC 2(UTF8-tail) /
%xED %x80-9F UTF8-tail /
%xEE-EF 2(UTF8-tail)
UTF8-4 = %xF0 %x90-BF 2( UTF8-tail ) /
%xF1-F3 3( UTF8-tail ) /
%xF4 %x80-8F 2( UTF8-tail )
UTF8-tail = %x80-BF
These rules are normatively defined in [RFC3629], but this document keeps them for convenience. ๐ฆ
See [RFC5198] for a discussion of normalization. Using normalization form NFC is RECOMMENDED. ๐ง
4.2. Changes on MIME Headers ๐
This specification updates Section 6.4 of [RFC2045]. [RFC2045] prohibits applying a content-transfer-encoding to every subtype of message/. This specification loosens that rule. It allows newly defined MIME types to permit a content-transfer-encoding, and it allows one for message/global (see Section 4.6). ๐ธ
Background: Normally, message/global is transferred over 8-bit-clean channels, and body parts get "identity" encodings, meaning no decoding is needed. But if a message containing a message/global gets downgraded from 8-bit to 7-bit as described in [RFC1652], an encoding may be applied to it. If the message keeps bouncing between a 7-bit environment and an environment implementing UTF8SMTP, several levels of encoding can pile up. This is expected to be rare in practice, and the authors figure that dealing with it in other ways would be more complicated than allowing nested encodings where needed. ๐๐
4.3. Syntax Extensions to RFC 2822 ๐
Okay so this is where we stretch the [RFC2822] rules so they can take UTF-8 characters ๐ธ. These are the extended rules, and the unchanged ones point back to [RFC2822] for folding white space ๐:
FWS = <see [RFC2822], folding white space>
CFWS = <see [RFC2822], folding white space>
ctext =/ UTF8-xtra-char
utext =/ UTF8-xtra-char
comment = "(" *([FWS] utf8-ccontent) [FWS] ")"
word = utf8-atom / utf8-quoted-string
Here's the best part โจ. Every [RFC2822] construct built on these pieces now lets UTF-8 through, comments and quoted strings included ๐ซถ. We do not touch <atext>, so UTF-8 can't sneak into <addr-spec>. That would also let UTF-8 into <message-id>, which is a big no because of the limitation in Section 4.5 ๐ญ. So instead we add a brand new <utf8-atext> to do the job ๐ฆ.
utf8-text = %d1-9 / ; all UTF-8 characters except
%d11-12 / ; US-ASCII NUL, CR, and LF
%d14-127 /
UTF8-xtra-char
utf8-quoted-pair = ("\" utf8-text) / obs-qp
utf8-qcontent = utf8-qtext / utf8-quoted-pair
utf8-quoted-string = [CFWS]
DQUOTE *([FWS] utf8-qcontent) [FWS] DQUOTE
[CFWS]
utf8-ccontent = ctext / utf8-quoted-pair / comment
utf8-qtext = qtext / UTF8-xtra-char
utf8-atext = ALPHA / DIGIT /
"!" / "#" / ; Any character except
"$" / "%" / ; controls, SP, and specials.
"&" / "'" / ; Used for atoms.
"*" / "+" /
"-" / "/" /
"=" / "?" /
"^" / "_" /
"`" / "{" /
"|" / "}" /
"~" /
UTF8-xtra-char
utf8-atom = [CFWS] 1*utf8-atext [CFWS]
utf8-dot-atom = [CFWS] utf8-dot-atom-text [CFWS]
utf8-dot-atom-text = 1*utf8-atext *("." 1*utf8-atext)
qcontent = utf8-qcontent
Now, to let UTF-8 into the Content-Description header field [RFC2045] ๐, we use this syntax:
description = "Content-Description:" unstructured CRLF
We also extend <utext> up above so UTF-8 works in every <unstructured> header field ๐.
Quick heads-up though ๐: none of this removes any limits on protocol elements. For example, every allowed timezone value in the Date: headers is still written in ASCII ๐ฐ๏ธ. And none of the new syntax changes what's allowed in a <msg-id>, which stays pure ASCII. No cap, that part is non-negotiable ๐
โโ๏ธ.
4.4. Change on addr-spec Syntax ๐ฌ
Internationalized email addresses are written in UTF-8, so every header field with <mailbox>es gets updated to allow UTF-8 ๐. Each one can also carry an extra, optional, all-ASCII alternate address ๐ซถ. Heads up: Message Submission Servers ("MSAs") and Message Transfer Agents (MTAs) may downgrade internationalized messages when they need to ๐. The how-to lives in [DOWNGRADE].
mailbox = name-addr / addr-spec / utf8-addr-spec
angle-addr =/ [CFWS] "<" utf8-addr-spec [ alt-address ] ">"
[CFWS] / obs-angle-addr
utf8-addr-spec = utf8-local-part "@" utf8-domain
utf8-local-part= utf8-dot-atom / utf8-quoted-string / obs-local-part
utf8-domain = utf8-dot-atom / domain-literal / obs-domain
alt-address = FWS "<" addr-spec ">"
Here are a few <mailbox> looks to vibe check โจ:
"DISPLAY_NAME" <ASCII@ASCII>
; traditional mailbox format
"DISPLAY_NAME" <non-ASCII@non-ASCII>
; UTF8SMTP but no ALT-ADDRESS parameter provided,
; message will bounce if UTF8SMTP extension is not supported
<non-ASCII@non-ASCII>
; without DISPLAY_NAME and quoted string
; UTF8SMTP but no ALT-ADDRESS parameter provided,
; message will bounce if UTF8SMTP extension is not supported
"DISPLAY_NAME" <non-ASCII@non-ASCII <ASCII@ASCII>>
; UTF8SMTP with ALT-ADDRESS parameter provided,
; ALT-ADDRESS can be used if downgrade is necessary
4.5. Trace Field Syntax ๐
"For" fields can now hold internationalized addresses, thanks to the new uFor syntax ๐ง. Received fields might need UTF-8 info too, so that info is allowed in there to keep those fields intact ๐ . The uFor syntax keeps the original UTF-8 email address between EAI-aware MTAs (that's email address internationalization, bestie ๐ธ). If downgrading is needed, the uFor parameter gets dropped following the procedure in [DOWNGRADE].
The "Return-Path" header shows the return address for the mail delivery ๐ฎ. So it gets upgraded to carry UTF-8 addresses too (check the revised <angle-addr> syntax in Section 4.4 of this document ๐). Trace field integrity stays safe, since that header gets added at the last MTA and is described in [RFC2821] ๐.
The <item-value> in "Received:" syntax now allows a UTF-8 email address in the "For" field ๐ฆ. <angle-addr> is extended to include UTF-8 email addresses too. And to allow UTF-8 email addresses in an <addr-spec>, <utf8-addr-spec> gets added to <item-value> ๐ซถ:
item-value =/ utf8-addr-spec
4.6. message/global ๐
Internationalized messages must only be transmitted as authorized by [RFC5336], or inside a non-SMTP environment that supports these messages ๐ธ. A message counts as a "message/global message" if:
- it contains UTF-8 header values as specified in this document, or
- it contains UTF-8 values in the headers fields of body parts.
The type message/global is basically a cousin of message/rfc822 ๐. The difference is that it can hold a message with UTF-8 characters in the message headers or in the body parts' headers ๐. If this type gets sent to a 7-bit-only system, it has to be encoded in MIME [RFC2045]. (Fun fact: a MIME-compliant system that doesn't recognize message/global would treat it as "application/octet-stream", per Section 5.2.4 of [RFC2046] ๐ค.)
SMTP servers and other systems that transfer a message/global body part MAY choose to down-convert it to a message/rfc822 body part, using the rules in [DOWNGRADE] ๐ฝ.
Here's the registration form, served up nice and neat ๐:
- Type name:
message - Subtype name:
global - Required parameters: none
- Optional parameters: none
- Encoding considerations: Any content-transfer-encoding is permitted. The 8-bit or binary content-transfer-encodings are recommended where permitted.
- Security considerations: See Section 5.
- Interoperability considerations: This media type gives you similar powers to message/rfc822 for email with international headers ๐ . When you need to embed or return that kind of content inside another message, you usually get to pick: this type with the content left alone, or a down-convert to message/rfc822. Both interoperate with the installed base, just with different vibes ๐ซถ. Systems that don't know international headers will usually treat a message/global body part as an unknown attachment, while they still understand the structure of a message/rfc822. Systems that do get message/global can offer better functionality than a down-converted message/rfc822 โจ. Which option is most interoperable depends on the software that's out there ๐คทโโ๏ธ.
- Published specification: RFC 5335
- Applications that use this media type: SMTP servers and email clients that support multipart/report generation or parsing ๐. Email clients that forward messages with international headers as attachments ๐.
- Additional information:
- Magic number(s): none
- File extension(s): The extension ".u8msg" is suggested.
- Macintosh file type code(s): A uniform type identifier (UTI) of "public.utf8-email-message" is suggested. This conforms to "public.message" and "public.composite-content", but does not necessarily conform to "public.utf8-plain-text" ๐.
- Person & email address to contact for further information: See the Author's Address section of this document.
- Intended usage: COMMON
- Restrictions on usage: This is a structured media type that embeds other MIME media types. The 8-bit or binary content-transfer-encoding MUST be used unless this media type is sent over a 7-bit-only transport ๐ซ.
- Author: See the Author's Address section of this document.
- Change controller: IETF Standards Process
Lowkey the most important rule here: the 8-bit or binary content-transfer-encoding MUST be used unless you're on a 7-bit-only transport. Don't skip it, bestie ๐ โโ๏ธ.
5. Security Considerations ๐
Okay, security time, and I need you to take this seriously ๐ญ. If a user has a non-ASCII mailbox address and an ASCII mailbox address, a digital certificate that identifies that user may have both addresses in the identity. Having several email addresses as identities in one certificate is already supported in PKIX (Public Key Infrastructure for X.509 Certificates) and OpenPGP ๐ชช.
UTF-8 often needs several octets to encode a single character, so internationalized local parts can make mail addresses longer ๐. As specified in [RFC2822], each line of characters MUST be no more than 998 octets, excluding the CRLF ๐จ.
Because internationalized local parts can make email addresses longer, any process that parses, stores, or handles email addresses or local parts must take extra care not to overflow buffers, truncate addresses, or exceed storage allotments ๐. When comparing addresses, they must also take care to use the entire lengths of the addresses ๐.
In this specification, a user is allowed to give an ASCII alternative address for a non-ASCII address ๐ญ. But these two addresses might go to different mailboxes, or even to different people ๐ฌ. That setup might come from a user's personal choice or from administration policy. We know that if ASCII and non-ASCII email get delivered to two different destinations based on MTA capability, that might violate the principle of least astonishment. Still, it isn't a "protocol problem" ๐คทโโ๏ธ.
The security impact of UTF-8 headers on email signature systems such as Domain Keys Identified Mail (DKIM), S/MIME, and OpenPGP is discussed in RFC 4952, Section 9 ๐๏ธ. A future document, [DOWNGRADE], will cover how downgrading affects those systems ๐ฎ.
6. IANA Considerations ๐๏ธ
IANA has registered the message/global MIME type using the registration form in Section 4.4 ๐.
7. Acknowledgements ๐ซถ
This document pulls together a lot of ideas that Paul Hoffman first described in Internet-Draft form, although many details changed from that earlier work ๐.
The author especially thanks Jeff Yeh for his efforts and contributions on editing previous versions ๐.
Most of the content in this document was provided by John C Klensin ๐. Some significant comments and suggestions came from Charles H. Lindsey, Kari Hurtta, Pete Resnick, Alexey Melnikov, Chris Newman, Yangwoo Ko, Yoshiro Yoneya, and other members of the JET team (Joint Engineering Team), and they were folded into the document. The editor sincerely thanks all of them ๐ฉท.
8. References โจ
8.1. Normative References โจ
Okay bestie, these are the docs this RFC needs you to have read, no skipping ๐
[RFC1652]Klensin, J., Freed, N., Rose, M., Stefferud, E., and D. Crocker, "SMTP Service Extension for 8bit-MIMEtransport", RFC 1652, July 1994. ๐ฌ[RFC2119]Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. ๐[RFC2821]Klensin, J., "Simple Mail Transfer Protocol", RFC 2821, April 2001. โ๏ธ[RFC2822]Resnick, P., "Internet Message Format", RFC 2822, April 2001. ๐[RFC3629]Yergeau, F., "UTF-8, a transformation format of ISO 10646", STD 63, RFC 3629, November 2003. ๐[RFC4952]Klensin, J. and Y. Ko, "Overview and Framework for Internationalized Email", RFC 4952, July 2007. ๐ฆ[RFC5198]Klensin, J. and M. Padlipsky, "Unicode Format for Network Interchange", RFC 5198, March 2008. ๐ธ[RFC5234]Crocker, D. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", STD 68, RFC 5234, January 2008. ๐งฉ[RFC5336]Yao, J., Ed. and W. Mao, Ed., "SMTP Extension for Internationalized Email Addresses", RFC 5336, September 2008. ๐
8.2. Informative References ๐ง
These ones are the "nice to know" reads. Still worth a look, just not the must-haves ๐
[DOWNGRADE]Fujiwara, K. and Y. Yoneya, "Downgrading mechanism for Email Address Internationalization", Work in Progress, July 2008. ๐[EAI-POP]Newman, C. and R. Gellens, "POP3 Support for UTF-8", Work in Progress, July 2008. ๐ฅ[IMAP-UTF8]Resnick, P. and C. Newman, "IMAP Support for UTF-8", Work in Progress, April 2008. ๐[RFC2045]Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies", RFC 2045, November 1996. ๐ซถ[RFC2046]Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part Two: Media Types", RFC 2046, November 1996. ๐ท[RFC2047]Moore, K., "MIME (Multipurpose Internet Mail Extensions) Part Three: Message Header Extensions for Non-ASCII Text", RFC 2047, November 1996. โญ
Author's Address ๐
Here's where you can reach the editor, Abel Yang, at TWNIC ๐
Abel Yang (editor)
TWNIC
4F-2, No. 9, Sec 2, Roosvelt Rd.
Taipei, 100
Taiwan
Phone: +886 2 23411313 ext 505
EMail: abelyang@twnic.net.tw
Full Copyright Statement โ๏ธ
The legal bits are super important, so I'm keeping this one word-for-word. No vibes-based edits here ๐
Copyright (C) The IETF Trust (2008).
This document is subject to the rights, licenses and restrictions
contained in BCP 78, and except as set forth therein, the authors
retain all their rights.
This document and the information contained herein are provided on an
"AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS
OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND
THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS
OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF
THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Intellectual Property ๐
This section is about who owns what and how to ask about it, so it's also kept exactly as written ๐ฆ
The IETF takes no position regarding the validity or scope of any
Intellectual Property Rights or other rights that might be claimed to
pertain to the implementation or use of the technology described in
this document or the extent to which any license under such rights
might or might not be available; nor does it represent that it has
made any independent effort to identify any such rights. Information
on the procedures with respect to rights in RFC documents can be
found in BCP 78 and BCP 79.
Copies of IPR disclosures made to the IETF Secretariat and any
assurances of licenses to be made available, or the result of an
attempt made to obtain a general license or permission for the use of
such proprietary rights by implementers or users of this
specification can be obtained from the IETF on-line IPR repository at
http://www.ietf.org/ipr.
The IETF invites any interested party to bring to its attention any
copyrights, patents or patent applications, or other proprietary
rights that may cover technology that may be required to implement
this standard. Please address the information to the IETF at
ietf-ipr@ietf.org.
That's the whole doc, bestie! ๐ธโจ
๐ฆ where this page comes from
- ietf: rfc5335 2008-09
- ๐ girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms