RFC 6855
IMAP Support for UTF-8
IMAP Support for UTF-8 💖
Okay bestie, let's start with the paperwork 📋✨ This is RFC 6855, a Standards Track document from the IETF, dated March 2013 with ISSN 2070-1721. It was edited by P. Resnick (Qualcomm Incorporated), C. Newman (Oracle), and S. Shen (CNNIC). It obsoletes RFC 5738 🦋
Abstract 🌸
This spec extends the Internet Message Access Protocol (IMAP) so it can handle UTF-8 encoded international characters in user names, mail addresses, and message headers. It replaces RFC 5738.
Status of This Memo 🎀
This is an Internet Standards Track document.
This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community, received public review, and was approved for publication by the Internet Engineering Steering Group (IESG). More info on Internet Standards is in Section 2 of RFC 5741.
For the current status of this document, any errata, and how to send feedback, check http://www.rfc-editor.org/info/rfc6855.
Copyright Notice 🌈
Copyright (c) 2013 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please read these carefully, because they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and are provided without warranty as described in the Simplified BSD License.
Table of Contents 🧚
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Conventions Used in This Document . . . . . . . . . . . . . . 2
3. "UTF8=ACCEPT" IMAP Capability and UTF-8 in IMAP
Quoted-Strings . . . . . . . . . . . . . . . . . . . . . . . . 3
4. IMAP UTF8 "APPEND" Data Extension . . . . . . . . . . . . . . 4
5. "LOGIN" Command and UTF-8 . . . . . . . . . . . . . . . . . . 5
6. "UTF8=ONLY" Capability . . . . . . . . . . . . . . . . . . . . 5
7. Dealing with Legacy Clients . . . . . . . . . . . . . . . . . 6
8. Issues with UTF-8 Header Mailstore . . . . . . . . . . . . . . 7
9. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 8
10. Security Considerations . . . . . . . . . . . . . . . . . . . 8
11. References . . . . . . . . . . . . . . . . . . . . . . . . . . 9
11.1. Normative References . . . . . . . . . . . . . . . . . . 9
11.2. Informative References . . . . . . . . . . . . . . . . . 10
Appendix A. Design Rationale . . . . . . . . . . . . . . . . . . 11
Appendix B. Acknowledgments . . . . . . . . . . . . . . . . . . . 111. Introduction ✨
This spec is part of the Email Address Internationalization protocols, which are described in the Email Address Internationalization Framework document [RFC6530]. It extends IMAP [RFC3501] so it can allow UTF-8 [RFC3629] in headers, as described in "Internationalized Email Headers" [RFC6532]. It also adds a mechanism for mailbox names that use the UTF-8 charset. Lastly, it creates two new IMAP capabilities so servers can advertise these new extensions 💅
This spec assumes the IMAP server will be running in a fully internationalized environment, meaning every client that accesses the server can accept non-ASCII message header fields and other information, as specified in Section 3. At least during a transition period, that assumption won't be realistic for many environments. Section 7 below talks about those issues 🙈
This spec replaces an earlier, experimental approach to the same problem [RFC5738].
2. Conventions Used in This Document 🌸
The key words MUST, MUST NOT, SHOULD, SHOULD NOT, and MAY in this document are to be interpreted as defined in "Key words for use in RFCs to Indicate Requirement Levels" [RFC2119].
The formal syntax uses the Augmented Backus-Naur Form (ABNF) [RFC5234] notation. The rules from IMAP [RFC3501], UTF-8 [RFC3629], Extensions to IMAP ABNF [RFC4466], and IMAP "LIST" command extensions [RFC5258] are also referenced. This doc assumes you already have a pretty solid understanding of these RFCs 🧠
3. "UTF8=ACCEPT" IMAP Capability and UTF-8 in IMAP Quoted-Strings 🦋
The UTF8=ACCEPT capability tells the client the server can open mailboxes that contain internationalized messages using the SELECT and EXAMINE commands, and that it can send UTF-8 responses to the LIST and LSUB commands. This capability also affects other IMAP extensions that can return mailbox names or their prefixes, like NAMESPACE [RFC2342] and ACL [RFC4314].
The UTF8=ONLY capability, which is described in Section 6, implies the UTF8=ACCEPT capability. A server is said to support UTF8=ACCEPT if it advertises either UTF8=ACCEPT or UTF8=ONLY.
A client MUST use the ENABLE command [RFC5161] with the UTF8=ACCEPT option (defined in Section 4 below) to tell the server that it accepts UTF-8 in quoted-strings and supports the UTF8=ACCEPT extension. The ENABLE UTF8=ACCEPT command is only valid in the authenticated state ✨
Here's the tea ☕ The IMAP base spec [RFC3501] forbids 8-bit characters in atoms or quoted-strings. So a UTF-8 string can only be sent as a literal. That's kind of annoying from a coding standpoint, and unless the server offers IMAP non-synchronizing literals [RFC2088], it means an extra round trip for every UTF-8 string the client sends. When the IMAP server supports UTF8=ACCEPT, it supports UTF-8 in quoted-strings with the following syntax:
quoted =/ DQUOTE *uQUOTED-CHAR DQUOTE
; QUOTED-CHAR is not modified, as it will affect
; other RFC 3501 ABNF non-terminals.
uQUOTED-CHAR = QUOTED-CHAR / UTF8-2 / UTF8-3 / UTF8-4
UTF8-2 = <Defined in Section 4 of RFC 3629>
UTF8-3 = <Defined in Section 4 of RFC 3629>
UTF8-4 = <Defined in Section 4 of RFC 3629>When the client uses this extended quoting mechanism, the server MUST reject, with a BAD response, any octet sequences with the high bit set that don't follow the formal syntax requirements of UTF-8 [RFC3629]. The IMAP server MUST NOT send UTF-8 in quoted-strings to the client unless the client has indicated it supports that syntax by using the ENABLE UTF8=ACCEPT command 🔥
If the server supports UTF8=ACCEPT, the client MAY use extended quoted syntax with any IMAP argument that permits a string (including astring and nstring). However, if characters outside the US-ASCII repertoire show up somewhere they don't belong, the results will be the same as if other syntactically valid but semantically invalid characters had been used. The following paragraphs describe the specific cases where UTF-8 characters are or aren't permitted 👑
All IMAP servers that support UTF8=ACCEPT SHOULD accept UTF-8 in mailbox names. Servers that also support the Mailbox International Naming Convention described in RFC 3501, Section 5.1.3, MUST accept UTF8-quoted mailbox names and convert them to the appropriate internal format. Mailbox names MUST comply with the Net-Unicode Definition ([RFC5198], Section 2), with one specific exception: they MUST NOT contain control characters (U+0000-U+001F and U+0080-U+009F), a delete character (U+007F), a line separator (U+2028), or a paragraph separator (U+2029) 🌷
Once an IMAP client has enabled UTF-8 support with the ENABLE UTF8=ACCEPT command, it MUST NOT issue a SEARCH command that contains a charset specification. If an IMAP server gets a SEARCH command like that in this situation, it SHOULD reject the command with a BAD response, since the charset labels conflict 💀
4. IMAP UTF8 "APPEND" Data Extension 🧚
If the server supports UTF8=ACCEPT, then it accepts UTF-8 headers in the APPEND command message argument. A client that sends a message with UTF-8 headers to the server MUST send them using the UTF8 data extension to the APPEND command. If the server also advertises the CATENATE capability [RFC4469], the client can use that same data extension to include such a message in a catenated message part. Here's the ABNF for the APPEND data extension and the CATENATE extension:
utf8-literal = "UTF8" SP "(" literal8 ")"
literal8 = <Defined in RFC 4466>
append-data =/ utf8-literal
cat-part =/ utf8-literalIf an IMAP server supports UTF8=ACCEPT and the IMAP client has not issued the ENABLE UTF8=ACCEPT command, the server MUST reject, with a NO response, any APPEND command that includes an 8-bit character in message header fields 🙅♀️
5. "LOGIN" Command and UTF-8 🎀
This spec doesn't extend the IMAP LOGIN command [RFC3501] to support UTF-8 usernames and passwords. Whenever a client needs to use UTF-8 usernames or passwords, it MUST use the IMAP AUTHENTICATE command, which already knows how to pass UTF-8 usernames and credentials 🔐
Even though using the AUTHENTICATE command this way makes it syntactically legal to have a UTF-8 username or password, there's no guarantee the user provisioning system the IMAP server uses will allow those identities. That's an implementation decision, and it may depend on which identity system the IMAP server is set up to use.
6. "UTF8=ONLY" Capability 🎀
okay bestie, so UTF8=ONLY is the capability that says the server supports UTF8=ACCEPT (see Section 4) AND that it requires UTF-8 support from clients 💅 no backup plan. the server will send UTF-8 in quoted-strings, and it will not accept the older international mailbox name convention (modified UTF-7 [RFC3501]). because these are incompatible changes to IMAP, there has to be an explicit server announcement AND client confirmation 🥺 so clients MUST use the ENABLE UTF8=ACCEPT command before using this server. a server that advertises UTF8=ONLY will reject, with a NO [CANNOT] response [RFC5530], any command that might need UTF-8 support and isn't preceded by an ENABLE UTF8=ACCEPT command. ngl that's a strict bouncer energy 🚫
IMAP clients that find support for a server announcing UTF8=ONLY problematic are encouraged to at least detect the announcement and show an informative error message to the end-user. no one should be left staring at a blank screen wondering what happened 🙈
since UTF8=ONLY includes support for UTF8=ACCEPT, the capability string will include at most one of those and never both. and for the client, ENABLE UTF8=ACCEPT is always the one used, never ENABLE UTF8=ONLY. only one of them is the main character here and it's UTF8=ACCEPT 👑
7. Dealing with Legacy Clients 🕰️
okay real talk, in most situations it's going to be difficult or impossible for the implementer or operator of an IMAP (or POP) server to know whether every client that might access it, or the mail store more generally, can support the facilities in this document. almost always, servers that follow this spec have to be ready to deal with clients that don't enable the relevant capabilities. unfortunately there's no completely satisfying way to handle that, other than for systems that want to receive email needing SMTPUTF8 to make sure every component is upgraded properly, including IMAP and any other clients users pick 🥲
when a message that needs SMTPUTF8 shows up and the client hasn't enabled UTF-8 capability, the server has a few choices: hiding the problematic message(s), creating in-band or out-of-band notifications or error messages, or trying to make a surrogate of the message so the client gets useful info about what happened. but like, surrogate messages can't be real substitutes for the original 😭 they will almost always be impossible to reply to (either at all or without losing info), and the new header fields or special constructs for server-client communication might go beyond what current email specs ask for (e.g., [RFC5322]). so yeah, they might confuse some legacy mail user agents (including IMAP clients) or not give users the info they expected. there are also trade-offs when you build surrogates: you can accept complexity and extra computation to try to keep as much info as possible (for example, in "Post-Delivery Message Downgrading for Internationalized Email Messages" [RFC6857]), or you can try to minimize those costs while still giving useful info (for example, in "Simplified POP and IMAP Downgrading for Internationalized Email" [RFC6858]). it's giving "choose your fighter" 🥊
implementations that choose to downgrade SHOULD use one of the standardized algorithms from RFC 6857 or RFC 6858. getting downgrade algorithms right, while keeping operational problems and harm to the email system to a minimum, is tricky and needs careful engineering 🛠️ these two algorithms are well understood and carefully designed, so no need to reinvent the wheel bestie.
and here's the tea: because these messages are really surrogates of the originals (even though people often say "downgraded" for convenience), they have relationships to the originals that the IMAP spec [RFC3501] didn't anticipate 🫠 that brings up two big concerns. first, digital signatures computed over and meant for the original message often won't apply to the surrogate, and they'll often fail signature verification. (some digital signatures can still be verified, but only if they cover parts of the original message that weren't changed when the surrogate was made.) second, servers that might be accessed by the same user with different clients or methods (e.g., POP or webmail systems alongside IMAP, or IMAP clients with different capabilities) need to be extremely careful that UIDVALIDITY [RFC3501] behaves the way the user would expect 😳 those issues get especially sensitive if the server caches the surrogate, or computes and stores it when the message arrives so it can offer either form depending on client capabilities.
also, to deal with the case where a server following this extension returns the same UIDVALIDITY to both legacy and UTF8=ACCEPT-aware clients, a client that got upgraded from being non-UTF8=ACCEPT-aware MUST discard its cache of messages downloaded from the server. yes, that means start fresh, no drama, just a clean slate ✨
the best (or "least bad") approach for any given environment depends on local conditions, local assumptions about user behavior, how much control the server operator has over client usage and upgrades, what options are actually available, and so on 💭 it's impossible, at least when this spec was published, to give good advice that works for every situation, or even for particular profiles of situations, other than "upgrade legacy clients as soon as possible". honestly that's the most iconic advice in the whole doc 📣
8. Issues with UTF-8 Header Mailstore 📬
when an IMAP server uses a mailbox format that supports UTF-8 headers, and it lets you select or examine that mailbox without issuing ENABLE UTF8=ACCEPT first, it's the server's responsibility to follow the IMAP base spec [RFC3501] and the Internet Message Format [RFC5322] for all header info sent over the wire 🎯 the question of handling messages with non-ASCII characters in legacy environments is covered in Section 7, so go check that out if you're curious 🧐
9. IANA Considerations 📋
this doc redefines two capabilities (UTF8=ACCEPT and UTF8=ONLY) in the "IMAP 4 Capabilities" registry [RFC3501]. three other capabilities from the experimental predecessor to this document (UTF8=ALL, UTF8=APPEND, UTF8=USER) are now OBSOLETE 🪦 so IANA updated the registry like this:
OLD:
+--------------+-----------------+
| UTF8=ACCEPT | [RFC5738] |
| UTF8=ALL | [RFC5738] |
| UTF8=APPEND | [RFC5738] |
| UTF8=ONLY | [RFC5738] |
| UTF8=USER | [RFC5738] |
+--------------+-----------------+
NEW:
+------------------------+---------------------+
| UTF8=ACCEPT | [RFC6855] |
| UTF8=ALL (OBSOLETE) | [RFC5738] [RFC6855]|
| UTF8=APPEND (OBSOLETE) | [RFC5738] [RFC6855]|
| UTF8=ONLY | [RFC6855] |
| UTF8=USER (OBSOLETE) | [RFC5738] [RFC6855]|
+------------------------+---------------------+10. Security Considerations 🔐
okay this part is SERIOUS bestie, so pay attention 🚨 the security considerations of UTF-8 [RFC3629] and SASLprep [RFC4013] apply to this spec, especially when it comes to using UTF-8 in usernames and passwords. other than that, this isn't believed to change the security considerations of IMAP.
special considerations, some with security implications, come up in two situations. first, when a server that follows this spec is accessed by a client that doesn't. second, in more complex situations where a given message is accessed by multiple clients that might use different protocols and/or support different capabilities. those issues are discussed in Section 7. read it. know it. live it. 💀
11. References 📚
11.1. Normative References 🧷
- [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
- [RFC3501] Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1", RFC 3501, March 2003.
- [RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO 10646", STD 63, RFC 3629, November 2003.
- [RFC4013] Zeilenga, K., "SASLprep: Stringprep Profile for User Names and Passwords", RFC 4013, February 2005.
- [RFC4466] Melnikov, A. and C. Daboo, "Collected Extensions to IMAP4 ABNF", RFC 4466, April 2006.
- [RFC4469] Resnick, P., "Internet Message Access Protocol (IMAP) CATENATE Extension", RFC 4469, April 2006.
- [RFC5161] Gulbrandsen, A. and A. Melnikov, "The IMAP ENABLE Extension", RFC 5161, March 2008.
- [RFC5198] Klensin, J. and M. Padlipsky, "Unicode Format for Network Interchange", RFC 5198, March 2008.
- [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", STD 68, RFC 5234, January 2008.
- [RFC5258] Leiba, B. and A. Melnikov, "Internet Message Access Protocol version 4 - LIST Command Extensions", RFC 5258, June 2008.
- [RFC5322] Resnick, P., Ed., "Internet Message Format", RFC 5322, October 2008.
- [RFC6530] Klensin, J. and Y. Ko, "Overview and Framework for Internationalized Email", RFC 6530, February 2012.
- [RFC6532] Yang, A., Steele, S., and N. Freed, "Internationalized Email Headers", RFC 6532, February 2012.
- [RFC6857] Fujiwara, K., "Post-Delivery Message Downgrading for Internationalized Email Messages", RFC 6857, March 2013.
- [RFC6858] Gulbrandsen, A., "Simplified POP and IMAP Downgrading for Internationalized Email", RFC 6858, March 2013.
11.2. Informative References 🔎
- [RFC2088] Myers, J., "IMAP4 non-synchronizing literals", RFC 2088, January 1997.
- [RFC2342] Gahrns, M. and C. Newman, "IMAP4 Namespace", RFC 2342, May 1998.
- [RFC4314] Melnikov, A., "IMAP4 Access Control List (ACL) Extension", RFC 4314, December 2005.
- [RFC5530] Gulbrandsen, A., "IMAP Response Codes", RFC 5530, May 2009.
- [RFC5738] Resnick, P. and C. Newman, "IMAP Support for UTF-8", RFC 5738, March 2010.
Appendix A. Design Rationale 🧠
this non-normative section talks about the reasons behind some of the design choices in this spec, so it's the "why" bit 💡
the UTF8=ONLY mechanism makes it easier to diagnose interoperability problems when legacy support goes away. when backwards compatibility isn't working anyway, the non-conforming "just-send-UTF-8 IMAP" has the advantage that it might work with some legacy clients. however, the difficulty of diagnosing interoperability problems caused by a "just-send-UTF-8 IMAP" mechanism is the reason the UTF8=ONLY capability mechanism was chosen. lowkey a very mature decision 🫡
Appendix B. Acknowledgments 🫶
the authors want to thank the participants of the EAI working group for their contributions to this document, with a special shoutout to Harald Alvestrand, David Black, Randall Gellens, Arnt Gulbrandsen, Kari Hurtta, John Klensin, Xiaodong Lee, Charles Lindsey, Alexey Melnikov, Subramanian Moonesamy, Shawn Steele, Daniel Taharlev, and Joseph Yee for their specific contributions to the discussion 🌸 the real MVPs 👑
Authors' Addresses 💖
okay bestie, these are the three editors who wrote this spec, so here's exactly how to reach them 🫶
Pete Resnick (editor) 👑
Pete Resnick (editor)
Qualcomm Incorporated
5775 Morehouse Drive
San Diego, CA 92121-1714
USA
Phone: +1 858 651 4478
EMail: presnick@qti.qualcomm.comChris Newman (editor) 🦋
Chris Newman (editor)
Oracle
800 Royal Oaks
Monrovia, CA 91016
USA
Phone:
EMail: chris.newman@oracle.comSean Shen (editor) 🌸
Sean Shen (editor)
CNNIC
No.4 South 4th Zhongguancun Street
Beijing, 100190
China
Phone: +86 10-58813038
EMail: shenshuo@cnnic.cnthree editors, three cities, one very iconic spec 🌈💅
📦 where this page comes from
- ietf: rfc6855 2013-03
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms