RFC 5738

IMAP Support for UTF-8

📅 published March 2010, from ietf

IMAP Support for UTF-8 🎀

Okay bestie, this is RFC 5738 💖 It's called IMAP Support for UTF-8, and it's an Experimental spec, so people are supposed to try it out and poke at it, not treat it as a finished standard. Here's the title-block tea ☕

  • Series: Network Working Group
  • Number: RFC 5738
  • Authors: P. Resnick (Qualcomm Incorporated) and C. Newman (Sun Microsystems)
  • Category: Experimental 🧪
  • Date: March 2010
  • Updates: RFC 3501 (that's IMAP4rev1 🦋)

Abstract ✨

This spec extends IMAP4rev1 so you can use UTF-8 encoded international characters in user names, mail addresses, and message headers. It's giving global inbox energy 🌍

Status of This Memo 📜

This document is not an Internet Standards Track specification. It's published for examination, experimental implementation, and evaluation, so it's basically a "try it and tell us" situation 🧪

It defines an Experimental Protocol for the Internet community. It's a product of the Internet Engineering Task Force (IETF) and represents the consensus of the IETF community. It got public review and was approved for publication by the Internet Engineering Steering Group (IESG). Not every IESG-approved doc is a candidate for any level of Internet Standard, so check out Section 2 of RFC 5741 if you're curious 🤓

For info on the current status of this document, any errata, and how to give feedback, go to http://www.rfc-editor.org/info/rfc5738 🔗

Copyright (c) 2010 IETF Trust and the persons identified as the document authors. All rights reserved. 💜

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as in effect on the date of publication. Please read these carefully, bestie, because they describe your rights and restrictions for this document. Code Components pulled out of this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they come without warranty as described in the Simplified BSD License 🙈

This document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. The people controlling the copyright in some of that material may not have given the IETF Trust the right to allow modifications outside the IETF Standards Process. So unless you get an adequate license from those copyright holders, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process. The exceptions are formatting it for publication as an RFC or translating it into languages other than English 🌸

Table of Contents 📚

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
   2.  Conventions Used in This Document  . . . . . . . . . . . . . .  3
   3.  UTF8=ACCEPT IMAP Capability  . . . . . . . . . . . . . . . . .  3
     3.1.  IMAP UTF-8 Quoted Strings  . . . . . . . . . . . . . . . .  3
     3.2.  UTF8 Parameter to SELECT and EXAMINE . . . . . . . . . . .  5
     3.3.  UTF-8 LIST and LSUB Responses  . . . . . . . . . . . . . .  5
     3.4.  UTF-8 Interaction with IMAP4 LIST Command Extensions . . .  6
       3.4.1.  UTF8 and UTF8ONLY LIST Selection Options . . . . . . .  6
       3.4.2.  UTF8 LIST Return Option  . . . . . . . . . . . . . . .  6
   4.  UTF8=APPEND Capability . . . . . . . . . . . . . . . . . . . .  7
   5.  UTF8=USER Capability . . . . . . . . . . . . . . . . . . . . .  7
   6.  UTF8=ALL Capability  . . . . . . . . . . . . . . . . . . . . .  7
   7.  UTF8=ONLY Capability . . . . . . . . . . . . . . . . . . . . .  8
   8.  Up-Conversion Server Requirements  . . . . . . . . . . . . . .  8
   9.  Issues with UTF-8 Header Mailstore . . . . . . . . . . . . . .  9
   10. IANA Considerations  . . . . . . . . . . . . . . . . . . . . .  9
   11. Security Considerations  . . . . . . . . . . . . . . . . . . . 11
   12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 11
     12.1. Normative References . . . . . . . . . . . . . . . . . . . 11
     12.2. Informative References . . . . . . . . . . . . . . . . . . 13
   Appendix A.  Design Rationale  . . . . . . . . . . . . . . . . . . 14
   Appendix B.  Examples Demonstrating Relationships between
                UTF8= Capabilities  . . . . . . . . . . . . . . . . . 15
   Appendix C.  Acknowledgments . . . . . . . . . . . . . . . . . . . 15

1. Introduction ✨

Okay so this spec extends IMAP4rev1 [RFC3501] so it can allow UTF-8 [RFC3629] in headers, the way "Internationalized Email Headers" [RFC5335] describes 🌈 It also adds a way to support mailbox names, login names, and passwords that use the UTF-8 charset. Main character energy! 💅

It creates five new IMAP capabilities so servers can advertise these new extensions. It also adds two new IMAP LIST selection options and a new IMAP LIST return option 🎀

2. Conventions Used in This Document 🧠

The key words "MUST", "MUST NOT", "SHOULD", "SHOULD NOT", and "MAY" in this document are to be interpreted as defined in "Key words for use in RFCs to Indicate Requirement Levels" [RFC2119] 💜

The formal syntax uses Augmented Backus-Naur Form (ABNF) [RFC5234] notation, including the core rules defined in Appendix B of [RFC5234]. Rules from IMAP4rev1 [RFC3501], UTF-8 [RFC3629], "Collected Extensions to IMAP4 ABNF" [RFC4466], and IMAP4 LIST Command Extensions [RFC5258] are also referenced 📚

In examples, "C:" means lines sent by the client, and "S:" means lines sent by the server 🫶 If one "C:" or "S:" label covers multiple lines, the line breaks between them are just for editing clarity. They're not part of the real protocol exchange, so don't panic about them 🙈

3. UTF8=ACCEPT IMAP Capability 🌸

The "UTF8=ACCEPT" capability says the server supports UTF-8 quoted strings, the "UTF8" parameter to SELECT and EXAMINE, and UTF-8 responses from the LIST and LSUB commands. It's the main vibe check for this whole spec 🦋

A client MUST use the ENABLE UTF8=ACCEPT command (defined in [RFC5161]) to tell the server it accepts UTF-8 quoted-strings. The ENABLE UTF8=ACCEPT command MUST only be used in the authenticated state. (Also, the "UTF8=ONLY" capability from Section 7 and the "UTF8=ALL" capability from Section 6 both imply the "UTF8=ACCEPT" capability. Check those sections for more details, bestie 👑)

3.1. IMAP UTF-8 Quoted Strings 💬

The IMAP4rev1 base spec [RFC3501] forbids 8-bit characters in atoms or quoted strings 🚫 So a UTF-8 string can only be sent as a literal. That's kind of annoying from a coding standpoint. Unless the server offers IMAP4 non-synchronizing literals [RFC2088], each UTF-8 string the client sends needs an extra round trip 😭 When the IMAP server advertises the "UTF8=ACCEPT" capability, it's telling the client it supports native UTF-8 quoted-strings with this syntax:

     string        =/ utf8-quoted

     utf8-quoted   = "*" DQUOTE *UQUOTED-CHAR DQUOTE

     UQUOTED-CHAR  = QUOTED-CHAR / UTF8-2 / UTF8-3 / UTF8-4
                ; UTF8-2, UTF8-3, and UTF8-4 are as defined in RFC 3629

When the client uses this quoting mechanism (specifically an octet sequence beginning with *" and ending with "), the server MUST reject octet sequences with the high bit set that fail to follow the formal syntax in [RFC3629] with a BAD response 🙅‍♀️

The IMAP server MUST NOT send utf8-quoted syntax to the client unless the client has signaled support for it with the ENABLE UTF8=ACCEPT command 🔒

If the server advertises "UTF8=ACCEPT", the client MAY use utf8-quoted syntax with any IMAP argument that allows a string (including astring and nstring) 🎉 But if characters outside the US-ASCII repertoire show up in a weird place, it's the same result as using any other syntactically valid but semantically invalid characters. For example, if the client puts UTF-8 characters in the user or password arguments (and the server hasn't advertised "UTF8=USER"), the LOGIN command will fail, just like it would with any other invalid user name or password 💀 The following paragraphs describe the specific cases where UTF-8 is and isn't allowed.

All IMAP servers that advertise "UTF8=ACCEPT" SHOULD accept UTF-8 in mailbox names. Servers that also support the "Mailbox International Naming Convention" described in RFC 3501, Section 5.1.3, MUST accept utf8-quoted mailbox names and convert them to the appropriate internal format 🗂️ Mailbox names MUST comply with the Net-Unicode Definition (Section 2 of [RFC5198]), with one specific exception: they MUST NOT contain control characters (0000-001F, 0080-009F), delete (007F), line separator (2028), or paragraph separator (2029) 🚫

An IMAP client MUST NOT issue a SEARCH command that mixes utf8-quoted syntax with a SEARCH CHARSET other than UTF-8. If an IMAP server gets a SEARCH command like that, it SHOULD reject the command with a BAD response, because the charset labels conflict 🙃

3.2. UTF8 Parameter to SELECT and EXAMINE 🔍

The "UTF8=ACCEPT" capability also says the server supports the "UTF8" parameter to SELECT and EXAMINE. When a mailbox is selected with the "UTF8" parameter, all IMAP commands related to message sizes, message headers, and MIME body headers change so they refer to the message with UTF-8 headers ✨ If the mailstore isn't UTF-8 header native, and the SELECT or EXAMINE command with the UTF-8 header modifier succeeds, the server MUST return results as if the mailstore were UTF-8 header native, following the upconversion requirements in Section 8. The server MAY reject the SELECT or EXAMINE command with the [NOT-UTF-8] response code, unless "UTF8=ALL" or "UTF8=ONLY" is advertised 🧚

Servers MAY include mailboxes that can only be selected or examined if the "UTF8" parameter is given. However, those mailboxes MUST NOT be included in the output of an unextended LIST, LSUB, or equivalent command 🙈 If a client tries to SELECT or EXAMINE those mailboxes without the "UTF8" parameter, the server MUST reject the command with a [UTF-8-ONLY] response code. Because of this, those mailboxes won't be reachable by IMAP clients written before this spec, and they're discouraged unless the server advertises "UTF8=ONLY" or implements IMAP4 LIST Command Extensions [RFC5258] 🥲

     utf8-select-param = "UTF8"
               ;; Conforms to <select-param> from RFC 4466

     C: a SELECT newmailbox (UTF8)
     S: ...
     S: a OK SELECT completed
     C: b FETCH 1 (SIZE ENVELOPE BODY)
     S: ... < UTF-8 header native results >
     S: b OK FETCH completed

     C: c EXAMINE legacymailbox (UTF8)
     S: c NO [NOT-UTF-8] Mailbox does not support UTF-8 access

     C: d SELECT funky-new-mailbox
     S: d NO [UTF-8-ONLY] Mailbox requires UTF-8 client

3.3. UTF-8 LIST and LSUB Responses 📋

After an IMAP client successfully issues the ENABLE UTF8=ACCEPT command, the server MUST NOT return any mailbox names in LIST results using the IMAP4 Mailbox International Naming Convention. Instead, the server MUST return any mailbox names with characters outside the US-ASCII repertoire using utf8-quoted syntax 💖

(The IMAP4 Mailbox International Naming Convention has been a bit of a mess in the past, so everyone wants this syntax to become obsolete as quickly as possible 😭🔥)

3.4. UTF-8 Interaction with IMAP4 LIST Command Extensions

Okay so when an IMAP server advertises both UTF8=ACCEPT and the "LIST-EXTENDED" [RFC5258] capability, the server MUST support the LIST extensions described in this section. No skipping, bestie 💅

3.4.1. UTF8 and UTF8ONLY LIST Selection Options

The "UTF8" LIST selection option tells the server to include mailboxes that only support UTF-8 headers in the output of the list command. ✨ The "UTF8ONLY" LIST selection option tells the server to include all mailboxes that support UTF-8 headers and to exclude mailboxes that don't support UTF-8 headers. 🫶 Note that "UTF8ONLY" implies "UTF8", so the client doesn't need to request both. Using either selection option also gives you UTF-8 mailbox names in the result, as described in Section 3.3, and it implies the "UTF8" List return option described in Section 3.4.2. Main character energy 👑

3.4.2. UTF8 LIST Return Option

If the client supplies the "UTF8" LIST return option, then the server MUST include either the "\NoUTF8" or the "\UTF8Only" mailbox attribute, as appropriate. 🎀 The "\NoUTF8" mailbox attribute means that trying to SELECT or EXAMINE that mailbox with the "UTF8" parameter will fail with a [NOT-UTF-8] response code. The "\UTF8Only" mailbox attribute means that trying to SELECT or EXAMINE that mailbox without the "UTF8" parameter will fail with a [UTF-8-ONLY] response code. ⚠️ Computing this info may be expensive on some server implementations, so this return option should not be used unless necessary. Lowkey a performance thing 💀

The ABNF [RFC5234] for these LIST extensions goes like this 🧚

list-select-independent-opt =/ "UTF8"

list-select-base-opt        =/ "UTF8ONLY"

mbx-list-oflag              =/ "\NoUTF8" / "\UTF8Only"

return-option               =/ "UTF8"

resp-text-code              =/ "NOT-UTF-8" / "UTF-8-ONLY"

4. UTF8=APPEND Capability ✨

If the "UTF8=APPEND" capability is advertised, then the server accepts UTF-8 headers in the APPEND command message argument. 💖 A client that sends a message with UTF-8 headers to the server MUST send them using the "UTF8" APPEND data extension. If the server also advertises the CATENATE capability (as specified in [RFC4469]), the client can use that same data extension to include such a message in a CATENATE message part. The ABNF for the APPEND data extension and CATENATE extension is:

utf8-literal   = "UTF8" SP "(" literal8 ")"

append-data    =/ utf8-literal

cat-part       =/ utf8-literal

A server that advertises "UTF8=APPEND" has to comply with the requirements of the IMAP base specification and [RFC5322] for message fetching. 🦋 Mechanisms for 7-bit downgrading that help with complying with the standards are discussed in Downgrading mechanism for Internationalized eMail Address (IMA) [RFC5504].

IMAP servers that do not advertise the "UTF8=APPEND" or "UTF8=ONLY" capability SHOULD reject an APPEND command that includes any 8-bit in the message headers with a "NO" response. 🙈

Note that the "UTF8=ONLY" capability described in Section 7 implies the "UTF8=APPEND" capability. See additional information in that section 🌸

5. UTF8=USER Capability 🌈

If the "UTF8=USER" capability is advertised, that means the server accepts UTF-8 user names and passwords and applies SASLprep [RFC4013] to both arguments of the LOGIN command. The server MUST reject UTF-8 that fails to comply with the formal syntax in RFC 3629 [RFC3629], or if it encounters Unicode characters listed in Section 2.3 of SASLprep RFC 4013 [RFC4013]. Strict but iconic 💅

6. UTF8=ALL Capability 🍓

The "UTF8=ALL" capability says that all server mailboxes support UTF-8 headers. Specifically, SELECT and EXAMINE with the "UTF8" parameter will never fail with a [NOT-UTF-8] response code. 🫶

Note that the "UTF8=ONLY" capability described in Section 7 implies the "UTF8=ALL" capability. See additional information in that section. ✨

Note that the "UTF8=ALL" capability implies the "UTF8=ACCEPT" capability. 💜

7. UTF8=ONLY Capability 🔥

The "UTF8=ONLY" capability lets an IMAP server advertise that it does not support the international mailbox name convention (modified UTF-7), and that it doesn't permit selection or examination of any mailbox unless the "UTF8" parameter is provided. 😭 Since this is an incompatible change to IMAP, a clear warning is necessary. IMAP clients that find implementing the "UTF8=ONLY" capability problematic are encouraged to at least detect the "UTF8=ONLY" capability and show an informative error message to the end-user. Be nice to your users, periodt 🎀

When an IMAP mailbox internally uses UTF-8 header native storage, the down-conversion step needed to permit selection or examination of the mailbox in a backwards compatible fashion will become more difficult to support. ⚠️

Although it is hoped that deployed IMAP servers will not advertise "UTF8=ONLY" for some years, this capability is meant to minimize the disruption when legacy support finally goes away. 🕊️

The "UTF8=ONLY" capability implies the "UTF8=ACCEPT" capability, the "UTF8=ALL" capability, and the "UTF8=APPEND" capability. A server that advertises "UTF8=ONLY" doesn't need to advertise the three implicit capabilities. Efficient, bestie 🧚

8. Up-Conversion Server Requirements 🦋

When an IMAP4 server uses a traditional mailbox format that includes 7-bit headers, and it chooses to permit access to that mailbox with the "UTF8" parameter, it MUST support minimal up-conversion as described in this section. 💅

The server MUST support up-conversion of the following address header-fields in the message header: From, Sender, To, CC, Bcc, Resent-From, Resent-Sender, Resent-To, Resent-CC, Resent-Bcc, and Reply-To. This up-conversion MUST include address local-parts in fields downgraded according to [RFC5504], address domains encoded according to Internationalizing Domain Names in Applications (IDNA) [RFC3490], and MIME header encoding [RFC2047] of display-names and any [RFC5322] comments. 🌸

The following charsets MUST be supported for up-conversion of MIME header encoding [RFC2047]: UTF-8, US-ASCII, ISO-8859-1, ISO-8859-2, ISO-8859-3, ISO-8859-4, ISO-8859-5, ISO-8859-6, ISO-8859-7, ISO-8859-8, ISO-8859-9, ISO-8859-10, ISO-8859-14, and ISO-8859-15. If the server supports other charsets in IMAP SEARCH or IMAP CONVERT [RFC5259], it SHOULD also support those charsets in this conversion. 🌍

Up-conversion of MIME header encoding of the following headers MUST also be implemented: Subject, Date ([RFC5322] comments only), Comments, Keywords, and Content-Description. 📝

Server implementations also SHOULD up-convert all MIME body headers [RFC2045], SHOULD up-convert or remove the deprecated (and misused) "name" parameter [RFC1341] on Content-Type, and MUST up-convert the Content-Disposition [RFC2183] "filename" parameter, except when any of these are contained within a multipart/signed MIME body part (see below). These parameters can be encoded using the standard MIME parameter encoding [RFC2231] mechanism, or via non-standard use of MIME header encoding [RFC2047] in quoted strings. 🎀

The IMAP server MUST NOT perform up-conversion of headers and content of multipart/signed, as well as Original-Recipient and Return-Path. 🚫

9. Issues with UTF-8 Header Mailstore 🍓

When an IMAP server uses a mailbox format that supports UTF-8 headers, and it permits selection or examination of that mailbox without the "UTF8" parameter, it is the server's responsibility to comply with the IMAP4rev1 base specification [RFC3501] and [RFC5322] with respect to all header information transmitted over the wire. 🫶 Mechanisms for 7-bit downgrading that help with complying with the standards are discussed in "Downgrading Mechanism for Email Address Internationalization" [RFC5504].

An IMAP server with a mailbox that supports UTF-8 headers MUST comply with the protocol requirements implicit from Section 8. 💖 However, the code needed for that compliance doesn't have to be part of the IMAP server itself in this case. For example, the minimal required up-conversion could be performed when a message is inserted into the IMAP-accessible mailbox. Smart and sneaky 🧚

10. IANA Considerations ✨

This adds five new capabilities ("UTF8=ACCEPT", "UTF8=USER", "UTF8=APPEND", "UTF8=ALL", and "UTF8=ONLY") to the IMAP4rev1 Capabilities registry [RFC3501].

This adds two new IMAP4 list selection options and one new IMAP4 list return option. 📋

1.  LIST-EXTENDED option name: UTF8

    LIST-EXTENDED option type: SELECTION

    Implied return options(s): UTF8

    LIST-EXTENDED option description: Causes the LIST response to
    include mailboxes that mandate the UTF8 SELECT/EXAMINE parameter.

    Published specification: RFC 5738, Section 3.4.1

    Security considerations: RFC 5738, Section 11

    Intended usage: COMMON

    Person and email address to contact for further information: see
    the Authors' Addresses at the end of this specification

    Owner/Change controller: iesg@ietf.org

2.  LIST-EXTENDED option name: UTF8ONLY

    LIST-EXTENDED option type: SELECTION

    Implied return options(s): UTF8

    LIST-EXTENDED option description: Causes the LIST response to
    include mailboxes that mandate the UTF8 SELECT/EXAMINE parameter
    and exclude mailboxes that do not support the UTF8 SELECT/EXAMINE
    parameter.

    Published specification: RFC 5738, Section 3.4.1

    Security considerations: RFC 5738, Section 11

    Intended usage: COMMON

    Person and email address to contact for further information: see
    the Authors' Addresses at the end of this specification

    Owner/Change controller: iesg@ietf.org

3.  LIST-EXTENDED option name: UTF8

    LIST-EXTENDED option type: RETURN

    Implied return options(s): none

    LIST-EXTENDED option description: Causes the LIST response to
    include \NoUTF8 and \UTF8Only mailbox attributes.

    Published specification: RFC 5738, Section 3.4.1

    Security considerations: RFC 5738, Section 11

    Intended usage: COMMON

    Person and email address to contact for further information: see
    the Authors' Addresses at the end of this specification

    Owner/Change controller: iesg@ietf.org

11. Security Considerations 🔐

The security considerations of UTF-8 [RFC3629] and SASLprep [RFC4013] apply to this specification, particularly with respect to use of UTF-8 in user names and passwords. Otherwise, this is not believed to alter the security considerations of IMAP4rev1. 😭🔐

12. References 📚

The reference list is coming up in the next part, so stay tuned 🧚

12.1. Normative References

okay bestie, these are the docs the spec leans on so hard it can't function without them 📚✨

  • [RFC1341] Borenstein, N. and N. Freed, "MIME (Multipurpose Internet Mail Extensions): Mechanisms for Specifying and Describing the Format of Internet Message Bodies", RFC 1341, June 1992.
  • [RFC2045] Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part One: Format of Internet Message Bodies", RFC 2045, November 1996.
  • [RFC2047] Moore, K., "MIME (Multipurpose Internet Mail Extensions) Part Three: Message Header Extensions for Non-ASCII Text", RFC 2047, November 1996.
  • [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
  • [RFC2183] Troost, R., Dorner, S., and K. Moore, "Communicating Presentation Information in Internet Messages: The Content-Disposition Header Field", RFC 2183, August 1997.
  • [RFC2231] Freed, N. and K. Moore, "MIME Parameter Value and Encoded Word Extensions: Character Sets, Languages, and Continuations", RFC 2231, November 1997.
  • [RFC3490] Faltstrom, P., Hoffman, P., and A. Costello, "Internationalizing Domain Names in Applications (IDNA)", RFC 3490, March 2003.
  • [RFC3501] Crispin, M., "INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1", RFC 3501, March 2003.
  • [RFC3629] Yergeau, F., "UTF-8, a transformation format of ISO 10646", STD 63, RFC 3629, November 2003.
  • [RFC4013] Zeilenga, K., "SASLprep: Stringprep Profile for User Names and Passwords", RFC 4013, February 2005.
  • [RFC4466] Melnikov, A. and C. Daboo, "Collected Extensions to IMAP4 ABNF", RFC 4466, April 2006.
  • [RFC4469] Resnick, P., "Internet Message Access Protocol (IMAP) CATENATE Extension", RFC 4469, April 2006.
  • [RFC5161] Gulbrandsen, A. and A. Melnikov, "The IMAP ENABLE Extension", RFC 5161, March 2008.
  • [RFC5198] Klensin, J. and M. Padlipsky, "Unicode Format for Network Interchange", RFC 5198, March 2008.
  • [RFC5234] Crocker, D. and P. Overell, "Augmented BNF for Syntax Specifications: ABNF", STD 68, RFC 5234, January 2008.
  • [RFC5258] Leiba, B. and A. Melnikov, "Internet Message Access Protocol version 4 - LIST Command Extensions", RFC 5258, June 2008.
  • [RFC5259] Melnikov, A. and P. Coates, "Internet Message Access Protocol - CONVERT Extension", RFC 5259, July 2008.
  • [RFC5322] Resnick, P., Ed., "Internet Message Format", RFC 5322, October 2008.
  • [RFC5335] Abel, Y., "Internationalized Email Headers", RFC 5335, September 2008.
  • [RFC5504] Fujiwara, K. and Y. Yoneya, "Downgrading Mechanism for Email Address Internationalization", RFC 5504, March 2009.

12.2. Informative References

these ones are just good to know background reading 🌸

  • [RFC2049] Freed, N. and N. Borenstein, "Multipurpose Internet Mail Extensions (MIME) Part Five: Conformance Criteria and Examples", RFC 2049, November 1996.
  • [RFC2088] Myers, J., "IMAP4 non-synchronizing literals", RFC 2088, January 1997.
  • [RFC2277] Alvestrand, H., "IETF Policy on Character Sets and Languages", BCP 18, RFC 2277, January 1998.
  • [RFC5721] Gellens, R. and C. Newman, "POP3 Support for UTF-8", RFC 5721, February 2010.

Appendix A. Design Rationale

this non-normative section is the big "why" behind some of the design choices above, and honestly the reasoning is so good 🧠💖

the whole idea of advertising that you can reach a mailbox in UTF-8 mode is meant to allow a graceful upgrade, even for servers juggling multiple mailbox formats. nobody wants to force a server's entire mailstore to convert to UTF-8 headers all at once 🙅‍♀️. so phasing in support for new mailboxes and slowly migrating the old ones is totally allowed by this design 🦋

"UTF8=USER" is optional because lots of identity systems are US-ASCII only, so it's helpful to tell the client up front that UTF-8 won't work. no surprises, no vibes-based debugging 💅

"UTF8=APPEND" is optional because it effectively requires IMAP server support for down-conversion, which is WAY more complicated than up-conversion 😭

the "UTF8=ONLY" mechanism makes it easier to diagnose interoperability problems once legacy support goes away 🔍. when backwards compatibility is broken anyway, just-send-UTF-8 IMAP has the upside that it might still work with some legacy clients. however, diagnosing the interoperability problems that a just-send-UTF-8 IMAP mechanism can cause is so hard that it's the reason the "UTF8=ONLY" capability mechanism was chosen instead 🎀

the up-conversion requirements are built to balance two things: getting rid of complicated encodings (like MIME header encodings) over time, without dumping a huge deployment burden on servers 🫶. since IMAP4 servers already need a MIME parser, this brings in extra server up-conversion requirements that aren't in POP3 Support for UTF-8 [RFC5721] 🌈

the set of mandatory charsets comes from two places: MIME requirements [RFC2049] and IETF Policy on Character Sets [RFC2277] 📚. requiring servers to up-convert widely deployed encoded ideographic charsets to UTF-8 would be reasonable for most scenarios, but some embedded devices might need unacceptably large tables 😬. so the open-ended recommendation to support widely deployed charsets sidesteps the political drama of trying to list those charsets out. the authors believe market forces, existing open-source software, and public conversion tables are enough to get the right charsets deployed ✨

Appendix B. Examples Demonstrating Relationships between UTF8=Capabilities

here's how the UTF8= capabilities combine, and the notes tell you which combos are just the same thing with a different spelling 🧚

     UTF8=ACCEPT UTF8=USER UTF8=APPEND
     UTF8=ACCEPT UTF8=ALL
     UTF8=ALL       ; Note, same as above
     UTF8=ACCEPT UTF8=USER UTF8=APPEND UTF8=ALL UTF8=ONLY
     UTF8=USER UTF8=ONLY ; Note, same as above

Appendix C. Acknowledgments

the authors want to thank the EAI working group participants for their contributions to this document 🫶. special thanks go to Harald Alvestrand, David Black, Randall Gellens, Arnt Gulbrandsen, Kari Hurtta, John Klensin, Xiaodong Lee, Charles Lindsey, Alexey Melnikov, Subramanian Moonesamy, Shawn Steele, Daniel Taharlev, and Joseph Yee for their specific contributions to the discussion 👑

Authors' Addresses

and here are the people who wrote it, with their contact details 💌

Pete Resnick
Qualcomm Incorporated
5775 Morehouse Drive
San Diego, CA  92121-1714
US

Phone: +1 858 651 4478
EMail: presnick@qualcomm.com
URI:   http://www.qualcomm.com/~presnick/

Chris Newman
Sun Microsystems
800 Royal Oaks
Monrovia, CA  91016
US

EMail: chris.newman@sun.com
📦 where this page comes from