RFC 5721

POP3 Support for UTF-8

📅 published February 2010, from ietf

RFC 5721: POP3 Support for UTF-8 💌✨

Title block, the tea 🫖💅

Who wrote it: R. Gellens (QUALCOMM Incorporated) and C. Newman (Sun Microsystems) 👯
Category: Experimental 🧪 (so it's not a Standards Track spec, just something people try out and evaluate)
Stream and ISSN: IETF stream, ISSN 2070-1721 🎀
Date: February 2010 📅
Obsoletes: no Obsoletes header 🙅‍♀️
Updates: POP3 [RFC1939] 🔄 (more on that drama in Section 1)

Abstract 🌸

okay so this spec glow-ups POP3 (the Post Office Protocol, version 3) so it can handle un-encoded international characters in user names, passwords, mail addresses, message headers, and protocol-level text error strings. it's giving global. 🌍💖

Status of This Memo 📝

this doc is not an Internet Standards Track spec. it's published so people can examine it, implement it experimentally, and evaluate it. 🔬

it defines an Experimental Protocol for the Internet community. it's a product of the Internet Engineering Task Force (IETF) and represents the consensus of the IETF community. it got public review and was approved for publication by the Internet Engineering Steering Group (IESG). not every doc the IESG approves is a candidate for any level of Internet Standard, so see Section 2 of RFC 5741 for that. ✨

for the current status, any errata, and how to give feedback, check http://www.rfc-editor.org/info/rfc5721. 🔗

Copyright Notice ©️

this document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as in effect on the date of publication. please read these carefully, because they describe your rights and restrictions for this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they're provided without warranty as described in the Simplified BSD License. 📜

this document may contain material from IETF Documents or IETF Contributions published or made publicly available before November 10, 2008. the people controlling the copyright in some of that material may not have granted the IETF Trust the right to allow modifications outside the IETF Standards Process. so unless you get an adequate license from those copyright holders, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process. the exceptions are formatting it for publication as an RFC or translating it into languages other than English. 🙃

Table of Contents 🗂️

1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  3
  1.1.  Conventions Used in This Document  . . . . . . . . . . . .  3
2.  LANG Capability  . . . . . . . . . . . . . . . . . . . . . . .  4
3.  UTF8 Capability  . . . . . . . . . . . . . . . . . . . . . . .  6
  3.1.  The UTF8 Command . . . . . . . . . . . . . . . . . . . . .  7
  3.2.  USER Argument to UTF8 Capability . . . . . . . . . . . . .  9
4.  Native UTF-8 Maildrops . . . . . . . . . . . . . . . . . . . .  9
5.  IANA Considerations  . . . . . . . . . . . . . . . . . . . . . 10
6.  Security Considerations  . . . . . . . . . . . . . . . . . . . 10
7.  References . . . . . . . . . . . . . . . . . . . . . . . . . . 10
  7.1.  Normative References . . . . . . . . . . . . . . . . . . . 10
  7.2.  Informative References . . . . . . . . . . . . . . . . . . 11
Appendix A.  Design Rationale  . . . . . . . . . . . . . . . . . . 12
Appendix B.  Acknowledgments . . . . . . . . . . . . . . . . . . . 12

1. Introduction 🦋

this doc is part of the Email Address Internationalization (EAI) experiment described in the EAI Framework document [RFC4952]. for background, check out the charter of the EAI working group. it should be evaluated in the context of EAI, bestie. 🧠 as part of the overall EAI work, email messages may be sent and delivered with un-encoded UTF-8 characters, and mail drops accessed using POP3 [RFC1939] might natively store UTF-8. 💅

this spec extends POP3 [RFC1939] using the POP3 extension mechanism [RFC2449] to permit un-encoded UTF-8 [RFC3629] in headers, as described in "Internationalized Email Headers" [RFC5335]. it also adds a mechanism for login names and passwords outside the ASCII character set, plus a mechanism for UTF-8 protocol-level error strings in a language that fits the user. 🌏

okay now the drama 👀 this doc updates POP3 [RFC1939]. an Experimental spec updating a Standards Track spec means that people taking part in the experiment have to consider the Standard updated. to avoid confusion, this Experimental doc does not have an "Updates" header. if and when a version of this doc moves to the Standards Track, an "Updates: 1939" header should be added. 🫠

some vocab: "down-conversion" refers to the process of modifying a message that has UTF8 headers [RFC5335] or body parts with 8bit content-transfer-encoding (as defined in MIME Section 2.8 [RFC2045]) into a conforming 7-bit Internet Message Format [RFC5322]. that conversion uses message header extensions for non-ASCII text [RFC2047] and other 7-bit encodings. down-conversion is specified by "Downgrading Mechanism for Email Address Internationalization" [RFC5504]. 🔁

1.1. Conventions Used in This Document 📚

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in "Key words for use in RFCs to Indicate Requirement Levels" [RFC2119].

in the examples, "C:" and "S:" mean lines sent by the client and server. if a single "C:" or "S:" label applies to multiple lines, the line breaks between those lines are just for editorial clarity and are not part of the actual protocol exchange. 🫶

note that examples always use 7-bit ASCII characters because of limitations of this document format. some examples for the "LANG" command may look silly as a result, sorry not sorry. 🙈

2. LANG Capability 🌐

per "POP3 Extension Mechanism" [RFC2449], this doc adds a new capability response tag to indicate support for a new command: LANG. here's the tag and the new command. 🍓

CAPA tag:

LANG

Arguments with CAPA tag:

none

Added Commands:

LANG

Standard commands affected:

All

Announced states / possible differences:

both / no

Commands valid in states:

AUTHENTICATION, TRANSACTION

Specification reference:

this document

Discussion:

POP3 lets most +OK and -ERR server responses include human-readable text that might be shown to the user. but the POP3 spec [RFC1939] limits that text to ASCII. ugh. 😤 the LANG capability and command let a POP3 client negotiate which language the server uses for that human-readable text. ✨

a server that advertises the LANG extension MUST use the language "i-default" as described in [RFC2277] as its default language until the client negotiates another supported language. a server MUST include "i-default" as one of its supported languages. 👑

the LANG command asks that human-readable text in all subsequent +OK and -ERR responses be localized to a language matching the language range argument (the "Basic Language Range" as described by [RFC4647]). if the command succeeds, the server returns a +OK response, then a single space, then the exact language tag that was selected, then another space, and the rest of the line is human-readable text in that language. this text, and all later protocol-level human-readable text, is encoded in the UTF-8 charset. 🌸

if the command fails, the server returns an -ERR response, and later human-readable response text keeps using whatever language was active before (usually i-default). 🙃

the special "*" language range argument means "use the language the server admin prefers." the preferred language MAY vary based on the currently active user. 🎀

if there's no argument and the POP3 server issues a positive response, the response is multi-line. after the initial +OK, the POP3 server responds with one line for each language tag it supports. that line is called a "language listing". 📋

to keep parsing simple, all POP3 servers are required to use a certain format for language listings. a language listing consists of the language tag [RFC5646] of the message, optionally followed by a single space and a human-readable description of the language, written in that language, using the UTF-8 charset. 🧠

Examples:

< Note that some examples do not include the correct character
accents due to limitations of this document format. >

< The server defaults to using English i-default responses until
the client explicitly changes the language. >

C: USER karen
S: +OK Hello, karen
C: PASS password
S: +OK karen's maildrop contains 2 messages (320 octets)

< Client requests deprecated MUL language.  Server replies
with -ERR response. >

C: LANG MUL
S: -ERR invalid language MUL

< A LANG command with no parameters is a request for
a language listing. >
C: LANG
S: +OK Language listing follows:
S: en English
S: en-boont English Boontling dialect
S: de Deutsch
S: it Italiano
S: es Espanol
S: sv Svenska
S: i-default Default language
S: .

< A request for a language listing might fail. >

C: LANG
S: -ERR Server is unable to list languages

< Once the client changes the language, all responses will be in
that language, starting with the response to the LANG command. >

C: LANG es
S: +OK es Idioma cambiado

< If a server does not support the requested primary language,
responses will continue to be returned in the current language
the server is using. >

C: LANG uga
S: -ERR es Idioma <<UGA>> no es conocido

C: LANG sv
S: +OK sv Kommandot "LANG" lyckades

C: LANG *
S: +OK es Idioma cambiado

3. UTF8 Capability 🦄

per "POP3 Extension Mechanism" [RFC2449], this doc adds a new capability response tag to indicate support for new server functionality, including a new command: UTF8. here's the tag, the new command, and the new functionality. 💜

CAPA tag:

UTF8

Arguments with CAPA tag:

USER

Added Commands:

UTF8

Standard commands affected:

USER, PASS, APOP, LIST, TOP, RETR

Announced states / possible differences:

both / no

Commands valid in states:

AUTHORIZATION

Specification reference:

this document

Discussion:

this capability adds the "UTF8" command to POP3. the UTF8 command switches the session from ASCII mode to UTF-8 mode. the main character energy of this whole spec 🌈✨

3.1. The UTF8 Command 💖

The UTF8 command turns on UTF-8 mode 🫶 It takes no parameters, so you just send it and go.

Maildrops can natively store UTF-8 or be limited to ASCII 🦋 UTF-8 mode has no effect on messages in an ASCII-only maildrop. Messages in native UTF-8 maildrops can be ASCII or UTF-8, using internationalized headers [RFC5335] and/or 8bit content-transfer-encoding, as defined in MIME Section 2.8 [RFC2045]. In UTF-8 mode, both UTF-8 and ASCII messages are sent to the client as-is, with no conversion. When not in UTF-8 mode, UTF-8 messages in a native UTF-8 maildrop MUST be down-converted (downgraded) to comply with unextended POP and Internet Mail Format. POP servers (unlike SMTP and Submit servers) are not required to use "Downgrading Mechanism for Email Address Internationalization" [RFC5504].

Discussion: The main argument against a single required downgrading mechanism for a POP server is that the only clients with any use for a standardized downgraded message (because they want to interpret downgrade headers, for example) are the ones that can support UTF-8 and, hence, will issue the UTF8 command in the first place. The counter argument is that clients which don't support UTF-8 might get upgraded in the future 🌸 It's desirable for an upgraded client to be able to interpret earlier downgraded messages in the local mail store, and that's most likely if the messages were downgraded using one standardized procedure. ✨

Therefore, while POP servers are not required to use "Downgrading Mechanism for Email Address Internationalization" [RFC5504], there are advantages to them doing so. 💅

Note that even in UTF-8 mode, MIME binary content-transfer-encoding is still not permitted. 🙈

The octet count (size) of a message reported in a response to the LIST command SHOULD match the actual number of octets sent in a RETR response (not counting byte-stuffing). Sizes reported elsewhere, such as in STAT responses and in non-standardized, free-form text in positive status indicators (following +OK), need not be accurate, but it is preferable if they are. 🍓

Discussion: Mail stores are either ASCII or native UTF-8, and clients either issue the UTF8 command or they don't. The message only needs converting when it is native UTF-8 and the client has not issued the UTF8 command, and in that case the server must down-convert it. The down-converted message may be larger. The server can pick from various strategies for down-conversion, including when to down-convert, whether to cache or store the down-converted form of a message (and if so, for how long), and whether to calculate or retain the size of a down-converted message independently of the down-converted content. If the server doesn't have immediate access to the accurate down-converted size, it may be faster to estimate it than to calculate it. 🧚 Servers are expected to normally follow the RFC 1939 [RFC1939] text on using the "exact size" in a scan listing, but there may be situations with maildrops containing very large numbers of messages where this might be a problem. If the server does estimate, reporting a scan listing size smaller than what the message turns out to be could be a problem for some clients 😭 In summary, it's better for servers to report accurate sizes, but if that isn't possible, high guesses are better than small ones. 🎀

Some POP servers include the message size in the non-standardized text response following +OK (the text production of RFC 2449 [RFC2449]), in a RETR or TOP response (possibly because some examples in POP3 [RFC1939] do so). There has been at least one known case of a client relying on this to know when it had received the whole message, rather than following the POP3 [RFC1939] rule of looking for a line consisting of a termination octet (".") and a CRLF pair. Any such client is non-compliant, but if a server does include the size in such text, it's better if the size is accurate. 💀

Clients MUST NOT issue the STLS command [RFC2595] after issuing UTF8. Servers MAY (but are not required to) enforce this by rejecting an STLS command issued after a successful UTF8 command with a "-ERR" response. (Because this is a protocol error rather than a failure based on conditions, an extended response code [RFC2449] is not specified.) 🔥

3.2. USER Argument to UTF8 Capability 👑

If the USER argument is included with this capability, it means the server accepts UTF-8 user names and passwords. 🌈

Servers that include the USER argument in the UTF8 capability response SHOULD apply SASLprep [RFC4013] to the arguments of the USER and PASS commands. ⭐

A client or server that supports APOP and permits UTF-8 in user names or passwords MUST apply SASLprep [RFC4013] to the user name and password used to compute the APOP digest. 🫶

When applying SASLprep [RFC4013], servers MUST reject UTF-8 user names or passwords that contain a Unicode character listed in Section 2.3 of SASLprep [RFC4013]. When applying SASLprep to the USER argument, the PASS argument, or the APOP username argument, a compliant server or client MUST treat them as a query string (i.e., unassigned Unicode codepoints are allowed). When applying SASLprep to the APOP password argument, a compliant server or client MUST treat it as a stored string (i.e., unassigned Unicode codepoints are prohibited). 🧐

The client does not need to issue the UTF8 command before using UTF-8 in authentication. However, clients MUST NOT use UTF-8 in USER, PASS, or APOP commands unless the USER argument is included in the UTF8 capability response. 🚫

The server MUST reject UTF-8 user names or passwords that fail to comply with the formal syntax in UTF-8 [RFC3629]. 🎀

Use of UTF-8 in the AUTH command is governed by the POP3 SASL [RFC5034] mechanism. 🦋

4. Native UTF-8 Maildrops ✨

When a POP3 server uses a native UTF-8 maildrop, it is the server's responsibility to comply with the POP3 base specification [RFC1939] and Internet Message Format [RFC5322] when not in UTF-8 mode. Mechanisms for 7-bit downgrading that help comply with the standards are described in "Downgrading Mechanism for Email Address Internationalization" [RFC5504]. 💖

5. IANA Considerations 📋

This specification adds two new capabilities ("UTF8" and "LANG") to the POP3 capability registry [RFC2449]. 🌸

6. Security Considerations 🚨

The security considerations of UTF-8 [RFC3629] and SASLprep [RFC4013] apply to this specification, particularly with respect to use of UTF-8 in user names and passwords. 🔐

The LANG * command might reveal the existence and preferred language of a user to an active attacker probing the system, if the active language changes in response to the USER, PASS, or APOP commands before the user's credentials are validated. Servers MUST implement a configuration to prevent this exposure. 😱

It is possible for a man-in-the-middle attacker to insert a LANG command into the command stream, making protocol-level diagnostic responses unintelligible to the user. A mechanism to integrity-protect the session, such as Transport Layer Security (TLS) [RFC2595], can be used to defeat such attacks. 🛡️

Modifying server authentication code (in this case, to support UTF-8) needs to be done with care to avoid introducing vulnerabilities (for example, in string parsing). 🧨

The UTF8 command description (Section 3.1) contains a discussion on reporting inaccurate sizes. An additional risk of doing so is that, if a client allocates buffers based on the reported size, it may overrun the buffer, crash, or have other problems if the message data is larger than reported. 💀🙈

7. References 📚

7.1. Normative References 🦋

[RFC1939]  Myers, J. and M. Rose, "Post Office Protocol - Version 3",
           STD 53, RFC 1939, May 1996.

[RFC2045]  Freed, N. and N. Borenstein, "Multipurpose Internet Mail
           Extensions (MIME) Part One: Format of Internet Message
           Bodies", RFC 2045, November 1996.

[RFC2047]  Moore, K., "MIME (Multipurpose Internet Mail Extensions)
           Part Three: Message Header Extensions for Non-ASCII Text",
           RFC 2047, November 1996.

[RFC2119]  Bradner, S., "Key words for use in RFCs to Indicate
           Requirement Levels", BCP 14, RFC 2119, March 1997.

[RFC2277]  Alvestrand, H., "IETF Policy on Character Sets and
           Languages", BCP 18, RFC 2277, January 1998.

[RFC2449]  Gellens, R., Newman, C., and L. Lundblade, "POP3 Extension
           Mechanism", RFC 2449, November 1998.

[RFC3629]  Yergeau, F., "UTF-8, a transformation format of ISO
           10646", STD 63, RFC 3629, November 2003.

[RFC4013]  Zeilenga, K., "SASLprep: Stringprep Profile for User Names
           and Passwords", RFC 4013, February 2005.

[RFC4647]  Phillips, A. and M. Davis, "Matching of Language Tags",
           BCP 47, RFC 4647, September 2006.

[RFC5322]  Resnick, P., Ed., "Internet Message Format", RFC 5322,
           October 2008.

[RFC5335]  Abel, Y., "Internationalized Email Headers", RFC 5335,
           September 2008.

[RFC5646]  Phillips, A. and M. Davis, "Tags for Identifying
           Languages", BCP 47, RFC 5646, September 2009.

7.2. Informative References 🧚

[RFC2595]  Newman, C., "Using TLS with IMAP, POP3 and ACAP",
           RFC 2595, June 1999.

[RFC4952]  Klensin, J. and Y. Ko, "Overview and Framework for
           Internationalized Email", RFC 4952, July 2007.

[RFC5034]  Siemborski, R. and A. Menon-Sen, "The Post Office Protocol
           (POP3) Simple Authentication and Security Layer (SASL)
           Authentication Mechanism", RFC 5034, July 2007.

[RFC5504]  Fujiwara, K. and Y. Yoneya, "Downgrading Mechanism for
           Email Address Internationalization", RFC 5504, March 2009.

Appendix A. Design Rationale 🎀

okay bestie, this part is non-normative, which just means it's the "here's WHY we did it" bit and not a rule you have to follow. 🧚 It's giving behind-the-scenes gossip. 💅

Servers doing up-conversion so that, at a minimum, RFC2047-encoded words get decoded into UTF-8 sounded super tempting ✨ because clients keep messing this up. 😭 But after a LOT of discussion, the EAI group decided the benefits didn't justify the burden. 💀

Interoperability problems with RFC 2047 and limited deployment of RFC 2231 mean we're hoping these 7-bit encoding mechanisms can get deprecated someday, once UTF-8 header support is everywhere. 🌈 fr fr we're manifesting it 🫶

USER is optional because nobody really understands the implementation burden of SASLprep [RFC4013] yet 🙈 and forcing it on everyone could seriously mess up deployment. not worth the drama rn 💅

Also, it's hard to come up with good examples for language negotiation without non-ASCII characters, but it's even harder to give examples for commands that use the UTF-8 charset un-encoded when the document format is limited to ASCII. 💜 So there are no plans to add examples for that part while this is still an experimental proposal. BUT implementers are encouraged to send examples to the document authors for a future revision. 🦋 we love a community contribution moment 🍓

Last one, and it's a good one 👑: down-conversion of native UTF-8 messages is mandatory when the UTF8 command isn't in use, but servers are not required to use "Downgrading Mechanism for Email Address Internationalization" [RFC5504] to do it. 🫠 As clients get UTF-8 support and get better at handling (like displaying and replying to) UTF-8 messages that got downgraded in transit, it's ideal if they can also handle messages in the local mail store that the POP server downgraded. 🌸 That's more likely when the POP server downgrades messages with the same mechanism an SMTP server uses. iykyk 😌

Appendix B. Acknowledgments 🌷

Big thank-yous to John Klensin, Tony Hansen, and the other EAI working group participants 🫶 who gave helpful suggestions and super interesting debate that made this spec way better. ✨ the group chat was clearly bringing the receipts 💅

Authors' Addresses 🦋

Randall Gellens
QUALCOMM Incorporated
5775 Morehouse Drive
San Diego, CA  92651
US

EMail: rg+ietf@qualcomm.com


Chris Newman
Sun Microsystems
800 Royal Oaks
Monrovia, CA  91016-6347
US

EMail: chis.newman@sun.com

📦 where this page comes from

ietf: rfc5721 2010-02
💖 girlified by claude-haiku-5-5 on 2026-10-17: a language model's rewrite of the original, under its terms