RFC 1522

RFC 1522: MIME (Multipurpose Internet Mail Extensions) Part Two: Message Header Extensions for Non-ASCII Text 🎀

Okay bestie, the deets first 💅 This one's from the Network Working Group, written by K. Moore at the University of Tennessee, dated September 1993. It's a Standards Track doc, and it obsoletes RFC 1342 ✨ (there's no Updates entry in its header 🙈).

---

Status of this Memo 📜

This RFC is a standards track protocol for the Internet community, and it's literally asking for discussion and suggestions for improvements 🫶 To check where it stands (standardization state and status), look at the current edition of the "Internet Official Protocol Standards." And the distribution of this memo is unlimited 🌈

Abstract 💭

So this memo adds an extension to the message format from RFC 1521 [1], so that character sets other than ASCII can show up in RFC 822 (STD 11) message headers 🦋 It was built to be super compatible with the mail software that already exists, and easy for mail readers that support RFC 1521 to implement.

1. Introduction ✨

RFC 1521 is the one that explains how to tag textual body parts with different character sets, and how to encode those body parts into plain printable ASCII 💖 This memo does the same kind of thing, but for the headers of an RFC 822 [2] message, so non-ASCII text can fit in there without confusing the software that's already out in the wild.

Honestly, the whole vibe here is protecting existing mail software from chaos 🙈 Like the encoding in RFC 1521, these techniques let you use non-ASCII characters in headers in a way that the quirks of current Internet mail programs won't mess up. Some mail relays are known to (a) delete some header fields while keeping others, (b) rearrange the order of addresses in To or Cc fields, (c) rearrange the (vertical) order of header fields, and/or (d) "wrap" headers at different places than the original message. Some mail readers also have trouble parsing headers that are legal under RFC 822 but use backslash-quoting to "hide" special characters like <, ,, or :, or that use other rarely used features of that spec 😭

It's sad that these programs don't parse RFC 822 headers properly, but breaking them would cause severe operational problems for the Internet mail system 💀 So the extensions in this memo don't rely on the little-used features of RFC 822. 🧚

Instead, certain sequences of "ordinary" printable ASCII characters (called "encoded-words") are reserved for encoded data 🍓 The syntax is set up so they're unlikely to "accidentally" show up as normal text in headers, and the characters inside encoded-words are limited to ones that don't have special meanings in the spot where the encoded-word appears.

Generally, an "encoded-word" is a run of printable ASCII characters that starts with =?, ends with ?=, and has two ?s in between. It names a character set and an encoding method, and it includes the original text encoded as graphic ASCII characters, following the rules of that encoding method ✨

A mail composer that implements this spec will let you type non-ASCII text into header fields, then translate those fields (or the right parts of them) into encoded-words before putting them into the message header 💅 A mail reader that implements it will spot encoded-words in certain parts of the header. Instead of showing the encoded-word "as is," it reverses the encoding and displays the original text in the designated character set 🦋

NOTES 📝

This memo leans super hard on notation and terms from STD 11, RFC 822 and RFC 1521 🤓 The syntax for the ABNF in this memo is defined in STD 11, RFC 822, and so are a lot of the terms used in the grammar for the header extensions here. Implementing this extension successfully means paying really careful attention to the details of both STD 11, RFC 822 and RFC 1521.

When this memo says "ASCII," it means the "7-Bit American Standard Code for Information Interchange," ANSI X3.4-1986. The MIME charset name for it is "US-ASCII." When it's not specifically talking about the MIME charset name, the document just says "ASCII," for brevity and to match STD 11, RFC 822 🌸 But implementors are warned: the character set name must be spelled US-ASCII in MIME message and body part headers. No typos, bestie 🙈

2. Syntax of encoded-words 🧩

An "encoded-word" is defined by the following ABNF grammar. It uses RFC 822 notation, with one exception: white space characters MAY NOT appear between components of an encoded-word 💖

   encoded-word = "=?" charset "?" encoding "?" encoded-text "?="

   charset = token    ; see section 3

   encoding = token   ; see section 4

   token = 1*<Any CHAR except SPACE, CTLs, and especials>

   especials = "(" / ")" / "<" / ">" / "@" / "," / ";" / ":" / "
               <"> / "/" / "[" / "]" / "?" / "." / "="

   encoded-text = 1*<Any printable ASCII character other
                     than "?" or SPACE>
                     ; (but see "Use of encoded-words in message
                     ; headers", section 5)

Both "encoding" and "charset" names are case-independent 🌈 So the charset name ISO-8859-1 is the same as iso-8859-1, and the encoding named "Q" can be spelled either "Q" or "q."

An encoded-word may not be more than 75 characters long, counting the charset, encoding, encoded-text, and delimiters 📏 If you need to encode more text than fits in one 75-character encoded-word, you can use multiple encoded-words separated by CRLF SPACE 🦋

There's no limit on how long a multiple-line header field can be, but each line of a header field that contains one or more encoded-words is limited to 76 characters 💅

The length restrictions aren't only there to make things easier for internetwork mail gateways. They also put a cap on how much lookahead a header parser has to do (while hunting for a final ?= delimiter) before it can decide whether a token is an encoded-word or something else 🧠

The characters that can show up in encoded-text get further restricted by the rules in section 5 🔥

3. Character sets 🔤

The "charset" part of an encoded-word names the character set for the unencoded text 💖 A charset can be any of the character set names allowed in the RFC 1521 "charset" parameter of a text/plain body part, or any character set name registered with IANA for use with the MIME text/plain content-type [3]. (See section 7.1.1 of RFC 1521 for a list of the charsets defined in that document.)

Some character sets use code-switching to flip between "ASCII mode" and other modes 🔀 If the unencoded text in an encoded-word has control codes that switch out of ASCII mode, it must also contain additional control codes so that ASCII mode is selected again at the end of the encoded-word. This rule applies separately to each encoded-word, including adjacent encoded-words within a single header field 🫶

When there's a chance of using more than one character set to represent the text in an encoded-word, and there are no private agreements between the sender and recipients of a message, it is recommended that members of the ISO-8859-* series be used in preference to other character sets. 🌸

4. Encodings 💅

Initially, the legal values for "encoding" are "Q" and "B." These encodings are described below. The "Q" encoding is recommended for use when most of the characters to be encoded are in the ASCII character set; otherwise, the "B" encoding should be used. Still, a mail reader which claims to recognize encoded-words MUST be able to accept either encoding for any character set it supports 🙌

Only a subset of the printable ASCII characters can be used in encoded-text 🎀 Space and tab characters aren't allowed, so the beginning and end of an encoded-word are obvious. The "?" character separates the different portions of an encoded-word from one another, so it can't appear in the encoded-text portion. Other characters are also illegal in certain contexts. For example, an encoded-word in a "phrase" before an address in a From header field may not contain any of the "specials" defined in RFC 822. And finally, certain other characters are disallowed in some contexts, so messages stay reliable when they pass through internetwork mail gateways 🚦

The "B" encoding automatically meets these requirements 💖 The "Q" encoding lets a wide range of printable characters be used in non-critical spots in the header (like Subject), with fewer characters available for other locations 🦋

4.1. The "B" encoding 🫧

The "B" encoding is identical to the "BASE64" encoding defined by RFC 1521 ✨

4.2. The "Q" encoding 🎀

The "Q" encoding is similar to the "Quoted-Printable" content-transfer-encoding defined in RFC 1521. It's designed so that text that's mostly ASCII can be read on an ASCII terminal without decoding first 👀

(1) Any 8-bit value can be represented by a = followed by two hexadecimal digits. For example, if the character set in use were ISO-8859-1, the = character would be encoded as =3D, and a SPACE as =20. (Upper case should be used for the hexadecimal digits "A" through "F.") 🔢

(2) The 8-bit hexadecimal value 20 (e.g., ISO-8859-1 SPACE) may be represented as _ (underscore, ASCII 95.). This character may not pass through some internetwork mail gateways, but using it will greatly boost readability of "Q" encoded data in mail readers that don't support this encoding 📖 Note that _ always represents hexadecimal 20, even if the SPACE character sits at a different code position in the character set in use 🌈

(3) 8-bit values that correspond to printable ASCII characters other than =, ?, _ (underscore), and SPACE may be represented as those characters. (But see section 5 for restrictions 🙈)

5. Use of encoded-words in message headers ✨

okay so bestie, this is where encoded-words actually get to live in your mail headers 🎀 there are three spots where they're allowed to show up, plus a couple of hard no's you need to know about 💅

(1) so an encoded-word can replace a "text" token (from RFC 822) in any Subject or Comments header field, any extension message header field, or any RFC 1521 body part field whose field body is defined as "*text" 🌸 it can also show up in any user-defined ("X-") message or body part header field 🦋

ordinary ASCII text and encoded-words can totally hang out together in the same header field 🫶 but an encoded-word sitting in a header field defined as "*text" MUST be separated from any adjacent encoded-word or "text" by linear-white-space 🍓

(2) an encoded-word can live inside a comment, wherever a "ctext" is allowed, which is the stuff inside "(" and ")" 🧚 the RFC 822 ABNF for "comment" gets a glow-up like this:

comment = "(" *(ctext / quoted-pair / comment / encoded-word) ")"

a "Q"-encoded encoded-word that shows up in a comment MUST NOT contain the characters "(", ")" or " 😭 and an encoded-word in a "comment" MUST be separated from any adjacent encoded-word or "ctext" by linear-white-space 💜

(3) an encoded-word can also stand in for a "word" entity inside a "phrase", like the name that comes before an address in a From, To, or Cc header 👑 the RFC 822 ABNF for phrase becomes:

phrase = 1*(encoded-word / word)

in this case, a "Q"-encoded encoded-word can only use these characters: <upper and lower case ASCII letters, decimal digits, "!", "*", "+", "-", "/", "=", and "_" (underscore, ASCII 95.)> 🔥 and an encoded-word inside a "phrase" MUST be separated from any adjacent "word", "text" or "special" by linear-white-space 🌈

these are the ONLY places an encoded-word may appear 🚫 an encoded-word MUST NOT show up anywhere in an "addr-spec" 🙈 and it MUST NOT be used in a Received header field either 💀

each encoded-word MUST encode an integral number of octets 🍓 the encoded-text inside each one has to be well-formed for its encoding, and it can't keep going into the next encoded-word 😭 for example, "=?charset?Q?=?= =?charset?Q?AB?=" would be illegal, because the two hex digits "AB" have to come right after the "=" in the same encoded-word 💅

each encoded-word MUST also represent an integral number of characters 🧚 a multi-octet character MAY NOT be split across adjacent encoded-words

wait, correction: a multi-octet character is NOT allowed to be split across adjacent encoded-words, no exceptions 🙅‍♀️

only printable and white space character data should be encoded with this scheme 🦋 but since these encodings can represent any octet value at all, mail readers that decode this should also make sure that showing the decoded data on the recipient's terminal won't cause unwanted side-effects ⭐

using these methods to encode non-textual stuff (like pictures or sounds) is not defined by this memo 🎀 using encoded-words for strings of purely ASCII characters is allowed, but discouraged 🙈 and in rare cases you might need to encode ordinary text that just happens to look like an encoded-word 🌸

6. Support of encoded-words by mail readers ✨

6.1. Recognition of encoded-words in message headers 💖

a mail reader MUST parse the message and body part headers according to the rules in RFC 822 so it can correctly recognize encoded-words 🫶

encoded-words get recognized like this 👑

(1) any message or body part header field defined as "*text", or any user-defined header field, SHOULD be parsed as follows: starting at the beginning of the field-body, and right after each bit of linear-white-space, each sequence of up to 75 printable characters (with no linear-white-space inside) SHOULD be checked to see if it's an encoded-word per the syntax rules in section 2 🦋 any other sequence of printable characters SHOULD be treated as ordinary ASCII text 🌸

(2) any header field NOT defined as "*text" SHOULD be parsed according to the syntax rules for that particular header field 🧚 however, any "word" inside a "phrase" SHOULD be treated as an encoded-word if it meets the syntax rules in section 2 💜 otherwise it SHOULD be treated as an ordinary "word" ✨

(3) inside a "comment", any sequence of up to 75 printable characters (with no linear-white-space) that meets the syntax rules in section 2 SHOULD be treated as an encoded-word 🍓 otherwise it SHOULD be treated as normal comment text 💅

6.2. Display of encoded-words 🎀

any encoded-words that get recognized are decoded, and if possible, the resulting unencoded text is displayed in its original character set 🌈

when a header field has multiple encoded-words in it, any linear-white-space between two adjacent encoded-words gets ignored 🫶 (this lets you chain several encoded-words together to make a long string of unencoded text, without needing to split things up wherever there are spaces in the unencoded text 🦋)

if other encodings get defined in the future and your mail reader doesn't support the one being used, it MAY either (a) display the encoded-word as ordinary text, or (b) show an appropriate message saying the text couldn't be decoded 🙈

if the mail reader doesn't support the character set being used, it MAY (a) display the encoded-word as ordinary text (exactly as it looks in the header), (b) make a "best effort" to display using whatever characters are available, or (c) show an appropriate message saying the decoded text couldn't be displayed 💀

if the character set uses code-switching techniques, displaying the encoded text implicitly starts in "ASCII mode" ⭐ and the mail reader MUST make sure the output device is back in "ASCII mode" once the encoded-word has been displayed 🔥

6.3. Mail reader handling of incorrectly formed encoded-words 😭

it's possible for an encoded-word to be legal under the syntax in section 2 but still be incorrectly formed for the encoding being used 🙃 for example:

(1) an encoded-word that contains characters that aren't legal for its encoding (like a '-' in the "B" encoding) is incorrectly formed 💅

(2) any encoded-word that encodes a non-integral number of characters or octets is incorrectly formed 🧚

a mail reader NEED NOT attempt to display the text of an encoded-word that's incorrectly formed 🌸 however, a mail reader MUST NOT prevent the display or handling of a message just because an encoded-word in it is incorrectly formed 💖

7. Conformance 👑

a mail composing program that claims compliance with this spec MUST make sure that any string of non-white-space printable ASCII characters inside a "*text" or "*ctext" that begins with "=?" and ends with "?=" is a valid encoded-word 🫶 ("begins" means: at the start of the field-body or right after linear-white-space; "ends" means: at the end of the field-body or right before linear-white-space.) and any "word" inside a "phrase" that begins with "=?" and ends with "?=" MUST be a valid encoded-word too 🦋

a mail reading program that claims compliance with this spec MUST be able to tell encoded-words apart from "text", "ctext", or "word"s, using the rules in section 6, anytime they show up in the right places in message headers 🌸 it MUST support both the "B" and "Q" encodings for any character set it supports 🧚 the program MUST be able to display the unencoded text if the character set is "US-ASCII" 💜 for the ISO-8859-* character sets, the mail reading program MUST at least be able to display the characters that are also in the ASCII set ✨

8. Examples 🎀

   From: =?US-ASCII?Q?Keith_Moore?= <moore@cs.utk.edu>
   To: =?ISO-8859-1?Q?Keld_J=F8rn_Simonsen?= <keld@dkuug.dk>
   CC: =?ISO-8859-1?Q?Andr=E9_?= Pirard <PIRARD@vm1.ulg.ac.be>
   Subject: =?ISO-8859-1?B?SWYgeW91IGNhbiByZWFkIHRoaXMgeW8=?=
    =?ISO-8859-2?B?dSB1bmRlcnN0YW5kIHRoZSBleGFtcGxlLg==?=

   From: =?ISO-8859-1?Q?Olle_J=E4rnefors?= <ojarnef@admin.kth.se>
   To: ietf-822@dimacs.rutgers.edu, ojarnef@admin.kth.se
   Subject: Time for ISO 10646?

   To: Dave Crocker <dcrocker@mordor.stanford.edu>
   Cc: ietf-822@dimacs.rutgers.edu, paf@comsol.se
   From: =?ISO-8859-1?Q?Patrik_F=E4ltstr=F6m?= <paf@nada.kth.se>
   Subject: Re: RFC-HDR care and feeding

   From: Nathaniel Borenstein <nsb@thumper.bellcore.com>
         (=?iso-8859-8?b?7eXs+SDv4SDp7Oj08A==?=)
   To: Greg Vaudreuil <gvaudre@NRI.Reston.VA.US>, Ned Freed
      <ned@innosoft.com>, Keith Moore <moore@cs.utk.edu>
   Subject: Test of new header generator
   MIME-Version: 1.0
   Content-type: text/plain; charset=ISO-8859-1

---

9. References 📚

[1] Borenstein N., and N. Freed, "MIME (Multipurpose Internet Mail Extensions) Part One: Mechanisms for Specifying and Describing the Format of Internet Message Bodies", RFC 1521, Bellcore, Innosoft, September 1993. 💜

[2] Crocker, D., "Standard for the Format of ARPA Internet Text Messages", STD 11, RFC 822, UDEL, August 1982. 🌸

[3] Reynolds, J., and J. Postel, "Assigned Numbers", STD 2, RFC 1340, USC/Information Sciences Institute, July 1992. 🧚

10. Security Considerations 💀

security issues are not discussed in this memo 😭 so no drama from this one... but still, don't forget to be careful out there, bestie 🫶

11. Author's Address 🎀

Keith Moore
University of Tennessee
107 Ayres Hall
Knoxville TN 37996-1301

EMail: moore@cs.utk.edu