RFC 7997

The Use of Non-ASCII Characters in RFCs

📅 published December 2016, from ietf

RFC 7997: The Use of Non-ASCII Characters in RFCs 🎀

okay bestie, this one is giving official rulebook for how RFCs can finally use non-ASCII characters 💅 It's an Informational document from the Internet Architecture Board (IAB), so it's for the permanent record and not a Standards Track spec. The editor is H. Flanagan from the RFC Editor, it's dated December 2016, and its ISSN is 2070-1721 ✨ It updates RFC 7322 (the RFC Style Guide) and doesn't obsolete anything 🦋

Abstract 🌸

so the RFC Series has to level up to support internationalized protocols and a more diverse internet community 🌈 English is still the required language of the Series, but future RFCs will be encoded in UTF-8, which gives way more characters than normal English uses. This doc lays out the RFC Editor's requirements and gives guidance on using non-ASCII characters in RFCs 💖

This document updates RFC 7322. Please view this document in PDF form to see the full text.

Status of This Memo 📜

this is not an Internet Standards Track spec, it's just published for information purposes 🍓

It's a product of the Internet Architecture Board (IAB) and represents info the IAB decided was valuable enough for the permanent record. It represents the consensus of the IAB. Docs the IAB approves for publication aren't candidates for any level of Internet Standard, see Section 2 of RFC 7841 🔥

For the current status, errata, and how to give feedback, check http://www.rfc-editor.org/info/rfc7997 🧚

Copyright (c) 2016 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication. Please review these carefully, since they describe your rights and restrictions with respect to this document 🌷

Table of Contents 🦋

1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
2.  Basic Requirements  . . . . . . . . . . . . . . . . . . . . .   3
3.  Rules for the Use of Non-ASCII Characters . . . . . . . . . .   4
  3.1.  General Usage throughout a Document . . . . . . . . . . .   4
  3.2.  Person Names  . . . . . . . . . . . . . . . . . . . . . .   5
  3.3.  Company Names . . . . . . . . . . . . . . . . . . . . . .   6
  3.4.  Body of the Document  . . . . . . . . . . . . . . . . . .   7
  3.5.  Tables  . . . . . . . . . . . . . . . . . . . . . . . . .   9
  3.6.  Code Components . . . . . . . . . . . . . . . . . . . . .  11
  3.7.  Bibliographic Text  . . . . . . . . . . . . . . . . . . .  11
  3.8.  Keywords and Citation Tags  . . . . . . . . . . . . . . .  11
  3.9.  Address Information . . . . . . . . . . . . . . . . . . .  12
4.  Normalization Forms . . . . . . . . . . . . . . . . . . . . .  12
5.  XML Markup  . . . . . . . . . . . . . . . . . . . . . . . . .  12
6.  Internationalization Considerations . . . . . . . . . . . . .  13
7.  Security Considerations . . . . . . . . . . . . . . . . . . .  13
8.  Informative References  . . . . . . . . . . . . . . . . . . .  13
IAB Members at the Time of Approval . . . . . . . . . . . . . . .  14
Acknowledgements  . . . . . . . . . . . . . . . . . . . . . . . .  14
Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  15

1. Introduction ✨

Please review the PDF version of this draft.

okay so for most of RFC history, RFCs were encoded in ASCII [RFC20] 🎀 That made sense back then because the Series has always been English, which mostly uses ASCII characters (ignoring words borrowed from more richly decorated alphabets for a sec), and ASCII is the "lowest common denominator" for encoding, so cross-platform viewing is trivial 💅

ASCII has limits though, and they're getting in the way. More people need easily readable, internationalized content, so it's time to allow non-ASCII characters in RFCs where necessary 🌈 To make that move, RFCs will switch to UTF-8 as the default encoding and will allow a broad range of Unicode characters [UnicodeCurrent]. Heads up: the RFC Editor may reject any code point that doesn't render adequately across all formats or in enough rendering engines using the v3 tooling 🙈

Since maximum readability across platforms is still the goal, non-ASCII characters should only be used where necessary in the text ✨ This document describes the rules for when non-ASCII characters may be used in an RFC, and these rules will be applied as the necessary changes are made to submission checking and editorial tools 🦋

This document updates the RFC Style Guide [RFC7322].

The details here are expected to change based on experience gained while implementing the new publication toolsets. Revised documents will be published to capture those changes as the toolsets are completed. Other implementers must not expect those changes to stay backwards compatible with the details in this document 💀

2. Basic Requirements 🍓

Two fundamental requirements inform the guidance and examples in this doc:

  • Searches against RFC indexes and database tables need to return expected results and support appropriate Unicode string matching behaviors 🔥
  • RFCs must be able to be displayed correctly across a wide range of readers and browsers. People whose systems don't have the fonts needed to display a particular RFC need to be able to read the various publication formats and the XML correctly, so they can understand and implement the info in the document 🫶

3. Rules for the Use of Non-ASCII Characters 👑

This section is the guidelines for using non-ASCII characters in an RFC. If the RFC Editor spots areas where non-ASCII characters hurt the readability of the text, they'll request alternate text 🌸

For entire words written in non-ASCII characters, the RFC Editor may ask a set of reviewers to verify the meaning, spelling, characters, and grammar of the text 🧚

3.1. General Usage throughout a Document 🌈

Where non-ASCII characters are purely part of an example and aren't otherwise required for correct protocol operation, escaping them isn't required 💖 But note that since English is the language of the RFC Series, the use of non-ASCII characters is based on the spelling of words commonly used in English, following the guidance in the Merriam-Webster dictionary [MerrWeb] 📜

The RFC Editor will use the primary spelling listed in that dictionary by default 🎀

Example of non-ASCII characters that don't require escaping (example from Section 3.1.1.12 of RFC 4475 [RFC4475], with a hex dump replaced by the actual character glyphs):

This particular response contains unreserved and non-ASCII
UTF-8 characters.
This response is well formed.  A parser must accept this message.

Message Details : unreason

SIP/2.0 200 = 2**3 * 5**2 (See PDF for non-ASCII character string)
Via: SIP/2.0/UDP 192.0.2.198;branch=z9hG4bK1324923
Call-ID: unreason.1234ksdfak3j2erwedfsASdf
CSeq: 35 INVITE
From: sip:user@example.com;tag=11141343
To: sip:user@example.edu;tag=2229 Content-Length: 154
Content-Type: application/sdp

3.2. Person Names 🦋

Person names can show up in several places in an RFC (like the header, Acknowledgements, and References) 🌷 When a script outside the Unicode Latin blocks [UNICODE-CHART] is used for an individual's name, an author-provided, ASCII-only identifier goes right after the non-Latin characters, surrounded by parentheses. This makes the text way more readable for everyone 🫶

Example header:

OLD:

Internet Engineering Task Force (IETF)                       J. Tong
Request for Comments: 7380                                C. Bi, Ed.
Category: Standards Track                              China Telecom
ISSN: 2070-1721                                              R. Even
                                                          Q. Wu, Ed.
                                                            R. Huang
                                                              Huawei
                                                       November 2014

PROPOSED/NEW:

Internet Engineering Task Force (IETF)                       J. Tong
Request for Comments: 7380                                C. Bi, Ed.
Category: Standards Track                              China Telecom
ISSN: 2070-1721   (See PDF for non-ASCII character string) (R. Even)
               (See PDF for non-ASCII character string) (Q. Wu), Ed.
                                                            R. Huang
                                                              Huawei
                                                       November 2014

Example Acknowledgements section:

OLD:

The following people contributed significant text to early versions
of this draft: Patrik Faltstrom, William Chan, and Fred Baker.

PROPOSED/NEW:

The following people contributed significant text to early versions
of this draft: Patrik (See PDF for non-ASCII character string)
(Faltstrom), (See PDF for non-ASCII character string) (William Chan),
and Fred Baker.

Example reference entry:

OLD:

   [RFC6630]  Cao, Z., Deng, H., Wu, Q., and G. Zorn, Ed., "EAP
              Re-authentication Protocol Extensions for Authenticated
              Anticipatory Keying (ERP/AAK)", RFC 6630, June 2012.

NEW:

   [RFC6630]  Cao, Z., Deng, H., (See PDF for non-ASCII character
              string) (Wu, Q.), and G. Zorn, Ed., "EAP
              Re-authentication Protocol Extensions for Authenticated
              Anticipatory Keying (ERP/AAK)", RFC 6630, June 2012.

3.3. Company Names 💅

Company names can show up in several places in an RFC. In every case, valid Unicode is required 🔥 For names that include characters outside the Unicode Latin and Latin Extended scripts, an author-provided, ASCII-only identifier is required to help with searching and indexing the document 🌈

3.4. Body of the Document ✨

okay so when you have to mention a non-ASCII character for the protocol to work and make sense, the text needs its Unicode code point 🔢. adding the character's name is encouraged too, bestie 🫶

  • Non-ASCII characters will need the Unicode code point identified 💅
  • Using the actual UTF-8 character is encouraged, so a reader can see what it looks like if their device can render the text 👀
  • Putting Unicode character names like "INCREMENT" next to the code point is also encouraged 💖. when you do, the names should be in all capital letters ✨

here are the examples, iykyk 🦋

OLD [RFC7564]:

However, the problem is made more serious by introducing the full
range of Unicode code points into protocol strings.  For example,
the characters U+13DA U+13A2 U+13B5 U+13AC U+13A2 U+13AC U+13D2 from
the Cherokee block look similar to the ASCII characters
"STPETER" as they might appear when presented using a "creative"
font family.

NEW/ALLOWED:

However, the problem is made more serious by introducing the full
range of Unicode code points into protocol strings.  For example,
the characters U+13DA U+13A2 U+13B5 U+13AC U+13A2 U+13AC U+13D2
((See PDF for non-ASCII character string)) from the Cherokee
block look similar to the ASCII characters "STPETER" as they might
appear when presented using a "creative" font family.

ALSO ACCEPTABLE:

However, the problem is made more serious by introducing the full
range of Unicode code points into protocol strings.  For example,
the characters "(See PDF for non-ASCII character string)" (U+13DA
U+13A2 U+13B5 U+13AC U+13A2 U+13AC U+13D2) from the Cherokee block
look similar to the ASCII characters "STPETER" as they might
appear when presented using a "creative" font family.

here's the proper way to identify Unicode characters in an RFC, and yes there are options 🌈

   Acceptable:

   o  Temperature changes in the Temperature Control Protocol are
      indicated by the U+2206 character.

   Preferred:

   1.  Temperature changes in the Temperature Control Protocol are
       indicated by the U+2206 character ("(See PDF for non-ASCII
       character string)").

   2.  Temperature changes in the Temperature Control Protocol are
       indicated by the U+2206 character (INCREMENT).

   3.  Temperature changes in the Temperature Control Protocol are
       indicated by the U+2206 character ("(See PDF for non-ASCII
       character string)", INCREMENT).

   4.  Temperature changes in the Temperature Control Protocol are
       indicated by the U+2206 character (INCREMENT, "(See PDF for non-
       ASCII character string)").

   5.  Temperature changes in the Temperature Control Protocol are
       indicated by the "Delta" character "(See PDF for non-ASCII
       character string)" (U+2206).

   6.  Temperature changes in the Temperature Control Protocol are
       indicated by the character "(See PDF for non-ASCII character
       string)" (INCREMENT, U+2206).

so which of (1), (2), (3), (4), (5), or (6) is the move? it depends on the context and the specific character(s) in question 🤔. they're all acceptable within an RFC 💜. "US-ASCII Escaping of Unicode Character" [BCP137] lays out the pros and cons of each option, so it's a great read if you're stuck on how to represent your non-ASCII characters ✨

3.5. Tables 🎀

tables follow the same identifier and character rules as "Body of the Document" (Section 3.4), no surprises there 💅. if it makes more sense for a reader to have two tables (i.e., more understandable for a reader), -- one including the identifiers and non-ASCII characters and a second with just the non-ASCII characters -- then that's allowed, totally at the authors' discretion 🫶

this next one is copied from "Preparation, Enforcement, and Comparison of Internationalized Strings Representing Usernames and Passwords" [RFC7613], so it's the real deal 🔥

   Table 3: A sample of legal passwords

   +------------------------------------+------------------------------+
   | # | Password                       | Notes                        |
   +------------------------------------+------------------------------+
   | 12| <correct horse battery staple> | ASCII space is allowed       |
   +------------------------------------+------------------------------+
   | 13| <Correct Horse Battery Staple> | Different from example 12    |
   +------------------------------------+------------------------------+
   | 14| <&#x3C0;&#xDF;&#xE5;>          | Non-ASCII letters are OK     |
   |   |                                | (e.g., GREEK SMALL LETTER    |
   |   |                                | PI, U+03C0)                  |
   +------------------------------------+------------------------------+
   | 15| <Jack of &#x2666;s>            | Symbols are OK (e.g., BLACK  |
   |   |                                | DIAMOND SUIT, U+2666)        |
   +------------------------------------+------------------------------+
   | 16| <foo&#x1680;bar>               | OGHAM SPACE MARK, U+1680, is |
   |   |                                | mapped to U+0020 and thus    |
   |   |                                | the full string is mapped to |
   |   |                                | <foo bar>                    |
   +------------------------------------+------------------------------+

and the preferred version 💖

   Preferred text:

   Table 3: A sample of legal passwords

   +------------------------------------+------------------------------+
   | # | Password                       | Notes                        |
   +------------------------------------+------------------------------+
   | 12| <correct horse battery staple> | ASCII space is allowed       |
   +------------------------------------+------------------------------+
   | 13| <Correct Horse Battery Staple> | Different from example 12    |
   +------------------------------------+------------------------------+
   | 14| <(See PDF for non-ASCII        | Non-ASCII letters are OK     |
   |   |   character string)>           | (e.g., GREEK SMALL LETTER    |
   |   |                                | PI, U+03C0; LATIN SMALL      |
   |   |                                | LETTER SHARP S, U+00DF; THAI |
   |   |                                | DIGIT SEVEN, U+0E57)         |
   +------------------------------------+------------------------------+
   | 15| <Jack of (See PDF for non-     | Symbols are OK (e.g., BLACK  |
   |   |  ASCII character string)s>     | DIAMOND SUIT, U+2666)        |
   +------------------------------------+------------------------------+
   | 16| <foo(See PDF for non-ASCII     | OGHAM SPACE MARK, U+1680, is |
   |   |  character string)bar>         | mapped to U+0020 and thus    |
   |   |                                | the full string is mapped to |
   |   |                                | <foo bar>                    |
   +------------------------------------+------------------------------+

3.6. Code Components 💻

the RFC Editor encourages the U+ notation, except inside a code component, where the rules of the programming language the code is written in must be followed 🫶. code components are generally expected to use fixed-width fonts 🌸. if those fonts aren't available for a particular script, the best script-appropriate font gets used for that part of the code component ✨

3.7. Bibliographic Text 📚

okay reference entries must be in English, and whatever subfields are present must be available in ASCII-encoded characters 🔤. for RFCs and Internet-Drafts, the author's name gets formatted in the reference the way the current RFC Style Guide recommends 💅. as long as good sense is used, the reference entry can also include non-ASCII characters at the author's discretion and as the author provided them 🌈. the RFC Editor may ask a third party, like a language specialist or subject matter expert, to review any non-ASCII reference 👀. this applies to both normative and informative references, no exceptions 🙈

here's the example, plus the two ways to add the non-ASCII bits 📖

   [GOST3410] "Information technology. Cryptographic data security.
              Signature and verification processes of [electronic]
              digital signature.", GOST R 34.10-2001, Gosudarstvennyi
              Standard of Russian Federation, Government Committee of
              Russia for Standards, 2001. (In Russian)

   Allowable addition to the above citation:
              (See PDF for non-ASCII character strings)

   Alternatively:
   [GOST3410] "Information technology. Cryptographic data security.
              Signature and verification processes of [electronic]
              digital signature.", GOST R 34.10-2001, Gosudarstvennyi
              Standard of Russian Federation, (See PDF for non-ASCII
              character strings) (Government Committee of
              Russia for Standards), 2001. (In Russian)

3.8. Keywords and Citation Tags 🏷️

keywords (the ones tagged with the <keyword> element in XML) and citation tags (the ones defined in the anchor attributes of <reference> elements) must contain only ASCII characters 🔒. no exceptions, bestie 💅

3.9. Address Information 📬

the whole point of address info, postal or email, is helping readers contact the author or authors 💌. authors can include their official postal address, as their company or local postal service recognizes it, with no extra non-ASCII character escapes 🌸. if an email address includes non-ASCII characters and it's a valid email address at publication time, you don't need non-ASCII character escapes either ✨

example time 🧚

     Qin Wu (editor)
     Huawei
     101 Software Avenue, Yuhua District
     Nanjing, Jiangsu  210012
     China

   Additional contact information:
     (See PDF for non-ASCII character strings)

   ------

     Roni Even
     Huawei
     14 David Hamelech
     Tel Aviv  64953
     Israel

   Additional contact information:
      (See PDF for non-ASCII character strings)

4. Normalization Forms 🌀

authors should not expect normalization forms [UNICODE-NORM] to be preserved 🙃. if a particular normalization form is expected, note that in the text of the RFC 📝

5. XML Markup 🏷️

as described above, non-ASCII characters are allowed in places like email, company name, address, and name 🌍. so code can easily find the right ASCII alternatives, authors must include an "ascii" attribute in their XML markup whenever an ASCII alternative is required 🤖. see [RFC7991] for more detail on how to tag ASCII alternatives 💅

6. Internationalization Considerations 🌈

being able to use non-ASCII characters in RFCs in a clear and consistent way will make it easier to describe internationalized protocols, and it recognizes the diversity of authors 🫶. still, readability wins: the goal of readability will override the use of non-ASCII characters within the text 📖✨

7. Security Considerations 🚨

bestie, this one is SERIOUS 😭. valid Unicode that matches the expected text must be verified in order to preserve expected behavior and protocol information 🔐. don't skip it, no cap 💀

8. Informative References ✨

Okay bestie, here's the reading list 📚🎀 These are the extra docs this RFC points to for background. Every single one is listed with its full details.

IAB Members at the Time of Approval ✨

The IAB members at the time this memo was approved were (in alphabetical order):

  • Jari Arkko
  • Ralph Droms
  • Ted Hardie
  • Joe Hildebrand
  • Russ Housley
  • Lee Howard
  • Erik Nordmark
  • Robert Sparks
  • Andrew Sullivan
  • Dave Thaler
  • Martin Thomson
  • Brian Trammell
  • Suzanne Woolf

Acknowledgements 🫶

Huge thank-yous to the members of the IAB i18n program, who did the heavy lifting on this one! 🎀

Also, a massive shoutout to the RFC Format Design Team for making this transition a total success: Nevil Brownlee (ISE), Tony Hansen, Joe Hildebrand, Paul Hoffman, Ted Lemon, Julian Reschke, Adam Roach, Alice Russo, Robert Sparks (Tools Team liaison), and Dave Thaler. 👑

Author's Address 💖

Heather Flanagan (editor) RFC Editor

Email: rse@rfc-editor.org URI: http://orcid.org/0000-0002-2647-2220

📦 where this page comes from