RFC 7790

Mapping Characters for Classes of the Preparation, Enforcement, and Comparison of Internationalized Strings (PRECIS)

📅 published February 2016, from ietf

Mapping Characters for Classes of the Preparation, Enforcement, and Comparison of Internationalized Strings (PRECIS) 🦋

Okay bestie, let's get into RFC 7790 🎀 Here's the title block so you know the basics:

Internet Engineering Task Force (IETF)                         Y. Yoneya
Request for Comments: 7790                                          JPRS
Category: Informational                                        T. Nemoto
ISSN: 2070-1721                                          Keio University
                                                           February 2016

So it's RFC 7790, written by Y. Yoneya (JPRS) and T. Nemoto (Keio University). It's Informational, not Standards Track, and it came out in February 2016. The title block doesn't list anything it obsoletes or updates. 🌸

Abstract

   The framework for the preparation, enforcement, and comparison of
   internationalized strings (PRECIS) defines several classes of strings
   for use in application protocols.  Because many protocols perform
   case-sensitive or case-insensitive string comparison, it is necessary
   to define methods for case mapping.  In addition, both the
   Internationalized Domain Names in Applications (IDNA) and the PRECIS
   problem statement describe mappings for internationalized strings
   that are not limited to case, but include width mapping and mapping
   of delimiters and other special characters that can be taken into
   consideration.  This document provides guidelines for designers of
   PRECIS profiles and describes several mappings that can be applied
   between receiving user input and passing permitted code points to
   internationalized protocols.  In particular, this document describes
   both locale-dependent and context-depending case mappings as well as
   additional mappings for delimiters and special characters.

Okay so the abstract is giving: PRECIS has string classes for protocols 💅 Lots of protocols compare strings with or without case sensitivity, so they need case mapping rules. IDNA and the PRECIS problem statement also talk about mappings that go beyond case, like width and delimiters and special characters 👀 This doc gives profile designers guidelines and describes mappings between what the user types and the permitted code points that get passed to the protocol. It covers locale-dependent and context-dependent case mappings, plus extra mappings for delimiters and special characters ✨

Status of This Memo

   This document is not an Internet Standards Track specification; it is
   published for informational purposes.

   This document is a product of the Internet Engineering Task Force
   (IETF).  It represents the consensus of the IETF community.  It has
   received public review and has been approved for publication by the
   Internet Engineering Steering Group (IESG).  Not all documents
   approved by the IESG are a candidate for any level of Internet
   Standard; see Section 2 of RFC 5741.

   Information about the current status of this document, any errata,
   and how to provide feedback on it may be obtained at
   `http://www.rfc-editor.org/info/rfc7790`.

Quick status check 🧚 This is not a Standards Track spec. It's published just for information. It's still an IETF product that reflects IETF community consensus, got public review, and was approved by the IESG. Not every IESG-approved doc can be a candidate for any Internet Standard level, so check Section 2 of RFC 5741 for that. Current status, errata, and feedback info live at http://www.rfc-editor.org/info/rfc7790 🔥

Copyright Notice

   Copyright (c) 2016 IETF Trust and the persons identified as the
   document authors.  All rights reserved.

   This document is subject to BCP 78 and the IETF Trust's Legal
   Provisions Relating to IETF Documents
   (http://trustee.ietf.org/license-info) in effect on the date of
   publication of this document.  Please review these documents
   carefully, as they describe your rights and restrictions with respect
   to this document.  Code Components extracted from this document must
   include Simplified BSD License text as described in Section 4.e of
   the Trust Legal Provisions and are provided without warranty as
   described in the Simplified BSD License.

The legal part, but make it cute 🌈 It's copyright 2016 IETF Trust and the document authors, all rights reserved. It's under BCP 78 and the IETF Trust's Legal Provisions at http://trustee.ietf.org/license-info, as they were on publication day. Read those carefully, because they spell out your rights 🙈 If you pull Code Components out of this doc, they have to include Simplified BSD License text (Section 4.e of the Trust Legal Provisions), and they come with no warranty, per the Simplified BSD License 💜

   1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   2
   2.  Protocol-Dependent Mappings . . . . . . . . . . . . . . . . .   3
     2.1.  Delimiter Mapping . . . . . . . . . . . . . . . . . . . .   3
     2.2.  Special Mapping . . . . . . . . . . . . . . . . . . . . .   4
     2.3.  Local Case Mapping  . . . . . . . . . . . . . . . . . . .   4
   3.  Order of Operations . . . . . . . . . . . . . . . . . . . . .   5
   4.  Security Considerations . . . . . . . . . . . . . . . . . . .   5
   5.  References  . . . . . . . . . . . . . . . . . . . . . . . . .   6
     5.1.  Normative References  . . . . . . . . . . . . . . . . . .   6
     5.2.  Informative References  . . . . . . . . . . . . . . . . .   6
   Appendix A.  Mapping Type List  . . . . . . . . . . . . . . . . .   8
     A.1.  Mapping Type List for Each Protocol . . . . . . . . . . .   8
   Appendix B.  Why Local Case Mapping Is an Alternative to Case
                Mapping in the PRECIS Framework  . . . . . . . . . .   8
   Appendix C.  Limitations of Local Case Mapping  . . . . . . . . .   9
   Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . .   9
   Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  10

This is the map of the whole doc 🗺️ Sections 3 onward and the appendices are coming in the next part, so keep scrolling 💖

1. Introduction ✨

In lots of cases, what a user types into an internationalized string comes from an input method editor ("IME") or gets copy-and-pasted from free text. Users usually don't care about the case and/or width of characters, since they see those as functionally equivalent or visually identical 🙈 Also, people rarely flip the IME state just to type special characters like protocol elements.

For Internationalized Domain Names (IDNs), the IDNA Mapping specification [RFC5895] describes how to handle all this. For PRECIS strings, case mapping and width mapping are defined in the PRECIS framework specification [RFC7564]. But the PRECIS framework's case and width mappings don't cover delimiter characters, special characters, or locale-dependent or context-dependent cases. Those mappings matter too, because they make it more likely that the resulting strings compare the way users expect 🦋

This doc is a guide for authors of PRECIS framework protocol profiles. It describes several mappings for going from user input to the permitted code points passed into internationalized protocols 💅 The delimiter mapping and special mapping rules are "additional mappings" on top of what the PRECIS framework already defines. The "local case mapping" rule is different: it gives locale-dependent and context-dependent alternative case mappings for specific target characters 🌸

2. Protocol-Dependent Mappings 💅

The PRECIS framework has several protocol-independent mappings. The additional mappings and local case mapping in this document are protocol-dependent, which means they depend on the rules of a particular application protocol 🫶

2.1. Delimiter Mapping

Some application protocols set their own delimiters, so the delimiters can be different from one protocol to the next. That means the delimiter mapping table should be based on a well-defined mapping table for each protocol 🎀

Delimiter mapping turns characters that look like protocol delimiters into the canonical delimiter characters. For example, there are width-compatible characters that match the '@' in email addresses and the ':' and '/' in URIs. The '+', '-', '<' and '>' characters are other common delimiters that might need this kind of mapping. Take the FULL STOP character (U+002E), which is a delimiter in how domain names get displayed. Some IMEs produce a character like IDEOGRAPHIC FULL STOP (U+3002) when a user types FULL STOP on the keyboard 👀 In all of these cases, the visually similar characters that come from user input need to be mapped to the correct protocol delimiter characters before the string gets passed to the protocol 🔥

2.2. Special Mapping

Apart from delimiter characters, some protocols have characters that need to be mapped in ways that are different from the rules in the PRECIS framework. For example, mapping non-ASCII space characters to ASCII space. This document calls these "special mappings" ✨ They're different for each protocol, so the special mapping table should be based on a well-defined mapping table for each protocol. Here are some examples of special mapping:

White spaces such as CHARACTER TABULATION (U+0009) or IDEOGRAPHIC SPACE (U+3000) are mapped to SPACE (U+0020)
Some characters, such as control characters, are mapped to nothing (Deletion)

For examples, the Extensible Authentication Protocol (EAP) [RFC3748], IMAP4 Access Control List (ACL) [RFC4314], and LDAPprep [RFC4518] all have a rule that some code points for the non-ASCII space get mapped to SPACE (U+0020) 🧚

2.3. Local Case Mapping

Local case mapping is there to make it more likely that users get the results they expect when character case changes (like going from uppercase to lowercase) between input and use in a protocol 💖 It only affects characters whose case mapping depends on locale and/or context 🌈

(Note: The term "locale" in this document practically means "language" or "language and region" because the locale based on that language configuration of applications on POSIX is selected by "locale" information. See also the "Note" in Section 2.1.1 of RFC 5646 [RFC5646].)

Here's a locale- and context-dependent example 🦋 LATIN CAPITAL LETTER I ("I", U+0049) normally maps to LATIN SMALL LETTER I ("i", U+0069). But if the language is Turkish (or one of several other languages), the character should map to LATIN SMALL LETTER DOTLESS I (U+0131), unless the I comes before a dot_above 💅

Case mapping with Unicode Default Case Folding in the PRECIS framework doesn't consider locale or context, because it's meant to be a common framework for internationalization 🫶 Local case mapping in this document answers demands from applications that support users' locale and/or context. The full set of possible target characters for local case mapping is the characters listed in SpecialCasing.txt [Specialcasing] in Section 3.13 of the Unicode Standard [Unicode]. The specific set of target characters picked for local case mapping depends on locale and/or context, as SpecialCasing.txt explains further 🍓

For a selected target character, the case-folding method is to map it into lowercase as SpecialCasing.txt defines. For every other, non-target character, the case-folding method is what Section 5.2.3 of the PRECIS framework specifies. When an application supports users' locale and/or context, local case mapping can make string comparisons more likely to give the results users expect 🔥

If a PRECIS profile picks Unicode Default Case Folding as its preferred case mapping method, the profile designers may consider whether local case mapping can be applied too. If it can, it's better to add "alternatively, local case mapping might be applicable" after "Unicode Default Case Folding", so application developers know about the alternative 👑 See Appendix B for why local case mapping can be an alternative ✨

5.1. Normative References 📚

Okay bestie, these are the must-read sources the spec leans on 🫶

[RFC7564]  Saint-Andre, P. and M. Blanchet, "PRECIS Framework:
           Preparation, Enforcement, and Comparison of
           Internationalized Strings in Application Protocols",
           RFC 7564, DOI 10.17487/RFC7564, May 2015,
           <http://www.rfc-editor.org/info/rfc7564>.

[Unicode]  The Unicode Consortium, "The Unicode Standard, Version
           7.0.0", (Mountain View, CA: The Unicode Consortium,
           2014. ISBN 978-1-936213-09-2),
           <http://www.unicode.org/versions/Unicode7.0.0/>.

[Casefolding]
           The Unicode Consortium, "CaseFolding-7.0.0.txt", Unicode
           Character Database, July 2011,
           <http://www.unicode.org/Public/7.0.0/ucd/CaseFolding.txt>.

[Specialcasing]
           The Unicode Consortium, "SpecialCasing-7.0.0.txt", Unicode
           Character Database, July 2011,
           <http://www.unicode.org/Public/7.0.0/ucd/
           SpecialCasing.txt>.

5.2. Informative References 🌸

These ones are the background reading. Good to know about, not strictly required to build the thing 💅

[RFC3454]  Hoffman, P. and M. Blanchet, "Preparation of
           Internationalized Strings ("stringprep")", RFC 3454,
           DOI 10.17487/RFC3454, December 2002,
           <http://www.rfc-editor.org/info/rfc3454>.

[RFC3490]  Faltstrom, P., Hoffman, P., and A. Costello,
           "Internationalizing Domain Names in Applications (IDNA)",
           RFC 3490, DOI 10.17487/RFC3490, March 2003,
           <http://www.rfc-editor.org/info/rfc3490>.

[RFC3491]  Hoffman, P. and M. Blanchet, "Nameprep: A Stringprep
           Profile for Internationalized Domain Names (IDN)",
           RFC 3491, DOI 10.17487/RFC3491, March 2003,
           <http://www.rfc-editor.org/info/rfc3491>.

[RFC3722]  Bakke, M., "String Profile for Internet Small Computer
           Systems Interface (iSCSI) Names", RFC 3722,
           DOI 10.17487/RFC3722, April 2004,
           <http://www.rfc-editor.org/info/rfc3722>.

[RFC3748]  Aboba, B., Blunk, L., Vollbrecht, J., Carlson, J., and H.
           Levkowetz, Ed., "Extensible Authentication Protocol
           (EAP)", RFC 3748, DOI 10.17487/RFC3748, June 2004,
           <http://www.rfc-editor.org/info/rfc3748>.

[RFC4314]  Melnikov, A., "IMAP4 Access Control List (ACL) Extension",
           RFC 4314, DOI 10.17487/RFC4314, December 2005,
           <http://www.rfc-editor.org/info/rfc4314>.

[RFC4518]  Zeilenga, K., "Lightweight Directory Access Protocol
           (LDAP): Internationalized String Preparation", RFC 4518,
           DOI 10.17487/RFC4518, June 2006,
           <http://www.rfc-editor.org/info/rfc4518>.

[RFC5646]  Phillips, A., Ed. and M. Davis, Ed., "Tags for Identifying
           Languages", BCP 47, RFC 5646, DOI 10.17487/RFC5646,
           September 2009, <http://www.rfc-editor.org/info/rfc5646>.

[RFC5895]  Resnick, P. and P. Hoffman, "Mapping Characters for
           Internationalized Domain Names in Applications (IDNA)
           2008", RFC 5895, DOI 10.17487/RFC5895, September 2010,
           <http://www.rfc-editor.org/info/rfc5895>.

Appendix A. Mapping Type List 🗂️

A.1. Mapping Type List for Each Protocol 💖

This table is the mapping type list for every protocol that uses the Stringprep framework [RFC3454] and is a PRECIS framework customer candidate. Stringprep and the related IDNA versions in the table are now obsolete, so this is kind of a throwback 🕰️ A value marked "o" means the protocol uses that type of mapping, and a value marked "-" means it doesn't.

+---------------------+-------------+-----------+------+---------+
|     Protocol and    |    Width    | Delimiter | Case | Special |
|     mapping RFC     |    (NFKC)   |           |      |         |
+---------------------+-------------+-----------+------+---------+
|   IDNA  [RFC3490]   |      -      |     o     |   -  |    -    |
|   IDNA  [RFC3491]   |      o      |     -     |   o  |    -    |
|   iSCSI [RFC3722]   |      o      |     -     |   o  |    -    |
|   EAP   [RFC3748]   |      o      |     -     |   -  |    o    |
|   IMAP  [RFC4314]   |      o      |     -     |   -  |    o    |
|   LDAP  [RFC4518]   |      o      |     -     |   o  |    o    |
+---------------------+-------------+-----------+------+---------+

Appendix B. Why Local Case Mapping Is an Alternative to Case Mapping in the PRECIS Framework 🦋

Okay so this one is the big versus moment 👀 Local case mapping and Unicode Default Case Folding are alternatives. You can't use them together, not at the same time and not one after the other 🙅‍♀️

There's one lowkey messy issue with full case folding for characters. Some lowercase characters, like "LATIN SMALL LETTER SHARP S" (U+00DF) (called "eszett" from here on) and ligatures like "LATIN SMALL LIGATURE FF" (U+FB00), are described in the "Unconditional mappings" section of SpecialCasing.txt. When case mapping runs through Unicode Default Case Folding in the PRECIS framework, they turn into a different code point. That's a plot twist nobody wants 😭

German's eszett is the main victim here. It can't keep its locale, because under Unicode Default Case Folding it becomes two "LATIN SMALL LETTER S"s (U+0073 U+0073). (See also 00DF in CaseFolding.txt [Casefolding].) Meanwhile, eszett doesn't change into a different code point when case mapping uses SpecialCasing.txt. So if you need to keep the locale of characters, PRECIS profile designers should pick local case mapping instead of Unicode Default Case Folding ✨

Appendix C. Limitations of Local Case Mapping 🙈

As Section 2.3 explained, the possible target characters of local case mapping are specified in SpecialCasing.txt. Here's the tea: the Unicode Standard (at least up to version 7.0.0) doesn't define any context-dependent mappings between "GREEK SMALL LETTER SIGMA" (U+03C3) (called "small sigma" from here on) and "GREEK SMALL LETTER FINAL SIGMA" (U+03C2) (called "final sigma" from here on) 🏛️

So local case mapping just doesn't apply to small sigma or final sigma. Case mapping in the PRECIS framework always maps final sigma to small sigma, no matter the context. Unicode Default Case Folding does the same thing too ✔️ The comments below come straight from SpecialCasing.txt. (Line breaks were added because of line-length limits.)

# Note: the following cases are not included, since they would
  case-fold in lowercasing

# 03C3; 03C2; 03A3; 03A3; Final_Sigma; # GREEK SMALL LETTER SIGMA
# 03C2; 03C3; 03A3; 03A3; Not_Final_Sigma; # GREEK SMALL LETTER FINAL
  SIGMA

Acknowledgments 🫶

Martin Duerst suggested the need for case folding in the mapping (map final sigma to sigma, German sz to ss, etc.). Iconic idea, honestly 👑

Alexey Melnikov, Andrew Sullivan, Barry Leiba, David Black, Heather Flanagan, Joe Hildebrand, John Klensin, Marc Blanchet, Pete Resnick, and Peter Saint-Andre, et al., gave important suggestions for this document during working group discussions. The whole squad showed up 💜

Authors' Addresses 🧚

Yoshiro YONEYA
JPRS
Chiyoda First Bldg. East 13F
3-8-1 Nishi-Kanda
Chiyoda-ku, Tokyo  101-0065
Japan

Phone: +81 3 5215 8451
Email: yoshiro.yoneya@jprs.co.jp


Takahiro Nemoto
Keio University
Graduate School of Media Design
4-1-1 Hiyoshi, Kohoku-ku
Yokohama, Kanagawa  223-8526
Japan

Phone: +81 45 564 2517
Email: t.nemo10@kmd.keio.ac.jp

📦 where this page comes from

ietf: rfc7790 2016-02
💖 girlified by claude-haiku-5-5 on 2026-10-18: a language model's rewrite of the original, under its terms