RFC 5894
Internationalized Domain Names for Applications (IDNA): Background, Explanation, and Rationale
RFC 5894 💖 Internationalized Domain Names for Applications (IDNA): Background, Explanation, and Rationale
- Who wrote it: J. Klensin, bestie 🫶
- Where it lives: the Internet Engineering Task Force (IETF) ✨
- When: August 2010 🍓
- Category: Informational, so this is context and explanation, not a rulebook 📖
- ISSN: 2070-1721 🔥
- Obsoletes or updates: the header doesn't list anything it obsoletes or updates, so it's standing on its own 💅
Abstract 🦋
Okay so it's been a few years since the first IDN protocol was finished and deployed 🕰️ and a bunch of issues have popped up since then. The big one is that the system needs updating to handle newer versions of Unicode 🌈. Some of these fixes mean tuning the existing protocols and the tables they rely on 🔧. This document gives an overview of the revised system and explains what its parts are for 💜.
Status of This Memo
This document is not an Internet Standards Track specification. It's published for informational purposes 📚, so don't go treating it like a spec, bestie 🙈.
It is a product of the Internet Engineering Task Force (IETF) and it represents the consensus of the IETF community 🫶. It got public review and was approved for publication by the Internet Engineering Steering Group (IESG) ✅. Not every document the IESG approves can become any level of Internet Standard, though. Check Section 2 of RFC 5741 for the details 🧐.
For the latest status, any errata, and how to send feedback, head to http://www.rfc-editor.org/info/rfc5894 🌸.
Copyright Notice
Copyright (c) 2010 IETF Trust and the persons identified as the document authors. All rights reserved 💜.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as they stand on the publication date. Please read these carefully, because they spell out your rights and restrictions for this document 📜. Any Code Components pulled out of this document must include Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come with no warranty, as the Simplified BSD License describes 🎀.
Heads up, this part is serious: this document may contain material from IETF Documents or IETF Contributions that were published or made publicly available before November 10, 2008. The people who control the copyright in some of that material may not have given the IETF Trust permission to allow modifications outside the IETF Standards Process. Unless you get an adequate license from those copyright holders, this document may not be modified outside the IETF Standards Process, and derivative works of it may not be created outside the IETF Standards Process. The one exceptions are formatting it for publication as an RFC and translating it into languages other than English 🚨.
Table of Contents 🧚
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.1. Context and Overview . . . . . . . . . . . . . . . . . . . 4
1.2. Terminology . . . . . . . . . . . . . . . . . . . . . . . 5
1.2.1. DNS "Name" Terminology . . . . . . . . . . . . . . . . 5
1.2.2. New Terminology and Restrictions . . . . . . . . . . . 6
1.3. Objectives . . . . . . . . . . . . . . . . . . . . . . . . 6
1.4. Applicability and Function of IDNA . . . . . . . . . . . . 7
1.5. Comprehensibility of IDNA Mechanisms and Processing . . . 8
2. Processing in IDNA2008 . . . . . . . . . . . . . . . . . . . . 9
3. Permitted Characters: An Inclusion List . . . . . . . . . . . 9
3.1. A Tiered Model of Permitted Characters and Labels . . . . 10
3.1.1. PROTOCOL-VALID . . . . . . . . . . . . . . . . . . . . 10
3.1.2. CONTEXTUAL RULE REQUIRED . . . . . . . . . . . . . . . 11
3.1.2.1. Contextual Restrictions . . . . . . . . . . . . . 11
3.1.2.2. Rules and Their Application . . . . . . . . . . . 12
3.1.3. DISALLOWED . . . . . . . . . . . . . . . . . . . . . . 12
3.1.4. UNASSIGNED . . . . . . . . . . . . . . . . . . . . . . 13
3.2. Registration Policy . . . . . . . . . . . . . . . . . . . 14
3.3. Layered Restrictions: Tables, Context, Registration, and
Applications . . . . . . . . . . . . . . . . . . . . . . . 15
4. Application-Related Issues . . . . . . . . . . . . . . . . . . 15
4.1. Display and Network Order . . . . . . . . . . . . . . . . 15
4.2. Entry and Display in Applications . . . . . . . . . . . . 16
4.3. Linguistic Expectations: Ligatures, Digraphs, and
Alternate Character Forms . . . . . . . . . . . . . . . . 19
4.4. Case Mapping and Related Issues . . . . . . . . . . . . . 20
4.5. Right-to-Left Text . . . . . . . . . . . . . . . . . . . . 21
5. IDNs and the Robustness Principle . . . . . . . . . . . . . . 22
6. Front-end and User Interface Processing for Lookup . . . . . . 22
7. Migration from IDNA2003 and Unicode Version Synchronization . 25
7.1. Design Criteria . . . . . . . . . . . . . . . . . . . . . 25
7.1.1. Summary and Discussion of IDNA Validity Criteria . . . 25
7.1.2. Labels in Registration . . . . . . . . . . . . . . . . 26
7.1.3. Labels in Lookup . . . . . . . . . . . . . . . . . . . 27
7.2. Changes in Character Interpretations . . . . . . . . . . . 28
7.2.1. Character Changes: Eszett and Final Sigma . . . . . . 28
7.2.2. Character Changes: Zero Width Joiner and Zero
Width Non-Joiner . . . . . . . . . . . . . . . . . . . 29
7.2.3. Character Changes and the Need for Transition . . . . 29
7.2.4. Transition Strategies . . . . . . . . . . . . . . . . 30
7.3. Elimination of Character Mapping . . . . . . . . . . . . . 31
7.4. The Question of Prefix Changes . . . . . . . . . . . . . . 31
7.4.1. Conditions Requiring a Prefix Change . . . . . . . . . 31
7.4.2. Conditions Not Requiring a Prefix Change . . . . . . . 32
7.4.3. Implications of Prefix Changes . . . . . . . . . . . . 32
7.5. Stringprep Changes and Compatibility . . . . . . . . . . . 33
7.6. The Symbol Question . . . . . . . . . . . . . . . . . . . 33
7.7. Migration between Unicode Versions: Unassigned Code
Points . . . . . . . . . . . . . . . . . . . . . . . . . . 35
7.8. Other Compatibility Issues . . . . . . . . . . . . . . . . 36
8. Name Server Considerations . . . . . . . . . . . . . . . . . . 37
8.1. Processing Non-ASCII Strings . . . . . . . . . . . . . . . 37
8.2. Root and Other DNS Server Considerations . . . . . . . . . 37
9. Internationalization Considerations . . . . . . . . . . . . . 38
10. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 38
10.1. IDNA Character Registry . . . . . . . . . . . . . . . . . 38
10.2. IDNA Context Registry . . . . . . . . . . . . . . . . . . 39
10.3. IANA Repository of IDN Practices of TLDs . . . . . . . . . 39
11. Security Considerations . . . . . . . . . . . . . . . . . . . 39
11.1. General Security Issues with IDNA . . . . . . . . . . . . 39
12. Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . 39
13. Contributors . . . . . . . . . . . . . . . . . . . . . . . . . 40
14. References . . . . . . . . . . . . . . . . . . . . . . . . . . 40
14.1. Normative References . . . . . . . . . . . . . . . . . . . 40
14.2. Informative References . . . . . . . . . . . . . . . . . . 411. Introduction 🌸
1.1. Context and Overview 💅
Internationalized Domain Names in Applications (IDNA) is a collection of standards that lets client apps turn some mnemonic strings written in Unicode into an ASCII-compatible encoding form, called "ACE" 🌈. The result is a valid DNS label that only uses LDH syntax (see the Definitions document [RFC5890]) ✨. The specific kind of ACE label IDNA uses is called an "A-label" 🎀.
Here's the best part, bestie 😭: a client can look up an exact A-label in the existing DNS. That means A-labels don't need any DNS extensions, DNS server upgrades, or updates to low-level client libraries 🙌. You can spot an A-label by its "xn--" prefix, which sits in front of the characters produced by the Punycode algorithm [RFC3492] 🔍. So a user application can recognize an A-label and convert it back into Unicode (or some local coded character set) to display it 🦋.
On the registry side, IDNA lets a registry offer Internationalized Domain Names (IDNs) for registration as A-labels 📝. A registry may offer any subset of valid IDNs, and it may apply any restrictions or bundling (that's grouping similar labels together in one registration) that fit its context 🧺. Registration is sometimes discussed separately from lookup, and it comes with a few specific requirements that don't apply to lookup at all 👑.
DNS clients and registries have different requirements for handling IDNs ⚖️. Registries are urged to register only exact, valid A-labels, while clients might do some mapping to turn otherwise-invalid user input into a valid A-label 💜.
The first version of IDNA was published in 2003 and is called IDNA2003 here, to contrast it with the current version, IDNA2008 (named after the year the IETF started working on it) 📅. IDNA2003 is made up of four documents: the IDNA base specification [RFC3490], Nameprep [RFC3491], Punycode [RFC3492], and Stringprep [RFC3454] 📚. The current set, IDNA2008, doesn't depend on any IDNA2003 specification except the one for Punycode encoding 🙌. When the text says "IDNA2008", "these specifications", or "these documents", it means the entire IDNA2008 set listed in a separate Definitions document [RFC5890] 🗂️. The characters that are valid in A-labels are identified by rules in the Tables document [RFC5892], but validity can be derived from the Unicode properties of those characters, with a very few exceptions 🧠.
DNS labels have traditionally been matched case-insensitively (as described in the DNS specifications [RFC1034][RFC1035]) 🔤. IDNA2003 kept that convention with a case-folding operation that generally maps capital letters to lowercase ones 🔡. But here's the tea ☕: if case rules get enforced for one language, another language can sometimes lose the ability to treat two characters separately 💀. Case-insensitivity is handled a little differently in IDNA2008 🌷.
IDNA2003 only used Unicode version 3.2 🕰️. To keep up with new characters added in newer Unicode versions, IDNA2008 decouples its rules from any particular Unicode version 🦄. Instead, the attributes of new characters in Unicode, plus a small number of exception cases, decide how and whether those characters can be used in IDNA labels 🧚.
This document gives informational context for IDNA2008, including terminology, background, and policy discussions 📖. It contains no normative material 🙈. The specifications for conformance to the IDNA2008 protocols appear entirely in the other documents in the series 💖.
1.2. Terminology 🎀
Terminology for IDNA2008 lives in the Definitions document [RFC5890] 📚. That document also has a road map to the IDNA2008 document collection 🗺️. No attempt should be made to understand this document without the definitions and concepts found there 🫶.
1.2.1. DNS "Name" Terminology 🎀
okay bestie, so the word "name" in DNS land is kinda confusing once IDNs enter the chat 😭. People talk about DNS labels like they're words or phrases from whatever natural language they speak. Historically, a ton of DNS "names" were mnemonics, little memory hooks that point to some concept, object, or organization 🧠. They're usually rooted in a language because humans think in language-based ways. BUT since they're mnemonics, they don't have to follow the spelling rules of any language. Nobody's requiring them to be legit "words" 🙅
This matters SO much 💅 because the realistic goal of an IDN effort isn't to cram the entire Klingon (or whatever language you love) novel into DNS labels 📚💀. The goal is to make a usefully broad range of mnemonics, written as naturally as possible, across a super wide range of scripts ✨.
1.2.2. New Terminology and Restrictions 🦋
IDNA2008 brings in new vocab 📖. The precise definitions live in the Definitions document for U-label, A-Label, LDH label (the stuff every valid pre-IDNA hostname was already following), Reserved LDH label (R-LDH label), XN-label, Fake A-label, and Non-Reserved LDH label (NR-LDH label).
There's also a new term, "putative label," for a label that might look like it meets certain definition constraints but hasn't been properly tested for validity yet 🧪.
These definitions are also illustrated in Figure 1 of the Definitions document. R-LDH labels have -- in the third and fourth character positions from the beginning of the label. In IDNA-aware applications, only a subset of these reserved labels is allowed, namely the A-label subset. A-labels are the R-LDH labels that begin with the case-insensitive string xn--. Labels with that prefix that aren't otherwise valid get sorted into the "Fake A-label" category 🚫. Non-Reserved labels (NR-LDH labels) are implicitly valid since they do not bear any resemblance to the labels specified by IDNA 🫶.
The creation of the Reserved-LDH category is required for three reasons: ✨
- to prevent confusion with pre-IDNA coding forms;
- to permit future extensions that would require changing the prefix, no matter how unlikely those might be (see Section 7.4); and
- to reduce the opportunities for attacks via the Punycode encoding algorithm itself.
As with other documents in the IDNA2008 set, this document uses the term "registry" to describe any zone in the DNS. That term, and the terms "zone" or "zone administration," are interchangeable 🌸.
1.3. Objectives 🎯
These are the main objectives in revising IDNA, and they're giving glow-up energy:
- Use a more recent version of Unicode and allow IDNA to be independent of Unicode versions, so IDNA2008 doesn't need an update every time implementations adopt code points from new Unicode versions 🌈.
- Fix a very small number of code point categorizations that have turned out to cause problems in the communities that use those code points 🔧.
- Reduce the dependency on mapping in favor of valid A-labels. This means pre-mapped forms that aren't valid IDNA labels will show up less often in various contexts 📉.
- Fix some details in the bidirectional code point handling algorithms 🔄.
1.4. Applicability and Function of IDNA 🌍
The IDNA specification solves the problem of extending the repertoire of characters that can be used in domain names so it includes a large chunk of the Unicode repertoire 💖.
IDNA does not extend DNS. Instead, applications (and, by implication, users) still see an exact-match lookup service. Either there's a single name that matches exactly (subject to the base DNS requirement of case-insensitive ASCII matching) or there's no match 🙃. This model has served existing apps well, but with or without internationalized domain names, it means users have to know the exact spelling of domain names they type into things like web browsers and mail user agents. The bigger character repertoire potentially makes the set of misspellings larger, especially since in some cases the same appearance (for example, on a business card) might visually match several Unicode code points or several sequences of code points 👀.
The IDNA standard does not require any applications to conform to it, nor does it retroactively change those applications 🫡. An application can elect to use IDNA to support IDNs while maintaining interoperability with existing infrastructure. For applications that want to use non-ASCII characters in public DNS domain names, IDNA is the only option defined at the time this specification is published. Adding IDNA support to an existing application entails changes to that application only, and leaves room for flexibility in front-end processing and, more specifically, in the user interface (see Section 6) 💅.
A great deal of the IDN discussion has focused on transition issues and how IDNs will work in a world where not all the components have been updated 🚧. Proposals that weren't chosen by the original IDN Working Group would have depended on updating user applications, DNS resolvers, and DNS servers so a user could apply an internationalized domain name in any form or coding acceptable under that method. While processing must be performed prior to or after access to the DNS, IDNA requires no changes to the DNS protocol, any DNS servers, or the resolvers on users' computers 🎉.
IDNA allows the graceful introduction of IDNs not only by avoiding upgrades to existing infrastructure (such as DNS servers and mail transport agents), but also by allowing some limited use of IDNs in applications through the ASCII-encoded representation of labels containing non-ASCII characters 🍓. Those names are user-unfriendly to read and type, so they're not optimal for user input 😵. But they can be used as a last resort to allow rudimentary IDN usage. For example, they might be the best choice for display if it were known that relevant fonts weren't available on the user's computer 🔤. To allow user-friendly input and output of IDNs, and to accept some characters as equivalent to those processed according to the protocol, applications need to be modified to conform to this specification 🧚.
This version of IDNA uses the Unicode character repertoire for continuity with the original version of IDNA 💜.
1.5. Comprehensibility of IDNA Mechanisms and Processing 🧠
One goal of IDNA2008 is to improve general understanding of how IDNA works, what characters are permitted, and what happens to them. The main goal of reducing the dependency on mapping helps with this too 💡. Comprehensibility and predictability for users and registrants are important design goals here, and end-user applications have an important role to play in increasing that comprehensibility 🔍.
Any system that tries to handle international characters runs into some common problems. For example, a User Interface (UI) can't display a character if no font containing that character is available 😩. In some cases, internationalization enables effective localization while maintaining some global uniformity, but it loses some universality 🌐.
It's hard even to suggest how end-user applications should cope when characters and fonts aren't available. Display functions are rarely controlled by the kinds of applications that would call upon IDNA, so suggestions like that will rarely be very effective 🤷♀️.
Conversion between local character sets and normalized Unicode, if needed, is part of these user interface issues. Those conversions add complexity to a system that doesn't use Unicode as its primary (or only) internal character coding system 🌀. If a label is converted to a local character set that doesn't have all the needed characters, or that uses different character-coding principles, the user interface program may have to add special logic to avoid or reduce loss of information 🛠️.
The major difficulty may lie in accurately identifying the incoming character set and applying the correct conversion routine. Even trickier, the local character coding system could be based on conceptually different assumptions than the ones Unicode uses (for example, choices of font encodings used for publications in some Indic scripts). Those differences may not easily yield unambiguous conversions or interpretations, even if each coding system is internally consistent and adequate to represent the local language and script 📚.
IDNA2008 shifts responsibility for character mapping and other adjustments from the protocol (where it lived in IDNA2003) to pre-processing that happens before IDNA itself is invoked 🔀. The intent is that this change leads to greater usage of fully-valid A-Labels or U-labels in display, transit, and storage, which should aid comprehensibility and predictability ✨. A careful look at pre-processing raises questions about what that pre-processing should do and at what point it becomes harmful; how universally consistent pre-processing algorithms can be; and how to stay compatible with labels prepared in an IDNA2003 context 🤔. Those issues are discussed in Section 6 and in the Mapping document [IDNA2008-Mapping] 📖.
2. Processing in IDNA2008 📬
IDNA2008 separates Domain Name Registration and Lookup in the protocol specification (RFC 5891, Sections 4 and 5 [RFC5891]). Most steps in the two processes are similar, but the separation reflects current practice, where per-registry (DNS zone) restrictions and special processing are applied at registration time but not during lookup 🗂️. Another significant benefit is that the separation makes it easier to incrementally add permitted character groups, which helps avoid freezing on one particular version of Unicode 🧊.
The actual registration and lookup protocols for IDNA2008 are specified in the Protocol document 📜.
3. Permitted Characters: An Inclusion List 🌟
IDNA2008 adopts the inclusion model. A code point is assumed to be invalid for IDN use unless it's included as part of a Unicode property-based rule or, in rare cases, included individually by an exception 🚪. When an implementation moves to a new version of Unicode, the rules may indicate new valid code points 🆕.
This section gives an overview of the model used to establish the algorithm and character lists of the Tables document [RFC5892], and it describes the names and applicability of the categories used there 🗺️. Note that including a character in the PROTOCOL-VALID category group (Section 3.1.1) doesn't mean it can be used indiscriminately. Some characters are associated with contextual rules that must be applied as well ⚠️.
The information in this section is provided to make the rules, tables, and protocol easier to understand 🫶. The normative generating rules that match this informal discussion appear in the Tables document, and the rules that actually determine what labels can be registered or looked up are in the Protocol document 💖.
3.1. A Tiered Model of Permitted Characters and Labels ✨
Okay so, moving to an inclusion model means writing a brand new list of characters that are allowed in IDNs 🌸 In IDNA2003, whether a character was valid never depended on context, and it was fixed forever (or until the standard got replaced, whichever came first). The problem is that context-free rules turned out to be impractical. Some characters, especially the ones Unicode calls "Join_Controls", are needed to write some scripts properly, but they have no visible effect in other scripts 🫠 IDNA2003 just banned them entirely by throwing them out. Now there's a consensus that, under certain conditions, these "joiner" characters are genuinely needed for useful mnemonics in some languages and scripts 🧚 Context-dependent rules basically exist to handle characters that would otherwise be banned outright, the ones that get used or perceived differently from script to script, and they let the standard be applied more appropriately when a string isn't handled the same way everywhere 💅
IDNA2008 sorts every possible Unicode code point into one of four categories: PROTOCOL-VALID, CONTEXTUAL RULE REQUIRED, DISALLOWED, and UNASSIGNED 🎀
3.1.1. PROTOCOL-VALID 💖
Characters in the PROTOCOL-VALID category (often shortened to PVALID) are allowed in IDNs. Their use can still be restricted by rules about the context they show up in, or by other rules that apply to the whole label they're embedded in. For example, any label containing a character from this category that has the "right-to-left" property must be used in context with the Bidi rules [RFC5893]. The name is meant to stress that being in this category does not mean a given registry has to accept every character in it 👑 Registries still get to use their judgment about which labels they'll accept, and they're expected to keep their rules consistent with those calls (see the Protocol document [RFC5891] and Section 3.3) 🫶
Characters placed in PROTOCOL-VALID are expected to never be removed from it or reclassified. Sure, characters could theoretically be removed from Unicode, but that would clash with the Unicode stability principles (see UTR 39: Unicode Security Mechanisms [Unicode52], Appendix F), so it should never happen 🔒
3.1.2. CONTEXTUAL RULE REQUIRED 🦋
Some characters aren't great for general use in IDNs, but they're necessary for plausible support of certain scripts 🌈 The two most famous examples are the ZERO WIDTH JOINER and ZERO WIDTH NON-JOINER characters (ZWJ, U+200D and ZWNJ, U+200C). Other characters might need special handling too, since they'd otherwise be DISALLOWED (usually because Unicode files them under punctuation or special symbols), but they still need to be allowed in limited contexts. Some characters get this treatment because they're a real danger for producing misleading labels, or for causing unacceptable ambiguity when labels are matched and interpreted 👀
3.1.2.1. Contextual Restrictions 🧠
Characters with contextual restrictions are marked CONTEXTUAL RULE REQUIRED and each one is tied to a rule. That rule decides whether the character is valid in a particular string, and also whether the rule itself gets applied at lookup as well as at registration 🍓
Here's the split: characters that indicate or prohibit joining (and ones similar to them) are known as CONTEXT-JOINER, or CONTEXTJ. Other characters that need contextual treatment are CONTEXT-OTHER, or CONTEXTO. Only the former need full testing at lookup time ✨
Quick reality check, though: these contextual rules cannot prevent every confusing or problematic use of the relevant characters 🙈 What they can do is confine those characters to scripts (and narrower contexts) where zone administrators know enough about them to deal with them properly 💜
For example, a registry working with an Indic script that needs ZWJ and/or ZWNJ as part of its writing system is expected to understand where those characters have a visible effect and where they don't, and to write its registration rules to match 🪷 A registry that mostly handles Latin or Cyrillic script might not even be aware the characters exist, let alone what happens when you embed them in labels from those scripts 😭 So that kind of registry should avoid accepting registrations containing those characters, at least in labels using characters from the Latin or Cyrillic scripts 🚫
3.1.2.2. Rules and Their Application 📜
Rules come with descriptions such as "Must follow a character from Script XYZ", "Must occur only if the entire label is in Script ABC", or "Must occur only if the previous and subsequent characters have the DFG property" 🌟 The actual rules may be DEFINED or NULL. If a rule is present, it can have the value "True" (the character may be used in any position in any label), the value "False" (the character may not be used in any label), or it can be a set of procedural rules that specify the context where the character is permitted 🎀
Here's a plot twist: it's way easier to spot these characters than to know whether they're actually needed in IDNs, or how to write exactly the right rule for each one. So a rule can have a null value in a given version of the tables. Characters tied to null rules aren't allowed to appear in putative labels for either registration or lookup 🙅♀️ That said, a later version of the tables might have a non-null rule, so stay tuned 💅
The actual rules and their descriptions live in Sections 2 and 3 of the Tables document [RFC5892]. That document also covers how to set up a registry for future rules 🗂️
3.1.3. DISALLOWED 🚫
Some characters just don't belong in IDNs, so they're excluded for both registration and lookup. IDNA-conforming applications doing name lookup should check that these characters are absent, and if they're present, the label strings should be rejected rather than converted to A-labels and looked up 🛑 Some of these characters are a problem for IDNs in their own right (like the FRACTION SLASH, U+2044), while others (like the various HEART symbols, e.g., U+2665, U+2661, and U+2765, see Section 7.6) simply fall outside the conventions for typical identifiers, which are basically letters and numbers 💖
Obviously this category would also include any code points that were removed entirely from Unicode, if that ever happened 🧩
Characters placed in DISALLOWED are expected to never be removed from it or reclassified. If a character gets classified as DISALLOWED by mistake, and the mistake is serious enough, there are only two options: introduce a new code point into Unicode and classify it as PROTOCOL-VALID, or have the IETF accept the big costs of an incompatible change and replace the relevant RFC with one that includes appropriate exceptions 😬
There is provision for exception cases, but in general, characters go into DISALLOWED if they fall into one or more of these groups:
- The character is a compatibility equivalent for another character. In slightly more precise Unicode terms, applying Normalization Form KC (NFKC) to the character gives you some other character 🪞
- The character is an uppercase form, or some other form that gets mapped to another character by Unicode case folding 🔠
- The character is a symbol or punctuation form, or more generally, something that is not a letter, a digit, or a mark used to form a letter or digit 🎭
3.1.4. UNASSIGNED 🌙
For convenience when processing and building tables, code points that don't have assigned values in a given version of Unicode are treated as belonging to a special UNASSIGNED category 🧁 These code points are prohibited in labels to be registered or looked up. This category differs from DISALLOWED in one big way: code points leave it when they get assigned in a later version of Unicode, at which point they're sorted into one of the other categories as appropriate 🦄
The reason for restricting UNASSIGNED characters is that the properties of those code points can't be fully known until actual characters are assigned to them 🤷♀️ Imagine an UNASSIGNED code point got included in a label being looked up, and then later it was assigned to a character that needed some contextual rules. Then un-updated IDNA-aware software might still allow lookup of labels containing that previously unassigned character, while updated software might restrict the same label at lookup, depending on the contextual rules. Super confusing 😵💫 So it should be clear that an UNASSIGNED character should never be permitted in a label to be registered as part of a domain name ⛔
3.2. Registration Policy
okay so these recommendations cannot and should not define registry policies 🙈 that's the registries' call, but they should totally come up with extra restrictions when needed to cut down on confusion and other problems 💅 For example, it's generally believed that labels mixing characters from more than one script are a bad practice, although there may be some important exceptions to that principle ✨ Some registries might choose to limit registrations to characters from a very small number of scripts 🌸 For lots of scripts, variant techniques can be super helpful for reducing the problems users might notice 🦋 Think of the JET specification for the CJK script [RFC3743] and its generalization [RFC4290], plus the tables from the Chinese Domain Name Consortium [RFC4713] that show how this works for Chinese 🎀
In general, users win when registries only allow characters from scripts that the registry or its advisers really understand 💖 If a registry wants to cut down on confusion by disallowing characters from historic writing systems, or characters that only show up in specialized, highly technical contexts, there's useful info in Section 2.4 (Specific Character Adjustments) of Unicode Identifier and Pattern Syntax [Unicode-UAX31], especially Table 4 (Candidate Characters for Exclusion from Identifiers) 👑 Section 3.1 (General Security Profile for Identifiers) in Unicode Security Mechanisms [Unicode-UTS39] is also worth a look 🧚
The requirement in Section 4.1 of the Protocol document [RFC5891] that registration procedures use only U-labels and/or A-labels is there so registrants are fully aware of exactly what they're registering, and so the canonical forms get used 🫶 It is not meant to say that registrants have to supply characters in some particular code sequence 🙅 How registrants enter their input, and how that's managed, is part of the relationship between registrants and registrars and between registries and registrars. That stuff is outside the scope of these standards 💜
It's also worth stressing that these policy principles apply at every level of the DNS, not just top level domain (TLD) or second level domain (SLD) registrations 🌈 Even a totally basic "anything is permitted that is valid under the protocol" policy is helpful, because it lets users and application developers know what to expect ⭐
3.3. Layered Restrictions: Tables, Context, Registration, and Applications
Okay this part is so important bestie 🔥 IDNA2008's character rules come from the realization that there's no single magic bullet for security, confusability, or any of the other issues tied to IDNs 😭 Instead, the specifications define a bunch of different approaches. The character tables are the first mechanism. Protocol rules about how those characters get applied or restricted in context are the second. Put those two together and you've got the limits of what the protocol can do 💅 As Section 3.2 talked about, registries are expected to restrict what they let people register, coming up with and using rules designed to balance confusion and risk on one side against maximum expressiveness in mnemonics on the other ✨
On top of that, user interface programs have a big role to play. They can warn people about label forms that look problematic, based on what they know about local contexts and conventions 🦋 Still, no approach based on naming or identifiers alone can protect against every single threat 🥺
4. Application-Related Issues
4.1. Display and Network Order
Domain names are always transmitted in network order, which is the order the code points are sent in over protocols 🧚 But they might have a different display order, meaning the order the code points show up on a screen or on paper 👀 When a domain name has characters that are normally written right to left, display order can be affected, though network order stays the same 💫 It gets even messier when left-to-right and right-to-left labels sit right next to each other inside one domain name 🙃 Deciding the display order is ultimately up to user agents, like Web browsers, mail clients, hosted Web applications, and many more, which can be super localized 🌍 So should a domain name abc.def, where both labels are written in right-to-left scripts, be displayed as fed.cba or cba.fed? Apps out there today are already all over the place, and you can find examples of both choices 😭
Things change again when an IDN shows up in an Internationalized Resource Identifier (IRI) [RFC3987] 🌸 An IRI or an internationalized email address has stuff in it besides the domain name. For example, IRIs include protocol identifiers and field delimiter syntax like http:// or mailto:, while email addresses use the @ to split local parts from domain names 💌
An IRI in network order starts with http:// followed by domain labels in network order, so you get http://abc.def 🎀
User interface programs aren't required to display and allow input of IRIs directly, but they often do anyway 💅 Implementers have to decide whether the overall direction of these strings will always be left to right, or right to left, for an IRI or email address 🤔 The natural order for someone typing a domain name on a right-to-left system is fed.cba. So should the right-to-left (RTL) user interface reverse the entire domain name every time someone types one? And does that change if the user types http:// right before a domain name, which kind of implies they're starting at the beginning of the network-order IRI? 🧐 Back in the 1980s and 1990s, systems mixed up labels read in network order (left to right) with systems that read those labels right to left, and experience from that era says you'd predict a LOT of confusion 💀
If every implementation of every application makes its own calls on these issues, users are going to build up heuristics that sometimes break when they switch apps 😩 Some display order conventions, if people voluntarily adopt them, would definitely cut down on confusion 🫶 But suggesting those conventions is beyond the scope of these specifications, so that's a bit of a plot twist 🌈
4.2. Entry and Display in Applications ✨
okay so apps can accept and display domain names using any character set or character coding system 🌈. The IDNA protocol doesn't necessarily change the interface between users and apps 🦋. An IDNA-aware app can take in and show internationalized domain names in two formats: the internationalized character set(s) the app supports (basically, a proper local representation of a U-label), and an A-label 💖.
Apps may show A-labels, but they're encouraged not to, unless it's for a special purpose, maybe debugging, or because the display can't handle anything else 🙈. Generally, apps should let users type A-labels in, but never really push them. A-labels are opaque and ugly, and malicious variations on them are not easy for users to spot 💀. So where possible, only show them when they're absolutely necessary. Since IDN labels can be rendered as either an A-label or a U-label, the app can reasonably give users an option to pick their preferred display method. Showing the U-label should normally be the default 💅.
Domain names get stored and moved around in tons of places. They show up in documents like mail messages and web pages, and they travel through lots of protocols, like the control commands of SMTP and its message body parts, plus the headers and body content in HTTP 📬. It's super important to remember that domain names appear both in domain name slots and inside the content passed over protocols, and it would be really helpful if protocols clearly spelled out what their domain name slots are 🧚.
In protocols and document formats that define how charsets get specified or negotiated, labels can be encoded in any charset the protocol or format allows 🎀. If a protocol or format only allows one charset, the labels must be given in that charset. Of course, not every charset can properly represent every label 😭. If a U-label can't be displayed in its entirety, the only option (without losing information) might be to display the A-label instead.
Where a protocol or document format allows IDNs, labels should use whatever character encoding and escape mechanism that protocol or format uses in the local environment. This exists to prevent weird situations where, say, UTF-8 domain names end up embedded in text that's otherwise in some other character coding 🫶.
All protocols that use domain name slots (see Section 2.3.2.6 in the Definitions document [RFC5890]) already know how to handle domain names in the ASCII charset. So A-labels can be handled inherently by those protocols 🌸.
IDNA2008 doesn't specify required mappings between one character or code point and others. There's a longer chat about mapping issues in Section 6, and specific recommendations are in the Mapping document [IDNA2008-Mapping]. In general, IDNA2008 prohibits characters that would be mapped to others by normalization or other rules. For example, mathematical characters based on Latin letters are accepted as input to IDNA2003, but they're prohibited in IDNA2008. Same deal with uppercase characters, double-width characters, and other variations: they're prohibited as IDNA input, even though mapping them as needed in user interfaces is strongly encouraged 👑.
Because the rules in the Tables document [RFC5892] mean that only strings not transformed by NFKC are valid, an app that chooses to do NFKC normalization before lookup is safe. That operation will never make the app unable to look up a valid string 🔥. However, as discussed above, the app can't guarantee that any other app will do that same mapping, so it should only be used with caution and for informed users ⭐.
In lots of cases, these prohibitions shouldn't affect what the user can type as lookup input. It's totally reasonable for systems with user interfaces to do some character mapping that fits the local environment, and this would normally happen before IDNA is actually invoked. At least conceptually, that mapping counts as part of the Unicode conversions discussed above and in the Protocol document [RFC5891]. But those changes stay local, only in environments where users will clearly understand the character forms are equivalent. For interchange between systems, it matters way more that U-labels and A-labels can be mapped back and forth without losing information 🦋.
One very specific, very important example of this strategy is case folding 💜. In the ASCII-only DNS, names are looked up and matched in a case-independent way, but no actual case folding happens. Names can be placed in the DNS in uppercase, lowercase, or any mix, and that form is preserved and returned in queries and so on. IDNA2003 faked that behavior for non-ASCII strings by doing case folding at registration time (so you only ever got lowercase IDNs in the DNS) and again when names were looked up.
As suggested earlier in this section, it seems desirable to do as little character mapping as possible, as long as Unicode works correctly. For example, Normalization Form C (NFC) mapping to resolve different codings for the same character is still necessary, even though the specifications require it be done before invoking the protocol. The goal is to make the mapping between A-labels and U-labels idempotent. Case mapping is no exception to this principle 🌈. If only lowercase characters can be registered in the DNS (meaning they're present in a U-label), then IDNA2008 should prohibit uppercase characters as input, even though user interfaces should probably map those characters. A few other considerations back this up. For example, in ASCII case mapping for individual characters, uppercase(character) is always equal to uppercase(lowercase(character)). That may not be true with IDNs. In some scripts that have case distinctions, a few characters don't have counterparts in one case or the other. The relationship between uppercase and lowercase can even depend on the language, with different languages (or even the same language in different areas) expecting different mappings 🙈. User interface programs can meet the expectations of users who are used to the case-insensitive DNS by doing case folding before IDNA processing. But the IDNA procedures themselves should neither require such mapping nor expect it when it isn't natural to the localized environment 💅.
4.3. Linguistic Expectations: Ligatures, Digraphs, and Alternate Character Forms ✨
Users have expectations about character matching or equivalence that come from their own languages and their orthography (basically, how their language is spelled out). These expectations might not always be met in a global system, especially when multiple languages use the same script but with different conventions 🌸. Some examples:
- A Norwegian user might expect a label with the ae-ligature to be treated as the same label as one using the Swedish spelling with a-diaeresis, even though applying that mapping to English would be astonishing to users 😭.
- A German user might expect a label with an o-umlaut and a label that had "oe" substituted (but was otherwise the same) to be treated as equivalent, even though that substitution would be a clear error in Swedish 💀.
- A Chinese user might expect automatic matching of Simplified and Traditional Chinese characters, but applying that matching for Korean or Japanese text would create considerable confusion 🙈.
- An English user might expect "theater" and "theatre" to match 🍓.
A bunch of languages use alphabetic scripts where a single phoneme gets written with two characters, called a "digraph". Examples are the "ph" in "pharmacy" and "telephone". (These characters can also appear consecutively without forming a digraph, as in "tophat".) Certain digraphs may be shown typographically by setting the two characters closer together than they'd be if used consecutively to represent different phonemes. Some digraphs are fully joined as ligatures. For example, the word "encyclopaedia" is sometimes set with a U+00E6 LATIN SMALL LIGATURE AE 🦋. When ligature and digraph forms have the same interpretation across all languages that use a given script, applying Unicode normalization usually resolves the differences and makes them match ✨. When they have different interpretations, matching has to use other methods (presumably chosen at the registry level), or users have to be educated that matching won't happen 👑.
This problem shows up clearly with the many words in the Norwegian language, where the "ae" ligature is the 27th letter of a 29-letter extended Latin alphabet. It's equivalent to the 28th letter of the Swedish alphabet (which also has 29 letters), U+00E4 LATIN SMALL LETTER A WITH DIAERESIS. You can't substitute "ae" for that one according to current orthographic standards 😤. That character (U+00E4) is also part of the German alphabet, where, unlike in the Nordic languages, the two-character sequence "ae" is usually treated as a totally acceptable alternate spelling for the "umlauted a" character. The inverse isn't true, though, and those two characters can't necessarily be combined into an "umlauted a" 🙈. This also applies to another German character, the "umlauted o" (U+00F6 LATIN SMALL LETTER O WITH DIAERESIS). For example, it can't be used when writing the name of the author "Goethe" 🖋️. It's also a letter in the Swedish alphabet, where (like the "a with diaeresis") it can't be correctly represented as "oe". And in the Norwegian alphabet it's represented not as "o with diaeresis" but as "slashed o", U+00F8 💜.
Some ligatures that have their own code points in Unicode got special handling in IDNA2003, and they now cause extra problems during the transition. See Section 7.2 🌈.
Additional cases with alphabets written right to left are described in Section 4.5 ⭐.
Picking a matching and comparison algorithm often needs information about the language being used, the context, or both. That information isn't available to IDNA or the DNS 🫶. So IDNA2008 makes no attempt to treat combined characters in any special way. A registry that knows the language context labels are being registered in, and where that language sometimes (or always) treats the two-character sequences as equivalent to the combined form, should give serious consideration to applying a "variant" model [RFC3743][RFC4290] or to prohibiting registration of one of the forms entirely. That helps cut down on the user confusion and fraud that could happen if the related strings were registered to different parties 🔥.
4.4. Case Mapping and Related Issues 🎀
okay so in the DNS, ASCII letters are stored with their case preserved 💅 Matching during the query process is case-independent, but none of the info that might be represented by choices of case has been lost ✨ That model has been accidentally helpful, bestie, because people have made DNS labels by smooshing words (or parts of words) together, and case has often been used to tell the pieces apart and make the labels more memorable 🦋 iconic, honestly.
Since DNS servers do not get involved in parsing IDNs, they cannot do case-independent matching 😭 So keeping the cases separate in lookup or registration, and doing matching at the server, is not feasible with IDNA or any similar approach. Matching of characters that are considered to differ only by case must be done, if desired, by programs invoking IDNA lookup, even though it wasn't done by ASCII-only DNS clients 🙈 That situation was recognized in IDNA2003, and nothing in IDNA2008 fundamentally changes it or could do so. In IDNA2003, all characters are case folded and mapped by clients in a standardized step ✨
Even in scripts that generally support case distinctions, some characters do not have uppercase forms 🌸 For example, the Unicode case-folding operation maps Greek Final Form Sigma (U+03C2) to the medial form (U+03C3), and maps Eszett (German Sharp S, U+00DF) to "ss". Neither of these mappings is reversible, because the uppercase of U+03C3 is the uppercase Sigma (U+03A3) and "ss" is an ASCII string 💀 IDNA2008 permits, at the risk of some incompatibility, slightly more flexibility in this area by avoiding case folding and treating these characters as themselves 🧚 Approaches to handling one-way mappings are discussed in Section 7.2.
Because IDNA2003 maps Final Sigma and Eszett to other characters, and the reverse mapping is never possible, neither Final Sigma nor Eszett can be represented in the ACE form of IDNA2003 IDN, nor in the native character (U-label) form derived from it 😭 With IDNA2008, both characters can be used in an IDN, so the A-label used for lookup for any U-label containing those characters is now different 💜 See Section 7.1 for a discussion of what kinds of changes might require the IDNA prefix to change. After extended discussions, the IDNABIS Working Group came to consensus that the change for these characters did not justify a prefix change 🫶
4.5. Right-to-Left Text 🌈
Okay, so, in order to be sure that the directionality of right-to-left text is unambiguous, IDNA2003 required that any label in which right-to-left characters appear both starts and ends with them, and that it does not include any characters with strong left-to-right properties (that excludes other alphabetic characters but permits European digits) 🍓 Any other string that contains a right-to-left character and does not meet those requirements is rejected. This is one of the few places where the IDNA algorithms (both in IDNA2003 and in IDNA2008) examine an entire label, not just individual characters 👑 The algorithmic model used in IDNA2003 rejects the label when the final character in a right-to-left string requires a combining mark in order to be correctly represented.
That prohibition is not acceptable for writing systems for languages written with consonantal alphabets to which diacritical vocalic systems are applied, and for languages with orthographies derived from them, where the combining marks may have different functionality 😭 In both cases, the combining marks can be essential components of the orthography. Examples of this are Yiddish, written with an extended Hebrew script, and Dhivehi (the official language of Maldives), which is written in the Thaana script (which is, in turn, derived from the Arabic script) 🌸 IDNA2008 removes the restriction on final combining characters with a new set of rules for right-to-left scripts and their characters. Those new rules are specified in the Bidi document [RFC5893] ✨
5. IDNs and the Robustness Principle 🦋
The "Robustness Principle" is often stated as "Be conservative about what you send and liberal in what you accept" (see, e.g., Section 1.2.2 of the applications-layer Host Requirements specification [RFC1123]). This principle applies to IDNA 💅 When you apply the principle to registries as the source ("sender") of all registered and useful IDNs, registries are responsible for being conservative about what they register and put out on the Internet. For IDNs to work well, zone administrators (registries) must have and require sensible policies about what gets registered (conservative policies!), and they have to implement and enforce them 🧚
Conversely, lookup applications are expected to reject labels that clearly violate global (protocol) rules (no one has ever seriously claimed that being liberal in what is accepted requires being stupid 💀). However, once you get past those global rules and deal with anything sensitive to script or locale, it is necessary to assume that garbage has not been placed into the DNS. In other words, you have to be liberal about what you're willing to look up in the DNS, rather than guessing about whether it should have been permitted to be registered 🫶
If a string cannot be successfully found in the DNS after the lookup processing described here, it makes no difference whether it simply wasn't registered or was prohibited by some rule at the registry 🙈 Application implementers should be aware that where DNS wildcards are used, the ability to successfully resolve a name does not guarantee that it was actually registered. Main character energy for wildcards, but not a guarantee 👑
6. Front-end and User Interface Processing for Lookup 🌸
Domain names may be identified and processed in many contexts. They may be typed in by users themselves, or embedded in an identifier such as an email address, URI, or IRI. They may occur in running text, or be processed by one system after being provided in another ✨ Systems may try to normalize URLs to determine (or guess) whether a reference is valid, or whether two references point to the same object, without actually looking the objects up (comparison without lookup is necessary for URI types that are not intended to be resolved). Some of these goals may be more easily and reliably satisfied than others 💅
While there are strong arguments for any domain name that is placed "on the wire" (transmitted between systems) to be in the zero-ambiguity forms of A-labels, it is inevitable that programs that process domain names will encounter U-labels or variant forms 🦋
An application that implements the IDNA protocol [RFC5891] will always take any user input and convert it to a set of Unicode code points. That user input may be acquired by any of several different input methods, all with differing conversion processes to be taken into consideration (e.g., typed on a keyboard, written by hand onto some sort of digitizer, spoken into a microphone and interpreted by a speech-to-text engine, etc.) 🎀 The process of taking any particular user input and mapping it into a Unicode code point may be a simple one. If a user strikes the "A" key on a US English keyboard, without any modifiers such as the "Shift" key held down, in order to draw a Latin small letter A ("a"), many (perhaps most) modern operating system input methods will produce to the calling application the code point U+0061, encoded in a single octet 🍓
Sometimes the process is somewhat more complicated 😭 A user might strike a particular set of keys to represent a combining macron, followed by striking the "A" key, in order to draw a Latin small letter A with a macron above it. Depending on the operating system, the input method chosen by the user, and even the parameters with which the application communicates with the input method, the result might be the code point U+0101 (encoded as two octets in UTF-8 or UTF-16, four octets in UTF-32, etc.), the code point U+0061 followed by the code point U+0304 (again, encoded in three or more octets, depending upon the encoding used), or even the code point U+FF41 followed by the code point U+0304 (and encoded in some form). These examples leave aside the issue of operating systems and input methods that do not use Unicode code points for their character set 🌈
In every case, applications (with the help of the operating systems on which they run and the input methods used) need to perform a mapping from user input into Unicode code points ✨
IDNA2003 used a model whereby input was taken from the user, mapped (via whatever input method mechanisms were used) to a set of Unicode code points, and then further mapped to a set of Unicode code points using the Nameprep profile [RFC3491]. In this procedure there are two separate mapping steps: first, a mapping done by the input method (which might be controlled by the operating system, the application, or some combination), and then a second mapping performed by the Nameprep portion of the IDNA protocol. The mapping done in Nameprep includes a particular mapping table to re-map some characters to other characters, a particular normalization, and a set of prohibited characters 🧚
Note that the result of the two-step mapping process means that the mapping chosen by the operating system or application in the first step might differ significantly from the mapping supplied by the Nameprep profile in the second step. This has advantages and disadvantages 💜 Of course, the second mapping regularizes what gets looked up in the DNS, making for better interoperability between implementations that use the Nameprep mapping. However, the application or operating system may choose mappings in their input methods, which, when passed through the second (Nameprep) mapping, result in characters that are "surprising" to the end user 🙈
The other important feature of IDNA2003 is that, with very few exceptions, it assumes that any set of Unicode code points provided to the Nameprep mapping can be mapped into a string of Unicode code points that are "sensible", even if that means mapping some code points to nothing (that is, removing the code points from the string). This allowed maximum flexibility in input strings 🫶
The present version of IDNA (IDNA2008) differs significantly in approach from the original version. First and foremost, it does not provide explicit mapping instructions. Instead, it assumes that the application (perhaps via an operating system input method) will do whatever mapping it requires to convert input into Unicode code points 👑 This has the advantage of giving the application flexibility to choose a mapping that is suitable for its user given specific user requirements, and it avoids the two-step mapping of the original protocol. Instead of a mapping, IDNA2008 provides a set of categories that can be used to specify the valid code points allowed in a domain name 🌸
In principle, an application ought to take user input of a domain name and convert it to the set of Unicode code points that represent the domain name the user intends. As a practical matter, of course, determining user intent is a tricky business, so an application needs to choose a reasonable mapping from user input. That may differ based on the particular circumstances of a user, depending on locale, language, type of input method, etc. It is up to the application to make a reasonable choice ✨
7. Migration from IDNA2003 and Unicode Version Synchronization 🦋
7.1. Design Criteria 💅
Okay bestie, so as mentioned above and in the IAB review and recommendations for IDNs [RFC4690], the IDNA2008 design has two main goals 🎯:
- to enable applications to be agnostic about whether they are being run in environments supporting any Unicode version from 3.2 onward.
- to permit incrementally adding new characters, character groups, scripts, and other character collections as they are incorporated into Unicode, doing so without disruption and, in the long term, without "heavy" processes (an IETF consensus process is required by the IDNA2008 specifications and is expected to be required and used until significant experience accumulates with IDNA operations and new versions of Unicode) 🌈
7.1.1. Summary and Discussion of IDNA Validity Criteria ✨
Here's the vibe check on when a label counts as valid under IDNA 🦋 (the actual rules are rigorously defined in the Protocol [RFC5891] and Tables [RFC5892] documents):
- The characters are "letters", marks needed to form letters, numerals, or other code points used to write words in some language. Symbols, drawing characters, and various notational characters are intended to be permanently excluded. There is no evidence that they are important enough to Internet operations or internationalization to justify expanding domain names beyond the general principle of "letters, digits, and hyphen". (Additional discussion and rationale for the symbol decision appears in Section 7.6.) 🔤
- Other than in very exceptional cases, e.g., where they are needed to write substantially any word of a given language, punctuation characters are excluded. The fact that a word exists is not proof that it should be usable in a DNS label, and DNS labels are not expected to be usable for multiple-word phrases (although they are certainly not prohibited if the conventions and orthography of a particular language cause that to be possible) 🙈
- Characters that are unassigned (have no character assignment at all) in the version of Unicode being used by the registry or application are not permitted, even on lookup. The issues involved in this decision are discussed in Section 7.7. 🚫
- Any character that is mapped to another character by a current version of NFKC is prohibited as input to IDNA (for either registration or lookup). With a few exceptions, this principle excludes any character mapped to another by Nameprep [RFC3491]. 💀
The principles above drive the design of rules that are specified exactly in the Tables document. Those rules identify the characters that are valid under IDNA. The rules themselves are normative, and the tables are derived from them, rather than vice versa. So the rules are the source of truth and the tables are just the receipts 🧾✨
7.1.2. Labels in Registration 🎀
Any label registered in a DNS zone must be validated -- i.e., the criteria for that label must be met -- in order for applications to work as intended. This principle is not new! Ever since the DNS was first deployed, zone administrators have been expected to verify that names meet "hostname" requirements [RFC0952] where those requirements are imposed by the expected applications. Other application contexts, such as the later addition of special service location formats [RFC2782], imposed new requirements on zone administrators too 📋. For zones that will contain IDNs, support for Unicode version-independence requires restrictions on all strings placed in the zone. In particular, for such zones (the exact rules appear in Section 4 of the Protocol document [RFC5891]):
- Any label that appears to be an A-label, i.e., any label that starts in
xn--, must be valid under IDNA, i.e., they must be valid A-labels, as discussed in Section 2 above. 🔒 - The Unicode tables (i.e., tables of code points, character classes, and properties) and IDNA tables (i.e., tables of contextual rules such as those that appear in the Tables document) must be consistent on the systems performing or validating labels to be registered. Note that this does not require that tables reflect the latest version of Unicode, only that all tables used on a given system are consistent with each other. 🧩
Under this model, registry tables will need to be updated (both the Unicode-associated tables and the tables of permitted IDN characters) to enable a new script or other set of new characters. The registry will not be affected by newer versions of Unicode, or newly authorized characters, until and unless it wishes to support them. 🌸 The zone administrator is responsible for verifying validity for IDNA as well as its local policies -- a more extensive set of checks than are required for looking up the labels. Systems looking up or resolving DNS labels, especially IDN DNS labels, must be able to assume that applicable registration rules were followed for names entered into the DNS. Trust the process, babe 🫶
7.1.3. Labels in Lookup 🔍
Any application processing a label through IDNA so it can be looked up in a DNS zone is required to (the exact rules appear in Section 5 of the Protocol document [RFC5891]):
- Maintain IDNA and Unicode tables that are consistent with regard to versions, i.e., unless the application actually executes the classification rules in the Tables document [RFC5892], its IDNA tables must be derived from the version of Unicode that is supported more generally on the system. As with registration, the tables need not reflect the latest version of Unicode, but they must be consistent. 📚
- Validate the characters in labels to be looked up only to the extent of determining that the U-label does not contain "DISALLOWED" code points or code points that are unassigned in its version of Unicode.
- Validate the label itself for conformance with a small number of whole-label rules. In particular, it must verify that:
- there are no leading combining marks,
- the Bidi conditions are met if right-to-left characters appear,
- any required contextual rules are available, and
- any contextual rules that are associated with joiner characters (and CONTEXTJ characters more generally) are tested. 🧐
- Do not reject labels based on other contextual rules about characters, including mixed-script label prohibitions. Such rules may be used to influence presentation decisions in the user interface, but not to avoid looking up domain names. 🎨
To clarify the rules about handling characters that require contextual rules: one can have a context-required character (i.e., one that requires a rule) but no rule. In that case, the character is treated the same way DISALLOWED characters are treated, until and unless a rule is supplied. That state is more or less equivalent to "the idea of permitting this character is accepted in principle, but it won't be permitted in practice until consensus is reached on a safe way to use it" 🙃
The ability to add a rule more or less exempts these characters from the prohibition against reclassifying characters from DISALLOWED to PVALID.
And, obviously, "no rule" is different from "have a rule, but the test either succeeds or fails." Totally different energy 💅
Lookup applications that follow these rules, rather than having their own criteria for rejecting lookup attempts, are not sensitive to version incompatibilities with the particular zone registry associated with the domain name, except for labels containing characters recently added to Unicode. 🦄
An application or client that processes names according to this protocol and then resolves them in the DNS will be able to locate any name that is registered, as long as those registrations are valid under IDNA and its version of the IDNA tables is sufficiently up to date to interpret all of the characters in the label. Messages to users should distinguish between "label contains an unallocated code point" and other types of lookup failures. A failure on the basis of an old version of Unicode may lead the user to want to upgrade to a newer version, but will have no other ill effects (this is consistent with behavior in the transition to the DNS when some hosts could not yet handle some forms of names or record types). Honestly iconic of them to be this considerate 🥹✨
7.2. Changes in Character Interpretations 🔄
As a consequence of the elimination of mapping, the current version of IDNA changes the interpretation of a few characters relative to its predecessors. This subsection outlines the issues and discusses possible transition strategies 🧠
7.2.1. Character Changes: Eszett and Final Sigma 🇩🇪
In those scripts that make case distinctions, there are a few characters for which an obvious and unique uppercase character has not historically been available to match a lowercase one, or vice versa. For those characters, the mappings used in constructing the Stringprep tables for IDNA2003, performed using the Unicode toCaseFold operation (see Section 5.18 of the Unicode Standard [Unicode52]), generate different characters or sets of characters. Those operations are not reversible and lose even more information than traditional uppercase or lowercase transformations, but are more useful than those transformations for comparison purposes. Two notable characters of this type are the German character Eszett (Sharp S, U+00DF) and the Greek Final Form Sigma (U+03C2). The former is case folded to the ASCII string "ss", the latter to a medial (lowercase) Sigma (U+03C3). Tiny letters, big drama 😭
7.2.2. Character Changes: Zero Width Joiner and Zero Width Non-Joiner 🧚
IDNA2003 mapped both ZERO WIDTH JOINER (ZWJ, U+200D) and ZERO WIDTH NON-JOINER (ZWNJ, U+200C) to nothing, effectively dropping these characters from any label in which they appeared and treating strings containing them as identical to strings that did not. As discussed in Section 3.1.2 above, those characters are essential for writing many reasonable mnemonics for certain scripts. However, treating them as valid in IDNA2008, even with contextual restrictions, raises approximately the same problem as exists with Eszett and Final Sigma: strings that were valid under IDNA2003 have different interpretations as labels, and different A-labels, than the same strings under this newer version. It's giving "same name, different vibes" and we need to talk about it 👑💜
7.2.3. Character Changes and the Need for Transition 💅✨
okay so the whole reason these characters are causing drama is that IDNA2008 got rid of the mandatory and standardized mappings (case folding and friends) so that A-labels and U-labels stay idempotent 🫶. but then Eszett and Final Sigma are kinda stuck in a bind 😭. if you disallow them, important words and mnemonics can't be spelled in a way that makes orthographic sense, no cap. if you allow them as their own distinct characters, there's zero information loss and registries get more flexibility 🦋, BUT IDNA2003 and IDNA2008 lookups might end up producing different A-labels. it's giving "pick your poison" energy 💀
the Working Group basically said "okay, we know there's going to be incompatibility either way, but it's not a big enough deal to justify a prefix change" 🎀. so Eszett and Final Form Sigma got treated as distinct and Protocol-Valid characters. periodt 👑
now, since these characters get interpreted differently in the older and newer versions of IDNA, transition strategies and policies are a MUST-have 🌸. wait, let me fix that, they are necessary 💜. some stuff can be handled by client programs (the ones doing lookups or causing them to happen), but the DNS is so diverse in how it's used that a lot of the responsibility lands on registries 🧚.
registries, especially the ones maintaining zones for third parties, have to figure out how to roll out a new service without creating confusion or seriously weakening or invalidating existing identifiers 🙈. honestly this is not a new problem. registries dealt with similar stuff when IDNs first showed up (especially for Latin-based scripts, where they could clash with existing labels already written in ASCII using more or less standard conventions) and again when other new string forms got permitted as labels 🌈.
7.2.4. Transition Strategies 🦋
there are a bunch of ways to handle introducing new characters or changing how existing characters are interpreted compared to their mapped forms in the earlier IDNA version. this is messy because the labels after the ToUnicode(ToASCII()) translation in IDNA2003 are still valid, but they don't really tell you what the registrant meant 😭. a string with "ss" could have just been "ss", or it could have been meant to contain an Eszett. a string with lowercase Sigma could have meant Final Sigma (you could guess from position in the string, but people have a long tradition of smooshing words together, so those guesses are unreliable 🙃). and strings without ZWJ or ZWNJ might have been meant to include them 💖.
no preference or claim to completeness here, but these are some of the strategies registries have used before for similar transitions:
- don't let registries hand out the newly available character at all. this could cause lookup failures if someone types a name expecting the IDNA2003 mapping, but it totally eliminates the chance of false matches 🛡️.
- run a "sunrise"-style period where holders of labels with "ss" (Eszett case), lowercase Sigma (Final Sigma case), or labels that might have had ZWJ or ZWNJ in context get priority (and maybe other perks) to register the matching string with Eszett, Final Sigma, or the right zero-width character 🌟.
- do some kind of "variant" setup where registrants get labels with both character forms 👯.
- do a different "variant" setup where registering extra strings that would produce the same A-label under IDNA2003 is either blocked entirely or only allowed for whoever already holds one of those names 🔐.
- ignore it and assume the marketplace or other mechanisms will sort it out 🤷.
either way, a registry (at any level of the DNS tree) that lets labels with these characters get registered, or is even thinking about it, will need to deal with how they relate to existing labels that might conflict. registries that already had lots of labels had to do exactly this when IDNs first launched 🍓.
7.3. Elimination of Character Mapping 🧼
like we talked about at length in Section 6, IDNA2003 (through Nameprep, see Section 7.5) mapped a bunch of characters into related ones. those mappings are no longer requirements in IDNA2008 🙌. these specs strongly prefer that only A-labels or U-labels get used in protocol contexts, and as much as is practical more generally 💜. IDNA2008 does still acknowledge that some mapping when the user types something into a lookup app can be appropriate and desirable 🫶. the issues are covered in Section 6, and specific recommendations live in the Mapping document [IDNA2008-Mapping] 📚.
7.4. The Question of Prefix Changes 🎀
the conditions that would have required changing the IDNA ACE prefix ("xn--", the one used in IDNA2003) were a huge concern for the community 😭. a prefix change would have been clearly necessary if the algorithms got modified in a way that created serious ambiguities during the transition in registrations. this section sums up the working group's conclusions about when a prefix change would have been needed and what it would have meant 🔥.
7.4.1. Conditions Requiring a Prefix Change 🚨
an IDN prefix change would be needed if a given string gets looked up or interpreted differently depending on which version of the protocol or tables is in use 👀. this upgrade would have required a prefix change if, and only if, one of these four conditions were met:
- converting an A-label to Unicode (so, a U-label) would have given one string under IDNA2003 and a different string under IDNA2008 🙃.
- in a significant number of cases, an input string that was valid under both IDNA2003 and IDNA2008 would have produced two different A-labels across the two versions. this is basically the same as the one above, except for a tiny number of edge cases that weren't considered serious enough to justify a prefix change (see Section 7.2) 🤏.
ngl, if the input string was valid under one version and not the other, this condition doesn't apply. see the first item in Section 7.4.2, below 💅.
- a fundamental change to the meaning of the string going into the DNS, for example if someone decided to put language or script info into the encoding alongside the string itself 🌍.
- so many characters got added to Unicode that the Punycode mechanism for block offsets would no longer point to the higher-numbered planes and blocks. this is super unlikely even long term, and definitely not happening in the next several years 🧚.
7.4.2. Conditions Not Requiring a Prefix Change 💖
because of the principles above, none of these changes needed a new prefix:
- prohibiting some characters as input to IDNA. this might make names that were already registered unreachable, but it doesn't change those names 🙈.
- adjusting IDNA tables or actions (including normalization definitions) for characters that were already invalid under IDNA2003 🧹.
- changing the style of the IDNA definition without changing what IDNA actually does 🎨.
7.4.3. Implications of Prefix Changes 💀
a prefix change was possible, but the costs would be seriously high 😭. registries couldn't have converted every IDNA2003 ("xn--") registration to a new form all at once while also syncing that change with lookup-supporting apps 📉. unless every existing registration just got declared invalid (and maybe even then), systems that needed to handle both old-prefix and new-prefix labels would first have to process a label under IDNA2008 rules and look it up, and then, if it wasn't found, process it under IDNA2003 rules and look it up again 🔁. that would probably slow down all IDN processing in the DNS a lot, especially since a fully-qualified name could mix labels registered with the old and new prefixes 🐌. it would also make DNS caching super hard 🙅♀️. on top of that, looking up the same input string as two separate A-labels would create room for confusion and attacks, since those labels could point to different targets and resolve to different DNS entries 🚩.
so, a prefix change should have been avoided if at all possible, and it was 🙌. that meant accepting some IDNA2003 character distinctions as irreversible, and/or giving special treatment to edge cases ✨.
7.5. Stringprep Changes and Compatibility 🧪
the Nameprep specification [RFC3491] is a huge part of IDNA2003, and it's a profile of Stringprep [RFC3454] 🧠. Nameprep is specific to IDNA, but Stringprep is used by a bunch of other protocols too. if Stringprep had been modified by IDNA2008, those changes (meant to improve IDN handling) could have caused problems for non-DNS uses, especially anything touching identification and authentication protocols 😬. several parts of IDNA2008 give interpretations to strings that IDNA2003 prohibited, or prohibit strings that IDNA2003 allowed 🔄. those include the new inclusion info in the Tables document [RFC5892], the reduction in the number of characters permitted as input for registration or lookup (Section 3), and even the changes to how right-to-left strings are handled, as described in the Bidi document [RFC5893] 🔀. IDNA2008 doesn't use Nameprep or Stringprep at all, so there are no side-effect changes to other protocols 🫧.
it's really important to keep IDNA processing separate from processing for various security protocols 🔐, because some constraints that make IDNs smooth and understandable might be unwanted or undesirable in other contexts. for example, good passwords and passphrases have totally different criteria from desirable IDNs: passwords should be hard to guess, while domain names should usually be easy to remember 🧠. similarly, internationalized Small Computer System Interface (SCSI) identifiers and other protocol components are likely to have different requirements than IDNs 💜.
7.6. The Symbol Question 🎀
Okay bestie, here's a big difference between this spec and the original IDNA: IDNA2003 let non-letter symbols into the protocol, like punctuation and line-drawing symbols 🙈 They were always discouraged in practice. The "IESG Statement" about IDNA and every version of the ICANN Guidelines both say that only language characters should be used in labels. This spec disallows symbols entirely 💅 There are a few reasons, and they're honestly so good:
- As we talked about elsewhere, the original IDNA spec assumed that as many Unicode characters as possible should be allowed in IDNs, either directly or by mapping to other characters 🌈 This spec uses an inclusion model instead. It takes the original "hostname" rules (LDH, see the Definitions document [RFC5890]), which have served the Internet really well, and extends them to a Unicode base rather than an ASCII base ✨
- Symbols are way more problematic than letters, because nobody necessarily agrees on whether a particular glyph matches a symbol 😭 There are no uniform naming conventions, and outline, solid, and shaded versions may or may not exist. Like, take a "heart" symbol in a logo that people read as "I love...". The user might read it as "I love..." or "I heart...", but you'd need serious Unicode coding knowledge to realize there's more than one "heart" character (for example U+2665, U+2661, and U+2765) and to know how to describe each one 🫶 This matters most when strings are supposed to be understood or transcribed by a listener after being read out loud.
- A screen reader used by blind Internet users has to read out IDN domain names and maybe retype them on the keyboard 🦋 That gets way more complicated when character names aren't obvious and intuitive to someone who knows the language.
- Here's a simpler example: say someone wants a "heart" or "star" symbol in a label. That's a problem because those names are ambiguous in the Unicode naming system (the real Unicode names need way more qualification). A user or would-be registrant has no way to know, unless they carefully study the code tables, whether a name is ambiguous (for example where there are multiple "heart" characters) or not. And the person looking at the label doesn't know whether to read it out loud as "heart", "love", "black heart", or any of the other examples below 💀
- Honestly, it's even worse than that 😭 A normal, casual user can't tell the difference between the hearts of U+2665 and U+2765, or the stars of U+2606 and U+2729, unless they somehow already know to look for a distinction. There's a white heart (U+2661) and few black hearts. So calling a label "a heart" is hopelessly ambiguous, because all we know is it contains one of several characters that look like hearts or have "heart" in their names. In cities where "Square" is a popular part of a location name, someone might also want a square symbol in a label, and there are way more squares of different flavors in Unicode than there are hearts or stars 🍓
The consequence is that symbols are a really poor basis for reliable communication 💜 Consistent with that, the Unicode standard recommends that strings used in identifiers not contain symbols or punctuation [Unicode-UAX31]. Of course, these symbol problems don't come up with actual pictographic languages and scripts, which would be treated like any other language characters. Don't mix the two up! 🌸
7.7. Migration between Unicode Versions: Unassigned Code Points 🧚
In IDNA2003, labels with unassigned code points get looked up on the assumption that, if they show up in labels and can be mapped and then resolved, the relevant standards must have changed and the registry only allocated assigned values ✨
In IDNA2008, strings containing unassigned code points must not be looked up or registered 🔥 Here's the summary: the status of an unassigned character with regard to the DISALLOWED, PROTOCOL-VALID, and CONTEXTUAL RULE REQUIRED categories can't be evaluated until a character is actually assigned and known. There are a few reasons for this, and the biggest ones are:
- Tests involving the context of characters (for example, some characters being permitted only next to others of specific types) and integrity tests on complete labels are needed. Unassigned code points can't be permitted, because you can't figure out whether particular code points will need contextual rules (and what those rules should be) before characters get assigned to them and their properties are fully understood 👑
- Nobody can know in advance, and with enough reliability, whether a newly assigned code point will be tied to a character that the rules in the Tables document [RFC5892] would disallow (like a compatibility character). In IDNA2003 there was no direct dependency on NFKC. Many entries in Stringprep's tables are based on NFKC, but IDNA2003 only depends on Stringprep. So allocating a compatibility character might produce some odd situations, but it wouldn't be a problem. In IDNA2008, compatibility characters are DISALLOWED unless character-specific exceptions are made. Letting strings with unassigned characters be looked up would break the principle that characters in DISALLOWED are not looked up 🙈
- The Unicode Standard says an unassigned code point normalizes (and, where relevant, case folds) to itself. If that code point is later assigned to a character, especially if the new code point has a combining class that determines where it goes relative to other combining characters, it could normalize to some other code point or sequence 😭
You could argue that these issues aren't important, and that it's better to keep looking up labels even if they contain unassigned characters, because all the important scripts and characters were coded as of Unicode 5.2 (or even earlier), so unassigned code points will only ever be assigned to obscure characters or archaic scripts 💅 Unfortunately, that doesn't look like a safe assumption, for at least two reasons. First, pretty much the same claim of completeness has been made for earlier versions of Unicode. The reality is that a script that's obscure to much of the world may still be super important to the people who use it. Cultural and linguistic preservation principles make it inappropriate to declare a script unimportant in IDNs 🌈 Second, we already have counterexamples, like the relationships that come with the new Han characters being added (whether in the BMP or in Unicode Plane 2) 🦋
Separate from the technical transition issues above, any addition of characters to an existing script to make it easier to use, or to better fit particular languages, may cause transition issues too ✨ Those additions can change the preferred way to write a particular string. Those changes might show up in keyboard transition modules, for example, which would have to differ from the ones for earlier versions of Unicode where the newer characters might not exist 🎀 That creates an inherent transition problem, because attempts to access labels might use either the old or the new conventions. That needs registry action whether or not the older conventions were used in labels. Needing to think about transition mechanisms is built into Unicode evolving to better fit writing systems, and it's independent of how IDNs are represented in the DNS or how transitions between versions of those mechanisms happen. The addition of Malayalam Chillu in Unicode 5.1.0 is a good example of this kind of transition 🧚
7.8. Other Compatibility Issues 🍓
The 2003 IDNA model has several odd artifacts from the context it was developed in 😬 Lots of these, if not all, are potential avenues for exploits, especially if the registration process lets "source" names (names that haven't been processed through IDNA and Nameprep) get registered. For example, IDNA2003 maps the Eszett, a character used in German, into the sequence "ss" instead of keeping it as itself or prohibiting it. So a string with that character that's otherwise ASCII isn't really an IDN (in the U-label sense defined above). After Nameprep maps out the Eszett, the result is an ASCII string, so it doesn't get an xn-- prefix, but the string a user sees looks like an IDN 🙈 IDNA2008 gets rid of this artifact. A character is either permitted as itself or prohibited. Special cases that only make sense in a particular linguistic or cultural context can be handled as localization matters where that's appropriate 💖
8. Name Server Considerations 🌸
8.1. Processing Non-ASCII Strings 💅
Existing DNS servers don't know the IDNA rules for handling non-ASCII forms of IDNs, so they need to be shielded from them 🫶 All the existing channels through which names can get into a DNS server database (for example, master files (as described in RFC 1034) and DNS update messages [RFC2136]) couldn't be IDNA-aware, because they predate IDNA. Other sections of this document provide that shielding by making sure internationalized domain names entering DNS server databases through those channels have already been converted to their equivalent ASCII A-label forms ✨
Because the Registration and Lookup algorithms are kept distinct in Sections 4 and 5 (respectively) of the Protocol document [RFC5891] (a domain name containing only ASCII code points can't be converted to an A-label), there can't be more than one A-label form for any given U-label 👑
As the clarifications to the DNS specification [RFC2181] specify, the DNS protocol explicitly allows domain labels to contain octets beyond the ASCII range (0000..007F), and this document doesn't change that 🦋 Still, even though octets 0080..00FF have a well-defined interpretation in the DNS, many application protocols only support ASCII labels. There's no defined interpretation of those non-ASCII octets as characters, and in particular no case-independent matching for them (see, for example, the clarification on DNS case insensitivity [RFC4343]). If labels containing these octets get returned to applications, unpredictable behavior could result 😭 The A-label form, which can't contain those characters, is the only standard representation for internationalized labels in the DNS protocol 💜
8.2. Root and Other DNS Server Considerations 🌐
okay so IDNs in A-label form are gonna be a little longer than normal domain names 📏, which means the root servers' bandwidth is probably going up by a small amount. bestie, that's not a crisis, it's just a vibe shift 💅
also, queries and responses for IDNs will probably be a bit longer than the ones we're used to, so EDNS0 support [RFC2671] matters even more 🫶 without it, queries and responses might get pushed over to TCP instead of UDP 🙈
9. Internationalization Considerations 🌍
okay this section is my fave 🥹 DNS labels and fully-qualified domain names are mnemonics that help people find and refer to stuff on the Internet ✨ IDNs widen that toolkit so it includes names based on languages and character sets that aren't Western European or Roman-derived 🌸
BUT domain "names" are not, in general, words in any language 💀 the IETF policy on character sets and languages (BCP 18 [RFC2277]) applies to situations where language identification is used to give language-specific contexts 📚 the DNS, by contrast, is global and international and ultimately has nothing to do with languages 🌏
so if you added languages (or similar context) to IDNs in general, or to DNS matching in particular, you'd get context-dependent matching in DNS 😭 that would be a seriously big change to the DNS protocol itself 🚨 it would also mean users would have to know which language goes with a label before they could look it up 🙃 and that knowledge usually isn't available, because lots of labels aren't words in any language, and some are words in more than one 🧚
10. IANA Considerations 📋
this section is the overview of the IANA registries IDNA needs 🗂️ the actual definitions and specs for the first two, which were newly created for IDNA2008, live in the Tables document [RFC5892] 💜 this document describes the registries, but it does not specify any IANA actions 🫶
10.1. IDNA Character Registry 🔤
the big categories "UNASSIGNED", "DISALLOWED", "PROTOCOL-VALID", and "CONTEXTUAL RULE REQUIRED" are sorted out by special categories and rules that are built into the Tables document 🧩 this registry of characters and scripts and their categories is not normative 🙅♀️ but it gets updated for each new version of Unicode and the characters it holds, which makes it super handy for programming and validation 🎀 the details of this registry are specified in the Tables document ✨
10.2. IDNA Context Registry 🧠
IANA has created and currently maintains a list of approved contextual rules for characters that the IDNA Character Registry marks as needing a Contextual Rule (aka the types of rules described in Section 3.1.2) 💅 the details for those rules are in the Tables document 📖
10.3. IANA Repository of IDN Practices of TLDs 🏛️
this registry used to be called the "IANA Language Character Set Registry" or "IANA Script Registry" (both kinda misleading names, ngl) 🙈 IANA keeps it at the request of ICANN 🌷 it's a central documentation repository for the IDN policies used by top level domain (TLD) registries who volunteer to contribute, and it's used alongside ICANN Guidelines for IDN use 💖
it is not an IETF-managed registry 🙅♀️ the protocol changes in this document may call for some revisions to the tables, but IDNA2008 has no direct effect on that registry, and no IANA action is required as a result 🦋
11. Security Considerations 🔐
11.1. General Security Issues with IDNA 🚨
this document is purely explanatory and informational, so it introduces no new security issues ✨ it would, of course, be a poor idea to try implementing from it 😬 an attempt like that would almost certainly lead to interoperability problems, and it might lead to security ones too 💀 a discussion of security issues with IDNA, including some relevant history, is in the Definitions document [RFC5890] 📚
the big warning: don't build your implementation from this document alone. the attempt would almost certainly cause interop problems, and it might cause security problems too 😭
12. Acknowledgments 🫶
the editor and contributors want to thank everyone who gave significant early review comments (before the working group existed), sometimes with actual text attached: Paul Hoffman, Simon Josefsson, and Sam Weiler 💖 some specific ideas came from suggestions, text, or comments about unclear sections from Vint Cerf, Frank Ellerman, Michael Everson, Asmus Freytag, Erik van der Poel, Michel Suignard, and Ken Whistler 🌸 thanks also to Vint Cerf, Lisa Dusseault, Debbie Garside, and Jefsey Morfin for conversations that led to big improvements, and to Ben Campbell, Martin Duerst, Subramanian Moonesamy, Peter Saint-Andre, and Dan Winship for catching specific errors and suggesting fixes 👑
a meeting was held on 30 January 2008 to try to reconcile differences in perspective and terminology about these specifications between the design team and members of the Unicode Technical Consortium 🤝 the discussions at and after that meeting were super helpful in focusing the issues and refining the specs 🎀 the active participants (in alphabetic order, as usual) were Harald Alvestrand, Vint Cerf, Tina Dam, Mark Davis, Lisa Dusseault, Patrik Faltstrom (by telephone), Cary Karp, John Klensin, Warren Kumari, Lisa Moore, Erik van der Poel, Michel Suignard, and Ken Whistler 🦋 thanks to Google for supporting that meeting, and to the participants for their contributions 💜
useful comments and text on the working group versions of the working draft came from many people in the IETF "IDNABIS" working group, and a bunch of document changes came out of that group's mailing list discussions 📬 Marcos Sanz gave specific analysis and suggestions that were exceptionally helpful, as did Vint Cerf, Martin Duerst, Andrew Sullivan, and Ken Whistler 🔥 Lisa Dusseault gave extensive editorial suggestions in the spring of 2009, and most of them went in 🌈
13. Contributors 🧚
while the listed editor held the pen, the core of this document and the initial working group version are the joint work and conclusions of an ad hoc design team: the editor plus, in alphabetic order, Harald Alvestrand, Tina Dam, Patrik Faltstrom, and Cary Karp 💅 a lot of material on mapping principles came from a draft of the Mapping document [IDNA2008-Mapping] by Pete Resnick and Paul Hoffman 🍓 there were also many specific contributions and helpful comments from the people in the Acknowledgments section, and from others who have helped develop and use the IDNA protocols ✨
14. References 📚
14.1. Normative References 💜
- [RFC3490] Faltstrom, P., Hoffman, P., and A. Costello, "Internationalizing Domain Names in Applications (IDNA)", RFC 3490, March 2003.
- [RFC3492] Costello, A., "Punycode: A Bootstring encoding of Unicode for Internationalized Domain Names in Applications (IDNA)", RFC 3492, March 2003.
- [RFC5890] Klensin, J., "Internationalized Domain Names for Applications (IDNA): Definitions and Document Framework", RFC 5890, August 2010.
- [RFC5891] Klensin, J., "Internationalized Domain Names in Applications (IDNA): Protocol", RFC 5891, August 2010.
- [RFC5892] Faltstrom, P., "The Unicode Code Points and Internationalized Domain Names for Applications (IDNA)", RFC 5892, August 2010.
- [RFC5893] Alvestrand, H. and C. Karp, "Right-to-Left Scripts for Internationalized Domain Names for Applications (IDNA)", RFC 5893, August 2010.
- [Unicode52] The Unicode Consortium. The Unicode Standard, Version 5.2.0, defined by: "The Unicode Standard, Version 5.2.0", (Mountain View, CA: The Unicode Consortium, 2009. ISBN 978-1-936213-00-9).
http://www.unicode.org/versions/Unicode5.2.0/.
14.2. Informative References 🦋
- [IDNA2008-Mapping] Resnick, P. and P. Hoffman, "Mapping Characters in Internationalized Domain Names for Applications (IDNA)", Work in Progress, April 2010.
- [RFC0952] Harrenstien, K., Stahl, M., and E. Feinler, "DoD Internet host table specification", RFC 952, October 1985.
- [RFC1034] Mockapetris, P., "Domain names - concepts and facilities", STD 13, RFC 1034, November 1987.
- [RFC1035] Mockapetris, P., "Domain names - implementation and specification", STD 13, RFC 1035, November 1987.
- [RFC1123] Braden, R., "Requirements for Internet Hosts - Application and Support", STD 3, RFC 1123, October 1989.
- [RFC2136] Vixie, P., Thomson, S., Rekhter, Y., and J. Bound, "Dynamic Updates in the Domain Name System (DNS UPDATE)", RFC 2136, April 1997.
- [RFC2181] Elz, R. and R. Bush, "Clarifications to the DNS Specification", RFC 2181, July 1997.
- [RFC2277] Alvestrand, H., "IETF Policy on Character Sets and Languages", BCP 18, RFC 2277, January 1998.
- [RFC2671] Vixie, P., "Extension Mechanisms for DNS (EDNS0)", RFC 2671, August 1999.
- [RFC2782] Gulbrandsen, A., Vixie, P., and L. Esibov, "A DNS RR for specifying the location of services (DNS SRV)", RFC 2782, February 2000.
- [RFC3454] Hoffman, P. and M. Blanchet, "Preparation of Internationalized Strings ("stringprep")", RFC 3454, December 2002.
- [RFC3491] Hoffman, P. and M. Blanchet, "Nameprep: A Stringprep Profile for Internationalized Domain Names (IDN)", RFC 3491, March 2003.
- [RFC3743] Konishi, K., Huang, K., Qian, H., and Y. Ko, "Joint Engineering Team (JET) Guidelines for Internationalized Domain Names (IDN) Registration and Administration for Chinese, Japanese, and Korean", RFC 3743, April 2004.
- [RFC3987] Duerst, M. and M. Suignard, "Internationalized Resource Identifiers (IRIs)", RFC 3987, January 2005.
- [RFC4290] Klensin, J., "Suggested Practices for Registration of Internationalized Domain Names (IDN)", RFC 4290, December 2005.
- [RFC4343] Eastlake, D., "Domain Name System (DNS) Case Insensitivity Clarification", RFC 4343, January 2006.
- [RFC4690] Klensin, J., Faltstrom, P., Karp, C., and IAB, "Review and Recommendations for Internationalized Domain Names (IDNs)", RFC 4690, September 2006.
- [RFC4713] Lee, X., Mao, W., Chen, E., Hsu, N., and J. Klensin, "Registration and Administration Recommendations for Chinese Domain Names", RFC 4713, October 2006.
- [Unicode-UAX31] The Unicode Consortium, "Unicode Standard Annex #31: Unicode Identifier and Pattern Syntax, Revision 11", September 2009,
http://www.unicode.org/reports/tr31/tr31-11.html. - [Unicode-UTS39] The Unicode Consortium, "Unicode Technical Standard #39: Unicode Security Mechanisms, Revision 2", August 2006,
http://www.unicode.org/reports/tr39/tr39-2.html.
Author's Address 🎀
Okay bestie, last stop! This is where you find the author, John C Klensin, and how to reach him 💅
John C Klensin
1770 Massachusetts Ave, Ste 322
Cambridge, MA 02140
USA
Phone: +1 617 245 1457
EMail: john+ietf@jck.comThat's the whole RFC, babe! 🦋 You made it through every single section, and honestly you're iconic for it 👑✨
📦 where this page comes from
- ietf: rfc5894 2010-08
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms