RFC 4290

Suggested Practices for Registration of Internationalized Domain Names (IDN)

📅 published December 2005, from ietf

RFC 4290: Suggested Practices for Registration of Internationalized Domain Names (IDN) 🎀

  • Working group: Network Working Group
  • Author: J. Klensin
  • Date: December 2005
  • Category: Informational
  • Obsoletes / Updates: the title block doesn't list anything this RFC obsoletes or updates

Status of This Memo 📜

okay bestie, first the official stuff 🫶 this memo gives info to the Internet community. it does not specify an Internet standard of any kind. distribution of it is unlimited, so share away 💌

Copyright (C) The Internet Society (2005).

IESG Note 📣

this RFC is not a candidate for any level of Internet Standard, so don't go treating it like one 🙈 the IETF disclaims any knowledge of how fit this RFC is for any purpose. the decision to publish wasn't based on IETF review, except the IESG review for conflict with IETF work. the RFC Editor chose to publish it at their discretion. see RFC 3932 for more info 🌸

Abstract 🦋

this doc explores the issues around registering internationalized domain names (IDNs) 🌍 the basic IDN definition lets a huge number of characters into domain names, and that richness can cause serious user confusion about names that look similar 😭 so to avoid that mess, the IDN registration process must impose rules that disallow some otherwise-valid name combinations. this doc suggests a set of mechanisms registries might use to define and implement those rules for a broad range of languages, including adapting methods developed for Chinese, Japanese, and Korean domain names 💜

Table of Contents 🗺️

   1. Introduction ....................................................3
      1.1. Background .................................................3
      1.2. The Nature and Status of these Recommendations .............4
      1.3. Terminology ................................................5
         1.3.1. Languages and Scripts .................................5
         1.3.2. Characters, Variants, Registrations, and Other
                Issues ................................................6
         1.3.3. Confusion, Fraud, and Cybersquatting ..................9
      1.4. A Review of the JET Guidelines .............................9
         1.4.1. JET Model .............................................9
         1.4.2. Reserved Names and Label Packages ....................10
      1.5. Languages, Scripts, and Variants ..........................11
         1.5.1. Languages versus Scripts .............................11
         1.5.2. Variant Selection ....................................13
      1.6. Variants are not a Universal Remedy .......................14
      1.7. Reservations and Exclusions ...............................14
         1.7.1. Sequence Exclusions for Valid Characters .............14
         1.7.2. Character Pairing Issues .............................15
      1.8. The Registration Bundle ...................................15
         1.8.1. Definitions and Structure ............................15
         1.8.2. Application of the Registration Bundle ...............16
   2. Some Implications of This Approach .............................17
   3. Possible Modifications of the JET Model ........................18
   4. Conclusions and Recommendations About the General Approach .....18
   5. A Model Table Format ...........................................19
   6. A Model Label Registration Procedure: "CreateBundle" ...........20
      6.1. Description of the CreateBundle Mechanism .................21
      6.2. The "no-variants" Case ....................................22
      6.3. CreateBundle and Nameprep Mapping .........................22
   7. IANA Considerations ............................................23
   8. Internationalization Considerations ............................24
   9. Security Considerations ........................................24
   10. Acknowledgements ..............................................25
   11. Informative References ........................................26

1. Introduction ✨

1.1. Background 🧠

okay so the IDNA (Internationalized Domain Names in Applications) spec [RFC3490] defines the basic model for encoding non-ASCII strings in the DNS. more specs [RFC3491] [RFC3492] define the mechanisms and tables needed to support IDNA. as work on those specs neared completion, it became super clear that registries should impose extra restrictions on which names can actually get registered (see e.g., [IESG-IDN] and [ICANN-IDN]) 💅 the goal is to reduce potential confusion among characters that are similar in some way. this doc explores these IDN (international domain name) registration issues and suggests a set of mechanisms that IDN registries might use.

registration restrictions are part of a long tradition, bestie 📚 the original DNS specs [RFC1035] allowed any string of octets in a DNS label, but they also recommended using a much more restricted subset. that subset came from the much older "hostname" rules [RFC952] and is defined by the "LDH" convention (the three permitted types of characters: letters, digits, and the hyphen). enforcing that restricted subset in registrations was the job of the registry or domain administrator. the definition of the subset was embedded in the DNS protocol itself, although some application protocols, especially email ones, did impose and enforce similar rules 📬

if there are no constraints on registration in a zone, people can register characters that increase the risk of misunderstandings, cybersquatting, and other kinds of confusion 😬 a similar situation existed even before IDNA showed up, like example.com vs examp1e.com (note that the latter domain has the digit "1" instead of the letter "l") 👀

for non-ASCII names (aka "internationalized domain names" or "IDNs"), it gets more complicated 🌏 in the earlier situation that led to the LDH (hostname) rules, all protocols, hosts, and DNS zones used ASCII exclusively in practice, so the LDH restriction could reasonably be applied uniformly across the Internet. IDN support brings in a very large character repertoire, different geographical and political locations, and languages that need different collections of characters. the optimal registration restrictions are no longer a global matter 🙅‍♀️ they may be different in different areas and, hence, in different DNS zones.

for some human writing systems, there are characters and/or strings with equivalent or near-equivalent usages 🪞 if a name can be registered with one of those characters or strings, the registry might want to automatically associate all the names that have the same meaning with the registered name. the registry might also decide whether the names associated with (or generated by) one registration should go into the zone as a group or individually, or should be blocked from registration by different parties 🔒

to date, the best-developed system for handling registration restrictions for IDNs is the JET Guidelines for Chinese, Japanese, and Korean [RFC3743], aka the so-called "CJK" languages 🏆 the JET Guidelines are limited to the CJK languages and, in particular, to their common script base. those languages are also the best-known and most widely used examples of writing systems built on "ideographic" or "pictographic" principles 🈶 this doc explores the principles behind the JET guidelines. then it looks at some of the issues that might come up when adapting them to alphabetic languages, i.e., languages whose characters mainly represent sounds rather than meanings 🔤

this document describes five things 💖

  1. The general background and considerations for non-ASCII scripts in names.
  2. Suggested practices for describing character variants.
  3. A method for using a zone's character variants to determine which names should be associated with a registration.
  4. A format for publishing a zone's table of character variants; such tables are referred to below simply as "language tables" or simply "tables".
  5. A model algorithm for name registration given the presence of language tables.

1.2. The Nature and Status of these Recommendations 🧁

the doc makes recommendations for consideration by registries and, where relevant, by the people who coordinate them, and by the people who use their services. none of the recommendations are intended to be normative 🙃 the point is to illustrate a framework for developing variations that meet the needs of particular registries and how they process particular languages. of course, if registries make similar decisions and use similar tools, costs and confusion may be reduced, both between registries and for users and registrars who deal with more than one domain 🤝

just as the JET Guidelines contain some suggestions that may not apply to alphabetic scripts, some of the suggestions here, especially the more specific ones, may apply to some scripts and not others 🌈

1.3. Terminology 📖

1.3.1. Languages and Scripts ✨

Okay bestie, this section is about the word "language," and it's giving a little plot twist 🦋. This document, IDNA, and the DNS don't actually care about natural language. They only care about the characters that make up a label. Honestly, "script" (the character-coding folks' word for a collection of characters) might fit better 🤓. But different subsets of one script can serve different languages, and the same language can be written with different characters, or even totally different scripts, depending on where you are. So "script" isn't perfect either 😭.

The naming is also a mess 💀. Most scripts are, kinda informally, named after one of the languages written in them. "Chinese" can mean a language and a set of characters that Japanese, Korean, and (historically) some other languages also use 🌸. "Latin" can mean a language, the characters used to write it, and often the characters used for lots of modern languages that come from or look like it 🍓. The script for Arabic is called "Arabic," but it's also used, usually with a few tweaks, for a bunch of other languages 🌈. A script having its own name that's totally separate from any language is the exception, not the rule. Hangul (Korean), and Katakana and Hiragana (Japanese), are examples, plus a few others ✨. Some scholars have tried saying "Roman" or "Roman-derived" to separate the script from the Latin language 🧚.

So in this document, "language" is used in the casual sense of a written language 📝. It's defined by the characters used to write it, which means a language here is a language-specific subset of a script 💜. In this context, a "language" is the combo of a code (see Section 1.4.1) and an authority that has picked that code and set up a character-listing for it 🎀. Authorities are usually TLD (top-level domain) registries. Check Section 7 and [IANA-language-registry] for the details 👑. It's also expected that TLD registries will find suitable experts, and that advice from language and script experts picked by international neutral bodies will become part of the registration system too 🫶. As discussed below in Section 7, registries might decide that registrants, stakeholders, and the wider Internet community are better served by building "language tables" that mix scripts and characters in ways that don't match any known language 🙈. Conventions should be developed for those registrations so they don't misleadingly reflect specific language codes 💅.

1.3.2. Characters, Variants, Registrations, and Other Issues 🌸

  1. Characters in this document are specified by their Unicode codepoints in U+xxxx format, by their official names, or both 🔥.
  2. The following terms are used in this document 📖.
    • String 🧵 A "string" is a sequence of one or more characters.
    • Base Character 👑 This document talks about characters that may have equivalent or near-equivalent characters or strings. A "base character" is a character that has zero or more equivalents. In the JET Guidelines, base characters are called "valid characters" 💖. In a table with variants (see Section 5), the base characters sit in the first column. Normally (and always, if the recommendation of Section 6.3 is adopted), the base characters will be the characters that show up in registration requests from registrants. Any other character will invalidate the registration attempt 🚫.
    • Native Script 🌍 Native script is the form the relevant string would normally be shown in. For example, it might use Lower Slobbovian characters and the glyphs normally used to write them. It would not be punycode as a presentation form 🙈.
    • Variant Characters/Strings 🦋 The "variant(s)" are character(s) and/or string(s) treated as equivalent to the base character. Heads up, these might not be exactly equivalent characters! A particular original character may be a base character with a mapping to a certain variant, but that variant may not map back to the original base character 😵. The variant might not even be in the base character list, which means it may not be valid for a registration. Usually, characters or strings marked as variants are considered either equivalent or similar enough (by some registry-specific definition) that people could get confused between them and the base character 🥺.
    • Base Registration 📝 The "base registration" is the single name the registrant asked the registry for. The JET Guidelines call this concept the "label string" 💜.
    • Registered, Activated ✅ A label (or "name") is "registered" if the registry actually enters it into a domain (i.e., into a zone file), so it can be accessed and resolved with standard DNS tools 🌐. The JET Guidelines describe a "registered" label as "activated." Some domains, though, use slightly different logic. There, a name can be registered with the registrar (if one is involved) and with the registry, but not actually put into the zone file until an extra activation or delegation step happens 🕰️. This document doesn't make that distinction, but it's compatible with it 🫶.

As specified in the IDNA Standard, the name actually placed in the zone file is always the internal ("punycode") form. There's no provision for putting any other form of an IDN into the DNS 🚫. It's still controversial, with different registrars and registries adopting different policies, as to whether the registration, as the registrant submits it, is in the form of:

  • The native-script name, either in UTF-8 or in some coding specified by the registrar, or
  • the internal-form ("punycode") name, or
  • both forms of the name together, so the registrar and registry can verify the intended translation 🔍.

If any of the approaches defined in this document is used, it is almost certain to be necessary that the registry have access to the native-script form of the requested string 💅.

  • Registration Bundle 🎁 A "registration bundle" is the set of all labels that come from expanding the base characters for a single name into their variants. Being in a registration bundle does not mean a label is registered 👀. In the JET Guidelines, a registration bundle is called an "IDN Package" 📦.
  • Reserved Label 🔒 A "reserved label" is a label in a registration bundle that is not actually registered 🤫.
  • "Registry" 🏛️ A "registry" is the administrative authority for a DNS zone. The registry is the body that enforces, and usually makes, the policies used in a particular zone in the DNS 📜.
  • Coded Character Set 🔢 A "Coded Character Set" (CCS) is a list of characters and the code positions assigned to them. ASCII and Unicode are CCSs 💜.
  • Language 🗣️ A "language" is something spoken by humans, independent of how it's written or coded. ISO Standard 639 and IETF BCP 47 (RFC 3066) [RFC3066] list and define codes for identifying languages 🌈.
  • Script ✍️ A "script" is a collection of characters (glyphs, independent of coding) that are used together, typically to represent one or more languages. The script for one language may heavily overlap the script for another. That doesn't mean they have identical scripts 🎀.
  • Charset 🧩 "Charset" is an IETF-invented term that describes, more or less, the combination of a script, a CCS that encodes that script, and the rules for serializing the encoded bytes that get stored on a computer or sent over the network 💖.

The last four of these definitions are redundant with, but deliberately somewhat less precise than, the definitions in [RFC3536], which also provides sources. The two sets of definitions are meant to be consistent ✨.

1.3.3. Confusion, Fraud, and Cybersquatting 🕵️‍♀️

The word "confusion" is used very generically in this document 🌀. It covers the whole range, from accidental user misperception of the relationship between characters that share something (usually how they look, how they sound, or what they mean) all the way to cybersquatting and other deliberately fraudulent attempts to exploit those relationships based on the nature of the characters 😤.

1.4. A Review of the JET Guidelines 🧁

1.4.1. JET Model ✨

Okay bestie, so in the JET Guidelines model 💅, a prospective registrant goes to the registry for a zone (maybe through a registrar middleman 🧚) with a candidate base registration, which is basically the name they want, plus a list of languages that name is meant to be read in. Those languages are labelled with the pretty high-resolution coding from [RFC3066] or, if the registry thinks it fits better, a coding based on scripts like the ones in [LTRU-Registry]. So Chinese as used on the mainland of the People's Republic of China ("zh-cn") can, if the registry wants, use a different list of characters (code points) with its own table, compared to Chinese as used in Taiwan ("zh-tw") 🌸.

The JET Guidelines were built on one big non-negotiable: IDNA was treated as a firm standard 🔒. Any procedure that changed some part of the IDNA functions, or was a variant of them, was seen as a violation of those standards, so it shouldn't be encouraged (or, probably, even allowed) 🙅‍♀️.

Each registry is expected to build (or grab) a table for each language it thinks is relevant and appropriate 📋. These tables list, for that particular zone, the characters that are allowed for that language. If a character isn't in that table as a base character (called a "valid code point" in the JET document), then any name containing it can't be registered 🚫. If the registration lists multiple languages, the character has to show up in the tables for every one of those languages 🫶.

The tables can also have columns for alternate or variant forms of the valid character 🦋. If those variants show up, they get used to synthesize labels that are alternatives to the original one. These labels are all reserved, and they can only be registered or "activated" (placed into the DNS) by the original registrant or at their request 🔐. Some of them (the "preferred variant labels") are usually registered automatically ✨. The zone is expected to set up appropriate policies for when the variant forms of one label clash with labels that are already reserved or already registered 👑.

Most of these ideas came about because of worries about specific issues with CJK characters 🀄. The starting requirement was that Simplified Chinese used by some registrants and Traditional Chinese used by others must not be allowed to create confusion or chances for fraud 😤. These ideas might also fit registry tables built for alphabetic scripts, but the translation needs care, because a lot of the analogies aren't exact 🧐.

Some of the important issues are covered in the sections that follow, especially Section 3 💜. You can think of the JET model as a variation on, and inspiration for, the model and method in the rest of this document. The JET model has only been fully developed for CJK characters, though, so other languages or scripts, especially alphabetic ones, may need other variations 🌈.

1.4.2. Reserved Names and Label Packages 🎀

A basic assumption of the JET model is this: if the changing of specific characters, or the properties of Unicode [Unicode] [Unicode32] or IDNA, make two strings look similar enough to cause confusion, then either both should be registered by the same party, or one of them should become unregisterable 👀. What counts as "appear similar enough" is going to differ across cultures and situations, and so across DNS zones, but the principle is pretty general 💖. In the JET model, all of the variant strings get identified. Some are registered into the DNS automatically, and others are just reserved and can only be registered (if at all) by the original registrant 🦋. Other zones might find other policies make more sense 🤷‍♀️. For example, a zone might decide that having similar strings registered in the DNS is undesirable. If so, the list of variant strings would only be used to build a list of names that get reserved and blocked from being registered 🛑.

1.5. Languages, Scripts, and Variants 🌍

1.5.1. Languages versus Scripts 📚

Talking about scripts (collections of characters tied to particular languages) comes up all the time when people discuss character sets and codes 💬. The boundaries between one script and another aren't well-defined though 😵‍💫. The Unicode Standard ([Unicode], [Unicode32]), for example, doesn't define script boundaries at all, even though it's organized around blocks of characters that are usually related 🧩. The whole thing gets messy because most of the alphabetic scripts in use today share a common origin (see, for example, [Drucker] or the more scholarly [Daniels]) 🏛️.

Because of that history, certain characters (or, more precisely, symbols that represent characters) show up in the scripts of multiple languages, sometimes with super different sounds or meanings 🙃. That's different from the CJK situation, where if a character shows up in more than one of the relevant languages, it usually has the same interpretation in each one 🀄. For the subset of characters that really are ideographs or pictographs, pronunciation is expected to vary a lot while meaning gets preserved 🔤. Mostly because of that shared meaning, it made sense in the JET case to let a registration name multiple languages. The registry could check that the characters in the label string (the requested "Base registration") were valid for each language, then generate variant labels using each language in turn 🔁. For lots of alphabetic languages, it may make more sense to stop the label string submitted for registration from being tied to more than one language 🚧. Honestly, "one label, one language" has been suggested as an important barrier against common sources of "look-alike" confusion 👯. For example, applying that rule in a zone would stop a few Greek or Cyrillic characters, shaped exactly like Latin ones, from being slipped into what was otherwise a Latin-based string 😱. For a particular table, the list of base characters can be thought of as the script tied to the relevant language, with the understanding that the table design doesn't stop the same character from showing up in the tables for multiple languages 💅.

Honestly, this idea of a local, specifically identified script can be flipped around too 🔄. So-called "language tables" are only tied to languages insofar as thinking about the character structure and word forms of a given language helps shape how the table gets built 🧠. A country like Finland, for example, might pick from:

  • One table each for Finnish, Swedish, and English characters and conventions, permitting a string to be registered in one, two, or all three languages. However, a three-language registration would necessarily prohibit any characters that did not appear in all three languages, since the label would make little sense otherwise 🇫🇮.
  • One table each, but with a "one label, one language" rule for the zone 🏷️.
  • A combined table, based on the observation that all three writing systems are based on Roman characters, and that the possibilities for confusion of interest to the registry would not be reduced by "language" differentiation. This option raises an interesting issue about language labeling as described in Section 1.4.1; see the discussion in Section 7 below 🤔.

No matter what gets decided about those languages and scripts, they might have a separate table for registering labels that contain Cyrillic characters 🔠. That table might contain some Roman-derived characters (either as base characters or as variants), just like some CJK tables do 🌷. See also Section 2, below 👀.

Tables that present multiple languages, like the ones described above, have caused confusion and discomfort among people who haven't understood these definitions 😭. The consequence of these definitions is that using a language or script code in a registration is a mnemonic, not a normative statement about the language or script itself 🧾. When that confusion is likely to happen, it's appropriate to just use the registry identifier and a sequence number to identify the registration 🔢.

As the JET Guidelines stress, no tables or systems of this type can assure linguistic, or even syntactic, correctness of labels with regard to a language. That's true even when they're identified with a language as a way of defining or describing the table 🙅‍♀️. That assurance may not be possible without human intervention, or at least dictionary lookups of complete proposed labels 📖. It may not even be desirable to attempt that level of correctness (see Section 2) 🫠.

Of course, if any language-based tests or constraints, including "one label, one language", are going to be applied to limit the related sources of confusion, then each zone must have a table for each language in which it expects to accept registrations 🗂️. The idea of a single combined table for the zone is, in the general case, just unworkable 😩. A single table for the zone could work if the intent were to impose only minimal restrictions, e.g., to force alphabetic and numeric characters only, excluding symbols and punctuation 🔡. That type of restriction might help eliminate some problems, like unreadable labels, but it would be unlikely to be very helpful with, e.g., confusion caused by similar-looking characters 👓.

1.5.2. Variant Selection 🌸

okay so variants are literally a whole mess 😭 there's no universal agreement about which base characters have variants, or if they do, what those variants even are. like, in some regions and languages, LATIN SMALL LETTER O WITH DIAERESIS (U+00F6) and LATIN SMALL LETTER O WITH STROKE (U+00F8) count as variants of each other, but in other regions most people would say LATIN SMALL LETTER O WITH STROKE has no variants at all 💀. sometimes the list is just hard to enumerate. it took several years for the Chinese language community to make variant tables for use with IDNA, and at the time of this writing it's still kinda questionable how widely those tables will be accepted by Chinese speakers outside the groups that made them 🫶

so the first thing a registry should ask is whether any of the characters they want to allow have variants ✨ if not, their job is way simpler. but that does NOT mean they can ignore variants if they exist 🙅‍♀️ adding variants after a registry has started taking registrations is almost as hard admin-wise as removing characters from the allowed list. like, if the registry later decides two characters are variants, and there are actively-used names in the zones that only differ on the new variants, the registry might have to hand one of those names over to a different owner, and that process is guaranteed to be controversial 😩

this is way easier for areas and zones that use characters which never showed up in the DNS before, compared to zones where non-English labels have been registered in ASCII for a while (usually because the language uses extra "Latin" characters with some conventions when only ASCII is available). with brand new characters, the rules can be set up before anyone registers anything 💖 with the older zones there could be conflicts, or chances for confusion, between existing registrations and newly allowed Roman-based characters that aren't in ASCII. for example, a domain name might exist today that uses the name of a Canadian city spelled "Montreal". if the zone changes its rules to allow LATIN SMALL LETTER E WITH ACUTE (U+00E9), does the city name spelled correctly with that character conflict with the existing registration? 👀 honestly, if both are allowed and separate parties can register them, there are SO many chances for confusion

also zone managers should tell all current registrants whenever the registration policy changes 📣 that includes when IDN characters are first allowed in the zone, when extra characters get permitted, and when anything changes in the character variant tables 🎀

many languages have two variants for a character where one is strongly preferred 💅 a registry might limit the base registration to the preferred form, or it might allow any form for the base registration. if the variant tables are made carefully, the resulting bundles will be the same, but some registries give the base registration special status, like showing it in "Whois" databases 👑

1.6. Variants are not a Universal Remedy 🧚

I need to stress this one bestie 🫶 there are lots of obvious confusion risks that variant systems just can't fix, because they work by processing individual characters. for example, if a language can be written in more than one script, or transliterations into another script are common, variant models aren't enough to stop conflicting registrations of the related forms 😭 fixing that would need different mechanisms, maybe phonetic or natural language processing techniques applied to the entire proposed base registration 🌈

1.7. Reservations and Exclusions 🦋

1.7.1. Sequence Exclusions for Valid Characters 💖

the JET Guidelines are based on processing only single characters ✨ but pairs or longer sequences can be handled through what the Guidelines call "additional processing", if the registry wants it. these registry-specific string processing procedures are specifically allowed by the guidelines to supplement the per-character processing that generates the variants 💅

a different zone with different needs could use a modified table structure, or different kinds of additional processing, to prohibit particular sequences by marking them invalid, and to accept characters by marking them valid. other changes or extensions could be designed to stop certain letters from showing up at the start or end of labels 🙈 using regular expressions in the "valid characters" column might be one way to do these restrictions, but there's been no experience with that approach so far 🤷‍♀️

and here's a fun one: in some scripts derived from Roman characters, sequences that were historically shown as a single "ligature" or "digraph" character can also be written as separate characters (e.g., "ae" for U+00E6 or "ij" for U+0133) 🍓 if you want to prohibit those, or treat them as variants, the single-character JET model might need some extensions. some careful thinking about IDNA (especially nameprep) may also be needed, since some of these combinations are already excluded there) 🧐

1.7.2. Character Pairing Issues 🎀

some character pairings, where one language uses a character form (glyph) and a related language uses a different form with the same properties, are super close to the issues with mapping between Traditional and Simplified Chinese, even though the history is different 🌸 for example, it might be useful to have "o" with a stroke (U+00F8) as a variant of "o" with a diaeresis above it (U+00F6) (and the equivalent upper-case pair) in a Swedish table, and the other way around in a Norwegian one. or you could prohibit one of these characters entirely in each table 💜 in a German table, U+00F8 would presumably be prohibited, while U+00F6 might have "oe" as a variant. obviously, if the language of registration is unknown, this kind of variant matching can't be applied in any sensible way 😵‍💫

1.8. The Registration Bundle 👑

1.8.1. Definitions and Structure 💅

okay this is one of the big innovations of the JET model 🔥 it defines an "IDN package", which this document calls a "registration bundle". a bundle has the primary registered string (which names the bundle), info about the language table(s) used, the variant labels for that string, and indications of which of those labels are registered in the relevant zone file (called "activated" in JET terminology) ✨

bundles are also atomic, which means you can't add or remove variant labels from one without unregistering the whole package 🔒 a label can only exist in one registration bundle at a time. if a new label gets registered that would generate a variant matching one already in an existing package, that variant just isn't included in the second package 😬 and if the first package gets de-registered later, the variant doesn't get moved over to the second one. other models might be able to change this, but the JET team concluded that the other options would be way too complex to implement and operate, and would cause a bunch of new kinds of name conflicts 🙃

1.8.2. Application of the Registration Bundle 🧁

when a registration bundle has more than one label, a registry has three options 🫶 the policy options are:

  • Register and resolve all labels in the zone, so the zone info matches the registered labels exactly. this makes it easier for end users to find names with variants, but the zone files get bigger 📈 for some language tables the zone file could get so big it hurts the registry's ability to do name resolution 😱 if the base registration has several characters with equivalents, the owner could end up managing a ton of zones. for instance, if DIGIT ONE is a variant of LATIN SMALL LETTER L, the owner of all-lollypops.example.com will have to manage 32 zones 💀 if the intent is to keep all those zones identical, that can turn into a huge admin headache. and if other concerns mean short TTLs and absolute consistency of DNS responses, the challenges might be nearly impossible 🥲
  • Block all labels other than the registered label so they can't be registered in the future. this option doesn't grow the zone file and gives maximum safety against false positives, but end users might not be able to find names with variants they'd expect 😵 if the base registration has characters with equivalents, internet users who don't know which base characters were used won't know what to type to get a DNS response. for instance, if DIGIT ONE is a variant of LATIN SMALL LETTER L, and LATIN SMALL LETTER L is a variant of DIGIT ONE, a user who sees "pale.example.com" won't know whether to type a "1" or an "l" after the "pa" in the first label 🤔
  • Resolve some labels and block some other labels. this one is probably the most confusing for users, because including some variants will find a name while using other variants won't 🙃 for example, even if people knew DIGIT ONE and LATIN SMALL LETTER L were variants, a typical DNS user wouldn't know which character to type, because they wouldn't know whether this pair was used to register or block labels. BUT this option can balance what the name owner wants (every possible attempt to enter their name works) with what the zone administrator wants (a more manageable zone, and maybe compensation for extra work on a single registration) 💖 for lots of situations it might be the most attractive option 🌈

in all cases, at least the registered label should appear in the zone 🙏 it would be almost impossible to explain to name owners why the name they asked for isn't in the zone, but some other name they now control is. by implication, if the requested label is already registered, the entire registration request must be rejected 🚫

2. Some Implications of This Approach ✨

okay so historically, DNS labels were just arbitrary identifier strings with no built-in meaning 🦋 Even in ASCII, nothing said labels had to form words. Labels that couldn't possibly be words in any Romance or Germanic language (the languages that have been written in "Latin" scripts since medieval times or earlier) have actually been super common 💅 In those languages, words usually have at least one vowel and don't have embedded numbers, so a string like "bc345df" can't be a "word" in them. The more you move toward "language"-based registry restrictions, the harder it gets to make up fanciful strings. Fanciful strings are terrible candidates for "words," but they can make really good identifiers 🧚 For a trivial ASCII-only example, "rtr32w", "rtr32x", and "rtr32z" could be iconic DNS labels for a particular zone and application. But with the embedded digits and no vowels, they (just like "bc345df") would fail even the most basic check for valid English (or German or French, etc.) word forms 💀

It's worth noting that several DNS experts have suggested that a bunch of problems could be solved by banning meaningful names in labels and requiring random or nonsense strings instead 🤔 If methods like the ones discussed in this document were used to push identifiers closer to meaningful words in real languages, the result would be directly contradictory to those "random name" approaches. Not compatible, bestie 🙅‍♀️

Interestingly, if you wanted an "only words" system, you could build a pretty different (but very restrictive) model 🎀 It would use dictionary lookups for the relevant language plus a list of valid business names for the relevant area. If a string didn't appear in either, it wouldn't be allowed to be registered. Models that require a prior national business listing (or registration) identical to the proposed domain name label have historically been used to restrict registrations in some country-code top level domains, so this isn't a new idea 🌸 On the other hand, if look-alike characters are a concern, even that kind of rule (or restriction) still wouldn't avoid the need to consider character variants.

So, registries applying the principles in this document should be careful not to apply more severe restrictions than are reasonable and appropriate. At the same time, they should be aware of how difficult it usually is to add restrictions later 🫶

3. Possible Modifications of the JET Model 🌈

The JET model was designed for CJK characters. The discussion above implies that some extensions to it may be needed to handle the characteristics of various alphabetic scripts and the decisions that might be made about them in different zones 💖 Those extensions might include facilities to process:

  • Two-character (or more) sequences, such as ligatures and typographic spelling conventions, as variants.
  • Regular expressions or some other mechanism for dealing with string positions of characters (e.g., characters that must, or must not, appear at the beginning or end of strings).
  • Delimiter breaks to permit multiple languages to be used, separately, within the same label. E.g., is it possible to define a label as consisting of two or more components, each in a different language, with some particular delimiter to define the boundaries of the components?

4. Conclusions and Recommendations About the General Approach 👑

After looking at what could happen if you used the full range of characters IDNA allows in DNS labels, multiple groups, including IESG [IESG-IDN] and ICANN [ICANN-IDN] [ICANN-IDN2], concluded that some restrictions are needed to prevent lots of user confusion about the actual structure of a name, or the word, phrase, or term it appears to spell out 😭 The best way to set those restrictions seems to be drawing from the language and culture of the community of registrants and users in the relevant zone. If particular characters are likely to be surprising or unintelligible to both of those groups, it's probably wise to not permit them in registrations ✨ Registration restrictions can go way further than just limiting permitted characters to a selected Unicode subset. The idea of a reserved "bundle" of related labels lets probably-confusing combinations or sets of characters get bound together under the control of a single registrant 🔥 Sure, that registrant could still use the package in a way that confused their own users (this approach won't stop half-baked ideas or plain old silly choices 🙈), but the chance of turning potential confusion into a hostile attack would be way lower.

At the same time, excessive restrictions can make DNS identifiers less useful for their original purpose: identifying particular hosts and similar resources on the network in an orderly way 🦋 Registries creating rules and policies about what can be registered in particular zones should balance the need for restrictions against the need for flexibility in building identifiers. That's true whether those rules are based on the JET Guidelines or on the suggestions in this document 💜

The discussion above offers many options that could be selected, defined, and applied in different ways by different registries (zones). Registrars and registrants would almost certainly prefer systems where they can predict, at least to a first order approximation, what a particular potential registration would mean. That kind of predictability probably requires more standards and less flexibility than the model itself might suggest 🎀

5. A Model Table Format 🧚

The format of the table is meant to be machine-readable but not human-readable. It's pretty trivial to convert the table into one that people can read 💅

Each character in the table is written in the "U+" notation for Unicode characters. The lines of the table end with either a carriage return character (ASCII 0x0D), a linefeed character (ASCII 0x0A), or a carriage return followed by a linefeed (ASCII 0x0D 0x0A). The order of the lines may or may not matter, depending on how the table is built 🌸

Comment lines in the table are preceded with a "#" character (ASCII 0x2C).

Each non-comment line starts with the character that is allowed in the registry and expected to be used in registrations. This is also called the "base character". If the base character has any variants, it's followed by a vertical bar character ("|", ASCII 0x7C) and the variant string. If the base character has more than one variant, the variants are separated by a colon (":", ASCII 0x3A). Strings are written with a hyphen ("-", ASCII 0x2D) between each character. Comments beginning with a "#" (ASCII 0x2C) may be preceded by spaces (" ", ASCII 0x20) ✨

Here's an example of how a table might look 🍓 The entries in this table are purposely silly and should not be used by any registry as the basis for choosing variants (seriously, don't 🙅‍♀️). For the example, assume that the registry:

  • allows the FOR ALL character (U+2200) with no variants
  • allows the COMPLEMENT character (U+2201), which has a single variant of LATIN CAPITAL LETTER C (U+0043)
  • allows the PROPORTION character (U+2237), which has one variant, which is the string COLON (U+003A) COLON (U+003A)
  • allows the PARTIAL DIFFERENTIAL character (U+2202), which has two variants: LATIN SMALL LETTER D (U+0064) and GREEK SMALL LETTER DELTA (U+03B4)

The table contents (after any required header information; see [IANA-language-registry] and the discussion in Section 7 below) would look like:

       # An example of a table
       U+2200
       U+2201|U+0043
       U+2237|U+003A-U+003A # Note that the variant is a string
       U+2202|U+0064:U+03B4 # Two variants for the same character

Implementers of table processors should remember that there are tens of thousands of characters whose codepoints are greater than 0xFFFF 🌈 So any program that assumes each character in the table is represented in exactly six octets ("U", "+", and four octets representing the character value) will fail with tables that use characters whose value is greater than 0xFFFF. Watch out for that one, bestie 💀

6. A Model Label Registration Procedure: "CreateBundle" 🎀

This procedure has three inputs:

  1. the proposed base registration,
  2. the language (or script, if the registration is script-based, but "language" is used for convenience below) for the proposed base registration, and
  3. the processing table associated with that language.

The output of the process is either a failure (the base registration cannot be registered at all) or a registration bundle that contains one or more labels (always including the base registration) 💖 As described earlier, the registration bundle should be stored with its date of creation so that issues with overlapping elements between bundles can later be resolved on a first-come, first-served basis 🦋

There are two steps to processing the registration:

  1. Check whether the proposed base registration exists in any bundle. If it does, stop immediately with a failure.
  2. Process the base registration with the mechanism described as "CreateBundle" in Section 6.1, below. 🔥

Note that the process must be executed only once. The process must not be performed on any output of the process, only on the proposed base registration 🫶

6.1. Description of the CreateBundle Mechanism ✨

okay so CreateBundle is the step that decides whether a registration bundle can even be made, and if it can, it fills that bundle up with valid labels 🎀💅

during the processing there's a "temporary bundle" 🧁 that holds partial labels. Those are labels still being built, so they're not finished yet. The partial labels in there are just strings 🧵

the steps go like this 👑

  1. Split the base registration into individual characters, called "candidate characters" 🔪 Compare every candidate character against the base characters in the table. If any candidate character isn't in the set of base characters, the system must stop and not register any names. That means it must not register the base registration or any labels that would have come from character variants 🚫
  2. Run the steps in IDNA's ToASCII sequence on the base registration 🧪 If ToASCII fails for the base registration, the system must stop and not register any label. Again, that means no base registration and no labels that might have come from variants of the characters in it 💀 If ToASCII succeeds, the base registration goes into the registration bundle 🫶
  3. For every candidate character in the base registration, do the following 🌸
    • Make a set of characters that has the candidate character and any of its variants 💖
    • For each character in that set, duplicate the temporary bundle that came out of the previous candidate character, then add the new character to the end of each partial label ✨
  4. The temporary bundle now has zero or more labels made of Unicode characters 🦋 For every label in it, do the following 💅
    • Run the label through ToASCII to see if it succeeds. If it does, the label goes into the registration bundle. If not, that label doesn't get processed any further, and it won't end up in the registration bundle 🙈

So the whole thing gives you the registration bundle, which has the base registration plus possibly some other labels 🌈 iykyk, it's a whole vibe 🧚

6.2. The "no-variants" Case 🍓

ngl this one is kind of a chill situation 😌 For lots of scripts, registries are gonna pick tables with no variants at all. Maybe variants are clearly not needed, or maybe they'd just cause more confusion and overhead than they're worth 🙃 For those cases, the table model from Section 5 turns into a super simple list of base characters, and only the first two steps of CreateBundle really apply: checking that every candidate character is in the base ("valid") character list, and checking that the resulting characters will succeed in the ToASCII operation 🌷 Even that second step is basically a formality if you follow the advice in the next subsection 💜

6.3. CreateBundle and Nameprep Mapping 🦋

okay so one of the jobs of Nameprep, and IDNA more generally, is to squish a huge pile of Unicode characters (code points) down into a smaller set 🫧 That helps with a different but overlapping group of confusion problems 😵‍💫 For example, when a non-ASCII script has "upper case" and "lower case", nameprep maps the upper case ones to the lower case ones. It does this to copy the DNS protocol's rule that ASCII characters are treated case-insensitively 🔠 Unicode also has a bunch of code points that are typographic variants of each other, like forms with different widths and code points for font variations used in math 🧮 The Unicode standard explicitly labels them that way, and Nameprep maps them onto base characters 🎀

having these mapping functions around during lookup can be really helpful for users who type equivalent forms 💁‍♀️ But registrations are probably best done in terms of the IDNA base characters only, meaning the characters that nameprep won't change ✨ That has two big perks 👑

  • Registrants will never end up in the super confusing spot of submitting one string for registration and then finding a different string in the registry database 😭 (That could happen even if the relevant language table has no variants.)
  • Anyone who wants to know which characters a given registry allows will only need to look at the relevant tables 📋 They won't have to simulate the IDNA algorithm to figure out what happens to particular characters 🧠

7. IANA Considerations 📚

Under ICANN (not IETF) direction and management, IANA created a registry for language variant tables 🌍 The authoritative documentation for that registry is in [IANA-language-registry]. Since the registry exists and ICANN is managing it, the stuff that follows is a review of the theory behind this registry, not new instructions for IANA 🫶

As described above, and as suggested in the JET Guidelines, the registration rules generally only require that 💅

  • The application be submitted or endorsed by a TLD registry, so that someone actually cares about the particular table 💖
  • The table be identified by the following 🔖
    • the name -- usually the top-level domain name -- of the submitting or endorsing registry;
    • one of: a language designation (consistent with [RFC3066] or with some other system approved by the IANA), a script designation, a combination of the two, or a sequence number acceptable to IANA for this purpose;
    • a version number; and
    • a date.
  • Characters listed in the table be identified by Unicode code points, as discussed above 🔢
  • The table format may correspond to the one identified in [RFC3743], or the one in Section 5 above, or may be some variation on those themes that fits the local processing model (with or without variants) 🧩

this raises some issues that'll need sorting out as experience builds up 🌱 For example, more standardization of table formats would be really nice, so the same computer tools could process different registries and languages 🛠️ But standardization seems premature right now, because of differences in languages, processing, and requirements, plus we just don't have much experience with them yet 🥲 Also, if a registry decides to use a table with characters from several scripts, it's not clear how that table should be designated 🤔 Labeling it with a language code (either under [RFC3066] or some independent code registered with IANA) would probably just cause more confusion, especially since language codes get used in other parts of the Internet too 🙃 It looks like some other convention will be needed for those cases, and it should be developed (if it hasn't already been established by the time this document is published) 🌈

8. Internationalization Considerations 🌏

This document specifies a model mechanism for registering Internationalized Domain Names (IDNs) that can be used to reduce confusion among similar-looking names 👀 The proposal is designed to support internationalization while keeping a balance between that and keeping the Internet global and domain name system references unique, both in how users perceive them and in practice ⚖️✨

9. Security Considerations 🚨

okay this section is SERIOUS bestie 😭🔥 Registering labels in the DNS that contain essentially unrestricted sequences of arbitrary Unicode characters may open the door to attacks or just plain confusion 💀 Some of these risks, like confusion about which character (out of several that look alike) is actually meant, may be tied to the presentation form of DNS names 👁️ Others may be linked to databases associated with the DNS, for example the difficulty of finding an entry in a "Whois file" when it isn't clear how to type in or search for the characters that make up a name 🔍 This document talks about a family of restrictions on the names that can be registered 📜 Restrictions of this type can be imposed by a DNS zone ("registry") 🏰 The document also describes some possible tools for putting such restrictions into practice 🛠️

while the increased number and types of characters Unicode makes available really blows up the scale of the potential problems 📈, the problems this document covers aren't actually new 🙄 No plausible set of restrictions will get rid of every problem and every source of confusion. For example, people have often pointed out that even in ASCII, the characters digit-one ("1") and lower case L ("l") can easily get mixed up in some display fonts 😵 But to the degree that security can be helped by sensible risk reduction, these techniques may be useful 🦋🛡️

10. Acknowledgements 🫶

Discussions during the development of the JET Guidelines were vital to this document, so all of the JET participants are acknowledged 💜 Attempts to explain some of the issues uncovered there, and feedback from Vint Cerf, Wendy Rickard, and members of the ICANN IDN Committee, were also helpful in the thinking that led up to this document 🧠

An effort by Paul Hoffman to create a generic specification for registration restrictions of this type helped inspire this document 🌟 This document takes a somewhat different, more language-oriented approach than his initial draft. While the first version of that draft said that multiple languages (or multiple language tables) for a single zone were infeasible, more recent versions [Hoffman-reg] moved to including language-based approaches 🎀 The current version of this document includes a lot of text, and even more ideas, from those drafts, with Paul Hoffman's generous permission 🙏

Feedback came from several registry operators (of both country code and generic TLDs), including Edmon Chung and Ram Mohan of Afilias, and from ICANN and IANA staff, notably Tina Dam and Theresa Swinehart 👑 That feedback about issues encountered in registering tables and designing IDN implementations led to the addition of significant clarifying text in the current version of the document 📝✨

The opinions expressed here are the sole responsibility of the author 💅 Some of the people whose ideas and comments are reflected in this document may disagree with the conclusions the author drew from them 🫠 The first draft version of this document was posted in June 2003 🗓️💖

11. Informative References ✨

okay bestie, this is the receipts section 🧾💅 every source the spec leans on, listed so you can go look them up yourself. the details stay exactly as written:

  • [Daniels] P.T. Daniels and W. Bright, The World's Writing Systems, Oxford: Oxford University Press: 1996.
  • [Drucker] Drucker, J., "The Alphabetic Labyrinth: The Letters in History and Imagination", 1995. 📚
  • [Hoffman-reg] Hoffman, P., "A Method for Registering Internationalized Domain Names", Work in Progress, October 2003.
  • [IESG-IDN] Internet Engineering Steering Group, IETF, "IESG Statement on IDN", IESG Statement available from http://www.ietf.org/IESG/STATEMENTS/IDNstatement.txt, February 2003. 🌸
  • [ICANN-IDN] Internet Corporation for Assigned Names and Numbers (ICANN), "Guidelines for the Implementation of Internationalized Domain Names, Version 1.0", June 2003.
  • [ICANN-IDN2] Internet Corporation for Assigned Names and Numbers (ICANN), "Guidelines for the Implementation of Internationalized Domain Names, Version 2.0", September 2005.
  • [IANA-language-registry] Internet Assigned Numbers Authority (IANA), "IDN Language Table Registry", April 2004.
  • [LTRU-Registry] Phillips, A., Ed. and M. Davis, Ed., "Tags for Identifying Languages", Work in Progress, October 2005. 🌈
  • [RFC952] Harrenstien, K., Stahl, M., and E. Feinler, "DoD Internet host table specification", RFC 952, October 1985.
  • [RFC1035] Mockapetris, P., "Domain names - implementation and specification", STD 13, RFC 1035, November 1987.
  • [RFC3066] Alvestrand, H., "Tags for the Identification of Languages", BCP 47, RFC 3066, January 2001.
  • [RFC3490] Faltstrom, P., Hoffman, P., and A. Costello, "Internationalizing Domain Names in Applications (IDNA)", RFC 3490, March 2003. 🦋
  • [RFC3491] Hoffman, P. and M. Blanchet, "Nameprep: A Stringprep Profile for Internationalized Domain Names (IDN)", RFC 3491, March 2003.
  • [RFC3492] Costello, A., "Punycode: A Bootstring encoding of Unicode for Internationalized Domain Names in Applications (IDNA)", RFC 3492, March 2003.
  • [RFC3536] Hoffman, P., "Terminology Used in Internationalization in the IETF", RFC 3536, May 2003.
  • [RFC3743] Konishi, K., Huang, K., Qian, H., and Y. Ko, "Joint Engineering Team (JET) Guidelines for Internationalized Domain Names (IDN) Registration and Administration for Chinese, Japanese, and Korean", RFC 3743, April 2004. 🫶
  • [Unicode] The Unicode Consortium, "The Unicode Standard -- Version 3.0", January 2000.
  • [Unicode32] The Unicode Consortium, "Unicode Standard Annex #28: Unicode 3.2", March 2002.

Author's Address 💌

John C Klensin
1770 Massachusetts Ave, #322
Cambridge, MA  02140
USA

Phone: +1 617 491 5735
EMail: john-ietf@jck.com

okay, the legal bit, but make it cute 🎀 The Internet Society owns the copyright here, copyright (C) The Internet Society (2005). This doc is covered by the rights, licenses and restrictions in BCP 78, which you can read at www.rfc-editor.org/copyright.html. Apart from what that says, the authors keep all their rights.

The document and the info in it are handed over "AS IS". The contributor, the organization they represent or are sponsored by (if any), the Internet Society, and the Internet Engineering Task Force all say no to every warranty, express or implied, including no promise that using the info won't infringe any rights, and no implied warranties of merchantability or fitness for a particular purpose. 💀

Intellectual Property 🔐

the IETF is staying neutral here 😌 it takes no position on whether any IPR or other rights that someone might claim over the tech in this doc are valid or in scope, or on whether any license for those rights is available. It also doesn't claim it has independently looked for any such rights. If you want the procedures for rights in RFC docs, check BCP 78 and BCP 79.

Copies of IPR disclosures sent to the IETF Secretariat, plus any assurances of licenses that will be made available, or the outcome of any attempt to get a general license or permission for implementers or users of this spec to use the proprietary rights, can all be found in the IETF's online IPR repository at http://www.ietf.org/ipr. 🗂️

The IETF also invites anyone to flag any copyrights, patents, patent applications, or other proprietary rights that might cover tech needed to implement this standard. Send that info to the IETF at ietf-ipr@ietf.org. 🙋‍♀️✨

Acknowledgement 🌷

the RFC Editor function is funded by the Internet Society right now, and that's the whole tea 🍵💖

📦 where this page comes from