RFC 3490

Internationalizing Domain Names in Applications (IDNA)

📅 published March 2003, from ietf

Internationalizing Domain Names in Applications (IDNA) 💖

okay bestie, this is RFC 3490 🎀 and it's giving "domain names finally go international" energy ✨

Title block tea ☕: it was written by P. Faltstrom (Cisco), P. Hoffman (IMC & VPNC), and A. Costello (UC Berkeley). It's a Standards Track document, published in March 2003. The header doesn't list anything it obsoletes or updates, so there's no lineage drama here 🧚

Status of this Memo

okay so this part says it's an Internet standards track protocol for the Internet community, and it's basically begging for discussion and suggestions for improvements 🙏 If you want the official standardization state, go check the current edition of "Internet Official Protocol Standards" (STD 1). Distribution of this memo is unlimited, so share away 🌸

Copyright (C) The Internet Society (2003). All Rights Reserved. 💅

Abstract

so until this came along, domain names had no standard way to use characters outside the ASCII repertoire 😭 This doc defines internationalized domain names (IDNs) and a mechanism called Internationalizing Domain Names in Applications (IDNA) to handle them in a standard way ✨

Here's the iconic part: IDNs pull characters from a huge repertoire (Unicode), but IDNA lets those non-ASCII characters be written using only the ASCII characters that host names already allow today 🦋 That backward-compatible form is needed in existing protocols like DNS, so IDNs can roll out without touching the existing infrastructure. IDNA is only for processing domain names, not free text, so don't try to use it for your diary entries 🙈

Table of Contents

   1. Introduction..................................................  2
      1.1 Problem Statement.........................................  3
      1.2 Limitations of IDNA.......................................  3
      1.3 Brief overview for application developers.................  4
   2. Terminology...................................................  5
   3. Requirements and applicability................................  7
      3.1 Requirements..............................................  7
      3.2 Applicability.............................................  8
         3.2.1. DNS resource records................................  8
         3.2.2. Non-domain-name data types stored in domain names...  9
   4. Conversion operations.........................................  9
      4.1 ToASCII................................................... 10
      4.2 ToUnicode................................................. 11
   5. ACE prefix.................................................... 12
   6. Implications for typical applications using DNS............... 13
      6.1 Entry and display in applications......................... 14
      6.2 Applications and resolver libraries....................... 15
      6.3 DNS servers............................................... 15
      6.4 Avoiding exposing users to the raw ACE encoding........... 16
      6.5  DNSSEC authentication of IDN domain names................ 16
   7. Name server considerations.................................... 17
   8. Root server considerations.................................... 17
   9. References.................................................... 18
      9.1 Normative References...................................... 18
      9.2 Informative References.................................... 18
   10. Security Considerations...................................... 19
   11. IANA Considerations.......................................... 20
   12. Authors' Addresses........................................... 21
   13. Full Copyright Statement..................................... 22

1. Introduction ✨

okay so IDNA works by letting applications use certain ASCII name labels (starting with a special prefix) to stand in for non-ASCII name labels 🧚 The lower layers don't even need to know, so IDNA doesn't need any infrastructure changes. Like, at all. It doesn't depend on changes to DNS servers, resolvers, or protocol elements, because the existing ASCII name service in DNS is totally sufficient for IDNA 💅

This document doesn't require any applications to conform to IDNA, but apps can opt in to support IDNs while still playing nicely with existing infrastructure 🎀 If an application wants non-ASCII characters in domain names, IDNA is the only option defined right now. Adding IDNA to an app only means changing the app itself, and it leaves room for flexibility in the user interface ✨

A big chunk of the IDN convo has been about transition issues: how this works when not every component has been updated yet 😭 The proposals the IDN Working Group didn't pick would have needed user applications, resolvers, and DNS servers all updated before a user could use an IDN. IDNA said nope to that and only depends on updates to user applications. No DNS protocol changes, no DNS server changes, no resolver changes on users' computers 🦋

1.1 Problem Statement

The IDNA spec solves the problem of extending the set of characters allowed in domain names so it includes the Unicode repertoire (with some restrictions) 🌈

IDNA doesn't extend the service DNS offers to applications. Instead, apps (and, by extension, users) still see an exact-match lookup service. Either there's a single exactly-matching name or there's no match at all 💀 This model has served existing apps well, but with or without IDNs, users have to know the exact spelling of the domain names they type into things like web browsers and mail user agents. A bigger character set potentially means more ways to misspell things, especially since on something like a business card, the same appearance might visually match several Unicode code points or several sequences of code points 🙈

IDNA lets IDNs roll out gracefully. It avoids upgrading existing infrastructure (like DNS servers and mail transport agents), and it allows some basic use of IDNs in apps by using the ASCII representation of the non-ASCII name labels. Those names are super user-unfriendly to read and type, so they aren't suitable for user input 😭 But they still let you (for instance) reply to email and click URLs, even though the displayed domain name is incomprehensible to the user. To allow user-friendly input and output of IDNs, applications need to be modified to conform to this specification 🎀

IDNA uses the Unicode character repertoire, which avoids the major delays that would come from waiting for a different, specific character set to be defined for IDN purposes by some other standards developing organization 👑

1.2 Limitations of IDNA

The IDNA protocol doesn't solve every linguistic issue with users inputting names in different scripts 💅 Lots of important language-based and script-based mappings aren't covered in IDNA and need to be handled outside the protocol. For example, names entered in a mix of traditional and simplified Chinese characters won't be mapped to a single canonical name. Another example: Scandinavian names entered with U+00F6 (LATIN SMALL LETTER O WITH DIAERESIS) won't be mapped to U+00F8 (LATIN SMALL LETTER O WITH STROKE) 🌸

IDNA also doesn't go deep on a bigger problem: how to make sure a user who's entering a domain name from visual info (like a business card or billboard) or aural info (like a phone call or radio) gets the IDN right with high probability 🙈 ASCII has this problem too, like the visual mix-up between the letter 'O' and the digit zero, but the bigger character set creates more chances for similar-looking and similar-sounding names. This is a complex issue tied to languages, computer input methods, and more. Also, the kind of matching and searching you'd need for a high chance of success wouldn't fit DNS's exact-matching role at all 🧚

1.3 Brief overview for application developers

Apps can use IDNA to support internationalized domain names anywhere ASCII domain names already work, including DNS master files and resolver interfaces 🦋 (Apps can also define their own protocols and interfaces that support IDNs directly with non-ASCII representations. IDNA doesn't prescribe any particular representation for new protocols, but it still defines which names are valid and how they get compared.)

The IDNA protocol lives completely inside applications. It's not a client-server or peer-to-peer protocol. Everything happens inside the app itself 💖 When used with a DNS resolver library, IDNA sits in as a "shim" between the application and the resolver library. When used for writing names into a DNS zone, IDNA runs just before the name is committed to the zone ✨

Section 4 of this document describes two operations:

  • The ToASCII operation is used before sending an IDN to something that expects ASCII names (such as a resolver), or before writing an IDN into a place that expects ASCII names (such as a DNS master file).
  • The ToUnicode operation is used when displaying names to users, for example names obtained from a DNS zone. 👑

Important note: the ToASCII operation can fail 😭 If it fails while processing a domain name, that name can't be used as an internationalized domain name, and the application has to have some way of dealing with that failure.

IDNA requires that implementations process input strings with Nameprep [NAMEPREP], which is a profile of Stringprep [STRINGPREP], and then with Punycode [PUNYCODE]. Implementations of IDNA MUST fully implement Nameprep and Punycode; neither Nameprep nor Punycode are optional. 🔥

2. Terminology ✨

okay bestie, let's get our vocab straight 💅 The key words "MUST", "SHALL", "REQUIRED", "SHOULD", "RECOMMENDED", and "MAY" in this document are to be interpreted as described in BCP 14, RFC 2119 [RFC2119]. 🎀

A code point is an integer value associated with a character in a coded character set. 🔢

Unicode [UNICODE] is a coded character set with tens of thousands of characters, so it's giving a lot 🌈 A single Unicode code point is written as "U+" followed by four to six hexadecimal digits, and a range of code points is written as two hexadecimal numbers separated by "..", with no prefixes. ✨

ASCII means US-ASCII [USASCII], a coded character set with 128 characters tied to code points in the range 0..7F. Unicode is basically an extension of ASCII, since it includes all the ASCII characters and gives them the same code points. It's the ultimate glow-up that keeps the OG crew 🦋

The term "LDH code points" is defined in this document to mean the code points for ASCII letters, digits, and the hyphen-minus. That's U+002D, 30..39, 41..5A, and 61..7A. "LDH" is short for "letters, digits, hyphen" 🧚

[STD13] talks about "domain names" and "host names," but honestly, lots of people use the two terms interchangeably 🙈 Because [STD13] wasn't super clear, a bunch of people who are sure they know the exact definitions still disagree on them, lol. So in this document, "domain name" is used in general. Whenever the host name syntax restrictions from [STD3] come up, this document cites [STD3] explicitly 💜

A label is one individual part of a domain name. Labels are usually separated by dots. For example, "www.example.com" is made of three labels: "www", "example", and "com". (The zero-length root label from [STD13] can be explicit, as in "www.example.com.", or implicit, as in "www.example.com". It is not considered a label in this specification 🌸) IDNA extends the set of characters you can use in labels that are text. For the rest of this document, "label" is shorthand for "text label," and "every label" means "every text label." 🎀

An "internationalized label" is a label to which the ToASCII operation (see section 4) can be applied without failing, with the UseSTD3ASCIIRules flag unset. This means every ASCII label that satisfies the [STD13] length restriction is an internationalized label. So "internationalized label" is a generalization that covers both old ASCII labels and new non-ASCII labels 🫶 Most Unicode characters can show up in internationalized labels, but ToASCII fails for some input strings, and those strings are not valid internationalized labels. 🙅‍♀️

An "internationalized domain name" (IDN) is a domain name where every label is an internationalized label. That means every ASCII domain name is also an IDN, so a name can be an IDN without containing any non-ASCII characters at all 🌈 This document doesn't try to define an "internationalized host name." Just like with ASCII names, some DNS zone administrators may put restrictions on which characters or strings can be registered as labels in their zones, beyond what DNS or IDNA imposes. Those restrictions have no effect on the syntax or semantics of DNS protocol messages. A query for a name that matches no records gets the same response no matter why the name isn't in the zone. Clients issuing queries or interpreting responses can't be assumed to know anything about zone-specific restrictions or conventions 🧐

In IDNA, label equivalence is defined using the ToASCII operation, which builds an ASCII form for a given label, whether or not the label was already ASCII. Two labels are equivalent if and only if their ASCII forms produced by ToASCII match using a case-insensitive ASCII comparison. ASCII labels already had a notion of equivalence, where upper case and lower case count as the same, and the IDNA version is just an extension of that older idea ✨ Equivalent labels in IDNA are treated as alternate forms of the same label, just like "foo" and "Foo" are 💅

To let internationalized labels work with existing applications, IDNA uses an "ACE label" (ACE stands for ASCII Compatible Encoding) 🦋 An ACE label is an internationalized label that can be rendered in ASCII and is equivalent to an internationalized label that can't be rendered in ASCII. For any internationalized label that can't be rendered in ASCII, the ToASCII operation converts it to an equivalent ACE label. An ASCII label gets left alone by ToASCII. ACE labels aren't suitable for showing to users 🙈 The ToUnicode operation converts any label to an equivalent non-ACE label. Formally, an ACE label is any label that the ToUnicode operation would change, while non-ACE labels are left unchanged by ToUnicode. Every ACE label begins with the ACE prefix specified in section 5. ToASCII and ToUnicode are specified in section 4 🧚

The "ACE prefix" is defined in this document as a string of ASCII characters that appears at the beginning of every ACE label. It's specified in section 5 🌸

A "domain name slot" is defined in this document as a protocol element, a function argument, a return value (and so on) that is explicitly designated for carrying a domain name. Examples of domain name slots include: the QNAME field of a DNS query; the name argument of the gethostbyname() library function; the part of an email address after the at-sign (@) in the From: field of an email message header; and the host portion of the URI in the src attribute of an HTML <IMG> tag. General text that just happens to contain a domain name is not a domain name slot. For example, a domain name in the plain text body of an email message is not occupying a domain name slot 💖

An "IDN-aware domain name slot" is defined in this document as a domain name slot explicitly designated for carrying an internationalized domain name, as defined in this document. The designation can be static, such as in the specification of the protocol or interface, or dynamic, such as the result of negotiation in an interactive session 👑

An "IDN-unaware domain name slot" is defined in this document as any domain name slot that is not an IDN-aware domain name slot. Obviously, this includes any domain name slot whose specification predates IDNA 🍓

3. Requirements and applicability ✨

3.1. Requirements

IDNA conformance means following these four requirements 🎀

  1. Whenever dots are used as label separators, the following characters MUST be recognized as dots: U+002E (full stop), U+3002 (ideographic full stop), U+FF0E (fullwidth full stop), and U+FF61 (halfwidth ideographic full stop). 🔥
  2. Whenever a domain name is put into an IDN-unaware domain name slot (see section 2), it MUST contain only ASCII characters. Given an internationalized domain name (IDN), you can get an equivalent domain name that meets this requirement by applying the ToASCII operation (see section 4) to each label. If dots are used as label separators, change all the label separators to U+002E 🫶
  3. ACE labels obtained from domain name slots SHOULD be hidden from users when it's known that the environment can handle the non-ACE form, except when the ACE form is explicitly requested. When it's not known whether the environment can handle the non-ACE form, the application MAY use the non-ACE form (which might fail, such as by not being displayed properly), or it MAY use the ACE form (which will look unintelligible to the user). Given an internationalized domain name, you can get an equivalent domain name with no ACE labels by applying the ToUnicode operation (see section 4) to each label. When requirements 2 and 3 both apply, requirement 2 takes precedence 🙈
  4. Whenever two labels are compared, they MUST be considered to match if and only if they are equivalent, meaning their ASCII forms (obtained by applying ToASCII) match using a case-insensitive ASCII comparison. Whenever two names are compared, they MUST be considered to match if and only if their corresponding labels match, regardless of whether the names use the same forms of label separators 💜

3.2. Applicability

IDNA is applicable to all domain names in all domain name slots, except where it's explicitly excluded 🌈

This means IDNA is applicable to lots of protocols that predate IDNA. Note that IDNs occupying domain name slots in those protocols MUST be in ASCII form (see section 3.1, requirement 2) 🧚

3.2.1. DNS resource records

IDNA does not apply to domain names in the NAME and RDATA fields of DNS resource records whose CLASS is not IN. This exclusion applies to every non-IN class, present and future, except where future standards override this exclusion by explicitly inviting the use of IDNA 🦋

There are currently no other exclusions on IDNA's applicability to DNS resource records. It depends entirely on the CLASS, not on the TYPE. This will stay true even as new types are defined, unless there's a compelling reason for a new type to complicate things by imposing type-specific rules 👑

3.2.2. Non-domain-name data types stored in domain names

Although IDNA enables the representation of non-ASCII characters in domain names, that does not mean IDNA enables non-ASCII characters in other data types that get stored in domain names. For example, an email address local part is sometimes stored in a domain label (hostmaster@example.com would be represented as hostmaster.example.com in the RDATA field of an SOA record). IDNA does not update the existing email standards, which allow only ASCII characters in local parts. So unless the email standards are revised to invite the use of IDNA for local parts, a domain label that holds the local part of an email address SHOULD NOT begin with the ACE prefix. Even if it does, it is to be interpreted literally as a local part that happens to begin with the ACE prefix 🙈💅

4. Conversion operations ✨

Okay bestie, this is where the actual magic happens 🪄 An application converts a domain name that's going into an IDN-unaware slot, or one that's about to be shown to a user. This section walks through the steps for that conversion, plus the ToASCII and ToUnicode operations 💖

The input to ToASCII or ToUnicode is a single label, which is a sequence of Unicode code points. Remember that all ASCII code points are also Unicode code points 🌸 If a domain name is written in a character set that isn't Unicode or US-ASCII, it has to be transcoded to Unicode first 🫶

Starting from a whole domain name, the application does these steps:

  1. Decide whether the domain name is a "stored string" or a "query string" as described in [STRINGPREP]. If this conversion follows the "queries" rule from [STRINGPREP], set the flag called AllowUnassigned.
  2. Split the domain name into individual labels as described in section 3.1. The labels do not include the separator.
  3. For each label, decide whether or not to enforce the restrictions on ASCII characters in host names [STD3]. (Applications already faced this choice before IDNA showed up, and they can keep making the decision the same way they always have. IDNA makes no new recommendations regarding this choice 🙈) If the restrictions are to be enforced, set the flag called UseSTD3ASCIIRules for that label.
  4. Process each label with either the ToASCII or the ToUnicode operation, whichever fits. Typically, you use ToASCII if you're about to put the name into an IDN-unaware slot, and you use ToUnicode if you're displaying the name to a user. Section 3.1 gives more detail on the applicable requirements 📝
  5. If ToASCII was applied in step 4 and dots are used as label separators, change all the label separators to U+002E (full stop) 🍓

The next two subsections define the ToASCII and ToUnicode operations used in step 4.

This description of the protocol uses specific procedure names, flag names, and so on, just to make the protocol easier to specify. These names, along with the actual steps of the procedures, are not required of an implementation. In fact, any implementation that has the same external behavior specified in this document conforms to this specification. Iconic behavior 👑

4.1. ToASCII

The ToASCII operation takes a sequence of Unicode code points that make up one label and transforms it into a sequence of code points in the ASCII range (0..7F). If ToASCII succeeds, the original sequence and the resulting sequence are equivalent labels ✨

It is important to note that ToASCII can fail 💀 ToASCII fails if any of its steps fails. If any step of ToASCII fails on any label in a domain name, that domain name MUST NOT be used as an internationalized domain name. How you handle that failure is up to the application 🫠

The inputs to ToASCII are a sequence of code points, the AllowUnassigned flag, and the UseSTD3ASCIIRules flag. The output of ToASCII is either a sequence of ASCII code points or a failure condition.

ToASCII never alters a sequence of code points that's already all in the ASCII range (although it could still fail). Applying ToASCII multiple times has exactly the same effect as applying it just once 🦋

ToASCII consists of the following steps:

  1. If the sequence contains any code points outside the ASCII range (0..7F), proceed to step 2. Otherwise, skip to step 3.
  2. Perform the steps specified in [NAMEPREP] and fail if there is an error. The AllowUnassigned flag is used in [NAMEPREP].
  3. If the UseSTD3ASCIIRules flag is set, perform these checks:
    • (a) Verify the absence of non-LDH ASCII code points. That is, verify the absence of 0..2C, 2E..2F, 3A..40, 5B..60, and 7B..7F.
    • (b) Verify the absence of leading and trailing hyphen-minus. That is, verify the absence of U+002D at the beginning and end of the sequence.
  4. If the sequence contains any code points outside the ASCII range (0..7F), proceed to step 5. Otherwise, skip to step 8.
  5. Verify that the sequence does NOT begin with the ACE prefix 🙅‍♀️
  6. Encode the sequence using the encoding algorithm in [PUNYCODE] and fail if there is an error.
  7. Prepend the ACE prefix.
  8. Verify that the number of code points is in the range 1 to 63 inclusive.

4.2. ToUnicode

The ToUnicode operation takes a sequence of Unicode code points that make up one label and returns a sequence of Unicode code points. If the input sequence is a label in ACE form, the result is an equivalent internationalized label that is not in ACE form. Otherwise, the original sequence is returned unaltered 🌈

ToUnicode never fails. If any step fails, the original input sequence is returned immediately in that step 🧚

The ToUnicode output never contains more code points than its input. Note that the number of octets needed to represent a sequence of code points depends on the particular character encoding being used.

The inputs to ToUnicode are a sequence of code points, the AllowUnassigned flag, and the UseSTD3ASCIIRules flag. The output of ToUnicode is always a sequence of Unicode code points.

  1. If all code points in the sequence are in the ASCII range (0..7F), skip to step 3.
  2. Perform the steps specified in [NAMEPREP] and fail if there is an error. (If step 3 of ToASCII is also performed here, it won't affect the overall behavior of ToUnicode, but it isn't necessary.) The AllowUnassigned flag is used in [NAMEPREP].
  3. Verify that the sequence begins with the ACE prefix, and save a copy of the sequence.
  4. Remove the ACE prefix.
  5. Decode the sequence using the decoding algorithm in [PUNYCODE] and fail if there is an error. Save a copy of the result of this step.
  6. Apply ToASCII.
  7. Verify that the result of step 6 matches the saved copy from step 3, using a case-insensitive ASCII comparison.
  8. Return the saved copy from step 5.

5. ACE prefix 🎀

The ACE prefix is used in the conversion operations (section 4). It's two alphanumeric ASCII characters followed by two hyphen-minuses. It cannot be any of the prefixes already used in earlier documents, which include the following: "bl--", "bq--", "dq--", "lq--", "mq--", "ra--", "wq--", and "zq--". The ToASCII and ToUnicode operations MUST recognize the ACE prefix in a case-insensitive manner 🔥

The ACE prefix for IDNA is "xn--" or any capitalization of it 💜

So an ACE label might look like xn--de-jg4avhby1noc0d, where de-jg4avhby1noc0d is the part of the ACE label generated by the encoding steps in [PUNYCODE] 🫧

While all ACE labels begin with the ACE prefix, not all labels that begin with the ACE prefix are actually ACE labels. Non-ACE labels that begin with the ACE prefix will confuse users, so they SHOULD NOT be allowed in DNS zones 😭

6. Implications for typical applications using DNS 🌸

In IDNA, applications do the processing needed to take in internationalized domain names from users, display internationalized domain names to users, and process the inputs and outputs from DNS and other protocols that carry domain names. Lots of moving parts, but it's giving teamwork 🫶

The components and the interfaces between them look like this:

                    +------+
                    | User |
                    +------+
                       ^
                       | Input and display: local interface methods
                       | (pen, keyboard, glowing phosphorus, ...)
   +-------------------|-------------------------------+
   |                   v                               |
   |          +-----------------------------+          |
   |          |        Application          |          |
   |          |   (ToASCII and ToUnicode    |          |
   |          |      operations may be      |          |
   |          |        called here)         |          |
   |          +-----------------------------+          |
   |                   ^        ^                      | End system
   |                   |        |                      |
   | Call to resolver: |        | Application-specific |
   |              ACE  |        | protocol:            |
   |                   v        | ACE unless the       |
   |           +----------+     | protocol is updated  |
   |           | Resolver |     | to handle other      |
   |           +----------+     | encodings            |
   |                 ^          |                      |
   +-----------------|----------|----------------------+
       DNS protocol: |          |
                 ACE |          |
                     v          v
          +-------------+    +---------------------+
          | DNS servers |    | Application servers |
          +-------------+    +---------------------+

The box labeled "Application" is where the application splits a domain name into labels, sets the appropriate flags, and performs the ToASCII and ToUnicode operations. This is described in section 4 ✨

6.1. Entry and display in applications

Applications can accept domain names using any character set or sets the application developer wants, and they can display domain names in any charset. In other words, the IDNA protocol doesn't affect the interface between users and applications 💅

An IDNA-aware application can accept and display internationalized domain names in two formats: the internationalized character set(s) the application supports, and as an ACE label. ACE labels that are displayed or input MUST always include the ACE prefix. Applications MAY allow input and display of ACE labels, but they aren't encouraged to, except as an interface for special purposes (possibly debugging), or to cope with display limitations as described in section 6.4.. ACE encoding is opaque and ugly, so it should only be shown to users who absolutely need it 🙈 Because name labels encoded as ACE name labels can be rendered either as the encoded ASCII characters or as the proper decoded characters, the application MAY have an option for the user to choose their preferred display method. If it does, rendering the ACE SHOULD NOT be the default 🦋

Domain names are often stored and transported in lots of places. For example, they show up in documents like mail messages and web pages. They travel through many parts of many protocols, such as both the control commands and the RFC 2822 body parts of SMTP, and the headers and body content in HTTP. It's important to remember that domain names appear both in domain name slots and in the content that gets passed over protocols 🧠

In protocols and document formats that define how to handle the specification or negotiation of charsets, labels can be encoded in any charset the protocol or document format allows. If a protocol or document format only allows one charset, the labels MUST be given in that charset 👑

In any place where a protocol or document format allows transmission of the characters in internationalized labels, internationalized labels SHOULD be transmitted using whatever character encoding and escape mechanism that protocol or document format uses at that place 🌈

All protocols that use domain name slots already have the capacity to handle domain names in the ASCII charset. That means ACE labels (internationalized labels that have been processed with the ToASCII operation) can inherently be handled by those protocols 🎀

6.2 Applications and resolver libraries ✨

Okay bestie, so apps don't usually poke DNS directly 💅 They hand the job to the operating system's functions, which are often called "the resolver library", and they talk to it through a programming interface (API) 🎀

Because these resolver libraries today only take domain names in ASCII, applications MUST prepare labels with the ToASCII operation before passing them in 🔥 Labels that come back from the resolver library only have ASCII characters, so internationalized labels that can't be shown directly in ASCII show up in the ACE form. And ACE labels always include the ACE prefix ✨

An operating system might have a set of libraries for doing ToASCII. The input to one of those might be in one or more charsets that apps use (UTF-8 and UTF-16 are likely picks for basically any OS, and script-specific charsets are likely on localized systems) 🌈

IDNA-aware applications MUST be able to work with both non-internationalized labels (the ones that follow [STD13] and [STD3]) and internationalized labels. Both, no skipping 🫶

It's expected that newer resolver libraries will someday accept domain names in charsets other than ASCII. Application developers might one day pass domain names in Unicode, or even in their local script, to a new API for the OS resolver libraries 🦋 So the ToASCII and ToUnicode operations might happen inside those newer libraries.

Domain names passed to resolvers, or put into the question section of DNS requests, follow the rules for "queries" from [STRINGPREP] 💜

6.3 DNS servers 🧚

Domain names stored in zones follow the rules for "stored strings" from [STRINGPREP] 🍓

For internationalized labels that can't be shown directly in ASCII, DNS servers MUST use the ACE form that ToASCII makes. Every IDN served by a DNS server MUST contain only ASCII characters. Strict, but it keeps things clean 💅

If a signaling system that lets old and new DNS clients and servers negotiate gets standardized in the future, the encoding of the query in the DNS protocol itself could switch from ACE to something else, like UTF-8 👑 Whether that should happen is a separate problem, though, and this memo doesn't talk about it 🙈

6.4 Avoiding exposing users to the raw ACE encoding 🙈

Any app that might show the user a domain name pulled from a domain name slot (like from gethostbyaddr, or part of a mail header) will need an update so users don't have to see the ACE 😭

If an app decodes an ACE name with ToUnicode but can't show every character in it (say the name has characters the output system can't display), the app SHOULD show the name in ACE format, which always includes the ACE prefix. That's better than showing the replacement character (U+FFFD) 💖 It makes it easier for the user to copy the name correctly into other programs. Programs that show the ACE form by default when they can't display every character in a name label SHOULD also have a mechanism to show the ToUnicode output with as many characters as possible, using replacement characters in the spots that can't be displayed ✨

ToUnicode doesn't touch labels that aren't valid ACE labels, even if they start with the ACE prefix. After ToUnicode has run, if a label still starts with the ACE prefix, it's not a valid ACE label, and it isn't equivalent to any of the intermediate Unicode strings that ToUnicode built. 🚨

6.5 DNSSEC authentication of IDN domain names 🔐

DNS Security [RFC2535] is a method for sending cryptographic verification info along with DNS messages 🌸 Public Key Cryptography is used together with digital signatures so that a requester of domain info can authenticate where the data came from. This means the data can be traced back to a trusted source, either directly or through a chain of trust that links the source to the top of the DNS hierarchy 👑

IDNA says that all internationalized domain names served by DNS servers that can't be shown directly in ASCII must use the ACE form made by ToASCII. This has to happen before a zone gets signed with that zone's private key 🔥 Because of that order, it matters that DNSSEC authenticates the ASCII domain name, not the Unicode form or the mapping between the Unicode and ASCII forms. With DNSSEC in the picture, this is the name that MUST be signed in the zone and MUST be validated against 🫶

One consequence for sites deploying IDNA alongside DNSSEC: any special-purpose proxies or forwarders used to turn user input into IDNs must sit earlier in the resolution flow than the DNSSEC-authenticating nameservers, or DNSSEC won't work 🌈

7. Name server considerations 🏰

Existing DNS servers don't know the IDNA rules for handling non-ASCII forms of IDNs, so they need to be shielded from them 🛡️ All the existing channels that let names into a DNS server database (for example, master files [STD13] and DNS update messages [RFC2136]) are IDN-unaware because they came before IDNA. That's why requirement 2 of section 3.1 of this document does the shielding, by making sure internationalized domain names entering server databases through those channels have already been converted to their equivalent ASCII forms 💜

It is imperative that there be only one ASCII encoding for a particular domain name 😤 Because of how ToASCII and ToUnicode were designed, no ACE labels decode to ASCII labels, so name servers can't end up holding multiple ASCII encodings of the same domain name 🎀

[RFC2181] explicitly allows domain labels to contain octets beyond the ASCII range (0..7F), and this document doesn't change that 🦋 Heads up though: there's no defined interpretation of octets 80..FF as characters. If labels with those octets get returned to applications, unpredictable behavior could result 💀 The ASCII form that ToASCII defines is the only standard representation for internationalized labels in the current DNS protocol ✨

8. Root server considerations 🌍

IDNs are likely to be a bit longer than current domain names, so the bandwidth the root servers need will probably go up by a small amount 📈 Queries and responses for IDNs will also probably be somewhat longer than typical queries today, so more of them may be forced onto TCP instead of UDP 🙈

9. References 📚

9.1 Normative References 💅

  • [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
  • [STRINGPREP] Hoffman, P. and M. Blanchet, "Preparation of Internationalized Strings ("stringprep")", RFC 3454, December 2002.
  • [NAMEPREP] Hoffman, P. and M. Blanchet, "Nameprep: A Stringprep Profile for Internationalized Domain Names (IDN)", RFC 3491, March 2003.
  • [PUNYCODE] Costello, A., "Punycode: A Bootstring encoding of Unicode for use with Internationalized Domain Names in Applications (IDNA)", RFC 3492, March 2003.
  • [STD3] Braden, R., "Requirements for Internet Hosts -- Communication Layers", STD 3, RFC 1122, and "Requirements for Internet Hosts -- Application and Support", STD 3, RFC 1123, October 1989.
  • [STD13] Mockapetris, P., "Domain names - concepts and facilities", STD 13, RFC 1034 and "Domain names - implementation and specification", STD 13, RFC 1035, November 1987.

9.2 Informative References 📖

10. Security Considerations 🔐✨

okay bestie, security is literally the vibe check of the whole internet 💅 because the DNS holds so much of it up, any change to how the DNS behaves can change the security of a huge chunk of the internet 😭🫶

this memo describes an algorithm that takes characters which aren't valid under STD3 and STD13 and encodes them into octet values that are valid 🦋 and no new security issues come from the encoding itself or from using those encoded values, except the ones the ACE encoding brings along on its own 🎀

domain names are how users find and connect to servers 🌸 so if one internationalized name ends up connecting people to different servers because different systems read it in different ways, that's a whole security problem 🙈 not me... the whole thing gets compromised 💀

when systems use local character sets besides ASCII and Unicode, this spec leaves the transcoding between that local character set and Unicode up to the application 🧚 if different apps (or different versions of the same app) use different transcoding rules, they could read the same name differently and end up at different servers 😭 and security protocols like TLS don't fix this, since they don't take local character sets into account 💔

because this document normatively refers to [NAMEPREP], [PUNYCODE], and [STRINGPREP], it also includes the security considerations from those documents 🌈

if this spec ever gets updated to use a newer Unicode normalization table, that new table has to be compared with the old one to catch backwards incompatible changes 👀 if there are any, they need handling, or you get security and operational problems 🔥 ways to handle the conflicts could include keeping the old normalization, dealing with the conflicting characters through operational means, or some other method 💜

Implementations MUST NOT use more recent normalization tables than the one referenced from this document, even if the operating system provides newer tables 🚨 if an application isn't sure which normalization tables the OS has, the application needs to include the normalization tables itself 🫶 using normalization tables other than the one referenced here could have security and operational implications 💀

to help stop people getting confused by characters that look alike 👀 it's suggested that implementations give visual indications when a domain name contains multiple scripts ✨ those same mechanisms can show when a name mixes simplified and traditional Chinese characters, or tell zero and one apart from O and l 🙈 DNS zone administrators may impose restrictions (subject to the limitations in section 2) that try to minimize homographs 💅

domain names (or parts of them) are sometimes compared against a set of privileged or anti-privileged domains 👑 in those situations it's especially important that the comparisons are done properly, as specified in section 3.1 requirement 4 😭 for labels already in ASCII form, the proper comparison just reduces to the same case-insensitive ASCII comparison that has always been used for ASCII labels 🌸

the introduction of IDNA means that any existing labels starting with the ACE prefix that would be altered by ToUnicode will automatically be ACE labels 🦋 and they'll be treated as equivalent to non-ASCII labels, whether or not the zone administrator or registrant meant that 💀 weird but true 🫠

11. IANA Considerations 📋

okay short and sweet: IANA has assigned the ACE prefix in consultation with the IESG 🎀✨

12. Authors' Addresses 🖊️

Patrik Faltstrom Cisco Systems Arstaangsvagen 31 J S-117 43 Stockholm Sweden

EMail: paf@cisco.com

Paul Hoffman Internet Mail Consortium and VPN Consortium 127 Segre Place Santa Cruz, CA 95060 USA

EMail: phoffman@imc.org

Adam M. Costello University of California, Berkeley

URL: http://www.nicemice.net/amc/

Copyright (C) The Internet Society (2003). All Rights Reserved. 🌈

this document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind 🦋 as long as the above copyright notice and this paragraph are included on all such copies and derivative works 💜 however, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations 🙅‍♀️ except as needed for developing Internet standards, in which case the procedures for copyrights defined in the Internet Standards process must be followed 📜 or as required to translate it into languages other than English 🌍

the limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns ♾️✨

this document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE 😭🫶

Acknowledgement 🙏

funding for the RFC Editor function is currently provided by the Internet Society 💖

📦 where this page comes from