RFC 8228

RFC 8228: Guidance on Designing Label Generation Rulesets (LGRs) Supporting Variant Labels 🎀

• Request for Comments: 8228 🌸
• Author: A. Freytag 💅
• Date: August 2017 📅
• Category: Informational (published for information, not a standards-track spec 🧠)
• ISSN: 2070-1721
• Obsoletes / Updates: this header doesn't list any RFCs it obsoletes or updates 🤷‍♀️

Abstract ✨

okay so LGRs (Label Generation Rulesets) are the rulebooks that decide which identifier labels are valid, plus the alternate spellings of those labels, which we call variants 🦋. They get used for identifier systems like Internationalized Domain Names (IDNs) 🌍. This doc is all about how to design LGRs that handle variant labels well 💖. The big thing is making sure the label generation rules stay consistent and behave properly even when variants are around 👑. Once you've made those design choices, you write them down using the XML format defined in RFC 7940 📝✨.

Status of This Memo 📜

ngl this part is the legal-ish stuff 😭. This document is not an Internet Standards Track spec. It's just published for info 🌸. It's a product of the IETF and was approved for publication by the IESG 💜. Not every IESG-approved document is up for any level of Internet Standard, so check Section 2 of RFC 7841 for that 💅.

For the current status, errata, and how to send feedback, go to http://www.rfc-editor.org/info/rfc8228 🔗.

Copyright Notice 🎀

Copyright (c) 2017 IETF Trust and the persons identified as the document authors. All rights reserved 🌈. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as they were in effect on the publication date. Please actually read those, because they spell out your rights and restrictions 🙈. Any Code Components pulled out of this document have to include the Simplified BSD License text described in Section 4.e of the Trust Legal Provisions, and they come with no warranty, as the Simplified BSD License describes ⚠️.

Table of Contents 🧚

1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   3
2.  Variant Relations . . . . . . . . . . . . . . . . . . . . . .   4
3.  Symmetry and Transitivity . . . . . . . . . . . . . . . . . .   5
4.  A Word on Notation  . . . . . . . . . . . . . . . . . . . . .   5
5.  Variant Mappings  . . . . . . . . . . . . . . . . . . . . . .   6
6.  Variant Labels  . . . . . . . . . . . . . . . . . . . . . . .   7
7.  Variant Types and Label Dispositions  . . . . . . . . . . . .   7
8.  Allocatable Variants  . . . . . . . . . . . . . . . . . . . .   8
9.  Blocked Variants  . . . . . . . . . . . . . . . . . . . . . .   9
10. Pure Variant Labels . . . . . . . . . . . . . . . . . . . . .  10
11. Reflexive Variants  . . . . . . . . . . . . . . . . . . . . .  11
12. Limiting Allocatable Variants by Subtyping  . . . . . . . . .  12
13. Allowing Mixed Originals  . . . . . . . . . . . . . . . . . .  14
14. Handling Out-of-Repertoire Variants . . . . . . . . . . . . .  15
15. Conditional Variants  . . . . . . . . . . . . . . . . . . . .  16
16. Making Conditional Variants Well Behaved  . . . . . . . . . .  18
17. Variants for Sequences  . . . . . . . . . . . . . . . . . . .  19
18. Corresponding XML Notation  . . . . . . . . . . . . . . . . .  21
19. IANA Considerations . . . . . . . . . . . . . . . . . . . . .  22
20. Security Considerations . . . . . . . . . . . . . . . . . . .  23
21. References  . . . . . . . . . . . . . . . . . . . . . . . . .  23
  21.1.  Normative References . . . . . . . . . . . . . . . . . .  23
  21.2.  Informative References . . . . . . . . . . . . . . . . .  23
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . .  24
Author's Address  . . . . . . . . . . . . . . . . . . . . . . . .  24

1. Introduction 🦋

LGRs define which labels are allowed, and they can be applied to identifier systems that run on labels, like the Domain Name System (DNS) [RFC1034] [RFC1035] 🌐. So far LGRs have mostly been used to set policy for IDNs built on IDNA2008 [RFC5890] [RFC5891] [RFC5892] [RFC5893] [RFC5894] in DNS 💖. This doc talks about making LGRs for that situation, but the techniques here are almost certainly useful for other internationalized identifiers too ✨.

Besides deciding whether a label is eligible, LGRs can also say when alternate representations of a label (so-called "variant labels") can exist, and what their status (disposition) is 🎀. Generally, variant labels are labels that look the same or mean the same as another label in the writing system or script the LGR supports 👀. Labels that are merely similar are different: those have some measurable degree of resemblance. Variant labels here are about real equivalence in meaning or appearance 💅. Deciding what counts as a good variant for a given writing system or context (especially in the DNS) is assumed to have been sorted out beforehand, so this doc doesn't cover it 🙈.

Once variant labels are identified, they usually get delegated to some entity along with the applied-for label, or they get permanently reserved, depending on the disposition from the LGR 🔐. Done right, variant labels can make an LGR more secure, but getting variant rules to behave well isn't always easy 😭. This doc goes over the basic considerations and constraints you need to keep in mind, and gives examples of use cases for the different types of variant specifications in an LGR 🌸.

This doc does not weigh in on whether variants are the right tool for a given problem, or on what basis they should be defined 🤷‍♀️. Its goal is to explain in more detail what various declarations do and what trade-offs come with design choices 🧠. It assumes any LGR is written in the XML representation defined in [RFC7940] and therefore follows whatever that RFC requires 💜. For clarity, the examples here use a more compact notation than the XML syntax in [RFC7940]. Even so, the reader is expected to already know the concepts from that RFC (see Section 4) 📚.

Users interact with an identifier system like the DNS through labels. Variants show up to them as variant labels: two or more labels that are functionally "same as" under the writing system's conventions, even though their code point sequences differ 🦋. An LGR specification defines variant mappings between code points first, and only in a second step derives variant labels from those mappings. For more on that process, see [RFC7940] 🧚.

The designer of an LGR controls whether some or all of the variant labels made from an original label are allocatable (available for allocation to the original applicant), or whether some or all of them are blocked (not allocatable to anyone) 👑. This doc explains how that disposition choice is made (see Section 7) 💅.

The disposition someone wants depends on what the users of a particular zone expect, and that isn't this doc's topic 🌈. This doc also doesn't cover LGRs defining custom label dispositions. Instead, it suggests ways to design an LGR that gets the chosen approach to variants, using the two standard dispositions: "allocatable" and "blocked" ✨.

The info here comes from real operational experience building LGRs for lots of languages and scripts using RFC 7940, and it's shared to help the wider community 💖. It doesn't change or alter the specification in RFC 7940 in any way 🫶.

2. Variant Relations 💜

A variant relation is basically a "same as" relation, which means it's an equivalence relation 🦋. The strictest version of "same as" is equality, and any equality has both symmetry:

  A = B => B = A

and transitivity:

  A = B and B = C => A = C

The variant relation, with its functional "same as" sense, has to satisfy the same constraints 🔥. Once we say A is the "same as" B, we're also claiming that B is the "same as" A 😭. This doc uses the symbol "~" to mean "has a variant relation with", so we get:

  A ~ B => B ~ A

If we make the same claim about B and C (B ~ C), we get A ~ C too, because if B is the "same as" both A and C, then A has to be the "same as" C 💅:

  A ~ B and B ~ C => A ~ C

3. Symmetry and Transitivity 🧚

Not every possible relation between labels is an equivalence, and the ones that aren't can fail to be transitive and might not be symmetric either 🙈. For example, confusability isn't transitive. Two labels can both be confusingly similar to a third without being confusable with each other, like when the third one's shape sits "in between" the other two 🌸. A relation based on identical or effectively identical appearance, though, does meet the transitivity test, and we'd count it as a variant relation ✨. Other forms of equivalence, like semantic equivalence, are examples of variant relations too 💖.

Using [RFC7940], you could define a set of mappings that's neither symmetric nor transitive, and it would still be formally valid 😬. But a symmetric and transitive set of mappings is strongly preferred as the basis for an LGR 👑. Part of the reason is the implementation benefits. For instance, if all mappings are symmetric and transitive, checking for collisions between labels with variants gets way simpler 🔥. So this doc only discusses relations that are symmetric and transitive 💅. Also, it's often pretty straightforward to check mechanically whether an LGR is symmetric and/or transitive, and to compute the mappings needed to make it so (but see Section 15) 🦋.

4. A Word on Notation 📝

[RFC7940] defines an XML schema for LGRs in general, and for variant code points and sequences in particular (see Section 18) 🎀. Honestly that XML is pretty verbose, and it can easily hide the important features from anyone who isn't trained to read XML 😵‍💫. So this doc uses a symbolic shorthand in its examples 💖. The shorthand is only a teaching tool. It isn't meant to replace or stand in for the XML syntax used to formally specify an LGR under [RFC7940] 🫶.

When it's time to capture the LGR in a formal definition, the notation from any example in this doc can be converted to XML as described in Section 18 ✨.

5. Variant Mappings ✨

okay so, up until now we've been treating variant relations like simple "same as" vibes 💅, but each one is really a symmetric pair of reciprocal mappings, so it goes both ways. bestie, this is the whole mood 🦋 In this document, the symbol --> means "maps to".

A ~ B => A --> B, B --> A

In an LGR, these mappings aren't defined straight between labels. They're defined between code points (or code point sequences; see Section 17) 🌸 In the transitive case, we start with

A ~ B => A --> B, B --> A

A ~ C => A --> C, C --> A

and then we also get

B ~ C => B --> C, C --> B

for a total of six possible mappings 😭 By convention, these get listed in tables ordered by source code point, like this:

A --> B
A --> C
B --> A
B --> C
C --> A
C --> B

As you can see, A, B, and C can each be mapped two ways 💖

6. Variant Labels 🧚

okay, so making a variant label is basically a little code point swap party 🎀 Each code point in the original label gets swapped, one at a time, for every variant code point that a mapping from the original code point points to. Say we have the label AAA (the letter "A" three times). Using the transitive example from above, the variant labels would be

AAB
ABA
ABB
BAA
BAB
BBA
BBB
AAC
...
CCC

ngl, so far we've only defined what the variant labels are. We haven't talked about what happens to them yet 👀 In the next section, we get into how to set up the variant mappings so that some variant labels are mutually exclusive (blocked), while others can be given out to the same applicant as the original label (allocatable). it's giving plot twist 🍓

7. Variant Types and Label Dispositions 👑

say we want to allow a variant relation between code points O and A, and maybe between O and B or O and C too. With transitivity, that gives us:

O ~ A ~ B ~ C

Now here's the tea 🫖 We want to tell apart someone applying for OOO from someone applying for the label ABC. If someone applies for OOO, we only want to allocate OOO. But if someone applies for ABC, we want to be able to allocate OOO, or ABC, or both, and NOT any of the other variant labels like OAO, BCO, or anything like that. (A real-world example: O is an unaccented letter, while A, B, and C are different accented forms of the same letter. Since unaccented letters are a common fallback, you might want an unaccented label to be allocatable as a variant, but not the other way around 💅)

So how do we actually specify that? The trick is to label the mappings A --> O, B --> O, and C --> O with the type "allocatable", and the mappings O --> A, O --> B, and O --> C with the type "blocked". In this document, the symbol x--> means "maps with type blocked", and the symbol a--> means "maps with type allocatable". So:

O  x--> A
O  x--> B
O  x--> C
A  a--> O
B  a--> O
C  a--> O

When we generate all the permutations of labels, the mapping types we use depend on which code point we start from 🌈 The full set of permuted variant labels is the same either way, but the disposition of each variant label depends on which label we started from. We call that starting label the "original" or "applied-for" label.

When you build an LGR with variants, every variant mapping should always be labeled with a type ([RFC7940] doesn't formally require a type, but any well-behaved LGR would be fully typed ✨). By default, these types line up directly with the dispositions for variant labels, and the most restrictive type decides the disposition of the variant label. Later on we'll see that sometimes it's useful to pick types from a wider set of values than the final dispositions, and then spell out explicitly how to get the label dispositions from them 🦋

8. Allocatable Variants 💖

If we start with AAA and use the mappings from Section 7, the permutation OOO comes out of applying the mapping A a--> O at each code point. In other words, only mappings with type "a" (allocatable) were used. To figure out whether we can allocate both OOO and the original label AAA, we keep track of the types of the mappings used to generate the label 🔥

We record the variant types for each variant mapping used to create the permutation in an ordered list. That ordered list is called a "variant type list". In running text, we usually show it inside square brackets. For example, [a x -] means the variant label came from a variant mapping with type "a" in the first code point position, "x" in the second, and the original code point in the third position (where "-" means "no variant mapping") 🌸

For our example permutation, the variant type list looks like this (brackets dropped):

AAA --> OOO : a a a

From the variant type list, we derive a "variant type set", written in curly braces. It's an unordered set of the unique variant types in the variant type list. For the variant type list [a a a], the variant type set is { a }, which has one element, "a" 🧚

Deciding whether to allow allocation of a variant label then comes down to deriving a disposition from the variant type set of the mappings used to build that label. For example, this derivation

if "all variants" = "a" => set label disposition to "allocatable"

would allow OOO to be allocated, because every variant mapping used to build that variant label from AAA has type "a" 🎀

The "all-variants" condition is tolerant of an extra "-" in the variant set (unlike the "only-variants" condition we'll cover in Section 10). So if we had started with AOA, OAA, or AAO, the variant set for the permuted variant OOO would have been { a - }, because in each case one of the code points stays the same as the original. The "-" just means that, since there's no mapping O --> O, there's no variant type for that O 🙈

Because the "all-variants" = "a" condition ignores the "-", the same derivation tells us OOO is an allocatable variant for each of AOA, OAA, and AAO. ngl that's a lot of allocatable energy 😭

Allocatable variant labels, especially lots of them per label, cost users of the LGR something. A well-behaved LGR will keep the number of allocatable variants as low as it can 💅

9. Blocked Variants 🔥

Blocked variants aren't available to any other registrant. That means they protect the applicant of the original label from someone else registering a label that is the "same as" under some user-perceived metric 🛡️ Blocked variants can be a really useful tool even for scripts where no allocatable labels are ever defined 🌈

If we start with OOO and use the mappings from Section 7, the permutation AAA comes out of applying only mappings with type "blocked". That means we can't allocate AAA, only the original label OOO. This matches the following derivation:

if "any variants" = "x" => set label disposition to "blocked"

Now, to stop ABO from being allocated as a variant label for AAA, we need to make sure the mapping A --> B has been defined with type "blocked", like so:

A  x--> B

Then:

AAA --> ABO: - x a.

So the set {x a} contains at least one "x", which satisfies the blocked-disposition derivation for ABO when AAA is the applied-for label 💖

If an LGR gives you a symmetric and transitive set of variant labels, then figuring out whether a label or its variants collide with another label or its variants can be implemented really efficiently ✨ Symmetry and transitivity mean that sets of labels which are mutual variants of each other are disjoint from all other such sets. Only labels inside the same set can be variants of each other. Identifying the variant set can be an O(1) operation, and you don't have to enumerate all the variants 🧚

10. Pure Variant Labels 🦋

Now, if we want to stop AOA from being allocated when we start from AAA, we need a rule that disallows mixing original code points with variant code points. That's easy with the "only-variants" qualifier, which requires the label to consist entirely of variants, and all of those variants to come from the same set of types 💅

if "only-variants" = "a" => set label disposition to "allocatable"

The two code points A in AOA weren't reached by variant mappings. Those code points are unchanged, and no variant mappings are defined for A --> A. So in our example, the set of variant mapping types is

AAA --> AOA:  - a -

But unlike the "all-variants" condition, "only-variants" requires a variant type set of { a }, corresponding to a variant type list of [a a a] (no "-" allowed) 🎀 By adding a final derivation

else if "any-variants" = "a" => set label disposition to "blocked"

and running that derivation only on whatever labels are left over, we disallow AOA when starting from AAA, but still allow OOO 🌸

Derivation conditions always get applied in order, and later derivations only apply to labels that didn't match any earlier conditions. That's what the "else" in the last example is doing. Put together, they form a cascade 🌈

11. Reflexive Variants 🍓

But what if we started from AOA? We'd expect the original label OOO to be allocatable, but using the mappings from Section 7, the variant type set would be

AOA --> OOO:  a - a

because the middle O hasn't changed from the original code point. This is where a reflexive mapping comes in 🙌 Since O is the "same as" O, we can map it to itself. That's normally redundant, but adding an explicit reflexive mapping lets us assign a disposition to that mapping:

O  a--> O

With that in place, the variant type list for AOA --> OOO becomes:

AOA --> OOO: a a a

and the label OOO passes the derivation condition

if "only-variants" = "a" => set label disposition to "allocatable"

exactly as we wanted ✨ You'll see reflexive variants used a lot whenever "only-variants" derivations are involved. And if any code point uses a reflexive variant, a well-behaved LGR should specify an appropriate reflexive variant for every code point 💜

12. Limiting Allocatable Variants by Subtyping ✨

okay so we already know the number of variant labels can get absolutely massive because of combinatorics 😭 sometimes the move is to sort variants into categories and say only labels whose variants come from the same category are allocatable. 🫶 for some LGRs, you can do that with a rule that stops code points from different categories showing up together in one allocatable label. for other LGRs, splitting the allocatable variants into subtypes may be the better move 💅

quick recap of the standard case: a code point C can have (up to) two kinds of variant mappings:

C  x--> X
C  a--> A

where a--> means a variant mapping with type "allocatable" and x--> means "blocked". for the rest of this section, we name the target code point with the matching uppercase letter 🌸

now subtyping lets us tell apart different flavours of allocatable variants 🦋 for example, we can define three new types: "s", "t", and "b". "s" and "t" are mutually incompatible, but "b" is compatible with either one (here "b" stands for "both", iykyk 💜). a real-world example is variant mappings for "simplified" or "traditional" Chinese, or for both.

with those subtypes, a code point C might have (up to) four kinds of variant mappings:

C  x--> X
C  s--> S
C  t--> T
C  b--> B

and explicit reflexive mappings of one of these types:

C  s--> C
C  t--> C
C  b--> C

as before, all mappings must have one and only one type, but each code point may map to any number of other code points 🎀

our compatibility rules for "b" with "t" or "s" come from how we pick the derivation conditions:

if "any-variants" = "x" =>  blocked
else if "only-variants" = "s" or "b" =>  allocatable
else if "only-variants" = "t" or "b" =>  allocatable
else if "any-variants" = "s" or "t" or "b" =>  blocked

an original label of four code points:

CCCC

can have lots of variant labels, like this one with its variant type list:

CCCC --> XSTB : x s t b

this one is blocked because getting from C to B needs an x--> 💀 (variant mappings are defined for specific source code points, so we have to show the starting label for each example, not just the code points in the variant label.)

CCCC --> SSBB : s s b b

this one is allocatable, because the type list only has allocatable mappings of subtype "s" or "b", which we defined as compatible ✨ the actual set of types is {s, b} with only two members, but it's easier to follow if we list each type.

CCCC --> TTBB : t t b b

also allocatable, because the type set {t, b} only has allocatable mappings of the mutually compatible subtypes "t" or "b" 🍓

CCCC --> SSTT : s s t t

NOT allocatable 🙅‍♀️ the type set has the incompatible subtypes "t" and "s", so the final derivation blocks it.

CCCC --> CSBB : c s b b
CCCC --> CTBB : c t b b

these two are only allocatable based on the subtype of the C --> C mapping, shown here as "c". depending on what we chose for the reflexive mapping's type, "c" could be "s", "t", or "b".

if the subtype is "s", the first label is allocatable. if it's "t", the second one is. if it's "b", both are allocatable 👑

so far this doesn't look like a huge reduction in allocatable variant labels, but that's only because we quietly assumed C could have all three allocatable types "s", "t", and "b" at once 🙈

in a real-world setup, "s", "t", and "b" get assigned so each code point C normally has at most one non-reflexive variant mapping with one of those subtypes, and all its other mappings are type "x" (blocked). this holds for most code points in existing tables (like the ones used in current IDN Top-Level Domains (TLDs)), though some code points have unusually complex variant relations and may have an extra mapping 🌈

---

13. Allowing Mixed Originals 🦋

okay so if you want to allow original labels (but NOT variant labels) that mix s and t, the scheme needs a little glow-up 💖 we have to tell reflexive mappings apart from non-reflexive ones. in this document, "r-n" means "a reflexive (identity) mapping of type 'n'". so the reflexive mappings from the last section become:

C  r-s--> C
C  r-t--> C
C  r-b--> C

with that, and with the derivations redefined as:

if "any-variants" = "x" =>  blocked
else if "only-variants" = "s" or "r-s" or "b" or "r-b" => allocatable
else if "only-variants" = "t" or "r-t" or "b" or "r-b" => allocatable
else if "any-variants" = "s" or "t" or "b"  => blocked
else => allocatable

any labels with only reflexive mappings of an otherwise mixed type (aka any mixed original label) now fall through, and their disposition gets set to "allocatable" in the final derivation ✨

in a well-behaved LGR, though, it's better to explicitly define the derivation for allocatable labels instead of relying on a fall through 🫶 with the derivation above, code points without any variant mappings fall through and become allocatable by default if they're part of an original label. in a big repertoire, it can be hard to figure out which code points that affects 😭 so instead, it's better to mark them with their own reflexive mapping type, "neither" or "r-n":

C  r-n--> C

with that in place, we can swap

     else =>  allocatable

for

     else if "only-variants" = "r-s" or "r-t" or "r-b" or "r-n"
          =>  allocatable
     else => invalid

this makes the intent super explicit 💅 and since every code point in the LGR has some kind of reflexive mapping, it's way easier to check that their types were assigned correctly 🧚

---

14. Handling Out-of-Repertoire Variants 💅

at first this might sound counterintuitive: why define variants that map to code points that aren't even in the repertoire? 🙈 but for zones where multiple LGRs are defined, there can be situations where a label that's valid under one LGR should be blocked because a label under another LGR is already delegated. this can happen whether or not the repertoires of the affected LGRs overlap, and where they do overlap, whether or not the labels are both restricted to the shared subset 💀

to handle that exclusion relation through variant definitions, we need to be able to map to some code point X that's outside an LGR's repertoire, R:

C  x--> X : where C = elementOf(R) and X != elementOf(R)

because of symmetry, we also have to specify the inverse mapping in the LGR:

X  x--> C : where X != elementOf(R) and C = elementOf(R)

this makes X a source of variant mappings, so we need to mark X as outside the repertoire. that way, any attempt to apply for a label containing X gets a disposition of "invalid", just like X had never been listed in the LGR 🦋 the mechanism uses reflexive variants, but with a new reflexive mapping type of "out-of-repertoire-var", shown as "r-o-->":

X  r-o--> X

this says X != elementOf(R), as long as the LGR has a suitable derivation, so that any label containing "r-o-->" gets "invalid", just like any other code point not in the repertoire. the derivation used is:

if "any-variant" = "out-of-repertoire-var" => invalid

it gets inserted ahead of any other derivation of the "any-variant" kind in the chain of derivations 👑 so instead of the minimum two symmetric variants, any out-of-repertoire variant now has a minimum of three variant mappings defined:

C  x--> X
X  x--> C
X  r-o--> X

where C = elementOf(R) and X != elementOf(R).

since no variant label containing a code point outside the repertoire could ever be allocated, the only logical choice for the non-reflexive mappings to out-of-repertoire code points is "blocked" 🫶

---

15. Conditional Variants 🌸

okay so variant mappings are based on whether code points are "same as" to the user 👀 but in some writing systems, code points change shape depending on where they sit in a word (positional forms). some code points match in some positions but not others. in those cases, the variant mapping only exists for some positions or, more generally, for some contexts. in other contexts, the mapping just doesn't exist 💖

for example, say two code points look the same at the end of a label (in final position) but not anywhere else. then they're only variants when they're in final position, and we write that like this:

final: C --> D

in cursively connected scripts, like Arabic, a code point can take its final form next to any following code point that breaks the cursive connection, not just at the end of a label ✨ (we're ignoring the isolated form to keep things simple; if we included it, "final" might be "final-or-isolate", for example.)

from symmetry, we expect the mapping D --> C to also only exist when D is in final position 🎀 (similar things apply to transitivity.)

sometimes a code point has a final form that's basically the same as another code point's, while sharing its initial and medial forms with a different one:

final: C --> D
!final: C --> E

here, the opposite of final is written as "!final" 🦋

since shapes change by position, when a context is applied to a variant mapping, it's treated independently from that same mapping in other contexts. that goes for the types too 🧚 for example, the mapping C --> F might be "allocatable" in final position but "blocked" everywhere else:

final:  C  a--> F
!final: C  x--> F

now, the type on the forward mapping is independent of the reverse symmetric mapping or any transitive mappings 💅 picture a case where the symmetric mapping is F a--> C, meaning every mapping from F to C is "allocatable":

final: F  a--> C
!final: F  a-->C

why not just write F a--> C? because the forward mapping is split by context. adding a context makes the two forward variant mappings distinct, and that has to be spelled out in the reverse mappings so that human and machine readers can easily check symmetry and transitivity in the LGR 🫶 (this is true even though the two opposite contexts "final" and "!final" should together cover every possible case.)

16. Making Conditional Variants Well Behaved 💅

okay so, to keep LGR with contextual variants well behaved, you really want to always use "fully qualified" variant mappings 🎀 that agree on the names of the context rules for both forward and reverse mappings. You also need to make sure no label can match more than one context for the same mapping. Using mutually exclusive contexts, like "final" and "!final", is an easy way to get that ✨

BUT (and this is the tea) you don't always need dual or multiple contexts that cover every single possible case. For example, these two contexts don't cover all the possible positional contexts:

final: C --> D
initial: C --> D.

A well-behaved LGR using these two contexts would define all symmetric and transitive mappings involving C, D, and their variants consistently in terms of the two conditions "final" and "initial". It would also make sure that both can't be satisfied at the same time by some label 🫶

Also, a well-behaved LGR never mixes a variant mapping with a context and the same variant mapping without one 😭

context: C --> D
C --> D

A parser can detect and flag this kind of inadvertent mixing of conditional and unconditional variants. Checking that two formally distinct contexts are never satisfied by the same label, though, depends on how labels and context rules interact. So that's on the LGR designer to make sure the LGR is well behaved 💜

A well-behaved LGR never assigns conditions to a reflexive variant. That's effectively the same as putting a context on the code point itself, and the latter is the preferred option 🌸

Finally, for symmetry to work the way you'd expect, the context has to be satisfied for both the original code point in the context of the original label, and for the variant code point in the variant label. In other words, the context should be "stable under variant substitution" anywhere in the label. It's giving consistency ✨

Positional contexts usually meet this last condition. For example, a code point that interrupts a cursive connection would likely share this property with any of its variants. But since it's possible in principle to define other kinds of contexts, the LGR designer needs to make sure the LGR is well behaved in this respect when it's designed 🦋

Because these constraints are hard to verify mechanically, an LGR designer is required to document why the LGR can be expected to meet them, and the details of the techniques used to get that outcome. Hmm, correction, the original says it is essential that the designer does this 🙈 This information should be found in the description element of the LGR.

Real talk: conditional contexts can be useful for some cases, but extra care is needed to make sure an LGR containing them is well behaved. LGR designers would be well advised to avoid conditional contexts and prefer unconditional rules whenever that's practical, even though it will doubtlessly reduce the number of labels practically available 🥺

---

17. Variants for Sequences 🧚

Variant mappings can be defined between sequences, or between a code point and a sequence. For example, you could define a "blocked" variant between the sequence "rn" and the code point "m", because they're practically indistinguishable in common UI fonts 👀

These variants are no different from variants defined between single code points, except if a sequence is defined so that there's a code point or shorter sequence that is a prefix (initial subsequence), and both that prefix and the remainder are also part of the repertoire. In that case, you can accidentally create duplicate variants with conflicting dispositions 💀

Here's an example that results in conflicting reflexive variants:

A  a--> C
AB  x--> CD

where AB is a sequence with an initial subsequence of A. For example, B might be a combining code point used in the sequence AB. If B only shows up in that sequence, there's no issue. But if B also shows up by itself, like this:

B  a--> D

then a label "AB" could correspond to either {A}{B}, which is the two code points, or {AB}, the sequence. (The curly braces show the sequence boundaries as they'd be applied during label validation and variant mapping.)

So a label AB would generate the "allocatable" variant label {C}{D} and the "blocked" variant label {CD}, which makes two variant labels with conflicting dispositions. Chaos 😭

For the "m" and "rn" example (and vice versa), there's no issue as long as "r" and "n" don't have variant mappings of their own. That way there can't be multiple variant labels for the same input. Still, it's better to avoid ambiguities altogether where you can 🌈

The easiest way to avoid an ambiguous segmentation into sequences is to never allow both a sequence and all of its constituent parts at the same time as independent parts of the repertoire. For example, don't define B by itself as a member of the repertoire 🍓

Sequences are often used for combining sequences, which consist of a base character B followed by one or more combining marks C. If you enumerate every sequence where a certain combining mark is expected, and you don't list the combining mark by itself in the LGR, then the mark can't show up outside those specifically enumerated contexts. Super clean 🫶

When enumeration isn't possible or practicable, other techniques can prevent ambiguous segmentation. One example is a context rule on code points that disallows B preceding C in any label, except as part of a predefined sequence or class of sequences. The details of those techniques are outside the scope of this document (see [RFC7940] for information on context rules for code points) 📚

---

18. Corresponding XML Notation 🎀

The XML format defined in [RFC7940] corresponds pretty directly to the notation used for variant mappings in this document. (There's no notation in the RFC for variant type sets, fyi 🙈) In an LGR document, a simple member of a repertoire that has no variants is listed like this:

<char cp="nnnn" />

where nnnn is the [UNICODE] code point value, written in the standard uppercase hexadecimal notation, padded to at least 4 digits, and without a leading "U+". For a code point sequence of length 2, the XML notation becomes:

<char cp="uuuu vvvvv" />

Variant mappings are defined by nesting <var> elements inside the <char> element. For example, a variant relation of type "blocked":

C  x--> X

is expressed as:

<char cp="nnnn">
  <var cp="mmmm" type="blocked" />
</char>

where "x-->" identifies a "blocked" type. (Other types include "a-->" for "allocatable", for example.) Here, nnnn and mmmm are the [UNICODE] code point values for C and X, respectively. Either C or X could be a code point sequence or a single code point. ✨

A reflexive mapping is specified the same way, except it always uses the same code point value for both the <char> and <var> elements. For example:

X  r-o--> X

would correspond to:

<char cp="nnnn"><var cp="nnnn" type="out-of-repertoire-var" /></char>

Multiple <var> elements can be nested inside a single <char> element, but their "cp" values must be distinct (unless attributes for context rules are present, and the combination of "cp" value and context attributes is distinct):

<char cp="nnnn">
  <var cp="kkkk" type="allocatable" />
  <var cp="mmmm" type="blocked" />
</char>

A set of conditional variants like this:

final: C  a--> K
!final: C  x--> K

would correspond to:

<var cp="kkkk" when="final" type="allocatable" />
<var cp="kkkk" not-when="final" type="blocked" />

where the string "final" references the name of a context rule. Context rules are defined in [RFC7940], and they conceptually correspond to regular expressions. The details of how to create and define these rules are outside the scope of this document 🧠 If the label matches the context defined in the rule, the variant mapping is valid and takes part in further processing. Otherwise, it's invalid and ignored. Using the "not-when" attribute inverts the sense of the match. The two attributes are mutually exclusive, so you can't use both 🙅‍♀️

A derivation of a variant label disposition like this:

if "only-variants" = "s" or "b" => allocatable

is expressed as:

<action disp="allocatable" only-variants= "s b" />

Instead of using "if" and "else if", the <action> elements implicitly form a cascade, where the first action triggered defines the disposition of the label. So the order of the action elements is significant! Order matters, bestie 🫵

For the full specification of the XML format, see [RFC7940] 💖

---

19. IANA Considerations 🌸

This document does not require any IANA actions. Nothing to do here, no cap 🙌

---

20. Security Considerations 🚨

As described in [RFC7940], variants may be used as a tool to reduce certain avenues of attack in security-relevant identifiers. They do this by allowing certain labels to be "mutually exclusive or registered only to the same user". 🔐 BUT, if variants are designed indiscriminately, they can themselves add risks to the security or usability of the identifiers. This can happen through an ambiguous definition, or by allowing too many allocatable variants per label. Yikes 😭

The information in this document is meant to let the reader design an LGR specification that is "well behaved" with respect to variants. As used here, this term refers to an LGR that is predictable in its effects to the LGR author (and reviewer), and more reliable in its implementation 🦋

A well-behaved LGR isn't just one that can be expressed in [RFC7940]. It also actively avoids certain edge cases that the schema doesn't prevent, such as the ones that would lead to ambiguities in the specified disposition for some variant labels. By applying the additional considerations introduced in this document, including adding certain declarations that are optional under the schema and may not change the results of processing a label, the LGR becomes easier to review, and its implementations easier to verify ✨

It's important to know that variants are an important part, but only a part, of an LGR design. There are lots of other features of an LGR this document doesn't touch. Also, whether to define variants at all, or which labels should be considered variants of each other, isn't addressed here 🫠

---

21. References 📚

21.1. Normative References

[RFC7940] Davies, K. and A. Freytag, "Representing Label Generation Rulesets Using XML", RFC 7940, DOI 10.17487/RFC7940, August 2016, https://www.rfc-editor.org/info/rfc7940.

21.2. Informative References ✨

[RFC1034] Mockapetris, P., "Domain names - concepts and facilities", STD 13, RFC 1034, DOI 10.17487/RFC1034, November 1987, <https://www.rfc-editor.org/info/rfc1034>. 📚

[RFC1035] Mockapetris, P., "Domain names - implementation and specification", STD 13, RFC 1035, DOI 10.17487/RFC1035, November 1987, <https://www.rfc-editor.org/info/rfc1035>. 📖

[RFC5890] Klensin, J., "Internationalized Domain Names for Applications (IDNA): Definitions and Document Framework", RFC 5890, DOI 10.17487/RFC5890, August 2010, <https://www.rfc-editor.org/info/rfc5890>. 🌍

[RFC5891] Klensin, J., "Internationalized Domain Names in Applications (IDNA): Protocol", RFC 5891, DOI 10.17487/RFC5891, August 2010, <https://www.rfc-editor.org/info/rfc5891>. 🦋

[RFC5892] Faltstrom, P., Ed., "The Unicode Code Points and Internationalized Domain Names for Applications (IDNA)", RFC 5892, DOI 10.17487/RFC5892, August 2010, <https://www.rfc-editor.org/info/rfc5892>. 💜

[RFC5893] Alvestrand, H., Ed. and C. Karp, "Right-to-Left Scripts for Internationalized Domain Names for Applications (IDNA)", RFC 5893, DOI 10.17487/RFC5893, August 2010, <https://www.rfc-editor.org/info/rfc5893>. ✨

[RFC5894] Klensin, J., "Internationalized Domain Names for Applications (IDNA): Background, Explanation, and Rationale", RFC 5894, DOI 10.17487/RFC5894, August 2010, <https://www.rfc-editor.org/info/rfc5894>. 🌸

[UNICODE] The Unicode Consortium, "The Unicode Standard", <http://www.unicode.org/versions/latest/>. 🔥

---

Acknowledgments 🫶

Contributions that have shaped this document have been provided by Marc Blanchet, Ben Campbell, Patrik Faltstrom, Scott Hollenbeck, Mirja Kuehlewind, Sarmad Hussain, John Klensin, Alexey Melnikov, Nicholas Ostler, Michel Suignard, Andrew Sullivan, Wil Tan, and Suzanne Woolf. 👑 Literally a whole squad of legends, bestie. 💅

---

Author's Address 🎀

Asmus Freytag

Email: asmus@unicode.org 💌