RFC 4518

Lightweight Directory Access Protocol (LDAP): Internationalized String Preparation

📅 published June 2006, from ietf

RFC 4518 🎀 Lightweight Directory Access Protocol (LDAP): Internationalized String Preparation

okay bestie, let's start with the title block 💅

Series: Network Working Group
RFC number: 4518
Author: K. Zeilenga, OpenLDAP Foundation
Category: Standards Track
Date: June 2006
Obsoletes / Updates: this title block doesn't list either one 🤷‍♀️ (section 1.4 is where the obsoleting happens, more on that later)

Status of This Memo ✨

this is an Internet standards track protocol for the Internet community, and it's asking for discussion and suggestions for improvements 🫶 for the standardization state and status of this protocol, check the current edition of the "Internet Official Protocol Standards" (STD 1). distribution of this memo is unlimited, so share away 🌸

Copyright Notice 📜

Copyright (C) The Internet Society (2006).

Abstract 🦋

okay so the older LDAP technical specs never really pinned down how character string matching is supposed to work 😭 and that caused a bunch of usability and interoperability problems. this document fixes that by defining string preparation algorithms for the character-based matching rules that LDAP uses. iykyk, it's giving clarity 💖

1. Introduction 🧚

1.1. Background

so an LDAP matching rule [RFC4517] is basically an algorithm that checks whether a presented value matches an attribute value, based on the criteria for that rule 💅 the answer you get back can be True, False, or Undefined:

True      - the attribute contains a matching value,

False     - the attribute contains no matching value,

Undefined - it cannot be determined whether the attribute contains
            a matching value.

for example, the caseIgnoreMatch matching rule can check whether the commonName attribute holds a certain value without caring about case or insignificant spaces 🍓

1.2. X.500 String Matching Rules

the X.520 spec [X.520] covers value syntaxes and matching rules for values that show up a lot in the directory [X.500], and honestly, it's great for a lot of stuff 🌈 but it's not enough for strings made of Unicode [Unicode] characters, and that's the gap we're filling 🔥

take the caseIgnoreMatch matching rule [X.520]. it's defined as just being a case-insensitive comparison where insignificant spaces get ignored. for printableString, that's fine because there's only one space character and case mapping is bijective (every case change can be reversed), so the simple definition works 👑 but for Unicode string types like universalString, it falls apart. a matching implementation that folds lowercase to uppercase gives different results than one that folds uppercase to lowercase 💀 and one implementation might treat space as only SPACE (U+0020), another might treat any character with the space separator (Zs) property as a space, and another might count anything in the whitespace (WS) category. chaos, honestly 🙈

that lack of precise specification has caused real interoperability problems. in certificate chain validation, it can even open up security vulnerabilities 😭 so this document lays out precise algorithms for preparing character strings before matching.

1.3. Relationship to "stringprep"

the character string preparation here is based on the "stringprep" approach [RFC3454] 💖 in stringprep, presented and stored values get prepared for comparison first, so that a plain character-by-character comparison gives the "correct" result.

this spec is a refinement of that approach. each algorithm adds two extra preparation steps:

a) before the Unicode preparation steps from "stringprep", the string gets transcoded to Unicode.
b) after the Unicode preparation steps from "stringprep", the string gets modified so that characters that don't matter to the matching rule are handled properly.

so preparing strings for X.500 [X.500] matching [X.501] involves these steps, in this order:

1) Transcode
2) Map
3) Normalize
4) Prohibit
5) Check Bidi (Bidirectional)
6) Insignificant Character Handling

we go through these in Section 2 ✨

it's also worth knowing that the Unicode character tables in this spec, or the ones it references, come from Unicode [Unicode] data. but for implementing this spec, the tables in this document are the ones that count 🎀

1.4. Relationship to the LDAP Technical Specification

this document is an integral part of the LDAP technical specification [RFC4510]. that spec obsoletes the previous LDAP technical specification [RFC3377] in its entirety 🦋

it also details the new LDAP internationalized character string preparation algorithms that [RFC4517] uses, and possibly other technical specs that define LDAP syntaxes and/or matching rules 🧚

1.5. Relationship to X.500

LDAP is defined [RFC4510] in X.500 terms, as an X.500 access mechanism 👑 so there's a strong desire for LDAP and X.500 syntax and semantics to line up. the string preparation algorithms here are based on the "Internationalized String Matching Rules for X.500" [XMATCH] proposal to ITU/ISO Joint Study Group 2 🌸

1.6. Conventions and Terms

the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be read as described in BCP 14 [RFC2119] 💅 these are the official requirement words, so they're not casual here, periodt ✨

character names in this doc use the code point and name notation from the Unicode Standard [Unicode]. for example, the letter "a" can be written as either <U+0061> or <LATIN SMALL LETTER A> 🍓 in the mapping lists and the prohibited character lists, the "U+" is dropped to make them easier to read. the comments for character ranges show up in square brackets (like [CONTROL CHARACTERS]) and those comments are not from the standard, they're just helpful labels 🫶

note: a glossary of Unicode terms is in [Glossary], and info about the Unicode character encoding model is in [CharModel] 📚

the term "combining mark" in this spec means any Unicode [Unicode] code point with a mark property (Mn, Mc, Me). Appendix A is the definitive list of combining marks, so go there when you need it 🔥

2. String Preparation 💖

okay this is the main event 👑 the following six-step process SHALL be applied to each presented value and each attribute value before character string matching rule evaluation:

1) Transcode
2) Map
3) Normalize
4) Prohibit
5) Check bidi
6) Insignificant Character Handling

if any step fails, the assertion evaluates to Undefined 💀

the character repertoire for this whole process is Unicode 3.2 [Unicode] ✨

also, this six-step process is meant to describe the expected matching behavior. implementations are free to use other processes, as long as the matching rule evaluation behaves the same way this spec describes 🦋

2.1. Transcode

each non-Unicode string value gets transcoded to Unicode 🧚

PrintableString [X.680] values get transcoded directly to Unicode 🌸

UniversalString, UTF8String, and bmpString [X.680] values don't need transcoding, since they're already Unicode-based (and in the case of bmpString, it's a subset of Unicode) 💅

TeletexString [X.680] values do get transcoded to Unicode. but there's no standard for mapping TeletexString values to Unicode, so that mapping is left as a local matter 😭

because of that and some other reasons, use of TeletexString is NOT RECOMMENDED 🙈

the output is the transcoded string 🎀

2.2. Map

the SOFT HYPHEN (U+00AD) and MONGOLIAN TODO SOFT HYPHEN (U+1806) code points get mapped to nothing 🫶 the COMBINING GRAPHEME JOINER (U+034F) and the VARIATION SELECTORs (U+180B-180D, FF00-FE0F) also get mapped to nothing. the OBJECT REPLACEMENT CHARACTER (U+FFFC) gets mapped to nothing too ✨

the CHARACTER TABULATION (U+0009), LINE FEED (LF) (U+000A), LINE TABULATION (U+000B), FORM FEED (FF) (U+000C), CARRIAGE RETURN (CR) (U+000D), and NEXT LINE (NEL) (U+0085) all get mapped to SPACE (U+0020) 🌈

every other control code point (like Cc) or code point with a control function (like Cf) gets mapped to nothing. here's the complete list of those code points:

U+0000-0008, 000E-001F, 007F-0084, 0086-009F, 06DD, 070F, 180E, 200C-200F, 202A-202E, 2060-2063, 206A-206F, FEFF, FFF9-FFFB, 1D173-1D17A, E0001, E0020-E007F.

the ZERO WIDTH SPACE (U+200B) gets mapped to nothing. all the other code points with the Separator (space, line, or paragraph) property (like Zs, Zl, or Zp) get mapped to SPACE (U+0020) 🎀 the complete list of those is:

U+0020, 00A0, 1680, 2000-200A, 2028-2029, 202F, 205F, 3000.

for the case ignore, numeric, and stored prefix string matching rules, characters get case folded per B.2 of [RFC3454] 💖

the output is the mapped string 🦋

2.3. Normalize

the input string gets normalized to Unicode Form KC (compatibility composed) as described in [UAX15] 🧚 the output is the normalized string ✨

2.4. Prohibit

all Unassigned code points are prohibited 🙅‍♀️ the Unassigned code points are listed in Table A.1 of [RFC3454].

characters that change display properties or are deprecated are also prohibited, per Section 5.8 of [RFC3454] 💀 those are listed in Table C.8 of [RFC3454].

Private Use code points are prohibited too. they're listed in Table C.3 of [RFC3454] 🔒

all non-character code points are prohibited, and they're listed in Table C.4 of [RFC3454] 🚫

surrogate codes are prohibited as well, listed in Table C.5 of [RFC3454] 🙈

and the REPLACEMENT CHARACTER (U+FFFD) code point is prohibited 🌸

the step fails if the input string contains any prohibited code point. otherwise, the output is the input string, unchanged 💅

2.5. Check bidi

bidirectional characters are ignored in this step 🌈

2.6. Insignificant Character Handling

this is where the string gets modified so that characters that don't matter to the matching rule are handled properly ✨ the modification is different from one matching rule to another 🦋

Section 2.6.1 applies to case ignore and exact string matching. Section 2.6.2 applies to numericString matching. Section 2.6.3 applies to telephoneNumber matching 🧚

2.6.1. Insignificant Space Handling 🌸

okay so for this bit, a "space" just means the SPACE (U+0020) code point with no combining marks hanging off it 🫶

NOTE The previous steps make sure the string can't contain any code points in the separator class, except SPACE (U+0020). 💅

For attribute values and non-substring assertion values, here's the deal ✨ If the input has no non-space characters at all, the output is exactly two SPACEs. If it has at least one non-space character, then:

the string starts with exactly one space character,
the string ends with exactly one SPACE character, and
any inner (non-empty) run of space characters gets swapped out for exactly two SPACE characters.

For example, the input "foo<SPACE>bar<SPACE><SPACE>" becomes "<SPACE>foo<SPACE><SPACE>bar<SPACE>" 🦋

Substring assertion values are a little more fussy 🙈 If the string being prepared has no non-space characters, the output is exactly one SPACE. Otherwise:

If the input is an initial substring, it gets modified to start with exactly one SPACE character.
If the input is an initial or any substring that ends in one or more space characters, it gets modified to end with exactly one SPACE character.
If the input is an any or final substring that starts in one or more space characters, it gets modified to start with exactly one SPACE character.
If the input is a final substring, it gets modified to end with exactly one SPACE character.

So for "foo<SPACE>bar<SPACE><SPACE>" as an initial substring, you get "<SPACE>foo<SPACE><SPACE>bar<SPACE>". As an any or final substring, that same input comes out as "foo<SPACE>bar<SPACE>" 🎀

Appendix B has the reasoning behind all this if you're curious, bestie 🧚

2.6.2. numericString Insignificant Character Handling 🍓

Same deal here: a space is the SPACE (U+0020) code point with no combining marks after it 💖

For numericString, every space is insignificant and gets removed, no exceptions 💀 For example, if the Form KC string is "<SPACE><SPACE>123<SPACE><SPACE>456<SPACE><SPACE>", the output is "123456". And if the Form KC string is "<SPACE><SPACE><SPACE>", the output is "" (an empty string) 🌈

2.6.3. telephoneNumber Insignificant Character Handling 📞

For telephoneNumber, a hyphen is any of these: HYPHEN-MINUS (U+002D), ARMENIAN HYPHEN (U+058A), HYPHEN (U+2010), NON-BREAKING HYPHEN (U+2011), MINUS SIGN (U+2212), SMALL HYPHEN-MINUS (U+FE63), or FULLWIDTH HYPHEN-MINUS (U+FF0D), as long as it has no combining marks after it. A space is the SPACE (U+0020) code point, again with no combining marks 🦋

Hyphens and spaces are both treated as insignificant and removed. So for the Form KC string:

<SPACE><HYPHEN>123<SPACE><SPACE>456<SPACE><HYPHEN>

you get "123456" ✨ And for the Form KC string:

<HYPHEN><HYPHEN><HYPHEN>

you get the (empty) output string "" 🥺

3. Security Considerations 🔐

Okay, security babes, listen up 👑 The security considerations from "Preparation of Internationalized Strings ("stringprep")" [RFC3454] generally apply to the algorithms described here. Read those too, no skipping! 😭

4. Acknowledgements 🫶

The approach in this document is based on design principles and algorithms from "Preparation of Internationalized Strings ('stringprep')" [RFC3454] by Paul Hoffman and Marc Blanchet, so big shoutout to them 💜 Some extra guidance came from Unicode Technical Standards, Technical Reports, and Notes 🌸

This document is a product of the IETF LDAP Revision (LDAPBIS) Working Group ✨

5. References 📚

5.1. Normative References 🦋

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC3454] Hoffman, P. and M. Blanchet, "Preparation of Internationalized Strings ("stringprep")", RFC 3454, December 2002.
[RFC4510] Zeilenga, K., "Lightweight Directory Access Protocol (LDAP): Technical Specification Road Map", RFC 4510, June 2006.
[RFC4517] Legg, S., Ed., "Lightweight Directory Access Protocol (LDAP): Syntaxes and Matching Rules", RFC 4517, June 2006.
[Unicode] The Unicode Consortium, "The Unicode Standard, Version 3.2.0" is defined by "The Unicode Standard, Version 3.0" (Reading, MA, Addison-Wesley, 2000. ISBN 0-201-61633-5), as amended by the "Unicode Standard Annex #27: Unicode 3.1" (http://www.unicode.org/reports/tr27/) and by the "Unicode Standard Annex #28: Unicode 3.2" (http://www.unicode.org/reports/tr28/).
[UAX15] Davis, M. and M. Duerst, "Unicode Standard Annex #15: Unicode Normalization Forms, Version 3.2.0". <http://www.unicode.org/unicode/reports/tr15/tr15-22.html>, March 2002.
[X.680] International Telecommunication Union - Telecommunication Standardization Sector, "Abstract Syntax Notation One (ASN.1) - Specification of Basic Notation", X.680(2002) (also ISO/IEC 8824-1:2002).

5.2. Informative References 🌷

[X.500] International Telecommunication Union - Telecommunication Standardization Sector, "The Directory -- Overview of concepts, models and services," X.500(1993) (also ISO/IEC 9594-1:1994).
[X.501] International Telecommunication Union - Telecommunication Standardization Sector, "The Directory -- Models," X.501(1993) (also ISO/IEC 9594-2:1994).
[X.520] International Telecommunication Union - Telecommunication Standardization Sector, "The Directory: Selected Attribute Types", X.520(1993) (also ISO/IEC 9594-6:1994).
[Glossary] The Unicode Consortium, "Unicode Glossary", <http://www.unicode.org/glossary/>.
[CharModel] Whistler, K. and M. Davis, "Unicode Technical Report #17, Character Encoding Model", UTR17, <http://www.unicode.org/unicode/reports/tr17/>, August 2000.
[RFC3377] Hodges, J. and R. Morgan, "Lightweight Directory Access Protocol (v3): Technical Specification", RFC 3377, September 2002.
[RFC4515] Smith, M., Ed. and T. Howes, "Lightweight Directory Access Protocol (LDAP): String Representation of Search Filters", RFC 4515, June 2006.
[XMATCH] Zeilenga, K., "Internationalized String Matching Rules for X.500", Work in Progress.

Appendix A. Combining Marks 🎀

This appendix is normative 💅

This table was derived from Unicode [Unicode] data files, and it lists all code points with the Mn, Mc, or Me properties. Treat this table as definitive when implementing this spec 🌈

      0300-034F 0360-036F 0483-0486 0488-0489 0591-05A1
      05A3-05B9 05BB-05BC 05BF 05C1-05C2 05C4 064B-0655 0670
      06D6-06DC 06DE-06E4 06E7-06E8 06EA-06ED 0711 0730-074A
      07A6-07B0 0901-0903 093C 093E-094F 0951-0954 0962-0963
      0981-0983 09BC 09BE-09C4 09C7-09C8 09CB-09CD 09D7
      09E2-09E3 0A02 0A3C 0A3E-0A42 0A47-0A48 0A4B-0A4D
      0A70-0A71 0A81-0A83 0ABC 0ABE-0AC5 0AC7-0AC9 0ACB-0ACD
      0B01-0B03 0B3C 0B3E-0B43 0B47-0B48 0B4B-0B4D 0B56-0B57
      0B82 0BBE-0BC2 0BC6-0BC8 0BCA-0BCD 0BD7 0C01-0C03
      0C3E-0C44 0C46-0C48 0C4A-0C4D 0C55-0C56 0C82-0C83
      0CBE-0CC4 0CC6-0CC8 0CCA-0CCD 0CD5-0CD6 0D02-0D03
      0D3E-0D43 0D46-0D48 0D4A-0D4D 0D57 0D82-0D83 0DCA
      0DCF-0DD4 0DD6 0DD8-0DDF 0DF2-0DF3 0E31 0E34-0E3A
      0E47-0E4E 0EB1 0EB4-0EB9 0EBB-0EBC 0EC8-0ECD 0F18-0F19
      0F35 0F37 0F39 0F3E-0F3F 0F71-0F84 0F86-0F87 0F90-0F97
      0F99-0FBC 0FC6 102C-1032 1036-1039 1056-1059 1712-1714
      1732-1734 1752-1753 1772-1773 17B4-17D3 180B-180D 18A9
      20D0-20EA 302A-302F 3099-309A FB1E FE00-FE0F FE20-FE23
      1D165-1D169 1D16D-1D172 1D17B-1D182 1D185-1D18B
      1D1AA-1D1AD

Appendix B. Substrings Matching ✨

okay bestie, this appendix is non-normative 🫶 (so it's just here to help you vibe with the main rules, not to set them in stone)

So here's the thing 💅 If substrings matching isn't in the picture, the insignificant space handling for case ignore/exact matching could get way simpler. You'd just need to swap every run of one or more spaces for one space, and if the string has any non-space characters, chop off all the leading and trailing spaces. Clean and easy 🧼

BUT when substrings matching IS in the picture, that simple version goes totally off the rails 🙈 Check these:

(CN=foo\20*\20bar) would match the CN value "foobar";
(CN=*\20foobar\20*) would match "foobar", but (CN=*\20*foobar*\20*) would not.

Quick note for anyone new to LDAP substrings matching 🌸: the LDAP filter [RFC4515] assertion (CN=A*B*C) says to "match any value (of the attribute CN) that begins with A, contains B after A, ends with C where C is also after B."

So case 1 shows that the simple version would treat leading and trailing spaces inside the substrings as insignificant, when they're actually only the leading, trailing, and multiple consecutive spaces of the whole string that get ignored 🎀 Case 2 is the sub-partitioning failure: if some substring matches a partition of the attribute value, then an assertion made by splitting that substring into several pieces should also match. The simple version breaks that rule 💔

To design space handling that works for substrings, you have to study the X.500 case exact/ignore matching rules first 🔍 X.520 [X.520] says:

The [substrings] rule returns TRUE if there is a partitioning of
the attribute value (into portions) such that:

   -  the specified substrings (initial, any, final) match
      different portions of the value in the order of the strings
      sequence;

   -  initial, if present, matches the first portion of the value;

   -  final, if present, matches the last portion of the value;

   -  any, if present, matches some arbitrary portion of the
      value.

So the substrings assertion (CN=foo\20*\20bar) matches the attribute value "foo<SPACE><SPACE>bar", because that value can be split into the portions "foo<SPACE>" and "<SPACE>bar", and both pieces meet those requirements ✨

X.520 also says:

[T]he following spaces are regarded as not significant:

   -  leading spaces (i.e., those preceding the first character
      that is not a space);

   -  trailing spaces (i.e., those following the last character
      that is not a space);

   -  multiple consecutive spaces (these are taken as equivalent
      to a single space character).

Here's the tea ☕ That statement is about the assertion values and attribute values as whole strings, NOT about each substring in an assertion value on its own. So if an assertion value and an attribute value match when you ignore insignificant characters, that assertion should also match any attribute value that only differs by having insignificant characters added or removed 🦋

That's why (CN=foo\20*\20bar) matches "foo<SPACE><SPACE><SPACE>bar" and "foo<SPACE>bar" too. Those values only differ from "foo<SPACE><SPACE>bar" by insignificant spaces getting added or removed, so the match holds up 💖

Now, astute readers will spot a few special cases where this space handling doesn't ignore spaces that could honestly be called insignificant 👀 For example, the assertion (CN=\20*\20*\20) doesn't match "<SPACE><SPACE><SPACE>" (insignificant spaces are in the value) or " " (insignificant spaces aren't in the value). BUT those cases don't have any real-world use that simple assertions like (cn=\20) can't already handle 🙌 And this little anomaly can only be fully fixed by a preparation algorithm used together with character-by-character partitioning and matching, so it's been considered acceptable 🌈

Author's Address

Kurt D. Zeilenga OpenLDAP Foundation

EMail: Kurt@OpenLDAP.org

Full Copyright Statement

This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights.

This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property

The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79.

Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr.

The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org.

Acknowledgement

Funding for the RFC Editor function is provided by the IETF Administrative Support Activity (IASA).

📦 where this page comes from

ietf: rfc4518 2006-06
💖 girlified by claude-haiku-5-5 on 2026-10-16: a language model's rewrite of the original, under its terms