RFC 2654

A Tagged Index Object for use in the Common Indexing Protocol

📅 published August 1999, from ietf

A Tagged Index Object for use in the Common Indexing Protocol 🎀

Title Block 📋

okay so the paperwork, bestie 💅 This is RFC 2654, filed under the Network Working Group in August 1999. It's an Experimental doc, so it's not a standard, just a proposal the community is invited to poke at 🧪 The authors are R. Hedberg (Catalogix), B. Greenblatt (Directory Tools and Application Services, Inc.), R. Moats (AT&T), and M. Wahl (Innosoft International, Inc.). The title block doesn't list any RFCs it obsoletes or updates, so it's standing on its own 🌸

Status of this Memo 🦋

ok so this memo defines an Experimental Protocol for the Internet community ✨ It is not an Internet standard of any kind. The authors are literally asking for discussion and suggestions for improvement, and the memo can be distributed with no limits 🫶

Copyright (C) The Internet Society (1999). All Rights Reserved. 💜

Abstract 🧚

okay the big idea: this doc sets up a way for information servers to swap indices of what's in their databases, using the Common Indexing Protocol (CIP) [1] 🍓 It defines the structure of the index info being exchanged and what the headers defined in CIP actually mean. The authors assume these structures can be used by X.500 DSAs, LDAP servers, Whois++ servers, CSO Ph servers, and many others 😭✨

Table of Contents 📚

1. Introduction  . . . . . . . . . . . . . . . . . . . . . . . . . 2
2. Background  . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3. Object  . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
4. The Tagged Index Object . . . . . . . . . . . . . . . . . . . . 5
4.1. The Agreement . . . . . . . . . . . . . . . . . . . . . . . . 5
4.2. Content Type  . . . . . . . . . . . . . . . . . . . . . . . . 8
4.3 Tagged Index BNF . . . . . . . . . . . . . . . . . . . . . . . 9
4.3.1. Header Descriptions . . . . . . . . . . . . . . . . . . . .10
4.3.2. Tokenization types  . . . . . . . . . . . . . . . . . . . .11
4.3.3. Tag Conventions . . . . . . . . . . . . . . . . . . . . . .11
4.4. Incremental Indexing  . . . . . . . . . . . . . . . . . . . .12
5. Examples  . . . . . . . . . . . . . . . . . . . . . . . . . . .13
5.1 The original database  . . . . . . . . . . . . . . . . . . . .13
5.1.1 "complete" consistency based full update . . . . . . . . . .14
5.1.2 "tag" consistency based full update  . . . . . . . . . . . .14
5.1.3 "unique" consistency based full update . . . . . . . . . . .15
5.2 First update . . . . . . . . . . . . . . . . . . . . . . . . . .16
5.2.1 "complete" consistency based incremental update  . . . . . .16
5.2.2 "tag" consistency based incremental update   . . . . . . . .17
5.2.3 "unique" consistency based incremental update  . . . . . . .17
5.3 Second update  . . . . . . . . . . . . . . . . . . . . . . . . . .18
5.3.1 "complete" consistency based incremental update  . . . . . .18
5.3.2 "tag" consistency based incremental update . . . . . . . . .19
5.3.3 "unique" consistency based incremental update  . . . . . . .20
6. Aggregation . . . . . . . . . . . . . . . . . . . . . . . . . .21
6.1 Aggregation of Tagged Index Objects  . . . . . . . . . . . . .21
7. Security Considerations . . . . . . . . . . . . . . . . . . . .21
8. References  . . . . . . . . . . . . . . . . . . . . . . . . . .22
9. Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . .23
Full Copyright Statement . . . . . . . . . . . . . . . . . . . . .24

1. Introduction ✨

okay so the Common Indexing Protocol (CIP), as defined in [1], is all about spreading searches across a bunch of copies of one kind of search engine so you get a global directory 🌍 It's scalable and flexible, and it can tie individual databases into distributed data warehouses that grow gracefully as the Internet grows 📈 CIP's approach doesn't depend on which access method is used to reach the data behind the indices, which is very giving independence 💅

Separate from CIP is the definition of the Index Object, which holds the info that Index Servers trade with each other 🎀 One Index Object that's already defined is the Centroid, which comes from the Whois++ protocol [2] 🦋

Here's the tea: the Centroid doesn't meet every requirement for swapping index info between information servers 😬 For one thing, it doesn't natively support incremental updates. If your server has millions of records, constantly sending complete dredges of the whole database is seriously bandwidth-heavy 💀 That's why the Tagged Index Object is designed specifically to carry index update information. The catch is that it makes the index object bigger 🍓 The Centroid also isn't built to always give boolean answers to queries. In the Centroid Model, "an index server will take a query in standard Whois++ format, search its collections of centroids and other forward information, determine which servers hold records which may fill that query, and then notifies the user's client of the next servers to contact to submit the query." [2] So swapping Centroids lets index servers give hints about which information server actually has the goods 🔥

The Tagged Index Object labels the different pieces of info with identifiers that link each attribute back to the object as a whole 🏷️ This "tagging" makes an index server way more capable of sending a specific query to the right information server 👑 Again though, that perk comes with a bigger index object, so it's a tradeoff, bestie 🫶

2. Background 🌸

so LDAP is defined in [3] 🧚 It's a way to access info arranged hierarchically, which makes a globally distributed database usually called the Directory Information Tree (DIT). One main vibe of LDAP servers is that several servers usually team up to manage a shared subtree of the DIT 🤝 LDAP servers are expected to answer requests about parts of the DIT they have data for, and parts they know nothing about in their own database 🙈 For example, the LDAP server for the part of the DIT in the United States (c=US) must be able to give a response to a Search operation about a part of the DIT in Sweden (c=se) 🇸🇪 Usually the answer is a referral to another LDAP server that should know more about that subtree. But there's no mechanism right now that lets these servers refer the LDAP client to that more knowledgeable server, sadly 😭 Typically an LDAP (v3) server is configured with the name of exactly one other LDAP server, and every client whose request falls outside the subtree that server knows about gets sent there 🙃 This specification defines a mechanism where LDAP servers can exchange index information, so referrals can point toward a clearly accurate destination 🎯

Next up, the X.500 series of recommendations defines the Directory Information Shadowing Protocol (DISP) [4], which lets X.500 DSAs exchange information in the DIT 🪞 Shadowing lets various info from various parts of the DIT get replicated across participating DSAs. DISP is built for exchanging entire portions of the DIT, while CIP and the Tagged Index Object are optimized for exchanging structural index info about the DIT and making tree navigation faster across info servers 🚀 So for exchanging index information, the Tagged Index Object is the better fit compared to DISP. DISP is more about DIT distribution and fault tolerance 🛡️ That makes DISP the better choice for actually exchanging data, to spread the load across several information servers. DISP is tailored specifically to X.500 (and other hierarchical directory systems), while the Tagged Index Object and CIP can be used in a wide variety of information server environments 🌈

Now, DISP lets one directory server collect info about big parts of the DIT, but gathering all the replicas for a significant portion of the DIT would take a huge database 😵 Also, as X.525 puts it: "Before shadowing can occur, an agreement, covering the conditions under which shadowing may occur is required. Although such agreements may be established in a variety of ways, such as policy statements covering all DSAs within a given DMD ...", where a DMD is a Directory Management Domain 📜 The reason is that DIT data is being swapped between DSAs, not just the info needed to maintain an index. In a lot of environments, an agreement like that just isn't appropriate, and gathering info for a meaningful chunk of the DIT could mean setting up many agreements 🥲

3. Object 🎀

okay the goal is an information server (or a network of them) that can answer real-world requests quickly ⚡ Like:

  • What is Tim Howes's email address? This is much harder than; What email address does Tim Howes at Netscape have ?
  • What is the X.509 certificate for Fred Smith at compuserve.com? One certainly doesn't want to search CompuServe's entire directory tree to find out this one piece of information. I also don't want to have to shadow the entire CompuServe directory subtree onto my server. If this request is being made because Fred is trying to log into my server, I'd certainly want to be able to respond to the BIND in real time. 😩
  • Who are all the people at Novell that have a title of programmer? 💻

All these requests can reasonably be translated into LDAP or Whois++, and other directory access protocol queries 🫧 They can also be handled in a straightforward way by the user's home information server, if that server has the right reference info in its database pointing to the database holding the source data 🏠 In that case, the first server can "chain" the request on the user's behalf. Alternatively, a precise referral could be returned 🎯 If the home information server wants to service (i.e. chain) the request based on the index information it has on hand, it can do that a few different ways:

  • issuing LDAP operations to the remote directory server 📨
  • issuing DSP operations to the remote directory server 📨
  • issuing DAP operations to the remote directory server 📨
  • issuing Whois++ operations to the remote Whois++ server 📨
  • ...

okay bestie, that's the first chunk! Part 2 is where Section 4 (The Tagged Index Object) starts getting into the nitty-gritty 🦋 Stay tuned 💖

4. The Tagged Index Object ✨

okay bestie, so this section defines a Tagged Index Object that Information Servers can swap with each other using CIP 🦋. Servers can often just use the Centroid definition (from [2]) to trade index info, but whoever designed this wanted a brand new construct for a few reasons 💅

  • When the Information Server gets a search request that deserves a referral, the server can send back a referral pointing the client at a server that is most likely able to answer correctly. False positive referrals (where the search hits entries in the index object and sends you to servers that don't actually hold the info you wanted) can be reduced, depending on which attribute tokenization types are used 🎯
  • Incremental updates could be allowed, which would eat up substantially less bandwidth than if full updates always had to be used 🫶

4.1. The Agreement

Before a Tagged Index Object can be exchanged, the organization that administers the object supplier and the organization that administers the object consumer must reach an agreement on how the servers will communicate. This agreement contains the following:

  • index-type: This specification describes the index type x-tagged-index-1 ✨
  • dsi: An OID that uniquely identifies the subtree and scope. This field is not explicitly necessary, as it may not provide information beyond what is contained in the base-uri below.
  • base-uri: One or more URIs that will form the base of any referrals created based on the index object that is governed by this agreement. For example, in the LDAP URL format [8] the base-uri would specify (among other items): the LDAP host, the base object to which this index object refers (e.g. c=SE), and the scope of the index object (e.g. single container) 🌸
  • supplier: The hostname and listening portnumber of the supplier server, as well as any alternative servers holding that same naming contexts, if the supplier is unavailable.
  • consumeraddr: This is a URI of the mailto: form, with the RFC 822 email address of the consumer server. Further versions of this draft allow other forms of URI, so that the consumer may retrieve the update via the WWW, FTP or CIP 📬
  • updateinterval: The maximum duration in seconds between occurrences of the supplier server generating an update. If the consumer server has not received an update from the supplier server after waiting this long since the previous update, it is likely that the index information is now out of date. A typical value for a server with frequent updates would be 604800 seconds, or every week. Servers whose DITs are only modified annually could have a much longer update interval 🗓️
  • attributeNamespace: Every set of index servers that together wants to support a specific usage of indexes has to agree on which attribute names to use in the index objects. The participating directory servers also have to agree on the mapping from local attribute names to the attribute names used in the index. Since one specific index server might be involved in several such sets, it has to have some way to connect an update to the proper set of indexes. One possible solution to this would be to use different DSIs 🧩
  • consistencybase: How consistency of the index is maintained over incremental updates:
    • "complete": every change or delete concerning one object has to contain all tokens connected to that object. This method must be supported by any server who wants to comply with this standard.
    • "tag": starting at a full update, every incremental update referring back to this full update has to maintain state information regarding tags, such that an object within the original database is assigned the same tag number every time. This method is optional.
    • "unique": every object in the Dataset has to have a unique value for a specific attribute in the index. An example of such an attribute could be the distinguishedName attribute. This method is also optional.
  • securityoption: Whether and how the supplier server should sign and encrypt the update before sending it to the consumer server. Options for this version of the specification are:
    • "none": the update is sent in plaintext
    • "PGP/MIME": the update is digitally signed and encrypted using PGP [9]
    • "S/MIME": the update is digitally signed and encrypted using S/MIME [10]
    • "SSLv3": the update is digitally signed and encrypted using an SSLv3 connection [11]
    • "Fortezza": the update is digitally signed and encrypted using Fortezza [5]

It is recommended that the "PGP/MIME" option be used when exchanging sensitive information across public networks, and both the supplier and consumer have PGP keys. The "Fortezza" option is intended for use in environments where security protocols are based on Fortezza-compatible devices. The "S/MIME" option can be used with both the supplier and consumer having RSA keys, and can make use of the PKCS protocols defined in the S/MIME specification. The "SSLv3" option can be used when both the supplier and consumer have access to SSL services, have server certificates, and can mutually authenticate each other 🔐

  • Security Credentials: The long-term cryptographic credentials used for key exchange and authentication of the consumer and supplier servers, if a security option was selected. For "PGP/MIME", this will be the trusted public keys of both servers. For "Fortezza", this will be the certificate paths of both servers to a common point of trust. For "S/MIME" and "SSLv3" these will be the certificates of the supplier and consumer.

Note that if the index server maintains the information that would appear in the agreement in a directory according to the definitions in [7], then no real formal agreement between the two parties needs to be put in place, and the information that is required for communication between the two index servers is derived automatically from the directory. 🌈

4.2. Content Type

The update consists of a MIME object of type application/cip-index-object. The parameters are:

  • "type": this has value application/index.obj.tagged.
  • "dsi": the DSI (if any) from the agreement.
  • "base-uri": A set of URIs, separated by spaces. In each URI, the hostname/portno must be distinct, and based on the "supplier" part of the agreement 🧁

The payload is mostly textual data but may include bytes with the high bit set. The originating information server should set the content-transfer-encoding as appropriate for the information included in the payload.

This object may be encapsulated in a wrapper content (such as multipart/signed) or be encrypted as part of the security procedures. The resulting content can then be distributed, for example via electronic mail. For example:

From: supplier@sup.com Date: Thu, 16 Jan 1997 13:50:37 -0500
Message-Id: <199701161850.NAA29295@sup.com>;
To: consumer@consumer.com       <<-- from consumer server address

Reply-to: supplier-admin@sup.com
MIME-Version: 1.0
Content-Type: application/index.obj.tagged;
dsi=1.3.6.1.4.1.1466.85.85.1.2.3.4.5.6.7.8.9.10.11.12.13.14.15.16;
base-uri="ldap://sup.com/dc=sup,dc=com ldap://alt.com/dc=sup,dc=com"

The payload is a series of CRLF-terminated lines. The payload is UTF-8. Some supplier servers may only be able to generate the printable US-ASCII subset of UTF-8, but all consumer servers must be able to handle the full range of Unicode characters when decoding the attribute values (in the "attr-value" field in the BNF below) 💖

4.3. Tagged Index BNF ✨

okay so the Tagged Index object has its own grammar, written in modified BNF. this is the blueprint for how every piece fits together, so it's kinda the most important bit 🫶

index-object = 0*(io-part SEP) io-part
io-part      = header SEP schema-spec SEP index-info
header       = version-spec SEP update-type SEP this-update SEP
                last-update context-size name-space SEP
version-spec = "version:" *SPACE "x-tagged-index-1"
update-type  = "updatetype:" *SPACE ( "total" |
               ( "incremental" [*SPACE "tagbased"|"uniqueIDbased" ] )
this-update  = "thisupdate:" *SPACE TIMESTAMP
last-update  = [ "lastupdate:" *SPACE TIMESTAMP SEP]
context-size = [ "contextsize:" *SPACE 1*DIGIT SEP]
schema-spec  = "BEGIN IO-Schema" SEP 1*(schema-line SEP)
               "END IO-Schema"
schema-line  = attribute-name ":" token-type
token-type   = "FULL" | "TOKEN" | "RFC822" | "UUCP" | "DNS"
index-info   = full-index | incremental-index
full-index   = "BEGIN Index-Info" SEP 1*(index-block SEP)
               "END Index-Info"
incremental-index = 1*(add-block | delete-block | update-block)
add-block    = "BEGIN Add Block" SEP 1*(index-block SEP)
               "END Add Block"
delete-block = "BEGIN Delete Block" SEP 1*(index-block SEP)
               "END Delete Block"
update-block = "BEGIN Update Block" SEP
               0*(old-index-block SEP)
               1*(new-index-block SEP)
               "END Update Block"
old-index-block = "BEGIN Old" SEP 1*(index-block SEP)
               "END Old"
new-index-block = "BEGIN New" SEP 1*(index-block SEP)
               "END New"
index-block  = first-line 0*(SEP cont-line)
first-line   = attr-name ":" *SPACE taglist "/" attr-value
cont-line    = "-" taglist "/" attr-value
taglist      = tag 0*("," tag) | "*"
tag          = 1*DIGIT ["-" 1*DIGIT]
attr-value   = 1*(UTF8)
attr-name    = 1*(NAMECHAR)
TIMESTAMP    = 1*DIGIT
NAMECHAR     = DIGIT | UPPER | LOWER | "-" | ";" | "."
SPACE        = <ASCII space, %x20>;
SEP          = (CR LF) | LF
CR           = <ASCII CR, carriage return, %x0D>;
LF           = <ASCII LF, line feed, %x0A>;
DIGIT        = "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" |
               "8" | "9"

UPPER        = "A" | "B" | "C" | "D" | "E" | "F" | "G" | "H" |
               "I" | "J" | "K" | "L" | "M" | "N" | "O" | "P" |
               "Q" | "R" | "S" | "T" | "U" | "V" | "W" | "X" |
               "Y" | "Z"
LOWER        = "a" | "b" | "c" | "d" | "e" | "f" | "g" | "h" |
               "i" | "j" | "k" | "l" | "m" | "n" | "o" | "p" |
               "q" | "r" | "s" | "t" | "u" | "v" | "w" | "x" |
               "y" | "z"

US-ASCII-SAFE  = %x01-09 / %x0B-0C / %x0E-7F
                ;; US-ASCII except CR, LF, NUL
UTF8           = US-ASCII-SAFE / UTF8-1 / UTF8-2 / UTF8-3
                          / UTF8-4 / UTF8-5
UTF8-CONT      = %x80-BF
UTF8-1         = %xC0-DF UTF8-CONT
UTF8-2         = %xE0-EF 2UTF8-CONT
UTF8-3         = %xF0-F7 3UTF8-CONT
UTF8-4         = %xF8-FB 4UTF8-CONT
UTF8-5         = %xFC-FD 5UTF8-CONT

so the attr-name field is only allowed to use the characters that LDAP and WHOIS++ attribute names already use 🌸 for other services with bigger character sets, the move is to make a profile that maps the names onto object identifiers. then those services use the digits and periods from that sequence to build their attr-name fields. smart, right? 🧚

also, and this is a big deal, updates to an index based on tagged index objects MUST be performed in the order the tagged index object itself specifies. no shuffling allowed 💅

4.3.1. Header Descriptions

the header section is made of one or more "header lines". these are the header lines that are defined:

  • "version": this line must always be present, and have the value x-tagged-index-1 for this version of the specification.
  • "updatetype": this line must always be present. it takes either total or incremental as its value. the first update a supplier server sends to a consumer server for a DSI must be a total update 🍓
  • "thisupdate": this line must always be present. the value is the number of seconds from 00:00:00 UTC January 1, 1970 at which the supplier constructed this update.
  • "lastupdate": this line must be present if the updatetype line has the value incremental. the value is the number of seconds from 00:00:00 UTC January 1, 1970 at which the supplier constructed the previous update sent to the consumer. this field lets the consumer figure out whether a previous update got missed 👀
  • "contextsize": this line may be present at the supplier's option. the value is a number, which is the approximate total number of entries in the subtree. this info is only for statistical purposes, so don't stress about it 💖

4.3.2. Tokenization Types

the Tagged Index Object inherits the "TOKEN" scheme for tokenization, as specified in [2]. on top of that, there are a few other tokenization schemes that are defined just for the Tagged Index Object. the table below shows these schemes and which characters delimit tokens for each one ✨

    Token Type      Tokenization Characters
    FULL    none
    TOKEN   white space, "@"
    RFC822  white space, ".", "@"
    UUCP    white space, "!"
    DNS     any character note a number, letter, or "-"

4.3.3. Tag Conventions

so in the tag list, a run of consecutive tags can be shortened using #-#. for example, 3,4,5,6,7,8,9,10 can be shortened to 3-10 🎀 tags are applied per entry. that means if two index lines in the same index object share a tag, those two lines always point to the same "record" in the directory. in LDAP terms, the two lines would be referring to the same directory object. and if two index lines in the same index object have different tags, those lines always point back to different records in the directory. the * in the tag position means that the specific token shows up in every record in the directory 🦋

the tag for the same underlying record can be different across two separate transmissions of a full-index. so receiving index servers should not make any assumptions about tag values across index object boundaries 🙈

4.4. Incremental Indexing

the tagged index object format lets information servers send only delta index data, instead of sending the whole total index info every single time. this is called incremental indexing, and it supports three basic operations: add, delete and replace 🔥 if the incremental updatetype is set in the tagged index object, the index object holds a snapshot of only the changes made since the index object named in the lastupdate header was sent. if the receiving index server never got that index object, it should request a total index object instead. and if the CIP protocol supports it, the index server may request the exact index object it missed 🧚

if the tagged index object has an Add Block, the lines in that block refer to new records that were added to the information base of the transmitting index server. the spec guarantees those records did not exist in any previously received tagged index object, so the receiving index server can insert this index info into the index it already keeps for the transmitting index server 💅

if the tagged index object has a Delete Block, the structure of that block depends on how consistency is maintained:

  • "completeRecord": every token connected to the record being deleted has to be included. the tag used to connect tokens in this message has no relation to the tags used in previously sent tagged index objects.
  • "uniqueIDBased": only the unique identifier has to be defined.
  • "tagBased": every token connected to the record has to be included, but each one has to be preceded by the tag used for this specific record in the preceding set, meaning the last full update and any incremental updates after it.

if the tagged index object has an Update Block, the lines in that block refer to records that were changed in the information base of the transmitting index server. again, what the block contains depends on how consistency is maintained 🌈

  • "completeRecord": all the tokens representing the old version of the record, as well as the new ones, have to be included.
  • "uniqueIDBased": the unique ID has to be included together with the tokens that have changed.
  • "tagBased": only the changed tokens are included, but both the old version (if there was one) and the new one (if there is one) have to be included.

the Update Block also supports indexing new attributes that weren't previously part of the tagged index object. for example, say the transmitting index server starts including index info on postal addresses. it could then send an Update Block in the index object that includes all the postal address index info for every record in its information base, and say that nothing else has changed 💜

5. Examples ⭐

in the following sections, the tagged index object is shown for each different consistencybase type, using the same scenario. each example starts with one full update, and then a set of updates follows it. the underlying information is shown in the LDIF [6] format 🫶

5.1 The original database ✨

dn: cn=Barbara Jensen, ou=Product Development, o=Ace Industry, c=US
objectclass: top
objectclass: person
objectclass: organizationalPerson
cn: Barbara Jensen
cn: Barbara J Jensen
cn: Babs Jensen
sn: Jensen
uid: bjensen
dn: cn=Bjorn Jensen, ou=Accounting, o=Ace Industry, c=US
objectclass: top
objectclass: person
objectclass: organizationalPerson
cn: Bjorn Jensen
sn: Jensen
title: Accounting manager
dn: cn=Gern Jensen, ou=Product Testing, o=Ace Industry, c=US
objectclass: top
objectclass: person
objectclass: organizationalPerson
cn: Gern Jensen
cn: Gern O Jensen
sn: Jensen
title: testpilot
dn: cn=Horatio Jensen, ou=Product Testing, o=Ace Industry, c=US
objectclass: top
objectclass: person
objectclass: organizationalPerson
cn: Horatio Jensen
cn: Horatio N Jensen
sn: Jensen
title: testpilot

5.1.1 "Complete" consistency based full update 💖

version: x-tagged-index-1
updatetype: total
thisupdate: 855938804
BEGIN IO-Schema
cn: TOKEN
sn: FULL
title: TOKEN
END IO-Schema
BEGIN Index-Info
cn: 1/Barbara
-1/J
-1/Babs
-*/Jensen
-2/Bjorn
-3/Gern
-3/O
-4/Horatio
-4/N
sn: */Jensen
title: 1/product
-1-2/manager
-1/accounting
-3,4/testpilot
END Index-Info

5.1.2 "tag" consistency based full update 🦋

version: x-tagged-index-1
updatetype: total
thisupdate: 855938804
BEGIN IO-Schema
cn: TOKEN
sn: FULL
title: TOKEN
END IO-Schema
BEGIN Index-Info
cn: 1/Barbara
-1/J
-1/Babs
-*/Jensen
-2/Bjorn
-3/Gern
-3/O
-4/Horatio
-4/N
sn: */Jensen

title: 1/product
-1-2/manager
-1/accounting
-3,4/testpilot
END Index-Info

5.1.3 "unique" consistency based full update ✨

Okay, last full-database version, and this one uses the "unique" ID approach. Every entry gets its own dn: so nothing is ambiguous 🫶

version: x-tagged-index-1
updatetype: total
thisupdate: 855938804
BEGIN IO-Schema
dn: FULL
cn: TOKEN
sn: FULL
title: TOKEN
END IO-Schema
BEGIN Index-Info
dn: 1/cn=Barbara Jensen, ou=Product Development, o=Ace Industry, c=US
-2/cn=Bjorn Jensen, ou=Accounting, o=Ace Industry, c=US
-3/cn=Gern Jensen, ou=Product Testing, o=Ace Industry, c=US
-4/cn=Horatio Jensen, ou=Product Testing, o=Ace Industry, c=US
cn: 1/Barbara
-1/J
-1/Babs
-*/Jensen
-2/Bjorn
-3/Gern
-3/O
-4/Horatio
-4/N
sn: */Jensen
title: 1/product
-1-2/manager
-1/accounting
-3,4/testpilot
END Index-Info

5.2 First update 💅

So Gern Jensen's entry above changes to this 👀

dn: cn=Gern Jensen, ou=Product Testing, o=Ace Industry, c=US
objectclass: top
objectclass: person
objectclass: organizationalPerson
cn: Gern Jensen
cn: Gern O Jensen
sn: Jensen
title: chiefpilot

5.2.1 First update using "complete" 🎀

version: x-tagged-index-1
updatetype: incremental
lastupdate: 855940000
thisupdate: 855938804
BEGIN IO-schema
cn: TOKEN
sn: FULL
title: FULL
END IO-Schema
BEGIN Update Block
BEGIN Old
cn: 1/Gern
cn: 1/O
cn: 1/Jensen
sn: 1/Jensen
title: 1/testpilot
END Old
BEGIN New
cn: 1/Gern
cn: 1/O
cn: 1/Jensen
sn: 1/Jensen
title: 1/chiefpilot
END New
END Update Block

5.2.2 First update using "tag" consistency 🦋

version: x-tagged-index-1
updatetype: incremental
lastupdate: 855940000
thisupdate: 855938804
BEGIN IO-schema
cn: TOKEN
sn: FULL
title: FULL
END IO-Schema
BEGIN Update Block
BEGIN Old
title: 3/testpilot
END Old
BEGIN New
title: 3/chiefpilot
END New
END Update Block

5.2.3 First update using "unique" ID's 🌸

version: x-tagged-index-1
updatetype: incremental
lastupdate: 855940000
thisupdate: 855938804
BEGIN IO-schema
cn: TOKEN
sn: FULL
title: FULL
END IO-Schema
BEGIN Update Block
BEGIN Old
dn: 1/cn=Gern Jensen, ou=Product Testing, o=Ace Industry, c=US
title: 1/testpilot
END Old
BEGIN New
dn: 1/cn=Gern Jensen, ou=Product Testing, o=Ace Industry, c=US
title: 1/chiefpilot
END New
END Update Block

5.3 Second update 🧚

Now the big one: one new person joins, one leaves, and everyone else picks up a new locality. Main character energy 👑

# Add a new entry
dn: cn=Bo Didley, ou=Marketing, o=Ace Industry, c=US
changetype: add
objectclass: top
objectclass: person
objectclass: organizationalPerson
cn: Bo Didley
sn: Didley
title: Policy Maker
# Delete an existing entry
dn: cn=Bjorn Jensen, ou=Accounting, o=Ace Industry, c=US
changetype: delete
# Modify all other entries: adding an additional locality value
dn: cn=Barbara Jensen, ou=Product Development, o=Ace Industry, c=US
changetype: modify
add: locality
locality: New Jersey
dn: cn=Gern Jensen, ou=Product Testing, o=Ace Industry, c=US
changetype: modify
add: locality
locality: New Orleans
dn: cn=Horatio Jensen, ou=Product Testing, o=Ace Industry, c=US
changetype: modify
add: locality
locality: New Caledonia

5.3.1 "complete" 🍓

version: x-tagged-index-1
updatetype: incremental
lastupdate: 855938804
thisupdate: 855939525
BEGIN IO-schema
cn: TOKEN
sn: FULL
title: FULL
locality: TOKEN
END IO-Schema
BEGIN Add Block
cn: 1/Bo
-1/Didley
sn: 1/Didley
title: 1/Policy
-1/maker
locality: 1/New
-1/York
END Add Block
BEGIN Delete Block
cn: 1/Bjorn
-1/Jensen
sn: 1/Jensen
title: 1/Accounting
-1/Manager
END Delete Block
BEGIN Update Block
BEGIN Old
cn: 1/Barbara
-1/J
-1-3/Jensen
-2/Gern
-2/O
-3/Horatio
sn: 1-3/Jensen
title: 1/Production
-1/Manager
-2/Testpilot
-3/Chiefpilot
END Old
BEGIN New
cn: 1/Barbara
-1/J
-1-3/Jensen
-2/Gern
-2/O
-3/Horatio

sn: 1-3/Jensen
title: 1/Production
-1/Manager
-2/Testpilot
-3/Chiefpilot
locality: 1/Jersey
-2/Orleans
-3/Caledonia
-1-3/New
END New
END Update Block

5.3.2 "tag" 🌈

version: x-tagged-index-1
updatetype: incremental
lastupdate: 855938804
thisupdate: 855939525
BEGIN IO-schema
cn: TOKEN
sn: FULL
title: FULL
locality: TOKEN
END IO-Schema
BEGIN Add Block
cn: 5/Bo
-5/Didley
sn: 5/Didley
title: 5/Policy
-5/maker
locality: 5/New
-5/York
END Add Block
BEGIN Delete Block
cn: 2/Bjorn
-2/Jensen
sn: 2/Jensen
title: 2/Accounting
-2/Manager
END Delete Block
BEGIN Update Block
BEGIN New
locality: 1/Jersey
-2/Orleans
-4/Caledonia
-1,2,4/New
END New
END Update Block

5.3.3 "unique" 💜

version: x-tagged-index-1
updatetype: incremental
lastupdate: 855938804
thisupdate: 855939525
BEGIN IO-schema
cn: TOKEN
sn: FULL
title: FULL
locality: TOKEN
END IO-Schema
BEGIN Add Block
dn: 1/cn=Bo Didley, ou=Marketing, o=Ace Industry, c=US
cn: 1/Bo
-1/Didley
sn: 1/Didley
title: 1/Policy
-1/maker
locality: 1/New
-1/York
END Add Block
BEGIN Delete Block
dn: 1/cn=Bjorn Jensen, ou=Accounting, o=Ace Industry, c=US
END Delete Block
BEGIN Update Block
BEGIN New
dn: 1/cn=Barbara Jensen, ou=Product Development, o=Ace Industry, c=US
-2/cn=Gern Jensen, ou=Product Testing, o=Ace Industry, c=US
-3/cn=Horatio Jensen, ou=Product Testing, o=Ace Industry, c=US
locality: 1/Jersey
-2/Orleans
-3/Caledonia
-1-3/New
END New
END Update Block

6. Aggregation 🔥

6.1. Aggregation of Tagged Index Objects

Aggregating two tagged index objects means merging their two lists of values and rewriting each tag list. The rewrite is done so the result looks like it came from a single source, which is honestly so clean 🫶 An index server that aggregates tagged index objects for export MUST ensure that the export URL (i.e. the base-uri of the CIP object) for the aggregate index object will route all queries that have "hits" on the index object to that server (otherwise, query routing will not succeed). Don't skip this one, bestie, or the routing just won't work 😭

7. Security Considerations 💀

Okay, this section is serious, so buckle up 🙈 This specification is a protocol for moving information between two servers. That information may be protected by laws in many countries, so you have to be really careful about how you tokenize the data. The goal is that protected data can't be rebuilt in full by the receiving server. This protocol has no built-in protection against spoofing or eavesdropping. BUT since it rides inside MIME messages (like all CIP index objects), it gets all the security perks and risks of regular MIME messages too. So anyone who wants to stop eavesdropping or spoofing can use the usual MIME signing and encryption techniques 🔐

Information Server administrators have to decide which parts of their databases belong in the Tagged Index Object. Period 💅

If the info is going outside the enterprise, information server developers are encouraged to add features that hide the organizational structure when they build the Tagged Index Object from the underlying database. To keep Tagged Index Objects safe while crossing the Internet, Index Servers should make use of SSL when completing the connection. To strongly verify who the peer index server really is, SSL version 3 certificate exchange should be implemented, and the identity in the peer's certificate should be verified with the Public Key Infrastructure. If electronic mail is used to exchange Tagged Index Objects, a secure messaging facility such as PGP/MIME or S/MIME should be used to sign or encrypt (or both) the information.

8. References 📚

  • [1] Allen, J. and M. Mealling, "The Architecture of the Common Indexing Protocol (CIP)," RFC 2651, August 1999.
  • [2] Weider, C., Fullton, J. and S. Spero, "Architecture of the Whois++ Index Service", RFC 1913, February 1996.
  • [3] Wahl, M., Howes, T. and S. Kille, "Lightweight Directory Access Protocol (v3)", RFC 2251, December 1997.
  • [4] ITU, "X.525 Information Technology - Open Systems Interconnection - The Directory: Replication", November 1993.
  • [5] "FORTEZZA Application Implementors Guide for the FORTEZZA Crypto Card (Production Version)", Document #PD4002102-1.01, SPYRUS, 1995.
  • [6] Good, G., "The LDAP Data Interchange Format (LDIF) - Technical Specification", Work in Progress.
  • [7] Hedberg, R., "LDAPv2 Client vs. the Index Mesh", RFC 2657, August 1999.
  • [8] Howes, T. and M. Smith, "The LDAP URL Format", RFC 2255, December 1997.
  • [9] Elkins, M., "MIME Security with Pretty Good Privacy (PGP)", RFC 2015, October 1996.
  • [10] Ramsdell, B., Editor, "S/MIME Version 3 Message Specification", RFC 2633, June 1999.
  • [11] Allen, C. and T. Dierks, "The TLS Protocol Version 1.0", RFC 2246, January 1999.

9. Authors' Addresses 🦋

Roland Hedberg Catalogix Dalsveien 53 0387 Oslo Norway

EMail: roland@catalogix.ac.se

Bruce Greenblatt Directory Tools and Application Services, Inc. 6841 Heaton Moor Drive San Jose, CA 95119 USA

Phone: +1-408-224-5349 EMail: bgreenblatt@directory-applications.com

Ryan Moats AT&T 15621 Drexel Circle Omaha, NE 68135-2358 USA

Phone: +1 402 894-9456 EMail: jayhawk@att.com

Mark Wahl Innosoft International, Inc. 8911 Capital of Texas Hwy, Suite 4140 Austin, TX 78759 USA

Phone +1 626 919 3600 EMail Mark.Wahl@innosoft.com

Copyright (C) The Internet Society (1999). All Rights Reserved.

This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English.

The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns.

This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Acknowledgement 🫶

Funding for the RFC Editor function is currently provided by the Internet Society. Thank you, Internet Society, for keeping the lights on ✨

📦 where this page comes from