RFC 2968

Mesh of Multiple DAG servers - Results from TISDAG

📅 published October 2000, from ietf

Mesh of Multiple DAG servers - Results from TISDAG 🦋

Okay bestie, let's get into it 💅 This is RFC 2968 from the Network Working Group, and it's an Informational doc (category: Informational) written by L. Daigle and T. Eklof. It was published in October 2000. The title block doesn't list anything it obsoletes or updates, so it's just doing its own thing 🌸

Status of This Memo ✨

This memo is here to share information with the Internet community 🧚 It doesn't specify an Internet standard of any kind, so it's not a spec you have to follow. Distribution is unlimited, so share away 🌈

Copyright (C) The Internet Society (2000). All Rights Reserved. 🎀

Abstract

Okay so the Common Indexing Protocol ([CIP1]) is built for more than just making query referral indexes. It's also meant to help make meshes of (loosely) affiliated referral indexes 🕸️💖 The whole point of a mesh like that is to split up indexing and/or searching work across different servers, kind of like sharing the group project ✨ Until now, the TISDAG project ([TISDAG], [DAGEXP]) was all about building one single referral index. So the obvious next move is wiring that into a bigger set of services that can work together 🫶

1. Introduction ✨

1.1 Overview of mesh possibilities

Two different options are on the table for taking the TISDAG service into a mesh model (or a mix of both) 👑 First, it should be possible to build a mesh of DAG-based services. Or it might be super interesting to use the mesh to bring in access to other kinds of services, like the Norwegian Directory of Directories 🧚 Either way, the big idea is that services working together should trade index objects, and the mesh's shape (hierarchical, or graph-like, preferably without loops!) decides how that trading happens 🦋

As the CIP docs explain ([CIP1]), there are lots of ways to build indexes across several referral servers 💅 For example, WDSP index objects could just get passed along untouched. Or a referral index server's contents could be rolled up into a brand new index object, and that new object would generate referrals back to that server 🌈

Here's the plan: the mesh gets built from index objects that have been aggregated over the participating services' servers. So referrals point to other recognized services, not to each individual participant 🎀 You can do this as a hierarchy, or as a level mesh that's one layer deep. The big reason to not just pass index objects along as-is (unaggregated) is that each service might support a different range of access protocols, have its own security requirements, and so on 😭 Referrals should go to a CAP or CAPs. That could be the standard ones the DAG system already uses, or new ones made for the specific needs of remote systems (like other query types) 🔥 Inside a given DAG system, referrals to these remote servers will look just like any other referral. A particular SAP or SAPs might also be set up to handle query fulfillment, which helps with translating between service variations and with secure access if the relationship between the services is restricted 💜

For the scenarios below, the main service is Country A in Scenario 1 and Country B in Scenario 2, and both are DAG-based services. The scenarios are written with interoperating DAG services in mind, but in most cases they'd work just as well if the remote service ran on some other architecture 🍓 Again, the key thing for building any kind of mesh is trading the CIP index object, not the internal system architecture 🫶

1.1.1 Scenario 1: Top Down 🌸

Let's say 2 countries tie their services together 🤝 A user makes a query in Country A. A certain number of hits land against the index objects of A's WDSPs. There's also a hit in Country B's aggregate index. There are 3 possible cases for how that needs to be handled 👀

Case 1:

Okay so in Case 1, Country A and Country B run services that are basically the same, in terms of the protocols, queries, and schema they support 💖 In this case, one referral should be generated per protocol that Country B's service supports. The referral can be passed all the way back to the client, as long as the client's protocol supports referrals ✨ Or the CAP can chain the referral through an appropriate SAP, the usual way. In other words, Country B's CAPs act as WDSPs for Country A's service 💅

Check out this illustration (only the relevant CAPs, SAPs, etc. are shown, the rest got cut for space 🙈):

             +-----------------+
        (1)  |-----+ Country A |     +-------+
      ------>|Prot1|   DAG     |     |A-WSDP1|
      <------| CAP |     +-----|     | Prot1 |
        (2)  |-----+     |Prot1|     +-------+
             |           | SAP |
      ----+  |           +-----|     +-------+
       (3)|  |    +-------+    |     |A-WDSP2|
          |  |    | RI-A  |    |     | Prot1 |
          |  +-----------------+     +-------+
          |
          |                          +-------+
          |                          |A-WDSP3|
          |                          | Prot2 |
          +----------------+         +-------+
                           |          [...]
                           |
                           |         +-----------------+
                           |         |-----+ Country B |     +-------+
                           +-------->|Prot1|   DAG     |     |B-WSDP1|
                                     | CAP |     +-----|     | Prot2 |
                                     |-----+     |Prot1|     +-------+
                                     |           | SAP |
                                     |           +-----|     +-------+
                                     |    +-------+    |     |B-WDSP2|
                                     |    | RI-B  |    |     | Prot1 |
                                     +-----------------+     +-------+
                                                              [...]

Where:

  • Prot[i] is some particular query protocol
  • RI-A has an index over all A-WDSP[i] and RI-B
  • RI-B has an index over all B-WDSP[i]
  • (1) is the query to the Country A DAG system, which yields a referral based on the index object from RI-B
  • (2) is that referral
  • (3) is the resolution of that referral, which the client takes to the Country B DAG system directly (to find out which, if any, B-WDSP[i] have relevant information)

Case 2:

Now Case 2 is when Country A and Country B both cover the same service type (like whitepages), but they don't use an identical set of protocols, allowed queries, or schema 🌈 Country B's index object that was sent to Country A's DAG service has to be built in terms of Country A's service, so the right hits can come up against it (meaning, so referrals to Country B's service can be generated) 🎀 But resolving the referral needs some extra protocol, schema, and query mapping 😤 A special SAP set up inside Country A's service can handle that, by mapping Country A's service onto Country B's published service 🧚 Country A can then choose to support just one of Country B's access protocols, and the designated SAP will always contact one type of CAP at Country B 💜

Or, Country B can set up a particular CAP that maps Country A's service into whatever fits best with the internal structure of its own service 🔥 In that setup, Country A's referral goes to a special CAP in Country B's service (which, again, looks like a WDSP to Country A's service). In fact, the client software might be able to handle the referral directly 🫶 The big difference between these two approaches is who's responsible for managing the relationship between the two service types. On one hand, Country A could handle it, if it knows its own service well and also knows the published access to Country B. On the other hand, Country B could be on the hook for setting up a CAP for every country that wants to connect to it 😬 That second option can sometimes be justified by how much internal optimization Country B can do, and because it lowers the overhead for Country A's service (since it can pass the referral straight back to the client software) ✨

Check out this illustration (only the relevant CAPs, SAPs, etc. are shown, the rest got cut for space 🙈):

             +-----------------+
        (1)  |-----+ Country A |     +-------+
      ------>|Prot1|   DAG     |     |A-WSDP1|
      <------| CAP |     +-----|     | Prot1 |
        (2)  |-----+     |Prot1|     +-------+
             |           | SAP |
      ----+  |           +-----|     +-------+
       (3)|  |    +-------+    |     |A-WDSP2|
          |  |    | RI-A  |    |     | Prot1 |
          |  +-----------------+     +-------+
          |
          |                          +-------+
          |                          |A-WDSP3|
          |                          | Prot2 |
          +----------------+         +-------+
                           |          [...]
                           |
                           |         +-----------------+
                           |         |-----+ Country B |     +-------+
                           |         |Prot3|   DAG     |     |B-WSDP1|
                           |         | CAP |     +-----|     | Prot3 |
                           |         |-----+     |Prot3|     +-------+
                           |         |---------+ | SAP |
                           |         |Country A| +-----|
                           +-------->|CAP:Prot1|       |
                                     |---------+       |     +-------+
                                     |    +-------+    |     |B-WDSP2|
                                     |    | RI-B  |    |     | Prot3 |
                                     +-----------------+     +-------+
                                                              [...]

Where:

  • Prot[i] is some particular query protocol
  • RI-A has an index over all A-WDSP[i] and RI-B
  • RI-B has an index over all B-WDSP[i]
  • (1) is the query to the Country A DAG system, which yields a referral based on the index object from RI-B
  • (2) is that referral
  • (3) is the resolution of that referral, which the client takes to the Country B DAG system directly, but to a CAP that is specifically designed to accommodate protocols from Country A's service, and map it (and schema) into Country B's service. Likely, all Country B referrals will be chained for the Country A client

Case 3:

Case 3 is basically a twist on Case 1 🧠 If Country A and Country B run services that are identical in every way except for the data (the WDSPs covered), it might make sense to NOT aggregate Country B's WDSP index objects, and just copy them to Country A's server instead 💖 Then Country A's CAPs could get access to Country B's SAPs, so chaining happens directly at the remote service (instead of involving Country A's SAPs and Country B's CAPs, like in the first example above) 🦋 The answer doesn't come from the tech itself. It totally depends on the kind of relationship that can be set up between Country A and Country B's services 👑

1.1.2. Scenario 2: Working Up 🦋

Okay so, the scenario we just did quietly assumed that Country A's server had already received index objects from Country B's server 👀. That's totally true if Country A's server sits higher up in a hierarchy of services (set up by agreements between the service operators), or if the network is made of servers that all share their index objects with every other server. In that second case, searching at any one server gives you the full range of results, because referrals get made to any other server that might have data matching the user's query. Sharing index objects lets each server manage its own local data while still spreading out the load on basic query handling. Iconic teamwork 🫶

BUT 😬 if the server network is hierarchical, or at least not-completely-connected, then queries run at a level other than the top, or inside one branch, won't actually get matched against every index object. So there might be other servers the query should be sent to if the whole space needs searching. Say Country B is actually lower in the hierarchy than Country A in the example above. A user querying Country B's service might be totally happy limiting the query to that country's info. That's true in enough real-life situations that the hierarchy becomes a legit way to scope queries, which means you don't have to flood the whole network with every single query or keep full copies of all data on every server 💅

Still very much in theoretical stages 🧪, but the DAG/IP has control constructs that let DAG components act according to the topology of the mesh. A CAP might use the polled-by system command to figure out which other servers exist at higher levels, and those would be worth contacting if the scope of the search needs to grow 🔍

In the example above, a CAP in Country B's system could notice that Country A's service was polling Country B, and so treat it as a logical target for widening the query 🎯 We still need more experience (mostly with server mesh topologies) before it's clear how to make the best use of these capabilities. Open questions, bestie:

  • should the CAP always broaden the scope? only if there are no local referrals? under user direction?
  • should the CAP use a local SAP to contact the remote service's CAP?
  • is it better to completely connect the mesh of servers, or produce some kind of hierarchy?
  • etc

2. Other considerations 🌸

Depending on the context where a mesh gets set up (like between national white pages services, or different units of a corporate organization, etc), it might be useful to let individual WDSPs indicate whether they're willing to have their data included in a DAG system's aggregated index object. That way the DAG system can receive referrals from other systems in the mesh 🕸️ main character energy for the whole network, honestly.

3. Security Considerations 💀

Okay, serious moment, bestie 🙈 This document describes different configurations for sharing information between information services. It introduces no security considerations beyond the ones that come with (and get addressed by) particular directory service access protocols. So the security drama lives in those protocols, not here.

Don't skip this one: the security story is handled by the directory service access protocols themselves, so that's where you look.

4. Acknowledgements 🧚

The work described in this document was carried out as part of an ongoing Ericsson project. If you want more info about that project, reach out to:

Bjorn Larsson bjorn.x.larsson@era.ericsson.se

5. Authors' Addresses 💜

Leslie L. Daigle Thinking Cat Enterprises

EMail: leslie@thinkingcat.com

Thommy Eklof Hotsip AB

EMail: thommy.eklof@hotsip.com

6. References 📚

Request For Comments (RFC) and Internet Draft documents are available from numerous mirror sites, so go shopping 🛍️

[CIP1]   Allen, J. and M. Mealling, "The Architecture of the Common
         Indexing Protocol (CIP)", RFC 2651, August 1999.

[TISDAG] Daigle, L. and R. Hedberg "Technical Infrastructure for
         Swedish Directory Access Gateways (TISDAG)," RFC 2967,
         October 2000.

[DAGEXP] Eklof, T. and L. Daigle, "Wide Area Directory Deployment
         Experiences", RFC 2969, October 2000.

[NDD]    Hedberg, R. and H. Alvestrand, "Technical Specification, The
         Norwegian Directory of Directories (NDD)", Work in Progress.

Copyright (C) The Internet Society (2000). All Rights Reserved.

Okay, the legal bit 📜 This document and translations of it may be copied and furnished to others. Derivative works that comment on or explain it, or help with implementing it, may be prepared, copied, published and distributed, in whole or in part, with no restrictions of any kind, as long as the copyright notice above and this paragraph are included on every copy and derivative work. However, this document itself may not be modified in any way (like removing the copyright notice or references to the Internet Society or other Internet organizations), except as needed for developing Internet standards, in which case the copyright procedures in the Internet Standards process must be followed, or as required to translate it into languages other than English.

The limited permissions above are perpetual and will not be revoked by the Internet Society or its successors or assigns 💖

This document and the information in it are provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Acknowledgement 🌈

Funding for the RFC Editor function is currently provided by the Internet Society. Iykyk, and now you do too ✨

📦 where this page comes from