RFC 5351
An Overview of Reliable Server Pooling Protocols
An Overview of Reliable Server Pooling Protocols 🦋
Okay bestie, let's get into RFC 5351! 💖 It's from the Network Working Group and it's dated September 2008. The authors are P. Lei at Cisco Systems, Inc., L. Ong at Ciena Corporation, M. Tuexen at Muenster Univ. of Applied Sciences, and T. Dreibholz at University of Duisburg-Essen. 🫶 Its category is Informational, so it's an overview and not a standard. The title block doesn't list anything it obsoletes or updates, so there's nothing extra to chase there. 🌸
Status of This Memo ✨
This memo is here to give info to the Internet community. 💅 It does not specify an Internet standard of any kind, and distribution of it is unlimited. 🌈
Abstract 🎀
Reliable Server Pooling (you'll see it written as "RSerPool") gives you a set of services and protocols that don't care what application they're in, and they're built for making fault-tolerant, highly available client/server apps. 😭 This document is a tour of the protocols and mechanisms in the RSerPool suite. 🔥
Table of Contents 🍓
1. Introduction ....................................................3
2. Aggregate Server Access Protocol (ASAP) Overview ................6
2.1. Pool Initialization ........................................6
2.2. Pool Entity Registration ...................................6
2.3. Pool Entity Selection ......................................7
2.4. Endpoint Keep-Alive ........................................7
2.5. Failover Services ..........................................7
2.5.1. Cookie Mechanism ....................................7
2.5.2. Business Card Mechanism .............................8
3. Endpoint Handlespace Redundancy Protocol (ENRP) Overview ........8
3.1. Initialization .............................................8
3.2. Server Discovery and Home Server Selection .................8
3.3. Failure Detection, Handlespace Audit, and Synchronization ..9
3.4. Server Takeover ............................................9
4. Example Scenarios ...............................................9
4.1. Example Scenario Using RSerPool Resolution Service .........9
4.2. Example Scenario Using RSerPool Session Services ..........11
5. Reference Implementation .......................................12
6. Security Considerations ........................................12
7. IANA Considerations ............................................12
8. Acknowledgements ...............................................12
9. References .....................................................13
9.1. Normative References ......................................13
9.2. Informative References ....................................131. Introduction 🌸
Okay so the RSerPool protocol suite lets client apps (called "pool users" 🧚) pick a server (called a "pool element") from a group of servers that all offer the same service (that group is the "pool"). 💜 The protocols are currently aimed at the Experimental Track. 🔬
The RSerPool architecture is all about high availability and load balancing. 👑 It lets a pool user figure out which server in the pool is the best fit at any given moment. ✨ The architecture is designed around these goals:
- application-independent protocol mechanisms 💅
- separation of server naming from IP addressing 🎀
- use of the end-to-end principle to avoid dependencies on intermediate equipment 🦋
- separation of session availability/failover functionality from the application itself 🌈
- facilitation of different server selection policies 🍓
- facilitation of a set of application-independent failover capabilities 🫶
- peer-to-peer structure 💖
The basic pieces of the RSerPool architecture are in Figure 1 below. 👀
.......................
______ ______ . +-------+ .
/ ENRP \ / ENRP \ . | | .
|Server| <----> |Server|<----------.----->| PE 1 | .
\______/ ENRP \______/ ASAP(1) . | | .
^ . +-------+ .
| . .
| ASAP(2) . Server Pool .
V . .
+-------+ . +-------+ .
| | . | | .
| PU |<---------->. | PE 2 | .
| | PU to PE . | | .
+-------+ . +-------+ .
. .
. +-------+ .
. | | .
. | PE 3 | .
. | | .
. +-------+ .
.......................
Figure 1A server pool is a set of one or more servers that offer the same application functionality. 🌟 Those servers are called Pool Elements (PEs). 🧁 You can use several PEs in one pool for fault tolerance or load sharing, for example. 💫 PEs register into and de-register out of the pool at an entity called the Endpoint haNdlespace Redundancy Protocol (ENRP) server, using the Aggregate Server Access Protocol (ASAP) [RFC5352]. That link is labelled ASAP(1) in the figure. 🦋
Each server pool is identified by a unique byte string called the pool handle (PH). 🎀 The pool handle maps the pool to a specific PE, found by its IP address (both IPv4 and IPv6 PE addresses are supported) and port number. 🌈 The Pool User (PU) specifies the pool handle whenever it wants to access a server in the pool. 💅 To turn the pool handle into the address it needs for a PE, the PU asks an ENRP server using ASAP. That link is labelled ASAP(2) in the figure. 🫶 The space of pool handles is treated as a flat space with limited operational scope (see RFC 3237 [RFC3237]). 🌸 The RSerPool protocol documents don't cover how pool handles get administered yet. 🙈 The protocols between the PU and the PE are application-specific. 💖 It's assumed that the PU and PE are set up to share a common set of protocols for application-layer communication, separate from the RSerPool mechanisms. ✨
RSerPool has a few tools to help clients move between servers when one fails. 🔥 It lets the client find alternative servers, either when it first discovers the pool or in real time. 🧚 The original server can also give the client a state cookie, which the client can forward to an alternative server to carry application-specific state. 🍓 That info goes straight between the PE and the PU, over the association labelled PU to PE in the figure. 👑
ENRP servers are meant to be a fully distributed and fault-tolerant registry service. 🌈 They use ENRP [RFC5353] to keep the data about the pool handle mapping space in sync. ✨ For PUs and PEs, all ENRP servers are functionally equal. 💜 Because ENRP keeps them synchronized, a PE can register or de-register with any one of them, and a PU can resolve a PH with any one of them. 🫶 Figure 2 below shows an illustration with 3 ENRP servers. 🦋
______ _____
... / ENRP \ / ENRP \ ...
PEs/PUs <---->|Server| <----> |Server|<----> PEs/PUs
... ASAP \______/ ENRP \______/ ASAP ...
^ ^
| |
| / ENRP \ |
+---->|Server|<----+
ENRP \______/ ENRP
^
| ASAP
v
...
PEs/PUs
...
Figure 2The requirements for the RSerPool framework are defined in RFC 3237 [RFC3237]. 📜 It's worth noting that RSerPool's load balancing requirements partly overlap with grid computing and high-performance computing. 🌈 The scope of those two areas is totally different though. 💀 Grid and high-performance computing also cover things like managing different administrative domains, data locking and synchronization, inter-session communication, and resource accounting for powerful computation services. 🧠 RSerPool, on the other hand, is just a lightweight take on load distribution and session management. ✨ Those functions are meant for systems with only small memory and CPU resources. 🌸 Anything further is outside the scope of RSerPool, and if it's needed, the application can provide it itself. 🫶
This document is a tour of the RSerPool protocol suite, focusing on the Aggregate Server Access Protocol (ASAP) [RFC5352] and the Endpoint Handlespace Redundancy Protocol (ENRP) [RFC5353]. 🦋 Along with those protocol specs, there's a common parameter format specification [RFC5354] for both protocols, a definition of server selection rules (pool policies) [RFC5356], and a security threat analysis [RFC5355]. 🔐💖
2. Aggregate Server Access Protocol (ASAP) Overview 🧚
ASAP gives you a straightforward set of mechanisms for creating and maintaining pools of redundant servers. 🎀 Those mechanisms include:
- registration of a new server into a server pool 🌷
- de-registration of an existing server from a pool 🍓
- resolution of a pool handle to a server or list of servers 🔍
- liveness detection for servers in a pool 💓
- failover mechanisms for handling a server failure 🛟
2.1. Pool Initialization 🌱
Pools come into existence when a PE registers the first instance of the pool name with an ENRP server. ✨ They disappear when the last PE de-registers. 🫧 In other words, starting the first PE on some machine creates the pool once that registration reaches the ENRP server. 🚀
It's assumed that information RSerPool needs, such as the address of an ENRP server to contact, is configured into the PE beforehand. 🛠️ How to automate that configuration isn't addressed at this time. 🙈
2.2. Pool Entity Registration 🎀
okay so a new server joins a pool by sending a Registration message over ASAP to an ENRP server 💅 in it, it tells the ENRP server which pool handle it wants to join, gives itself a PE identifier (picked randomly, no cap), shares how long it plans to stay in the pool, and says which transport protocols and selection policy it supports. The ENRP server it contacts first is its Home ENRP server 🏠. That server keeps a list of the PE's subscriptions and runs periodic audits to make sure the PE is still responsive 👀
when a PE wants to leave, it goes through similar steps to de-register 🦋 or, if it'd rather be low-key about it, it can just let the lifetime it registered with run out, and then it's gracefully removed from the pool 🫶
2.3. Pool Entity Selection 🌸
when an endpoint wants to connect to a server in the pool, it makes an ASAP Handle Resolution message and sends it to its Home ENRP server ✨ the ENRP server looks up the handle using what it knows about the pool servers and sends back a Handle Resolution Response message over ASAP 💖 that response has a list of the IP addresses of one or more pool servers that can be contacted 🧚
how the list gets made can involve a bunch of server selection policies. the RSerPool protocol suite has a few basic ones built in, and it also lets external server selection input handle the more complex stuff 🔥
2.4. Endpoint Keep-Alive 💜
ENRP servers check on pool elements using the ASAP Endpoint Keep-Alive message 👀 a PE answers an ASAP Keep-Alive message with an Endpoint Keep-Alive Ack response 💅
also, a PU can tell its Home ENRP server that the PE it was using has gone unresponsive by sending an ASAP Endpoint Unreachable message to that ENRP server 😭
2.5. Failover Services ⭐
RSerPool stays application-independent, but it still gives you some simple hooks for failover of a single session with a pool element 🫶 failover that depends on application state or transaction status can't really be defined without knowing the specific application, so that's on the app 🙈 still, a few simple RSerPool mechanisms give any application some level of failover 🌈
2.5.1. Cookie Mechanism 🍓
cookies may optionally be generated by the ASAP layer and sent from the PE to the PU every so often 🍪 the PU only keeps the last cookie it got 💜 if failover happens, the PU sends that last cookie to the new PE 🦋 this is a simple way to share state between PEs 🫶 please note the old PE should sign the cookie, and the receiving PE should verify that signature 🔐 the PU doesn't read any structure into the cookie. it just stores it and passes it along to the new PE ✨
2.5.2. Business Card Mechanism 💳
a PE can send a business card to its peer (a PE or a PU) that has its pool handle plus guidance on which other PEs the peer should use for failover 🎀 this lets a PE tell a PU which one it sees as the "next best" PE to use if something goes wrong 🥹 that choice can be based on pool stuff, like load balancing, or user stuff, like PEs with the most up-to-date state information 👑
3. Endpoint Handlespace Redundancy Protocol (ENRP) Overview 👑
a set of server pools, called a handlespace, is managed by ENRP servers ✨ pools aren't valid across the whole Internet. they only work inside smaller domains called the operational scope 🌍 the ENRP servers use the ENRP protocol to keep up a distributed, fault-tolerant, real-time registry service 💅 they also talk to each other to swap info like pool membership changes, handlespace data synchronization, and so on 🫶
3.1. Initialization 🌱
each ENRP server starts by generating a 32-bit server ID that it uses in later messages. it stays the same for the whole life of the server 🔢 then it tries to find all the other ENRP servers in its server pool scope 🔍 it can do that with a pre-defined Mentor server, or by sending Presence messages on a well-known multicast channel and picking one from the replies to be its Mentor 🧚 a Mentor can be any peer ENRP server 💖 the server asks its Mentor for the latest handlespace data using Handle Table Requests 📨 the answer comes back in Handle Table Response messages, and the server unpacks it into its local database 🎁 after that, the ENRP server is ready to serve ENRP services ✨
3.2. Server Discovery and Home Server Selection 🔎
PEs can now sign up with the brand new ENRP server by using ASAP messages 💖 they find it once the server sends an ASAP Server Announce message on the well-known ASAP multicast channel 📣 PEs only need to register with one ENRP server, because the other ENRP servers in the pool will sync up their pool element info using the ENRP protocol 🫶
a PE might have a configured list of ENRP servers to talk to, given as IP addresses. in that case it starts setting up associations with some of them and makes the first one that responds its Home ENRP server 🏠
or, it can listen on the multicast channel for a set period, and start an association when it hears an ENRP server 👂 the first server it sets up an association with can then become its Home ENRP server 🥰
3.3. Failure Detection, Handlespace Audit, and Synchronization 🕵️♀️
ENRP servers send ENRP Presence messages to all their peers to show they're still alive 💚 these Presence messages also include a checksum computed over all PE identities that the ENRP server is the Home ENRP server for 🔢 each ENRP server keeps an up-to-date list of its peers and can also work out the checksum it expects from a given peer, based on its local handlespace database 🧮 by comparing the expected sum with the sum a peer reports (this is the handlespace audit), an inconsistency can be caught 🚨 when that happens, the handlespace, limited to the PEs owned by that peer, can be requested for synchronization, the same way as in Section 3.2 🔁
3.4. Server Takeover 🦸♀️
if an ENRP server is found to be unresponsive, the remaining ENRP servers negotiate which other server takes over the Home ENRP role for the PEs of the failed peer 🤝 once they agree on the takeover, the ENRP server taking over those PEs sends a notification to its peers (via ENRP) and to the PEs it took over (via ASAP) 📬 the handoff, like, fully handled 💅✨
4. Example Scenarios 📚
4.1. Example Scenario Using RSerPool Resolution Service 🧁
RSerPool can be used in a 'standalone' way. the application uses RSerPool to find the address of a primary server in the pool, then talks straight to that server without using RSerPool services again 💫 if that first server fails, the app calls RSerPool again to find the next one 🔄
for pool user ("client") apps, if an ASAP implementation is on the client system, there are usually only three changes needed in the app source code 👩💻
- instead of hardcoding hostnames of the primary, secondary, tertiary servers, and so on, the app user gives a pool handle 🎫
- instead of a DNS-based service (like the Unix library function
getaddrinfo()) to turn a hostname into an IP address, the app calls an RSerPool service primitive, provisionally namedGETPRIMARYSERVER. it takes a pool handle as input and returns the IP address of the primary server 🪄 the app then uses that IP address just like it would have used the IP address DNS returned before 💅 - without extra RSerPool services, failure detection and failover logic has to be designed into each app 🛠️ but when the primary server fails, instead of running DNS translation again on a secondary server's hostname, the app calls a service primitive provisionally named
GETNEXTSERVER. it does two things in one go 🎯- first, it tells the RSerPool layer that the server returned by an earlier
GETPRIMARYSERVERorGETNEXTSERVERcall has failed 💀 - second, it gives back the IP address of the next server to contact, based on the best info the RSerPool layer has at that moment (like the available pool elements, the pool element policy in effect for the pool, and so on) 🧠
- first, it tells the RSerPool layer that the server returned by an earlier
Note: at the time of this document, a full API for use with RSerPool protocols has not yet been defined. 📝
for pool element ("server") apps where an ASAP implementation is available, two changes are needed in the app source code 🖥️
- the server should invoke the
REGISTERservice primitive when it starts, to add itself to the server pool using an appropriate pool handle 🎉 this also covers the address(es), protocol or mapping id, port (if the mapping requires one), and pooling policy (or policies) 📋 - the server should invoke the
DEREGISTERservice primitive to remove itself from the server pool when it shuts down 👋
using these RSerPool services gets you limited benefits compared to using all of them, but they're still super useful compared to not using RSerPool at all 🌈 first, the client user only has to supply one string, the pool handle, instead of a whole list of servers 🎀 second, the server to use can be picked dynamically by the server selection mechanism (a "pool policy" performed by ASAP; see ASAP [RFC5352]) 🎯 finally, when failures happen, they're reported to the pool through the signaling in ASAP [RFC5352] and ENRP [RFC5353], so other clients will eventually find out that this server has failed, once other elements of the RSerPool architecture confirm it 🫶💖
4.2. Example Scenario Using RSerPool Session Services ✨
Okay bestie, here's the big one. When you use the full RSerPool suite, EVERYTHING between the pool user and the pool element goes through the RSerPool framework 🌸. That includes starting a session, ending it, and sending and receiving data. So you have to change your app to call the RSerPool service primitives (aka the API) instead of the transport layer primitives from TCP, Stream Control Transmission Protocol (SCTP), or whatever transport you're using. It's a whole glow-up 💅
Same as the last scenario, you're building sessions (not plain connections or associations), and the destination is a pool handle instead of a list of IP addresses plus a port number. The difference is that failover from one pool element to another is totally automatic 🦋 and can be invisible to the app, as long as the app saved enough state in a state cookie 🎀:
- The RSerPool framework control channel handles the maintenance stuff, like keeping pool element lists, policies, and so on, fresh and up to date.
- Since RSerPool manages the application data (the data channel), any unsent data (stuff RSerPool hasn't handed to the underlying transport yet) gets automatically redirected to the newly picked pool element during failover. If the transport layer can retrieve unsent data (like SCTP can), that retrieved data can also be automatically re-sent to the new pool element. Iconic 👑
- An application server (pool element) can attach a state cookie (see Section 2.5.1), and the ASAP layer at the pool user passes it along to another pool element automatically when a failover happens. The new pool element can use that cookie to rebuild whatever state it needs to pick up a session or transaction that got interrupted by a failure between the pool user and the original pool element. Not me being impressed 🥹
- The application client (pool user) can provide a callback function that runs on the pool user side whenever a failover happens. That callback can run any app-specific failover code, like sending a special message (or a sequence of messages) that helps the new pool element rebuild any state it needs to continue an in-process session 🧚
- Picture this: in a peer-to-peer app, PU A is talking to PE B, and PU A also happens to be a PE in pool X 🫶. PU A can hand PE B a "business card" that says it's a member of pool X. If A fails, or the communication link between A and B fails, PE B can use that business card to reach an equivalent PE to PU A in pool X. Main character energy ⭐
- And if the app at PU A knows about certain PEs in pool X that B should preferably contact when A becomes unreachable from B, PU A can give that list to the ASAP layer, and it'll be included in A's business card (see Section 2.5.2). Thoughtful, honestly 🌈
Pro tip from your favourite networking nerd: the failover magic only works if the state cookie actually has enough info in it. Save your state, bestie! 🍓
5. Reference Implementation 🌷
A reference implementation of RSerPool is available at [RSerPoolPage] and it's described in [Dre2006]. Go check it out 🧚
6. Security Considerations 🔐
This document doesn't spell out any security requirements beyond the ones already covered in the ENRP and ASAP protocol specs. The full security threat analysis of RSerPool lives in THREATS [RFC5355], so if you're doing anything serious with this, go read that one. Don't skip it, it's not optional, it's giving "serious security risk" energy 💀
7. IANA Considerations 📋
This document doesn't need any extra IANA actions beyond the ones already identified in the ENRP [RFC5353] and ASAP [RFC5352] protocol specs. Everything's already handled, so we're good ✨
8. Acknowledgements 💜
The authors want to thank Maureen Stillman, Qiaobing Xie, Randall Stewart, Scott Bradner, and many others for their invaluable comments. Real ones, the whole crew 🫶
9. References 📚
9.1. Normative References
[RFC3237] Tuexen, M., Xie, Q., Stewart, R., Shore, M., Ong, L.,
Loughney, J., and M. Stillman, "Requirements for
Reliable Server Pooling", RFC 3237, January 2002.
[RFC5352] Stewart, R., Xie, Q., Stillman, M., and M. Tuexen,
"Aggregate Server Access Protocol (ASAP)", RFC 5352,
September 2008.
[RFC5353] Xie, Q., Stewart, R., Stillman, M., Tuexen, M., and
A. Silverton, "Endpoint Handlespace Redundancy
Protocol (ENRP)", RFC 5353, September 2008.
[RFC5354] Stewart, R., Xie, Q., Stillman, M., and M. Tuexen,
"Aggregate Server Access Protocol (ASAP) and Endpoint
Handlespace Redundancy Protocol (ENRP) Parameters",
RFC 5354, September 2008.
[RFC5355] Stillman, M., Ed., Gopal, R., Guttman, E., Holdrege,
M., and S. Sengodan, "Threats Introduced by Reliable
Server Pooling (RSerPool) and Requirements for
Security in Response to Threats", RFC 5355,
September 2008.
[RFC5356] Dreibholz, T. and M. Tuexen, "Reliable Server Pooling
Policies", RFC 5356, September 2008.9.2. Informative References
[RSerPoolPage] Dreibholz, T., "Thomas Dreibholz's RSerPool Page",
<http://tdrwww.iem.uni-due.de/dreibholz/rserpool/>.
[Dre2006] Dreibholz, T., "Reliable Server Pooling --
Evaluation, Optimization and Extension of a Novel
IETF Architecture", Ph.D. Thesis University of
Duisburg-Essen, Faculty of Economics, Institute for
Computer Science and Business Information Systems,
March 2007, <http://duepublico.uni-duisburg-essen.de/
servlets/DerivateServlet/Derivate-16326/
Dre2006-final.pdf>.Authors' Addresses 🦋
Peter Lei
Cisco Systems, Inc.
955 Happfield Dr.
Arlington Heights, IL 60004
US
Phone: +1 773 695-8201
EMail: peterlei@cisco.com
Lyndon Ong
Ciena Corporation
PO Box 308
Cupertino, CA 95015
US
EMail: Lyong@Ciena.com
Michael Tuexen
Muenster Univ. of Applied Sciences
Stegerwaldstr. 39
48565 Steinfurt
Germany
EMail: tuexen@fh-muenster.de
Thomas Dreibholz
University of Duisburg-Essen, Institute for Experimental Mathematics
Ellernstrasse 29
45326 Essen, Nordrhein-Westfalen
Germany
Phone: +49 201 183-7637
Fax: +49 201 183-7673
EMail: dreibh@iem.uni-due.de
URI: http://www.iem.uni-due.de/~dreibh/Full Copyright Statement 🌈
Copyright (C) The IETF Trust (2008). The rights are set out in BCP 78, and apart from what's in there, the authors keep all their rights 🎀
This document and the info in it are provided on an "AS IS" basis, and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. No promises, babe 🙈
Intellectual Property 💖
The IETF doesn't take a position on whether any Intellectual Property Rights or other rights that someone might claim over the tech in this document are valid or what they cover. It also doesn't say whether any license for those rights is available, and it hasn't independently gone looking for any such rights 🔍 You can find the procedures for rights in RFC documents in BCP 78 and BCP 79.
Copies of IPR disclosures sent to the IETF Secretariat, and any assurances of licenses that might be made available (or the result of an attempt to get a general license or permission to use those proprietary rights for implementers or users of this spec), can be found in the IETF online IPR repository at http://www.ietf.org/ipr.
The IETF invites anyone who's interested to let it know about any copyrights, patents, patent applications, or other proprietary rights that might cover technology needed to implement this standard. Send that info to the IETF at ietf-ipr@ietf.org 💜
📦 where this page comes from
- ietf: rfc5351 2008-09
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms