RFC 1794
DNS Support for Load Balancing
RFC 1794: DNS Support for Load Balancing ๐
okay bestie, this one is from the Network Working Group ๐ It's written by T. Brisco at Rutgers University, it's Informational, and it's dated April 1995 ๐ธ Nothing in the title block says it obsoletes or updates any other RFC, so it's doing its own thing ๐ฆ
Status of This Memo ๐ธ
okay so this memo gives information to the Internet community ๐ It does not specify an Internet standard of any kind, so it's not a standard, bestie. And distribution of it is unlimited, so anyone can share it ๐ซถ
1. Introduction โจ
so this RFC is here to do three things ๐ First, it chronicles a little foray into the IETF DNS Working Group. Second, it talks through other possible alternatives to provide or simulate load balancing support for DNS. And third, it offers an ultimate, super flexible solution for DNS support that balances loads of many different types ๐ฅ
2. History ๐
ngl the history of this goes back way before my time ๐ญ so there are definitely some holes here. hopefully other authors can fill them in ๐ซถ
okay so at the start, "load balancing" was meant to let the Domain Name System (DNS) [1] agents support the idea of "clusters" (a term borrowed from VMS) ๐ A cluster is a bunch of machines that are all functionally similar or the same, where it doesn't really matter which one gets picked, as long as the processing load is reasonably well spread across a series of actual different hosts ๐ง
around 1986, a bunch of different schemes started surfacing as hacks to the Berkeley Internet Name Domain server (BIND) distribution ๐ The most widely spread of these were the "Shuffle Address" (SA) modifications by Bryan Beecher, or possibly Marshall Rose's "Round Robin" code ๐
the SA records, however, did a round-robin ordering of the Address resource records, and didn't do much about the particular loads on the target machines ๐ Then Matt Madison (of TGV) implemented some changes that used VMS facilities to review the system loads and return A RRs in the order of least-loaded to most-loaded ๐ฅ
the problem with SAs was that load was not actually a factor ๐ and TGV's approach relied on VMS-specific facilities to order the records. The SA RRs also required changes to the DNS specification (in file syntax and in record processing). Both were viewed as drawbacks and not as general solutions ๐
most of the Internet waited in anticipation of an IETF-approved method for simulating "clusters" ๐๐ง
through a few IETF DNS Working Group sessions (Chaired by Rob Austein of Epilogue), everyone collectively agreed that a number of criteria must be met ๐
- A) Backwards compatibility with the existing DNS RFC.
- B) Information changes frequently.
- C) Multiple addresses should be sent out.
- D) It must interact with other RRs appropriately.
- E) It must be able to represent many types of "loads".
- F) It must be fast.
(A) would ensure that the installed base of BIND and other DNS implementations would keep operating and interoperating properly ๐ซถ
(B) would permit very fast update times, to enable modeling of real-time data โก Five minutes was thought of as a normal interval, though changes as fast as every sixty seconds could be imagined ๐
(C) would cover the possibility of a host's address being advertised as optimal, yet the machine crashing during the period within the TTL of the RR. The second-most preferable address would be advertised second, the third-most preferable third, and so on. This would allow a reasonable stab at recovery during machine failures ๐ฉน
(D) would ensure correct handling of all ancillary information, such as MX, RP, and TXT information, as well as reverse lookup information. It needed to be ensured that processes such as mail handling continued to work in an unsurprising and predictable manner ๐ฌ
(E) would ensure the flexibility everyone wished for ๐ A breadth of "loads" were wished to be represented by various members of the DNS Working Group. Some were pretty eclectic, like the address ordering by the RTT to the host, and some were pragmatic, like balancing the CPU load evenly across a series of hosts. All represented valid concerns within their own context. Having a separate RR type for each was unthinkable (primarily, because it would violate goal A) ๐
(F) needed to ensure a few things ๐ Primarily, the time to calculate the information used to order the addressing information did not exceed the TTL of the information distributed. In other words, elements with a TTL of five minutes didn't take six minutes to calculate ๐ญ Similarly, it seems a fairly clear goal in the DNS RFC that clients should not be kept waiting. Request processing should continue regardless of the state of any other processing occurring โณ
3. Possible Alternatives ๐ฆ
during various discussions with the DNS Working Group and with the Load Balancing Committee, it was noted that no existing solution dealt with all the wishes appropriately ๐ฅบ One of the major successes of the DNS is its flexibility, and it was felt that this needed to be retained in all aspects ๐ช It was conceived that perhaps not only address information would need to be changed rapidly, but other records may also need to change rapidly. At least this could not be ruled out, bestie, because who knows what technologies lurk in the future ๐ฎ
of primary concern to many was the ability to interact with older implementations of DNS ๐ง The DNS is implemented widely now, and changes to critical portions of the protocol could cause havoc for years ๐ It became rapidly apparent through conversations with Jon Postel and Dave Crocker (Area Director) that modifications to the protocol would be viewed dimly ๐
4. A Flexible Model ๐ง
after many hours of discussions, Rob Austein suggested that the changes could be implemented without changes to the protocol ๐คฏ If zone transfer behavior could be subtly changed, then the zone transfer process could accommodate the changing of various RR information. What was needed was a smarter program to do the zone transfers. So changes were made to BIND that would permit the specification of the program that does the zone transfers for particular zones ๐
there is no specification that a secondary has to receive updates from its primary server in any specific manner. It only needs to check periodically, and obtain new zone copies when changes have been made ๐ Conceivably, the zone transfer agent could obtain the information from any number of sources (e.g., a load average daemon, a round-robin sorter) and present the information back to the nameserver for distribution ๐
a number of questions arose from this concept, and all seem to have been dealt with accordingly ๐ Primarily, the DNS protocol doesn't guarantee ordering. Even so, it is clear that the ordering is predictive: information read in twice in the same order will be presented twice in the same order to clients ๐ Clients, of course, may reorder this information, but that is deemed a "local issue" because it is configurable by the remote systems administrators (e.g., sortlists, etc.) ๐ ๏ธ The zone transfer agent would have to account for any "mis-ordering" that may occur locally, but remote reordering (e.g., client-side sortlists) of RRs is impossible to predict ๐ฎ Since local mis-ordering is consistent, the zone transfer agents could easily account for this โ
secondarily, but perhaps more subtly, the problem arises that zone transfers aren't used by primary nameservers, only by secondary nameservers ๐ค To clarify this, the idea of "fast" or "volatile" subzones must be dealt with. In a volatile environment (where address or other RR ordering changes rapidly), the refresh rate of a zone must be set very low, and the TTL of the RRs handed out must similarly be very low ๐ There is no use in handing out information with TTLs of an hour when the conditions for ordering the RRs change minutely โฐ There must be a relatively close relationship between the refresh rates and TTLs of the information ๐ซถ Of course, with very low refresh rates, zone transfers between the primary and secondary would have to occur frequently. Given that primary and secondary nameservers should be topologically and geographically far apart, moving that much data that frequently is seen as prohibitive ๐ซ
also, the longer the propagation time between the primary and secondary, the larger the window in which circumstances can change, thus invalidating the secondary's information ๐ฌ It is generally thought that passing volatile information on to a secondary is fairly useless. If secondaries want accurate information, then they should calculate it themselves and not obtain it via zone transfers ๐งฎ This avoids the problem with secondaries losing contact with the primaries (but access to the targets of the volatile domain is still reachable), but the secondary has information that is growing stale ๐
what is essentially necessary is a secondary (with no primary) which can calculate the necessary ordering of the RR data for itself. This also avoids the problem of different versions of domain servers predictively ordering RR information in different predictive fashions ๐ฏ For a volatile zone, there is no primary DNS agent, but rather a series of autonomous secondary agents. Each autonomous secondary agent is, of course, capable of calculating the ordering or content of the volatile RRs itself ๐
5. Implementation โจ
Okay bestie, with some help from Masataka Ohta (Tokyo Institute of Technology), I modified BIND so you can specify the zone transfer program (zone transfer agent) for particular domains ๐ . It looks like this:
transfer <domain-name> <program-name>
Right now I define a separate subdomain with a few hosts in it, all volatile information ๐ฆ. The zone has a refresh rate of 300, and a minimum TTL of 300 indicated. The configuration file is volatile.hosts. Every 300 seconds, a program called doAxfer runs to do the zone transfer. It reads the file volatile.hosts.template and the file volatile.hosts.list. The addresses in volatile.hosts.list get rotated a random number of times, then substituted (in order) into volatile.hosts.template to generate the file volatile.hosts. Then doAxfer exits with a value of 1, which tells the nameserver the zone transfer worked, so the file should be read in and the info gets distributed โจ. The result is a host with multiple addresses, and those addresses get randomized every five minutes (300 seconds) ๐.
Two bugs keep plaguing us in this endeavor ๐ญ. First, BIND considers any TTL under 300 seconds "irrational" and swaps in 300 instead. That really hampers volatile zones. In the fastest possible case, a 0 TTL, information would get used once and then thrown away ๐. Presumably the new RR information could be calculated every 5 seconds, and the RRs handed out with a TTL of 0. It must be considered that one limitation on how fast a zone can be is going to be how fast a machine can calculate new information ๐ง.
The other bug is that, same as with TTLs, BIND considers any zone refresh rate under 15 minutes to be similarly irrational. Obviously a 15 minute zone refresh rate is unacceptable for this sort of application ๐.
For a work-around, the current code sets these same hard-coded values to 60 seconds. Sixty seconds is still large enough to avoid any residual bugs tied to small timer values, but it's short enough to make fast subzones useful ๐.
This version of BIND is currently in release within Rutgers University, running both "fast" and normal zones ๐ธ.
6. Performance
Okay, the performance of fast zones isn't exactly stellar, but it's not much more than the normal CPU loads BIND already induces ๐. Testing was done on a Sun Sparc-2 being used as a normal workstation, but no resolvers were using the name server, so basically the nameserver was idle ๐ด. With no fast subzones, BIND racked up 11 CPU seconds in 24 hours. With one fast zone, six address records, and a refresh every 300 seconds (5 minutes), BIND used 1 minute 4 seconds of CPU time. Same setup, but refreshed every sixty seconds, and BIND used 5 minutes and 38 seconds of CPU time ๐ฅ.
Honestly, no big surprise here: the CPU load on the serving machine was linear to the frequency of the refresh time โจ. The sixty second refresh setup used about five times as much CPU time as the 300 second one ๐. You can easily extrapolate that overall CPU utilization would be linear to the number of zones and the frequency of the refresh period ๐ฆ. All of this is based on a shell script that always said a zone update was necessary. A smarter program should realize when reordering the RRs isn't needed and skip those periodic zone reloads ๐ซถ.
7. Acknowledgments
Most of the ideas in this document came from conversations and proposals with so many people ๐ฅน. That includes, but isn't limited to, Robert Austein, Stuart Vance, Masataka Ohta, Marshall Rose, and the members of the IETF DNS Working Group ๐.
8. References
[1] Mockapetris, P., "Domain Names - Implementation and Specification", STD 13, RFC 1035, USC/Information Sciences Institute, November 1987. ๐
9. Security Considerations
Security issues are not discussed in this memo. ๐จ๐ฑ Not gonna lie, this is a total cliffhanger, so keep it in mind before you deploy anything ๐.
10. Author's Address
Thomas P. Brisco
Associate Director for Network Operations
Rutgers University
Computing Services, Telecommunications Division
Hill Center for the Mathematical Sciences
Busch Campus
Piscataway, New Jersey 08855-0879
USA
Phone: +1-908-445-2351
EMail: brisco@rutgers.edu
๐ฆ where this page comes from
- ietf: rfc1794 1995-04
- ๐ girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms