RFC 1713

Tools for DNS debugging 🎀

okay bestie, this RFC is an FYI from the Informational category, so it's not a standard, it's a friendly guide 🫶 Here's the title block in one glance:

• RFC number: 1713
• Series: Network Working Group
• Author: A. Romao, from FCCN
• FYI number: 27
• Date: November 1994
• Category: Informational
• Obsoletes / Updates: nothing is listed in this title block

Status of this Memo 🦋

this memo gives info to the Internet community 💖 It is not an Internet standard of any kind, and it can be shared freely with unlimited distribution ✨

Abstract 🌸

okay so DNS (Domain Name System) is literally everywhere and nobody notices it 🙈 The sad part is that admins usually don't look for problems as long as the apps that need name-to-address mapping keep working. This doc lists some tools domain admins can use to spot those weird anomalies and fix them 💅

1. Introduction ✨

Right now more than 3,800,000 computers are connected across the global Internet [1]. That's several millions of end-users who can reach any of those machines just by naming it 🌈 All of this works because of the world's widest distributed database, the Domain Name System, which gives distributed apps lots of services. The most iconic one is translating names into IP addresses and back again 🔥

You're using DNS every time you do an FTP or Telnet, when your gopher client follows a link to a remote server, when you click a hypertext item and have to reach the server defined by the URL, or when you talk to someuser@some.host. It's also there when your mail gets routed through a set of gateways before it reaches the final recipient, and when you post an article to Usenet and want it propagated all over the world 🦋 Those are the most visible uses, but a lot more apps rely on it too, like network security, monitoring and accounting tools, just to name a few 💜

DNS owes a huge chunk of its success to distributed administration 👑 Each component (called a zone, which is the same as a domain in most cases) is its own independent entity. It's responsible for what happens inside its domain of authority, for how and what information changes, and for letting the tree grow downwards by creating new components 🧚

BUT this distributed nature also causes inconsistencies, and it's giving chaos 😭 Lots of admins make mistakes when they configure their domains and when they delegate authority to sub-domains. Plenty of them don't even know how to do these things properly, so the problems just sit there and spread 💀 Bad implementations of DNS clients and servers cause trouble too, especially really old ones. They either don't follow the standards or they're error prone, and that creates or allows a lot of the problems above 🙈

All these anomalies make DNS less efficient than it could be, which causes trouble for network operations and affects the whole Internet 🍓 This document shows how important it is to manage DNS properly, and it covers what's already available to help admins take better care of their domains 🫶

2. DNS debugging 🔥

To help find problems in DNS configurations and/or implementations, there's a set of tools built specifically for this purpose ✨ Honestly, there are probably a lot of people in charge of domain administration who have no idea these tools exist, and who are even less aware of the anomalies that might be hiding in their configs 😭 This section describes some of these programs, including their scope, motivations and availability. It's meant as an intro to DNS debugging and as a guide for anyone who wants to check how healthy their domains and servers are 💖

Some prior knowledge is assumed, both about DNS basics and about other tools like dig and nslookup. Those aren't analyzed in detail here, because hopefully they're well-known enough from daily usage 🎀

2.1. Host 💜

Host is a program that retrieves DNS information from name servers 🧚 That info can be used for simple things like address-to-name mapping, or for more advanced stuff, like performing sanity checks on the data. It was created at Rutgers University, but then Eric Wassenaar from Nikhef did a major rewrite, and he still seems to be actively improving it 👑 You can get it from ftp://ftp.nikhef.nl/pub/network/host_YYMMDD.tar.Z, where YYMMDD is the date of the latest release.

By default, host just maps host names to Internet addresses, querying either the default servers or a specific one 💅 But you can also get any kind of data (resource records) by specifying different query types and classes, and by asking for verbose or debugging output from any name server. You can control parameters like recursion, retry times, timeouts, and whether to use virtual circuits or datagrams, when talking to name servers 🌈 This lets you simulate resolver behavior, which is super useful for finding problems with resolver operations (aka any application using the resolver library) ✨ As a query program, it can be just as powerful as nslookup or dig 🔥

As a debugger, host analyzes some set of the DNS space (for example, an entire zone) and produces reports with the results of its operation 🦋 To do that, it first does a zone transfer, which may be recursive, so it gets info from a zone and all its sub-zones. Then it analyzes that data based on the arguments you gave on the command line 💖 Zone transfers are done by contacting the authoritative name servers for that zone, so the request has to be allowed from those servers. Some of them refuse zone transfers (except from secondaries) to avoid congestion, so keep that in mind 🙈

⚠️ Host can flag anomalies about authority, like lame delegations (described below), and some more exotic stuff like extrazone hosts. An extrazone host is a host of the form host.some.dom.ain, where some.dom.ain is not a delegated zone of dom.ain.

These errors only show up when you ask for them on the command line. You might also get a bunch of other messages as secondary effects of host's operations. Some of those are mere warnings, which can be suppressed, and some are serious errors 💀 Warnings aren't as simple as they look, though. Most of them come from misconfigured zones, so it might not be a good idea to just ignore them 🫶

Error messages are about serious anomalies. They can come from the packets exchanged with the queried servers (like size errors, or invalid ancounts, nscounts and the like), or from the DNS info itself (also called "status messages" in the program's docs) 🌸 Those include inconsistencies between SOA records shown by different servers for a domain, unexpected address-to-name mappings, and name servers that aren't responding, aren't reachable, aren't running, or don't exist at all, and so on 😭

Host does all its querying online. That means it only works with data it received from name servers, so if you want different kinds of reports on the same piece of data, you have to query a name server more than once. You can arrange your arguments so you get everything in one run, but if you forget something or have to run it again, you get extra zone transfers, extra load on name servers, and extra DNS traffic 💜

Host is an excellent tool if you use it carefully 🦋 Like most querying programs, it can generate lots of traffic from one simple command. Still, its resolver simulation and debug features make it great for finding many common, and some not-so-common, DNS configuration errors. It also generates useful reports and statistics about the DNS tree ✨ For example, RIPE (Reseaux IP Europeens) NCC uses it to generate a monthly European hostcount, which gives an overview of Internet usage evolution in Europe 🌍 Along with these counts, error reports are generated, one per country, and all the info is available in the RIPE archive 👑

2.2. Dnswalk 🧚

Dnswalk is a DNS debugger written in Perl by David Barr from Pennsylvania State University 💖 You can find the latest version at ftp://ftp.pop.psu.edu/pub/src/dnswalk. The software comes with a small document where the author shares some useful advice, so it's worth a read 🎀

The program checks domain configurations that are stored locally, with data arranged hierarchically in directories that resemble the DNS tree organization of domains 🌈 To set up this info, dnswalk may first perform zone transfers from authoritative name servers. You can do a recursive transfer of a domain and its sub-domains, but be careful, because that can generate a huge amount of traffic 🙈 If the data is already present, dnswalk may skip these transfers, as long as the data is up to date ✨

Dnswalk looks for inconsistencies in resource records, like MX and aliases pointing to aliases or to unknown hosts, incoherent PTR, A and CNAME records, invalid characters in names, missing trailing dots, unnecessary glue information, and so on 💅 It also checks authority information, namely lame delegations and domains with only one name server 🔥 It's easy to use: you just specify the domain to analyze, plus some optional parameters, and the program does the rest 🦋 The catch is that only one domain (and its sub-domains, if that's the case) can be checked at a time 💀

While checking data, dnswalk uses dig and resolver routines (gethostbyXXXX from the Perl library) a lot. It uses them to get things like authority information from the servers of the analyzed domains, and to get names from IP addresses so it can verify the existence of PTR records and aliases 🫶 So besides the zone transfers, you can count on extra traffic just from running the program. That might not be negligible if you're debugging a fairly large amount of data and care about query retries and timeouts 😭

2.3. Lamers 🙈

okay so a lame delegation is a seriously big DNS config mistake, and sadly it's super common 💀. It happens when a name server is listed in the NS records for some domain, but it's actually not a server for that domain. So queries get sent to the wrong servers, who don't know nothing (at least not as expected) about the domain you're asking about 😭. Sometimes those hosts (if they even exist!) don't run name servers at all. So the queries time out and get resent, only to fail anyway, and that creates (more) unnecessary traffic 🫠.

It's honestly so easy to make one 💅. The most common case is when an admin changes the NS list for their domain and drops one or more servers from it, without telling the parent domain admins who gave them authority over the domain. From then on, the parent's name server keeps announcing one or more servers for the domain, and those servers get queries about stuff they don't know about 🙃. (On the flip side, servers can be added to the list without the parent's servers knowing, which hides valuable info from them. That's not a lame delegation, but it shouldn't happen either 👀.) Other examples are putting a name in an NS list without telling the admin of that host, or when a server suddenly stops providing name service for a domain.

To detect this and warn DNS admins all over the world, Bryan Beecher from University of Michigan wrote lamers 🎀, a program that analyzes named (the famous BIND name server) logging info [2]. To get useful logs, named got a patch to detect and log lame delegations. This patch was originally written by Don Lewis from Silicon Systems, and it's now part of the latest BIND release thanks to Bryan Beecher, so it's expected to be widely available soon ✨. Lamers is a small shell script that scans those logs and reports the lame delegations it finds. The reports go out by mail to the hostmasters of the affected domains, as stated in the SOA record for each one. If that's not possible, the message goes to the postmasters of the affected name servers instead. Bounces need manual processing, since they're caused by careless setup of those records or invalid postmaster addresses. The errors found by the U-M servers also get posted twice a month on the USENET newsgroup comp.protocols.tcp-ip.domains 📰.

If you ever get one of these reports, study it carefully so you can find and fix problems in your domain, or see if your servers are being affected by the spread of erroneous info 🧐. Even better, lamers could run on your own servers to catch more lame delegations, because U-M can't see them all! 🫶 Also, if you get mail reporting a lame delegation affecting your domain or some of your hosts, please don't just ignore it or flame the senders. They're really trying to help!! 🥹

You can get lamers from ftp://terminator.cc.umich.edu/dns/lame-delegations 🦋.

---

2.4. DOC 🔍

Authority info is one of the most significant parts of the DNS data, because the whole mechanism depends on it to traverse the domain tree correctly 🌳. Incorrect authority info leads to problems like lame delegations or, in extreme cases, the inaccessibility of a domain 😱. Picture this: the info given about all of a domain's name servers is wrong. Since you can't contact the real servers, you might end up unable to reach anything inside that domain 💀. This may be exaggerated, but if you've been on the DNS business long enough, you've probably seen some enlightened examples of this scenario 🙈.

To look for this kind of problem, Paul Mockapetris and Steve Hotz, from the Information Sciences Institute, wrote a C-shell script called DOC (Domain Obscenity Control) 🍓. It's an automated domain testing tool that uses dig to query the appropriate name servers about authority for a domain, and then analyzes the responses.

DOC only looks at authority data, because the authors anticipated that people would complain about things like invasion of privacy 🙅‍♀️. Also, back when it was written, most domains were so messy that they thought there wouldn't be much point checking anything deeper until the basic problems were fixed 🧹.

Only one domain gets analyzed each time ☝️. The program checks whether all the servers for the parent domain agree about the delegation info for the domain. Then DOC picks a list of name servers for the domain (obtained from one of the parent's servers) and starts checking their info, querying each one. It looks for the SOA record, checks whether the response is authoritative, compares the various records retrieved, gets each one's list of NS, and compares the lists (both among these servers and the parent's). For those servers inside the domain, it also looks for their PTR records 🔎.

Due to several factors, DOC seems to have frozen since its first public release, back in 1990 🥶. Within the distribution there's an RFC draft about automated domain testing, which was never published. Nevertheless, it may be useful reading 📖. The software can be fetched from ftp://ftp.uu.net/networking/ip/dns/doc.2.0.tar.Z 💜.

---

2.5. DDT 🧚

DDT (Domain Debug Tools) is a package of programs that scan DNS info for error detection 🔥. It was originally developed by Jorge Frazao from PUUG - Portuguese UNIX Users Group, and later rewritten by the author, who was at the time at the Faculty of Sciences of University of Lisbon 🇵🇹. Each program specializes in a given set of anomalies: there's a checker for authority info, another for glue data, mail exchangers, reverse-mappings, and miscellaneous errors found in all kinds of resource records. As a whole, they do a rather extensive checking of DNS configurations 💅.

These tools work on cached DNS data, meaning data stored locally after performing zone transfers from the appropriate servers. The transfers are presently done by a slightly modified version of BIND's named-xfer, called ddt-xfer, which allows recursive transfers. This means the tools don't query name servers online each time they run ✨. This option was taken for several reasons [3]: (1) efficiency, since it reads data from disk and avoids network transit delays, (2) reduced network traffic, because data only has to be fetched once and then the programs can run over it as many times as you wish, and (3) accessibility. In countries with limited Internet access, as was the case in Portugal when DDT was in its first stages, this may be the only practical way to use the tools 🌸.

Point (2) deserves some special considerations 👀. First, it's not entirely true that there aren't any additional queries while processing the information. One of the tools, the authority checker, queries (via dig) each domain's purported name servers to test the consistency of the authority info they provide about the domain 🕵️‍♀️. Second, someone might argue that when the actual tests are done, the info used may be out of date. That's true, but this is just the DNS nature: if you obtain a piece of info, you can't be sure it's still valid one second later 🫠. Furthermore, if your source wasn't the primary for the domain, you can't even be sure it was valid in the exact moment you got it in the first place. But experience shows that if you see an error, it's likely to still be there in the next version of the domain info. And if it isn't, nothing was lost by having detected it in the past 🦋. On the other side, of course there's little point in checking one-month-old data... 💀

The list of errors looked for includes:

• lame delegations
• version number mismatches between servers (this may be a transient problem)
• non-existing servers
• domains with only one server
• unnecessary glue information
• MX records pointing to hosts not in the analyzed domain (this may not be an error, it's just to point out possibly strange or expensive mail-routing policies)
• MX records pointing to aliases
• A records without the respective PTR, and vice-versa
• missing trailing dots
• hostnames with no data (A or CNAME records)
• aliases pointing to aliases
• and some more 👑

Given the specialized nature of each tool, it's possible to look for a well-defined set of errors, instead of having the data analyzed in all possible ways 🎀.

Except for ddt-xfer, all the programs are written in Perl 🐪. A new release may come out in the near future, after a thorough review of the methods used, the set of errors checked for, and some bug fixing. In particular, a Perl version of ddt-xfer is expected 🤞. In the meantime, the latest version is available from ftp://ns.dns.pt/pub/dns/ddt-2.0.1.tar.gz 🌈.

2.6. The Checker Project 🦋

okay bestie, DNS traffic has been getting researchers super curious for ages, bc most of it is literally unnecessary 😭. Observations showed DNS eats something like twenty times more bandwidth than it should [4]. And the causes? Deficient resolver and name server implementations spread all over the world, from personal machines to super-computers, running all sorts of operating systems 💀.

The panacea hasn't been found yet (people claim the latest official version of BIND is a great step forward [5]), so researchers have been hunting for where the anomalies come from as a first step toward a fix 🔍. The Checker Project is one such effort, developed at the University of Southern California [6]. It's a set of C code patched into BIND's named that monitors server activity and builds a database with the history of that operation (queries and responses). You can then generate reports from that database that summarize activity and spot behavioral patterns in client requests, looking for anomalies 📊. The named code change is small and simple unless you want PEC checking enabled (see below). You can find sources and documentation at ftp://catarina.usc.edu/pub/checker ✨.

Checker only does this kind of collection and reporting. It does not try to enforce any rules on the administrators of defective sites by any means whatsoever 🙅‍♀️. The authors hope that simply showing the evidence is reason strong enough for those administrators to get their problems fixed 🫶.

An interesting feature is PEC (proactive error checking) 🧚. The server pretends to be unresponsive for some queries: it randomly picks a name and starts refusing replies for queries on that name during a pre-determined period. Those queries still get recorded, though, so people can reason about the retry and timeout schemes used by name servers and resolvers 🕵️‍♀️. The expectation is that properly implemented clients will choose another name server to query, while defective ones keep trying the same server 👀. This feature seems to still be under testing, since it isn't completely clear yet how to interpret the results 🤷‍♀️. A PEC-only error checker is available from USC that is much simpler than the full error checker. It examines another name server client every 30 minutes to see if that client causes excessive load ⏰.

Presently, Checker has been running on a secondary for the US domain for more than a year with little trouble 💅. The authors feel confident it should run on any BSD platform (at least SunOS) without problems, and it is planned to be included as part of the BIND name server 🌸.

Checker is part of a research project led by Peter Danzig from USC, aimed at implementing probabilistic error checking mechanisms like PEC on distributed systems [7] 🧠. DNS is one such system, and it was chosen as the platform for testing the validity of these techniques over the NSFnet 🌐. The hope is to gain enough knowledge to improve the performance and reliability of distributed systems 🚀. Some of the targets for these checkers to detect are undetected server failures, query loops, bad retransmission backoff algorithms, misconfigurations, and resubmission of requests after negative replies 🎯.

2.7. Others 🎀

All the tools described above came from systematic work on DNS debugging, and some of them were built as part of research projects 🎓. For the sake of completeness, though, a few other programs get mentioned here too. These seem just as serious, but they were developed in a somewhat ad-hoc way, without any real intention of being used outside the environments where they were born 🐣. That's just an impression, and it's arguable, but there was no real need to give any of them an entire section. That doesn't mean they aren't valuable contributions! In some cases they may be exactly what you're looking for, without having to install a complete package just to do some testing on your domain 🍓.

The reference taken was the contrib directory in the latest BIND distribution (where some of the above programs can also be found) 📂. In there you'll find tools for creating your DNS configuration files and NIS maps from /etc/hosts and vice-versa, or for generating PTR from A records. These things matter because they help avoid common typing errors and inconsistencies between those tables 🙈. You'll also find syntax checkers for zone files, programs for querying and monitoring name servers, all the small programs presented in [8], and more 🧁. It's worth spending some time looking at them, because maybe you'll find that program you were planning to write yourself 💖. The latest public version of BIND can be found at ftp://gatekeeper.dec.com/pub/misc/vixie/4.9.2-940221.tar.gz. As of this writing, BIND-4.9.3 is in its final beta stages, and a public release is expected soon, also at gatekeeper.dec.com 🎉.

You may also want to consider a version control system like SCCS or RCS to keep your configuration files consistent through updates, or to use tools like M4 macros to generate those files 🗂️. As mentioned above, it's important to avoid human-generated errors, since they create problems that are difficult to track down. They're often hidden behind some mistyped name 😭. Errors like this may end up in many queries for a non-existing name, just to mention the less serious kind. See [9] for a description of the most common errors made while configuring domains 📝.

---

3. Why look after DNS? 🔍

Several pieces of software were presented to help people administer and debug their name services 🛠️. They differ a lot in how they do things, in scope, and in requirements, so it can be hard to even pick one to work with 😵‍💫. For one thing, people's expectations of these tools vary depending on their kind of involvement with DNS 💅. If you're responsible for a big domain, e.g., a top-level one or a big institution with many hosts and sub-domains, you probably want to see how well the tree below your node is organized. The consequences of errors tend to propagate upwards, affecting your own domain and servers 📈. For that you need a program that recursively descends the domain tree and analyzes each domain per se and the interdependencies between them all 🌳. You'll have to consider how deep you want your analysis to be, and what effects it will have on the network infrastructure. In other words, will it generate traffic only inside a campus network, no matter how big it is, or will it be spread over, say, a whole country? (Of course, your kind of connectivity plays an important role here 🌍.)

You may simply want to run some sanity checks on your own domain, without any further concerns ✅. Or you may want to join some kind of global effort to monitor name server traffic, either for research purposes or just to point out the "trouble-queries" flowing around 🚨.

Whatever your interest, you can almost surely find a tool to suit it 💜. Eliminating problems like those described in this document is a major contribution to the efficiency of an important piece of the Internet mechanism 👑. To get an idea of how important it is, think about all the applications that depend on it, not just the ones that get addresses out of names 🤯. Many systems rely on DNS to store, retrieve, and spread the information they need. Internet electronic mail was already mentioned (see [10] for details), and work is in progress to integrate X.400 operations with DNS [11]. Others include "remote printing" services [12], distributed file systems, and network routing purposes, among others 🖨️. These features may be accomplished by some standard, well-known resource records [13], or by new, experimental ones [14, 15]. Even if some of them won't succeed, one may well expect more load on the DNS burden 📦.

The ubiquitous DNS deserves a great deal of attention, perhaps much more than it generally gets 🥺. One could say it's a victim of its own success 🌈. If a user triggers an excessive number of queries just to get one request satisfied, they won't worry about it (in fact, they won't even notice it), won't complain to their system administrator, and things just keep going like this 🙃. Of course, DNS was designed to resist and provide its services despite all these anomalies. But by doing so, it's frequently forgotten, as long as people can Telnet or ftp 💤. As DNS takes on new responsibilities, as pointed out in the paragraph above, the problems described in this text will grow more serious, and new ones may appear (notably security ones [16], with a lot of work currently in progress addressing security in DNS). All of that happens if nothing is done to purge them 😱.

---

References 📚

• [1] Lottor, M., "Internet Domain Survey, October 1994", http://www.nw.com/zone/WWW/report.html, October 1994.
• [2] Beecher, B., "Dealing With Lame Delegations", Univ. Michigan, LISA VI, October 1992.
• [3] Frazao, J. and J. L. Martins, "Ddt - Domain Debug Tools, A Package to Debug the DNS Tree", Dept. Informatica Faculdade Ciencias Univ. Lisboa, DI-FCUL-1992-04, January 1992.
• [4] Danzig, P., "Probabilistic Error Checkers: Fixing DNS", Univ. Southern California, Technical Report, February 1992.
• [5] Kumar, A., J. Postel, C. Neuman, P. Danzig and S. Miller, "Common DNS Implementation Errors and Suggested Fixes", RFC 1536, USC/Information Sciences Institute, October 1993.
• [6] Miller, S. and P. Danzig, "The Checker Project, Installation and Operator's Manual", Univ. Southern California, TR CS94-560, 1994.
• [7] Danzig, P., K. Obraczka and A. Kumar, "An Analisys of Wide-Area Name Server Traffic", Univ. Southern California, TR 92-504, 1992.
• [8] Albitz, P. and C. Liu, "DNS and BIND", O'Reilly and Associates Inc., October 1992.
• [9] Beertema, P., "Common DNS Data File Configuration Errors", RFC 1537, CWI, October 1993.
• [10] Partridge, C., "Mail Routing and the Domain System", STD 14, RFC 974, CSNET CIC BBN Laboratories Inc., January 1986.
• [11] Allocchio, C., A. Bonito, B. Cole, S. Giordano and R. Hagens, "Using the Internet DNS to Distribute RFC1327 Mail Address Mapping Tables", RFC 1664, GARR, Cisco Systems Inc., Centro Svizzero Calcolo Scientifico, ANS, August 1994.
• [12] Malamud, C. and M. Rose, "Principles of Operation for the TPC.INT Subdomain: General Principles and Policy", RFC 1530, Internet Multicasting Service, Dover Beach Consulting Inc., October 1993.
• [13] Rosenbaum, R., "Using the Domain Name System to Store Arbitrary String Attributes", RFC 1464, Digital Equipment Corporation, May 1993.
• [14] Everhart, C., L. Mamakos, R. Ullmann and P. Mockapetris (Ed.), "New DNS RR Definitions", RFC 1183, Transarc, Univ. Maryland, Prime Computer, Information Sciences Institute, October 1990.
• [15] Manning, B., and R. Colella, "DNS NSAP Resource Records", RFC 1706, USC/Information Sciences Institute, NIST, October 1994.
• [16] Gavron, E., "A Security Problem and Proposed Correction With Widely Deployed DNS Software", RFC 1535, ACES Research Inc., October 1993

---

Security Considerations 🔐

Plot twist: security issues are not discussed in this memo 💀. It's only briefly mentioned at the end of section 3, so please don't skip that bit, bestie 🙈.

Author's Address ✨

okay so this is where you can find Artur 🫶 he's the author of this RFC and his details are right here, bestie 💖

   Artur Romao
   DI - Faculdade de Ciencias e Tecnologia
   Universidade Nova de Lisboa
   Quinta da Torre
   P-2825 Monte de Caparica
   Portugal

   Phone: +351 1 294 28 44
   Fax:   +351 1 295 77 86
   EMail: artur@fct.unl.pt

that's the whole document, babe 🎀🦋