RFC 8906

A Common Operational Problem in DNS Servers: Failure to Communicate

πŸ“… published September 2020, from ietf

RFC 8906: A Common Operational Problem in DNS Servers: Failure to Communicate πŸ’Œ

okay bestie, first the official paperwork πŸ“œ This is RFC 8906, titled A Common Operational Problem in DNS Servers: Failure to Communicate. It's an IETF document by M. Andrews and R. Bellis, both from ISC. It's a Best Current Practice, so it's BCP 231 πŸŽ€ It came out in September 2020 with ISSN 2070-1721. The header doesn't list anything it obsoletes or updates ✨

Abstract ✨

The DNS is a query/response protocol 🧚 When servers fail to answer queries, or answer them wrong, you get immediate operational headaches and long-term problems for protocol development 😭

This document points out some common kinds of queries that certain servers either ignore or answer incorrectly πŸ‘€ It also suggests steps zone operators can take to find the problem and fix it πŸ”§

The document doesn't look at the DNS data itself, just the structure of the responses πŸ™ˆ

Status of This Memo

This memo documents an Internet Best Current Practice πŸ’…

This document comes from the Internet Engineering Task Force (IETF). It reflects the consensus of the IETF community, got public review, and was approved for publication by the Internet Engineering Steering Group (IESG). If you want more info on BCPs, check Section 2 of RFC 7841 πŸ“š

For the current status of this document, any errata, and how to send feedback, go to https://www.rfc-editor.org/info/rfc8906 πŸ¦‹

Copyright (c) 2020 IETF Trust and the persons identified as the document authors. All rights reserved πŸ’œ

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents, which you can find at https://trustee.ietf.org/license-info as in effect on the date of publication. Please read these carefully, because they spell out your rights and restrictions for this document πŸ“œ Code Components taken from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they come without warranty as described in the Simplified BSD License 🫢

Table of Contents πŸ—ΊοΈ

1.  Introduction
2.  Consequences
3.  Common Kinds of Queries That Result in No or Bad Responses
  3.1.  Basic DNS Queries
    3.1.1.  Zone Existence
    3.1.2.  Unknown/Unsupported Type Queries
    3.1.3.  DNS Flags
    3.1.4.  Unknown DNS Opcodes
    3.1.5.  TCP Queries
  3.2.  EDNS Queries
    3.2.1.  EDNS Queries: Version Independent
    3.2.2.  EDNS Queries: Version Specific
    3.2.3.  EDNS Options
    3.2.4.  EDNS Flags
    3.2.5.  Truncated EDNS Responses
    3.2.6.  DO=1 Handling
    3.2.7.  EDNS over TCP
4.  Firewalls and Load Balancers
5.  Packet Scrubbing Services
6.  Whole Answer Caches
7.  Response Code Selection
8.  Testing
  8.1.  Testing: Basic DNS
    8.1.1.  Is the server configured for the zone?
    8.1.2.  Testing Unknown Types
    8.1.3.  Testing Header Bits
    8.1.4.  Testing Unknown Opcodes
    8.1.5.  Testing TCP
  8.2.  Testing: Extended DNS
    8.2.1.  Testing Minimal EDNS
    8.2.2.  Testing EDNS Version Negotiation
    8.2.3.  Testing Unknown EDNS Options
    8.2.4.  Testing Unknown EDNS Flags
    8.2.5.  Testing EDNS Version Negotiation with Unknown EDNS
            Flags
    8.2.6.  Testing EDNS Version Negotiation with Unknown EDNS
            Options
    8.2.7.  Testing Truncated Responses
    8.2.8.  Testing DO=1 Handling
    8.2.9.  Testing EDNS Version Negotiation with DO=1
    8.2.10. Testing with Multiple Defined EDNS Options
  8.3.  When EDNS Is Not Supported
9.  Remediation
10. Security Considerations
11. IANA Considerations
12. References
  12.1.  Normative References
  12.2.  Informative References
Acknowledgements
Authors' Addresses

1. Introduction ✨

The DNS [RFC1034] [RFC1035] is a query/response protocol 🧚 When a server fails to respond to queries or responds incorrectly, that causes immediate operational problems and long-term problems with protocol development 😭

Not responding is basically indistinguishable from packet loss unless you dig into the query-response patterns πŸ•΅οΈβ€β™€οΈ On top of that, a missing response makes DNS clients send unnecessary queries and slows down the resolution process 🐌

Here's the tricky part: since nameservers or middleboxes that drop Extension Mechanisms for DNS (EDNS) [RFC6891] queries look exactly like packet loss, the loss sometimes gets misread as "no EDNS support." That can lead to DNSSEC validation failures πŸ’€

Servers that don't respond to queries make developers nervous about deploying new standards πŸ™ˆ These servers need to be found and fixed.

The DNS has response codes that cover almost any response you could imagine 🌈 A nameserver should be able to answer any imaginable query with them. There shouldn't be any need to drop a query just because the nameserver doesn't understand it 🫢

Unless a nameserver is under attack, it should respond to every DNS request sent to it. When one is under attack, it may want to drop packets πŸ›‘οΈ A common attack uses a nameserver as an amplifier by sending spoofed packets, because response packets are bigger than queries and large amplification factors are available, especially if EDNS is supported πŸ’₯ Limiting the rate of responses is reasonable in that situation, and the client should retry πŸ” But this only works if legitimate clients aren't forced to guess whether EDNS queries are accepted. As long as some servers don't respond to EDNS requests, clients have no way to tell whether silence means packet loss, EDNS not being supported, or rate limiting because the server is under attack 😡 Misclassifying server behaviour is unavoidable when rate limiting is used, until the population of servers that fail to respond to well-formed queries drops to near zero.

Nameservers should respond to queries even when the queried name isn't one the server is configured to answer for πŸ’β€β™€οΈ Misconfigured nameservers are super common in the DNS, and getting queries for zones the server isn't configured for doesn't necessarily mean the server is under attack. Parent zone operators are advised to regularly check that their delegating NS records match those of the delegated zone, and to fix them when they don't (Section 4.2.2 of [RFC1034], Paragraph 3). Doing this regularly should reduce the number of broken delegations πŸŽ€

This document doesn't try to identify every possible error, and it doesn't give an exhaustive list of tests 🌸

2. Consequences πŸ’₯

Not following the guidance in relevant DNS RFCs has multiple bad consequences 😩 Some come straight from the non-compliant behaviour, and others come from workarounds that get forced onto recursive servers. Fixing known issues now will cut down on future interoperability problems as DNS keeps evolving and clients start using newly introduced DNS features ✨ In particular, the base DNS specification [RFC1034] [RFC1035] and the EDNS specification [RFC6891] need to be followed when implemented.

Here are some known consequences:

  • The AD (Authenticated Data) bit in a response can't be trusted to mean anything, because some servers incorrectly copy the flag bit from the request to the response [RFC1035] [RFC4035]. The use of the AD bit in requests is defined in [RFC6840] πŸ™„
  • Widespread non-response to EDNS queries has led recursive servers to assume EDNS isn't supported and that falling back to plain DNS is required, which can potentially cause DNSSEC validation failures πŸ’”
  • Widespread non-response to EDNS options forces recursive servers to figure out whether it's the specific EDNS option or EDNS in general that's causing the non-response. In the limited time available to resolve a query before the client times out, that isn't possible 🫠
  • Incorrectly returning FORMERR when an EDNS option is present means the recursive server can't tell whether the server is just broken in handling that option, or doesn't support EDNS at all πŸ€·β€β™€οΈ
  • Mishandling of unknown query types has contributed to the abandonment of the transition of the SPF type πŸ₯€
  • Mishandling of unknown query types has slowed down the development of DNS-Based Authentication of Named Entities (DANE), and resulted in extra rules being specified to reduce the chance of interacting with a broken server when making TLSA queries πŸ”

These consequences will only grow if measures aren't put in place to remove non-compliant servers from the ecosystem πŸ“ˆ Working around issues caused by non-compliance with RFCs isn't sustainable 😀

Most (if not all) of these consequences could have been avoided if action had been taken to remove non-compliant servers as soon as people were aware of them. That means actively seeking out broken implementations and servers, and informing their developers and operators that they need to fix their servers πŸ”§πŸ’…

3. Common Kinds of Queries That Result in No or Bad Responses πŸŽ€

This section is split into Basic DNS requests and EDNS requests πŸ“

3.1. Basic DNS Queries

3.1.1. Zone Existence

If a zone is delegated to a server, that server should respond to an SOA query for that zone with an SOA record. Not responding at all is always incorrect, no matter how the server is configured πŸ™…β€β™€οΈ Responding with anything other than an SOA record in the answer section means there's a bad delegation 🚩

3.1.2. Unknown/Unsupported Type Queries

Some servers don't respond to unknown or unsupported types 😭 If a server gets a query for a type it doesn't recognise, or doesn't implement, it's expected to return the appropriate response as if it did recognise the type but has no data for it. That means either NOERROR or NXDOMAIN. The exception is queries for Meta-RR types, which may return NOTIMP 🧩

3.1.3. DNS Flags

Some servers don't respond to DNS queries that have various DNS flags set, whether those flags are defined or still reserved 🚫 When this was written, there were servers that didn't respond to queries with the AD flag set to 1, and servers that didn't respond to queries with the last reserved flag set πŸ’€

Servers should respond to such queries. If the server doesn't know what a flag means, it must not copy it to the response (Section 4.1.1 of [RFC1035]). If the server doesn't understand the meaning of a request, it should reply with a FORMERR response with unknown flags set to zero 🫢

3.1.3.1. Recursive Queries

A non-recursive server is supposed to respond to recursive queries as if the Recursion Desired (RD) bit isn't set [RFC1034] 🌿

3.1.4. Unknown DNS Opcodes

Using previously undefined opcodes is something to expect 🌈 Since DNS was first defined, two new opcodes have been added: UPDATE and NOTIFY πŸ†•

NOTIMP is the expected rcode for an unknown or unimplemented opcode πŸ™Œ

      |  NOTE: while new opcodes will most probably use the current
      |  layout structure for the rest of the message, there is no
      |  requirement that anything other than the DNS header match.
part 1 of 5 done bestie πŸŽ€ more parts are coming, so stay tuned 🫢

3.1.5. TCP Queries πŸŽ€

okay so every DNS server is supposed to answer queries over TCP [RFC7766] πŸ’… firewalls SHOULD NOT block TCP connection attempts, but if one does, it SHOULD cleanly end the connection by sending TCP RESET or sending ICMP/ICMPv6 Administratively Prohibited messages πŸ™ˆ dropping TCP connections is giving excessive delays to the whole resolution process 😭

3.2. EDNS Queries πŸ¦‹

EDNS queries are specified in [RFC6891] ✨

3.2.1. EDNS Queries: Version Independent 🌸

bestie, to find servers that ignore EDNS queries, first check that the server answers regular DNS queries πŸ‘€ then send a bunch of otherwise identical EDNS queries, then make the original query again πŸ” you need a whole series of EDNS queries because at least one DNS implementation answers the first EDNS query with FORMERR and then ignores later queries from the same address until a regular DNS query shows up πŸ’€ the EDNS query SHOULD specify a UDP buffer size of 512 bytes so you don't falsely decide it doesn't support EDNS just because the response packet was too big 🧚

if the server answers the first and last queries but ignores most or all of the EDNS ones, it's probably faulty πŸ™ƒ repeat the test a bunch of times so packet loss doesn't trick you into a false positive πŸ“

firewalls might also block bigger EDNS responses, but there's no easy way to check authoritative servers to see if the firewall is misconfigured 😀

3.2.2. EDNS Queries: Version Specific πŸ’œ

some servers answer EDNS version 0 queries just fine but ignore EDNS queries with version numbers higher than zero πŸ₯² servers SHOULD respond with BADVERS to EDNS queries with version numbers they don't support ✨

some servers handle EDNS version 0 correctly but fail to set QR=1 when answering EDNS versions they don't support 😩 those responses MAY be thrown away as invalid (since QR is not 1) or treated as requests when the source port of the original request was port 53 🫠

3.2.3. EDNS Options 🌈

some servers ignore EDNS queries that have EDNS options set ❌ the original EDNS specification left this behaviour undefined [RFC2671], but [RFC6891] clarified the correct behaviour πŸ™Œ unknown EDNS options are supposed to be ignored by the server πŸ’–

3.2.4. EDNS Flags 🌷

some servers ignore EDNS queries that have EDNS flags set πŸ™…β€β™€οΈ servers SHOULD ignore EDNS flags they don't understand and MUST NOT add them to the response [RFC6891] πŸŽ€

3.2.5. Truncated EDNS Responses 🫧

some EDNS-aware servers forget to include an OPT record when they send a truncated response 😭 an OPT record is supposed to be included in a truncated response [RFC6891] πŸ“¦

some EDNS-aware servers don't honour the advertised EDNS UDP buffer size and send oversized responses anyway πŸ’€ [RFC6891] Servers MUST send UDP responses no larger than the advertised EDNS UDP buffer size πŸ›‘

3.2.6. DO=1 Handling 🍭

some nameservers only send back an EDNS response when the DNSSEC OK (DO) bit [RFC3225] is 1 in the query, which is so not it πŸ™…β€β™€οΈ servers that support EDNS SHOULD always respond to EDNS requests with EDNS responses πŸ’…

some nameservers clearly support DNSSEC but still fail to copy the DO bit to the response, even though they return RRSIG records to EDNS queries with DO=1 πŸ‘» nameservers that support DNSSEC are expected to copy the DO bit from the request to the response 🫢

3.2.7. EDNS over TCP 🐚

some EDNS-aware servers wrongly limit TCP response sizes to the advertised UDP response size πŸ™ƒ this breaks resolution for clients whose response sizes go past the advertised UDP response size, even though the server and the client can both send and receive bigger TCP responses 😀 it basically defeats setting TC=1 in UDP responses πŸ’”

4. Firewalls and Load Balancers πŸ›‘οΈ

firewalls and load balancers can change the externally visible behaviour of a nameserver πŸ‘€ tests for conformance SHOULD be done from outside any firewall so the system gets tested as a whole 🧠

firewalls and load balancers SHOULD NOT drop DNS packets they don't understand πŸ™…β€β™€οΈ they SHOULD either pass the packets or generate an appropriate error response ✨

requests for unknown query types are totally normal client behaviour and should not be construed as an attack πŸ₯° nameservers have always been expected to handle these queries πŸ’ͺ

requests for unknown query classes are totally normal client behaviour and should not be construed as an attack πŸ₯° nameservers have always been expected to handle these queries πŸ’ͺ

requests with unknown opcodes are totally normal client behaviour and should not be construed as an attack πŸ₯° nameservers have always been expected to handle these queries πŸ’ͺ

requests with unassigned flags set (DNS or EDNS) are expected client behaviour and should not be construed as an attack 🫢 the behaviour for unassigned flags is to ignore them in the request and to not set them in the response 🎯 dropping DNS/EDNS packets with unassigned flags makes it really hard to deploy extensions that use them, since you'd have to reconfigure and update firewalls every time 😀

requests with unknown EDNS options are expected client behaviour and should not be construed as an attack 🫢 the correct behaviour for unknown EDNS options is to ignore their presence when building a reply 🧁

requests with unknown EDNS versions are expected client behaviour and should not be construed as an attack 🫢 the correct behaviour for unknown EDNS versions is to return BADVERS along with the highest EDNS version the server supports πŸ” dropping EDNS packets breaks EDNS version negotiation πŸ’”

firewalls SHOULD NOT assume there will only be a single response message to a request πŸ“¨ there have been proposals to use EDNS to signal that multiple DNS messages should be returned, instead of one UDP message that gets fragmented at the IP layer 🧩

DNS, and EDNS especially, are designed so clients can use new features with older servers without validating every option 🌟 indiscriminate blocking of messages breaks that design 🚫

however, there may be times when a nameserver mishandles messages with a particular flag, EDNS option, EDNS version field, opcode, type or class field, or some combination of them, to the point where the nameserver's integrity is compromised 😱 firewalls SHOULD offer the ability to selectively reject messages with an appropriately constructed response based on all of these fields while waiting for a fix from the nameserver vendor πŸ› οΈ FORMERR and REFUSED are two potential error codes to return πŸ™

5. Packet Scrubbing Services 🧼

packet scrubbing services filter out unwanted traffic, including but not limited to denial-of-service traffic 🚿 this is often done with heuristic analysis of the traffic πŸ”

packet scrubbing services can affect the externally visible behaviour of a nameserver in a similar way to firewalls πŸ‘€ if an operator uses a packet scrubbing service, they SHOULD check that legitimate queries aren't being blocked βœ…

unlike firewalls, packet scrubbing services are also turned on and off in response to denial-of-service attacks 🎒 you need to be careful when choosing a scrubbing service 🫣

ideally, operators SHOULD run these tests against a packet scrubbing service to make sure the tests aren't seen as attack vectors πŸ§ͺ

6. Whole Answer Caches 🧺

whole answer caches take a previously built answer and return it to a later query for the same question πŸ“¦ but they can return the wrong response if they don't take all the relevant attributes of the query into account 😬

in addition to the standard tuple <qname,qtype,qclass>, a non-exhaustive set of attributes that MUST be considered includes: RD, AD, CD, OPT record, DO, EDNS buffer size, EDNS version, EDNS options, and transport πŸ“‹βœ¨

7. Response Code Selection 🎯

picking the right response code matters a lot πŸ’… response codes SHOULD be chosen with how clients will handle them in mind 🧠

for unimplemented opcodes, NOTIMP is the expected response code πŸ™Œ note: newly implemented opcodes may change the message format by extending the header, changing the structure of the records, and so on 🧩 servers aren't expected to parse these, so they SHOULD respond with NOTIMP rather than FORMERR (which is what you'd expect for a parse error with a known opcode) πŸ’β€β™€οΈ

for unimplemented type codes, and when there are no other errors, the only valid response is NOERROR if the qname exists and NXDOMAIN otherwise 🫢 for Meta-RRs, NOTIMP may be returned instead πŸ”„

if a zone can't be loaded because it has unimplemented type codes that aren't encoded as unknown record types according to [RFC3597], the expected response is SERVFAIL, since the whole zone should be rejected (Section 5.2 of [RFC1035]) 🚫 if a zone does load, then Section 4.3.2 of [RFC1034] applies πŸ“œ

if the server supports EDNS and gets a query with an unsupported EDNS version, the correct response is BADVERS [RFC6891] πŸ”

if the server doesn't support EDNS at all, FORMERR is the expected error code πŸ₯Ί that said, a minimal EDNS server implementation needs to parse the OPT records and respond with an empty OPT record in the additional section in most cases 🧁 there's no need to interpret any EDNS options in the request, since unsupported EDNS options are expected to be ignored [RFC6891] ✨ EDNS flags can be ignored too 🫧 the only part of the OPT record that needs checking is the version field, to figure out whether BADVERS needs to be sent πŸ”

8. Testing πŸ§ͺ

testing is split into two sections: "Basic DNS", which all servers SHOULD meet, and "Extended DNS", which SHOULD be met by all servers that support EDNS (a server counts as supporting EDNS if it gives a valid EDNS response to any EDNS query) πŸ’– if a server doesn't support EDNS, it SHOULD still respond to all the tests, just with error responses πŸ™ƒ

these tests query for records at the apex of a zone the server is nominally configured to serve 🎯 all the tests should use the same zone 🫢

it's a good idea to run all of the tests below in parallel so you don't waste time on multiple timeouts when servers don't respond ⏱️ there are 16 queries sent to each nameserver (assuming no packet loss) testing different aspects of Basic DNS and Extended DNS 🌸

the tests below use dig from BIND 9.11.0 [ISC] πŸ› οΈ replace $zone with the name of the zone you're testing with πŸ“ replace $server with the name or address of the server being tested πŸ¦‹

when testing, recursive servers set RD=1 and you choose a zone name that is known to exist and isn't served by the recursive server 🧐 the root zone (".") is often a good pick, since it's DNSSEC signed πŸ” RD=1, rather than RD=0, SHOULD be present in the responses for all tests involving the opcode QUERY πŸ“¨ non-authoritative answers (AA=0) are expected when talking to a recursive server πŸ™Œ AD=1 is only expected if the server is validating responses and either AD=1 or DO=1 is set in the request, otherwise AD=0 is expected πŸ€“

8.1. Testing: Basic DNS 🌷

this first set of tests covers Basic DNS server behaviour, and all servers SHOULD pass these tests πŸ’…βœ¨

8.1.1. Is the server configured for the zone? 🌸

Okay bestie, first check: does the server even know about this zone? 🧐 We ask for the SOA record of the configured zone with no DNS flag bits set and no EDNS. Super plain, super simple.

We expect the SOA record for the zone back in the answer section, rcode set to NOERROR, and the Authoritative Answer (AA) and Query/Response (QR) bits set in the header. The Recursion Available (RA) bit may also be set [RFC1034]. We do not expect an OPT record to come back [RFC6891]. πŸ’…

dig +noedns +noad +norec soa $zone @$server

expect: status: NOERROR
expect: the SOA record to be present in the answer section
expect: flag: aa to be present
expect: flag: rd to NOT be present
expect: flag: ad to NOT be present
expect: the OPT record to NOT be present

8.1.2. Testing Unknown Types πŸ¦‹

Okay this one is giving detective energy. πŸ•΅οΈβ€β™€οΈ To find servers that choke on unknown or unsupported types, you make an initial query for an A record, then a bunch of queries for an unallocated type, then another A record query at the end. IANA keeps a registry of allocated types [IANA-DNS] so you know which ones are actually free.

If the server answers the first and last queries but ghosts the queries for the unallocated type, it's probably faulty. 😬 You should repeat the test a few times too, so packet loss doesn't trick you into a false positive. Nobody wants a fake drama πŸ™„

We ask for the TYPE1000 RRset at the configured zone's name, with no DNS flag bits set and no EDNS. TYPE1000 was picked because IANA is unlikely to allocate it anytime soon, and it isn't in a range reserved for private use [RFC6895]. Honestly, any unallocated type code would work here. πŸ“

We expect no records in the answer section, rcode set to NOERROR, and the AA and QR bits set in the header. RA may also be set [RFC1034]. We do not expect an OPT record to come back [RFC6891].

dig +noedns +noad +norec type1000 $zone @$server

expect: status: NOERROR
expect: an empty answer section.
expect: flag: aa to be present
expect: flag: rd to NOT be present
expect: flag: ad to NOT be present
expect: the OPT record to NOT be present

8.1.3. Testing Header Bits πŸŽ€

Time to poke at the header bits, and yes, it's as fun as it sounds. 🫢

8.1.3.1. Testing CD=1 Queries πŸ’–

We ask for the SOA record of the configured zone with only the CD DNS flag bit set, every other DNS bit clear, and no EDNS.

We expect the SOA record back in the answer section, rcode set to NOERROR, and the AA and QR bits set in the header. We do not expect an OPT record to come back. If the server supports DNSSEC, CD should be set in the response [RFC4035]. Otherwise, CD should be clear [RFC1034]. ✨

dig +noedns +noad +norec +cd soa $zone @$server

expect: status: NOERROR
expect: the SOA record to be present in the answer section
expect: flag: aa to be present
expect: flag: rd to NOT be present
expect: flag: ad to NOT be present
expect: the OPT record to NOT be present

8.1.3.2. Testing AD=1 Queries 🌈

Same vibe as before, but now we ask for the SOA record of the configured zone with only the AD DNS flag bit set, everything else clear, and no EDNS. πŸ¦„

We expect the SOA record back in the answer section, rcode set to NOERROR, and the AA and QR bits set in the header. We do not expect an OPT record to come back. The whole point here is to catch anything blocking queries that carry the AD bit. It's not about what value AD ends up with in the response. πŸ™ˆ

dig +noedns +norec +ad soa $zone @$server

expect: status: NOERROR
expect: the SOA record to be present in the answer section
expect: flag: aa to be present
expect: flag: rd to NOT be present
expect: the OPT record to NOT be present

AD use in queries is defined in [RFC6840]. πŸ“š

8.1.3.3. Testing Reserved Bit 🧚

Ooh, the reserved bit. πŸ‘€ We ask for the SOA record of the configured zone with only the final reserved DNS flag bit set, every other DNS bit clear, and no EDNS.

We expect the SOA record back in the answer section, rcode set to NOERROR, and the AA and QR bits set in the header. RA may be set. The final reserved bit must not be set [RFC1034]. We do not expect an OPT record to come back [RFC6891].

This check is all about making sure queries with that last unassigned header flag still work, and that the flag doesn't get copied into the response. πŸ™…β€β™€οΈ

dig +noedns +noad +norec +zflag soa $zone @$server

expect: status: NOERROR
expect: the SOA record to be present in the answer section
expect: MBZ to NOT be in the response (see below)
expect: flag: aa to be present
expect: flag: rd to NOT be present
expect: flag: ad to NOT be present
expect: the OPT record to NOT be present

MBZ (Must Be Zero) is a dig-specific indication that the flag bit has been incorrectly copied. See Section 4.1.1 of [RFC1035], which says:

"Z  Reserved for future use.  Must be zero in all queries and
   responses."

8.1.3.4. Testing Recursive Queries 🍭

Last header check, and it's a vibe. ✨ We ask for the SOA record of the configured zone with only the RD DNS flag bit set and no EDNS.

We expect the SOA record back in the answer section, rcode set to NOERROR, and the AA, QR and RD bits set in the header. RA may also be set [RFC1034]. We do not expect an OPT record to come back [RFC6891]. πŸ’œ

dig +noedns +noad +rec soa $zone @$server

expect: status: NOERROR
expect: the SOA record to be present in the answer section
expect: flag: aa to be present
expect: flag: rd to be present
expect: flag: ad to NOT be present
expect: the OPT record to NOT be present

8.1.4. Testing Unknown Opcodes 🌸

Now we're gonna send a message with an opcode nobody's using yet. πŸ™ƒ Build a DNS message that's just a header with opcode set to 15 (currently not allocated), no DNS header bits set, and empty question, answer, authority and additional sections. Nothing fancy, just vibes.

dig +noedns +noad +opcode=15 +norec +header-only @$server

expect: status: NOTIMP
expect: opcode: 15
expect: all sections to be empty
expect: flag: aa to NOT be present
expect: flag: rd to NOT be present
expect: flag: ad to NOT be present
expect: the OPT record to NOT be present

8.1.5. Testing TCP πŸ¦‹

Does the server take TCP? 🧐 Start by checking that it answers over UDP, so you know it's alive, then try the same query over TCP. If the TCP attempt fails, send an extra query over UDP to confirm the server under test is still up and running. Don't blame the server too early, bestie. 🫢

We ask for the SOA record of the configured zone with no DNS flag bits set and no EDNS. This query is to be sent using TCP.

We expect the SOA record back in the answer section, rcode set to NOERROR, and the AA and QR bits set in the header. RA may also be set [RFC1034]. We do not expect an OPT record to come back [RFC6891].

dig +noedns +noad +norec +tcp soa $zone @$server

expect: status: NOERROR
expect: the SOA record to be present in the answer section
expect: flag: aa to be present
expect: flag: rd to NOT be present
expect: flag: ad to NOT be present
expect: the OPT record to NOT be present

The requirement that TCP be supported is defined in [RFC7766]. πŸ‘‘


8.2. Testing: Extended DNS πŸ”₯

Okay, next up are the EDNS tests, and these are a whole mood. πŸ’… If any one of these passes (meaning there's at least some EDNS support), then all of them should pass too. Some servers support EDNS but fumble plain EDNS queries, so a plain EDNS query isn't a reliable sign that EDNS is missing. Don't be fooled, bestie! 😭

8.2.1. Testing Minimal EDNS ✨

We ask for the SOA record of the configured zone with no DNS flag bits set. EDNS version 0 is used, without any EDNS options or EDNS flags set.

We expect the SOA record back in the answer section, rcode set to NOERROR, and the AA and QR bits set in the header. RA may also be set [RFC1034]. We expect an OPT record to come back. There should be no EDNS flags in the response. The EDNS version field should be 0, and there should be no EDNS options present [RFC6891]. πŸŽ€

dig +nocookie +edns=0 +noad +norec soa $zone @$server

expect: status: NOERROR
expect: the SOA record to be present in the answer section
expect: an OPT record to be present in the additional section
expect: EDNS Version 0 in response
expect: flag: aa to be present
expect: flag: ad to NOT be present

+nocookie disables sending an EDNS COOKIE option, which is otherwise enabled by default in BIND 9.11.0 (and later). πŸ“

8.2.2. Testing EDNS Version Negotiation 🌈

Now we test version negotiation, which is kinda dramatic. 🎭 We ask for the SOA record of a zone the server is nominally configured to serve, with no DNS flag bits set. We use EDNS version 1, without any EDNS options or EDNS flags set.

We expect the SOA record for the zone to NOT be returned in the answer section. Instead, the extended rcode should be set to BADVERS, and the QR bit should be set in the header. RA may also be set [RFC1034]. We expect an OPT record to come back. There should be no EDNS flags in the response. The EDNS version field should be 0 in the response, since no other EDNS version has been specified yet [RFC6891]. πŸ™ƒ

dig +nocookie +edns=1 +noednsneg +noad +norec soa $zone @$server

expect: status: BADVERS
expect: the SOA record to NOT be present in the answer section
expect: an OPT record to be present in the additional section
expect: EDNS Version 0 in response
expect: flag: aa to NOT be present
expect: flag: ad to NOT be present

+noednsneg has been set, because dig supports EDNS version negotiation, and we only want to see the response to the initial EDNS version 1 query. 🧚

8.2.3. Testing Unknown EDNS Options πŸ’–

Last one for this part, and it's giving experimental. πŸ§ͺ We ask for the SOA record of the configured zone with no DNS flag bits set. We use EDNS version 0 without any EDNS flags, and include an EDNS option with a value IANA hasn't assigned yet. We picked the unassigned code 100 for the example below, but any unassigned EDNS option code could have been chosen for this test. 🌸

We expect the SOA record back in the answer section, rcode set to NOERROR, and the AA and QR bits set in the header. RA may also be set [RFC1034]. We expect an OPT record to come back. There should be no EDNS flags in the response. The EDNS version field should be 0, since EDNS versions other than 0 haven't been specified yet. There should also be no EDNS options present, because unknown EDNS options are supposed to be ignored by the server (Section 6.1.1 of [RFC6891]). πŸ‘‘

dig +nocookie +edns=0 +noad +norec +ednsopt=100 soa $zone @$server

expect: status: NOERROR
expect: the SOA record to be present in the answer section
expect: an OPT record to be present in the additional section
expect: OPT=100 to NOT be present
expect: EDNS Version 0 in response
expect: flag: aa to be present
expect: flag: ad to NOT be present

8.2.4. Testing Unknown EDNS Flags πŸŽ€

Okay bestie, we're asking for the SOA record of the configured zone πŸ’…. This query has no DNS flag bits set at all. We use EDNS version 0 with no EDNS options, and then we flip on an unassigned EDNS flag bit (0x40 in this case) πŸ¦‹.

We expect the SOA record for the zone in the answer section, rcode set to NOERROR, and the AA and QR bits set in the header. RA may also be set [RFC1034]. We expect an OPT record back. There should be no EDNS flags in the response, because unknown EDNS flags are supposed to be ignored. The EDNS version field should be 0, and there should be no EDNS options present [RFC6891] ✨.

Check that EDNS queries with unknown flags work (EDNS supported):

dig +nocookie +edns=0 +noad +norec +ednsflags=0x40 soa $zone @$server

expect: status: NOERROR
expect: the SOA record to be present in the answer section
expect: an OPT record to be present in the additional section
expect: MBZ not to be present
expect: EDNS Version 0 in response
expect: flag: aa to be present
expect: flag: ad to NOT be present

MBZ (Must Be Zero) is a dig-specific indication that a flag bit has been incorrectly copied, as per Section 6.1.4 of [RFC6891] πŸ™ˆ.

8.2.5. Testing EDNS Version Negotiation with Unknown EDNS Flags πŸ¦‹

Same vibe again: we ask for the SOA record of the configured zone with no DNS flag bits set πŸ’–. This time we use EDNS version 1 with no EDNS options, and we set an unassigned EDNS flag bit (0x40 in this case) 🌸.

We expect the SOA record to NOT be returned in the answer section. The extended rcode should be BADVERS, and the QR bit should be set in the header. RA may also be set [RFC1034]. An OPT record should come back. There should be no EDNS flags in the response, since unknown EDNS flags are supposed to be ignored. The EDNS version field should be 0, because EDNS versions other than 0 are yet to be specified, and there should be no EDNS options present [RFC6891] 🧚.

Check that EDNS version 1 queries with unknown flags work (EDNS supported):

dig +nocookie +edns=1 +noednsneg +noad +norec +ednsflags=0x40 soa \
    $zone @$server

expect: status: BADVERS
expect: SOA record to NOT be present
expect: an OPT record to be present in the additional section
expect: MBZ not to be present
expect: EDNS Version 0 in response
expect: flag: aa to NOT be present
expect: flag: ad to NOT be present

8.2.6. Testing EDNS Version Negotiation with Unknown EDNS Options 🌈

Ask for the SOA record of the configured zone, with no DNS flag bits set πŸ“. EDNS version 1 is used, and an unknown EDNS option is present. We picked an unassigned code of 100 for the example below, but any unassigned EDNS option code could have been chosen for this test πŸ‘‘.

We expect the SOA record to NOT be returned in the answer section, with the extended rcode set to BADVERS and the QR bit set in the header. RA may also be set [RFC1034]. An OPT record should be returned. There should be no EDNS flags in the response. The EDNS version field should be 0, since EDNS versions other than 0 are yet to be specified, and there should be no EDNS options present [RFC6891] πŸ’….

Check that EDNS version 1 queries with unknown options work (EDNS supported):

dig +nocookie +edns=1 +noednsneg +noad +norec +ednsopt=100 soa \
    $zone @$server

expect: status: BADVERS
expect: SOA record to NOT be present
expect: an OPT record to be present in the additional section
expect: OPT=100 to NOT be present
expect: EDNS Version 0 in response
expect: flag: aa to NOT be present
expect: flag: ad to NOT be present

8.2.7. Testing Truncated Responses πŸ¦‹

Okay, this one is a little dramatic 😭. We ask for the DNSKEY records of the configured zone, which must be a DNSSEC signed zone. The query has no DNS flag bits set, EDNS version 0 with no EDNS options, and the only EDNS flag set is DO. The EDNS UDP buffer size is set to 512. The whole point is to try to get a truncated response out of the server, because most signed DNSKEY responses are bigger than 512 bytes 🫢. This test will not give a valid result if the zone is not signed.

We expect a response with the rcode set to NOERROR, and the AA and QR bits set. AD may be set if the server supports DNSSEC, otherwise it should be clear. TC and RA may also be set [RFC1035] [RFC4035]. An OPT record should be present in the response, and there should be no EDNS flags other than DO. The EDNS version field should be 0, and there should be no EDNS options present [RFC6891] ✨.

If TC is not set, it is not possible to confirm that the server correctly adds the OPT record to the truncated responses or not. πŸ™ˆ
dig +norec +dnssec +bufsize=512 +ignore dnskey $zone @$server
expect: NOERROR
expect: OPT record with version set to 0

8.2.8. Testing DO=1 Handling πŸ’œ

Ask for the SOA record of the configured zone, which does not need to be DNSSEC signed 🌸. The query has no DNS flag bits set, EDNS version 0 with no EDNS options, and the only EDNS flag set is DO.

We expect the SOA record in the answer section, rcode set to NOERROR, and the AA and QR bits set in the response. AD may be set if the server supports DNSSEC, otherwise it should be clear. RA may also be set [RFC1034]. An OPT record should be returned. There should be no EDNS flags other than DO in the response, and DO should be present if the server supports DNSSEC. The EDNS version field should be 0, and there should be no EDNS options present [RFC6891] πŸ”₯.

Check that DO=1 queries work (EDNS supported):

dig +nocookie +edns=0 +noad +norec +dnssec soa $zone @$server

expect: status: NOERROR
expect: the SOA record to be present in the answer section
expect: an OPT record to be present in the additional section
expect: DO=1 to be present if an RRSIG is in the response
expect: EDNS Version 0 in response
expect: flag: aa to be present

8.2.9. Testing EDNS Version Negotiation with DO=1 🧚

Same SOA record, same zone, which again does not need to be DNSSEC signed πŸ’–. This query has no DNS flag bits set, and it uses EDNS version 1 with no EDNS options. The only EDNS flag set is DO.

We expect the SOA record for the zone NOT to be returned in the answer section. The extended rcode should be BADVERS, and the QR bit should be set in the header. RA may also be set [RFC1034]. An OPT record should be returned. There should be no EDNS flags other than DO in the response, which should be there if the server supports DNSSEC. The EDNS version field should be 0, and there should be no EDNS options present [RFC6891] πŸ‘‘.

Check that EDNS version 1, DO=1 queries work (EDNS supported):

dig +nocookie +edns=1 +noednsneg +noad +norec +dnssec soa \
    $zone @$server

expect: status: BADVERS
expect: SOA record to NOT be present
expect: an OPT record to be present in the additional section
expect: DO=1 to be present if the EDNS version 0 DNSSEC query test
        returned DO=1
expect: EDNS Version 0 in response
expect: flag: aa to NOT be present

8.2.10. Testing with Multiple Defined EDNS Options 🌈

Ask for the SOA record of the configured zone with no DNS flag bits set πŸ“. EDNS version 0 is used, and a bunch of defined EDNS options are present: NSID [RFC5001], DNS COOKIE [RFC7873], EDNS Client Subnet [RFC7871], and EDNS Expire [RFC7314] πŸ¦‹.

We expect the SOA record in the answer section, rcode set to NOERROR, and the AA and QR bits set in the header. RA may also be set [RFC1034]. An OPT record should be returned. There should be no EDNS flags in the response, and the EDNS version field should be 0. Any of the requested EDNS options that the server supports and its permitted configuration allows may be returned [RFC6891] πŸ’….

Check that EDNS queries with multiple defined EDNS options work:

dig +edns=0 +noad +norec +cookie +nsid +expire +subnet=0.0.0.0/0 \
    soa $zone @$server

expect: status: NOERROR
expect: the SOA record to be present in the answer section
expect: an OPT record to be present in the additional section
expect: EDNS Version 0 in response
expect: flag: aa to be present
expect: flag: ad to NOT be present

8.3. When EDNS Is Not Supported πŸ™ˆ

If EDNS is not supported by the nameserver, we expect a response to each of the above queries. That response may be a FORMERR error response, or the OPT record may just be ignored 🫢.

Some nameservers only return an EDNS response when a particular EDNS option or flag (e.g., DO=1) is present in the request. This behaviour is not compliant behaviour, and it may hide other incorrect behaviour from the above tests. Retesting with the triggering option/flag present will expose this misbehaviour 😭πŸ”₯.

9. Remediation ✨

Okay bestie, so nameserver operators are basically expected to test their own infrastructure for compliance with the standards πŸ’…. The tests above should be run when new systems go live, and then repeated periodically so everything keeps interoperating smoothly πŸ”.

Domain registrants who don't maintain their own DNS infrastructure are entitled to a DNS service that conforms to standards and interoperates well 🫢. If a registrant realises their DNS operator doesn't have a well-maintained or compliant infrastructure, they should insist that the provider fix the issues, and switch providers if the provider won't 😀πŸ”₯.

If an operator runs into problems because of how nameservers outside their control behave, the tests above can help narrow down the precise issue(s), and then the problem can be reported to the relevant party πŸ”πŸ§š.

If you don't already know the contact info for the operator of a misbehaving nameserver, these are the methods of communication that could be considered:

  • the RNAME of the zone authoritative for the name of the misbehaving server
  • the RNAME of zones for which the offending server is authoritative
  • administrative or technical contacts listed in the registration information for the parent domain of the name of the misbehaving server, or for zones for which the nameserver is authoritative
  • the registrar or registry for such zones
  • DNS-specific, operational fora (e.g., mailing lists)

Parent zone operators may want to regularly test the authoritative nameservers of their child zones 🌸. Still, parent operators can have wildly different abilities when it comes to notification or remediation, depending on whether they have a direct relationship with the child operator πŸ₯². Lots of Top-Level Domain (TLD) registries, for example, can't directly contact their registrants, so they may need to communicate through the relevant registrar πŸ“¬. In those cases, it may be most efficient for registrars to take on the job of testing their registrants' nameservers, since they have the direct relationship πŸ’œ.

When notification doesn't fix a misbehaving nameserver, parent operators can choose to remove the NS record sets (and glue records below) that refer to the faulty server, until the servers are fixed πŸ™ˆ. This should only be done as a last resort, and with due consideration, because removing a delegation can have unanticipated side effects πŸ’€. For example, other parts of the DNS tree may depend on names below the removed zone cut, and the parent operator may end up being responsible for causing new DNS failures to occur 😭.

Removing a delegation is a big deal, babe. Other parts of the DNS tree might be quietly relying on it, so think it through before you pull the trigger ⚠️.

10. Security Considerations 🚨

Testing protocol compliance can potentially result in false reports of attempts to attack services, coming from Intrusion Detection Services and firewalls πŸ‘€. All of the tests are well-formed (though not necessarily common) DNS queries πŸŽ€. None of the tests listed above should cause any harm to a protocol-compliant server πŸ›‘οΈ.

Relaxing firewall settings to ensure EDNS compliance could potentially expose a critical implementation flaw in the nameserver 😱πŸ”₯. Nameservers should be tested for conformance before relaxing firewall settings, no exceptions 🫠.

When removing delegations for non-compliant servers, there can be a knock-on effect on other zones that need these zones to be operational in order for the nameserver addresses to be resolved πŸŒͺοΈπŸ’”. Think of it like a domino situation, and not the cute kind πŸ™ƒ.


11. IANA Considerations πŸ¦‹

This document has no IANA actions. Nothing to register, nothing to update, just vibes ✨.


12. References πŸ“š

12.1. Normative References

12.2. Informative References

Acknowledgements 🫢

The contributions of Matthew Pounsett and Tim Wicinski are gratefully acknowledged. Huge thanks to them, they're absolute legends πŸ‘‘πŸ’–.

Authors' Addresses πŸ’Œ

M. Andrews
Internet Systems Consortium
PO Box 360
Newmarket, NH 03857
United States of America

Email: marka@isc.org


Ray Bellis
Internet Systems Consortium
PO Box 360
Newmarket, NH 03857
United States of America

Email: ray@isc.org
πŸ“¦ where this page comes from