RFC 5782

DNS Blacklists and Whitelists

📅 published February 2010, from ietf

DNS Blacklists and Whitelists 🦋

Okay bestie, the title block is giving RFC 5782 💅. It's DNS Blacklists and Whitelists by J. Levine of Taughannock Networks, published in February 2010. It's Informational, not a Standards Track thing, and it comes from the Internet Research Task Force (IRTF). The ISSN is 2070-1721 ✨. The title block doesn't list anything it obsoletes or updates, so there's no drama there 🌸.

Abstract ✨

Spam and other anti-social behavior on the Internet made people start sharing blacklists and whitelists of IP addresses or domains 🔥. The DNS became the de-facto standard way to distribute them, and this memo documents how DNS-based blacklists and whitelists are structured and used, plus the protocol for querying them 💜.

Status of This Memo 📜

This document is not an Internet Standards Track specification. It's published for informational purposes 🎀.

It's a product of the Internet Research Task Force (IRTF), which publishes research and development results. Those results might not be suitable for deployment 🧚. This RFC represents the consensus of the Anti-Spam Research Group of the IRTF. Documents approved for publication by the IRSG are not a candidate for any level of Internet Standard, so check Section 2 of RFC 5741 if you want the receipts 🫶.

If you want the current status, any errata, or a way to send feedback, go to http://www.rfc-editor.org/info/rfc5782 🌈.

Copyright (c) 2010 IETF Trust and the persons identified as the document authors. All rights reserved 💖.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents, in effect on the date of publication. The legal stuff lives at http://trustee.ietf.org/license-info. Please read these carefully, because they explain your rights and restrictions for this document 👑.

Table of Contents 🗂️

1. Introduction ....................................................2
2. Structure of an IP Address DNSBL or DNSWL .......................3
   2.1. IP Address DNSxL ...........................................3
   2.2. IP Address DNSWL ...........................................4
   2.3. Combined IP Address DNSxL ..................................4
   2.4. IPv6 DNSxLs ................................................5
3. Domain Name DNSxLs ..............................................6
4. DNSxL Cache Behavior ............................................7
5. Test and Contact Addresses ......................................7
6. Typical Usage of DNSBLs and DNSWLs ..............................8
7. Security Considerations .........................................9
8. References .....................................................10
   8.1. Normative References ......................................10
   8.2. Informative References ....................................10

1. Introduction ✨

In 1997, Dave Rand and Paul Vixie, two well-known Internet software engineers, started keeping a list of IP addresses that had sent them spam or done other stuff they found objectionable 🙈. Word spread fast, and they began distributing the list as a BGP feed so people could block all traffic from listed IP addresses at their routers. The list became known as the Real-time Blackhole List (RBL) 🖤.

Lots of network managers wanted to use the RBL to block unwanted e-mail, but they weren't ready to deal with a BGP feed 😭. Rand and Vixie made a DNS-based distribution scheme that quickly got way more popular than the original BGP version. Other people made their own DNS-based blacklists, either to compete with the RBL or to complement it by listing different categories of IP addresses 💅. Some people call every DNS-based blacklist an "RBL," but the term properly belongs to the Mail Abuse Prevention System (MAPS) RBL, the descendant of the original list. (In the United States, RBL is a registered service mark of Trend Micro [MAPSRBL].) 🍓

The conventional term now is DNS blacklist or blocklist, or DNSBL. Some people also publish DNS-based whitelists, or DNSWLs 🌸. Network managers usually use DNSBLs to block traffic and DNSWLs to preferentially accept traffic. A DNSBL and a DNSWL have the same structure, so from here on we use the abbreviation DNSxL to mean either one 🦋.

This document defines the structure of DNSBLs and DNSWLs. It covers how they're structured, how they operate, and how they're used. It does not describe or recommend policies for adding or removing addresses to or from DNSBLs and DNSWLs, and it doesn't recommend policies for using them either 🧚. The authors expect management policies to be covered in a companion document.

This document is a product of the Anti-Spam Research Group (ASRG) of the Internet Research Task Force. It represents the ASRG's consensus on practices that improve interoperability of DNS-based blacklists and whitelists 🫶.

Requirements Notation: The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119], with respect to recommendations for improving technical interoperability of DNSxLs. 💜

2. Structure of an IP Address DNSBL or DNSWL 🧱

A DNSxL is a zone in the DNS [RFC1034] [RFC1035]. The zone's resource records identify hosts that are on a blacklist or whitelist. Hosts were originally encoded into DNSxL zones by transforming their IP addresses, but now host names are sometimes encoded too. Most DNSxLs still use IP addresses 🌈.

2.1. IP Address DNSxL 💖

An IPv4 address DNSxL has a structure adapted from rDNS. (Reverse DNS, or rDNS, is the IN-ADDR.ARPA [RFC1034] and IP6.ARPA [RFC3596] domains used to map IP addresses to domain names 🔍.) Each IPv4 address listed in the DNSxL has a matching DNS entry. The entry's name is made by reversing the order of the octets in the text form of the IP address, then appending the DNSxL's domain name ✨.

Say the DNSxL is called bad.example.com and the IPv4 address to list is 192.0.2.99. The DNS entry name would be 99.2.0.192.bad.example.com. Each entry in the DNSxL MUST have an A record. DNSBLs SHOULD have a TXT record that describes the reason for the entry. DNSWLs MAY have a TXT record that describes the reason for the entry. The contents of the A record MUST NOT be used as an IP address. The A record contents conventionally have the value 127.0.0.2, but they MAY have other values as described below in Section 2.3. The TXT record explains why the IP address is listed in the DNSxL. It's often used as the text of an SMTP error response when an SMTP client tries to send mail to a server that uses the list as a DNSBL, or as explanatory text when the DNSBL is used in a scoring spam filter 🎀. The DNS records for this entry might be:

99.2.0.192.bad.example.com    A      127.0.0.2
99.2.0.192.bad.example.com    TXT
         "Dynamic address, see http://bad.example.com?192.0.2.99"

Some DNSxLs use the same TXT record for every entry. Others give each entry, or each range of entries, its own TXT record explaining why it's listed. That reason often includes the URL of a web page with more info 🌸. Client software MUST check the A record and MAY check the TXT record.

If a range of addresses is listed in the DNSxL, the DNSxL MUST contain an A record (or a pair of A and TXT records) for every address in the DNSxL. On the flip side, if an IP address is not listed in the DNSxL, there MUST NOT be any records for that address 🙅‍♀️.

2.2. IP Address DNSWL 🦋

SMTP has no way for a server to tell a client why a request was accepted, so TXT records in DNSWLs aren't super useful 🤷‍♀️. Some DNSWLs include TXT records anyway to document why entries are there.

It's possible, and occasionally handy, for one DNSxL to be used as a DNSBL in one context and as a DNSWL in another. For example, a DNSxL that lists the dynamically assigned addresses on a particular network could be used as a DNSWL on that network's outgoing mail server or intranet web server, and as a DNSBL for mail servers on other networks 🧚.

2.3. Combined IP Address DNSxL 🧩

Plenty of organizations keep a DNSxL with multiple entry types, where the entries of each type form a sublist. Say an organization publishes a DNSBL of sources of unwanted e-mail and wants to show why each address is included. It could have one sublist for addresses listed because of sender policy, a second for open relays, a third for hosts compromised by malware, and so on 💅. (So far, every DNSxL with sublists that we know of is meant for use as a DNSBL, but sublist techniques work just as well for DNSWLs.)

There are three common ways to represent a DNSxL with multiple sublists: subdomains, multiple A records, and bit-encoded entries. DNSxLs with sublists SHOULD use subdomains plus one of the other methods 🌈.

Sublist subdomains are just subdomains of the main DNSxL domain. For example, if bad.example.com had two sublists, 'relay' and 'malware', the entries for 192.0.2.99 would be 99.2.0.192.relay.bad.example.com or 99.2.0.192.malware.bad.example.com. If a DNSxL has entries for both the main domain and its sublists, sublist names MUST be at least two characters and contain non-digits. That way there's no name collision with entries in the main domain, where the names are made of digits or single hex characters 🔥.

To cut down the number of DNS lookups, multiple sublists can also be encoded as bit masks or as multiple A records. With bit masks, the A record for each IP address is the logical OR of the bit masks for all the lists that address appears on. For example, the bit masks for the two sublists might be 127.0.0.2 and 127.0.0.4. An IP address on both lists would then get 127.0.0.6 💖:

99.2.0.192.bad.example.com    A      127.0.0.6

With multiple A records, each sublist gets its own assigned value, such as 127.0.1.1, 127.0.1.2, and so on. The IP address gets one A record for each sublist it appears on 🍓:

99.2.0.192.bad.example.com    A      127.0.1.1
99.2.0.192.bad.example.com    A      127.0.1.2

There's no widely used convention for mapping sublist names to bits or values. The one convention everyone sort of follows is that all A values SHOULD be in the 127.0.0.0/8 range. That way, if a value gets mistakenly used as an IP address, it won't send unwanted network traffic 🙈.

DNSxLs that return multiple A records sometimes return multiple TXT records too. But since there's no way to match TXT records to A records, those TXT records aren't very useful. Other combined DNSxLs return a single TXT record 🌸.

2.4. IPv6 DNSxLs 🌐

okay so IPv6 DNSxLs are basically the same vibe as IPv4 ones, just adapted from the IP6.ARPA domain in [RFC3596] 🧚 Each entry's name MUST be a 32-component hex nibble-reversed IPv6 address with the DNSxL domain tacked on the end. The entries hold A and TXT records, and you read them the same way as IPv4 DNSxLs 💅

here's an example. say we want to represent the address:

2001:db8:1:2:3:4:567:89ab

in the DNSxL ugly.example.com. the entry might look like this:

b.a.9.8.7.6.5.0.4.0.0.0.3.0.0.0.2.0.0.0.1.0.0.0.8.b.d.0.1.0.0.2.
             ugly.example.com. A 127.0.0.2
                               TXT "Spam received."

combined IPv6 sublist DNSxLs work the same way as IPv4 ones, except you swap the four IPv4 octets for the 32 nibbles of the IPv6 address 🫶

ngl this next bit is lowkey so iconic 🔥 a single DNSxL could technically hold both IPv4 and IPv6 addresses, because the different lengths mean there's no confusion. BUT if the DNSxL is served from traditional zone files with wildcards, there's no way to say how long the name is that a wildcard matches. so wildcard names would actually be ambiguous for DNSxLs served that way 🙈

3. Domain Name DNSxLs 🌸

a few DNSxLs list domain names instead of IP addresses. these are sometimes called RHSBLs, which stands for right-hand-side blacklists 🍓 The names of their entries MUST contain the listed domain name followed by the name of the DNSxL. The entries hold A and TXT records, and they're interpreted the same way as in IPv4 DNSxLs.

say the DNSxL is called doms.example.net and the domain invalid.edu gets listed. the entry would be named invalid.edu.doms.example.net, like this:

invalid.edu.doms.example.net    A      127.0.0.2
invalid.edu.doms.example.net    TXT    "Host name used in phish"

name-based DNSBLs are way less common than IP address based ones 👀 and there's no agreed convention for wildcards yet, so nobody's really settled on that one.

name-based DNSWLs can be built the same way as DNSBLs, and people have used them as simple reputation systems. the octet values in the A record stand for reputation scores and confidence values, usually on a 0-100 or 0-255 scale 💜 Vouch By Reference [RFC5518] is a certification system that's similar in design and operation to a name-based DNSWL ✨

4. DNSxL Cache Behavior ⏰

the per-record time-to-live and zone refresh intervals of DNSBLs and DNSWLs vary a LOT depending on how the list is managed 💅 The Time to Live (TTL) and refresh times SHOULD be chosen to reflect how fast the DNSxL is expected to change.

for example, a list of IP addresses assigned to dynamically allocated dialup and DHCP users could be expected to change slowly. so the TTL might be several days, and the zone might get refreshed once a day 😴 on the other hand, a list of IP addresses that have been seen sending spam might change every few minutes. so those would have comparably short TTL and refresh intervals ⚡

5. Test and Contact Addresses 🧪

okay this section is giving "check your work" energy 💅

IPv4-based DNSxLs MUST contain an entry for 127.0.0.2 for testing purposes. IPv4-based DNSxLs MUST NOT contain an entry for 127.0.0.1 🎀

DNSBLs that return multiple values SHOULD have multiple test addresses. so, for example, a DNSBL that can return 127.0.0.5 would have a test record for 127.0.0.5. that test record returns an A record with the value 127.0.0.5, plus a matching TXT record 🦋

IPv6-based DNSxLs MUST contain an entry for ::FFFF:7F00:2 (::FFFF: 127.0.0.2). They MUST NOT contain an entry for ::FFFF:7F00:1 (::FFFF: 127.0.0.1). These are the IPv4-Mapped IPv6 Address [RFC4291] equivalents of the IPv4 test addresses 🧚

domain-name-based DNSxLs MUST contain an entry for the reserved domain name "TEST" from [RFC2606]. They MUST NOT contain an entry for the reserved domain name "INVALID" 🌈

DNSxLs also MAY contain A and/or AAAA records at the apex of the DNSxL zone that point to a web server. that way, anyone who wants to learn about the bad.example.net DNSBL can check http://bad.example.net 🍓

the combo of a test address that MUST exist and an address that MUST NOT exist lets a client system check that a domain still has DNSxL data in it 🙌 It also protects against DNSxLs that install a wildcard returning an A record for every query, whether they do it on purpose or by accident 👀 DNSxL clients SHOULD periodically check the appropriate test entries to make sure the DNSxLs they're using are still operating 💖

6. Typical Usage of DNSBLs and DNSWLs 📬

DNSxLs can be served from standard DNS servers, or from specialized servers like rbldns [RBLDNS] and rbldnsd [RBLDNSD]. those specialized ones accept lists of IP addresses and Classless Inter-Domain Routing (CIDR) ranges, then make the right DNS records on the fly ✨

organizations that lean heavily on a DNSxL usually set up a private mirror of it. they can do that with the standard Full Zone Transfer (AXFR) and Incremental Zone Transfer (IXFR), or by fetching a file with the addresses and CIDR ranges for the specialized servers 🫶 if a /24 or larger range of addresses is listed, and the zone's server uses traditional zone files, the DNSxL MAY use wildcards to keep the zone file small 🧁 for example, if the whole range 192.0.2.0/24 were listed, the DNSxL's zone could hold one wildcard for *.2.0.192.bad.example.com.

DNSBL clients are most often mail servers, or spam filters that get called from mail servers 📧 there's no rule saying DNSBLs can only be used for mail though. other services, like Internet Relay Chat (IRC), use them to check the client hosts trying to connect to a server 🌸

a client MUST interpret any returned A record as meaning that an address or domain is listed in a DNSxL 🚨 mail servers that test combined lists usually handle them the same as single lists. they treat any A record as meaning the IP address is listed, without figuring out which of the many reasons it might have been listed 🙈 DNSxL clients SHOULD be able to use bit masks and value range tests on returned A record values, so they can pick out particular sublists of a combined list 🔍

mail servers usually check a list of DNSxLs on every incoming SMTP connection, with the DNSxL names set in the server's configuration 📮 a common pattern is to check each list in turn until it finds a DNSBL entry (then it rejects the connection) or a DNSWL entry (then it accepts the connection). if the address isn't on any list at all, which is the usual case for legit mail, the mail server accepts the connection 🥰 another approach uses DNSxL entries as inputs to a weighting function, which works out an overall score for each message 🧮

the mail server uses its normal local DNS cache to limit traffic to the DNSxL servers, and to speed up rechecks of IP addresses it's seen recently ⚡ long-running mail servers MAY cache DNSxL data internally, but they MUST respect the TTL values and discard expired records ⏳

there's also an alternate approach, where DNSxLs get checked in a spam filtering package after a message has arrived. in that case, the IP(s) to test are usually pulled from "Received:" header fields or from URIs in the message body 📨 the DNSxL results can make a simple accept/reject call, or feed into a scoring system 💯

packages that test multiple header fields MUST be able to tell apart the values in lists that have sublists 🧠 for example, an entry saying an IP address is assigned to dialup users might be a strong sign that a message should be rejected if that IP address sends mail straight to the recipient system. but that same entry shouldn't cause a rejection if the message was relayed through an ISP's mail server 🙃

name-based DNSBLs have been used to check the domain names of e-mail addresses and the host names found in mail headers. they've also been used to check the domains found in URLs in message bodies 🔗

7. Security Considerations 🔐

okay this one is SERIOUS bestie, so read it like it's a plot twist 😭

any system manager who uses DNSxLs is trusting the people who run the lists with part of their server management. they SHOULD make sure the list's management policies match the policies the system manager actually wants to use 🧐 a badly chosen DNSBL might block addresses that send mail the system manager and its users really want to receive 💀 DNSBL management can also change over time. in some cases, when a DNSBL operator wanted to shut it down, they either removed every entry, or installed a wildcard that lists 0/0. that causes unexpected and unwanted results for anyone using the DNSBL 🚨

the A records in a DNSxL zone (other than the ones at the apex of the zone) stand for blacklist and/or whitelist entries, NOT IP addresses 🙅‍♀️ if a client tries to use those A records as IP addresses, for example by trying to use a DNSxL entry name as a web or FTP server, you'll get peculiar results 🫠 if the DNSxL operator ignored the advice in Section 2.3 and put values in the A records outside the 127/8 range, those peculiar results might not stay limited to the host that misused the records 😱 and the reverse is also true: if a system tries to use a zone that isn't a DNSxL as a blacklist or whitelist, even MORE peculiar results will happen 🌀 this has actually been seen in the wild, when an abandoned DNSBL domain got re-registered and the new owner installed a wildcard with an A record pointing to a web server 🙈 to avoid this kind of mess, systems that use DNSxLs SHOULD check for the test entries described in Section 5. that confirms a domain actually has the structure of a DNSxL. they SHOULD NOT use any DNSxL domain that doesn't have correct test entries 🛑

since DNSxL users usually make a query for every incoming e-mail message, the DNSxL operator can pull approximate mail volume statistics out of the DNS server logs 📊 this has been used a few times to estimate how much mail individual IP addresses or IP blocks send [SENDERBASE] [KSN] 👀

like any other DNS-based service, DNSBLs and DNSWLs are open to various kinds of DNS attacks, which are described in [RFC3833] ⚠️

8. References 📚

8.1. Normative References

  • [RFC1034] Mockapetris, P., "Domain names - concepts and facilities", STD 13, RFC 1034, November 1987.
  • [RFC1035] Mockapetris, P., "Domain names - implementation and specification", STD 13, RFC 1035, November 1987.
  • [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
  • [RFC2606] Eastlake, D. and A. Panitz, "Reserved Top Level DNS Names", BCP 32, RFC 2606, June 1999.
  • [RFC3596] Thomson, S., Huitema, C., Ksinant, V., and M. Souissi, "DNS Extensions to Support IP Version 6", RFC 3596, October 2003.
  • [RFC4291] Hinden, R. and S. Deering, "IP Version 6 Addressing Architecture", RFC 4291, February 2006.
  • [RFC5518] Hoffman, P., Levine, J., and A. Hathcock, "Vouch By Reference", RFC 5518, April 2009.

8.2. Informative References

Author's Address 💌

okay so this is where we find out who wrote the whole thing 🫶 John Levine, from Taughannock Networks, bestie. his mailing address is right here, along with his phone, email, and website 🌸

John Levine
Taughannock Networks
PO Box 727
Trumansburg, NY  14886

Phone: +1 607 330 5711
EMail: standards@taugh.com
URI:   http://www.taugh.com
📦 where this page comes from