dnsdbq(1)

📦
version 2.6.8 · 3 versions
, in debian trixie, nixos 26.05, nixpkgs unstable, ubuntu questing, ubuntu resolute, ubuntu stonking

💖 dnsdbq — DNSDB query tool 💖

okay bestie, gather round, because this is dnsdbq(1) from the General Commands Manual and it's genuinely iconic. it's a whole command-line tool that builds and fires off queries at Passive DNS systems, and it's the one everybody actually uses in production. no cap. 🔥

✨ NAME ✨

dnsdbq — the DNSDB query tool. that's it, that's the name, short and slay.

🌸 SYNOPSIS 🌸

dnsdbq [-acdfgGhIjmqSsUv468] [-A timestamp] [-B timestamp]
       [-b bailiwick] [-D asn_domain] [-i ip] [-J input_file]
       [-k sort_keys] [-L output_limit] [-l query_limit] [-M max_count]
       [-C cookie_file] [-N hex[/rrtype[,...]]] [-n name[/rrtype[,...]]]
       [-O offset] [-o timeout] [-p output_type]
       [-R hex[/rrtype[,...][/bailiwick]]]
       [-r name[/rrtype[,...][/bailiwick]]] [-T transform[,...]]
       [-t rrtype[,...]] [-u server_sys] [-V verb] [-0 function=thing]

like yes, that is a LOT of brackets. she contains multitudes. 💅

🦋 DESCRIPTION 🦋

so here's the tea: dnsdbq builds queries and sends them off to Passive DNS systems that hand back data in the IETF Passive DNS Common Output Format. and bestie, it's commonly used as an actual production command line interface to those systems. main character energy fr fr. 👑

it can show you responses in a bunch of different formats, and its default query type is a "lookup" query. if you want, it can also do a "summarize" query type instead. heads up though — different Passive DNS systems, or even different versions of the same system, might implement different query features. so it's giving "your mileage may vary" but make it cute. 🌈

🌟 FARSIGHT SECURITY'S DNSDB 🌟

Farsight Security's "DNSDB" is one of those Passive DNS systems, and you get to it by specifying the system dnsdb.

you're gonna need to get an API key from Farsight to use dnsdbq with DNSDB. no key, no queries, bestie. 🔑

Farsight's passive DNS infrastructure does this whole complex process called "bailiwick reconstruction", where an RRset's position inside the DNS hierarchy gets approximated. and it's doing that for two reasons:

  1. to give you context about where a given DNS record sits within the DNS hierarchy.
  2. to stop "untrustworthy" records — the ones that come from intentional or unintentional cache poisoning attempts — from getting replicated by downstream consumers. like, blocked. not today. 💀

for example, say you've got the fully qualified domain name www.dachshund.example.com. valid bailiwicks would be dachshund.example.com, example.com, or com. see how it's walking up the tree? iconic behavior honestly. 🐕

✨ OPTIONS ✨

okay this is the big one, buckle up. every single one, no skips:

  • -a — turns on ASINFO/CIDR annotation for IP addresses in A (IPv4 address) RRsets. the metadata that gets appended depends on which data source you picked with -D.
  • -A timestamp — sets a forward time fence. only results seen by the passive DNS on or after this time get selected. also peep -c. and go read the TIMESTAMP FORMATS section for the whole scoop on this one.
  • -B timestamp — sets a backward time fence. only results seen by the passive DNS sensor network on or before this time get selected. also peep -c. and yeah, TIMESTAMP FORMATS section, go look.
  • -b bailiwick — specify the bailiwick. heads up, this one's only valid with -r queries.
  • -c — okay so by default, -A and -B (separately or together) will select partial overlaps of database tuples and time search criteria. if you want to match only completely bracketed tuples, add the -c flag, aka "completeness" mode, aka "strict" mode. can only be specified once, and for reasons of geometry it affects both -A and -B if you gave both. math said so. 🧚
  • -C cookie_file — specify a cookie file for libcurl to use (read-only; analogous to curl(1) -b). typically you'd use this for service authentication. 🍪
  • -D asn_domain — specify the data source for ASINFO/CIDR annotations, if you enabled them with -a. default is asn.routeviews.org, but you might wanna try aspath.routeviews.org.
  • -d — enable debug mode. repeat it for more debug output. more -d, more words. 🔍
  • -f — specify batch lookup mode, which lets one or more queries run. queries get read from standard input and are expected to be in one of these formats:
    • RRset (name) query: rrset/name/NAME[/RRTYPE[,...][/BAILIWICK]]
    • RRset (raw) query: rrset/raw/HEX[/RRTYPE[,...][/BAILIWICK]]
    • Rdata (name) query: rdata/name/NAME[/RRTYPE][,...]
    • Rdata (IP address) query: rdata/ip/ADDR[,PFXLEN]
    • Rdata (raw) query: rdata/raw/HEX[/RRTYPE][,...]
    • Change query options: $OPTIONS {options}

Where options :==

[-A timestamp] [-B timestamp] [-c] [-g] [-G]
[-l query_limit] [-L output_limit] [-O offset]

$OPTIONS alone on a line lets you change command line options mid-batch. and if no options are given, the query parameters get reset to whatever was given on the command line, if any, or else to defaults. clean slate energy. ✨

a line starting with a # is ignored as a comment. ghosted. 🙈

any internal slash (/) or comma (,) characters inside the search names of a batch entry must be URL-encoded (for example, %2F or %2C). so to search for the domain 212.0/24.150.104.24.in-addr.arpa, the search string would be specified as 212.0%2F150.104.24.in-addr.arpa.

for raw queries, the hex value is an even number of hexadecimal digits specifying a raw octet string. the "raw" wire-format encodings are standardized, and the embedding of these in dnstable is documented in the dnstable-encoding(5) manual page. this whole topic is explained in detail at <https://www.farsightsecurity.com/blog/txt-record/dnsdb-rawhex-20161125>.

in batch lookup mode, each answer will be followed by a -- marker, so programmatic users know when it's safe to send the next lookup — or, if lookups are pipelined, so they know when one answer has ended and another begun. this option cannot be mixed with -n, -r, -R, or -i. check the EXAMPLES section for more on how to use -f.

if two -f options are given, then each answer will also be preceded by a ++ marker giving the query string (as read from the batch input), so you can identify each answer when you've got a very large batch input. and the -- marker will include an error/noerror indicator plus a short message describing the outcome. with two -f options and also -m, answers can appear in a different order than the batched questions, and the

  • -g — return graveled results if available. default is to return aggregated results ("rocks"). gravel is a feature for providing Volume Across Time. note that not all Passive DNS system APIs support this feature, and not all time ranges contain granular results ("gravel").
  • -G — undoes the effect of -g, so you get rocks back instead of gravel. (this one's for use in $OPTIONS inside batch files.)
  • -h — emit usage and quit. bye bestie 👋
  • -I — request info from the API server about the API key itself, which may include rate limit, query quota, query allowance, or privilege levels. the output format and content depend on the server_sys argument (see -u) and on the -p argument. -I -p json prints the raw info. -I -p text prints it in a more understandable textual form, including converting any epoch integer times into UTC formatted times. 🕰️
  • -i ip — specify an rdata ip ("right-hand side") query. the value is one of: an IPv4 address, an IPv6 address, an IPv4 network with prefix length, an IPv4 address range, or an IPv6 network with prefix length. if you're doing a network lookup, the delimiter between the network address and the prefix length is a single comma (,), not the usual slash (/), so it doesn't clash with the HTTP URI path name separator. see the EXAMPLES section for more on separator substitution rules.
  • -J input_file — opens input_file and reads newline-separated JSON objects out of it, in preference to -f (batch mode) or query mode. this is how you reprocess the output from a prior invocation that used -j (or -p json). sorting, limits, and time fences will work. specification of a domain name, RRtype, Rdata, or offset is not supported at this time. if input_file is -, then standard input (stdin) will be read. 🍓
  • -j — synonym for -p json. two names, one vibe.
  • -k sort_keys — when sorting with -s or -S, selects one or more comma-separated sort keys, among "first", "last", "duration", "count", "name", "type", and/or "data". the default order is first,last,duration,count,name,type,data (if sorting is requested). names are sorted right to left (by TLD then 2LD etc). data is sorted either by name if present, or else by numeric value (e.g., for A and AAAA RRsets). you can give several -k options after different -s and -S options, to sort ascending for some keys and descending for others. mix and match, bestie. 💅
  • -l query_limit — query for that limit's number of responses. if specified as 0, then the DNSDB API server will return the maximum limit of results allowed. if -l is not specified, then the query won't specify a limit, and the DNSDB API server may use its own default limit.
  • -L output_limit — clamps the number of objects per response (under -[R|r|N|n|i|f]) or for all responses (under -[fm|ff|ffm]) output to output_limit. if unset, and if batch and merge modes haven't been selected with the -f and -m options, then the -L output limit defaults to the -l limit's value. otherwise the default is no output limit.
  • -M max_count — for the summarize verb, stops summarizing when the count reaches that max_count, which must be a positive integer. the resulting total count may exceed max_count, since it'll include the entire count from the last RRset examined. the default is to not constrain the maximum count. the number of RRsets summarized is also limited by the query_limit.
  • -m — used only with -f. this causes multiple (up to ten) API queries to execute in parallel. in this mode there will be no -- marker, and the combined output of all queries is what gets subject to sorting, if any. if two -f flags are specified together with -m, the output will not be merged, can appear in any order, will be sorted separately for each response, and will have normal -- and ++ markers. (see the -f option above.) parallel queries, go off. ⚡
  • -N hex[/rrtype[,...]] — specify a raw rdata data ("right-hand side") query. hex is as described above for -f.
  • -n name — specify an rdata name ("right-hand side") query. the value is a DNS domain name in presentation format, or a left-hand (*.example.com) or right-hand (www.example.*) wildcard domain name. note that left-hand wildcard queries are somewhat more expensive than right-hand wildcard queries. budget accordingly, queen. 💸
  • -O offset — offset the results returned by the query by #offset. this gives you incremental results transfers. cannot be negative. the default is 0, which means no offset.
  • -o timeout — sets the timeout, in seconds, for the initial connection to the database server and for each transaction you make to that server. ⏱️
  • -p output_type — pick your output type, bestie. You can say:
    • text — presentation output meant to be human-readable. This is the default. dns is a synonym, for compatibility with older programmatic callers. ✨
    • json — newline delimited JSON output. See also https://jsonlines.org/.
    • csv — comma separated value output. This format is information losing, since it cannot express multiple resource records that are in a single RRset. Instead, each resource record is expressed in a separate line of output. See the DNSDBQ_TIME_FORMAT environment variable for controlling how timestamps are formatted for this option.
    • minimal — outputs only the owner name or rdata, one per line and deduplicated; for use by shell scripts. This is incompatible with sorting. 🫠
  • -q — makes the program reticent about warnings. 🤫
  • -R hex[/rrtype[,...][/bailiwick]] — specify a raw rrset owner data ("left-hand side") query. Hex is as described above for -f.
  • -r name[/type[,...][/bailiwick]] — specify an RRset ("left-hand side") name query. See the discussion in -n above as to the format of and limitations on query names.
  • -s — sort output in ascending key order. Limits (if any) specified by -l and -L will be applied before and after sorting, respectively. In batch mode, the -f, -ff, and -ffm option sets will cause each batch entry's result to be sorted independently, whereas with -fm, all outputs will be combined before sorting. This means with -fm there will be no output until after the last batch entry has been processed, due to store and forward by the sort process. 🥲
  • -S — sort output in descending key order. See the discussion for -s above.
  • -T transform[,...] — specify one or more transforms to be applied to the output:
    • datefix — always show dates in the format selected by the DNSDBQ_TIME_FORMAT environment variable, not in database format.
    • reverse — show the DNS owner name (rrname) in TLD-first order (so, COM.EXAMPLE rather than EXAMPLE.COM).
    • chomp — strip away the trailing dot (.) from the DNS owner name (rrname).
    • qdetail — annotate the response to include query details, mostly for use by -V summarize. This is incompatible with sorting.
  • -t rrtype[,...] — specify the resource record type(s) desired. Default is ANY. If present, this option should precede any -R, -r, -N, or -n options. This option is not allowed if the -i option is present. Valid values include those defined in DNS RFCs, including ANY. A special-case supported in DNSDB is ANY-DNSSEC, which matches on DS, RRSIG, NSEC, DNSKEY, NSEC3, NSEC3PARAM, and DLV resource record types.

If multiple rrtype values are specified, each will be sent separately to the database server, consuming quota if there is a quota. Such queries will be sent simultaneously in parallel, which may have a load impact on the server. 🔥

  • -u server_sys — specifies the Passive DNS system and thus its syntax for RESTful URLs. Can be "dnsdb" or "circl". The default is "dnsdb". See also environment variable DNSDBQ_SYSTEM.
  • -V verb — the verb to perform, i.e. the type of query, either "lookup" or "summarize". The default is the "lookup" verb. As an option, you can specify the "summarize" verb, which gives you an estimate of result size. At a glance, it provides information on when a given domain name, IP address or other DNS asset was first-seen and last-seen by the global sensor network, as well as the total sensor network observation count. This verb respects the database limit (see -l) in that the resulting summary will only be of rows that would have been returned by the "lookup" verb. See also -M. 👑
  • -0 function=thing — this is a developer tool meant to feed automated testing systems. 🛠️
  • -U — turns off TLS certificate verification (unsafe). 🚨
  • -v — report the version of dnsdbq and exit.
  • -4 — use to force connecting to the DNSDB server via IPv4.
  • -6 — use to force connecting to the DNSDB server via IPv6.
  • -8 — normally dnsdbq requires that -n or -r arguments are 7-bit ASCII clean. Non-ASCII values should be queried using PUNYCODE IDN encoding. This -8 option allows using arbitrary 8 bit values.

TIMESTAMP FORMATS ⏰

Timestamps may be one of following forms.

  • positive unsigned integer : in Unix epoch format.
  • negative unsigned integer : negative offset in seconds from now.
  • YYYY-MM-DD [HH:MM:SS] : in absolute form, in UTC time, as DNSDB does its fencing using UTC time.
  • %uw%ud%uh%um%us : the relative form with explicit labels (w=weeks, d=days, h=hours, m=minutes, s=seconds). Calculates offset from UTC time, as DNSDB does its fencing using UTC time.

When using batch mode with the second or forth cases, using relative times to now, the value for "now" is set when dnsdbq starts. 🕰️

A few examples of how to use timefencing options.

# tuples ending after Aug 22, 2015 (midnight)
$ dnsdbq ... -A 2015-08-22
# tuples starting before Jan 22, 2013 (midnight)
$ dnsdbq ... -B 2013-01-22
# tuples starting or ending from 2015 (midnight to midnight)
$ dnsdbq ... -B 2016-01-01 -A 2015-01-01
# tuples ending after 2015-08-22 14:36:10
$ dnsdbq ... -A "2015-08-22 14:36:10"
# tuples ending within the last 60 minutes
$ dnsdbq ... -A "-3600"
# tuples ending after "just now"
$ date +%s
1485284066
$ dnsdbq ... -A 1485284066
# batch mode with only tuples ending within last 60 minutes,
# even if feeding inputs to dnsdbq in batch mode takes hours.
$ dnsdbq -f ... -A "-3600"

EXAMPLES 🌟

A few examples of how to specify IP address information.

# specify a single IPv4 address
$ dnsdbq ... -i 128.223.32.35
# specify an IPv4 CIDR
$ dnsdbq ... -i 128.223.32.0/24
# specify a range of IPv4 addresses
$ dnsdbq ... -i 128.223.32.0-128.223.32.32

Perform an RRset query for a single A record for farsightsecurity.com. The output is serialized as JSON and is piped to the jq program (a command-line JSON processor, see https://stedolan.github.io/jq/) for pretty printing.

$ dnsdbq -r farsightsecurity.com/A -l 1 -j -a | jq .
{
  "count": 6350,
  "time_first": 1380123423,
  "time_last": 1427869045,
  "rrname": "farsightsecurity.com.",
  "rrtype": "A",
  "bailiwick": "farsightsecurity.com.",
  "rdata": [
    "66.160.140.81"
  ],
  "dnsdbq-rdata": [
    {
      "asinfo": [ 6939 ],
      "cidr": "66.160.128.0/18",
      "rdata": "66.160.140.81"
    }
  ]
}

Note the "dnsdbq-rdata" element added due to the use of the -a option. 💅

Perform a batched operation for a several different rrset and rdata queries. Output is again serialized as JSON and redirected to a file.

$ cat batch.txt
rrset/name/wikipedia.org
rrset/name/dmoz.org
rrset/raw/0366736902696f00/A
rdata/name/pbs.org
rdata/name/opb.org
rdata/ip/198.35.26.96
rdata/ip/23.21.237.0,24
rdata/raw/0b763d73706631202d616c6c
$ dnsdbq -j -f < batch.txt > batch-output.json
$ head -1 batch-output.json | jq .
{
  "count": 2411,
  "zone_time_first": 1275401003,
  "zone_time_last": 1484841664,
  "rrname": "wikipedia.org.",
  "rrtype": "NS",
  "bailiwick": "org.",
  "rdata": [
    "ns0.wikimedia.org.",
    "ns1.wikimedia.org.",
    "ns2.wikimedia.org."
  ]
}

ASINFO/CIDR LOOKUPS 🔍

When the -a option is used, every address seen in a response will cause a DNS lookup under the domain specified by the -D option. This stream of DNS queries might be an intolerable information leak depending on the nature of the underlying research, and it could also lead to unusably bad performance depending on the placement of your configured recursive DNS service. 😬

For best results, always use an on-server or on-LAN recursive DNS service, and consider whether to configure that recursive DNS service to be a "stealth secondary" of the zone denoted by the -D option. For the default -D value, more information can be found online at http://archive.routeviews.org/dnszones/.

Use of DNS lookups to retrieve ASINFO/CIDR metadata can be extremely fast and surveillance-free, but some attention must be paid in order to obtain that outcome. For occasional low-volume use, your current recursive DNS placement and configuration is probably good enough.

Note that while Passive DNS information is historical, the ASINFO/CIDR annotations made possible using the -a and -D options are based on current information. Internet routing system information may have changed since the DNS data was recorded. More information about this can be found online at https://github.com/dnsdb/dnsdbq/blob/master/README. 🌐

FILES 📁

~/.isc-dnsdb-query.conf, ~/.dnsdb-query.conf, /etc/isc-dnsdb-query.conf, or /etc/dnsdb-query.conf: configuration file which can specify the API key and other variables. The first of these files which is readable will be used, alone, in its entirety. See the DNSDBQ_CONFIG_FILE environment variable which can specify a different configuration file to use.

The variables which can be set in the configuration file are as follows:

  • DNSDBQ_SYSTEM — contains the default value for the -u option described above. The last setting found for any given variable will prevail.
  • DNSDB_API_KEY, APIKEY — contains the user's DNSDB apikey (no default).
  • DNSDB_SERVER — contains the URL of the DNSDB API server (default is https://api.dnsdb.info), and optionally the URI prefix for the database. The default URI prefix for system "dnsdb2" is "/dnsdb/v2/lookup"; the default for "dnsdb1" is "/lookup".
  • CIRCL_AUTH, CIRCL_SERVER — enable access to a passive DNS system compatible with the CIRCL.LU system. 🦋

🌸 ENVIRONMENT 🌸

bestie these are the little gremlins in your shell that change how dnsdbq behaves before you even type a flag. set them, unset them, whatever — just know they're there 💖

  • DNSDBQ_CONFIG_FILE — tells dnsdbq which config file to read, and it overrides the internal search list. like, no cap, it wins. 👑
  • DNSDB_API_KEY — holds your apikey. heads up tho: the older APIKEY environment variable has been retired, though it can still be used in the configuration file. 🫠 also, and this is the important part, environment variables are unprotected, and putting your API key in an unprotected place could cause inadvertant sharing. 💀 protect the key bestie, it's giving "left my diary open on the lunch table".
  • DNSDB_SERVER — holds the URL of the DNSDB API server, and optionally a URI prefix to be used (default is "/lookup"). If it's not set, the configuration file gets consulted instead. 🦋
  • DNSDBQ_TIME_FORMAT — controls how human readable date times show up in the output. ✨
    • If "iso" (the default) then ISO8601 (RFC3339) format is used, for example; "2018-09-06T22:48:00Z".
    • If "csv" then an Excel CSV compatible format is used; for example, "2018-09-06 22:48:00".
  • HTTPS_PROXY — holds the URL of the HTTPS proxy you wish to use. if you wanna know what values it takes, see https://curl.se/libcurl/c/CURLOPT_PROXY.html. 🌈

💅 EXIT STATUS 💅

okay so this part is actually kind of iconic in how chill it is:

  • Success (exit status zero) happens if a connection could be established to the back end database server — even if no records matched the search criteria. yeah. zero results is still a slay. finding nothing is not a failure, it's just a very specific kind of nothing. 🌸
  • Failure (exit status nonzero) happens if no connection could be established, maybe because of a network or service failure, or a configuration error like specifying the wrong server hostname. 🚨
so like: "i connected and there was nothing" = 🫶. "i couldn't even connect" = 😭. different vibes entirely.

🎀 SEE ALSO 🎀

go read these too, they're the friend group:

and that's the whole page bestie!! 🌟 this one's from the dnsdbq(1) era dated 2018-01-30 — iconic, historic, still eating. go query something cursed, drink water, and remember: environment variables are unprotected, so guard that API key like it's your only charger at a sleepover. 💜✨🦋

📦 where this page comes from