DNSHISTORY(1)

📦 version 1.3, in debian bookworm, ubuntu jammy

💖 DNSHISTORY(1) 💖

okay bestie, sit DOWN. this is dnshistory from the User Commands section and it is genuinely iconic. it takes your messy little log files, does DNS lookups on the IP addresses in them, and keeps a cute little history database so you never have to ask twice. 🦋

✨ NAME ✨

dnshistory — processes various log file formats doing dns IP Address lookups. Store these pairs in a database for later retrieval.

like. it reads the logs. it finds the IPs. it asks the DNS who they are. it writes it all down. main character energy fr fr. 👑

🌸 SYNOPSIS 🌸

dnshistory [OPTION]... [--file=FILE]

dnshistory is Copyright © 2004, 2005, 2006, 2007 Stephen McInerney

💅 DESCRIPTION 💅

so rn dnshistory processes Apache CLF and Combined logs, Squid access logs, FTP xferlog files and iptables based logs. and the log format? auto-detected. it just knows. ✨

dnshistory has five modes of operation:

  • Do Lookups. The default mode. Given a web log file, dnshistory will perform DNS reverse lookups on each unique IP Address and store the results in a history database.
  • Do Translations. Given a raw web log file, dnshistory will make use of a previously created history database and send to STDOUT the same web log but with addresses replaced by the Fully Qualified Domain Name as previously looked up.
  • Do Recombining. Given two web log files, one raw and one previously translated (eg. by using dnstran): Create a history database from the values in these separate log files.
  • Do Dump. Dump a given history database to STDOUT.
  • Do Import. Import a previously dumped history database from a given file.
  • Show History. Given one or more IP Addresses on the command line, show the history of those addresses.

The lookups make use of threads for maximum speed, and use the standard resolution libraries on a system. Thus hosts files, NIS, LDAP and other name resolution methods should work transparently. no cap, it just slots into whatever your system already does. 🫶

It is strongly recommended that for massive lookups a DNS server is "nearby". Preferably not a forwarding server. like literally, keep your DNS bestie close or this gets rough. 😭

dnshistory can read .gz files. STDIN is assumed to not be gz encoded. (so don't try to pipe zipped stuff in, it's giving chaos 🚫)

⭐ OPTIONS ⭐

  • -L, --dolookups — The default mode. Given a log file, either via STDIN or via '-f', do the lookups and store the results. this is the one that runs if you say nothing at all 💖
  • -T, --dotranslate — Given a log file, either via STDIN or via '-f', lookup each IP Address from the history database; replace the IP Address with the FQDN and send the newly updated log line to STDOUT. glow-up for your log file honestly ✨
  • -R, --dorecombine=FILE — Given a previously translated file (eg. via dnstran) via this option for the names, do the lookups for a file given via STDIN or '-f' and store the results. This file can be gz encoded.
The date/time of each stored entry is taken as being the actual time for the lookup stored in the recombine log file. This is probably incorrect, but "Good Enough". 💀
  • -D, --dodump — Dump the history database to STDOUT.
  • -I, --doimport=FILE — Given a previously dumped database, import that into a new database. Will fail and exit if the chosen database already exists. it's not gonna overwrite your stuff, periodt.
  • -S, --showhistory — Given one or more IP Addresses show their history. Address are the last item(s) on the command line. Addresses with no as yet discovered FQDN will display 'NONAME'. The Date/Time displayed is formatted as YYYY-MM-DD:hh:mm:ss, vs the 'seconds since epoch' for "--dodump"
  • --logtype=LOGTYPE — By default dnshistory will attempt to autodetect what type of logfile is being processed. By using this option, the autodetection is overridden. The choices are: auto, clf or www, squid, ftp or iptables.
  • -c, --cache=SIZE — Set the size of the memory cache to use. Value is in Mb. Default is 20Mb.
  • -d, --database=FILE — Change the default database file to use to store stateful data.
  • -f, --file=FILE — Web Log File to process. This file can be gz encoded. Will use STDIN if not set
  • -h, --help — Help screen. Very brief. (it said what it said 🙈)
  • -l, --maxlookups=NUMBER — The maximum number of lookups to attempt. The default is 1. This has not shown to be at all useful in testing... 💀
  • -m, --maxthreads=NUMBER — How many name lookup threads to spawn off. The default is 100. Setting this too high can do evil things to bandwidth and the CPU usage of any queried DNS server(s).
If doing lots of DNS queries, setting this too high can have a very negative impact on the ability to successfully resolve anything. so like, don't be greedy bestie, more threads ≠ more slay 😭
  • -t, --timeout=VALUE — The time in seconds before a stored DNS value is deemed "old". The default is 7 days.
  • -v, --verbose — Verboseness of a run. More v's will increase the level of verbosity, up to a maximum of 5. All of the higher levels are only of value for debugging purposes.
  • -V, --version — Display the version information and exit
  • -w, --wait — Delay time between query retries within a single run

🌈 RESULTS 🌈

  • At verbose level 1 (-v) some success/failure counters will be displayed. As well as any problematic log lines to STDERR.
  • At verbose level 2 (-vv) lines that may not match up (eg. Due to dnstran modifying referrers or URL's) will be sent to STDERR.

🍓 EXAMPLES 🍓

A typical run, using a database in /tmp/ (/tmp/c.db), and a log file in the current directory (test.log). First, do the lookups:

dnshistory -d /tmp/c.db -f test.log

Then the translation run for input into, for example, a web log analyser:

dnshistory -T -d /tmp/c.db -f test.log | webalizer ....

Do three attempts on failed queries, with a 2 second delay between retries:

dnshistory -l 3 -w 2 -d /tmp/c.db -f test.log

Lookup and Display the history of three IP Addresses: 127.0.0.1, 192.168.1.254, 10.10.10.10

dnshistory -S -d /tmp/c.db 127.0.0.1 192.168.1.254 10.10.10.10

Import a previously dumped database via dnsdb.dump

dnshistory -I dnsdb.dump -d /tmp/d.db

🧚 FILES 🧚

  • /var/lib/dnshistory/dnshistory.db — The default history database file.

💀 BUGS 💀

  • Ignores IP Addresses located in the URL and Referrer fields. (yeah. it does that. we don't talk about it 🙈)

👑 AUTHOR 👑

Stephen McInerney <spm@stedee.id.au>

and that's the whole page bestie!! this one's a Linux manual, January 2007 edition, and it's genuinely one of the good ones — threads, auto-detection, gz support, the works. go do your lookups, stay hydrated, and remember: keep that DNS server nearby. 💖✨🎀🦋🌸💅

📦 where this page comes from