ASNCOUNTER(1)

📦
version 0.7.0 · 2 versions
, in ubuntu stonking

💖 ASNCOUNTER(1) 💖

collect hits per ASN and netblock — okay bestie, this is a manual page, it's ASNCOUNTER(1) in the manual, and we are going to get through it together. Grab a boba. 🧋

✨ NAME ✨

asncounter — collect hits per ASN and netblock 🦋

🌸 DESCRIPTION 🌸

So the whole vibe here is: you count the number of hits (HTTP, packets, etc) per autonomous system number (ASN) and the related network blocks. That's it, that's the gig. 💅

And like, why do you care? Because this is useful when you get a LOT of traffic on a server and you're sitting there like "who is doing this 😭" — it helps you figure out which network is responsible for the traffic, so you can direct abuse complaints or block whole networks. And on core routers? It's also for figuring out who your peers are and who you might wanna seek particular peering agreements with. Networking main character energy fr fr. 👑

⭐ SYNOPSIS ⭐

asncounter OPTIONS [ADDRESS ...]

🎀 OPTIONS 🎀

  • -h, --help — show this help message and exit
  • --cache-directory, -C CACHE_DIRECTORY — where to store pyasn cache files, default: ~/.cache/pyasn
  • --no-prefixes — disable prefix count
  • --no-asn — disable ASN count
  • --no-resolve-asn — disable ASN to name resolution in output
  • --top, -t N — only show top N entries, default: 10
  • --input, -i INPUT — input file, default: stdin
  • --input-format, -I {line,tuple,tcpdump,scapy} — input format, default: line
  • --scapy-filter SCAPY_FILTER — BPF filter to apply to incoming packets, default: ip and not src host 0.0.0.0 and not src net 192.168.0.0/24
  • --interface [INTERFACE] — open an interface instead of stdin for packets, implies -I scapy, auto-detects by default
  • --output, -o OUTPUT — write stats or final prometheus metrics to the given file, default: stdout
  • --output-format, -O {tsv,prometheus,null} — output format, choices: tsv, prometheus, null, default: tsv
  • --port, -p [PORT] — start a prometheus server on the given port, default disabled, port 8999 if unspecified
  • --refresh, -R — download a recent RIB cache file and exit
  • --dump-prefixes, -D ASN — dump the aggregated prefixes of the given ASN to stdout and exit
  • --repl — run a REPL thread in main loop
  • --manhole — setup a REPL socket with manhole
  • --debug — more debugging output
  • ADDRESS — zero or more IP addresses to parse directly from the command line, before the input stream is read. disables the default stdin reading and --input-format cannot be changed. 👀

🌈 INPUT FORMATS 🌈

The --input-formats option warrants more discussion, bestie. Sit down. 🫶

🍓 line

The line input formats treats each line in the stream as an IP address, counting as one hit.

Empty lines are skipped, and comments — whatever follows the pound (#) sign — are trimmed. Whatever cannot be parsed as an IP address is loggd as a warning and skipped. 💀

This, for example, counts as a hit on two different IP addresses, for a total of two hits. It will also yield a warning:

192.0.2.1 # comment
2001:DB8::1
# comment
garbage that generates a warning

🦋 tuple

Same as the line input format, except the count is specified in a second, whitespace-separated field.

This, for example, will count one hit for the first IP address, and two for the second one, and will generate a warning.

192.0.2.1 1 # comment
2001:DB8::1 2
# comment
garbage that generates a warning

The “count” field can be anything: to represent a count, but also sizes, timings, asncounter doesn’t care. ✨

The counts are actually parsed as floats, as Python understand them.

The default output format (tsv) will round the numbers to the nearest even integer. This, for example, adds up to 5, which might be surprising to some (because Python rounds 2.5 to 2 and not 3):

192.0.2.2 3.4
192.0.2.2 2.5

This is known as the “rounding half to even” rule or the IEEE 754 standard. 📐

If the --output-format is set to prometheus floats will be recorded as accurately as Python allows. In that context, the above correctly sums up to 5.9. 🤯

🧚 tcpdump

The tcpdump format is a bit of an odd ball: it parses a tcpdump(1) line with a regular expression to extract the source IP address, and counts that.

It could be extended to count the packets sizes but currently does not do so. Likewise, it only tracks the left (source) side of packets, and not the destination, but could be extended to track both.

This approach likely can’t deal with a multi-gigabit per second small packet attack (2 million packets per second or more). But in a real production environment, it could easily deal with regular the 100-200 megabit per second traffic, where tcpdump and asncounter each took about 2% of one core to handle about 3-5 thousand packets per second. 🔥

💜 scapy

The scapy input format is also special: instead of parsing text lines, it parses packets.

With the --interface flag, it will open the default interface unless one is provided (e.g. --interface is generally equivalent to --interface eth0 if eth0 is the primary interface). This requires elevated privileges. 😤

This is much slower than the tcpdump parser (close to full 100% CPU usage) in a 100-200mbps scenario like above, but could eventually be leveraged to implement byte counts, which are harder to extract from tcpdump because of the variability of its output.

This only counts packets, regardless of direction, and, like tcpdump, only keeps track of source IP addresses. Like tcpdump, it could also be improved by tracking sizes instead of counts, but does not currently do so. 🫠

👑 OUTPUT FORMATS 👑

The --output-format argument also warrants a little more discussion. 📝

💅 tsv

TSV stands for Tab-Separated Values. It’s a poorly designed output formats that dumps two tables where rows are separated by newlines and columns by tabs. One table shows per ASN counts, the other shows per prefix counts. 😭

As mentioned in the above tuple section, counts are rounded when recorded in tsv mode. This is to simplify the display, in theory, the underlying Counter supports floats as well.

If more precision, long term storage, or alerting are needed, the prometheus output format is preferred.

This format is useful because it doesn’t require any dependency outside of the standard library (and, obviously, pyasn). 🎀

🌟 prometheus

The prometheus output format keeps tracks of counters inside Prometheus data structures. With the --port flag, it will open up a port (defaulting to 8999) where metrics will be exposed over HTTP, without any special security, on all interfaces. 🚨

no special security, bestie. on all interfaces. iykyk. 🙈

Otherwise, upon completion, results will be written in a textfile collector-compatible format.

🌸 null

The null output formats doesn’t display anything. It can be used for debugging, but internally uses the same recorder as the tsv format. 🤫

💖 EXAMPLES 💖

🍓 Simple web log counter

Okay so first up, the classic: this pulls the IP addresses out of your current access logs and reports the ratios. Main character energy fr.

> awk '{print $2}' /var/log/apache2/*access*.log | asncounter
INFO: using datfile ipasn_20250527.1600.dat.gz
INFO: collecting addresses from <stdin>
INFO: loading datfile /home/anarcat/.cache/pyasn/ipasn_20250527.1600.dat.gz...
INFO: finished reading data
INFO: loading /home/anarcat/.cache/pyasn/asnames.json
count   percent ASN AS
12779   69.33   66496   SAMPLE, CA
3361    18.23   None    None
366 1.99    66497   EXAMPLE, FR
337 1.83    16276   OVH, FR
321 1.74    8075    MICROSOFT-CORP-MSN-AS-BLOCK, US
309 1.68    14061   DIGITALOCEAN-ASN, US
128 0.69    16509   AMAZON-02, US
77  0.42    48090   DMZHOST, GB
56  0.3 136907  HWCLOUDS-AS-AP HUAWEI CLOUDS, HK
53  0.29    17621   CNCGROUP-SH China Unicom Shanghai network, CN
total: 18433
count   percent prefix  ASN AS
12779   69.33   192.0.2.0/24    66496   SAMPLE, CA
3361    18.23   None
298 1.62    178.128.208.0/20    14061   DIGITALOCEAN-ASN, US
289 1.57    51.222.0.0/16   16276   OVH, FR
272 1.48    2001:DB8::/48   66497   EXAMPLE, FR
235 1.27    172.160.0.0/11  8075    MICROSOFT-CORP-MSN-AS-BLOCK, US
94  0.51    2001:DB8:1::/48 66497   EXAMPLE, FR
72  0.39    47.128.0.0/14   16509   AMAZON-02, US
69  0.37    93.123.109.0/24 48090   DMZHOST, GB
53  0.29    27.115.124.0/24 17621   CNCGROUP-SH China Unicom Shanghai network, CN

And bestie, you can absolutely do this in real time too, obviously:

tail -F /var/log/apache2/*access*.log | awk '{print $2}' | asncounter

That report up there? It shows up when the process gets killed. But if you want a report without interrupting the parser, just send SIGHUP 💅

pkill -HUP asncounter

Not me forgetting you can count sizes too, with --input-format=tuple. Say your size field lives in the 10th column — this will sum the sizes instead of just counting the hits:

tail -F /var/log/apache2/*access*.log | awk '{print $1 $10}' |
asncounter --input-format=tuple

And if your logs happen to hold that info, you can add up processing times too, for example. 👑

🌈 tcpdump parser

Next vibe: yank the IP addresses out of incoming TCP/UDP packets on eth0 and report the top 5:

> tcpdump -c 10000 -q -i eth0 -n -Q in "(udp or tcp)" | asncounter --top 5 --input-format tcpdump
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
INFO: collecting IPs from stdin, using datfile ipasn_20250523.1600.dat.gz
INFO: loading datfile /root/.cache/pyasn/ipasn_20250523.1600.dat.gz...
INFO: loading /root/.cache/pyasn/asnames.json
ASN     count   AS
136907  7811    HWCLOUDS-AS-AP HUAWEI CLOUDS, HK
8075    254     MICROSOFT-CORP-MSN-AS-BLOCK, US
62744   164     QUINTEX, US
24940   114     HETZNER-AS, DE
14618   82      AMAZON-AES, US
prefix  count
166.108.192.0/20        1294
188.239.32.0/20 1056
166.108.224.0/20        970
111.119.192.0/20        951
124.243.128.0/18        667

A query that's lowkey similar to the HTTP log parser might look like this:

tcpdump -i eth0 -n -Q in "tcp and (port 80 or port 443)" | grep 'Flags \[S\]' | asncounter --input-format=tcpdump --repl

... otherwise you're gonna get different results than a pure packet count, because different connections yield different numbers of packets! This one up here counts connection attempts, which is still not the same as an actual HTTP hit — the connection could get refused before it even reaches the webserver, or aborted before it gets logged properly. 😭

It's still a solid estimate tho, and it's especially useful when you don't log IP addresses at all, like on high traffic caching servers. ✨

Heads up: we use grep up there because tcpdump's tcp[tcpflags] & tcp-syn != 0 only works for IPv4 packets — a disappointing (but understandable) limitation. 💀

🦋 scapy parser

Wanna grab IP addresses straight off the network interface, bypassing tcpdump entirely? Iconic, honestly:

asncounter --interface

🎀 REPL

With --repl, you drop into a Python shell where you can get your real-time statistics, interactively. It's giving ✨interactive✨:

> awk '{print $2}' /var/log/apache2/*access*.log | asncounter --repl --top 2
INFO: using datfile ipasn_20250527.1600.dat.gz
INFO: collecting addresses from <stdin>
INFO: starting interactive console, use recorder.display_results() to show current results
INFO: recorder.asn_counter and .prefix_counter dictionaries have the full data
Python 3.11.2 (main, Apr 28 2025, 14:11:48) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> INFO: loading datfile /home/anarcat/.cache/pyasn/ipasn_20250527.1600.dat.gz...
INFO: finished reading data

>>> recorder.display_results()
INFO: loading /home/anarcat/.cache/pyasn/asnames.json
count   percent ASN AS
13008   69.38   66496   SAMPLE, CA
3422    18.25   None    None
total: 18748
count   percent prefix  ASN AS
13008   69.38   192.0.2.0/24    66496   SAMPLE, CA
3422    18.25   None
total: 18748
>>> recorder.asn_counter
Counter({66496: 13008, None: 3422, [...]})
>>> recorder.prefix_counter
Counter({'192.0.2.0/24': 13008, None: 3422, [...]})

So you can get the actual number of hits for an AS even when it's not listed in the --top entries, like this:

>>> recorder.asn_counter.get(66496)
13008

💅 Blocking whole networks

Real talk: asncounter does not block anything. It only counts. Periodt. If you actually wanna block attackers or act on the collected data, you need another mechanism entirely.

If you do wanna block the network blocks, you can feed the shown netblocks straight into (say) Linux's netfilter firewall, or Nginx's access module / geo module. For example, this rejects traffic from a network with iptables:

iptables -I INPUT -s 192.0.2.0/24 -j REJECT

or with nftables:

nft insert rule inet filter INPUT 'ip saddr 192.0.2.0/24 reject'
Lowkey this is gonna become impractical with a large number of networks — look into IP sets to scale that up. 🌱

With Nginx, you can block a network using the deny directive:

deny 192.0.2.0/24;

That'll hand back a 403 status code. And if you wanna be fancier, you can return a tailored status code and build a bigger list with the geo module:

geo $geo_map_deny {
    default 0;

    192.0.2.0/24 1;
}

if ($geo_map_deny) {
  return 429;
}

Many networks can be listed in the geo block relatively effectively. 💪

pyasn doesn't (unfortunately) give you an easy command line interface to extract the data you'd need to block an entire AS. 🙈

And that is exactly why asncounter hands you the --dump-prefixes flag:

asncounter --dump-prefixes 64496

You can also do it in Python, from inside the --repl loop:

print("\n".join(sorted(recorder.asn_all_prefixes(64496))))

This gives you the list of ALL prefixes associated with AS64496 — which is actually empty in this case, because AS64496 is an example AS from RFC5398. 🫠

Note: in that Python code, the list of prefixes is not aggregated by default, but it is in --dump-prefixes, if netaddr is installed. In that case you can pass aggregate=True to reduce the set from Python.

🌸 Aggregating results

Aggregating a large number of netblocks might genuinely be worth it, for performance reasons. Network block announcements can be spread across multiple contiguous blocks for all kinds of reasons, and they can often be unified into smaller sets. For IPv4-only, iprange is good (and fast) enough:

> grep -v :: networks > networks-ipv4
> iprange < networks-ipv4 > networks-ipv4-filtered
> wc -l networks*
  588 networks
  495 networks-ipv4
  181 networks-ipv4-filtered

If you've got it installed, the netaddr Python package can do that for you too — and it supports IPv6 as well:

import netaddr
print("\n".join([str(n) for n in netaddr.cidr_merge(recorder.asndb.get_as_prefixes(64496))]))

And guess what, asncounter can aggregate those results directly now, for example:

print(recorder.asn_all_prefixes_str(66496, aggregate=True))

... but, as above, that requires the netaddr package to be available. It's also done automatically by the --dump-prefixes flag, like we said before:

asncounter --dump-prefixes 64496

🧚 Selective blocking

A more delicate approach 💐 is to block all the network blocks from one specific ASN that showed up in the result sets, instead of blocking the entire netblock.

The recorder.as_prefixes and recorder.as_prefixes_str functions can do this for you, merging multiple ASNs and aggregating with netaddr too:

print(recorder.asn_prefixes_str(66496, 66497, aggregate=True))
Heads up bestie: the asn_prefixes selectors are not implemented in Prometheus mode. ⚠️

Also remember you can pull the list of current ASNs and prefixes just by peeking at the dictionary keys:

print("\n".join(recorder.asn_counter.keys()))
print("\n".join(recorder.prefix_counter.keys()))

And this one can't be done with the --dump-prefixes flag, because that flag starts from scratch and dumps the entire AS — not just the networks you actually encountered during parsing. 🎯

✨ ENVIRONMENT ✨

  • $XDG_CACHE_HOME/.cache/pyasn — where to store the pyasn cache files 🗂️
  • $XDG_RUNTIME_DIR or $XDG_STATE_HOME/.local/state — where the manhole socket will be created if --manhole is used without arguments 🕳️

💀 SIGNALS 💀

  • SIGHUP — reload the cache files 🔄
  • SIGUSR1 — refresh and reload the cache files, equivalent to --refresh, but it blocks the main thread which can lead to problems for high traffic environments 🚨
  • SIGWINCH — display the current statistics 📊

👑 FILES 👑

  • ~/.cache/pyasn/ — default storage location for pyasn cache files.
  • /run/$UID/asncounter-manhole-$PID or ~/.local/.state/asncounter-manhole-$PID — default location for the debugging manhole socket, if enabled. 🐛

💀 LIMITATIONS 💀

okay bestie, time for the ✨honesty hour✨ section. she's not perfect and she knows it. here's what asncounter does not do, so you don't @ her later:

  • only counts, does not calculate bandwidth, but could be extended to do so 🙈
  • does not actually do any sort of mitigation or blocking, purely an analysis tool; if you want such mitigation, hook up asncount in Prometheus and AlertManager with web hooks, this is not a fail2ban rewrite 💅
  • test coverage is relatively low, 37% as of this writing. most critical paths are covered, although not the scapy parser or the RIB file download procedures 💀
  • requires downloading RIB files, could be improved by talking directly with a BGP router daemon like Bird or FRR
  • only a small set of tcpdump outputs have been tested
  • the REPL shell does not have proper readline support (keyboard arrows, control characters like “control a” do not work) 😭
Note that this documentation and test code uses sample AS numbers from RFC5398, IPv4 addresses from RFC5737, and IPv6 addresses from RFC3849. Some more well known entities (e.g. Amazon, Facebook) have not been redacted from the output for clarity.

🌸 Performance considerations 🌸

As mentioned above, this will unlikely tolerate multi-gigabit denial of service attacks. The tcpdump parser, however, is pretty fast and should be able to sustain a saturated gigabit link under normal conditions. The scapy parser is slower. 🐌

Memory usage seems reasonable: on startup, it uses about 250MB of memory, and a long-running process with about 40 000 blocks was using about 400MB.

By extrapolation, it is expect that data on the full routing table (currently 1.2 million entries) could be held within 12 GB of memory, although that would be a rare condition, only occurring on a core router with traffic from literally the entire internet. 🌈

🎀 Security considerations 🎀

There’s an unknown in the form of the C implementation of a Radix tree in pyasn. asncounter itself should be fairly safe: it does not trust its inputs, and the worse it can do is likely a resource exhaustion attack on high traffic.

It can run completely unprivileged as long as it has access to the input files, although in many scenarios people will not bother to drop privileges before calling it and it will not, itself, attempt to do so. 👑

Privileges can be dropped with systemd-run, for example:

systemd-run --pipe --property=DynamicUser=yes \
   --property=CacheDirectory=asncounter \
   --setenv=XDG_CACHE_HOME=/var/cache/asncounter \
   -- asncounter

This interacts poorly with --repl option, as it tries to reopen the tty for stdin. You might have better luck with sharing a debug socket with --manhole:

systemd-run --pipe --property=DynamicUser=yes \
   --property=CacheDirectory=asncounter \
   --setenv=XDG_CACHE_HOME=/var/cache/asncounter \
   -- asncounter --manhole=/var/cache/asncounter/asncounter-manhole

Then you can open a Python debugging shell for further diagnostics with:

nc -U /var/cache/asncounter/asncounter-manhole
Note that the source distribution ships with a more advanced and locked down asncounter-exporter.service which exposes a Prometheus exporter but generally provides a good way to start asncounter securely under systemd. If asncounter was installed through the Debian package, the unit is not enabled by default. It can be enabled and started with:
systemctl enable --now asncounter.service

Metrics are exposed over the port 8999 and a “manhole” is opened in /var/cache/asncounter/asncounter-manhole if available.

A timer called asncounter-refresh can be used to automatically refresh those databases. ⏰

👑 AUTHOR 👑

Antoine Beaupré anarcat@debian.org

💜 SEE ALSO 💜

tcpdump(8), fail2ban(1), netaddr(1), grepcidr(1)

and that's the whole page bestie, we ate and left absolutely no crumbs 🍓💅 she's ASNCOUNTER(1) fr fr — go forth, count those ASNs, stay unprivileged, and hydrate 💖🦋✨

📦 where this page comes from