NFDUMP(1)
๐ NFDUMP(1) ๐
okay so bestie, welcome to nfdump(1) from the General Commands Manual โ yes, the flow-analysis queen herself ๐
. This whole page is about nfdump 1.7.10, so keep that version in your pocket.
๐ NAME ๐
nfdump โ flow display and analysis program. She displays flows, analyzes flows, and honestly? iconic behavior.
๐ฆ SYNOPSIS ๐ฆ
nfdump -r flowpath [-w outfile] [-f filterfile] [-C config]
[-R filelist] [-M dirlist] [-O order] [-m] [-P filter]
[-t timewin] [-c num] [-a] [-A aggregation] [-b] [-B] [-I] [-g]
[-G geoDB] [-H torDB] [-s statistic] [-n num] [-o format] [-6]
[-q] [-N] [-i ident] [-v flowfile] [-E flowfile] [-x flowfile]
[-W workers] [-z=<compress>] [-J compress] [-j] [-y] [-D] [-X]
[-Z] [-T] [-V] [-h] [filter]
๐ DESCRIPTION ๐
nfdump reads the flow data from one or more binary files, created by any nfdump collector nfcapd, nfpcapd and sfcapd. It processes and lists the flows in many different output formats and can create a wide range of statistics.
This manual describes nfdump 1.7.10.
nfdump has a very powerful flow filter to process flows. The filter syntax is very similar to tcpdump, but adapted and extended for flow filtering. A flow filter may also contain arrays of many thousand IP addresses etc. to search for specific records.
nfdump can aggregate flows according to a user defined number of elements. This masks certain elements and allows to sum up flow records matching the same values.
The combination of flow filtering and aggregation as input for any flow statistics allows complex flow processing. Pre-filtered and aggregated flow data may also be written back into a binary flow file, which again may be processed with nfdump.
nfdump can enrich the listing of flows with geo location information and AS information, unless AS information is already available in the flow records. IP addresses can be tagged with a two letter country code, or with a longer location label containing the geographic region, country and city. The geo location and AS information is retrieved from the optional geoDB database, created by the geolookup program from the nfdump tools. geolookup uses the Maxmind database GeoDB or GeoLite2 to create a binary lookup database for nfdump. Please check the geolookup(1) man page for more details.
โจ the options are as follows, bestie โจ
-rflowpathโ Reads flow records from this path.flowpathmay be a single file, or a directory containing any number of flow files or sub directories. All files are processed in the order, as listed by the OS.-woutfileโ Writes all processed records intooutfileinstead of printing. Theoutfileis a binary flow file and may be processed again withnfdump. This can be useful to limit flows according to a flow filter and/or specific flow aggregation.-ffilterfileโ Reads the flow filter fromfilterfile. This can be useful for very long or structured filters, with comments and long lists. Note: Any filter specified directly on the command line takes precedence over thefilterfile.-Cconfigโ Read more options from fileconfig.nfdumptries to read by default%prefix/etc/nfdump.conf. This may be overwritten by the environment variableNFCONFwhich again may be overwritten by this option-C. In order to prevent reading any config file, even if it would exist set-C none. A config file is not required, but may be handy for often used output formats etc.-Oorderโ Sets an output order for records to be printed as text output. This order applies after all records processing, such as filtering, and aggregation and before printing.flowsโ Sort according to the number of flowspacketsโ Sort according to input plus output packetsipacketsโ Sort according to input packetsopacketsโ Sort according to output packetsbytesโ Sort according to input plus output bytesibytesโ Sort according to input bytesobytesโ Sort according to output bytesppsโ Sort according to input plus output packets per secondippsโ Sort according to input packets per secondoppsโ Sort according to out packets per secondbpsโ Sort according to input plus output bits per secondibpsโ Sort according to input bits per secondobpsโ Sort according to output bytes per secondbppโ Sort according to input plus output bytes per packetibppโ Sort according to input bytes per packetobppโ Sort according to output packetststartโ Sort according to start time of flow - former-mtendโ Sort according to end time of flowsdurationโ Sort according to duration of flows
-mโ Deprecated. Equivalent to-O tstart.-Pfilterโ Apply a post filter to the resulting flow table. Unlike the regular netflow filter, which is applied while reading the flow records, this filter is applied after flow analysis - stat generation or aggregation - and before the records are printed. This allows filtering on values which only exist after aggregation or statistics, such as the aggregated flows, packets or bytes counters of an aggregated record. The filter syntax is identical to the regular netflow filter syntax.-ttimewinโ Set time window to process flows. The time window is specified as:YYYY/MM/dd.hh:mm:ss[-YYYY/MM/dd.hh:mm:ss]. Any parts of the time spec may be omitted e.gYYYY/MM/ddexpands toYYYY/MM/dd.00:00:00-infinityand processes all flow from a given day onwards. This option is legacy. New filters should use thefirst seenandlast seenfilter primitives instead, e.g.
% nfdump -r flowfile 'first seen >= 2024-07-11T09:15:00 and last seen <= 2024-07-11T10:00:00'
-cnumโ Limit the number of records to be processed to the firstnumrecords, which passed the filter.-aโ Aggregate flow records. The default aggregation is done at connection level by taking the 5-tupleprotocol,srcip,dstip,srcportanddstport. This way of aggregation may be overwritten by option-A.-Aaggregationโ Sets the list of elements in a flow record to be aggregated.aggregationis a','separated list of any number ofv9/ipfixelements. The following elements are accepted:protoโ IP protocolsrcipโ Source IP addressdstipโ Destination IP addresssrcip4/netโ IPv4 source IP address with applied netmasksrcip6/netโ IPv6 source IP address with applied netmaskdstip4/netโ IPv4 destination IP address with applied netmaskdstip6/netโ IPv6 destination IP address with applied netmasksrcnetโ Apply netmasksrcmaskin netflow record for source IPdstnetโ Apply netmaskdstmaskin netflow record for dest IPsrcportโ Source portdstportโ Destination portsrcmaskโ Source maskdstmaskโ Destination masksrcvlanโ Source vlan labeldstvlanโ Destination vlan labelsrcasโ Source AS numberdstasโ Destination AS numbersrcasnโ Source AS number, printed with AS organisation namedstasnโ Destination AS number, printed with AS organisation namenextasโ BGP Next ASprevasโ BGP Previous ASinifโ SNMP input interface numberoutifโ SNMP output interface numbernextโ IP next hopbgpnextโ BGP next hopinsrcmacโ In source MAC addressoutdstmacโ out destination MAC addressindstmacโ In destination MAC addressoutsrcmacโ Out source MAC addresstosโ Source type of servicesrctosโ Source type of Servicedsttosโ Destination type of Serviceethertypeโ Ethernet type.minttlโ Minimum IP TTL.maxttlโ Maximum IP TTL.flagsโ TCP flagsmpls1โ MPLS label 1mpls2โ MPLS label 2mpls3โ MPLS label 3mpls4โ MPLS label 4mpls5โ MPLS label 5mpls6โ MPLS label 6mpls7โ MPLS label 7mpls8โ MPLS label 8mpls9โ MPLS label 9mpls10โ MPLS label 10routerโ IP address of exporting routerexpโ Exporter SysIDodidโ observation domain IDopidโ observation point IDsrcgeoโ Source geographic location.dstgeoโ Destination geographic location.xsrcipโ X-late source IP address, if compiled with NSEL supportxdstipโ X-late destination IP address, if compiled with NSEL supportxsrcportโ X-late source port, if compiled with NSEL supportxdstportโ X-late destination port, if compiled with NSEL support
nfdumpautomatically compiles the appropriate output format for the selected aggregation elements unless an explicit output format-ois given. The automatic output format is identical to-o 'fmt:%ts %td <fields> %pkt %byt %bps %bpp %fl'where<fields>represents the selected aggregation tags.
-bโ Aggregate flow records as bidirectional flows. This automatically implies-a. Aggregation is done on connection level by taking the 5-tupleprotocol,srcip,dstip,srcportanddstport. The reverse order applies for the corresponding reverse flow. Input and output packets/bytes are counted and reported separately. Both flows are merged into a single record with corresponding input and output counters. An appropriate output format is selected automatically, which may be overwritten by any-oformat option.-Bโ Similar to option-bbut tries to guess the correct client to server direction. Automagically swaps flows ifsrc port is < dst portfor TCP and UDP flows andsrc port < 1024anddst port > 1024. Some exporters do not really care sending the flows in proper order. Itโs considered to be a convenience option.-Iโ Print flow statistics of a single file or the summary of all the files specified by-r flowpath.-gโ Print for each flow file given by-r flowpatha one line summary, which can be easily used by gnu plot.-GgeoDB โ tells nfdump to use geoDB as the geo lookup DB for geo location and AS lookups. nfdump tries to read the environment variableNFGEODBfor the path of geoDB. The option-GoverwritesNFGEODBorgeodb.pathinnfdump.conf. And if you wanna make absolutely sure it reads no geoDB file at all, even one that literally exists, set-G none. See alsogeolookup(1). ๐-HtorDB โ same vibe but for Tor: use torDB as the tor lookup DB for tor exit node lookups. nfdump tries to read the environment variableNFTORDBfor the path of torDB. The option-HoverwritesNFTORDBortordb.pathinnfdump.conf. To prevent reading any torDB file, even if one exists, set-H none. See alsotorlookup(1). ๐ง
-sstatistic[:p[/orderby]]โ generates the Top N flow record or flow element statistic. If you optionally add:pto statistic, the statistic gets additionally split up into the transport layer protocols. By default the statistic is transport protocol independent. Each statistic may be ordered by the optional parameter orderby. ๐ฅ
orderby can be flows, packets, ipkg, opkg, bytes, ibyte, obyte, pps, ipps, opps, bps, ibps, obps, or bpp, ibpp, obpp โ where the i and o prefixed variants order by the input-only or output-only counter/rate, respectively, instead of the input+output sum. You may specify more than one orderby option, which results in the same statistic but ordered differently. If no orderby is given, the statistic is ordered by flows. You can specify as many -s flow element statistics as needed on the command line for the same run. โจ
statistic can be:
recordโ aggregated netflow records.srcipโ source IP addressesdstipโ destination IP addressesipโ any (src or dst) IP addressessrcgeoโ Two letter src geo country code of src IP addressdstgeoโ Two letter dst geo country code of dst IP addressgeoโ Two letter src/dst geo country code of src/dst IP addressnhipโ next hop IP addressesnhbipโ BGP next hop IP addressesrouterโ exporting router IP addresssrcportโ source portsdstportโ destination portsportโ any (source or destination) portsflagsโ TCP flagstosโ type of service - default srcsrctosโ src type of servicedsttosโ dst type of servicedirโ flow directions ingress/egresssrcasโ source AS numbersdstasโ destination AS numberssrcasnโ source AS organisations and numbersdstasnโ destination AS organisations and numbersasโ any (source or destination) AS numbersasnโ any (source or destination) AS org and numbersinifโ input interfaceoutifโ output interfaceifโ any interfacesrcmaskโ src maskdstmaskโ dst masksrcvlanโ src vlan labeldstvlanโ dst vlan labelvlanโ any vlan labelinsrcmacโ input src MAC addressoutdstmacโ output dst MAC addressindstmacโ input dst MAC addressoutsrcmacโ output src MAC addresssrcmacโ any src MAC addressdstmacโ any dst MAC addressinmacโ any input MAC addressoutmacโ any output MAC addressmaskโ any maskprotoโ IP protocolstunprotoโ IP tunnel protocolsmpls1โ MPLS label 1mpls2โ MPLS label 2mpls3โ MPLS label 3mpls4โ MPLS label 4mpls5โ MPLS label 5mpls6โ MPLS label 6mpls7โ MPLS label 7mpls8โ MPLS label 8mpls9โ MPLS label 9mpls10โ MPLS label 10nbarโ nbar IDja3โ ja3 hashesja4โ JA4 client fingerprints.ja4sโ JA4 server fingerprints.ethertypeโ Ethernet type.clโ Client latency.slโ Server latency.alโ Application latency.minttlโ Minimum IP TTL.maxttlโ Maximum IP TTL.nextasโ BGP next AS.prevasโ BGP previous AS.macโ Any MAC address.odidโ observation domain IDopidโ observation point IDivrfโ ingress vrfevrfโ egress vrf- NSEL/ASA statistics ๐
eventโ NSEL/ASA eventxeventโ NSEL/ASA extended eventnatโ NAT event type.natsrcipโ NSEL/ASA nat src IP addressnatdstipโ NSEL/ASA nat dst IP addressnatipโ NSEL/ASA nat src/dst IP addressesnatsrcportโ NSEL/ASA nat src portnatdstportโ NSEL/ASA nat dst portnatportโ NSEL/ASA nat src/dst portiaclโ NSEL/ASA ingress ACLeaclโ NSEL/ASA egress ACL
Example:
% nfdump -s srcip -s ip/flows/bytes -s record/bytes
-nnum โ sets the number of records to be printed to num. This option applies to-sstatistics as well as to ordered output-O-or-aggregated-records-a. The default is set to10for statistics and unlimited for the other use cases. Wanna disable the limit entirely? Set num to0. ๐ข-oformat โ sets the output format to print flow records. It has many different output formats already predefined. format may be one of the options below: ๐rawโ Print the full flow record on multiple lines, including every available extension.fmt:user โ Print the flow records according the format user. This is a very flexible and powerful way to format flow records. See the section OUTPUT below for more details on how to compile your own format.csv:user โ Print the flow records as user defined csv format. Use a','separated list of user defined output tokens, using the same%tagsyntax as fmt format. Composite tags that combine several values into one field, such as%sap,%dap,%gsa,%gda,%gsap,%gdap,%gra,%ttl,%ipl,%opland%n, are only available in fmt format and are rejected in csv format, since each csv token is expected to expand to a single atomic column. See the section OUTPUT below for more details on how to compile your own csv format. โ ๏ธjsonโ Print all selected records as JSON objects in one JSON array.ndjsonโ Print one complete JSON object per line, without an enclosing array. Suitable for log processors such as Logstash.csvโ Print the predefined comma-separated format corresponding toline.csv-fastโ Print a compact predefined CSV representation for high-throughput consumers.nullโ Suppress flow-record output while retaining applicable summary/statistics output.
Already predefined fmt formats:
lineโ Print each flow on one line. Default format.longโ Print each flow on one line with more detailsbilineโ Same asline, but for bi-directional flowsbilongโ Same aslong, but for bi-directional flowsglineโ Same asline, but add country code to IPs. If a geoDB file is supplied this is the default output formatglongโ Same aslong, but add country code to IPsextendedโ Print each flow on one line with even more details.nselโ Print format for NSEL event records. Default format if NSEL/NAT support has been compiled in.natโ Print format for NAT event records.
The nfdump config file may contain additional formats. If you want to add new formats or change existing ones, check the config file. ๐
IPv6 addresses are printed condensed in any fmt defined format to prevent cluttering the output with large blank blocks. A condensed IPV6 uses max 16 characters. If it is longer, then the middle part of the IP is cut out and replaced be "..". For previewing an output, this fits most needs. For a listing with the full IPV6 addresses add option -6. ๐ฆ
-6โ print full length of IPv6 addresses in output instead of condensed. Long girlies stay long ๐-qโ Quiet mode. Suppress the header line and the statistics at the bottom of text outputs. ๐คซ-Nโ Print plain numbers in output without scaling. Easier for output parsing with 3rd party tools. ๐ข-iident โ change the ident label in the file, specified by-rto ident-vflowfile โ Verify the consistency of flowfile and print the file parameters and number of records. ๐ต๏ธ-Eflowfile โ Print the exporter and sampler list if found in flowfile. Additional statistics per exporter are printed with number of flows, packets and sequence errors. ๐-xflowfile โ Scans and prints extension maps located in flowfile ๐บ๏ธ-Dโ Deprecated nameserver option. It is accepted for compatibility and ignored; name resolution follows the system resolver configuration. ๐ต (she's retired but still shows up to the party)-jโ Legacy shorthand for-z=bz2.-yโ Legacy shorthand for-z=lz4.-z=lzoโ Compress flow files with LZO1X-1 compression. Fastest compression. โก-z=bz2โ Compress flow files with bz2 compression. Slow but most efficient. May be used for archiving files or if you are really short of space. ๐๏ธ-z=lz4[:level]โ Compress flow files with LZ4 compression. Fast and efficient. Optional level should be between1..10. Changing the level results in smaller files but uses up more time to compress. Levels > 5 may need more workers. See-W. ๐จ-z=zstd[:level]โ Compress flow files with ZSTD compression. Fast and efficient. Optional level should be between1..10. Changing the level results in smaller files but uses up more time to compress. Levels > 5 may need more workers. See-W. ๐ธ-W numโ sets how many worker threads you get. the default depends on how many cores are online, and the value must not exceed the number of online CPU cores, periodt.
worker threads are what compress data blocks when you're writing flow files with -w file. using multiple workers can significantly improve throughput, especially when using bz2 compression or higher compression levels for lz4 or zstd. slay.
worker threads also get used when filtering flows with a given filter expression. increasing the number of workers may improve performance for large or complex filters. ngl that's the good stuff.
heads up bestie: nfdump uses additional internal threads for reading and processing files. those ones are not user-configurable โ not yours to touch, iykyk.
-J compressโ changes the compression for any number of files given by option-r flowpath. set compress to0for no compression, or to any of:1orLZO,2orBZ2,3orLZ4,4orZSTD. this one's for archiving flow files and changing the compression so you use less disk space ๐-Xโ compiles the filter syntax and dumps the filter engine table to stdout. this is for debugging purpose only, so likeโฆ don't make it your whole personality ๐-Zโ check the filter syntax and exit. sets the return value accordingly. the vibe check for your filter โจ-R filelistโ select a range of files. this option is mainly used by old NfSen and documented here as legacy option. the forms go like this:/any/dirโ read recursively all files in directorydir./dir/fileโ read all files beginning withfile./dir/file1:file2โ read all files fromfile1tofile2.- when using in combination with a sub hierarchy:
/dir/sub1/sub2/file1:sub3/sub4/file2โ read all files fromsub1/sub2/file1tosub3/sub4/file2, iterating over all required hierarchy levels.
note: files are read in alphabetical order. no cap, that's just how it goes ๐ซถ
-M dirlistโ read the same file hierarchy from multiple directories. this option is mainly used by old NfSen and documented here as legacy option. example:/any/path/to/dir1:dir2:dir3etc. and it will be expanded to the directories:/any/path/to/dir1,/any/path/to/dir2and/any/path/to/dir3. any number of colon separated directories may be given. a path ending with a wildcard@such as/any/path/to@will expand automatically in a list of all existing sub directories/any/path/to/dir1:dir2:dir3. the files to read are specified by-ror-Rand are expected to exist in all the given directories.
โ ๏ธ the options-rand-Rmust not contain any directories when used in combination with-M. literally don't do it.
-Tโ tag IP addresses with a prependingcntrl-Acharacter, to allow output parsers to hook in. this option is mainly used by old NfSen and documented here as legacy option. retro icon behavior ๐-Vโ printnfdumpversion and exit. short and sweet, we love a quick flex ๐-hโ print help text on stdout with all options and exit. your little map of everything ๐
๐ฆ THE FILTER ๐ฆ
okay so filter selects which records will be further processed. if no filter is given, all records will be processed. otherwise, only those flows matching the filter will be processed. any IP address in a filter may be specified as IPv4 or IPv6 โ flexible, we stan ๐
the filter syntax is similar to tcpdump but adapted and extended for flow records. the filter can be either specified on the command line after all options, or in a separate file. it can span several lines. anything after a # is treated as a comment and ignored to the end of the line. there is virtually no limit in the length of the filter expression. all keywords are case insensitive, bestie โ type however you want โจ
a single filter primitive filters a single element of a flow record. a filter consists of one or more primitives, which are linked together:
expr and expr
expr or expr
not expr and (expr)
in all expressions, where a number is a valid argument, the number may be given as a normal decimal number or as a hex number prefixed by 0x such as 0x22. a decimal number may also contain a multiplication factor such as K, M, G, T which multiplies the number by the corresponding factor. for example 1K, 2G etc. math girlies rise ๐งฎ
string arguments may be single or double quoted or not quoted at all, if the string is not a reserved filter key word โ src geo CH but src geo 'IN'
๐ธ Possible filter primitives ๐ธ
@include fileโ expands the content offileinto the current filter. like a lil copy-paste but make it legal ๐anyโ matches every flow record. it is useful as an explicit always-true term in a generated filter. the yes-girl of primitives ๐ซถcount comp numberโ true if the comparison with the record counter matchesnumber. each record gets assigned a record number at the time it is read from file. therefore this record number is not unique and may change, depending on the order files are read. so don't come for it ๐ญident stringโ true if the recordidentfield matchesstring. this filter can be used to filter out different sources.ipv4โ true if source and destination IP of a record are IPv4 IPs.ipv6โ true if source and destination IP of a record are IPv6 IPs.min ttl comp num,max ttl comp numโ true ifmin/maxIP ttl matches comparison.ttl comp numโ true if min or max IP ttl matches comparison.ttl equalโ true if min and max IP ttl have the same value. twins! ๐ฏetherType numberโ true if the Ethernet type field matchesnumber.fwdstat status,fwdstat comp numberโ true if the forwarding-status field matches its numeric value or a supported status name.proto protocolโ true if the record protocol field matchesprotocol.protocolcan be a string such astcp,udp,icmp,ah,esp,ipip, and many more, or a protocol number, such as6,17for protocoltcpandudp.tun proto protocolโ true if the record tunnel protocol field matchesprotocol.protocolmay be a string or protocol number.ip ipaddr,src ip ipaddr,dst ip ipaddrโ true if the respective IP field of the record matchesipaddr.ipaddrmay be an IPv4 or IPv6 address or a symbolic hostname. in this case a DNS lookup resolves the hostname to one or more IP addresses. if more than one IP results, all IPs are chained together in an or chain:(IP or IP or IP). ifipis not specified withsrcordstthe source or destination IP may match. ifipaddris set totorthen flows are listed, if the respective ip addr is a to tor exit not. for this filter to work, you need a workingnftordb. very main character of it ๐ฅhost ipaddrโhostis just a synonym forip(see above). same energy, different name ๐ip in [iplist],src ip in [iplist],dst ip [iplist]โ true if the respective IP field of the record is iniplist.iplistis a space or,separated list of IP addresses or networks in CIDR notation. this is the preferred way to search in large list of IP addresses and networks and is much more efficient than to chain all IP addresses together:(IP1 or IP2 or IP3). theiplistmay contain several hundreds to thousand IPs and/or networks. for just a few IPs use an or chain, otherwise use aniplist. ifipis not specified withsrcordstthe source or destination IP may match. efficiency queen ๐net network netmask,src net network netmask,dst net network netmask,net network/netbits,src net network/netbits,dst net network/netbitsโ true if the respective IP field of the record matches the network if the correspondingnetmaskornetbitsare applied to the IP address. ifnetis not specified withsrcordstthe source or destination IP may match.geo string,src geo string,dst geo stringโ true, if the 2-letter country code resolved by geolookup of the source or destination IP address matchesstring. this filter works only, if a validgeoDBis specified. see geo location option above. the 2-letter country code corresponds to the maxmind DB definitions. ifgeois not specified withsrcordstthe source or destination geo location code may match.
๐ please note: country codes which match nfdump filter language reserved words such asIN,LTetc must be explicitly quoted to be recognized as string. quote it or it's not happening, bestie.
tun ip ipaddr,src tun ip ipaddr,dst tun ip ipaddrโ true if the respective tunnel IP field of the record matchesipaddr. iftun ipis not specified withsrcordstthe source or destination tunnel IP may match.port comp num,src port comp num,dst port comp numโ true if the comparison of the respective port field matchesnum. seecompfor the comparator details. ifportis not specified withsrcordstthe source or destination port may match.port in [portlist],src port in [portlist],dst port in [portlist]โ true if the respective port field of the record is inportlist.portlistis a space or,separated list of port numbers. this is the preferred way to search in large list of port numbers and is much more efficient than to chain all ports together:(PORT1 or PORT2 or PORT3).portlistmay contain several hundreds to thousand of port numbers. ifportis not specified withsrcordstthe source or destination port may match. ๐icmp type num,icmp code numโ true if the respective icmp field of the record matchesnum. this automatically impliesproto icmp.engine type comp num,engine id comp numโ true if the respective exporter engine field matchesnum(0through255).exporter id comp num,exporter sysid comp numโ true if the collector exporter ID matchesnum(0through65535).
Okay so the filter primitives keep rolling, bestie, and they are lowkey iconic:
if num,in if num,out if numโ hitstruewhen the respective interface fields of the record matchnum. This ID may correspond to the SNMP ID of the interface but depends on the exporter. Ififisn't specified withinorout, the input or output interface may match.as comp num,src as comp num,dst as comp num,prev as comp num,next as comp numโ hitstruewhen the comparison of the respective AS fields matchesnum. nfdump supports 32-bit AS numbers every where. Withoutsrc,dst,prevornextthe source or destination AS may match. Seecompfor the comparator details.as in [aslist],src as in [aslist],dst as in [aslist],prev as in [aslist],next as in [aslist]โ hitstruewhen the respective AS field of the record is inaslist.aslistis a space or','separated list of AS numbers. This is the preferred way to search in large list of AS numbers and is much more efficient than to chain all ports together.aslistmay contain several hundreds to thousand of AS numbers. Ifasis not specified withsrc,dst,prevornextthe source or destination AS may match.mask bits,src mask bits,dst mask bitsโ hitstruewhen the respective mask bit field of the record matchesbits. Ifmaskis not specified withsrcordstthe source or destination mask bits may match.vlan num,src vlan num,dst vlan numโ hitstruewhen the respective vlan field of the record matchesnum. Ifvlanis not specified withsrcordstthe source or destination vlan may match.flags tcpflagsโ hitstruewhen the respective tcp flags field of the record matches any of the giventcpflags.tcpflagsis a string combination of all flags to be tested:Aโ ACK.Sโ SYN.Fโ FIN.Rโ Reset.Pโ Push.Uโ Urgent.Xโ All flags on.
The order of the flags within tcpflags is not relevant. Flags not mentioned are treated as don't care. In order to get those flows with only the SYN flag set, use the syntax:
flags S and not flags AFRPU
router ip ipaddrโ hitstruewhen the ip address of the sending router matchesipaddras valid IPv4/IPv6 address.next ip ipaddrโ hitstruewhen the fieldnext-ipof the record matchesipaddras valid IPv4/IPv6 address.bgp next ip ipaddrโ hitstruewhen the fieldbgpnext-ipof the record matchesipaddras valid IPv4/IPv6 address.mac macaddr,in mac macaddr,in src mac macaddr,in dst mac macaddr,out mac macaddr,out src mac macaddr,out dst mac macaddrโ hitstruewhen the respective mac address field of the record matchesmacaddr. By prependingmacwith any combination of a direction specifier as defined by CISCO v9 the test is limited to those mac addresses only. Otherwise multiple matches are possible. Without any specifiers any mac address is tested againstmacaddr.mpls labelN comp numberโ hitstruewhen the comparison of the mpls label N with N as mpls label number 1..10 matchesnumber. Filters according a specific number in the mpls label stack.mpls eos comp numberโ hitstruewhen the comparison of the end of stack mpls label matchesnumber.mpls expN comp numberโ hitstruewhen the comparison of the experimental bits 0..7 of mpls label N with N as mpls label number 1..10 matchesnumber.packets comp num,in packets comp num,out packets comp numโ hitstruewhen the comparison of the packet counter in the flow record matchesnum.nummay contain any valid scaling factor such ask,m,g. Example:packets > 1k. For a single flowpacketsandin packetsis equivalent and describes the number of packets from source to destination. In case of a bi-directional flow (sent by an exporter or combined by option--B) the packet counter for the reverse flow can be tested without packet.bytes comp num,in bytes comp num,out bytes comp numโ hitstruewhen the comparison of the byte counter in the flow record matchesnum.nummay contain any valid scaling factor such ask,m,g. Example:bytes > 1k.bytesandin bytesis equivalent and describes the number of bytes from source to destination. In case of a bi-directional flow (sent by an exporter or combined by option--B) the byte counter for the reverse flow can be tested without bytes.flows comp numโ hitstruewhen the comparison of the flow counter in the flow record matchesnum.nummay contain any valid scaling factor such ask,m,g. For each received flow, the flow counter is set to 1, unless the exporter sends this information. If multiple flows are aggregated, this counter is increased respectively.tos numโ hitstruewhen the type of service field of the flow record matchesnum.flowdir directionโ hitstruewhen the flow direction field in the flow record matchesdirection.directionmay beingress,egress,0for ingress, or1for egress.duration comp timeโ hitstruewhen the calculated duration of a flow(tend - tstart)compares totime. The duration is specified in msec (milliseconds).first seen comp timeISO8601,last seen comp timeISO8601โ hitstruewhen the start/end time of a flow compares totimeISO8601. The time follows ISO8601 format. e.g.2024-07-11T09:15:10.020. Up to msec may be specified. A string may be incomplete e.g.2024-07-11T09which is automatically extended to2024-07-11T09:00:00.000. At least a valid year is required.pps comp numโ hitstruewhen the calculated value ofin-packets/duration(packets per second) compares with the numbernum.nummay contain any valid scaling factor such ask,m,g.bps comp numโ hitstruewhen the calculated value of8*in-bytes/duration(bits per second) compares with the numbernum.nummay contain any valid scaling factor such ask,m,g.bpp comp numโ hitstruewhen the calculated value ofin-bytes/in-packets(bytes per packet) compares with the numbernum.nummay contain any valid scaling factor such ask,m,g.observation domain id comp number,observation point id comp numberโ hitstruewhen the comparison of the observation domain ID or point ID field respectively matchesnumber.payload filtersโ Some exporters, such asyafor the nfdump collectornfpcapcan send payload data along the netflow information. If such payloads are sent it can be filtered according the filter primitives below:payload content 'string'โ hitstruewhen the stringstringis found in the payload data.stringmust be quoted with single or double quotes:'string',โstringโ.payload regex 'regex',payload regex 'regex' flagsโ hitstruewhenregexmatches the payload data.regexsearches over the full payload length. A' 'byte does not stop the match process.regexmust be quoted with single or double quotes:'regex'orโregexโ. The regex engine understands the following reduced syntax:(...)โ subexpressions/capture ranges|โ the "or" operator^and$โ anchors.โ matches any single character[...]and[^...]โ character classes?,*,+โ simple quantifiers*?,+?,??โ lazy quantifiers{<num>},{<num1>,<num2>}โ complex quantifiers\dโ numeric character\wโ alpha numeric character\rโ carriage return\nโ new line
flags are optional and can be:
mโ multilineiโ case insensitive matchingsโ dot'.'includes newlines
payload dns definedโ hitstruewhen the payload decodes to a valid dns query/answer. Impliessrc or dst port 53.payload dns name dnsnameโ hitstruewhen any of the payload dns queries contain the stringdnsname. It may be the full name or a sub string. Impliessrc or dst port 53.payload dns address IPโ hitstruewhen any of the payload dns answers or additional records contain the stringIP. It may be the full or partial IP address. Impliessrc or dst port 53.payload ssl definedโ hitstruewhen the payload contains the start of a valid SSL/TLS handshake.payload ssl version versionโ hitstruewhen the payload contains the start of a valid SSL handshake and the SSL/TLS version matchesversion. Valid versions are1.0,2.0,3.0.payload tls versionversion โ true when the payload has the start of a legit TLS handshake and the TLS version matches version. Valid versions are1.0,1.1,1.2,1.3โจpayload tls snisniname โ true when the payload has the start of a legit TLS handshake and the TLS SNI name contains the string sniname ๐payload ja3md5string โ true when the payload has the start of a legit SSL/TLS handshake and the calculated ja3 value of that handshake matches md5string. Depending on whether it's a client or server SSL handshake, either ja3 or ja3s gets calculated ๐payload ja3 definedโ true when the payload has the start of a legit SSL/TLS handshake and a valid ja3 value can actually be calculated. Lowkey iconic for masking out all the flow records with no SSL/TLS traffic so you can generate a-s ja3statistic ๐ฆpayload ja4ja4string โ true when the payload has the start of a legit SSL/TLS handshake and the calculated ja4 value of the client TLS handshake matches ja4string ๐ธpayload ja4sja4Sstring โ true when the payload has the start of a legit SSL/TLS handshake and the calculated ja4 value of the server TLS handshake matches ja4Sstring ๐payload ja4 definedโ true when the payload has the start of a legit SSL/TLS handshake and a valid ja4/ja4s value can be calculated. Same serve as above: perfect for masking out all the flow records with no SSL/TLS traffic so you can generate a-s ja3or a-s ja3sstatistic โญpayload ja4s definedโ true if a valid JA4 server fingerprint can be calculated from the payload ๐
๐ก๏ธ OpenBSD pflog implemented elements
pf actionaction โ true when the respective pflog action field compares to one ofpass,block,scrub,noscrub,nat,nonat,binat,nobinat,rdr,nordr,synblock,defer,match,divert,rt,afrtpf reasonreason โ true when the respective pflog reason field compares to one ofmatch,bad-offset,fragment,short,normalize,memory,bad-timestamp,congestion,ip-option,proto-cksum,state-mismatch,state-insert,state-limit,src-limit,synproxy,translate,no-routepf ruleruleNr โ true when the respective pflog rule number field matches ruleNrpf dirin|outโ true when the respective pflog rule direction field matchesinoroutpf interfaceinterfaceName โ true when the respective pflog rule interface name field matches the string interfaceName
๐ nprobe implemented elements
client latencycomp timeserver latencycomp time
โ true when the respective latency field in the flow record compares to time. And heads up bestie, time is specified in msec โฑ๏ธ
๐ง CISCO ASA, network security event logging (NSEL) and NAT event logging (NEL) specific filters:
asa eventevent โ true if the NSEL event type of an event record matches event, which may be:ignore,create,term,delete,denyasa eventcomp number โ true if the comparison of the NSEL event type of an event record matches number as a number ๐ขasa event deniedreason โ true if the event denied type of an event record matches reason, which may beingress,egress,interface,nosynasa xeventcomp num โ true if the comparison of the extended event field of the event record matches numnat eventevent โ true if the NEL event type of an event record matches event. event may beadd,deletenat eventcomp number โ true if the comparison of the NEL event type of an event record matches number as a numbernat ipipaddrsrc nat ipipaddrdst nat ipipaddr
โ true if the field of the translated source or destination IP address matches ipaddr. If nat ip is specified without src or dst, both IP addresses may match ๐
nat portportsrc nat portportdst nat portport
โ true if the field of the translated source or destination port matches port. If port is specified without src or dst, both ports may match
nat netnetwork/masksrc nat netnetwork/maskdst nat netnetwork/mask
โ true if the translated source or destination IP address matches network when the mask mask is applied. If it's specified without src or dst, both IP addresses may match
nat pblock startcomp numbernat pblock stepcomp numbernat pblock endcomp numbernat pblock sizecomp number
โ true if the comparison of the start, step or end of the NAT port block in the event record matches number
port in nat pblocksrc port in nat pblockdst port in nat pblock
โ true if the source or destination port field matches the NAT port block range ๐ฏ
ingress ACLcomp numberingress ACEcomp numberingress XACEcomp number
โ true if the comparison of the respective ingress field matches number
egress ACLcomp number โ true if the comparison of the egress field matches numberegress ACEcomp numberegress XACEcomp number
โ true if the comparison of the respective egress access-control entry field matches number
ingress vrfnumber โ true if the field of the ingress vrf field of the event record matches numberegress vrfnumber โ true if the egress VRF field matches number
๐ comp (the little comparison legend)
So many filter elements support comparing against a number, and here's the whole menu of comparators supported for each of those filters:
=, ==, >, <, >=, <=
And because bash would otherwise snatch your symbols and start drama, there are alternative comparators available too:
EQ, LT, GT, LE, GE
If comp is just... omitted? No stress, == is assumed. Slay ๐ซถ
โจ OUTPUT FORMAT โจ
Okay bestie, this is the part where it gets custom. This section is literally all about how output formats get compiled. nfdump comes with a whole drawer full of already pre-defined output formats โ stuff like raw, json, ndjson, csv, etc. โ but the one-line formats you get with option -o can be compiled from various elements of a flow record ๐. And because a flow record contains so. many. different. elements, it's often genuinely useful to compile your own output format for whatever specific needs you have. Main character energy for your columns, basically ๐
.
๐ธ Format description ๐ธ
So the output format is specified by -o "fmt: string" for line output, or -o "csv: string" for csv output. The String contains the field tags to be printed, plus other characters if needed. A tag starts with a % sign followed by the field name. Tags are separated by spaces for line format, or by ',' for csv format. Characters or other strings that don't start with a % sign get copied literally to the output. Copy-paste realness, no cap โจ.
Example:
-o "fmt:%ts %td %pr %sap -> %dap %pkt %byt %fl"
-o "csv:%ts,%td,%pr,%sa,%sp,%da,%dp,%pkt,%byt,%fl"
The difference between fmt and csv is the printing of the header line. Also, csv does not have a flow summary at the end. csv output can be processed by any other post processing tool that understands csv data ๐ฆ. The examples above show the definitions of the predefined formats line and csv. They add the elements tstart, duration, protocol, source IP address/port, followed by the literal characters -> and destination ip address/port, packets, bytes, flows counter. Depending on the task, different output formats are required to see the required fields of a flow record. You can either extend a predefined format or specify a new one at the command line. Obsessed with the flexibility tbh ๐ญ.
Example: extend the predefined format long with the IP address of the sending router
-o "fmt:%long %ra"
Predefined formats can be extended by simply adding their name with a % sign somewhere in the format string โ as described under the output option -o. It's giving "stack them like bracelets" ๐.
๐ Format definition ๐
nfdump has already many formats predefined. Most of the time, these formats are good enough. Sometimes you may need different formats, which can be compiled as described above. And in order to prevent adding the same often-used output format each and every time you run nfdump, a new output format may be defined in the config file nfdump.conf โจ. The file nfdump.conf.dist contains the definition of the already hard coded formats. These may be uncommented and changed according to your specific needs. New formats may be added using the following syntax:
fmt.newname = "fmt:%ts %td %pr %sap -> %dap %pkt %byt %fl"
โฆwith newname being any new or existing definition of output formats. Existing formats are overwritten with the new definition ๐. You may also define a custom csv output format such as:
csv.newname = "csv:%tsr,%ter,%pr,%sa,%sp,%da,%dp,%pkt,%byt,%fl"
๐ Please note, thatnewnamemust be unique for all formats infmtas well ascsv. Pick a name, any name โ but only one.
๐ฆ Tag definition ๐ฆ
The following list contains all tags which are available to compile the output format. Unless noted otherwise, every tag below is available in both fmt and csv format. But heads up โ the composite tags that combine several values into a single field (%sap, %dap, %gsa, %gda, %gsap, %gdap, %gra, %ttl, %ipl, %opl, %n) are available in fmt format only ๐. Noted! Noted!
%<format>โ Inserts the predefined format at this position. e.g.%line%cntโ Record counter. record numbers are assigned dynamically assigned while reading read from file.%idtโ Ident string. Source identifier string-ito nfcapd and constant for all flows in a file.%nfvโ Netflow version.%tsโ Start Time - first seen%tfsโ First seen - identical to%ts%tsrโ Start Time, but in fractional seconds since the epoch (1970-01-01) UNIX format.%tsgโ Start Time GMT - first seem%teโ End Time - last seen%terโ End Time, in fractional seconds%tegโ End Time GMT - last seen%trโ Time the flow was received by the collector%trrโ Time the flow was received, in fractional seconds%trgโ Time GMT the flow was received by the collector%tdโ Duration of flow. Displayed inddHHMMSS.msec%tdsโ Duration of flow in seconds.%prโ Transport protocol%expโ Exporter ID%engโ Engine Type/ID%lblโ Flowlabel%saโ Source Address%daโ Destination Address%sapโ Source Address:Port%dapโ Destination Address:Port%gsapโ Source Address(country code):Port%gdapโ Destination Address(country code):Port%gsaโ Source address annotated with geographic data.%gdaโ Destination address annotated with geographic data.%spโ Source Port%dpโ Destination Port%minttlโ IP min ttl%maxttlโ IP max ttl%ttlโ IP min/max ttl%itโ ICMP-type%icโ ICMP-code%snโ Source Network, mask applied%dnโ Destination Network, mask applied%nhโ Next-hop IP Address%nhbโ BGP Next-hop IP Address%raโ Router IP Address%graโ Router IP address annotated with geographic data.%sasโ Source AS%dasโ Destination AS%nasโ Next AS%pasโ Previous AS%inโ Input Interface num%outโ Output Interface num%inamโ Input interface name.%onamโ Output interface name.%pktโ Packets - default input%ipktโ Input Packets%opktโ Output Packets%bytโ Bytes - default input%ibytโ Input Bytes%obytโ Output Bytes%flโ Flows%flgโ TCP Flags%tosโ Tos - default src%stosโ Src Tos%dtosโ Dst Tos%dirโ Direction: ingress, egress%bfdโ Bidirectional-flow direction.%endโ Flow-end reason.%smkโ Src mask%dmkโ Dst mask%fwdโ Forwarding Status%svlnโ Src vlan label%dvlnโ Dst vlan label%ismcโ Input Src Mac Addr%odmcโ Output Dst Mac Addr%idmcโ Input Dst Mac Addr%osmcโ Output Src Mac Addr%mpls1โ MPLS label 1%mpls2โ MPLS label 2%mpls3โ MPLS label 3%mpls4โ MPLS label 4%mpls5โ MPLS label 5%mpls6โ MPLS label 6%mpls7โ MPLS label 7%mpls8โ MPLS label 8%mpls9โ MPLS label 9%mpls10โ MPLS label 10%mplsโ MPLS labels 1-10%bpsโ bps - bits per second%ppsโ pps - packets per second%bppโ bps - Bytes per package%scโ src IP 2 letter country code%dcโ dst IP 2 letter country code%slocโ src IP geo location info%dlocโ dst IP geo location info%sasnโ src AS organisation name%dasnโ dst AS organisation name%storโ src IP 2 letter tor exit info: TX tor exit node%dtorโ dst IP 2 letter tor exit info: TX tor exit node%nโ new line char\n%iplโ input payload%oplโ output payload%nbidโ nbar ID%ja3โ ja3 hash%ja4โ JA4 fingerprint.%sniโ sni name in tls handshake%tlsโ TLS version.%nbnamโ nbar name%odidโ observation domainID%opidโ observation pointID
๐ OpenBSD pflog specific formats ๐
%pfifnโ pflog interface name%pfactโ pflog action%pfreaโ pflog reason%pfdirโ pflog direction%pfruleโ pflog rule nr
๐ NSEL specific formats ๐
%nfcโ NSEL connection ID%evtโ NSEL event%tevtโ NSEL/NAT event timestamp.%xevtโ NSEL extended event%msecโ NSEL event time in msec%iaclโ NSEL ingress ACL%eaclโ NSEL egress ACL%xsaโ NSEL XLATE src IP address%xdaโ NSEL XLATE dst IP address%xspโ NSEL XLATE src port%xdpโ NSEL SLATE dst port%xsapโ Xlate Source Address:Port%xdapโ Xlate Destination Address:Port%unameโ NSEL user name
๐ซถ NEL/NAT specific formats ๐ซถ
%ivrfโ NAT ingress VRF ID%vrfโ Ingress VRF ID (compatibility alias for%ivrf).%ivrfnamโ Ingress VRF name.%evrfโ NAT egress VRF ID%evrfnamโ Egress VRF name.%nsaโ NAT src IP address%ndaโ NAT dst IP address%nsapโ NAT translated source address and port.%ndapโ NAT translated destination address and port.%nspโ NAT src port%ndpโ NAT dst port%pbstartโ NAT pool block start%pbendโ NAT pool block end%pbstepโ NAT pool block step%pbsizeโ NAT pool block size%flidโ Flow ID%isidโ Nokia NAT in service ID%osidโ Nokia NAT out service ID%natsโ Nokia NAT string
๐ Nprobe formats ๐
%clโ Client latency%slโ Server latency%alโ Application latency
โญ Additional core tags โญ
%ethโ Ethernet type.%fragโ IP fragment flags.
๐ EXIT STATUS ๐
Short and sweet, icon behaviour honestly: nfdump returns 0 on success and 255 if processing failed ๐. That's it. That's the whole vibe check.
โจ EXAMPLES โจ
okay bestie, this is the part where it all clicks ๐ real commands, real vibes, copy these and go off ๐
nfdump reads files made by any previous version of nfdump 1.6.x โ with some limitations for versions earlier than 1.6.17 (iykyk, the old ones are a little crusty ๐ญ). so if you wanna convert flow files over to the new 1.7.x binary format, you literally just read and write, like this:
% nfdump -r oldfile -w newfile
wanna print a stat about the top 20 IP addresses, and do it twice โ once sorted by flows, once sorted by bytes? slay, here:
% nfdump -r flowfile -s ip/flows/bytes -n 20
print two statistics, one about the source IP and one about the destination IP address, limited to flows with either source or destination port 443 ๐:
% nfdump -r flowfile -s srcip/bytes -s dstip/bytes -n 20 'port
443'
print a statistic about the IP pairs that exchanged the most traffic ๐ the -A srcip,dstip is doing the heavy lifting here:
% nfdump -r flowfile -s record/bytes -A srcip,dstip
print all flows in raw format where there's an HTTP header in the payload โ even if the flow is not on port 80 ๐ (yes, ports lie, we don't):
% nfdump -r flowfile -o raw โpayload regex 'GET|POST'โ
print a statistic about all the ja3 md5 sums for the flows where a valid ja3 can actually be calculated โจ note the -n 0, which is the "don't cap me" energy:
% nfdump -r flowfile -s ja3 -n 0 'payload ja3 defined'
aggregate all flows and write the result back out to a binary file, sorted by the start time ๐:
% nfdump -r flowfile -a -Otstart -w newfile
๐ SEE ALSO ๐
- https://www.iana.org/assignments/ipfix/ipfix.xhtml
- https://datatracker.ietf.org/doc/html/rfc3954
nfcapd(1)nfpcapd(1)sfcapd(1)geolookup(1)
๐ BUGS ๐
no software without bugs, bestie โ not even this one ๐ญ. please report any bugs back to me. that's it, that's the whole section. periodt.
and that's a wrap on the whole page ๐ซถ this one's the Debian build, dated $Mdocdate$, and it's NFDUMP(1) โ go forth and dump those flows, you absolute icon ๐
โจ
๐ฆ where this page comes from
- epel 10: nfdump 1.7.10-1.el10_3 ยท BSD-3-Clause AND GPL-2.0-or-later
- Copyright (c) 2024, Peter Haag
- ๐ girlified by deepseek-ai/DeepSeek-V4.1-Flash-1017 on : a language model's rewrite of the original, under its terms