nfstrace(8)

📦
version 0.4.3.2+git20200805+b220d04 · 3 versions
, in debian trixie, ubuntu noble, ubuntu questing

💖 nfstrace(8) 💖

okay so this is the manual page for nfstrace, straight out of the System Manager's Manual section, and bestie it is giving network-forensics main character energy. let's get into it 🫶

✨ NAME ✨

nfstrace — NFS and CIFS tracing/monitoring/capturing/analyzing tool 💅

🌸 SYNOPSIS 🌸

nfstrace [ -m drain|live|dump|stat ] [ -i interface ] [ -f filter ]
         [ -s 1..65535 ] [ -t Milliseconds ] [ -b MBytes ] [ -p promisc ]
         [ -d in|out|inout ] [ -a path#opt1,opt2=val,... ] [ -I path ]
         [ -O path ] [ -C 'shell command' ] [ -D MBytes ] [ -E interfaces|plugins ]
         [ -M 1..4000 ] [ -Q 1..65535 ] [ -T true|false ] [ -Z username ]
         [ -v 0|1|2 ]

🦋 DESCRIPTION 🦋

so like, nfstrace does live Ethernet 1 Gbps – 10 Gbps packets capturing and it helps you figure out which NFS and CIFS procedures are hiding in your raw network traffic. and that's not even all of it — it also does filtration, dumping, compression, statistical analysis, visualization, and it hands you the API for custom pluggable analysis modules. she does it all, no cap 👑

nfstrace captures raw packets off an Ethernet interface using the libpcap interface to Linux (LSF) or FreeBSD (BPF) implementations. rn it's assumed that libpcap delivers correct TCP and UDP packets. the assembling of IP packets from ethernet frames and the IP packets defragmentation? those happen in the operating system's kernel, not here. 💅

the application has been tested on workstations with integrated 1 Gbps NICs (Ethernet 1000baseT/Full). iconic.

rn nfstrace supports these protocols:

Ethernet > IPv4 | IPv6 > UDP | TCP > NFSv3 | NFSv4 | NFSv4.1 | CIFSv1 | CIFSv2

and she can operate in four different modes, bestie:

  • online analysis (--mode=live): performs online capturing, filtration and live analysis of detected NFS procedures using a pluggable analysis module, or prints them out to stdout (-T or --trace options).
  • online dumping (--mode=dump): performs online traffic capturing, filtration and dumping to the output file (specified with -O or --ofile options).
  • offline analysis (--mode=stat): performs offline filtration of the .pcap file that contains previously captured traces, and performs analysis using a pluggable analysis module, or prints found NFS procedures to stdout (-T or --trace options).
  • offline dumping (--mode=drain): performs a reading of traffic from the .pcap file (specified with -I or --ifile options), filtration, dumping to the output .pcap file (specified with -O or --ofile options) and removing all the packets that are not related to NFS procedures. bye irrelevant packets 🗑️

nfstrace uses the libpcap file format for input and output files, so any external tool (e.g. Wireshark) can be used to inspect filtered traces. so yes, you can still go look at it in Wireshark, we're not gatekeeping 🫶

since nfstrace internally uses libpcap, which gives a portable interface to the native system API for capturing network traffic, filtration is performed in the operating system's kernel. nfstrace gives you a special option (-f or --filtration) for specifying custom filters in BPF syntax (see pcap-filter(7)).

the default BPF filter in nfstrace is 'port 2049 or port 445', which means each packet delivered to user-space from the kernel satisfies the following conditions: it has an IPv4 or IPv6 header, and it has a TCP and UDP header with source or destination port number equal to 2049 (default NFS port) or 445 (default CIFS port).

but real talk bestie — the reality is that this filter is very heavy and support of IPv6 is experimental. so if you want faster filtration of IPv4-only traffic, we suggest you use this BPF filter instead: 'ip and port 2049 or port 445'. ✨

💅 OPTIONS 💅

nfstrace can usually be run without arguments — in that case, default arguments will be used. easy like that 😌

  • -m, --mode=live|dump|drain|stat — set the running mode (see the description above) (default: live).
  • -i, --interface=INTERFACE — the listen interface, it is required for live and dump modes (default: searches for the lowest numbered, configured up interface (except loopback)).
  • -f, --filtration=filter — specify the packet filter in BPF syntax; for the expression syntax, see pcap-filter(7) (default: 'port 2049 or port 445').
  • -s, --snaplen=1..65535 — set the max length of captured raw packet (bigger packets will be truncated). can be used only for UDP (default: 65535).
  • -t, --timeout=milliseconds — set the read timeout that will be used while capturing (default: 100).
  • -b, --bsize=MBytes — set the size of the operating system capture buffer in MBytes; note that this option is crucial for capturing performance (default: 20). 🔥
  • -p, --promisc — put the capturing interface into promiscuous mode (default: true).
  • -d, --direction=in|out|inout — set the direction for which packets will be captured (default: inout).
  • -a, --analysis=PATH#opt1,opt2=val,... — specify the path to an analysis module and set its options (if any).
  • -I, --ifile=PATH — specify the input file for stat mode, '-' means stdin (default: nfstrace-{filter}.pcap).
  • -O, --ofile=PATH — specify the output file for dump mode, '-' means stdout (default: nfstrace-{filter}.pcap).
  • --log=PATH — specify the log file (default: nfstrace.log.{timestamp}).
  • -C, --command='shell command' — execute command for each dumped file.
  • -D, --dump-size=MBytes — set the size of the dumping file portion, 0 means no limit (default: 0).
  • -E, --enum=interfaces|plugins — enumerate all available network interfaces and and/or all available plugins, then exit; please note that interfaces can't be listed unless nfstrace was built against the recent version of libpcap that supports the pcap_findalldevs() function (default: none).
  • -M, --msg-header=1..4000 — truncate RPC messages to this limit (specified in bytes) before passing to a pluggable analysis module (default: 512).
  • -Q, --qcapacity=1..65535 — set the initial capacity of the queue with RPC messages (default: 4096).
  • -T, --trace — print collected NFSv3 or NFSv4 procedures, true if no modules were passed with the -a option.
  • -Z, --droproot=username — drop root privileges after opening the capture device.
  • -v, --verbose=0|1|2 — specify verbosity level (default: 1).
  • -h, --help — print help message and usage for modules passed with the -a option, then exit.

🧚 ANALYSIS MODULES 🧚

🍓 Operation Breakdown Analyzer

the Operation Breakdown (OB) analyzer calculates the average frequency of NFS and CIFS procedures and computes the standard deviation of latency. math girl summer 💅

$ nfstrace -a libbreakdown.so -h
nfstrace 0.4.0 (Release)
built on Linux-3.16.1-1-generic
by C++ compiler GNU 4.9.1
Usage: ./nfstrace [OPTIONS]...

Usage example:

    $ nfstrace -m stat -a libreakdown.so

the breakdown analyzer produces a .dat file in the current directory for each detected NFS session, which can be visualized using nst.sh and breakdown_nfsv3.plt or breakdown_nfsv4.plt (according to NFS version).

$ nst.sh -a breakdown_nfsv4.plt -d . -p
'breakdown_10.6.137.47:903*.dat'

🎀 Watch

the Watch plugin mimics the old nfswatch utility: it monitors NFS and CIFS traffic and displays it in the terminal using ncurses. it supports NFSv3, NFSv4, NFSv41, CIFSv1 and CIFSv2.

by default the watch plugin will update its screen every second, but you can specify another timeout in milliseconds:

$ nfstrace -a libwatch.so#2000

🌈 JSON Analyzer

the JSON analyzer calculates a total amount of each supported application protocol operation. it accepts TCP-connections on a particular TCP-endpoint (host:port), sends a respective JSON to the TCP-client and closes the connection. suggested to be used in live mode.

available options:

  • host=hostname — network interface to listen (default is to listen all interfaces).
  • port=port — IP-port to bind to (default: 8888).
  • workers=workers — amount of worker threads (default: 10).
  • duration=duration — max serving duration in milliseconds (default: 500).
  • backlog=backlog — listen backlog (default: 15).

💜 Example of use

start nfstrace in one console:

$ sudo nfstrace -i eth0 -a analyzers/libjson.so#host=localhost
...

then make a TCP-request to nfstrace in another console to fetch current statistics:

$ telnet localhost 8888
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
{
  "nfs_v3":{
    "null":32,
    "getattr":4582,
    ...
  },
  "nfs_v4":{
    ...
  },
  ...
}Connection closed by foreign host.

🎀 EXAMPLES 🎀

okay bestie, this is where it gets <em>fun</em> — real commands, real vibes, zero gatekeeping. Let's go. 💅

✨ Available options ✨

This command shows you all the available options of the application and any plugged-in analysis modules (you attach those with --analysis or -a). And yes, you can pass more than one module here. Slay.

nfstrace --help --analysis=libjson.so

🛰️ Online tracing 🛰️

This one runs nfstrace in online analysis mode (that's what --mode or -m sets) with no pluggable analysis module attached. It captures NFS and CIFS traffic transferred over TCP or UDP where the source or destination port number equals 2049, then just prints it all out to stdout (that's -T or --trace). Capturing ends when nfstrace receives SIGINT (Control-C). Iconic behavior, honestly.

Note: capturing from a network interface requires <strong>superuser privileges</strong>. No cap.
nfstrace --mode=live --filtration='ip and port 2049' --trace

🧠 Online analysis 🧠

This command runs nfstrace in online analysis mode. Just like the previous example, it captures NFS and CIFS traffic over TCP or UDP with source or destination port number equals to 2049, and then it performs Operation Breakdown analysis using the pluggable analysis module libbreakdown.so. Main character energy.

nfstrace -m live -f 'ip and port 2049' --analysis=libbreakdown.so

💾 Online dumping and offline analysis 💾

This example runs nfstrace in online dumping and offline analysis modes.

At first, nfstrace captures NFS traffic transferred over TCP or UDP with source or destination port number equals to 2049 and dumps the captured packets to the dump.pcap file (specified with --ofile or -O options).

At the second run, nfstrace performs offline Operation Breakdown analysis using the pluggable analysis module libbreakdown.so. Two runs, one slay.

# Dump captured packets to dump.pcap
nfstrace -m dump -f 'ip and port 2049' --ofile=dump.pcap

# Analyse dump.pcap using libbreakdown.so
nfstrace -m stat --ifile=dump.pcap -a libbreakdown.so

🗜️ Online dumping, compression and offline analysis 🗜️

This example runs nfstrace in online dumping and offline analysis modes. Since a dump file can easily exhaust disk space, compression makes sense. We love a space-saving queen.

At first, nfstrace captures NFS traffic transferred over TCP or UDP with source or destination port number equals to 2049 and dumps captured packets to the dump.pcap file.

Note that compression is done by the external tool (executed in a script passed with --command or -C options) and it will be executed when capturing is done. The output file can be inspected using some external tool.

At the second run, nfstrace performs offline analysis. Again, the external tool (bzcat in this example) is used in order to decompress the previously saved dump. nfstrace will read stdin (note the -I – option) and perform offline analysis using the Operation Breakdown analyzer.

# Dump captured procedures to dump.pcap file.
# Compress output using bzip2 when capturing ends.
nfstrace -m dump -f 'ip and port 2049' -O dump.pcap --command 'bzip2 -f -9'

# Extract dump.pcap from dump.pcap.bz2 to stdin.
# Read stdin and analyze data with libbreakdown.so module.
bzcat dump.pcap.bz2 | nfstrace -m stat -I - -a libbreakdown.so

🍰 Online dumping with file limit, compression and offline analysis 🍰

This example is similar to the previous one except for one thing: the output dump file can be very huge and cause problems in some situations, so nfstrace provides the ability to split it into parts. Big file? Not on our watch.

At first, nfstrace is invoked in online dumping mode. Everything is similar to the previous example except for the -D (--dump-size) option: it specifies the size limit in MBytes, so the dump file will be split according to this value.

At the second run, nfstrace performs offline analysis of captured packets using the Operation Breakdown analyzer.

Please note that <strong>only the first dump file has the pcap header</strong>. Important detail, bestie!

# Dump captured procedures multiple files and compress them.
nfstrace -m dump -f 'ip and port 2049' -O dump.pcap -D 1 -C bzip2 -f -9

# get list of parts in the right order:
# dump.pcap.bz2
# dump.pcap-1.bz2
# dump.pcap-2.bz2
parts=$(ls dump.pcap*.bz2 | sort -n -t - -k 2)

# Extract dump.pcap from main dump.pcap.bz2 and parts to stdin.
# Read stdin and analyze data with libbreakdown.so module.
bzcat $parts | nfstrace --mode=stat -I - --analysis=libbreakdown.so

📈 Visualization 📈

This example demonstrates the ability to plot a graphical representation of data collected by the Operation Breakdown analyzer.

nst.sh is a shell script that collects data generated by analyzers and passes it to a gnuplot script specified with the -a option.

breakdown.plt is a gnuplot script that understands the output data format of the Operation Breakdown analyzer and generates .png files with plots.

Note: gnuplot must be installed. <em>No gnuplot, no plot, no vibe.</em>
# Extract dump.pcap from dump.pcap.bz2 to stdin.
# Read stdin and analyze data with libbreakdown.so module.
bzcat trace.pcap.bz2 | nfstrace -m stat -I - -a libbreakdown.so

# Generate plot according to *.dat files generated by
# libbreakdown.so analyzer.
nst.sh -a breakdown.plt -d . -p 'breakdown*.dat' -v

📂 FILES 📂

🔌 API headers 🔌

/usr/include/nfstrace/api/

🧩 Pluggable Analysis Modules (PAMs) 🧩

/usr/lib/nfstrace/

🐞 BUGS 🐞

Any problems, bugs, questions, desirable enhancements, etc. should be sent to <nfstrace@epam.com>. They're listening, bestie.

👑 AUTHORS 👑

  • Vitali Adamenka — <Vitali_Adamenka@epam.com>
  • Yauheni Azaranka — <Yauheni_Azaranka@epam.com>
  • Alexey Costroma — <Alexey_Costroma@epam.com>
  • Dzianis Huznou — <Dzianis_Huznou@epam.com>
  • Pavel Karneliuk — <Pavel_Karneliuk@epam.com>
  • Andrey Kuznetsov — <Andrey_Kuznetsov@epam.com>
  • Mikhail Litvinets — <Mikhail_Litvinets@epam.com>
  • Ilya Storozhilov — <Ilya_Storozhilov@epam.com>

Legends, every single one. 🫶

👀 SEE ALSO 👀

bpf(4), pcap(3), nfsstat(8), pcap-filter(7).

And that's the whole page, bestie! This was nfstrace(8), version 0.4.3, dated 2024-04-07. Periodt. 💅✨

📦 where this page comes from