fprobe(8)

๐Ÿ’– fprobe โ€” a NetFlow probe ๐Ÿ’–

okay bestie, sit down, hydrate, because this is the fprobe(8) page and it's lowkey iconic. it lives in the System Manager's Manual, so yeah, section 8, the big-leagues section, and it's here to tell you everything about this little traffic-gobbling cutie. ๐Ÿฆ‹


โœจ NAME โœจ

fprobe โ€” a NetFlow probe. that's it. that's the whole name. she's simple, she's elegant, she eats packets and whispers flows at your collector. ๐Ÿ’…


๐ŸŒธ SYNOPSIS ๐ŸŒธ

fprobe [options] remote:port[/[local][/type]] ...

yeah so it's fprobe, then some options, then a whole list of collectors, and honestly the collector syntax is doing the most but we'll get there. keep reading, bestie. ๐Ÿซถ


๐Ÿ’œ DESCRIPTION ๐Ÿ’œ

fprobe is a libpcap-based tool that collects network traffic data and emits it as NetFlow flows towards the specified collector. no cap. she sniffs, she counts, she ships it off. main character energy from start to finish. ๐Ÿ‘‘


โœจ OPTIONS โœจ

bestie there are a LOT of these. buckle up. every single one matters, i'm not skipping a single one, that's not the vibe. ๐Ÿ’–

  • -h โ€” display the short help. tiny little hello, love that for us. ๐Ÿ‘‹
  • -p โ€” don't put the interface into promiscuous mode. but heads up, even if you use this option, the interface might still be in promiscuous mode for some other reason, so don't come for me if it is. ๐Ÿ™ˆ
  • -i <interface> โ€” listen on interface. if you don't specify one, fprobe will use the result of the pcap_lookupdev() function. on Linux systems with 2.2 or later kernels, you can pass an interface argument of any to capture packets from all interfaces. note that captures on the any device will not be done in promiscuous mode. you may also use - as the interface name to process files produced by tcpdump with the -w flag โ€” fprobe will read data from stdin</c>. yes ma'am, stdin. ๐ŸŽ€
  • -f <expression> โ€” the filter expression selects which packets will be captured. if no expression is given, all packets on the net will be captured. otherwise, only packets for which expression is true will be captured. warning bestie: fprobe uses a silly IP-packet detection method, so it is a bad idea to leave the filter empty. for general use ip (-fip) is a good filter expression. read the tcpdump manual for detailed expression syntax. ๐Ÿ“–
  • -s <seconds> โ€” how often to scan for expired flows. [default=5]
  • -g <seconds> โ€” fragmented flow lifetime. [default=30]
  • -d <seconds> โ€” idle flow lifetime (inactive timer). [default=60]
  • -e <seconds> โ€” active flow lifetime (active timer). [default=300]
  • -n <version> โ€” NetFlow version to use (1, 5, 7). [default=5]
  • -a <address> โ€” use address as the source for NetFlow flow. ๐Ÿ“ฎ
  • -x <inputID>[:<outputID>] โ€” workaround for SNMP interface indexes. [default=0] the second parameter may be omitted โ€” in that case its value will be equal to the first. see the BUGS section, because this one is giving "we did what we could". ๐Ÿ˜ญ
  • -b <flows> โ€” memory bulk size. [default=200 or 10000] note that the maximum and default values depend on compiling options (the --with-membulk parameter). yes, it's compile-time dependent, stay mad about it. ๐Ÿ’€
  • -m <kilobytes> โ€” memory limit for the flows cache (0=no limit). [default=0]
  • -q <flows> โ€” pending queue length. [default=100] each captured packet at first goes into a special buffer called the pending queue. the purpose of this buffer is to separate the most time-critical packet capture thread from the other stuff. very "don't text me while i'm busy" of it. ๐Ÿซถ
  • -B <kilobytes> โ€” kernel capture buffer size (0=don't change). [default=0] increasing the kernel capture buffer size is the most adequate way to prevent packet loss. unfortunately, at present there is no straight way to set the buffer size through libpcap, so this option is a hack. moreover, right now this hack takes effect only on socket()-based capture mechanisms: that means it works on Linux and does not work on BSD systems with their bpf(). note that the maximum allowed size of the buffer in Linux is limited and generally relatively small, so it should need to change the maximum:
sysctl -w net/core/rmem_max=4194304
  • -r <priority> โ€” real-time priority (0=disabled). [default=0] if the parameter is greater than zero, fprobe will use real-time scheduling policy to prevent packet loss. note that the possible values for this option depend on the operating system. ๐Ÿ˜Œ
  • -t <B:N> โ€” emitting rate limit (0:0=no limit). [default=0:0] produce an N nanosecond delay after each B bytes sent. this option may be useful with slow interfaces and slow collectors. note that the suspension time may be longer than requested because the argument value is rounded up to an integer multiple of the sleep resolution (which depends on the operating system and hardware) or because of the scheduling of other activity by the system. see the BUGS section. ๐Ÿ”ฅ
  • -S <bytes> โ€” snaplen (0=whole packet). [default=256] number of bytes to capture from the packet on the wire. โœ‚๏ธ
  • -K <bytes> โ€” link layer header size. by default fprobe takes this information from libpcap, but sometimes the obtained size is unsuitable for our purpose. this occurs, for example, on trunk interfaces in a VLAN environment, where the link layer header contains an additional VLAN header. see the EXAMPLES section. ๐ŸŒˆ
  • -k โ€” don't exclude the link layer header from packet size. by default fprobe counts only the IP-part of the packet.
  • -c <directory> โ€” directory to chroot to.
  • -u <user> โ€” user to run as.
  • -v <level> โ€” maximum displayed log level. (0=EMERG, 1=ALERT, 2=CRIT, 3=ERR, 4=WARNING, 5=NOTICE, 6=INFO, 7=DEBUG) [default=6]
  • -l <[dst][:id]> โ€” log destination (0=none, 1=syslog, 2=stdout, 3=both) and log/pidfile identifier. [default=1] this option allows you to select the opportune log destination and process identifier. the identifier helps to distinguish the pidfile and logs of one fprobe process from another. note that if the log destination contains stdout (equal to 2 or 3), fprobe will run in the foreground. ๐Ÿ‘€
  • remote:port/local/type โ€” the parameters remote and port respectively define the address and port of the NetFlow collector. the local parameter allows binding a certain local IP address with the specified collector. if the parameter is omitted, the value (if any) of the -a option will be used. the type parameter determines the emitting behavior. it may be m for mirroring (by default) and r for collectors round-robin rotating. you may specify multiple collectors. as many as you want, go off. ๐ŸŽ‰

๐ŸŒธ EXAMPLES ๐ŸŒธ

web traffic trivial capturing:

fprobe -ippp0 -f"tcp&&port 80" localhost:2055

capturing from a trunk interface:

fprobe -ieth0 -f"vlan&&ip" -K18 localhost:2055

reasonable configuration to run under heavy load:

fprobe -fip -B4096 -r2 -q10000 -t10000:10000000 localhost:2055

send packets to the collector at 10.1.1.1:2055 and distribute them between the collectors at 10.1.1.2:2055 and at 10.1.1.3:2055 on a round-robin basis:

fprobe 10.1.1.1:2055 10.1.1.2:2055//r 10.1.1.3:2055//r

look at that last one. two little //rs doing all the work. chefs kiss. ๐Ÿ‘จโ€๐Ÿณ๐Ÿ’‹


๐Ÿ’€ BUGS ๐Ÿ’€

bestie. we have to talk. ๐Ÿซ 

SNMP interfaces indexes and packet direction.

unfortunately libpcap doesn't provide any routing-related information about a captured packet, therefore there are no straight ways to determine and distinguish input and output interfaces. however, the -x option can at least tell that a flow was passed through a certain interface. also, you may launch several instances of the program with a tricky set of filters to mark out each possible packet direction:

fprobe -x1:2 -ieth1 -f"ip&&dst net 10.2" localhost:2055
fprobe -x2:1 -ieth2 -f"ip&&dst net 10.1" localhost:2055

Slow interfaces and slow collectors.

there may be problems with slow interfaces and slow collectors. it effects as emitted packets loss. on the one hand, the silent non-blocking sendto() implementation can't guarantee that a packet was really sent to the collector โ€” it may be dropped by the kernel due to outgoing buffer shortage (slow interface's problem) and on the other hand, a packet may be dropped on the collector's machine due to a similar reason โ€” incoming buffer shortage (slow collector's problem).

use the -t option as a workaround for this issue. ๐Ÿ› ๏ธ


๐Ÿฆ‹ SEE ALSO ๐Ÿฆ‹

go read those, they're the supporting cast and they ate. โœจ


and that's the whole page bestie!! ๐ŸŽ€ this was fprobe version 1.1, dated 2005-01-29, and she's still out here collecting flows like the icon she is. go forth, capture responsibly, and don't leave your filter empty, i'm begging you. ๐Ÿ’–๐Ÿ’…๐Ÿซถ