RFC 2722

Traffic Flow Measurement: Architecture

๐Ÿ“… published October 1999, from ietf

RFC 2722: Traffic Flow Measurement: Architecture โœจ

bestie, this is the RFC that lays out how to measure network traffic flows ๐Ÿฆ‹ It's Informational, so it's a friendly guide and not a strict standard. Part 1 of 11 covers the title block, the front matter, and the start of the actual architecture. More parts are coming, so keep your eyes peeled ๐Ÿ‘€

The title block, the tea โ˜•

  • Who wrote it: N. Brownlee from The University of Auckland, C. Mills from GTE Laboratories, Inc, and G. Ruth from GTE Internetworking. Three legends, honestly ๐Ÿซถ
  • Category: Informational ๐Ÿ’œ
  • Date: October 1999 ๐Ÿ“
  • Obsoletes: RFC 2063, which means this one replaces it ๐Ÿ’…
  • Updates: The header doesn't list anything being updated, so there's nothing to report there ๐ŸŒธ

Network Working Group                                        N. Brownlee
Request for Comments: 2722                    The University of Auckland
Obsoletes: 2063                                                 C. Mills
Category: Informational                            GTE Laboratories, Inc
                                                                 G. Ruth
                                                     GTE Internetworking
                                                            October 1999

Status of This Memo โœจ

Okay so this part is the legal-ish bit, but it matters ๐Ÿ’… This memo is here to give information to the Internet community. It does not set any Internet standard at all, so you can read it as a guide and not a rulebook ๐Ÿ“– And it can be shared as much as you want, no limits ๐ŸŒˆ

Copyright (C) The Internet Society (1999). All Rights Reserved.

Abstract ๐Ÿฆ‹

Basically, this doc gives a general framework for describing network traffic flows. It presents an architecture for measuring and reporting those flows, explains how that fits into the bigger picture of network traffic flow architecture, and shows how it can be used on the Internet ๐ŸŒธ

Table of Contents ๐Ÿ“š

1  Statement of Purpose and Scope                                   3
   1.1  Introduction . . . . . . . . . . . . . . . . . . . . . . .  3

2  Traffic Flow Measurement Architecture                            5
   2.1  Meters and Traffic Flows . . . . . . . . . . . . . . . . .  5
   2.2  Interaction Between METER and METER READER . . . . . . . .  7
   2.3  Interaction Between MANAGER and METER  . . . . . . . . . .  7
   2.4  Interaction Between MANAGER and METER READER . . . . . . .  8
   2.5  Multiple METERs or METER READERs . . . . . . . . . . . . .  9
   2.6  Interaction Between MANAGERs (MANAGER - MANAGER) . . . . . 10
   2.7  METER READERs and APPLICATIONs . . . . . . . . . . . . . . 10

3  Traffic Flows and Reporting Granularity                         10
   3.1  Flows and their Attributes . . . . . . . . . . . . . . . . 10
   3.2  Granularity of Flow Measurements . . . . . . . . . . . . . 13
   3.3  Rolling Counters, Timestamps, Report-in-One-Bucket-Only  . 15

4  Meters                                                          17
   4.1  Meter Structure  . . . . . . . . . . . . . . . . . . . . . 17
   4.2  Flow Table . . . . . . . . . . . . . . . . . . . . . . . . 19
   4.3  Packet Handling, Packet Matching . . . . . . . . . . . . . 20
   4.4  Rules and Rule Sets  . . . . . . . . . . . . . . . . . . . 23
   4.5  Maintaining the Flow Table . . . . . . . . . . . . . . . . 28
   4.6  Handling Increasing Traffic Levels . . . . . . . . . . . . 29

5  Meter Readers                                                   30
   5.1  Identifying Flows in Flow Records  . . . . . . . . . . . . 30
   5.2  Usage Records, Flow Data Files . . . . . . . . . . . . . . 30
   5.3  Meter to Meter Reader:  Usage Record Transmission  . . . . 31

6  Managers                                                        32
   6.1  Between Manager and Meter:  Control Functions  . . . . . . 32
   6.2  Between Manager and Meter Reader:  Control Functions . . . 33
   6.3  Exception Conditions . . . . . . . . . . . . . . . . . . . 35
   6.4  Standard Rule Sets . . . . . . . . . . . . . . . . . . . . 36

7  Security Considerations                                         36
   7.1  Threat Analysis  . . . . . . . . . . . . . . . . . . . . . 36
   7.2  Countermeasures  . . . . . . . . . . . . . . . . . . . . . 37

8  IANA Considerations                                             39
   8.1  PME Opcodes  . . . . . . . . . . . . . . . . . . . . . . . 39
   8.2  RTFM Attributes  . . . . . . . . . . . . . . . . . . . . . 39

9  APPENDICES                                                      41
   Appendix A: Network Characterisation  . . . . . . . . . . . . . 41
   Appendix B: Recommended Traffic Flow Measurement Capabilities . 42
   Appendix C: List of Defined Flow Attributes . . . . . . . . . . 43
   Appendix D: List of Meter Control Variables . . . . . . . . . . 44
   Appendix E: Changes Introduced Since RFC 2063 . . . . . . . . . 45

10 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . 45
11 References  . . . . . . . . . . . . . . . . . . . . . . . . . . 46
12 Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . 47
13 Full Copyright Statement  . . . . . . . . . . . . . . . . . . . 48

1. Statement of Purpose and Scope ๐ŸŒŸ

1.1. Introduction ๐Ÿ’–

This doc describes an architecture for measuring and reporting traffic flows on data networks. It has the following vibes:

  • The traffic flow model works with any protocol. You can mix and match address attributes at the 'adjacent' layer (more on that below), the network layer, and the transport layer of the stack ๐Ÿงš
  • Traffic flow attributes are defined so they work across several networking protocol stacks, which means measurement tools are actually useful in multi-protocol setups ๐ŸŒˆ
  • Users can write 'rule sets' to say exactly which flow data they want, so they collect what they need and ignore everything else ๐ŸŽ€
  • The data reduction work happens as close to the measurement point as possible. That cuts down how much data has to be collected (and sent over the network for storage) and makes life easier for the analysis apps ๐Ÿ’…

'Adjacent' is a layer-neutral word for the next layer down in whatever protocol layering you're using. It usually means the link layer (MAC addresses), but it doesn't say anything for or against tunnelling or other layering tricks ๐Ÿฆ‹

The architecture defines common metrics for measuring traffic flows. Using the same metrics means flow data can be swapped and compared across different platforms, which is so useful for:

  • Understanding how existing networks behave ๐Ÿ‘€
  • Planning network growth and expansion ๐ŸŒธ
  • Quantifying network performance ๐Ÿ“ˆ
  • Checking that network service quality is good โœ…
  • Attributing network usage to users ๐Ÿซถ

The architecture uses address attributes that are defined the same way at the Adjacent, Network and Transport layers. That lets implementations work well in multi-protocol environments ๐Ÿ”ฅ In this doc, 'usage data' is the generic term for whatever data you get from this measurement architecture ๐Ÿ’œ

In principle you could define address attributes for higher layers, but doing that in a general way would be really hard ๐Ÿ™ˆ Still, if an RTFM traffic meter ran inside an application server (where it could see application-specific usage info directly), the rest of the RTFM architecture could collect application-specific data too. Using the same model for network-level and application-level measurement could make it easier to build generic analysis apps that process and correlate both traffic and usage info. Experimental work in this area is described in the RTFM 'New Attributes' document [RTFM-NEW] โœจ

This document isn't a protocol spec, bestie ๐Ÿ™…โ€โ™€๏ธ It specifies and structures the info a traffic flow measurement system needs to collect, describes requirements such a system must meet, and outlines tradeoffs an implementor might make ๐ŸŽ€

Sometimes it's useful to use traffic info gathered through flow measurement instead of network statistics collected in other ways, for performance reasons. Quantifying network performance isn't the main goal of this architecture, but the measured flow data can still be used as a sign of how the network is performing ๐ŸŒˆ

Then there's cost recovery, which is all about "who pays for what." The big question is how to build a tariff (who gets billed, how much, for what, and based on what info). Tariff issues include fairness, predictability (how well subscribers can forecast their charges), practicality (of gathering the data and running the tariff), incentives (like encouraging off-peak use), and cost recovery goals (100% recovery, subsidisation, or profit making). This doc doesn't cover any of that, sadly ๐Ÿ’€

Background on why this approach was chosen is in the 'Internet Accounting Background' RFC [ACT-BKG] ๐Ÿ“š

2. Traffic Flow Measurement Architecture ๐Ÿงš

A traffic flow measurement system is used by Network Operations people to help manage and develop a network. It's basically the tool for measuring and understanding the network's traffic flows, and that info is useful for lots of reasons, as mentioned in section 1 (above) ๐Ÿ’…

The following sections outline a model for traffic flow measurement. It draws from working drafts of the OSI accounting model [OSI-ACT] ๐ŸŒธ

2.1. Meters and Traffic Flows โœจ

okay so the heart of the whole traffic measurement model is these network entities called traffic METERS ๐Ÿฆ‹ A meter watches packets as they zoom past one single point in the network and sorts them into groups. For each group, it keeps track of stuff like how many packets and bytes it saw. These METERED TRAFFIC GROUPS can be a user, a host, a network, a group of networks, a specific transport address (like an IP port number), or any combo of those, depending on how the meter is configured ๐Ÿ’…

we're assuming routers and traffic monitors all over the network have meters built in so they can measure traffic ๐ŸŒธ The question of where to put meters is covered in the 'Internet Accounting Background' RFC [ACT-BKG]. The big deal with meters is that they can squish traffic info down into something super compact by aggregating it ๐Ÿซถ

now for the TRAFFIC FLOW, which is basically an artificial, logical version of a call or a connection ๐Ÿ’œ A flow is a chunk of traffic that has a start and stop time and belongs to one of those metered groups we just talked about. The attribute values (source and destination addresses, packet counts, byte counts, and so on) tied to a flow are totals that reflect everything that happened during the DURATION between the start and stop times โณ The start time of a flow is locked in and never changes, but the stop time can keep moving later as the flow gets older ๐Ÿ“

here's the tea: for connectionless protocols like IP, there is by definition no way to tell whether a packet with a certain source and destination combo is part of a stream or not, because every packet is totally independent ๐Ÿ™ˆ So a traffic meter has a set of 'rules' in its configuration that describe which flows it cares about, based on their attribute values. It pulls attribute values out of each packet it sees and uses them to decide which flow that packet belongs to โœจ Sorting packets into 'flows' like this is an economical and practical way to measure network traffic and split it into well-defined groups, and honestly it's iconic ๐Ÿ‘‘

usage info that can't be derived from traffic flows can also be useful ๐Ÿงš For example, an application might want to record when people access different information resources, or a host might want to record the username (subscriber id) for a particular network session. The traffic flow architecture has a spot for this kind of thing. In the future the measurement model might be extended to collect this info from applications and hosts, so it can fill in values for higher-layer flow attributes ๐ŸŒˆ

besides FLOWS and METERS, the traffic flow measurement model also has MANAGERS, METER READERS and ANALYSIS APPLICATIONS, which get their own sections later on ๐ŸŽ€ The diagram below shows how they all connect. The numbers on it point to sections in this document:

                      MANAGER
                     /       \
                2.3 /         \ 2.4
                   /           \
                  /             \                      ANALYSIS
              METER  <----->  METER READER  <----->   APPLICATION
                       2.2                    2.7
  • MANAGER: A traffic measurement manager is an application that configures 'meter' entities and controls 'meter reader' entities ๐Ÿง  It sends configuration commands to the meters and keeps an eye on whether each meter and meter reader is running properly. It can be super convenient to put the meter reader and manager functions together inside one single network entity ๐Ÿ’–
  • METER: Meters get placed at measurement points that Network Operations personnel pick ๐Ÿ“ Each meter selectively records network activity based on its configuration settings. It can also aggregate, transform and process that recorded activity further before storing it. The processed, stored results are called the 'usage data' ๐Ÿ’พ
  • METER READER: A meter reader moves usage data from meters so that analysis applications can get to it ๐Ÿšš
  • ANALYSIS APPLICATION: An analysis application processes the usage data to produce info and reports that are useful for network engineering and management ๐Ÿ“Š Examples include:
    • TRAFFIC FLOW MATRICES, showing the total flow rates for many of the possible paths within an internet ๐Ÿ—บ๏ธ
    • FLOW RATE FREQUENCY DISTRIBUTIONS, summarizing flow rates over a period of time ๐Ÿ“ˆ
    • USAGE DATA showing the total traffic volumes sent and received by particular hosts ๐Ÿซง

the traffic measurement system as a whole is easiest to get once you think about how its components interact with each other ๐Ÿ”ฅ Those interactions are described in the sections that follow.


2.2. Interaction Between METER and METER READER

the info traveling along this path is the usage data itself ๐Ÿ’œ A meter holds its usage data in an array of flow data records called the FLOW TABLE. A meter reader can collect the data in whatever way works best. For example, it could upload a copy of the whole flow table using a file transfer protocol, or it could read the records in the current flow set one at a time using a suitable data transfer protocol ๐ŸŒท Also, the meter reader doesn't have to read complete flow data records. A subset of the attribute values might be totally enough ๐Ÿ™Œ

a meter reader can collect usage data from one or more meters ๐Ÿ’– Data can be collected from the meters at any time, and there is no requirement for collections to be synchronized in any way ๐Ÿซถ


2.3. Interaction Between MANAGER and METER

a manager is responsible for configuring and controlling one or more meters ๐ŸŽ€ Each meter's configuration includes stuff like:

  • Flow specifications, e.g. which traffic flows are to be measured, how they are to be aggregated, and any data the meter is required to compute for each flow being measured ๐Ÿ“‹
  • Meter control parameters, e.g. the 'inactivity' time for flows (if no packets belonging to a flow are seen for this time, the flow is considered to have ended, i.e. to have become idle) ๐Ÿ˜ด
  • Sampling behaviour. Normally every packet will be observed. Sometimes it may be necessary to use sampling techniques so that only some of the packets get observed (see following note) ๐ŸŽฒ

a note about sampling: current experience with the measurement architecture shows that a carefully-designed and implemented meter compresses the data well enough that, on the LANs and WANs we have today, sampling is seldom, if ever, needed ๐Ÿ’… For this reason, the architecture does not prescribe sampling algorithms. If sampling is needed, e.g. for metering a very-high-speed network with fine-grained flows, the sampling technique should be carefully chosen so that it does not bias the results ๐Ÿ‘€ For a good intro to this topic, check out the IPPM Working Group's RFC "Framework for IP Performance Metrics" [IPPM-FRM] ๐Ÿ“š

a meter can run several rule sets at the same time on behalf of one or more managers, and any manager can download a set of flow specifications (i.e. a 'rule set') to a meter ๐Ÿง Control parameters that apply to an individual rule set should be set by the manager after it downloads that rule set ๐Ÿ› ๏ธ

one manager should be designated as the 'master' for a meter ๐Ÿ‘‘ Parameters like sampling behaviour, which affect the overall operation of the meter, should only be set by the master manager ๐Ÿ”’


2.4. Interaction Between MANAGER and METER READER

a manager is responsible for configuring and controlling one or more meter readers โœจ A meter reader may only be controlled by a single manager ๐Ÿซถ For every meter it's collecting usage data from, a meter reader needs to know at least the following:

  • The meter's unique identity, i.e. its network name or address ๐Ÿท๏ธ
  • How often usage data is to be collected from the meter โฐ
  • Which flow records are to be collected (e.g. all flows, flows for a particular rule set, flows which have been active since a given time, etc.) ๐Ÿ—‚๏ธ
  • Which attribute values are to be collected for the required flow records (e.g. all attributes, or a small subset of them) ๐ŸŽฏ

since redundant reporting may be used to make usage data more reliable, exchanges among multiple entities must be considered as well ๐Ÿฆ‹ Those get discussed below.


2.5. Multiple METERs or METER READERs

                    -- METER READER A --
                   /         |          \
                  /          |           \
          =====METER 1     METER 2=====METER 3    METER 4=====
                              \          |           /
                               \         |          /
                                -- METER READER B --

several uniquely identified meters can report to one or more meter readers ๐ŸŒˆ The diagram above gives an example of how multiple meters and meter readers could be used ๐Ÿงฉ

in that diagram, meter 1 is read by meter reader A, and meter 4 is read by meter reader B ๐Ÿ’… Meters 1 and 4 have no redundancy, so if either one fails, the usage data for their network segments will be lost ๐Ÿ’€

meters 2 and 3, though, both measure traffic on the same network segment ๐Ÿซง If one of them fails, the other keeps collecting the segment's usage data. Meters 2 and 3 are read by meter reader A and by meter reader B. If one meter reader fails, the other will keep collecting usage data from both meters ๐Ÿ™Œ

the architecture does not require multiple meter readers to be synchronized ๐Ÿ•ฐ๏ธ In the situation above, meter readers A and B could both collect usage data at the same intervals, but not necessarily at the same times. Because collections are asynchronous, it's unlikely that usage records from two different meter readers will agree exactly ๐Ÿฅบ

if identical usage records were required from a single meter, a manager could get them by using two identical copies of a ruleset in that meter ๐Ÿชž Let's call them RS1 and RS2, and assume that RS1 is running. When a collection is about to happen, the manager switches the meter from RS1 to RS2 and directs the meter reader(s) to read flow data for RS1 from the meter. For the next collection, the manager switches back to RS1, and so on ๐Ÿ” Here's the catch, though: it is not possible to get identical usage records from more than one meter, since there's no way for a manager to switch rulesets in more than one meter at the same time ๐Ÿ˜ค

if there's only one meter reader and it fails, the meters keep running ๐Ÿ’ช When the meter reader restarts, it can collect all of the accumulated flow data. If this happens, time resolution will be lost (because of the missed collections), but overall traffic flow information won't ๐Ÿซถ The only exception would be if the traffic volume was high enough to 'roll over' counters for some flows during the failure. That's covered in the section on 'Rolling Counters' ๐Ÿ”„

2.6. Interaction Between MANAGERs (MANAGER - MANAGER) ๐Ÿ’…

okay so syncing up multiple management systems? that's the whole job of network management protocols ๐Ÿ™ˆ this architecture only covers the management controls it actually needs to run traffic flow measurement. it doesn't get into the bigger drama, like simultaneous or interleaved (possibly conflicting) commands from multiple network management stations, or passing control from one station to another ๐Ÿซถ

2.7. METER READERs and APPLICATIONs โœจ

once a meter reader has assembled a collection of usage data, an analysis application can process it. the reports the apps produce and the data they need are outside the scope of this architecture, so we're not going there ๐Ÿ’œ

ngl though, analysis apps usually eat a lot of input data ๐Ÿ˜ญ an important part of running a traffic flow measurement system is storing the flow data and regularly reducing it into daily, weekly or monthly summary files for further analysis. and yes, the details of that data handling are also outside this architecture ๐Ÿ“


3. Traffic Flows and Reporting Granularity ๐Ÿฆ‹

okay remember the flow definition from section 2.1? it was described in abstract terms like this:

    "A TRAFFIC FLOW is an artifical logical equivalent to a call or
    connection, belonging to a (user-specieied) METERED TRAFFIC
    GROUP."

in practical terms, a flow is a stream of packets the meter observes as they pass across a network between two end points (or from a single end point). the traffic meter summarizes those packets for analysis purposes ๐ŸŽ€

3.1. Flows and their Attributes ๐ŸŽ€

every traffic meter keeps a table of 'flow records' for the flows it sees โœจ a flow record holds the values of the ATTRIBUTES that matter for its flow. these attributes might include:

  • ADDRESSES for the flow's source and destination. these comprise the protocol type, the source and destination addresses at various network layers (extracted from the packet header), and the number of the interface on which the packet was observed.
  • First and last TIMES when packets were seen for this flow, i.e. the 'creation' and 'last activity' times for the flow.
  • COUNTS for 'forward' (source to destination) and 'backward' (destination to source) components (e.g. packets and bytes) of the flow's traffic. how 'source' and 'destination' get specified for flows is covered in the section on packet matching below ๐Ÿ‘€
  • OTHER attributes, e.g. the index of the flow's record in the flow table and the rule set number for the rules which the meter was running while the flow was observed. the values of these attributes give a way to tell apart flows observed by a meter at different times ๐Ÿงš

the attributes listed in this document (Appendix C) are a basic set, the useful minimum ๐Ÿ’– IANA considerations for allocating new attributes are set out in section 8 below.

a flow's METERED TRAFFIC GROUP is specified by the values of its ADDRESS attributes. for example, if a flow's address attributes were specified as source address = IP address 10.1.0.1 and destination address = IP address 26.1.0.1, then only IP packets from 10.1.0.1 to 26.1.0.1 (and back) would be counted in that flow. but if a flow's address attributes only specified source address = IP address 10.1.0.1, then all IP packets from and to 10.1.0.1 would be counted in that flow. main character energy for the address, no cap ๐Ÿ’…

the addresses that specify a flow's address attributes may include one or more of these types:

  • The INTERFACE NUMBER for the flow, i.e. the interface on which the meter measured the traffic. together with a unique address for the meter, this uniquely identifies a particular physical-level port ๐Ÿ”Œ
  • The ADJACENT ADDRESS, i.e. the address in the next layer down from the peer address in a particular instantiation of protocol layering. 'adjacent' will usually imply the link layer, but it doesn't implicitly advocate or dismiss any particular form of tunnelling or layering ๐ŸŒˆ

for example, if flow measurement is being performed using IP as the network layer on an Ethernet LAN [802-3], an adjacent address will normally be a six-octet Media Access Control (MAC) address. for a host connected to the same LAN segment as the meter, the adjacent address will be the MAC address of that host. for hosts on other LAN segments, it will be the MAC address of the adjacent (upstream or downstream) router carrying the traffic flow ๐Ÿ›ฃ๏ธ

  • The PEER ADDRESS, which identifies the source or destination of the packet for the network layer (n) at which traffic measurement is being performed. the form of a peer address depends on the network-layer protocol in use, and the measurement network layer (n) ๐Ÿ“
  • The TRANSPORT ADDRESS, which identifies the source or destination port for the packet, i.e. its (n+1) layer address. for example, if flow measurement is being performed at the IP layer, a transport address is a two-octet UDP or TCP port number ๐ŸŽ€

the four definitions above specify addresses for each of the four lowest layers of the OSI reference model: Physical layer, Link layer, Network layer and Transport layer ๐Ÿค“ a FLOW RECORD stores both the VALUE for each of its addresses (as described above) and a MASK that specifies which bits of the address value are being used and which are ignored. note that if address bits are being ignored, the meter will set them to zero, but their actual values are undefined.

one of the key features of this traffic measurement architecture is that attributes have essentially the same meaning across different protocols, so analysis applications can use the same reporting formats for all protocols ๐Ÿซถ this is straightforward for peer addresses. even though the form of addresses differs between protocols, the meaning of a 'peer address' stays the same. it gets harder to keep that correspondence at higher layers. for example, at the Network layer, IP, Novell IPX and AppleTalk all use port numbers as a 'transport address', but CLNP and DECnet have no notion of ports ๐Ÿ˜ญ

reporting by adjacent intermediate sources and destinations, or simply by meter interface (most useful when the meter is embedded in a router), supports hierarchical Internet reporting schemes as described in the 'Internet Accounting Background' RFC [ACT-BKG]. that means backbone and regional networks can measure usage down to just the next lower level of granularity (i.e. to the regional and stub/enterprise levels, respectively), with the final breakdown by end user (e.g. by source IP address) done by the stub/enterprise networks ๐ŸŒธ

in cases where network addresses are dynamically allocated (e.g. dial-in subscribers), further subscriber identification will be necessary if flows are to be ascribed to individual users ๐Ÿ™ˆ provision is made to further specify the metered traffic group through an optional SUBSCRIBER ID as part of the flow id. a subscriber ID may be associated with a particular flow either through the current rule set or by unspecified means within a meter. at this time, a subscriber ID is an arbitrary text string, and later versions of the architecture may specify details of its contents ๐Ÿงš

3.2. Granularity of Flow Measurements โœจ

okay so GRANULARITY is the 'control knob' ๐ŸŽ›๏ธ that lets an application and/or the meter trade off the overhead of usage reporting against how much detail you actually get ๐Ÿ’… A coarser granularity means more aggregation, and a finer granularity means more detail. So the number of flows measured (and stored) at a meter can be regulated just by changing the granularity of their attributes. Flows are like an adjustable pipe ๐ŸŽ€: you can run lots of fine-granularity streams, each one measured individually, or bundle everything into one coarse-granularity pipe. Time granularity can be controlled by varying the reporting interval, i.e. the time between meter readings โฐ

Flow granularity is controlled by adjusting the level of detail for the following:

  • The metered traffic group (address attributes, discussed above).
  • The categorisation of packets (other attributes, discussed below).
  • The lifetime/duration of flows (the reporting interval needs to be short enough to measure them with sufficient precision).

The set of rules that decides each packet's metered traffic group is known as the meter's CURRENT RULE SET ๐Ÿ“œ As we'll see, the current rule set is an integral part of the reported information, i.e. the recorded usage information cannot be properly interpreted without a definition of the rules used to collect it. no rules, no vibes ๐Ÿ’œ

Settings for these granularity factors may vary from meter to meter. They're determined by the meter's current rule set, so they'll change if network Operations personnel reconfigure the meter to use a new rule set. It's expected that the collection rules will change rather infrequently; nonetheless, the rule set in effect at any time must be identifiable via a RULE SET NUMBER ๐Ÿ”ข Granularity of metered traffic groups is further specified by additional ATTRIBUTES, which include:

  • Attributes which record information derived from other attribute values. Six of these are defined (SourceClass, DestClass, FlowClass, SourceKind, DestKind, FlowKind), and their meaning is determined by the meter's rule set. For example, one could have a subroutine in the rule set which determined whether a source or destination peer address was a member of an arbitrary list of networks, and set SourceClass/DestClass to one if the source/dest peer address was in the list or to zero otherwise.
  • Administratively specified attributes such as Quality of Service and Priority, etc. These are not defined at this time.

Settings for these granularity factors may vary from meter to meter. They're determined by the meter's current rule set, so they'll change if Network Operations personnel reconfigure the meter to use a new rule set.

A rule set can aggregate groups of addresses in two ways ๐Ÿงฉ The simplest is to use a mask in a single rule to test for an address within a masked group. The other way is to use a sequence of rules to test for an arbitrary group of (masked) address values, then use a PushRuleTo rule to set a derived attribute (e.g. FlowKind) to indicate the flow's group.

The LIFETIME of a flow is the time interval that began when the meter observed the first packet belonging to the flow and ended when it saw the last packet ๐Ÿฆ‹ Flow lifetimes are very variable, but many (if not most) are rather short. A meter can't measure lifetimes directly. Instead, a meter reader collects usage data for flows that have been active since the last collection, and an analysis application may compare the data from each collection to figure out when each flow actually stopped.

The meter does, however, need to reclaim memory (i.e. records in the flow table) being held by idle flows ๐Ÿงน The meter configuration includes a variable called InactivityTimeout, which specifies the minimum time a meter must wait before recovering the flow's record. In addition, before recovering a flow record the meter should be sure that the flow's data has been collected by all meter readers which registered to collect it. These two wait conditions are desired goals for the meter. They're not difficult to achieve in normal usage, however the meter cannot guarantee to fulfil them absolutely. this part is important, bestie ๐Ÿ™ˆ

These 'lifetime' issues are considered further in the section on meter readers (below). A complete list of the attributes currently defined is given in Appendix C later in this document.

3.3. Rolling Counters, Timestamps, Report-in-One-Bucket-Only

Once a usage record is sent, the decision needs to be made whether to clear any existing flow records or to keep them and add to their counts when recording subsequent traffic on the same flow ๐Ÿค” The second method, called rolling counters, is recommended and has several advantages. Its main perk is greater reliability: the system can often survive the loss of some usage records, like what might happen if a meter reader failed and later restarted. The next usage record will very often contain yet another reading of many of the same flow buckets that were in the lost usage record. The 'continuity' of data that rolling counters provide can also supply information for "sanity" checks on the data itself, to guard against errors in calculations ๐Ÿ’–

The use of rolling counters does introduce a new problem: how do you tell a follow-on flow record apart from a new flow record? Consider the following example ๐Ÿ‘€

                     CONTINUING FLOW        OLD FLOW, then NEW FLOW

                     start time = 1            start time = 1
Usage record N:      flow count = 2000      flow count = 2000 (done)

                     start time = 1            start time = 5
Usage record N+1:    flow count = 3000      new flow count = 1000

Total count:                 3000                    3000

In the continuing flow case, the same flow was reported when its count was 2000, and again at 3000: the total count to date is 3000. In the OLD/NEW case, the old flow had a count of 2000. Its record was then stopped (perhaps because of temporary idleness), but then more traffic with the same characteristics arrived, so a new flow record was started and quickly reached a count of 1000. The total flow count from both the old and new records is 3000.

The flow START TIMESTAMP attribute is enough to sort this out ๐Ÿ•ฐ๏ธ In the example above, the CONTINUING FLOW record in the second usage record has an old FLOW START timestamp, while the NEW FLOW contains a recent FLOW START timestamp. A flow with sporadic bursts of activity interspersed with long quiet periods will produce a sequence of flow activity records, each with the same set of address attributes, but with increasing FLOW START times.

Each packet is counted in at most one flow for each running ruleset, so a single packet doesn't get counted multiple times ๐Ÿ”’ The record of a single flow is informally called a "bucket." If multiple, sometimes overlapping, records of usage information are required (aggregate, individual, etc), the network manager should collect the counts at a granularity detailed enough that aggregate and combination counts can be reconstructed in post-processing of the raw usage data. Alternatively, multiple rulesets could be used to collect data at different granularities.

Here's a juicy example ๐Ÿ“ Say a meter needs to record both 'total packets coming in interface #1' and 'total packets arriving from any interface sourced by IP address = a.b.c.d', using a single rule set. Sure, a bucket can be declared for each case, but it's not clear how to handle a packet that satisfies both criteria. It must only be counted once. By default it will be counted in the first bucket for which it qualifies, and not in the other bucket. Plus, it's not possible to reconstruct this information by post-processing. the fix is to define not two, but THREE buckets, each one collecting a unique combination of the two criteria:

        Bucket 1:  Packets which came in interface 1,
                   AND were sourced by IP address a.b.c.d

        Bucket 2:  Packets which came in interface 1,
                   AND were NOT sourced by IP address a.b.c.d

        Bucket 3:  Packets which did NOT come in interface 1,
                   AND were sourced by IP address a.b.c.d

        (Bucket 4:  Packets which did NOT come in interface 1,
                   AND were NOT sourced by IP address a.b.c.d)

Now the desired information can be reconstructed by post-processing. "Total packets coming in interface 1" can be found by adding buckets 1 & 2, and "Total packets sourced by IP address a.b.c.d" can be found by adding buckets 1 & 3. Bucket 4 isn't explicitly required, since its information isn't of interest, but it's supplied in parentheses for completeness ๐ŸŒธ

Alternatively, the above could be achieved by running two rule sets (A and B), as follows:

        Bucket 1:  Packets which came in interface 1;
                   counted by rule set A.

        Bucket 2:  Packets which were sourced by IP address a.b.c.d;
                   counted by rule set B.

4. Meters โœจ

A traffic flow meter is a device for collecting data about traffic flows at a given point within a network ๐Ÿ” We'll call this the METERING POINT. The header of every packet passing the network metering point is offered to the traffic meter program.

A meter could be implemented in various ways, including:

  • A dedicated small host, connected to a broadcast LAN (so that it can see all packets as they pass by) and running a traffic meter program. The metering point is the LAN segment to which the meter is attached.
  • A multiprocessing system with one or more network interfaces, with drivers enabling a traffic meter program to see packets. In this case the system provides multiple metering points: traffic flows on any subset of its network interfaces can be measured.
  • A packet-forwarding device such as a router or switch. This is similar to (b) except that every received packet should also be forwarded, usually on a different interface.

4.1. Meter Structure โœจ

okay so this is the meter's whole layout, and it's kinda iconic ๐Ÿงš

                   packet                     +------------------+
                   header                     | Current Rule Set |
                     |                        +--------+---------+
                     |                                 |
                     |                                 |
             +-------*--------+    'match key'  +------*-------+
             |    Packet      |---------------->|    Packet    |
             |   Processor    |                 |   Matching   |
             |                |<----------------|    Engine    |
             +--+----------+--+  'flow key'     +--------------+
                |          |
                |          |
         Ignore *          | Count (via 'flow key')
                           |
                        +--*--------------+
                        | 'Search' index  |
                        +--------+--------+
                                 |
                        +--------*--------+
                        |                 |
                        |   Flow Table    |
                        |                 |
                        +--------+--------+
                                 |
                        +--------*--------+
                        | 'Collect' index |
                        +--------+--------+
                                 |
                                 *
                            Meter Reader

So here's how it goes, step by step ๐Ÿ’–

  • Incoming packet headers arrive at the top left of the diagram and get passed to the PACKET PROCESSOR.
  • The packet processor hands them to the Packet Matching Engine (PME), where they get classified.
  • The PME is a Virtual Machine running a pattern matching program contained in the CURRENT RULE SET. The Packet Processor invokes it, it executes the rules in the current rule set as described in section 4.3 below, and then it returns instructions on what to do with the packet. literally the whole plot ๐ŸŽ€
  • Some packets are classified as 'to be ignored'. They get discarded by the Packet Processor ๐Ÿ’€
  • Other packets are matched by the PME, which returns a FLOW KEY describing the flow the packet belongs to ๐Ÿฆ‹
  • The flow key is used to locate the flow's entry in the FLOW TABLE. A new entry is created when a flow is first seen. The entry's data fields (e.g. packet and byte counters) get updated ๐Ÿ“ˆ
  • A meter reader may collect data from the flow table at any time. It may use the 'collect' index to find the flows to be collected within the flow table. ๐Ÿซถ

The discussion above assumes that a meter only runs a single rule set. A meter can, however, run several rule sets concurrently ๐Ÿ‘‘ To make that work, the meter keeps a table of current rulesets. The packet processor matches each packet against every current ruleset, which produces a single flow table holding flows from all the rule sets. One way to build this is to use the Rule Set Number attribute in each flow as part of the flow key. ngl that's clever ๐Ÿง 

A packet may only be counted once in a rule set (as explained in section 3.3 above), but it may be counted in any of the current rulesets. The overall effect is somewhat similar to running several independent meters, one for each rule set. it's giving multiverse ๐ŸŒˆ


4.2. Flow Table

Every traffic meter maintains a 'flow table', i.e. a table of TRAFFIC FLOW RECORDS for the flows the meter has seen ๐ŸŒธ Details of how the flow table is maintained are in section 4.5 below. A flow record contains attribute values for its flow, including:

  • Addresses for the flow's source and destination. These include addresses and masks for various network layers (extracted from the packet header), plus the identity of the interface the packet was observed on.
  • First and last times when packets were seen for this flow.
  • Counts for 'forward' (source to destination) and 'backward' (destination to source) components of the flow's traffic.
  • Other attributes, e.g. the state of the flow record (discussed below). ๐Ÿ’…

The state of a flow record may be:

  • INACTIVE: The flow record is not being used by the meter. ๐Ÿ™ˆ
  • CURRENT: The record is in use and describes a flow which belongs to the 'current flow set', i.e. the set of flows recently seen by the meter. โญ
  • IDLE: The record is in use and the flow it describes is part of the current flow set. In addition, no packets belonging to this flow have been seen for a period specified by the meter's InactivityTime variable. ๐Ÿ˜ญ

4.3. Packet Handling, Packet Matching โœจ

okay bestie, every packet header the traffic meter program gets goes through the same little routine ๐Ÿ’…

  • Pull the attribute values out of the packet header and use them to build a MATCH KEY for the packet ๐Ÿ”‘
  • Match the packet's key against the current rule set (more on that in a sec ๐Ÿ‘€)

The rule set decides whether the packet gets counted or ignored. If it's counted, the matching process spits out a FLOW KEY for the flow the packet belongs to. That flow key is used to find the flow's record in the flow table. If there isn't a record yet, a new flow record might get created ๐ŸŒธ Then the matching flow record's data gets updated.

For example, the rule set could say that packets to or from any host in IP network 130.216 are counted ๐Ÿฆ‹ It could also say that flow records get created for every pair of 24-bit (Class C) subnets inside network 130.216 ๐Ÿงš

Each packet's match key gets passed to the meter's PATTERN MATCHING ENGINE (PME) for matching. The PME is a Virtual Machine that runs a set of instructions called RULES, so a RULE SET is basically a program for the PME ๐Ÿ’ป A packet's match key contains the source (S) and destination (D) interface identities, plus address values and masks.

If measured flows were unidirectional, meaning only packets going one way got counted, this would be super simple ๐Ÿ“ The PME would get called once to match the packet. Any flow key from a successful match would find the flow's record in the flow table, and that flow's counters would get updated.

But flows are actually bidirectional ๐Ÿ’œ They cover the forward and reverse packets of a protocol interchange, or 'session'. Keeping two sets of counters in the meter's flow record makes the flow data way easier to handle, because analysis programs don't have to piece the 'forward' and 'reverse' parts of a session back together ๐Ÿซถ The catch is that bidirectional flows are harder for the meter, since it has to figure out whether a packet is a 'forward' one or a 'reverse' one. To do that, the meter will often need to invoke the PME twice, once for each possible packet direction ๐Ÿ”ฅ

The diagram below shows the algorithm the traffic meter uses to process each packet. You read it from left to right and top to bottom, starting at the top left corner and ending at the bottom right corner. S is the flow's source address (its set of source address attribute values from the packet header), and D is its destination address.

                   Ignore
   --- match(S->D) -------------------------------------------------+
        | Suc   | NoMatch                                           |
        |       |          Ignore                                   |
        |      match(D->S) -----------------------------------------+
        |       | Suc   | NoMatch                                   |
        |       |       |                                           |
        |       |       +-------------------------------------------+
        |       |                                                   |
        |       |             Suc                                   |
        |      current(D->S) ---------- count(D->S,r) --------------+
        |       | Fail                                              |
        |       |                                                   |
        |      create(D->S) ----------- count(D->S,r) --------------+
        |                                                           |
        |             Suc                                           |
       current(S->D) ------------------ count(S->D,f) --------------+
        | Fail                                                      |
        |             Suc                                           |
       current(D->S) ------------------ count(D->S,r) --------------+
        | Fail                                                      |
        |                                                           |
       create(S->D) ------------------- count(S->D,f) --------------+
                                                                    |
                                                                    *

There are a few cases to think about ๐Ÿง 

  • The packet is recognised as one that is to be ignored ๐Ÿ™ˆ
  • The packet would match in either direction. One situation where this could happen is a rule set that matches flows within network X (Source = X, Dest = X) but also says flows get created for each subnet within network X, say subnets y and z. If, for example, a packet shows up for y->z, the meter has to check that flow z->y isn't already current before it creates y->z ๐Ÿ‘€
  • The packet matches in one direction only. If its flow is already current, its forward or reverse counters get incremented. Otherwise, the flow gets added to the flow table and then counted ๐Ÿ’–

The algorithm uses four functions:

  • match(A->B) is the PME. It uses the meter's current rule set to match the attribute values in the packet's match key. A->B means the assumed source address is A and the destination is B, i.e. the packet was travelling from A to B. match() returns one of three results:
    • 'Ignore' means the packet matched, but this flow isn't to be counted ๐Ÿšซ
    • 'NoMatch' means the packet didn't match. It might still match with its direction reversed, i.e. from B to A ๐Ÿ”„
    • 'Suc' means the packet did match, so it belongs to a flow that is to be counted โœ…
  • current(A->B) succeeds if the flow A-to-B is current, meaning it has a record in the flow table whose state is Current, and fails otherwise ๐ŸŒŸ
  • create(A->B) adds the flow A-to-B to the flow table. It sets the values for attributes that stay constant, like addresses, and zeroes the flow's counters ๐Ÿงธ
  • count(A->B,f) increments the 'forward' counters for flow A-to-B. count(A->B,r) increments the 'reverse' counters for flow A-to-B. Here, 'forward' means the counters for packets travelling from A to B. Note that count(A->B,f) is identical to count(B->A,r) ๐Ÿชž

When you're writing rule sets, remember that the meter will normally try to match each packet in the reverse direction if the forward match doesn't succeed. It's super important that the rule set doesn't contain inconsistencies that mess with this process ๐Ÿšจ

For example, say a rule set counts packets from source network A to destination network B, but ignores packets from source network B. That's a classic inconsistent rule set ๐Ÿ’€ Packets from network B should be counted as reverse packets for the A-to-B flow, so the rule set is basically contradicting itself.

You could avoid this by designing a language for writing rule files and then writing a compiler for it, which makes it way easier to produce correct rule sets ๐Ÿ› ๏ธ An example of such a language is described in the 'SRL' document [RTFM-SRL]. Another option is writing a 'rule set consistency checker' program that can catch problems in hand-written rule sets ๐Ÿ”

Normally, though, the best way to dodge these problems is to write rule sets that only classify flows in the forward direction, and let the meter handle the reverse-travelling packets ๐Ÿ‘‘

Occasionally, a rule set needs to know which direction a packet is being matched in. Say a rule set wants to save some attribute values (source and destination addresses, maybe) for any 'unusual' packets. The rule set will have a sequence of tests for all the 'usual' source addresses, followed by a rule that runs a 'NoMatch' action. If the match fails in the S->D direction, the NoMatch action makes it retry. If it fails in the D->S direction, the packet can be counted as an 'unusual' packet ๐Ÿ•ต๏ธโ€โ™€๏ธ

To count an 'unusual' packet, we need to know the matching direction, and that's what the MatchingStoD attribute is for ๐ŸŽ€ To use it, you follow the source address tests with a rule that checks whether the matching direction is S->D (MatchingStoD value is 1). If it is, a 'NoMatch' action runs. Otherwise, the packet has failed to match in both directions, so we can save whatever attribute values we care about and count the 'unusual' packet ๐Ÿฆ‹

4.4. Rules and Rule Sets โœจ

okay bestie, so a rule set is basically an array of rules ๐ŸŽ€ and rule sets live inside a meter as entries in an array of rule sets ๐Ÿ’…

Rule set 1 (the first entry in the rule set table) is built into the meter and you CANNOT change it ๐Ÿ”’ The meter runs it when it starts up ๐Ÿš€ It gives a very coarse reporting granularity, so it's mainly there to check that the meter is running before a 'useful' rule set gets downloaded into it ๐Ÿงš

A meter also keeps an array of 'tasks' ๐Ÿ“‹ which decide what rule sets the meter is running. Each task has a 'current' rule set (the one it normally uses) and a 'standby' rule set (used when the overall traffic level is unusually high ๐Ÿ“ˆ). If a task gets told to use rule set 0, it stops measuring ๐Ÿ›‘ All its packets get ignored until a non-zero rule set is made current again ๐Ÿ’€

Each rule in a rule set is an instruction for the Packet Matching Engine (PME), which means it's an instruction for a Virtual Machine ๐Ÿค– PME instructions have five component fields in two logical groups, and it looks like this:

   +-------- test ---------+    +---- action -----+
   attribute & mask = value:    opcode,  parameter;

The test group lets the PME check the value of an attribute ๐Ÿ‘€ It does this by ANDing the attribute value with the mask and comparing the result with the value field. Also, no explicit range test exists ๐Ÿ™…โ€โ™€๏ธ but you can still do a range when the range can be covered by a mask, e.g. attribute value less than 2048 ๐Ÿ”ข

The PME keeps a Boolean called the 'test indicator' ๐Ÿšฆ which decides whether a rule's test gets performed at all. It starts out set (true) โœจ

The action group says what action may be performed when the rule runs ๐ŸŽฌ Opcodes contain two flags, 'goto' and 'test', which you can see in the table further down. Execution starts at rule 1, the first rule in the rule set, and goes like this:

   If the test indicator is true:
      Perform the test, i.e. AND the attribute value with the
         mask and compare it with the value.
      If these are equal the test has succeeded; perform the
         rule's action (below).
      If the test fails execute the next rule in the rule set.
      If there are no more rules in the rule set, return from the
         match() function indicating NoMatch.

   If the test indicator is false, or the test (above) succeeded:
      Set the test indicator to this opcode's test flag value.
      Determine the next rule to execute.
         If the opcode has its goto flag set, its parameter value
            specifies the number of the next rule.
         Opcodes which don't have their goto flags set either
            determine the next rule in special ways (Return),
            or they terminate execution (Ignore, NoMatch, Count,
            CountPkt).
      Perform the action.

The PME keeps two 'history' data structures ๐Ÿ“š The first one is the 'return' stack. It just records the index (that's the 1-origin rule number) of each Gosub rule as it runs, and Return rules pop their Gosub rule index off it ๐Ÿ”„ Heads up: when the Ignore, NoMatch, Count and CountPkt actions run, PME execution ends no matter what, even if the PME is in a subroutine (aka the 'return' stack isn't empty) ๐Ÿ›‘

The second one is the 'pattern' queue ๐Ÿงต which saves info for later so it can be used to build a flow key. A flow key gets built by zeroing all its attribute values, then copying the attribute number, mask and value info from the pattern queue in the order it was enqueued ๐Ÿ”‘

An attribute number identifies the attribute actually used in a test ๐ŸŽฏ Usually it's the rule's attribute field, unless the attribute is a 'meter variable' ๐Ÿช„ Meter variables get explained after the opcode actions table below.

The opcodes are:

         opcode         goto    test

      1  Ignore           0       -
      2  NoMatch          0       -
      3  Count            0       -
      4  CountPkt         0       -
      5  Return           0       0
      6  Gosub            1       1
      7  GosubAct         1       0
      8  Assign           1       1
      9  AssignAct        1       0
     10  Goto             1       1
     11  GotoAct          1       0
     12  PushRuleTo       1       1
     13  PushRuleToAct    1       0
     14  PushPktTo        1       1
     15  PushPktToAct     1       0
     16  PopTo            1       1
     17  PopToAct         1       0

And here's what each of them actually does ๐Ÿ’–

  • Ignore: stops matching โœ‹ and returns from the match() function saying the packet is to be ignored ๐Ÿ™ˆ
  • NoMatch: stops matching โœ‹ and returns from the match() function indicating failure ๐Ÿ’”
  • Count: stops matching โœ‹ Saves this rule's attribute number, mask and value in the PME's pattern queue ๐Ÿ“ฅ then builds a flow key for the flow this packet belongs to ๐Ÿ”‘ Returns from the match() function indicating success โœ… The meter then uses the flow key to find the flow record for this packet's flow ๐Ÿ—‚๏ธ
  • CountPkt: same as Count, except the masked value from the packet header (as it would have been used in the rule's test) gets saved in the PME's pattern queue instead of the rule's value ๐ŸŽ€
  • Gosub: calls a rule-matching subroutine ๐Ÿ“ž Pushes the current rule number onto the PME's return stack โฌ†๏ธ sets the test indicator, then goes to the specified rule ๐Ÿ‘‰
  • GosubAct: same as Gosub, except the test indicator gets cleared before going to the specified rule ๐Ÿงน
  • Return: returns from a rule-matching subroutine ๐Ÿ”™ Pops the number of the calling gosub rule off the PME's 'return' stack and adds this rule's parameter value to it to work out the 'target' rule ๐ŸŽฏ Then it clears the test indicator and goes to the target rule ๐Ÿ

A subroutine call shows up in a rule set as a Gosub rule followed by a little group of following rules ๐Ÿงฉ Since a Return action clears the test flag, the action of one of those 'following' rules gets executed. This lets the subroutine hand back a result, on top of anything it saved in the PME's pattern queue ๐ŸŽ

  • Assign: sets the attribute specified in this rule to the parameter value specified for this rule ๐Ÿ’… Sets the test indicator, then goes to the specified rule ๐Ÿ‘‰
  • AssignAct: same as Assign, except the test indicator gets cleared before going to the specified rule ๐Ÿงน
  • Goto: sets the test indicator, then goes to the specified rule ๐Ÿšถโ€โ™€๏ธ
  • GotoAct: clears the test indicator, then goes to the specified rule ๐Ÿšถโ€โ™€๏ธ
  • PushRuleTo: saves this rule's attribute number, mask and value in the PME's pattern queue ๐Ÿ“ฅ Sets the test indicator, then goes to the specified rule ๐Ÿ‘‰
  • PushRuleToAct: same as PushRuleTo, except the test indicator gets cleared before going to the specified rule ๐Ÿงน

PushRuleTo actions can be used to save the value and mask used in a test, or (if the test is not performed) to save an arbitrary value and mask ๐Ÿงƒ

  • PushPktTo: saves this rule's attribute number, mask, and the masked value from the packet header (as it would have been used in the rule's test) in the PME's pattern queue ๐Ÿ“ฆ Sets the test indicator, then goes to the specified rule ๐Ÿ‘‰
  • PushPktToAct: same as PushPktTo, except the test indicator gets cleared before going to the specified rule ๐Ÿงน

PushPktTo actions can be used to save a value from the packet header using a specified mask ๐ŸŽญ The easiest way to set this up is to use a zero value for the PushPktTo rule's value field, and to GoToAct to the PushPktTo rule, so that its test doesn't get executed ๐Ÿง 

  • PopTo: deletes the most recent item from the pattern queue ๐Ÿ—‘๏ธ so the info saved by an earlier 'push' action is removed ๐Ÿซง Sets the test indicator, then goes to the specified rule ๐Ÿ‘‰
  • PopToAct: same as PopTo, except the test indicator gets cleared before going to the specified rule ๐Ÿงน

Okay, now for the extra attributes ๐ŸŒˆ Besides the attributes that apply directly to packets (like SourcePeerAddress, DestTransAddress, etc.), the PME has several more. They are:

  • Null: tests on the Null attribute always succeed โœ…
  • MatchingStoD: tells you whether the PME is matching the packet with its addresses in 'wire order' or with its addresses reversed ๐Ÿ”„ Its value is 1 if the addresses are in wire order (StoD), and zero otherwise ๐Ÿ”ข
  • v1 .. v5: v1, v2, v3, v4 and v5 are 'meter variables' ๐Ÿช„ They're a way to pass parameters into rule-matching subroutines ๐Ÿ’Œ Each one may hold the number of a normal attribute, and its value gets set by an Assign action ๐Ÿ“ When a meter variable shows up as the attribute of a rule, its value says which actual attribute gets tested ๐ŸŽฏ For example, if v1 had been assigned SourcePeerAddress as its value, a rule with v1 as its attribute would actually test SourcePeerAddress ๐Ÿ‘€
  • SourceClass, DestClass, FlowClass, SourceKind, DestKind, FlowKind: these six attributes can be set by executing PushRuleTo actions ๐Ÿง They let the PME save info built up during matching into flow records ๐Ÿ’พ Their values can be tested in rules too, so you can set them early in a rule set and test them later on โฐ

The opcodes above (with their 'goto' and 'test' values) are a minimum set ๐Ÿ’œ but honestly it's one that has proved very effective in current meter implementations ๐Ÿ† From time to time it might be useful to add more opcodes ๐Ÿงฉ IANA considerations for allocating those are set out in section 8 below ๐Ÿ“œ

4.5 Maintaining the Flow Table ๐Ÿงนโœจ

okay so the flow table is basically a 1-origin array of flow records ๐ŸŽ€ (an implementation can use whatever data structure is most suitable, no judgment ๐Ÿ’…). When the meter starts up there are no known flows yet, so every flow record is in the 'inactive' state ๐Ÿฆ‹

Each time a packet gets matched to a flow that isn't in a current flow set, a flow record is made for it, and its state is 'current' ๐Ÿ’–. To pick a record for the new flow, the meter searches the flow table for an 'inactive' record first. If there aren't any inactive records left, it looks for an 'idle' one instead ๐Ÿซถ. ngl, the order of records in the table has no special meaning, so don't read into it ๐Ÿ’€

A meter's memory management routines SHOULD aim to minimise the time spent finding flow records for new flows, so that the setup overhead for each new flow stays low ๐Ÿ”ฅ

A 'meter reader' MAY collect flow data at any time ๐ŸŒธ. There's no requirement for collections to be synchronized. The reader MAY collect the data in any suitable manner. For example, it could upload a copy of the whole flow table using a file transfer protocol, or it could read the records in the current flow set row by row using a suitable data transfer protocol ๐Ÿ“ฆ

The meter keeps track of collections. In particular, it maintains ReaderLastTime variables, which remember the time the last collection was made by each reader ๐Ÿง . A second variable, InactivityTime, specifies the minimum time the meter will wait before deciding that a flow is idle โณ

The meter MUST recover records used for idle flows, if only to stop itself running out of flow records. Recovered flow records go back to the 'inactive' state ๐Ÿงน. There are a few recovery strategies possible, including the following:

One possible recovery strategy is to recover idle flow records as soon as possible after their data has been collected by all readers that have registered to do so. To do this, the meter could run a background process that scans the flow table looking for 'current' flows whose 'last packet' time is earlier than the meter's LastCollectTime ๐Ÿ”Ž

Another recovery strategy is to leave idle flows alone for as long as possible. This would be fine if you were only interested in measuring total traffic volumes ๐Ÿ“Š. It could be implemented by having the meter search for collected idle flows only when it ran low on 'inactive' flow records.

One further factor a meter SHOULD consider before recovering a flow is the number of meter readers that have collected the flow's data ๐Ÿค”. If there are multiple meter readers operating, each reader SHOULD collect a flow's data before its memory is recovered ๐Ÿ‘ฏโ€โ™€๏ธ

Of course a meter reader may fail, so the meter can't wait forever for it ๐Ÿ˜ญ. Instead, the meter MUST keep a table of active meter readers, with a timeout specified for each one. If a meter reader fails to collect flow data within its timeout interval, the meter SHOULD delete that reader from the meter's active meter reader table ๐Ÿšซ

4.6 Handling Increasing Traffic Levels ๐Ÿšฆ

Under normal conditions, the meter reader specifies which set of usage records it wants to collect, and the meter provides them ๐Ÿ’Œ. If, however, memory usage rises above the high-water mark, the meter SHOULD switch to a STANDBY RULE SET so that the rate at which new flows are created goes down ๐Ÿ“‰

When the manager becomes aware that the meter is using its standby rule set, usually as part of a regular poll, it could decrease the interval between collections ๐Ÿ‘€. This would shorten the time that flows sit in memory waiting to be collected, letting the meter free up flow memory faster ๐Ÿƒโ€โ™€๏ธ

The meter could also put more effort into recovering flow memory, so there are fewer idle flows in memory ๐Ÿงฝ. When the situation returns to normal, the manager MAY request that the meter switch back to its normal rule set โœจ

5 Meter Readers ๐Ÿ“š

Usage data is accumulated by a meter (e.g. in a router) as memory permits ๐Ÿงบ. It's collected at regular reporting intervals by meter readers, as specified by a manager. The collected data is recorded in stable storage as a FLOW DATA FILE, as a sequence of USAGE RECORDS ๐Ÿ’พ

The following sections describe the contents of usage records and flow data files. Heads up though: at this stage, the details of those records and files are not specified in the architecture ๐Ÿ™ˆ. Specifying a common format for them would be a worthwhile future development ๐ŸŒˆ

5.1 Identifying Flows in Flow Records ๐Ÿ”

Once a packet has been classified and is ready to be counted, an appropriate flow data record MUST already exist in the flow table. Otherwise, one MUST be created. The flow record has a flexible format where unnecessary identification attributes MAY be omitted ๐Ÿงฉ. Which attributes of the flow record to use, and what values to put in them, is decided by the current rule set.

Note that the combination of start time, rule set number and flow subscript (row number in the flow table) gives you a unique flow identifier, no matter what the values of its other attributes are ๐Ÿ†”

The current rule set may specify extra information, e.g. a computed attribute value such as FlowKind, which gets placed in the attribute section of the usage record ๐Ÿง . That means that if a particular flow is matched by the rule set, the corresponding flow record SHOULD be marked not only with the qualifying identification attributes, but also with the extra information. Using this feature, several flows can each carry the same FlowKind value, so the resulting usage records can be used in post-processing, or between meter reader and meter as a criterion for collection ๐ŸŽฏ

5.2 Usage Records, Flow Data Files ๐Ÿ“

The collected usage data will be stored in flow data files on the meter reader, one file for each meter. On top of the measured usage data, flow data files MUST contain information uniquely identifying the meter the data was collected from ๐Ÿชช

A USAGE RECORD contains the descriptions of, and values for, one or more flows. Quantities are counted in terms of number of packets and number of bytes per flow ๐Ÿ”ข. Other quantities, e.g. short-term flow rates, may be added later. Work on those extensions is described in the RTFM 'New Attributes' document [RTFM-NEW] ๐Ÿ”ฎ

Each usage record contains the metered traffic group identifier of the meter (a set of network addresses), a time stamp, and a list of reported flows (FLOW DATA RECORDS). A meter reader builds up a file of usage records by regularly collecting flow data from a meter, using that data to build usage records, and concatenating them to the tail of a file. Such a file is called a FLOW DATA FILE ๐Ÿ—‚๏ธ

A usage record contains the following information in some form:

+-------------------------------------------------------------------+
|    RECORD IDENTIFIERS:                                            |
|      Meter Id (& digital signature if required)                   |
|      Timestamp                                                    |
|      Collection Rules ID                                          |
+-------------------------------------------------------------------+
|    FLOW IDENTIFIERS:            |    COUNTERS                     |
|      Address List               |       Packet Count              |
|      Subscriber ID (Optional)   |       Byte Count                |
|      Attributes (Optional)      |    Flow Start/Stop Time         |
+-------------------------------------------------------------------+

5.3 Meter to Meter Reader: Usage Record Transmission ๐Ÿ“ก

The usage record contents are the raison d'etre of the whole system ๐Ÿ’…. The accuracy, reliability, and security of transmission are the primary concerns of the meter/meter reader exchange ๐Ÿ”. Since errors can happen on networks, and Internet packets may be dropped, some mechanism is needed to make sure the usage information is transmitted intact ๐Ÿซก

Flow data is moved from meter to meter reader through a series of protocol exchanges between them. This can be done in various ways, moving individual attribute values, complete flows, or the entire flow table (i.e. all the active and idle flows) ๐Ÿ”„. One possible method is to use SNMP. The 'Traffic Flow Measurement: Meter MIB' RFC [RTFM-MIB] has the details ๐Ÿ“–. But that's simply one example, and the transfer of flow data from meter to meter reader is not specified in this document ๐Ÿคทโ€โ™€๏ธ

The reliability of the data transfer method under light, normal, and extreme network loads SHOULD be understood before choosing among collection methods ๐Ÿง

In normal operation, the meter will be running a rule file that provides the required degree of flow reporting granularity, and the meter reader(s) will collect the flow data often enough to let the meter's garbage collection mechanism keep memory usage stable ๐Ÿงนโœจ

In the worst case, traffic may increase to the point where the meter is in danger of running completely out of flow memory ๐Ÿ˜ฑ. The meter implementor MUST decide how to handle this. For example, they could switch to a default (extremely coarse granularity) rule set, send a trap message to the manager, or attempt to dump flow data to the meter reader ๐Ÿ›Ÿ

Users of the Traffic Flow Measurement system SHOULD analyse their requirements carefully and assess for themselves whether it's more important to collect flow data at normal granularity (increasing the collection frequency as needed to keep up with traffic volumes), or to accept flow data with a coarser granularity ๐Ÿง๐Ÿ’€

Similarly, it may be acceptable to lose flow data for a short time in return for making sure the meter keeps running properly, i.e. is not overwhelmed by rising traffic levels ๐Ÿฆ‹

6 Managers ๐Ÿง‘โ€๐Ÿ’ผ

A manager configures meters and controls meter readers ๐Ÿ‘‘. It does this via the interactions described below ๐Ÿซถ

6.1. Between Manager and Meter: Control Functions โœจ

okay bestie, this is where the manager gets to be the boss ๐Ÿ’… Here's the control stuff it can do to the meter:

  • DOWNLOAD RULE SET: a meter may hold an array of rule sets ๐ŸŽ€ One of them, the 'default' rule set, is built into the meter and cannot be changed. It's a diagnostic feature, so when a meter starts up it's always running a ruleset everyone knows about ๐Ÿงš

Every other rule set has to be downloaded by the manager. The manager can use any suitable protocol exchange to do it, like an FTP file transfer or a series of SNMP SETs (one for each row of the rule set) ๐ŸŒธ

  • SPECIFY METER TASK: once the rule sets are downloaded, the manager must tell the meter which rule sets will be the 'current' and 'standby' ones for each task the meter runs ๐Ÿ‘‘
  • SET HIGH WATER MARK: this is a percentage of the flow table capacity. The meter uses it to decide when to switch to its standby rule set, which makes the flows more granular and saves the meter's flow memory ๐Ÿซถ After that switch, the manager may also change the polling frequency or the meter's control parameters, so the meter can reclaim memory from idle flows faster ๐Ÿฆ‹ The meter keeps a separate high water mark value for each task it's currently running.

If the high traffic levels keep going, the meter's normal rule set may have to be rewritten to permanently reduce the reporting granularity ๐Ÿ˜ญ

  • SET FLOW TERMINATION PARAMETERS: the meter should have the good sense to purge flow records from its tables when lack of resources could cause data loss ๐Ÿ’… Those records can include:
    • Flows that have already been reported to all registered meter readers and show no activity since the last report,
    • Oldest flows, or
    • Flows with the smallest number of observed packets.
  • SET INACTIVITY TIMEOUT: this is the time in seconds since the last packet was seen for a flow โณ Flow records may be reclaimed if they've been idle for at least this long and have been collected under the current collection criteria ๐Ÿ“

It might be useful if a manager could set the FLOW TERMINATION PARAMETERS to different values for different tasks. Current meter implementations only have single ('whole meter') values for these parameters, and experience so far suggests that's a good enough amount of control for the tasks โœจ

6.2. Between Manager and Meter Reader: Control Functions ๐ŸŒˆ

okay so there are a bunch of parameters that have to be set for traffic flow measurement to work properly, and the good settings can change as network traffic changes ๐Ÿฆ‹ That's why dynamic network management is better than static meter configurations. A lot of these operations are about space tradeoffs. If the meter's memory runs out, either the collection interval has to go down or a coarser granularity of aggregation has to be used to cut the number of active flows ๐Ÿ˜ญ

Increasing the collection interval basically stores data in the meter. The usage data in transit is limited by the effective bandwidth of the virtual link between the meter and the meter reader. Those network resources are usually also carrying user data (which is the whole point of the network), so the traffic flow measurement traffic should be kept to an affordable fraction of the bandwidth. ("Affordable" is a policy decision made by the Network Operations personnel ๐Ÿ’ผ) Anyway, the operations below aren't independent variables. Each value you set has a direct, measurable effect on the others ๐Ÿ”ฅ

Network management operations, let's go:

  • MANAGER and METER READER IDENTIFICATION: the manager should make sure meters are read by the correct set of meter readers, and take steps to stop unauthorised access to usage information ๐Ÿ” The meter readers that get identified should be ready to poll if needed and accept data from the right meters. Alternate meter readers can be identified in case both the primary manager and the primary meter reader are unavailable. Alternate managers can be identified the same way ๐Ÿซถ
  • REPORTING INTERVAL CONTROL: the usual reporting interval should be picked to handle normal traffic patterns. But a meter can still run out of memory during traffic spikes, even with a properly set reporting interval ๐Ÿ“ˆ So there should be some mechanism for the meter to tell the manager it's in danger of running out of memory (by declaring a ' high water' condition ๐Ÿšจ). The manager can then arbitrate: by decreasing the polling interval, by letting nature take its course, or by telling the meter to ask for help sooner next time ๐Ÿ™ˆ
  • GRANULARITY CONTROL: this is the catch-all for every parameter that can be tuned and traded off to optimise the system's ability to reliably measure and store information on all the traffic (or as close to all of it as an administration needs) ๐Ÿง  Granularity:
    • Controls the amount of address information identifying each flow, and
    • Determines the number of buckets into which user traffic will be lumped together.

Since granularity is controlled by the meter's current rule set, the manager can only change it by asking the meter to switch to a different rule set ๐Ÿ’… That new rule set could be downloaded when it's needed, or it could have been downloaded as part of the meter's initial configuration.

  • FLOW LIFETIME CONTROL: flow termination parameters include timeout parameters for obsoleting inactive flows and removing them from tables, plus maximum flow lifetimes โณ This is intertwined with the reporting interval and granularity, so it has to be set in line with the other parameters ๐Ÿ’œ

6.3. Exception Conditions ๐Ÿ’€

Exception conditions have to be handled, especially when the meter runs out of space for flow data ๐Ÿ˜ฑ Packets can only be counted in a single flow, so that an active task never counts any packet twice. That means discarding records loses information. Here's how the mechanisms for dealing with it work:

  • METER OUTAGES: if a meter outage is coming (controlled restarts, etc.), the meter could send a trap to the manager. The manager could then ask one or more meter readers to pick up the data from the meter ๐ŸŒ™

After an uncontrolled outage like a power failure, the meter could send a trap to the manager saying it has restarted. The manager could then download the meter's correct rule set and tell the meter reader(s) that the meter is running again โœจ Or, the meter reader might notice from its regular poll that a meter has failed and restarted. It could then tell the manager about it, instead of relying on a trap from the meter ๐Ÿฆ‹

  • METER READER OUTAGES: if the collection system is down or isolated, the meter should try to tell the manager it can't communicate with the collection system ๐Ÿ“ก Usage data is kept in the flows' rolling counters, so it can be recovered once the meter reader restarts ๐Ÿซถ
  • MANAGER OUTAGES: if the manager fails for any reason, the meter should keep measuring, and the meter reader(s) should keep gathering usage records ๐ŸŒธ
  • BUFFER PROBLEMS: the network manager may notice a 'low memory' condition in the meter. This usually comes down to the interaction between these controls:
    • The reporting interval is too infrequent, or
    • The reporting granularity is too fine.

Either of these can get worse with low throughput or low bandwidth on the circuits carrying the usage data ๐Ÿข The manager can change any of these parameters in response to the meter's (or meter reader's) plea for help ๐Ÿฅบ

6.4. Standard Rule Sets ๐Ÿ“

the rule table is super flexible, but it can also get really complex ๐ŸŒ€ So it can help to develop some rule sets for common applications:

  • PROTOCOL TYPE: the meter records packets by protocol type. This is the default rule table for Traffic Flow Meters ๐Ÿงš
  • ADJACENT SYSTEMS: the meter records packets by the MAC address of the Adjacent Systems (the neighbouring originator or next-hop) ๐Ÿ›ฐ๏ธ (Variants on this table are "report source" or "report sink" only.) This strategy might be used by a regional or backbone network that wants to know how much aggregate traffic flows to or from its subscriber networks ๐ŸŒ
  • END SYSTEMS: the meter records packets by the IP address pair contained in the packet. (Variants on this table are "report source" or "report sink" only.) This strategy might be used by an End System network to get detailed host traffic matrix usage data ๐Ÿ’ป
  • TRANSPORT TYPE: the meter records packets by transport address. For IP packets, this gives usage information for the various IP services ๐ŸŽ€
  • HYBRID SYSTEMS: combinations of the above, e.g. for one interface report End Systems and for another interface report Adjacent Systems. This strategy might be used by an enterprise network to learn details about local usage and use an aggregate count for the shared regional network ๐Ÿขโœจ

7. Security Considerations ๐Ÿ”

โš ๏ธ Okay, this part is seriously important, so read it carefully. Security is not a vibe to skip past ๐Ÿ˜ค

7.1. Threat Analysis ๐Ÿ•ต๏ธ

A traffic flow measurement system may be subject to the following kinds of attacks:

  • ATTEMPTS TO DISABLE A TRAFFIC METER: an attacker may try to disrupt traffic measurement so that users can't be charged for network usage ๐Ÿ’ธ For example, a network probe sending packets to a large number of destination and transport addresses could cause a sudden rise in the number of flows in a meter's flow table, forcing it to use its coarser standby rule set ๐Ÿšจ
  • UNAUTHORIZED USE OF SYSTEM RESOURCES: an attacker may want to gain an advantage or cause mischief (e.g. denial of service) by subverting any of the system elements: meters, meter readers or managers ๐Ÿ˜ˆ
  • UNAUTHORIZED DISCLOSURE OF DATA: any data that's sensitive to disclosure can be read through active or passive attacks unless it's suitably protected ๐Ÿ‘€ Usage data may or may not be this type. Control messages, traps, etc. are not likely to be considered sensitive to disclosure.
  • UNAUTHORIZED ALTERATION, REPLACEMENT OR DESTRUCTION OF DATA: likewise, any data whose integrity is sensitive can be altered, replaced/injected or deleted through active or passive attacks unless it's suitably protected ๐Ÿ’€ Attackers may modify message streams to falsify usage data or interfere with the proper operation of the traffic flow measurement system. Therefore, all messages, both those containing usage data and those containing control data, should be considered vulnerable to such attacks ๐Ÿซฃ

7.2. Countermeasures โœจ

okay so bestie, here's how we actually protect this thing ๐Ÿ›ก๏ธ๐Ÿ’– the following countermeasures are recommended for the threats we just went through:

  • ATTEMPTS TO DISABLE A TRAFFIC METER can't be fully stopped, ngl ๐Ÿ˜ญ but flow data records from real network attacks have been super useful for figuring out what went down. The best move is to give the meter three or more times as much flow memory as it needs in normal operation, and to collect the flow data pretty often so recovering flow memory after an attack takes as little time as possible ๐Ÿงš
  • UNAUTHORIZED USE OF SYSTEM RESOURCES gets handled with authentication and access control services ๐Ÿ”
  • UNAUTHORIZED DISCLOSURE OF DATA gets handled with a confidentiality (encryption) service ๐Ÿซถ
  • UNAUTHORIZED ALTERATION, REPLACEMENT OR DESTRUCTION OF DATA gets handled with an integrity service ๐ŸŒธ

A Traffic Measurement system has to address all of these concerns, no exceptions ๐Ÿ’… since a high degree of protection is required, strong cryptographic methods are recommended. The security requirements for communication between each pair of traffic measurement system elements are in the table below. It's assumed that meters don't talk to other meters, and meter readers don't talk directly to other meter readers. (If they need to synchronize, the manager handles it, see Section 2.5.) Each entry in the table shows which kinds of security services are required. Here's the basic gist:

           Security Service Requirements for RTFM elements

  +------------------------------------------------------------------+
  | from\to |    meter     | meter reader | application |  manager   |
  |---------+--------------+--------------+-------------+------------|
  | meter   |     N/A      |  authent     |     N/A     |  authent   |
  |         |              |  acc ctrl    |             |  acc ctrl  |
  |         |              |  integrity   |             |            |
  |         |              |  confid **   |             |            |
  |---------+--------------+--------------+-------------+------------|
  | meter   |   authent    |     N/A      |  authent    |  authent   |
  | reader  |   acc ctrl   |              |  acc ctrl   |  acc ctrl  |
  |         |              |              |  integrity  |            |
  |         |              |              |  confid **  |            |
  |---------+--------------+--------------+-------------+------------|
  | appl    |     N/A      |  authent     |             |            |
  |         |              |  acc ctrl    |     ##      |    ##      |
  |---------+--------------+--------------+-------------+------------|
  | manager |  authent     |  authent     |     ##      |  authent   |
  |         |  acc ctrl    |  acc ctrl    |             |  acc ctrl  |
  |         |  integrity   |  integrity   |             |  integrity |
  +------------------------------------------------------------------+

N/A = Not Applicable ** = optional ## = outside RTFM scope

  • When any two elements talk to each other, they should mutually authenticate one another ๐Ÿค this shows up as 'authent' in the table. Once authentication is done, an element should check that the requested type of access is allowed, which is the 'acc ctrl' bit on the table ๐Ÿ”‘
  • Whenever information is transferred, its integrity should be protected ๐Ÿงฑ
  • Whenever usage data is transferred, it should be possible to make sure its confidentiality is protected, if it's sensitive to disclosure. That's the 'confid' label on the table ๐Ÿคซ

Security protocols aren't specified in this document, so no spoilers there ๐Ÿ™ˆ the system elements' management and collection protocols are responsible for providing enough data integrity, confidentiality, authentication and access control services. Main character behavior from the protocols, honestly ๐Ÿ‘‘


8. IANA Considerations ๐ŸŒˆ

The RTFM Architecture, as laid out in this document, has two sets of assigned numbers. This section covers how to assign them, using the example policies from the "Guidelines for IANA Considerations" document [IANA-RFC], so it's all very official ๐Ÿ’ผ

8.1. PME Opcodes ๐Ÿงš

The Pattern Matching Engine (PME) is a virtual machine that runs RTFM rules as its instructions. The PME opcodes show up in the 'action' field of an RTFM rule. The current list of opcodes, and their values for the PME's 'goto' and 'test' flags, are set out in section 4.4 above ("Rules and Rulesets).

The PME opcodes are pivotal to the whole RTFM architecture, since they must be implemented in every RTFM meter ๐Ÿ”ฅ so any new opcodes must be allocated through an IETF Consensus action [IANA-RFC]. No shortcuts, periodt ๐Ÿ’…

Opcodes are simply non-negative integers, but new opcodes should be allocated sequentially so the total opcode range stays as small as possible ๐Ÿ“

8.2. RTFM Attributes ๐ŸŽ€

Attribute numbers in the range 0-511 are globally unique, and they're allocated according to an IETF Consensus action [IANA-RFC]. Appendix C of this document allocates a basic (i.e. useful minimum) set of attributes, and those get numbers in the range 0 to 63. The RTFM working group is working on an extended set of attributes, which will have numbers in the range 64 to 127 โœจ

Vendor-specific attribute numbers are in the range 512-1023, and they'll be allocated using the First Come First Served policy [IANA-RFC]. Vendors who need attribute numbers should submit a request to IANA giving the attribute names, and IANA will allocate the next available numbers to them ๐Ÿฆ‹

Attribute numbers 1024 and higher are Reserved for Private Use [IANA-RFC]. Implementors who want to experiment with new attributes should use attribute numbers in this range ๐Ÿงช

Attribute numbers are simply non-negative integers. When writing specifications for attributes, implementors must give enough detail for the new attributes to be easily added to the RTFM Meter MIB [RTFM-MIB]. In particular, they must indicate whether the new attributes may be:

  • tested in an IF statement
  • saved by a SAVE statement or set by a STORE statement
  • read from an RTFM meter

(IF, SAVE and STORE are statements in the SRL Ruleset Language [RTFM-SRL] ๐Ÿ“)


9. APPENDICES ๐ŸŽ€

9.1. Appendix A: Network Characterisation ๐ŸŒ

Internet users have wildly diverse requirements. Networks differ in size, speed, throughput and processing power, among other things ๐ŸŒˆ there's a whole range of traffic flow measurement capabilities and requirements. For traffic flow measurement purposes, the Internet can be seen as a continuum that changes in character as traffic passes through these representative levels:

           International                    |
           Backbones/National        ---------------
                                    /               \
           Regional/MidLevel     ----------   ----------
                                /     \    \ /    /     \
           Stub/Enterprise     ---   ---   ---   ----   ----
                               |||   |||   |||   ||||   ||||
           End-Systems/Hosts   xxx   xxx   xxx   xxxx   xxxx

Note that mesh architectures can also be built out of these components, and these are just descriptive terms ๐Ÿ•ธ๏ธ a single network might include any or all of the descriptions below, although some networks are clearly just one type.

BACKBONE networks are typically bulk carriers that connect other networks ๐Ÿš› individual hosts (apart from network management devices and backbone service hosts) typically aren't directly connected to backbones.

REGIONAL networks are closely related to backbones, and they only differ in size, the number of networks connected through each port, and geographical coverage. Regionals may have directly connected hosts, acting as hybrid backbone/stub networks. A regional network is a SUBSCRIBER to the backbone ๐Ÿ’œ

STUB/ENTERPRISE networks connect hosts and local area networks. They are SUBSCRIBERS to regional and backbone networks ๐Ÿข

END SYSTEMS, colloquially HOSTS, are SUBSCRIBERS to any of the above networks ๐Ÿ’ป

Providing uniform identification of the SUBSCRIBER at a finer granularity than end-system (e.g. user/account) is beyond the scope of the current architecture ๐Ÿ™…โ€โ™€๏ธ although an optional attribute in the traffic flow measurement record may carry system-specific 'user identification' labels, so meters can use proprietary or non-standard schemes to attribute network traffic to responsible parties.

Initial recommended traffic flow measurement conventions are outlined here, organized by these Internet building blocks. It's important to understand what complexity reporting adds at each network level. The hierarchy is described top-down in the previous section, but reporting requirements are easier to address bottom-up ๐Ÿงฉ

            End-Systems
            Stub Networks
            Enterprise Networks
            Regional Networks
            Backbone Networks

END-SYSTEMS are currently responsible for allocating network usage to end-users, if that capability is wanted. From the Internet Protocol perspective, end-systems are the finest granularity that can be identified without protocol modifications ๐Ÿ” even if a meter broke protocol boundaries and tracked higher-level protocols, not all packets could be correctly allocated by user, and the definition of "user" itself varies widely from operating system to operating system (e.g. how to trace network usage back to users from shared processes) ๐Ÿ˜ตโ€๐Ÿ’ซ

STUB and ENTERPRISE networks will usually collect traffic data either by end-system network address or by network address pair, if detailed reporting is needed in the local area network. If no local reporting is required, they may record usage information in the exit router to track external traffic only ๐Ÿšช (These are the only networks which routinely use attributes to perform reporting at granularities finer than end-system or intermediate-system network address.)

REGIONAL networks are intermediate networks ๐ŸŒ‰ in some cases, subscribers will be enterprise networks, in which case the intermediate system network address is enough to identify the regional's immediate subscriber. In other cases, individual hosts or a disjoint group of hosts may constitute a subscriber. Then end-system network address pairs need to be tracked for those subscribers. When the source may be an aggregate entity (such as a network, or an adjacent router representing traffic from a world of hosts beyond) and the destination is a singular entity (or vice versa), the meter is said to be operating as a HYBRID system ๐Ÿชž

At the regional level, if the overhead is tolerable, it may be advantageous to report usage both by intermediate system network address (e.g. adjacent router address) and by end-system network address or end-system network address pair โœจ

BACKBONE networks are the highest level networks, operating at higher link speeds and traffic levels ๐Ÿš€ the high volume of traffic will in most cases rule out detailed traffic flow measurement. Backbone networks will usually account for traffic by their adjacent routers' network addresses ๐Ÿซก

more of this doc is coming, bestie ๐Ÿ’œ keep the vibes going

9.3. Appendix C: List of Defined Flow Attributes โœจ

okay so this appendix is basically the checklist of every flow attribute defined so far ๐Ÿ“‹ more will get added as the Traffic Measurement Architecture keeps growing ๐ŸŒฑ

heads up though, this table is only a super quick summary. the Meter MIB [RTFM-MIB] is the real source of truth for attributes and their allowed values ๐Ÿ’… also, MIB variables that represent flow attributes get 'flowData' stuck on the front of their names, which just tells you they live in the MIB's flowData table ๐Ÿงš

    0  Null

    4  SourceInterface        Integer     Source Address
    5  SourceAdjacentType     Integer
    6  SourceAdjacentAddress  String
    7  SourceAdjacentMask     String
    8  SourcePeerType         Integer
    9  SourcePeerAddress      String
   10  SourcePeerMask         String
   11  SourceTransType        Integer
   12  SourceTransAddress     String
   13  SourceTransMask        String

   14  DestInterface          Integer     Destination Address
   15  DestAdjacentType       Integer
   16  DestAdjacentAddress    String
   17  DestAdjacentMask       String
   18  DestPeerType           Integer
   19  DestPeerAddress        String
   20  DestPeerMask           String
   21  DestTransType          Integer
   22  DestTransAddress       String
   23  DestTransMask          String
   26  RuleSet                Integer     Meter attribute

   27  ToOctets               Integer     Source-to-Dest counters
   28  ToPDUs                 Integer
   29  FromOctets             Integer     Dest-to-Source counters
   30  FromPDUs               Integer
   31  FirstTime              Timestamp   Activity times
   32  LastActiveTime         Timestamp
   33  SourceSubscriberID     String      Session attributes
   34  DestSubscriberID       String
   35  SessionID              String

   36  SourceClass            Integer     'Computed' attributes
   37  DestClass              Integer
   38  FlowClass              Integer
   39  SourceKind             Integer
   40  DestKind               Integer
   41  FlowKind               Integer

   50  MatchingStoD           Integer     PME variable

   51  v1                     Integer     Meter Variables
   52  v2                     Integer
   53  v3                     Integer
   54  v4                     Integer
   55  v5                     Integer

   65
   ..  'Extended' attributes (to be defined by the RTFM working group)
  127

9.4. Appendix D: List of Meter Control Variables ๐ŸŽ›๏ธ

here's the lineup of meter control variables, sorted by where they live ๐Ÿ’–

    Meter variables:
       Flood Mark                    Percentage
       Inactivity Timeout (seconds)  Integer

    'per task' variables:
       Current Rule Set Number       Integer
       Standby Rule Set Number       Integer
       High Water Mark               Percentage

    'per reader' variables:
       Reader Last Time              Timestamp

9.5. Appendix E: Changes Introduced Since RFC 2063 ๐Ÿฆ‹

okay, history lesson time ๐Ÿ“œ the first version of the Traffic Flow Measurement Architecture came out as RFC 2063 back in January 1997. here are the biggest glow-ups since then ๐Ÿ’…

  • a Traffic Meter can now run multiple rule sets at the same time ๐Ÿ”ฅ that makes a meter way more useful, and it only needed minimal changes to the architecture
  • 'NoMatch' replaced 'Fail' as an action ๐Ÿ™ˆ the name was agreed on at the Working Group 1996 meeting in Montreal. it makes more sense, since a match failing doesn't mean it's over: it might get tried again with the packet's addresses reversed
  • the 'MatchingStoD' attribute was added โœจ it's a Packet Matching Engine (PME) attribute that tells you addresses are being matched in StoD (i.e. 'wire') order. you can use it to do different things when the match gets retried, which makes some kinds of rule sets way simpler. it was discussed and agreed to at the San Jose meeting in 1996
  • computed attributes (Class and Kind) can now be tested inside a rule set ๐Ÿ™Œ this got rid of an earlier restriction that was honestly not needed
  • the attribute numbers list got extended with ranges: one for 'basic' attributes (in this document) and one for 'extended' attributes (still being developed by the RTFM Working Group) ๐ŸŒˆ
  • the 'Security Considerations' section was completely rewritten ๐Ÿ” it now covers an evaluation of traffic measurement security risks and their countermeasures

10. Acknowledgments ๐Ÿซถ

an initial draft of this document was produced under the IETF's Internet Accounting Working Group, with help from the SNMP, RMON and SAAG working groups ๐Ÿ’œ big thanks to Stephen Stibler (IBM Research) for his patient, careful comments while this memo was being put together ๐ŸŒธ

11. References ๐Ÿ“š

  • [802-3] IEEE 802.3/ISO 8802-3 Information Processing Systems - Local Area Networks - Part 3: Carrier sense multiple access with collision detection (CSMA/CD) access method and physical layer specifications, 2nd edition, September 21, 1990.
  • [ACT-BKG] Mills, C., Hirsch, G. and G. Ruth, "Internet Accounting Background", RFC 1272, November 1991.
  • [IANA-RFC] Alvestrand, H. and T. Narten, "Guidelines for Writing an IANA Considerations Section in RFCs", BCP 26, RFC 2434, October 1998.
  • [IPPM-FRM] Paxson, V., Almes, G., Mahdavi, J. and M. Mathis, "Framework for IP Performance Metrics", RFC 2330, May 1998.
  • [OSI-ACT] International Standards Organisation (ISO), "Management Framework", Part 4 of Information Processing Systems Open Systems Interconnection Basic Reference Model, ISO 7498-4, 1994.
  • [RTFM-MIB] Brownlee, N., "Traffic Flow Measurement: Meter MIB", RFC 2720, October 1999.
  • [RTFM-NEW] Handelman, S., Stibler, S., Brownlee, N. and G. Ruth, "RTFM: New Attributes for Traffic Flow Measurment", RFC 2724, October 1999.
  • [RTFM-SRL] Brownlee, N., "SRL: A Language for Describing Traffic Flows and Specifying Actions for Flow Groups", RFC 2723, October 1999.

12. Authors' Addresses ๐Ÿ’Œ

Nevil Brownlee
Information Technology Systems & Services
The University of Auckland
Private Bag 92-019
Auckland, New Zealand

Phone: +64 9 373 7599 x8941
EMail: n.brownlee@auckland.ac.nz


Cyndi Mills
GTE Laboratories, Inc
40 Sylvan Rd.
Waltham, MA 02451, U.S.A.

Phone: +1 781 466 4278
EMail: cmills@gte.com


Greg Ruth
GTE Internetworking
3 Van de Graaff Drive
P.O. Box 3073
Burlington, MA 01803, U.S.A.

Phone: +1 781 262 4831
EMail: gruth@bbn.com
Copyright (C) The Internet Society (1999). All Rights Reserved.

This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English.

The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns.

This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Acknowledgement ๐ŸŒท

Funding for the RFC Editor function is currently provided by the Internet Society. ๐Ÿ’–

๐Ÿ“ฆ where this page comes from