RFC 2063
Traffic Flow Measurement: Architecture
RFC 2063: Traffic Flow Measurement: Architecture 🦋
Part 1 of 9 ✨
Okay bestie, so this one is from January 1997 and it's written by three people: N. Brownlee (The University of Auckland), C. Mills (BBN Systems and Technologies), and G. Ruth (GTE Laboratories, Inc.). It's labelled Category: Experimental 🧪, which means it's an experiment and not an Internet standard. The header doesn't list anything it obsoletes or updates, so it's standing on its own 💅.
Status of this Memo 📜
Let's get the official vibe check out of the way first. This memo defines an Experimental Protocol for the Internet community, and it does not specify an Internet standard of any kind. The authors are asking for discussion and suggestions for improvement, and distribution of the memo is unlimited 🌸.
Abstract 🎀
This document describes an architecture for measuring and reporting network traffic flows. It talks about how that fits into a bigger network traffic flow architecture, and how it can be used on the Internet. It's meant to be a starting point for the Realtime Traffic Flow Measurement Working Group 🫶.
Table of Contents 🗂️
Here's the full table of contents, copied exactly, page numbers and all:
1 Statement of Purpose and Scope 2
2 Traffic Flow Measurement Architecture 4
2.1 Meters and Traffic Flows . . . . . . . . . . . . . . . . . . 4
2.2 Interaction Between METER and METER READER . . . . . . . . . 6
2.3 Interaction Between MANAGER and METER . . . . . . . . . . . 6
2.4 Interaction Between MANAGER and METER READER . . . . . . . . 7
2.5 Multiple METERs or METER READERs . . . . . . . . . . . . . . 7
2.6 Interaction Between MANAGERs (MANAGER - MANAGER) . . . . . . 8
2.7 METER READERs and APPLICATIONs . . . . . . . . . . . . . . . 8
3 Traffic Flows and Reporting Granularity 9
3.1 Flows and their Attributes . . . . . . . . . . . . . . . . . 9
3.2 Granularity of Flow Measurements . . . . . . . . . . . . . . 11
3.3 Rolling Counters, Timestamps, Report-in-One-Bucket-Only . . 13
4 Meters 15
4.1 Meter Structure . . . . . . . . . . . . . . . . . . . . . . 15
4.2 Flow Table . . . . . . . . . . . . . . . . . . . . . . . . . 17
4.3 Packet Handling, Packet Matching . . . . . . . . . . . . . . 17
4.4 Rules and Rule Sets . . . . . . . . . . . . . . . . . . . . 21
4.5 Maintaining the Flow Table . . . . . . . . . . . . . . . . . 24
4.6 Handling Increasing Traffic Levels . . . . . . . . . . . . . 25
5 Meter Readers 26
5.1 Identifying Flows in Flow Records . . . . . . . . . . . . . 26
5.2 Usage Records, Flow Data Files . . . . . . . . . . . . . . . 27
5.3 Meter to Meter Reader: Usage Record Transmission. . . . . . 27
6 Managers 28
6.1 Between Manager and Meter: Control Functions . . . . . . . 28
6.2 Between Manager and Meter Reader: Control Functions . . . 29
6.3 Exception Conditions . . . . . . . . . . . . . . . . . . . . 31
6.4 Standard Rule Sets . . . . . . . . . . . . . . . . . . . . 32
7 APPENDICES 33
7.1 Appendix A: Network Characterisation . . . . . . . . . . . . 33
7.2 Appendix B: Recommended Traffic Flow Measurement Capabilities 34
7.3 Appendix C: List of Defined Flow Attributes . . . . . . . . 35
7.4 Appendix D: List of Meter Control Variables . . . . . . . . 36
8 Acknowledgments 36
9 References 37
10 Security Considerations 37
11 Authors' Addresses 371 Statement of Purpose and Scope 🌈
This section is the "why are we even doing this" part 💖. It describes an architecture for traffic flow measurement and reporting in data networks, and it has the following characteristics:
- The traffic flow model can be consistently applied to any protocol or application at any network layer (for example, the network, transport, or application layers) 🧚.
- Traffic flow attributes are defined so they're valid for multiple networking protocol stacks, and traffic flow measurement implementations are useful in MULTI-PROTOCOL environments 👑.
- Users can state their traffic flow measurement requirements in a simple way. That lets them collect the flow data they need while ignoring everything else 🍓.
- The data reduction work needed to produce the requested traffic flow information happens as close as possible to the network measurement point. This cuts down the volume of data that has to be obtained (and sent across the network for storage), and it minimises the amount of processing needed in traffic flow analysis applications 🔥.
The architecture defines common metrics for measuring traffic flows. Since everyone uses the same metrics, traffic flow data can be swapped and compared across lots of different platforms 🫶. This data is useful for:
- Understanding how existing networks behave 🔍,
- Planning network development and expansion 🌸,
- Quantifying network performance 📊,
- Verifying the quality of network service, and 💅
- Attributing network usage to users 🎀.
The traffic flow measurement architecture is deliberately built so specific protocol implementations can stretch coverage into multi-protocol environments and into other protocol layers. An example would be usage measurement for application-level services ✨. Using the same model for both network-level and application-level measurement may make it easier to build generic analysis applications that process and/or correlate any or all levels of traffic and usage information 🧠. In this document, the term 'usage data' is used as a generic term for the data obtained using the traffic flow measurement architecture 💜.
This document is not a protocol specification. Instead, it specifies and structures the information a traffic flow measurement system needs to collect, describes requirements that such a system must meet, and outlines tradeoffs an implementor may make 🧩.
For performance reasons, it may be desirable to use traffic information gathered through traffic flow measurement in place of network statistics obtained in other ways 🚀. The quantification of network performance isn't the main purpose of this architecture, but the measured traffic flow data may be used as an indication of network performance 📈.
A cost recovery structure decides "who pays for what." The big question here is how to build a tariff: who gets billed, how much, for which things, and based on what information 💸. Tariff issues include fairness, predictability (how well subscribers can forecast their network charges), practicality (of gathering the data and administering the tariff), incentives (for example, encouraging off-peak use), and cost recovery goals (100% recovery, subsidisation, or profit making) 🙈. This document doesn't cover issues like these 🤷♀️.
Background information explaining why this approach was selected is provided by 'Traffic Flow Measurement: Background' RFC [1] 📚.
2 Traffic Flow Measurement Architecture 🦋
Okay, now we're getting into the good stuff 🔥. A traffic flow measurement system is used by network Operations personnel to manage and develop a network. It gives them a tool for measuring and understanding the network's traffic flows 👀. This information is useful for lots of purposes, as mentioned in section 1 (above) 🌸.
The following sections outline a model for traffic flow measurement. The model draws from working drafts of the OSI accounting model [2] 🧁. Future extensions are expected as the model gets refined to cover additional protocol layers 🌈.
(part 1 of 9, more coming bestie 🫶💖)
2.1 Meters and Traffic Flows 🦋
okay so the big idea here is that the whole traffic measurement model is built around little network entities called traffic METERS 🧚 Meters count stuff like how many packets and bytes showed up, and then they sort those numbers into ACCOUNTABLE ENTITIES using other details like source and destination addresses. An accountable entity is basically whoever (or whatever) is responsible for some activity on the network 💅 It could be a user, a host, a network, a group of networks, etc, depending on how granular the meter's configuration is set to be. bestie this is giving total main character energy 💖
We assume that routers or traffic monitors all over a network are equipped with meters to measure traffic 🌸 Where you put the meters is its own whole topic, and it's covered in the 'Traffic Flow Measurement: Background' RFC [1] ✨ The really iconic thing about meters is that they summarize entity usage information in a super succinct way 🫶
Now for the purpose of traffic flow measurement, we define a TRAFFIC FLOW 🎀 It's an artificial logical equivalent to a call or connection, basically a chunk of traffic that has a start and stop time and was generated by one particular accountable entity 😭 The attribute values for a flow (source/destination addresses, packet counts, byte counts, etc.) are aggregate quantities that reflect everything happening in the DURATION between the start and stop times. The start time is fixed for a given flow, but the end time can keep moving later as the flow ages 🌈
Here's the tea 🍓 For connectionless protocols like IP, there is by definition no way to tell whether a packet with a particular source/destination combo is part of a stream of packets or not, because each packet is completely independent 💀 So a traffic meter has a set of 'rules' in its configuration that specify which flows it cares about, based on their attribute values. It pulls attribute values out of each packet it sees and uses them to decide which flow that packet belongs to 🧚 Sorting packets into 'flows' like this is an economical and practical way to measure network traffic and pin it on accountable entities. slay 👑
Usage information that can't be derived from traffic flows might also be interesting ✨ For example, an application might want to record accesses to different information resources, or a host might want to record the username (subscriber id) for a particular network session 🙈 The traffic flow architecture has provision for doing this. In the future the measurement model will be extended to gather this kind of information from applications and hosts so it can provide values for higher-layer flow attributes 🦋
So as well as FLOWS and METERS, the model includes MANAGERS, METER READERS and ANALYSIS APPLICAIONS, which get explained in the following sections 💜 The diagram below shows how they relate to each other. The numbers on the diagram point to sections in this document 🎀
MANAGER
/ \
2.3 / \ 2.4
/ \
/ \ ANALYSIS
METER <-----> METER READER <-----> APPLICATION
2.2 2.7- MANAGER: A traffic measurement manager is an application that configures 'meter' entities and controls 'meter reader' entities 👑 It uses the data requirements of analysis applications to figure out the right configuration for each meter, and to make sure each meter reader is running properly. It can be super convenient to combine the functions of meter reader and manager into a single network entity 💅
- METER: Meters get placed at measurement points that network Operations personnel decide on 🌸 Each meter selectively records network activity as its configuration tells it to. It can also aggregate, transform and further process the recorded activity before the data is stored. The processed and stored results are called the 'usage data.' 🍓
- METER READER: A meter reader reliably transports usage data from meters so that it's available to analysis applications 🫶
- ANALYSIS APPLICATION: An analysis application processes the usage data to produce information and reports that are useful for network engineering and management purposes 🔥 Examples include:
- TRAFFIC FLOW MATRICES, showing the total flow rates for many of the possible paths within an internet 🧚
- FLOW RATE FREQUENCY DISTRIBUTIONS, indicating how flow rates vary with time ⭐
- USAGE DATA showing the total traffic volumes sent and received by particular hosts 💖
The whole traffic measurement system is best understood by looking at how its components interact with each other 🦋 Those interactions are described in the following sections ✨
2.2 Interaction Between METER and METER READER 🌈
The information traveling along this path is the usage data itself 💜 A meter holds usage data in an array of flow data records called the FLOW TABLE 🎀 A meter reader can collect the data in any suitable manner. For example, it might upload a copy of the whole flow table using a file transfer protocol, or read the records in the current flow set one at a time using a suitable data transfer protocol 🫶 Note that the meter reader doesn't need to read complete flow data records, since a subset of their attribute values may well be sufficient ✨
A meter reader may collect usage data from one or more meters 🌸 Data may be collected from the meters at any time. There is no requirement for collections to be synchronized in any way 🦋
2.3 Interaction Between MANAGER and METER 💅
A manager is responsible for configuring and controlling one or more meters 👑 At the time of writing, a meter can only be controlled by a single manager; in the future this restriction may be relaxed 🧚 Each meter's configuration includes information such as:
- Flow specifications, e.g. which traffic flows are to be measured, how they are to be aggregated, and any data the meter is required to compute for each flow being measured 🍓
- Meter control parameters, e.g. the maximum size of its flow table, and the 'inactivity' time for flows. If no packets belonging to a flow are seen for this time, the flow is considered to have ended, i.e. to have become idle 🙈
- Sampling rate. Normally every packet will be observed. It may sometimes be necessary to use sampling techniques to observe only some of the packets 🌈 (Sampling algorithms are not prescribed by the architecture; it should be noted that before using sampling one should verify the statistical validity of the algorithm used 🫶) Current experience with the measurement architecture shows that a carefully-designed and implemented meter compresses the data such that on normal LANs and WANs of today, sampling is really not needed 💖
2.4 Interaction Between MANAGER and METER READER 🎀
A manager is responsible for configuring and controlling one or more meter readers 💜 A meter reader may only be controlled by a single manager 👑 A meter reader needs to know at least the following for every meter it is collecting usage data from:
- The meter's unique identity, i.e. its network name or address 🦋
- How often usage data is to be collected from the meter ⭐
- Which flow records are to be collected (e.g. all active flows, the whole flow table, flows seen since a given time, etc.) 🧚
- Which attribute values are to be collected for the required flow records (e.g. all attributes, or a small subset of them) 🌸
Since redundant reporting may be used to increase the reliability of usage data, exchanges among multiple entities must be considered as well 🫶 These are discussed below ✨
2.5 Multiple METERs or METER READERs 🌸
-- METER READER A --
/ | \
/ | \
=====METER 1 METER 2=====METER 3 METER 4=====
\ | /
\ | /
-- METER READER B --Several uniquely identified meters may report to one or more meter readers 🧚 The diagram above gives an example of how multiple meters and meter readers could be used 🦋
In the diagram above, meter 1 is read by meter reader A, and meter 4 is read by meter reader B. Meters 1 and 4 have no redundancy, so if either fails, usage data for their network segments will be lost 😭
Meters 2 and 3, however, measure traffic on the same network segment 🍓 One of them may fail, leaving the other collecting the segment's usage data. Meters 2 and 3 are read by meter reader A and by meter reader B. If one meter reader fails, the other will continue collecting usage data 💪
The architecture does not require multiple meter readers to be synchronized 🌈 In the situation above, meter readers A and B could both collect usage data at the same intervals, but not necessarily at the same times. Note that because collections are asynchronous, it is unlikely that usage records from two different meter readers will agree exactly 💀
If precisely synchronized collections are required, this can be achieved by having one manager request each meter to begin collecting a new set of flows, then allowing all meter readers to collect the usage data from the old sets of flows 🎀
If there is only one meter reader and it fails, the meters keep running 🫶 When the meter reader is restarted, it can collect all of the accumulated flow data. Should this happen, time resolution will be lost (because of the missed collections), but overall traffic flow information will not ✨ The only exception would occur if traffic volume was high enough to 'roll over' counters for some flows during the failure; this is addressed in the section on 'Rolling Counters.' 🔥
2.6 Interaction Between MANAGERs (MANAGER - MANAGER) 👑
Synchronization between multiple management systems is the province of network management protocols 💅 This traffic flow measurement architecture specifies only the network management controls necessary to perform the traffic flow measurement function. It does not address the more global issues of simultaneous or interleaved (possibly conflicting) commands from multiple network management stations, or the process of transferring control from one network management station to another 🙈
2.7 METER READERs and APPLICATIONs 🦋
Once a collection of usage data has been assembled by a meter reader, it can be processed by an analysis application 🧚 Details of analysis applications, such as the reports they produce and the data they require, are outside the scope of this architecture 💜
It should be noted, however, that analysis applications will often require considerable amounts of input data 😭 An important part of running a traffic flow measurement system is the storage and regular reduction of flow data so as to produce daily, weekly or monthly summary files for further analysis 🍓 Again, details of such data handling are outside the scope of this architecture 🌸
3 Traffic Flows and Reporting Granularity ✨
A flow was defined in section 2.1 above in abstract terms as follows 🎀
"A TRAFFIC FLOW is an artifical logical equivalent to a call or connection, belonging to an ACCOUNTABLE ENTITY."
In practical terms, a flow is a stream of packets passing across a network between two end points (or being sent from a single end point), which have been summarized by a traffic meter for analysis purposes 🦋
3.1. Flows and their Attributes 💖
okay so every traffic meter keeps a table of 'flow records' 🦋 for the flows it has seen. each flow record holds the values of the ATTRIBUTES it cares about, and these can include:
- ADDRESSES for the flow's source and destination. these are the protocol type, the source and destination addresses at various network layers (pulled straight from the packet), and the number of the interface the packet was observed on.
- TIMES: the first and last times packets were seen for this flow, aka the 'creation' and 'last activity' times ⏰
- COUNTS for the 'forward' (source to destination) and 'backward' (destination to source) parts of the flow's traffic, like packets and bytes 📦 how 'source' and 'destination' get assigned for flows is explained in the packet matching section below.
- OTHER attributes, like info the meter computes itself ✨
a flow's ACCOUNTABLE ENTITY is specified by the values of its ADDRESS attributes. for example, if a flow's address attributes only say source address = IP address 10.1.0.1, then every IP packet from and to that address gets counted in that flow. if the address list says source address = IP address 10.1.0.1, destination address = IP address 26.1.0.1, then only IP packets between 10.1.0.1 and 26.1.0.1 get counted. so specific 🫶
the addresses that specify a flow's address attributes can be one or more of these types:
- The INTERFACE NUMBER for the flow, aka the interface the meter measured the traffic on. together with a unique address for the meter, this uniquely identifies one particular physical-level port 🔌
- The ADJACENT ADDRESS, aka the [n-1] layer address of the immediate source or destination on the packet's path. like, if flow measurement happens at the IP layer on an Ethernet LAN [3], an adjacent address is a six-octet Media Access Control (MAC) address. for a host on the same LAN segment as the meter, the adjacent address is that host's MAC address. for hosts on other LAN segments, it's the MAC address of the adjacent (upstream or downstream) router carrying the traffic flow 🗺️
- The PEER ADDRESS, which identifies the source or destination of the PEER-LEVEL packet. the form of a peer address depends on the network-layer protocol in use, and on the network layer [n] where the traffic measurement is happening 🧩
- The TRANSPORT ADDRESS, which identifies the source or destination port for the packet, aka its [n+1] layer address. for example, if flow measurement happens at the IP layer, a transport address is a two-octet UDP or TCP port number 🚪
those four definitions give addresses for each of the four lowest layers of the OSI reference model: Physical, Link, Network and Transport layer. a FLOW RECORD stores both the VALUE for each of its addresses (as described above) and a MASK that says which bits of the address value are being used and which are ignored. ngl the mask is a whole vibe 🎀 if address bits are being ignored, the meter sets them to zero, but their actual values are undefined 🙈
one of the key features of the traffic measurement architecture is that attributes basically mean the same thing across different protocols, so analysis applications can use the same reporting formats for all protocols. this is easy for peer addresses: even though the form of addresses differs between protocols, the meaning of a 'peer address' stays the same. it gets harder at higher layers. for example, at the Network layer, IP, Novell IPX and AppleTalk all use port numbers as a 'transport address,' but CLNP and DECnet have no notion of ports 😭 more work is needed here, especially picking attributes that will suit the higher layers of the OSI reference model 🌈
reporting by adjacent intermediate sources and destinations, or just by meter interface (most useful when the meter is embedded in a router), supports the hierarchical Internet reporting schemes described in the 'Traffic Flow Measurement: Background' RFC [1]. that means backbone and regional networks can measure usage down to just the next lower level of granularity (the regional and stub/enterprise levels, respectively), and the final breakdown by end user (e.g. by source IP address) is handled by the stub/enterprise networks 🌸
in cases where network addresses are dynamically allocated (like mobile subscribers), further subscriber identification is necessary if flows are going to be ascribed to individual users 🧚 this is where an optional SUBSCRIBER ID comes in, as part of the flow id. a subscriber ID may be associated with a particular flow either through the current rule set, or by proprietary means within a meter (for example, via protocol exchanges with one or more multi-user hosts). right now a subscriber ID is an arbitrary text string. later versions of the architecture may specify its contents in more detail 💜
3.2. Granularity of Flow Measurements 🦋
GRANULARITY is the 'control knob' an application and/or the meter uses to trade off the overhead of usage reporting against the level of detail you get back 🎛️ coarser granularity means more aggregation, finer granularity means more detail. so the number of flows measured (and stored) at a meter can be regulated by changing the granularity of the accountable entity, the attributes, or the time intervals. flows are like an adjustable pipe: lots of fine-granularity streams can carry the data with each stream measured individually, or the data can get bundled into one coarse-granularity pipe 🚰
flow granularity is controlled by adjusting the level of detail at which these get reported:
- The accountable entity (address attributes, discussed above).
- The categorisation of packets (other attributes, discussed below).
- The lifetime/duration of flows (the reporting interval needs to be short enough to measure them with sufficient precision) ⏱️
the set of rules that decides each packet's accountable entity is known as the meter's CURRENT RULE SET. the current rule set becomes an integral part of the reported information, aka the recorded usage info can't be properly interpreted without knowing the rules that were used to collect it 🧠
the rule set is basically part of the receipt. no rules, no understanding the numbers 💀
settings for these granularity factors can vary from meter to meter. they're determined by the meter's current rule set, so they'll change if network Operations personnel reconfigure the meter to use a new rule set. collection rules are expected to change pretty rarely, but the rule set in effect at any time must be identifiable via a RULE SET ID. granularity of accountable entities is further specified by additional ATTRIBUTES, including:
- Meter variables, such as the index of the flow's record in the flow table, and the rule set id for the rules the meter was running while the flow was observed. these values give a way to tell apart flows the meter observed at different times 🕰️
- Attributes that record information derived from other attribute values. six of these are defined (SourceClass, DestClass, FlowClass, SourceKind, DestKind, FlowKind), and their meaning is determined by the meter's rule set. for example, you could have a subroutine in the rule set that figures out whether a source or destination peer address is a member of an arbitrary list of networks, and sets SourceClass/DestClass to one if the source/dest peer address is in the list, or to zero otherwise 🧮
- Administratively specified attributes, like Quality Of Service and Priority, etc. these aren't defined at this time 🤷♀️
- Higher-layer (especially application-level) attributes. these aren't defined at this time either 🫠
settings for these granularity factors can vary from meter to meter. they're determined by the meter's current rule set, so they'll change if network Operations personnel reconfigure the meter to use a new rule set 🔁
the LIFETIME of a flow is the time interval that began when the meter observed the flow's first packet and ended when it saw the last one. flow lifetimes are super variable, but many (if not most) are pretty short 🐣 a meter can't measure lifetimes directly. instead, a meter reader collects usage data for flows that have been active since the last collection, and an analysis application can compare the data from each collection to figure out when each flow actually stopped 🕵️♀️
the meter does, however, need to reclaim memory (aka records in the flow table) being held by idle flows 🧹 the meter configuration includes a variable called InactivityTimeout, which specifies the minimum time a meter must wait before it recovers the flow's record. also, before recovering a flow record, the meter must be sure that the flow's data has been collected by at least one meter reader 🫶
these 'lifetime' issues get covered further in the section on meter readers (below). a complete list of the attributes currently defined is given in Appendix C later in this document 📚
3.3 Rolling Counters, Timestamps, Report-in-One-Bucket-Only ✨
Okay so once a usage record gets sent, you have to decide: do you wipe the existing flow records, or keep them around and keep adding to their counts as more traffic on that same flow shows up? The second option is called rolling counters, and bestie, it's the one that's recommended 🎀. It has a few perks, and the biggest is reliability. The system can often survive losing some usage records, like if a meter reader crashes and restarts later. The next usage record very often has another reading of a lot of the same flow buckets that were in the lost one 🫶. That continuity can also be used for "sanity" checks on the data, to catch errors in the calculations 🧮.
Rolling counters do bring a new puzzle though: how do you tell a follow-on flow record apart from a brand new one? 🤔 Check out this example 👀
CONTINUING FLOW OLD FLOW, then NEW FLOW
start time = 1 start time = 1
Usage record N: flow count = 2000 flow count = 2000 (done)
start time = 1 start time = 5
Usage record N+1: flow count = 3000 new flow count = 1000
Total count: 3000 3000So in the continuing flow case, the same flow got reported when its count was 2000, and then again at 3000. The total count to date is 3000, easy 💅. In the OLD/NEW case, the old flow had a count of 2000. Its record then got stopped (maybe because of temporary idleness, or MAX LIFETIME policy). Then more traffic with the same characteristics showed up, so a new flow record started and quickly hit a count of 1000. The total from both the old and new records is 3000 ✨.
The flow START TIMESTAMP attribute is what resolves this. In the example above, the CONTINUING FLOW record in the second usage record has an old FLOW START timestamp, while the NEW FLOW has a recent FLOW START timestamp. Iykyk 🙈.
Each packet gets counted in one and only one flow, so it doesn't get counted twice. The record of a single flow is informally called a "bucket" 🪣. If you need multiple, sometimes overlapping, records of usage information (aggregate, individual, etc), the network manager should collect the counts at a granularity that's detailed enough that the aggregate and combination counts can be rebuilt during post-processing of the raw usage data 🧠.
For example, say a meter needs to record both 'total packets coming in interface #1' and 'total packets arriving from any interface sourced by IP address = a.b.c.d.' You could declare a bucket for each case, but then it's not clear how to handle a packet that satisfies both criteria. It must only be counted once. By default it gets counted in the first bucket it qualifies for, and not in the other one. Also, you can't rebuild this information in post-processing. The fix is to define not two, but THREE buckets, each one collecting a unique combination of the two criteria:
Bucket 1: Packets which came in interface 1,
AND were sourced by IP address a.b.c.d
Bucket 2: Packets which came in interface 1,
AND were NOT sourced by IP address a.b.c.d
Bucket 3: Packets which did NOT come in interface 1,
AND were sourced by IP address a.b.c.d
(Bucket 4: Packets which did NOT come in interface 1,
AND NOT sourced by IP address a.b.c.d)Now the desired info can be rebuilt in post-processing. "Total packets coming in interface 1" comes from adding buckets 1 & 2, and "Total packets sourced by IP address a.b.c.d" comes from adding buckets 1 & 3. Bucket 4 isn't explicitly needed, since nobody's interested in its info, but it's included in parentheses for completeness 💖.
4 Meters 🧚
A traffic flow meter is a device for collecting data about traffic flows at a given point within a network. We'll call that point the METERING POINT 📍. The header of every packet that passes the network metering point gets offered to the traffic meter program.
A meter could be built in a few different ways, including:
- A dedicated small host, connected to a LAN (so it can see all packets as they pass by) and running a 'traffic meter' program. The metering point is the LAN segment the meter is attached to.
- A multiprocessing system with one or more network interfaces, with drivers that let a traffic meter program see packets. In this case the system provides multiple metering points, so traffic flows on any subset of its network interfaces can be measured 🌈.
- A packet-forwarding device such as a router or switch. This is similar to the option above, except that every received packet should also be forwarded, usually out a different interface 🚀.
The discussion in the following sections assumes that a meter may only run a single rule set. It's possible, though, for a meter to run several rule sets at the same time, matching each packet against every active rule set and producing a single flow table with flows from all the active rule sets. The overall effect is similar to running several independent meters, one for each rule set 👯.
4.1 Meter Structure 🏗️
Here's an outline of the meter's structure in the following diagram.
Briefly, the meter works like this:
- Incoming packet headers arrive at the top left of the diagram and get passed to the PACKET PROCESSOR.
- The packet processor passes them to the Packet Matching Engine (PME), where they get classified.
- The PME is a Virtual Machine running a pattern matching program contained in the CURRENT RULE SET. It gets invoked by the Packet Processor, and returns instructions on what to do with the packet.
- Some packets are classified as 'to be ignored.' The Packet Processor discards them 🗑️.
- Other packets are matched by the PME, which returns a FLOW KEY describing the flow the packet belongs to.
- The flow key is used to locate the flow's entry in the FLOW TABLE. A new entry gets created when a flow is first seen. The entry's packet and byte counters get updated 🔢.
- A meter reader may collect data from the flow table at any time. It may use the 'collect' index to locate the flows to be collected within the flow table 📥.
packet +------------------+
header | Current Rule Set |
| +--------+---------+
| |
+--------*---------+ +----------*-------------+
| Packet Processor |<----->| Packet Matching Engine |
+--+------------+--+ +------------------------+
| |
Ignore * | Count via flow key
|
+--*--------------+
| 'Search' index |
+--------+--------+
|
+--------*--------+
| |
| Flow Table |
| |
+--------+--------+
|
+--------*--------+
| 'Collect' index |
+--------+--------+
|
*
Meter Reader4.2 Flow Table 📋
Every traffic meter keeps a table of TRAFFIC FLOW RECORDS for the flows it has seen. A flow record holds attribute values for its flow, including:
- Addresses for the flow's source and destination. These include addresses and masks for various network layers (pulled out of the packet), plus the number of the interface the packet was observed on.
- First and last times when packets were seen for this flow ⏰.
- Counts for the 'forward' (source to destination) and 'backward' (destination to source) parts of the flow's traffic.
- Other attributes, e.g. the state of the flow record (more on that below) 🫧.
A flow record can be in one of these states:
- INACTIVE: The flow record isn't being used by the meter 😴.
- CURRENT: The record is in use and describes a flow that belongs to the 'current flow set,' i.e. the set of flows the meter has seen recently 🌟.
- IDLE: The record is in use and the flow it describes is part of the current flow set. On top of that, no packets belonging to this flow have been seen for a period specified by the meter's InactivityTime variable 💤.
4.3 Packet Handling, Packet Matching ✨
okay so bestie, every packet header the traffic meter program receives goes through the same little routine 💅 Step one is to pull the attribute values out of the packet header and use them to build a MATCH KEY for the packet. Step two is to match that key against the current rule set, which I'll explain in detail below 🎀
- Extract attribute values from the packet header and use them to create a MATCH KEY for the packet.
- Match the packet's key against the current rule set, as explained in detail below.
The rule set decides whether the packet gets counted or ignored 🙈 If it gets counted, the matching process spits out a FLOW KEY for the flow the packet belongs to. The meter uses that flow key to look up the flow's record in the flow table. If there's no record for that flow yet, a brand new flow record might get created ✨ Then the counts on the matching flow record go up. Iconic 🦋
Here's an example. The rule set could say that packets to or from any host in IP network 130.216 get counted. It could also say that flow records should be made for every pair of 24-bit (Class C) subnets inside network 130.216 🌸
Each packet's match key gets passed to the meter's PATTERN MATCHING ENGINE (PME) for matching. The PME is a Virtual Machine that runs a set of instructions called RULES. In other words, a RULE SET is a program for the PME 💜 A packet's match key holds an interface number, a source address (S) and a destination address (D). It doesn't hold any attribute masks for those attributes, though, just the values 🧚
Now, if measured flows were unidirectional (meaning only the counted packets going in one direction), life would be easy 🍓 The PME would get called once for the packet. Any flow key from a successful match would find the flow's record in the flow table, and that flow's counters would get updated. Done and dusted.
But flows are bidirectional 😭 They reflect the forward and reverse packets of a protocol interchange, aka a 'session.' Keeping two sets of counters in the meter's flow record makes the flow data way easier to deal with, because analysis programs don't have to stitch the 'forward' and 'reverse' parts of a session back together 🫶 Bidirectional flows are harder for the meter, though, since it has to figure out whether a packet is a 'forward' one or a 'reverse' one. To work that out, the meter will often have to call the PME twice, once for each possible packet direction. The meter is working hard, bestie 💪
The diagram below shows the algorithm the traffic meter uses to process each packet. You read it from left to right and top to bottom, so from the top left corner down to the bottom right corner 🌈 S is the flow's source address (its set of source address attribute values from the packet), and D is its destination address.
Ignore
--- match(S->D) -------------------------------------------------+
| Suc | Fail |
| | Ignore |
| match(D->S) -----------------------------------------+
| | Suc | Fail |
| | | |
| | +-------------------------------------------+
| | |
| | Suc |
| current(D->S) ---------- count(D->S,r) --------------+
| | Fail |
| | |
| create(D->S) ----------- count(D->S,r) --------------+
| |
| Suc |
current(S->D) ------------------ count(S->D,f) --------------+
| Fail |
| Suc |
current(D->S) ------------------ count(D->S,r) --------------+
| Fail |
| |
create(S->D) ------------------- count(S->D,f) --------------+
|
*There are several cases to think about, babe 🧐 They are:
- The packet is recognised as one which is TO BE IGNORED.
- The packet MATCHES IN BOTH DIRECTIONS. One situation where this could happen is a rule set that matches flows within network X (Source = X, Dest = X) but says flows should be created for each subnet within network X, say subnets y and z. If, for example, a packet is seen for y->z, the meter has to check that flow z->y isn't already current before it creates y->z 👀
- The packet MATCHES IN ONE DIRECTION ONLY. If its flow is already current, its forward or reverse counters go up. Otherwise it gets added to the flow table and then counted ✨
The algorithm uses four functions, and they go like this 💖
match(A->B) is the PME. It uses the meter's current rule set to match the attribute values in the packet's match key. A->B means the assumed source address is A and the destination address is B, so the packet was travelling from A to B. match() returns one of three results:
'Ignore'means the packet matched, but this flow isn't supposed to be counted.'Fail'means the packet didn't match. It might still match with its direction reversed, i.e. from B to A 🙃'Suc'means the packet did match, so it belongs to a flow that's going to be counted 🔥
current(A->B) succeeds if the flow A-to-B is current, meaning it has a record in the flow table whose state is Current. Otherwise it fails 💅
create(A->B) adds the flow A-to-B to the flow table. It sets the values for the attributes that stay constant, like addresses, and zeroes out the flow's counters 🌸
count(A->B,f) increments the 'forward' counters for flow A-to-B. count(A->B,r) increments the 'reverse' counters for flow A-to-B. 'Forward' here means the counters for packets travelling from A to B. Fun fact: count(A->B,f) is identical to count(B->A,r) 🦋
Heads up, bestie: when you write rule sets, remember that the meter will normally try to match each packet in both directions. It's super important that your rule set doesn't contain inconsistencies that mess with this process 🚨
Let's say a rule set counts packets from source network A to destination network B, but ignores packets from source network B. That's a classic inconsistent rule set 💀 Packets from network B should be counted as reverse packets for the A-to-B flow, so ignoring them breaks everything.
There are two ways to dodge this problem. One is to design a language for specifying rule files and write a compiler for it, which makes it way easier to produce correct rule sets 🧚♀️ The other is to write a 'rule set consistency checker' program that can catch problems in hand-written rule sets 🔍
For now, the best way to avoid these problems is to write rule sets that only classify flows in the forward direction, and then trust the meter to handle the reverse-travelling packets 🫶💖
4.4. Rules and Rule Sets 📜✨
okay so a rule set is basically an array of rules 🎀. Rule sets live inside a meter as entries in an array of rule sets. One member of that array is the CURRENT RULE SET, which is just the one the meter is actually using to sort incoming packets rn 💅.
Rule set 1 is built into the meter and cannot be changed 🔒. It runs when the meter starts up and gives a super coarse reporting granularity. Its main job is proving the meter is alive before a "useful" rule set gets downloaded 🧚.
If the meter is told to use rule set 0, it stops measuring 💀. Every packet gets ignored until a non-zero rule set is made current.
Each rule is structured like this:
+-------- test ---------+ +---- action -----+
attribute & mask = value: opcode, parameter;Opcodes have two flags, goto and test 🦋. The PME keeps a Boolean indicator called the 'test indicator,' which starts out set (on). Execution begins at rule 1, the first rule in the set, and goes like this:
- If the test indicator is on:
- Perform the test, i.e. AND the attribute value with the mask and compare it with the value.
- If these are equal, the test has succeeded, so perform the rule's action (below).
- If the test fails, execute the next rule in the rule set.
- If there are no more rules in the rule set, return from the match() function indicating failure.
- If the test indicator is off, or the test (above) succeeded:
- Set the test indicator to this rule's test flag value.
- Determine the next rule to execute.
- If the opcode has its goto flag set, its parameter value specifies the number of the next rule.
- Opcodes which don't have their goto flags set either determine the next rule in special ways (Return), or they terminate execution (Ignore, Fail, Count, CountPkt).
- Perform the action.
The PME maintains two 'history' data structures 📚. The first, the 'return' stack, simply records the index (i.e. 1-origin rule number) of each Gosub rule as it is executed, and Return rules pop their Gosub rule index. The second, the 'pattern' queue, is used to save information for later use in building a flow key. A flow key is built by zeroing all its attribute values, then copying attribute and mask information from the pattern stack in the order it was enqueued.
The opcodes are:
opcode goto test
1 Ignore 0 -
2 Fail 0 -
3 Count 0 -
4 CountPkt 0 -
5 Return 0 0
6 Gosub 1 1
7 GosubAct 1 0
8 Assign 1 1
9 AssignAct 1 0
10 Goto 1 1
11 GotoAct 1 0
12 PushRuleTo 1 1
13 PushRuleToAct 1 0
14 PushPktTo 1 1
15 PushPktToAct 1 0And here's what each one actually does 🔥:
- Ignore: Stop matching, and return from the match() function indicating that the packet is to be ignored.
- Fail: Stop matching, and return from the match() function indicating failure.
- Count: Stop matching. Save this rule's attribute name, mask and value in the PME's pattern queue, then construct a flow key for the flow to which this packet belongs. Return from the match() function indicating success. The meter will use the flow key to locate the flow record for this packet's flow.
- CountPkt: As for Count, except that the masked value from the packet is saved in the PME's pattern queue instead of the rule's value.
- Gosub: Call a rule-matching subroutine. Push the current rule number on the PME's return stack, set the test indicator, then goto the specified rule.
- GosubAct: Same as Gosub, except that the test indicator is cleared before going to the specified rule.
- Return: Return from a rule-matching subroutine. Pop the number of the calling gosub rule from the PME's 'return' stack and add this rule's parameter value to it to determine the 'target' rule. Clear the test indicator, then goto the target rule.
A subroutine call looks like a Gosub rule followed by a small group of following rules 🌸. Since a Return action clears the test flag, the action of one of these 'following' rules will be executed. This lets the subroutine return a result, on top of any info it saved in the PME's pattern queue.
- Assign: Set the attribute specified in this rule to the value specified in this rule. Set the test indicator, then goto the specified rule.
- AssignAct: Same as Assign, except that the test indicator is cleared before going to the specified rule.
- Goto: Set the test indicator, then goto the specified rule.
- GotoAct: Clear the test indicator, then goto the specified rule.
- PushRuleTo: Save this rule's attribute name, mask and value in the PME's pattern queue. Set the test indicator, then goto the specified rule.
- PushRuleToAct: Same as PushRuleTo, except that the test indicator is cleared before going to the specified rule.
PushRuleTo actions can be used to save the value and mask used in a test, or (if the test is not performed) to save an arbitrary value and mask 🦋.
- PushPktTo: Save this rule's attribute name and mask, together with the masked value from the packet, in the PME's pattern queue. SET the test indicator, then goto the specified rule.
- PushPktToAct: Same as PushPktTo, except that the test indicator is cleared before going to the specified rule.
PushPktTo actions can be used to save a value from the packet using a specified mask. The test in PushPktTo rules will almost never be executed 🙈.
As well as the attributes that apply directly to packets (such as SourcePeerAddress, DestTransAddress, etc.), the PME implements several further attributes. These are:
Null: Tests performed on theNullattribute always succeed.v1 .. v5:v1,v2,v3,v4andv5are 'meter variables'. They provide a way to pass parameters into rule-matching subroutines. Each may hold the name of a normal attribute, and its value is set by an Assign action. When a meter variable appears as the attribute of a rule, its value specifies the actual attribute to be tested. For example, ifv1had been assignedSourcePeerAddressas its value, a rule withv1as its attribute would actually testSourcePeerAddress. 🧚SourceClass,DestClass,FlowClass,SourceKind,DestKind,FlowKind: These six attributes may be set by executingPushRuletoactions. They allow the PME to save (in flow records) information which has been built up during matching. Since their values are only defined when matching is complete (and the flow key is built), their values may not be tested in rules 💀.
4.5. Maintaining the Flow Table 🗂️💖
The flow table may be thought of as a 1-origin array of flow records. (A particular implementation may, of course, use whatever data structure is most suitable.) When the meter starts up there are no known flows, and all the flow records are in the 'inactive' state 😴.
Each time a packet is seen for a flow that is not in the current flow set, a flow record is set up for it, and the state of that record is 'current.' When selecting a record for the new flow, the meter searches the flow table for an 'inactive' record. There is no particular significance in the ordering of records within the table 🌈.
Flow data may be collected by a 'meter reader' at any time. There is no requirement for collections to be synchronized. The reader can collect the data in any suitable manner. For example, it could upload a copy of the whole flow table using a file transfer protocol, or it could read the records in the current flow set row by row using a suitable data transfer protocol 📥.
The meter keeps information about collections. In particular, it maintains a LastCollectTime variable, which remembers the time the last collection was made. A second variable, InactivityTime, specifies the minimum time the meter will wait before considering that a flow is idle ⏰.
The meter must recover records used for idle flows, if only to prevent it running out of flow records. Recovered flow records are returned to the 'inactive' state. A variety of recovery strategies are possible, including the following 👑:
One possible recovery strategy is to recover idle flow records as soon as possible after their data has been collected. To implement this, the meter could run a background process which scans the flow table looking for 'current' flows whose 'last packet' time is earlier than the meter's LastCollectTime. This would be suitable when one is interested in measuring flow lifetimes ⭐.
Another recovery strategy is to leave idle flows alone for as long as possible. This would be suitable if one was only interested in measuring total traffic volumes. It could be implemented by having the meter search for collected idle flows only when it runs out of 'inactive' flow records 🍓.
One further factor a meter should consider before recovering a flow is the number of meter readers which have collected the flow's data. If there are multiple meter readers operating, network Operations personnel should be able to specify the minimum number of meters (or perhaps a specific list of meters) which should collect a flow's data before its memory can be recovered. This issue will be further developed in the future 🔮.
4.6. Handling Increasing Traffic Levels 📈🫶
Under normal conditions, the meter reader specifies which set of usage records it wants to collect, and the meter provides them 💜.
If memory usage rises above the high-water mark, the meter should switch to a STANDBY RULE SET so as to increase the granularity of flow collection and decrease the rate at which new flows are created 🚨. When the manager, usually as part of a regular poll, becomes aware that the meter is using its standby rule set, it could decrease the interval between collections. The meter should also increase its efforts to recover flow memory, so as to reduce the number of idle flows in memory. When the situation returns to normal, the manager may request the meter to switch back to its normal rule set ✨.
5. Meter Readers 📬✨
okay so a meter (think a router 🖥️) piles up usage data for as long as its memory allows. Then meter readers come by at regular reporting intervals to grab it, on whatever schedule the manager has specified. 💅 Everything they collect gets saved to a disk file called a FLOW DATA FILE 💾, which is built out of a bunch of USAGE RECORDS.
The next sections cover what usage records and flow data files look like. heads up tho, the architecture does not pin down the details of these at this stage 🙈. A common format for them would be a seriously worthwhile thing to build later.
5.1. Identifying Flows in Flow Records
Once a packet has been classified and is ready to be counted, there has to be a matching flow data record already in the flow table. If there isn't one, it gets created 🧁. The flow record is super flexible, so any identification attributes that aren't needed can just be left out. Which attributes go in, and what values they get, is decided by the current rule set.
Here's the lowkey important bit 🔑: the combination of start time, rule set id and subscript (row number in the flow table) gives you a unique flow identifier, no matter what the other attributes are.
The current rule set can also add extra info, like a computed attribute value such as FlowKind, into the attribute section of the usage record. So if a particular flow is matched by the rule set, its flow record should be marked not only with the qualifying identification attributes, but also with that extra info 🎀. Since several flows can each carry the same FlowKind value, the resulting usage records can be used in post-processing, or between the meter reader and the meter as a criterion for collection. it's giving organisation 💜.
5.2. Usage Records, Flow Data Files
The collected usage data is stored in flow data files on the meter reader, with one file for each meter 🗂️. Besides the measured usage data, flow data files must contain information that uniquely identifies the meter the data came from 🪪.
A USAGE RECORD contains the descriptions of, and values for, one or more flows. Quantities are counted as number of packets and number of bytes per flow 🔢. Each usage record contains the entity identifier of the meter (a network address), a timestamp, and a list of reported flows (FLOW DATA RECORDS). A meter reader builds up a file of usage records by regularly collecting flow data from a meter, using that data to make usage records, and concatenating them onto the tail of a file. That file is called a FLOW DATA FILE 🦋.
A usage record contains the following information in some form:
+-------------------------------------------------------------------+
| RECORD IDENTIFIERS: |
| Meter Id (& digital signature if required) |
| Timestamp |
| Collection Rules ID |
+-------------------------------------------------------------------+
| FLOW IDENTIFIERS: | COUNTERS |
| Address List | Packet Count |
| Subscriber ID (Optional) | Byte Count |
| Attributes (Optional) | Flow Start/Stop Time |
+-------------------------------------------------------------------+5.3. Meter to Meter Reader: Usage Record Transmission
Usage records are literally the reason this whole system exists 💖. So the accuracy, reliability, and security of transmission are the primary concerns of the meter/meter reader exchange 🔐. Errors can happen on networks, and Internet packets may get dropped, so some mechanism is needed to make sure the usage info arrives intact.
Flow data moves from meter to meter reader through a series of protocol exchanges between them 🔁. This can happen in various ways, like moving individual attribute values, complete flows, or the entire flow table (i.e. all the active flows). One possible method is to use SNMP 🛰️; the 'Traffic Flow Measurement: Meter MIB' document [4] gives the details. Note that this is simply one example, because the transfer of flow data from meter to meter reader is not specified in this document.
The reliability of the data transfer method under light, normal, and extreme network loads should be understood before you pick among collection methods 🧐.
In normal operation the meter runs a rule file that provides the required degree of flow reporting granularity. The meter reader(s) collect the flow data often enough to let the meter's garbage collection mechanism keep memory usage stable 🧹.
In the worst case, traffic may climb to the point where the meter is in danger of running completely out of flow memory 😭. The meter implementor has to decide how to handle this. For example, they could switch to a default (extremely coarse granularity) rule set, send a trap to the manager, or try dumping flow data to the meter reader.
Users of the Traffic Flow Measurement system should analyse their requirements carefully. They need to decide for themselves whether it matters more to collect flow data at normal granularity (raising the collection frequency as needed to keep up with traffic volumes), or to accept flow data at a coarser granularity. Similarly, it may be fine to lose flow data for a short time, in return for making sure the meter keeps running properly and isn't overwhelmed by rising traffic levels 🫶.
6. Managers 👑
A manager configures meters and controls meter readers. It does this through the interactions described below 💅.
6.1. Between Manager and Meter: Control Functions
- DOWNLOAD RULE SET: A meter may hold an array of rule sets. One of these, the 'default' rule set, is built into the meter and cannot be changed. The others must be downloaded by the manager 📥. A manager may use any suitable protocol exchange to do this, for example an FTP file transfer or a series of SNMP SETs, one for each row of the rule set.
- SWITCH TO SPECIFIED RULE SET: Once the rule sets have been downloaded, the manager must tell the meter which rule set it is to actually run (i.e. which is to be the current rule set), and which is to be the standby rule set 🔀.
- SET HIGH WATER MARK: A percentage value that the meter interprets to know when to switch to its standby rule set. This increases the granularity of the flows and conserves the meter's flow memory 🧠. Once that has happened, the manager may also change the polling frequency or the meter's control parameters, so the meter can recover memory from idle flows faster 🏃♀️.
If the high traffic levels persist, the meter's normal rule set may have to be rewritten to permanently reduce the reporting granularity 📉.
- SET FLOW TERMINATION PARAMETERS: The meter should have the good sense, in situations where lack of resources may cause data loss, to purge flow records from its tables 🧽. Such records may include:
- Flows that have already been reported to at least one meter reader, and show no activity since the last report,
- Oldest flows, or
- Flows with the smallest number of unreported packets.
- SET INACTIVITY TIMEOUT: This is a time in seconds since the last packet was seen for a flow ⏱️. Flow records may be reclaimed if they have been idle for at least this amount of time, and have been collected in accordance with the current collection criteria.
6.2. Between Manager and Meter Reader: Control Functions
Lots of parameters have to be set for traffic flow measurement to work properly 🎛️. Viable settings can change as network traffic characteristics change, so dynamic network management is preferred over static meter configurations 🌊. A lot of these operations come down to space tradeoffs. If memory at the meter runs out, either the reporting interval has to go down or a coarser granularity of aggregation has to be used, so more data fits into less space 📦.
Increasing the reporting interval effectively stores data in the meter. Usage data in transit is limited by the effective bandwidth of the virtual link between the meter and the meter reader. These limited network resources are usually also used to carry user data (the whole purpose of the network), so the level of traffic flow measurement traffic should be kept to an affordable fraction of the bandwidth 💸. ("Affordable" is a policy decision made by the network Operations personnel.) At any rate, it must be understood that the operations below do not represent the setting of independent variables. On the contrary, each value set has a direct and measurable effect on the behaviour of the other variables 🔗.
Network management operations follow:
- MANAGER and METER READER IDENTIFICATION: The manager should make sure meters report to the correct set of collection stations, and take steps to prevent unauthorised access to usage information 🔒. The collection stations identified should be prepared to poll if necessary, and to accept data from the appropriate meters. Alternate collection stations may be identified in case both the primary manager and the primary collection station are unavailable. Similarly, alternate managers may be identified 🧑🤝🧑.
- REPORTING INTERVAL CONTROL: The usual reporting interval should be selected to cope with normal traffic patterns. However, a meter may still run out of memory during traffic spikes even with a correctly set reporting interval 📈. So some mechanism must be available for the meter to tell the manager it is in danger of exhausting its memory (by declaring a 'high water' condition). The manager can then arbitrate, by decreasing the polling interval, letting nature take its course, or by telling the meter to ask for help sooner next time 🙏.
- GRANULARITY CONTROL: Granularity control is a catch-all for all the parameters that can be tuned and traded off to optimise the system's ability to reliably measure and store information on all the traffic (or as close to all the traffic as an administration requires) 🎯. Granularity:
- Controls flow-id granularities for each interface, and
- Determines the number of buckets into which user traffic will be lumped together 🪣.
Since granularity is controlled by the meter's current rule set, the manager can only change it by requesting that the meter switch to a different rule set. The new rule set could be downloaded when required, or it could have been downloaded as part of the meter's initial configuration 🧁.
- FLOW LIFETIME CONTROL: Flow termination parameters include timeout parameters for obsoleting inactive flows and removing them from tables, as well as maximum flow lifetimes ⏳. This is intertwined with reporting interval and granularity, so it must be set in accordance with the other parameters 🧶.
6.3 Exception Conditions ✨
okay so exception conditions are a whole thing and they must be handled, especially when the meter runs out of buffer space 😭 bestie, the tea is that packets can only be counted in one flow at any given time (so nothing gets counted twice 💅), which means if you toss records you lose information. here's how we deal with it:
- METER OUTAGES: if a meter outage is coming (controlled crashes, etc.) the meter could send a trap to the manager. then the manager could ask one or more meter readers to grab the usage record from the meter 🧚
after an uncontrolled outage like a power failure, the meter could send a trap to the manager saying it restarted. then the manager could download the meter's correct rule set and tell the meter reader(s) that the meter is back up ✨ or, alternatively, the meter reader might notice from its regular poll that a meter failed and restarted. then it could tell the manager directly, instead of waiting on a trap from the meter 🦋
- METER READER OUTAGES: if the collection system is down or isolated, the meter should try to let the manager know it can't reach the collection system 📞 the usage data is safe though! it lives in the flows' rolling counters, so it can be recovered once the meter reader is restarted 🫶
- MANAGER OUTAGES: if the manager fails for any reason, the meter should keep measuring and the meter reader(s) should keep gathering usage records. the show goes on 💖
- BUFFER PROBLEMS: the network manager might realise there's a 'low memory' condition in the meter 🥺 this can usually be traced back to how these controls interact:
- The reporting interval is too infrequent,
- The reporting granularity is too fine, or
- The throughput/bandwidth of circuits carrying the usage data is too low.
the manager may change any of these parameters when the meter (or meter reader's) begs for help 🙏
6.4 Standard Rule Sets 🍓
the rule table is super flexible, but ngl it can get really complex 🥴 so it's helpful to make some rule sets for the common stuff:
- PROTOCOL TYPE: the meter records packets by protocol type. this will be the default rule table for Traffic Flow Meters 🎀
- ADJACENT SYSTEMS: the meter records packets by the MAC address of the Adjacent Systems (neighbouring originator or next-hop). (variants on this table are "report source" or "report sink" only.) this strategy might be used by a regional or backbone network that wants to know how much aggregate traffic flows to or from its subscriber networks 🌈
- END SYSTEMS: the meter records packets by the IP address pair contained in the packet. (variants on this table are "report source" or "report sink" only.) this strategy might be used by an End System network to get detailed host traffic matrix usage data 👑
- TRANSPORT TYPE: the meter records packets by transport address. for IP packets this gives usage info for the various IP services 💜
- HYBRID SYSTEMS: combinations of the above, e.g. for one interface report End Systems, and for another interface report Adjacent Systems. this strategy might be used by an enterprise network that wants detail about local usage and an aggregate count for the shared regional network 🫶
7 APPENDICES 🎀
7.1 Appendix A: Network Characterisation 🌸
internet users have wildly diverse requirements, bestie 🦋 networks differ in size, speed, throughput, and processing power (among other things), and there's a whole range of traffic flow measurement capabilities and requirements. for traffic flow measurement purposes, the Internet can be viewed as a continuum that changes character as traffic passes through these representative levels:
International |
Backbones/National ---------------
/ \
Regional/MidLevel ---------- ----------
/ \ \ / / \
Stub/Enterprise --- --- --- ---- ----
||| ||| ||| |||| ||||
End-Systems/Hosts xxx xxx xxx xxxx xxxxnote that mesh architectures can also be built out of these components, and these are just descriptive terms 💅 a single network can cover any or all of the descriptions below, although some networks can clearly be identified as a single type.
BACKBONE networks are typically bulk carriers that connect other networks. individual hosts (except for network management devices and backbone service hosts) typically aren't directly connected to backbones 🌈
REGIONAL networks are closely related to backbones and differ only in size, the number of networks connected via each port, and geographical coverage. regionals may have directly connected hosts, acting as hybrid backbone/stub networks. a regional network is a SUBSCRIBER to the backbone 🧚
STUB/ENTERPRISE networks connect hosts and local area networks. STUB/ENTERPRISE networks are SUBSCRIBERS to regional and backbone networks ✨
END SYSTEMS, colloquially HOSTS, are SUBSCRIBERS to any of the above networks 💖
providing a uniform identification of the SUBSCRIBER at a finer granularity than end-system (e.g. user/account) is beyond the scope of the current architecture 🙈 but an optional attribute in the traffic flow measurement record may carry system-specific "accountable (billable) party" labels, so meters can implement proprietary or non-standard schemes for attributing network traffic to responsible parties 🍓
7.2 Appendix B: Recommended Traffic Flow Measurement Capabilities 🦋
here are the initial recommended traffic flow measurement conventions, organised by these Internet building blocks. it's important to understand what complexity reporting adds at each network level. the hierarchy was described top-down in the previous section, but reporting requirements are easier to address bottom-up 🫶
End-Systems
Stub Networks
Enterprise Networks
Regional Networks
Backbone NetworksEND-SYSTEMS are currently responsible for allocating network usage to end-users, if that capability is wanted 💅 from the Internet Protocol perspective, end-systems are the finest granularity that can be identified without protocol modifications. even if a meter broke protocol boundaries and tracked higher-level protocols, not all packets could be correctly allocated by user, and the definition of user varies too much from operating system to operating system (e.g. how to trace network usage back to users from shared processes) 🤯
STUB and ENTERPRISE networks will usually collect traffic data either by end-system network address or by network address pair if detailed reporting is needed in the local area network 🎀 if no local reporting is needed, they may record usage in the exit router to track external traffic only. (these are the only networks that routinely use attributes to report at granularities finer than end-system or intermediate-system network address 👑)
REGIONAL networks are intermediate networks 🌸 in some cases, subscribers will be enterprise networks, and then the intermediate system network address is enough to identify the regional's immediate subscriber. in other cases, individual hosts or a disjoint group of hosts may be a subscriber, so end-system network address pairs need to be tracked for those subscribers. when the source may be an aggregate entity (such as a network, or an adjacent router representing traffic from a world of hosts beyond) and the destination is a singular entity (or vice versa), the meter is said to be operating as a HYBRID system 🌈
at the regional level, if the overhead is tolerable, it may be advantageous to report usage both by intermediate system network address (e.g. adjacent router address) and by end-system network address or end-system network address pair ✨
BACKBONE networks are the highest-level networks, operating at the highest link speeds and traffic levels 🚀 the high volume of traffic will in most cases rule out detailed traffic flow measurement, so backbone networks will usually account for traffic by adjacent routers' network addresses 💜
7.3 Appendix C: List of Defined Flow Attributes 💖
this appendix is a checklist of the attributes defined so far 📋 more will be added later as the Traffic Measurement Architecture develops further 🧚
0 Null
1 Flow Subscript Integer Flow table info
2 Flow Status Integer
4 Source Interface Integer Source Address
5 Source Adjacent Type Integer
6 Source Adjacent Address String
7 Source Adjacent Mask String
8 Source Peer Type Integer
9 Source Peer Address String
10 Source Peer Mask String
11 Source Trans Type Integer
12 Source Trans Address String
13 Source Trans Mask String
14 Destination Interface Integer Destination Address
15 Destination Adjacent Type Integer
16 Destination Adjacent Address String
17 Destination AdjacentMask String
18 Destination PeerType Integer
19 Destination PeerAddress String
20 Destination PeerMask String
21 Destination TransType Integer
22 Destination TransAddress String
23 Destination TransMask String
24 Packet Scale Factor Integer 'Other' attributes
25 Byte Scale Factor Integer
26 Rule Set Number Integer
27 Forward Bytes Counter Source-to-Dest counters
28 Forward Packets Counter
29 Reverse Bytes Counter Dest-to-Source counters
30 Reverse Packets Counter
31 First Time TimeTicks Activity times
32 Last Active Time TimeTicks
33 Source Subscriber ID String Session attributes
34 Destination Subscriber ID String
35 Session ID String
36 Source Class Integer 'Computed' attributes
37 Destination Class Integer
38 Flow Class Integer
39 Source Kind Integer
40 Destination Kind Integer
41 Flow Kind Integer
51 V1 Integer Meter variables
52 V2 Integer
53 V3 Integer
54 V4 Integer
55 V5 Integer7.4 Appendix D: List of Meter Control Variables 🦄
Current Rule Set Number Integer
Standby Rule Set Number Integer
High Water Mark Percentage
Flood Mark Percentage
Inactivity Timeout (seconds) Integer
Last Collect Time TimeTicks8. Acknowledgments ✨
This document was first put together under the IETF's Internet Accounting Working Group, with help from the SNMP, RMON and SAAG working groups 🫶. This version writes down the implementation work the Internet Accounting Working Group did, and it's meant to be a starting point for the Realtime Traffic Flow Measurement Working Group 🌸. Huge thanks to Stephen Stibler (IBM Research) for his patient and careful comments while this memo was being prepared 💖.
9. References 📚
[1] Mills, C., Hirsch, G. and G. Ruth, "Internet Accounting Background", RFC 1272, Bolt Beranek and Newman Inc., Meridian Technology Corporation, November 1991.
[2] International Standards Organisation (ISO), "Management Framework," Part 4 of Information Processing Systems Open Systems Interconnection Basic Reference Model, ISO 7498-4, 1994.
[3] IEEE 802.3/ISO 8802-3 Information Processing Systems - Local Area Networks - Part 3: Carrier sense multiple access with collision detection (CSMA/CD) access method and physical layer specifications, 2nd edition, September 21, 1990.
[4] Brownlee, N., "Traffic Flow Measurement: Meter MIB", RFC 2064, The University of Auckland, January 1997.
10. Security Considerations 🔐
Okay bestie, real talk, this one's a bit of a plot twist 😭. Security issues are not discussed in detail in this document. Instead, the meter's management and collection protocols are the ones on the hook for providing sufficient data integrity and confidentiality 💀. So if you're building or running this stuff, the protection has to come from those protocols, not from the meter spec itself 🛡️✨.
11. Authors' Addresses 🦋
Here's who wrote this, with their contact details, so you can reach out if you're confused 🌈.
Nevil Brownlee Information Technology Systems & Services The University of Auckland
Phone: +64 9 373 7599 x8941
EMail: n.brownlee @auckland.ac.nzCyndi Mills BBN Systems and Technologies
Phone: +1 617 873 4143
EMail: cmills@bbn.comGreg Ruth GTE Laboratories, Inc
Phone: +1 617 466 2448
EMail: gruth@gte.comAnd that's the whole RFC, bestie! 🎀💅👑
📦 where this page comes from
- ietf: rfc2063 1997-01
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms