RFC 3955

Evaluation of Candidate Protocols for IP Flow Information Export (IPFIX)

📅 published October 2004, from ietf

RFC 3955: Evaluation of Candidate Protocols for IP Flow Information Export (IPFIX) 🦋

okay bestie, this is RFC 3955 💖 It's an Informational doc from October 2004, written by S. Leinen at SWITCH 🎀. Informational means it's a review for the internet community, not a standard. It doesn't obsolete or update anything, since the title block doesn't list any. The whole thing is about picking which protocol should become the basis for IPFIX ✨

Status of this Memo 📜

This memo gives information for the Internet community 🌸 It does not specify an Internet standard of any kind. Distribution of this memo is unlimited 💜

Copyright (C) The Internet Society (2004) 🌈

Abstract 🫶

So this doc evaluates the five candidate protocols for IPFIX 🔥 The evaluation is based on the requirements document made by the IPFIX Working Group. The protocols get characterized, grouped into broad categories, and checked against specific requirements 🍓 The big conclusion? A recommendation to pick NetFlow v9 as the basis for the IPFIX spec 👑 periodt 💅

Table of Contents 🧚

1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . .   2
2. Protocol Summaries . . . . . . . . . . . . . . . . . . . . . .   2
   2.1.  CRANE. . . . . . . . . . . . . . . . . . . . . . . . . .   3
   2.2.  Diameter . . . . . . . . . . . . . . . . . . . . . . . .   4
   2.3.  LFAP . . . . . . . . . . . . . . . . . . . . . . . . . .   4
   2.4.  NetFlow v9 . . . . . . . . . . . . . . . . . . . . . . .   5
   2.5.  Streaming IPDR . . . . . . . . . . . . . . . . . . . . .   6
3. Broad Classification of Candidate Protocols .  . . . . . . . .   7
   3.1.  Design Goals . . . . . . . . . . . . . . . . . . . . . .   7
   3.2.  Data Representation. . . . . . . . . . . . . . . . . . .   8
   3.3.  Protocol Flow. . . . . . . . . . . . . . . . . . . . . .   9
4. Item-Level Compliance Evaluation . . . . . . . . . . . . . . .  10
   4.1.  Meter Reliability (5.1). . . . . . . . . . . . . . . . .  10
   4.2.  Sampling (5.2) . . . . . . . . . . . . . . . . . . . . .  11
   4.3.  Overload Behavior (5.3). . . . . . . . . . . . . . . . .  12
   4.4.  Timestamps (5.4) . . . . . . . . . . . . . . . . . . . .  12
   4.5.  Time Synchronization (5.5) . . . . . . . . . . . . . . .  12
   4.6.  Flow Expiration (5.6). . . . . . . . . . . . . . . . . .  13
   4.7.  Ignore Port Copy (5.9) . . . . . . . . . . . . . . . . .  13
   4.8.  Information Model (6.1). . . . . . . . . . . . . . . . .  13
   4.9.  Data Model (6.2) . . . . . . . . . . . . . . . . . . . .  13
   4.10. Data Transfer (6.3). . . . . . . . . . . . . . . . . . .  14
5. Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . .  18
   5.1.  Recommendation . . . . . . . . . . . . . . . . . . . . .  19
6. Security Considerations. . . . . . . . . . . . . . . . . . . .  19
7. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . .  19
8. References . . . . . . . . . . . . . . . . . . . . . . . . . .  20
   8.1.  Normative References . . . . . . . . . . . . . . . . . .  20
   8.2.  Informative References . . . . . . . . . . . . . . . . .  20
Appendix.  A Note on References to the Candidate Protocol
           Documents. . . . . . . . . . . . . . . . . . . . . . .  22
Author's Address. . . . . . . . . . . . . . . . . . . . . . . . .  22
Full Copyright Statement. . . . . . . . . . . . . . . . . . . . .  23

1. Introduction ✨

okay so the IP Flow Information Export (IPFIX) Working Group was chartered to pick a protocol for exporting flow information from traffic-observing devices, like routers or dedicated probes 🦋 An evaluation team was formed to review the submitted protocols. Each protocol had an advocate, who wrote a specific evaluation document for their protocol against the requirements document [1] 💅 The spec for each protocol was one or several Internet-Drafts, and sometimes those referred normatively to documents from outside the IETF 🌈

This document is the evaluation of those submitted protocols against the requirements document, and on a bigger picture level, against the working group charter 🎀

The following IPFIX candidate protocol submissions were evaluated:

  • CRANE [7], [8]
  • Diameter [9], [10]
  • LFAP [11], [12], [13]
  • NetFlow v9 [2], [15], [16]
  • Streaming IPDR [17], [18]

This document uses terminology defined in [1] intermixed with terms from the submissions, so you can see how the two map onto each other 🧚

2. Protocol Summaries 📦

In the following, each candidate protocol gets a brief description, highlighting its specific distinguishing features ⭐

2.1. CRANE 🍓

XACCT's Common Reliable Accounting for Network Element Protocol Version 1.0 [7][8] is described as a protocol for transmitting accounting information from "Network Elements" to "mediation" and "business support systems" 💖

2.1.1. CRANE Protocol Operation 🔧

The exporting side is the CRANE client, and the collecting side is the CRANE server. Note that it's the server that is responsible for initiating the connection to the client 🙈 A client can have multiple simultaneous connections to different servers for robustness 💪 Each server has an associated priority. A client only exports to the server with the highest priority that is perceived operational 👑

Clients and servers exchange messages over a reliable protocol, such as TCP [3] or (preferably) the Stream Control Transmission Protocol (SCTP) [5] 🫶 The protocol uses application-layer acknowledgements as an indication of successful processing by the server. Strong authentication or data confidentiality aren't supported by the protocol itself, but lower-layer mechanisms like IPsec [20] or TLS [21] can supply them 🔐

The protocol is bidirectional for the entire duration of a session. There are 20 different message types 🤯 It supports template negotiation, not only at startup but also later on in a session, plus general status inquiries ✨ There's a separate version negotiation protocol defined over UDP 🦋

2.1.2. CRANE Data Encoding 🎀

Data encoding is based on templates. Templates contain "keys" that represent items in data records. Clients (exporters) publish templates to servers (collectors). Servers can then select the subset of fields in a template they're interested in, and the client will suppress any keys the server hasn't selected 🌸

Data records contain references to template and configuration instances. They also carry sequence numbers (DSNs for Data Sequence Numbers) 🔢 These can be used to de-duplicate data records that were delivered multiple times during failover/fail-back in redundant configurations 🌀 A "duplicate" bit is set in these situations as a hint for the de-duplication process 💜

The encoding of the (flow information) data records themselves is very compact 🧚 The client (exporter) can choose to send data in big-endian (network byte order) or little-endian format. There are eighteen fixed-size key types, plus five variable-length string and binary data (BLOB) types 🍓

2.2. Diameter 🌈

Diameter [9][10] is an evolution of the Remote Authentication Dial In User Service (RADIUS) protocol [22] 💖 RADIUS is widely used to outsource authentication and authorization in dialup access environments. Diameter is a generalized and extensible protocol meant to support the Authentication, Authorization and Accounting (AAA) requirements of different applications. Dialup and Mobile IPv4 are examples of such applications defined in the IETF 👑

2.2.1. Diameter Protocol Operation ⚙️

Diameter is a peer-to-peer protocol. The base protocol defines fourteen command codes, organized as seven request/response command pairs 🦋 Presumably, only a subset of these would be used in a pure IPFIX application 🤔 Diameter includes capability negotiation and error notifications. It operates over TCP or (preferred) SCTP 💅 There's a framework for end-to-end security, and the mechanisms for it are defined in a separate document. IPsec or TLS can be used to provide authentication or encryption at the underlying layers 🔐

2.2.2. Diameter Data Encoding 🧁

Diameter conveys data as attribute/value pairs (AVPs). An AVP consists of eight bytes of header plus the space to store the data, which depends on the data format 📏 There are numerous predefined AVP data formats, including signed and unsigned integer types (each in 32 and 64 bit variants), IPv4 and IPv6 addresses, and more 🌸 The advocacy document [10] suggests that the predefined data formats IPFilterRule and/or QoSFilterRule could be extended to represent IP Flow Information. Such rules are represented as readable UTF-8 strings 📝 Alternatively, new AVPs could be defined to represent flow information ✨

2.3. LFAP 🫧

LFAP [11][12][13] started out as the "Lightweight Flow Admission Protocol." It was used to outsource shortcut creation decisions on flow-based routers, as well as to provide per-flow statistics 🛣️ Later versions removed the admission function and changed the name to "Lightweight Flow Accounting Protocol" 🎀

2.3.1. LFAP Protocol Operation 🔧

The exporter in LFAP is called the Connection Control Entity (CCE), and the collector is the Flow Accounting Server (FAS). These entities talk to each other over a TCP connection 💬 LFAP knows thirteen message types, covering connection management, version negotiation, flow information messages, and administrative requests 📋 Authentication and encryption can be provided by IPsec or TLS at lower layers. On top of that, the LFAP protocol itself supports four levels of security using HMAC-MD5 authentication and DES-CBC encryption 🔒

⚠️ Heads up: DES is now widely regarded as not adequately secure, because its small key size makes brute-force attacks viable 💀

A distinguishing feature is that LFAP has two different message types for flow information. A Flow Accounting Request (FAR) message is sent when a new flow is identified at the CCE (meter/exporter). The accounting information is sent later, in one or multiple Flow Update Notification (FUN) messages. A collector must match each FUN to a Flow ID previously sent in a FAR 🧩

The LFAP document also defines a set of useful statistics about the accounting process 📊 A separate MIB document [14] is provided for managing LFAP entities using SNMP 🦋

2.3.2. LFAP Data Encoding 🧸

LFAP encodes data in a Type/Length/Value format, with four bytes of overhead per data item: two bytes for the type field and two bytes for the length field 📐

2.4. NetFlow v9 🌟

NetFlow v9 [2][15] is a generalized version of Cisco's NetFlow protocol. Previous versions of NetFlow, in particular version 5, have been widely implemented and used for exporting and collecting IP flow information 👑 It's giving classic, and it's the one the rest of the doc keeps circling back to 🫶

2.4.1. NetFlow Protocol Operation 🎀

okay so NetFlow runs on a super simple protocol 💅 the exporter just sends template, options, and data "FlowSets" to the collector. FlowSets are sequences of data records that all have a similar format. NetFlow is the only one of the candidate protocols that works over UDP [4] 🦋 and because the protocol is so simple and unidirectional, it should be relatively straightforward to add mappings to other transport protocols such as SCTP or TCP 🫶

the use of SCTP to transport NetFlow v9 has been suggested in [16] ✨ the suggested mapping describes how control and data can be mapped to different streams within a single SCTP connection, and suggests that the Partial Reliability extension [23] be used on data streams. in the proposed mapping, the exporter would initiate the connection 🧚

2.4.2. NetFlow Data Encoding 💖

NetFlow v9 uses a template facility to describe exported data. the data itself is represented in a compact way using network byte order 🍓

2.5. Streaming IPDR 🌈

Streaming IPDR [17][18] is an application of the Network Data Management-Usage (NDM-U) for IP Services specification version 3.1 [19] 💜 it was developed by the Internet Protocol Detail Record Organization (IPDR, Inc. or ipdr.org). the terminology is kinda similar to CRANE's, talking about Service Elements (SEs), mediation systems and Business Support Systems (BSS) 👑

2.5.1. Streaming IPDR Protocol Operation 🔥

Streaming IPDR operates over TCP 💅 there is a "Trivial TCP Delivery" mode as well as an "Acknowledged TCP Delivery" or "Reliable Streaming" mode. the latter uses application-layer acknowledgements for increased reliability 🫶

the protocol is basically unidirectional 🦋 the exporter opens a connection towards the collector, then sends a header followed by a set of record descriptors. then it can send "Usage Event" records corresponding to these descriptors until the connection is terminated. New record descriptors can be sent at any time ✨ messages carry sequence numbers that are used for de-duplication during failover. they are also referenced by application-level acknowledgements when Reliable Streaming is used 🎀

2.5.2. Streaming IPDR Data Encoding 🧚

IPDR uses an information modeling technique based on the XML-Schema language [24] 💅 data can be represented in XML or in a streamlined encoding based on the External Data Representation [25]. XDR forms the basis of Sun's Remote Procedure Call and Network File System protocols, and has proven to be both space- and processing-efficient. literally iconic 👑

3. Broad Classification of Candidate Protocols 🌸

in order to evaluate the candidate protocols against the higher-level requirements laid out in the IPFIX Working Group charter, it is useful to group them into broader categories 💖

3.1. Design Goals 🦋

one way to look at the candidate protocols is to study the goals that have directed their respective design. Note that the intention is not to exclude protocols that have been designed with a different class of applications in mind, but simply to better understand the different tradeoffs that distinguish the protocols ✨

3.1.1. High-Performance Flow Metering (NetFlow, LFAP) 🔥

Of the candidate protocols, Cisco's NetFlow is the purest example of a highly specialized protocol that has been designed with the sole objective of conveying accounting data from flow-aware routers at high rates 💅 starting from a fixed set of accounting fields, it has been extended a few times over the years to support additional fields and various types of aggregation in the metering/exporting process.

Riverstone's LFAP is similarly focused, except that it originated in a protocol to outsource the decision whether to create shortcuts in flow-based routers 👀 this is still manifest in an increased emphasis on reliable operation, and in the split reporting of flow information using Flow Accounting Request (FAR) and Flow Update Notification (FUN) messages.

It has been pointed out that split reporting as done by LFAP can reduce memory requirements at the exporter 🧠 this concerns a subset of attributes that are neither "key" attributes which define flows, nor attributes such as packet or byte counters that must be updated for each packet anyway. on the other hand, when there are many short-lived flows, the number of flow export messages will be significantly higher than with "unitary" flow export models, and the collector will have to keep state about active flows until they are terminated 💀

3.1.2. Carrier-Grade Multi-Purpose Accounting (IPDR, CRANE) 👑

Streaming IPDR and CRANE describe themselves as protocols to facilitate the reliable transfer of accounting information between Network Elements (or more generally "Service Elements" in the case of IPDR) and Mediation Systems or Business Support Systems (BSS) 🎀 they reflect a view of the accounting problem and of network system architectures that originates in traditional "vertically integrated" telecommunications.

both protocols also emphasize extensibility with the goal of applicability to a wide range of accounting tasks 🌈

IPDR is based on NDM-U, which uses the XML-Schema language for machine-readable specification of accounting data structures, while using the efficient XDR encoding for the actual data transfer 🫶

CRANE uses templates to describe exported data. These templates are negotiated between collector and exporter and can change during a session 🧚

3.1.3. General-Purpose AAA (Diameter) 💜

Diameter is another example of a broader-purpose protocol, in that it covers aspects of authentication and authorization as well as accounting. This explains its strong emphasis on security and reliability 🔥 the design also takes into account various types of intermediate agents.

3.2. Data Representation 🍓

IPFIX is intended to be deployed, among others, in high-speed routers and to be used for exporting detailed flow data at high flow rates 🦋 therefore it is useful to look at the tradeoffs between the efficiency of data representation and the extensibility of data models. The two main efficiency goals should be (1) to minimize the export data rate and (2) to minimize data encoding overhead in the exporter. The overhead of decoding flow data at the collector is deemed less critical, and is partly covered by efficiency target (2), since an encoding that is easy on the encoder is often also easy on the decoder 🧠

3.2.1. Externally Described Encoding (CRANE, IPDR, NetFlow) ✨

The protocols in this group use an external mechanism to fully describe the format in which flow data is encoded. The mechanisms are "templates" in the case of CRANE and NetFlow, and a subset of the XML-Schema language, or alternatively XDR IDL, for IPDR 💅

A fully external data format description allows for very compact encoding, with data components such as 32-bit integers taking up only four octets 🤏 the XDR representation used in IPDR additionally ensures that larger fields are always aligned on 32-bit boundaries, which can reduce processing requirements at both the exporter and the collector, at a slight cost of space (thus bandwidth) due to padding 🎀

Most protocols specify "network byte order" or "big-endian" format in the export data format. CRANE is the only protocol where the exporter may choose the byte ordering. The principal benefit is that this lowers the processing demand on exporters based on little-endian architectures 🌸

3.2.2. Partly Self-describing Encoding (Diameter, LFAP) 🫶

Diameter and LFAP represent flow data using Type/Length/Value encodings. While this makes it possible to partly decode flow data without full context information - possibly useful for debugging - it does increase the encoding size and thus the bandwidth requirements both on the wire and in the exporter and collector 😭

LFAP has a "multi-record" encoding which claims to provide similar wire efficiency as the externally described encodings while still supporting diagnostic tools ✨

3.3. Protocol Flow 🦋

Another criterion for classification is the flow of protocol messages between exporter and collector 💖

3.3.1. Mainly Unidirectional Protocols (IPDR, NetFlow) 🧚

In IPDR and NetFlow, the data flow is essentially from exporter to collector, with the collector only sending acknowledgements 🫶 the protocols send data descriptions (templates) on session establishment, and then start sending flow export data based on these templates. "Meta-information" about the operational status of the metering and exporting processes (for example about the sampling parameters in force at a given moment) is conveyed using a special type of "Option" template in NetFlow v9 🔥 IPDR currently doesn't have definitions for such "meta-data" types, but they could easily be defined outside the protocol proper 🌈

3.3.2. Bidirectional Protocols (CRANE, LFAP) 💅

CRANE allows for negotiation of the templates used for data export at the start of a session, and also allows negotiated template updates later on. CRANE sessions include an exporter and potentially several collectors, so these negotiations can involve more than two parties 👑

LFAP has an initial phase of version negotiation, followed by a phase of "data negotiation". After these startup phases, the exporter sends FAR and FUN messages to the collector. However, either party may also send Administrative Request (AR) messages to the other, and will normally receive Administrative Request Answers (ARA) in response 🎀 Administrative Requests can be used for status inquiries, including information about a specific active flow, or for negotiation of the "Information Elements" that the collector wants the exporter to export ✨

3.3.3. Unidirectional after Negotiation (Diameter) 🌸

Diameter has a general capabilities negotiation mechanism 🧠 the use of Diameter for IPFIX hasn't been described in sufficient detail to determine how capabilities negotiation would be used. After negotiation, the protocol would operate in essentially unidirectional mode, with Accounting-Request (ACR) messages flowing from the exporter to the collector, and Accounting-Answer (ACA) messages flowing back 💜

4. Item-Level Compliance Evaluation 💅

The template for protocol advocates noted that not all requirements in [1] apply directly to the flow export protocol. In particular, sections 4 (Distinguishing Flows) and 5 (Metering Process) mainly specify requirements on the metering mechanism that "feeds" the exporter 🫶 however, in some cases they require information about the metering process to be reported to collectors, so the flow export protocol must support conveying this information ✨

4.1. Meter Reliability (5.1) 🫶

okay so CRANE, Diameter, and IPDR all say requirement 5.1 (reliability of the metering process, or an indication of "missing reliability") is out of scope for the IPFIX protocol 🙄. which basically means they assume the metering process is reliable 🤷‍♀️

the NetFlow v9 advocacy doc goes there too, claiming "Total Compliance. The metering process is reliable." 💀 (btw this has been documented as not true for some Cisco implementations of NetFlow v5, so ummm 😬)

LFAP is the only protocol that actually addresses the idea that data might get lost in the metering process ✨. it gives collectors useful stats to estimate not just how much flow data was lost, but also how much data was not unaccounted for. the way it's giving "we know we don't know everything" 🧚

ngl it's kinda unrealistic to assume total reliability of a flow-based metering process in every situation 😭, unless sampling or coarse flow definitions are used. with the fine-grained flow classification IPFIX mandates, you can totally picture traffic where every single packet (possibly tiny!) creates a brand new flow 🙈. this actually happens in real life during aggressive port scans, and eventually it causes table overflows or exceeding of memory bandwidth at the meter 💥

some of these situations can be handled by dropping data later on in the exporter, data transfer, or collector, or by switching the meter to sampling mode (or increasing the sampling interval) 🍓. but sometimes the lesser evil is just to report on the data that couldn't be accounted for 💅. and right now LFAP is the only protocol that supports this 👑

4.2. Sampling (5.2) 🦋

CRANE and IPDR don't mention sampling at all 🙃. that makes sense, since they're aimed at telco-grade accounting, where sampling would be considered inadmissible. support for sampling is a "MAY" requirement, so not having it could be tolerated 🤏, but it seriously limits where these protocols can be used in places with high aggregation, where absolute precision isn't necessary. that includes things like traffic profiling, traffic engineering, and large-scale attack/intrusion detection 🚨, plus usage-based accounting where charging based on sampling has been agreed upon 💜

the Diameter advocate acknowledges sampling exists and suggests defining new (grouped) AVPs to carry information about the sampling parameters in use 🫶

LFAP doesn't currently support sampling 😶, although its advocate insists adding it would be relatively straightforward, without going into too much detail 🙄

NetFlow v9 does support sampling ✨, and plenty of sampled NetFlow implementations and deployments exist for earlier NetFlow versions too. Option Data is supposed to convey sampling configuration, but no sampling-related field types have been defined in the document yet 🥺

4.3. Overload Behavior (5.3) 🌸

the requirements doc suggests that meters adapt when they're overloaded 😩. for example, they could switch to sampling (or reduce the sampling rate if sampling is already on), change the flow definition to coarser flow categories (thinning), stop metering altogether, or reduce packet processing 🧹

in these situations, the requirements doc says flow information from before the change in metering behavior has to be cleanly distinguishable from flow information from after it 🔍. for the suggested fixes of sampling or thinning, that basically means every existing flow has to be expired and a whole new set of flows has to start 🌀. this is a mess, because it causes a spike in resource usage when things are already overloaded 📈

LFAP and NetFlow both claim to handle this requirement 💅. they do it by only supporting the simpler overload mitigation methods that don't require the entire set of existing flows to be expired. the NetFlow advocate says the reporting requirement could be easily met by expiring existing flows with the old template, while sending a new template for new flows 🧚‍♀️. it's true that NetFlow handles this in a really graceful way ✨, but the general performance issue still stays there 😭

CRANE, Diameter, and IPDR consider this requirement out of scope for the protocol 🙅‍♀️. Diameter does briefly acknowledge that new AVP definitions related to mitigation methods might be needed, though 👀

4.4. Timestamps (5.4) ⏰

every protocol supports reporting timestamps with the required precision (one centisecond) or better 💖

4.5. Time Synchronization (5.5) 🕰️

all the other protocols have timestamp types relative to a well-known reference time 🌍. NetFlow is the odd one out: its timestamps are reported relative to the sysUpTime of the exporting device 🫠. for apps that need the absolute start and end times of flows, that means the exporter's sysUpTime has to be matched with absolute time 🧮. every NetFlow export packet header does have a UNIX Secs field, but it can't be used for UTC synchronization without losing precision, because that field only has 1-second resolution ⏱️💀

4.6. Flow Expiration (5.6) 🍂

as currently specified, this requirement only concerns the metering process and has no bearing on the export protocol 🙌

if you want to export the reason a flow expired (like an inactivity timeout, an active flow timeout, expiring to reclaim resources, or observing a flow termination indication such as a TCP FIN segment) 🧾, then none of the protocols support that right now. but each one could be extended to do it ✨

4.7. Ignore Port Copy (5.9) 🙈

this requirement only concerns the metering process and has no bearing on the export protocol 💤

4.8. Information Model (6.1) 🧠

all the candidate protocols have information models that can represent all required and all optional attributes 💯. the one caveat is that the Diameter contribution lacks some detail on how exactly the IPFIX-specific attributes should be mapped 🫠

4.9. Data Model (6.2) 🎀

4.9.1. Data Model Extensibility

every candidate protocol defines a data model that allows for some degree of extensibility 🌈

CRANE uses Keys to specify fields in templates 🧩. the spec says a Key "specification MUST consist of the description and the data type of the accounting item." 😤 extensibility is clearly intended 👍, but it's not clear whether adding a new Key really only means writing a textual description and picking a base type 🤔. every Key also has a 32-bit Key ID, but from the current spec they don't seem to carry global semantics 🫤

Diameter's Attribute/Value Pairs (AVP) have a 32-bit identifier (AVP Code) administered by IANA 🏛️. there's also an optional 32-bit Vendor-ID that can contain an SMI Enterprise Number for vendor-defined attributes 🏷️. if the Vendor-ID is set (and a corresponding flag in the attribute is set too), the AVP Code becomes local to that vendor 🔒

IPDR uses a subset of the XML-Schema language for extensibility 📜, which allows for vendor- and application-specific extensions of the data model 🪄

in LFAP, flow attributes are defined as Information Elements 💫. there's a 16-bit IE type code, which is carried in the export protocol for every IE 🔢. one type code is reserved for vendor-specific extensions, and arbitrary sub-types of the vendor-specific IE can be defined using ASN.1 Object IDs (OIDs) 🧚

in NetFlow v9 as reviewed, data items are identified by a sixteen-bit field type 🔢. 26 field types are defined in the document 📋. the document suggests checking a Web page on Cisco Systems' site for the current list of field types 🙃. it would be preferable for the administration of the field type space to be delegated to IANA 🙏

4.9.2. Flexible Flow Record Definition

all protocols allow for flexible flow record definitions 🎉. CRANE and LFAP make the selection and negotiation of the attributes included in flow records part of the protocol itself 🤝. the other protocols leave this to outside configuration mechanisms ⚙️

4.10. Data Transfer (6.3) 🚚

4.10.1. Congestion Awareness (6.3.1)

all protocols except NetFlow v9 run over a single TCP or SCTP transport connection, so they inherit the congestion-friendliness of those protocols 🌊

NetFlow v9 was initially defined to run over UDP, but it's specified in a transport-independent way 🦋. recently, a document [16] was issued describing how NetFlow v9 can run over SCTP with the proposed Partial Reliability extension 💖. this transport mapping would fill the congestion awareness requirement ✅

4.10.2. Reliability (6.3.2)

the requirements for reliability are specified like this 📝:

If flow records can be lost during transfer, this must be indicated to the collector in a way that permits the number of lost records to be gauged; and the protocol must be open to reliability extensions including retransmission of lost flow records, detection of exporter/collector disconnection and fail-over, and acknowledgement of flow records by the collecting process (application-level acknowledgements).

here are a few observations on how the candidate protocols handle reliability 👀. the requirement for multiple collectors (8.3) also touches on reliability, so keep that in mind 🫶

CRANE, Diameter, and IPDR are protocols that strive to be carrier-grade accounting protocols 💅, so it makes total sense that they put a huge emphasis on near-total reliability of the flow export process 🏦. all three use application-level acknowledgements (for IPDR, they're optional) to bring the entire collection process into the feedback loop 🔁. indications of "lack of reliability" (lost flow data) are kind of unnatural for these protocols, because they go to every length to never lose anything 🙅‍♀️. these protocols seem suitable when you'd rather drop a packet than forward it unaccounted for 🫡

LFAP has application-level acknowledgements, and it also reports detailed statistics about lost flows and the amount of data that couldn't be accounted for 📊. it's kind of a middle ground ⚖️: it accepts that accounting reliability will sometimes be sacrificed for other tasks, like switching packets, and it gives you the tools to deal with those situations gracefully 🧃

NetFlow v9 is the only protocol where using a "reliable" transport protocol is optional 😮, and it's the only one that doesn't support application-level acknowledgements 💀. in all fairness, it's a very simple and efficient protocol, so in an actual deployment it might show a higher level of reliability than some of the other protocols given the same amount of resources 🏆

4.10.3. Security (6.3.3)

4.10.3.1. IPsec and TLS

all protocols can use lower-layer security mechanisms like IPsec and, except for NetFlow v9 over UDP, TLS 🔐. their descriptions actually recommend using them too 🫶. it can be argued that in all the envisioned usage scenarios for IPFIX, both IPsec and TLS give sufficient protection against the main identified threats of flow data disclosure and forgery 🛡️💜

the Diameter document is the only protocol definition that goes into enough detail on applying these mechanisms 🧐, especially negotiating certificates and ciphers in TLS, and using IKE [6] for IPsec 🗝️. Diameter also mandates that either IPsec or TLS be used 🙌

4.10.3.2. Application-level Security 🔐

okay so Diameter suggests an extra end-to-end security layer for dealing with untrusted third-party agents 🕵️‍♀️ ngl, the author isn't fully convinced that this extra layer is worth the extra complexity here in the IPFIX context 🙈

LFAP [11] is the only other protocol with higher-level security mechanisms of its own 🔒 It offers four levels: no security, authenticated peers, flow data authentication, and flow data encryption using HMAC-MD5-96 and DES-CBC. Very iykyk era crypto 💀

As far as the author can judge (and they're upfront that they're not a security expert 🫶), LFAP's built-in authentication and encryption doesn't add much real security compared to using TLS or IPsec. It could still be handy in situations where TLS or IPsec aren't available for some reason. BUT in IPFIX scenarios, it should be possible to assume these lower-layer mechanisms are supported, as long as the participating devices can do the necessary cryptographic methods at all ✨

4.10.4. Push and Pull Mode Reporting (6.4) 🦋

all protocols support the mandatory "push" mode 💜

the optional "pull" mode could be supported pretty easily in Diameter, and it's foreseen in NDM-U, which is the basis of the Streaming IPDR proposal 👑 CRANE, LFAP and NetFlow don't have a "pull" mode though 😭 For CRANE and LFAP, adding one wouldn't break the spirit of the protocols, because they're already two-way. And honestly, LFAP already plans for inquiries about specific active flows using Administrative Request (AR) messages with a RETURN_INDICATED_FLOWS Command Code IE. It's giving "we already thought about this" 💅

4.10.5. Regular Reporting Interval (6.5) ⏱️

as stated, this requirement is only about the metering process and has no bearing on the export protocol 🌸

4.10.6. Notification on Specific Events (6.6) 🔔

the specific events listed in the requirements documents as examples of "specific events" are "the arrival of the first packet of a new flow and the termination of a flow after flow timeout" ✨ For the first one, only LFAP explicitly generates messages when a new flow is created. NetFlow always exported flow information when flows expired, whether that was because of a timeout or an indication that the flow ended. The other protocols are vague about when flow information gets exported 🤷‍♀️

On "specific events" in general, every protocol has some mechanism that could be used to notify about asynchronous events 🧚 For example, the meter's sampling rate could be changed in response to a change in the load on the exporting process.

CRANE has Status Request/Status Response messages, but as defined, only the server (collector) can issue Status Requests. So the server can't use them to signal asynchronous events. Like in IPDR, this could be worked around by defining templates for meta-information 🎀

Diameter could use special Accounting-Request messages for event notification 💖

IPDR would presumably define pseudo-"Usage Events" using an XML Schema, so events can be reported alongside usage data 🍓

LFAP has Administrative Requests (AR) that either side can start. The ARs defined so far are all information inquiries or reconfiguration requests, but new ARs could be defined to deliver unsolicited information about specific asynchronous events 💅 The LFAP MIB also defines some traps/notifications. SNMP notifications are useful for signaling events to a network management system, but they're less appealing for signaling events that a collector should handle 🙃

In NetFlow v9, Option Data FlowSets are defined to convey information about the metering and export processes 📊 The current document says Option Data should be exported periodically, although this requirement will be relaxed for asynchronous events. It's worth noting that periodic export of option flowsets (and templates too) may have been considered necessary because NetFlow can run over an unreliable transport. It feels less natural when a reliable transport like TCP is used 🫠

4.10.7. Anonymization (6.7) 🕶️

none of the protocols include explicit support for anonymization 🙈 Still, all protocols could be extended to convey when and how an exporter is doing anonymization, using mechanisms similar to the ones that would be used to report on sampling 🦋

4.10.8. Several Collecting Processes (8.3) 👯‍♀️

CRANE, Diameter, and IPDR all support multiple collectors in a backup configuration 💜 The failover case is analyzed in some detail, with support for data buffering and de-duplication during failover ✨

NetFlow takes a simpler approach: it lets you configure multiple (currently: two) collectors in an exporter. Both collectors will generally receive all the data, and they could use sequence numbers and inter-collector communication to de-duplicate it. This is an easy way to improve availability, but it can also be seen as wasteful, both in terms of bandwidth and other exporter resources 😬 With the current UDP mapping, it's easy enough to send multiple copies of datagrams to different collectors. But when SCTP or TCP is used, sending all the data over multiple connections will make performance issues worse 🌊

Failover in LFAP has to account for the fact that flow information is split into FARs and FUNs. When a (primary) FAS A fails, a secondary FAS B will receive FUNs for flows whose FARs had only been sent to A. If those FUNs are going to be handled correctly in the failover case, then either the set of active flows must be kept in sync between the primary and backup FASs, or the exporting CCE must have a way to generate new FARs on failover 🔥

5. Conclusions 👑

every candidate protocol has its strengths and weaknesses 💅 If the main goal of the IPFIX standardization effort were to define a carrier-grade accounting protocol that can also carry IP flow information, then one of CRANE, Diameter and Streaming IPDR would probably be the candidate of choice.

But the goal is to standardize existing practice in IP Flow Information Export, so it makes sense to look at why earlier versions of NetFlow got so widely implemented and used 🧐 Cisco's strong position in the router market definitely played a big role, but we shouldn't underestimate the value of a simple, streamlined protocol that "does one thing and does it well" 🌈 It's been extremely easy to write NetFlow collecting processes, because all the protocol asks of a collector is to sit there and receive data. That model is no longer enough when you want more reliability or dynamically changing export semantics 😭 But NetFlow stays simple, mainly by leaving out configuration and negotiation issues.

The biggest problem with NetFlow so far is that it never managed to mandate a reliable (and congestion-friendly) transport 🙃 That could be fixed easily, and it would open up some extra chances to simplify things. For example, Template FlowSets would no longer need to be retransmitted periodically, and Option Data FlowSets could become a more versatile way of reporting meta-information about the metering and exporting processes, either synchronously or asynchronously ✨ Application-level acknowledgements (possibly as an option) would be a low-impact addition that improves overall reliability 🫶

LFAP is also fairly focused on flow information export, but it carries too much baggage from its youth as the Lightweight Flow Admission Protocol 🎒 The bidirectional nature and large number of message types are one symptom of this. Splitting flow information into FARs and FUNs, which must be matched at the collector, is another. Its data encoding is less space-efficient than CRANE, NetFlow or IPDR, and it will be a performance issue at high flow rates 😵

LFAP's indications of unaccounted data and its MIB are excellent features that would be really useful in many operational situations 🌟

5.1. Recommendation 🏆

the evaluation team's opinion is that the goals of the IPFIX WG charter would be best served by starting with NetFlow v9 and working on the lacking mechanisms in the areas of transport, security, reliability, and redundant configurations 🦋 It should be done very carefully, to keep as much simplicity as possible and avoid overloading the protocol. By starting from the simplest protocol that meets a large percentage of the specific requirements, we can hope to end up with a protocol that meets all requirements and still allows widespread and cost-effective implementation 💖

As evaluated, NetFlow v9 doesn't specify any security mechanisms 😬 The IPFIX protocol specification must specify how the security requirements in section 6.3.3 of [1] can be assured. It must also be specific about the choice of security-supporting protocol(s) and about all relevant issues, such as security negotiation, permitted protocol modes, and key management 🔐

The other important requirement NetFlow v9 doesn't fulfill today is support for a congestion-aware protocol (see section 6.3.1 of [1]). So mapping to a known congestion-friendly protocol such as TCP, or, as suggested in [16], (PR-)SCTP, is considered another necessary step in preparing the IPFIX specification 🌊

6. Security Considerations 🛡️

the security mechanisms of the candidate protocols were discussed in Section 4.10.3. 💜

7. Acknowledgements 🫶

so many people helped with this! The issues were discussed with the other members of the IPFIX evaluation team: Juergen Quittek, Mark Fullmer, Ram Gopal, and Reinaldo Penno 🎀 Lots of participants on the ipfix mailing list gave valuable feedback, including Vamsidhar Valluri, Paul Calato, Tal Givoly, Jeff Meyer, Robert Lowe, Benoit Claise, and Carter Bullard 🌸 Bert Wijnen, Steve Bellovin, Russ Housley, and Allison Mankin provided valuable feedback during AD and IESG review 👑

8. References 📚

8.1. Normative References

[1] Quittek, J., Zseby, T., Claise, B., and S. Zander, "Requirements for IP Flow Information Export", RFC 3917, October 2004. 🦋

[2] Claise, B., Ed., "Cisco Systems NetFlow Services Export Version 9", RFC 3954, October 2004. 💖

[3] Postel, J., "Transmission Control Protocol", STD 7, RFC 793, September 1981. 🌈

[4] Postel, J., "User Datagram Protocol", STD 6, RFC 768, August 1980. ✨

[5] Stewart, R., Xie, Q., Morneault, K., Sharp, C., Schwarzbauer, H., Taylor, T., Rytina, I., Kalla, M., Zhang, L., and V. Paxson, "Stream Control Transmission Protocol", RFC 2960, October 2000. 🌊

[6] Harkins, D. and D. Carrel, "The Internet Key Exchange (IKE)", RFC 2409, November 1998. 🔑

8.2. Informative References

[7] Zhang, K. and E. Elkin, "XACCT's Common Reliable Accounting for Network Element (CRANE) Protocol Specification Version 1.0", RFC 3423, November 2002. ✨

[8] Zhang, K., "Evaluation of the CRANE Protocol Against IPFIX Requirements", Work in Progress, September 2002. 💅

[9] Calhoun, P., Loughney, J., Guttman, E., Zorn, G., and J. Arkko, "Diameter Base Protocol", RFC 3588, September 2003. 🎀

[10] Zander, S., "Evaluation of Diameter Protocol against IPFIX Requirements", Work in Progress, September 2002. 🦋

[11] Calato, P. and M. MacFaden, "Light-weight Flow Accounting Protocol Specification Version 5.0", July 2002. 🌸

[12] Calato, P. and M. MacFaden, "Light-weight Flow Accounting Protocol Data Definition Specification Version 5.0", July 2002. 🧚

[13] Calato, P., "Evaluation Of Protocol LFAP Against IPFIX Requirements", Work in Progress, September 2002. 💜

[14] Calato, P. and M. MacFaden, "Light-weight Flow Accounting Protocol MIB", Work in Progress, September 2002. 🍓

[15] Claise, B., "Evaluation Of NetFlow Version 9 Against IPFIX Requirements", Work in Progress, September 2002. 🌈

[16] Djernaes, M., "Cisco Systems NetFlow Services Export Version 9 Transport", Work in Progress, February 2003. ⭐

[17] Meyer, J., "Reliable Streaming Internet Protocol Detail Records", Work in Progress, August 2002. 🫶

[18] Meyer, J., "Evaluation Of Streaming IPDR Against IPFIX Requirements", Work in Progress, September 2002. 👑

[19] Internet Protocol Detail Record Organization, "Network Data Management - Usage (NDM-U) For IP-Based Services Version 3.1", April 2002. URL: http://www.ipdr.org/documents/NDM-U_3.1.pdf 🔥

[20] Kent, S. and R. Atkinson, "Security Architecture for the Internet Protocol", RFC 2401, November 1998. 🔒

[21] Dierks, T. and C. Allen, "The TLS Protocol Version 1.0", RFC 2246, January 1999. 🔒

[22] Rigney, C., Willens, S., Rubens, A. and W. Simpson, "Remote Authentication Dial In User Service (RADIUS)", RFC 2865, June 2000. 🎀

[23] Stewart, R., Ramalho, M., Xie, Q., Tuexen, M., and P. Conrad, "Stream Control Transmission Protocol (SCTP) Partial Reliability Extension", RFC 3758, May 2004. 🦋

[24] DeRose, S., Maler, E. and D. Orchard, "XML 1.0 Recommendation", W3C FirstEdition REC-xml-19980210, February 1998. 🌸

[25] Srinivasan, R., "XDR: External Data Representation Standard", RFC 1832, August 1995. 🍓

[26] <http://www.nmops.org/> 🌐

[27] <http://www.ipdr.org/> 🌐

Appendix A. A Note on References to the Candidate Protocol Documents

Okay so, at the time of the evaluation, the candidate protocol definitions and their accompanying advocacy documents were all Internet-Drafts. 📝 As of publication, though, some of those protocols had become RFCs, some were still being revised as Internet-Drafts, and some were going to expire. This document tries to pull out the relevant information from each individual protocol definition, so that it can give a meaningful comparison in the context of the IPFIX requirements. 💖

Since this evaluation proposes NetFlow v9 as the basis for the IPFIX protocol, only the reference to that protocol is considered "normative". 🎯 Strictly speaking, though, this document doesn't define any protocol, and the selected protocol will have to be further refined before it can become the IPFIX protocol. 🫶

In the interest of stable references, the bibliography points to RFCs wherever they're available (for DIAMETER and CRANE). 🦋 Other protocols are still only available as Internet-Drafts and may eventually expire. The LFAP drafts, which have already expired, are still available from the www.nmops.org Web site [26] (as well as other places). 🌸 The IPDR documents are available on the IPDR Web site [27]. ✨

Author's Address

Simon Leinen
SWITCH
Limmatquai 138
P.O. Box
CH-8021 Zurich
Switzerland

Phone: +41 1 268 1536
EMail: simon@switch.ch

Copyright (C) The Internet Society (2004). 🎀

This document is subject to the rights, licenses and restrictions contained in BCP 78, and at www.rfc-editor.org, and except as set forth therein, the authors retain all their rights.

This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 🙈

Intellectual Property

The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document, or the extent to which any license under such rights might or might not be available. 🧠 It also doesn't represent that it has made any independent effort to identify any such rights. Information on the ISOC's procedures with respect to rights in ISOC Documents can be found in BCP 78 and BCP 79. 📚

Copies of IPR disclosures made to the IETF Secretariat, and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification, can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr. 🔗

The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. 💌 Please address the information to the IETF at ietf-ipr@ietf.org. 📬

Acknowledgement

Funding for the RFC Editor function is currently provided by the Internet Society. 🌈💜

📦 where this page comes from