RFC 7011
Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of Flow Information
Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of Flow Information 🎀
okay so this is the big one, the IPFIX spec, and it's officially a standard 👑 Here's the tea on the title block:
- Request for Comments: 7011 💖
- STD: 77, so it's an Internet Standard too 💅
- Authors: B. Claise, Ed. (Cisco Systems, Inc.), B. Trammell, Ed. (ETH Zurich), and P. Aitken (Cisco Systems, Inc.) 🧚
- Category: Standards Track
- Date: September 2013
- ISSN: 2070-1721
- Obsoletes: RFC 5101 💀 that one's retired now
- Updates: nothing is listed
Abstract 🦋
ngl this abstract is the perfect little summary of the whole vibe ✨ IPFIX is the protocol for sending traffic flow info over the network. If an Exporting Process wants to get flow data to a Collecting Process, you need one shared way to represent the data and one standard way to send it, and that's exactly what this doc sets up 🫶 It explains how IPFIX Data Records and Template Records get carried over a number of transport protocols from an IPFIX Exporting Process to an IPFIX Collecting Process. And yes, it obsoletes RFC 5101, so that's the update 🔥
Status of This Memo 📜
So this is an Internet Standards Track doc, which is a big deal 👑 It's a product of the Internet Engineering Task Force (IETF), it represents the consensus of the IETF community, it got public review, and the Internet Engineering Steering Group (IESG) approved it for publication. If you want more on Internet Standards, check out Section 2 of RFC 5741 💜
Info about the current status, any errata, and how to send feedback lives at http://www.rfc-editor.org/info/rfc7011 🌸
Copyright Notice 🌈
Copyright (c) 2013 IETF Trust and the persons identified as the document authors. All rights reserved. 🎀
This doc is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) as they stood on the publication date. Please read these carefully, because they spell out your rights and restrictions for this document 🙈 Code Components pulled out of this document must include Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come without warranty, as described in the Simplified BSD License.
Table of Contents 🧚
1. Introduction ....................................................5
1.1. Changes since RFC 5101 .....................................5
1.2. IPFIX Documents Overview ...................................6
2. Terminology .....................................................7
2.1. Terminology Summary Table .................................13
3. IPFIX Message Format ...........................................13
3.1. Message Header Format .....................................15
3.2. Field Specifier Format ....................................16
3.3. Set and Set Header Format .................................18
3.3.1. Set Format .........................................18
3.3.2. Set Header Format ..................................19
3.4. Record Format .............................................20
3.4.1. Template Record Format .............................20
3.4.2. Options Template Record Format .....................23
3.4.2.1. Scope .....................................23
3.4.2.2. Options Template Record Format ............24
3.4.3. Data Record Format .................................27
4. Specific Reporting Requirements ................................28
4.1. The Metering Process Statistics Options Template ..........29
4.2. The Metering Process Reliability Statistics
Options Template ..........................................29
4.3. The Exporting Process Reliability Statistics
Options Template ..........................................31
4.4. The Flow Keys Options Template ............................32
5. Timing Considerations ..........................................32
5.1. IPFIX Message Header Export Time and Flow Record Time .....32
5.2. Supporting Timestamp Wraparound ...........................33
6. Linkage with the Information Model .............................34
6.1. Encoding of IPFIX Data Types ..............................34
6.1.1. Integral Data Types ................................34
6.1.2. Address Types ......................................34
6.1.3. float32 ............................................34
6.1.4. float64 ............................................34
6.1.5. boolean ............................................35
6.1.6. string and octetArray ..............................35
6.1.7. dateTimeSeconds ....................................35
6.1.8. dateTimeMilliseconds ...............................35
6.1.9. dateTimeMicroseconds ...............................35
6.1.10. dateTimeNanoseconds ...............................36
6.2. Reduced-Size Encoding .....................................36
7. Variable-Length Information Element ............................37
8. Template Management ............................................38
8.1. Template Withdrawal and Redefinition ......................40
8.2. Sequencing Template Management Actions ....................42
8.3. Additional Considerations for Template Management
over SCTP .................................................43
8.4. Additional Considerations for Template Management
over UDP ..................................................44
9. The Collecting Process's Side ..................................45
9.1. Collecting Process Handling of Malformed IPFIX Messages ...46
9.2. Additional Considerations for SCTP Collecting Processes ...46
9.3. Additional Considerations for UDP Collecting Processes ....46
10. Transport Protocol ............................................47
10.1. Transport Compliance and Transport Usage .................47
10.2. SCTP .....................................................48
10.2.1. Congestion Avoidance ..............................48
10.2.2. Reliability .......................................49
10.2.3. MTU ...............................................49
10.2.4. Association Establishment and Shutdown ............49
10.2.5. Failover ..........................................50
10.2.6. Streams ...........................................50
10.3. UDP ......................................................50
10.3.1. Congestion Avoidance ..............................50
10.3.2. Reliability .......................................51
10.3.3. MTU ...............................................51
10.3.4. Session Establishment and Shutdown ................51
10.3.5. Failover and Session Duplication ..................51
10.4. TCP ......................................................52
10.4.1. Congestion Avoidance ..............................52
10.4.2. Reliability .......................................52
10.4.3. MTU ...............................................52
10.4.4. Connection Establishment and Shutdown .............53
10.4.5. Failover ..........................................53
11. Security Considerations .......................................54
11.1. Applicability of TLS and DTLS ............................55
11.2. Usage ....................................................56
11.3. Mutual Authentication ....................................56
11.4. Protection against DoS Attacks ...........................57
11.5. When DTLS or TLS Is Not an Option ........................58
11.6. Logging an IPFIX Attack ..................................58
11.7. Securing the Collector ...................................59
11.8. Privacy Considerations for Collected Data ................59
12. Management Considerations .....................................60
13. IANA Considerations ...........................................61
Appendix A. IPFIX Encoding Examples ...............................62
A.1. Message Header Example ....................................62
A.2. Template Set Examples .....................................63
A.2.1. Template Set Using IANA Information Elements ..........63
A.2.2. Template Set Using Enterprise-Specific Information
Elements ..............................................64
A.3. Data Set Example ..........................................65
A.4. Options Template Set Examples .............................66
A.4.1. Options Template Set Using IANA Information Elements ..66
A.4.2. Options Template Set Using Enterprise-Specific
Information Elements ..................................66
A.4.3. Options Template Set Using an Enterprise-Specific
Scope .................................................67
A.4.4. Data Set Using an Enterprise-Specific Scope ...........68
A.5. Variable-Length Information Element Examples ..............69
A.5.1. Example of Variable-Length Information Element with
Length Less Than 255 Octets ...........................69
A.5.2. Example of Variable-Length Information Element with
3-Octet Length Encoding ...............................70
Normative References ..............................................71
Informative References ............................................71
Acknowledgments ...................................................74
Contributors ......................................................751. Introduction ✨
okay so picture traffic on a data network as a bunch of flows zipping through network elements 🚀 For admins (and plenty of other people with their own reasons), it's often super interesting, useful, or even necessary to get info about those flows. A Collecting Process should be able to take in the Flow information passing through multiple network elements across the data network 🧚 That only works if everyone agrees on how flow info is represented and how it gets from the network elements to the collection point, and that's the whole job of this doc. It spells out how the different flows are represented, plus the extra data you need to interpret a flow, the packet format, the transport mechanisms used, security concerns, and more 💖
(more parts coming, bestie 🫶 this is part 1 of 16!)
1.1. Changes since RFC 5101 🎀
okay so this doc is the glow-up of [RFC5101] 💅 It obsoletes the Proposed Standard revision of the IPFIX Protocol Specification [RFC5101], but the protocol here is still interoperable with the one in [RFC5101] 🫶 Here's what changed, bestie:
- All outstanding technical and editorial errata on [RFC5101] have been addressed. 🧹
- As the [IANA-IPFIX] registry is now the normative reference for all Information Element definitions (see [RFC7012]), all definitions of Information Elements in Section 4 have been replaced with references to that registry. 📖
- The encoding of the
dateTimeSeconds,dateTimeMilliseconds,dateTimeMicroseconds, anddateTimeNanosecondsdata types, and the related encoding of the IPFIX Message HeaderExport Timefield, have been clarified, especially with respect to the epoch upon which the timestamp data types are based. ⏰ - A new Section 5.2 has been added to address wraparound of these timestamp data types after they overflow in the years 2032-2038. 🌈
- Clarifications on encoding, especially in Section 6, have been made: all IPFIX values are encoded in network byte order. 🔢
- Template management, as described in Section 8, has been simplified and clarified: the specification has been relaxed with respect to [RFC5101], especially concerning potential failures in Template ID reuse. Additional corner cases in template management have been addressed. The new template management language is interoperable with that in [RFC5101] to the extent that the behavior was defined in the prior specification. 🧩
- Section 11.3 (Mutual Authentication) has been improved to refer to current practices in Transport Layer Security (TLS) mutual authentication; references to Punycode were removed, as these are covered in [RFC6125]. 🔐
- Editorial improvements, including structural changes to Sections 8, 9, and 10 to improve readability, have been applied. Behavior common to all transport protocols has been separated out, with exceptions per transport specifically noted. All template management language (on both Exporting and Collecting Processes) has been unified in Section 8. ✨
- A new Section 12 on management considerations has been added. 🎀
1.2. IPFIX Documents Overview 📚
okay so IPFIX basically gives network admins a way to get their hands on IP Flow information 👀 The architecture for exporting that measured IP Flow information from an IPFIX Exporting Process to a Collecting Process is defined in [RFC5470], following the requirements in [RFC3917]. This document is the part that specifies how IPFIX Data Records and Templates get carried over a bunch of transport protocols from IPFIX Exporting Processes to IPFIX Collecting Processes 🚚
Four IPFIX optimizations/extensions are currently specified 🦋 There's a bandwidth-saving method for the IPFIX protocol [RFC5473], an efficient method for exporting bidirectional flows [RFC5103], a method for the definition and export of complex data structures [RFC6313], and the specification of the Protocol on IPFIX Mediators [IPFIX-MED-PROTO], which is based on the IPFIX Mediation Framework [RFC6183]. Iconic lineup, honestly 👑
A "file-based transport" for IPFIX is discussed in [RFC5655]. It defines how IPFIX Messages can be stored in files for document-based workflows and for archival purposes 🗂️
IPFIX has a formal description of IPFIX Information Elements (their names, data types, and additional semantic information), as specified in [RFC7012]. The registry is maintained by IANA [IANA-IPFIX] 💅 The inline export of the Information Element type information is specified in [RFC5610].
The framework for packet selection and reporting [RFC5474] lets network elements select subsets of packets by statistical and other methods, and export a stream of reports on the selected packets to a Collector 🎯 The set of packet selection techniques (Sampling, Filtering, and hashing) standardized by the Packet Sampling (PSAMP) protocol is described in [RFC5475]. The PSAMP protocol [RFC5476], which uses IPFIX as its export protocol, specifies the export of packet information from a PSAMP Exporting Process to a PSAMP Collector. Instead of exporting PSAMP Packet Reports, the stream of selected packets may also serve as input to the generation of IPFIX Flow Records. Like IPFIX, PSAMP has a formal description of its Information Elements: their names, types, and additional semantic information. The PSAMP information model is defined in [RFC5477] 🧚
[RFC6615] specifies a MIB module for monitoring, and [RFC6728] specifies a data model for configuring and monitoring IPFIX and PSAMP-compliant devices using the Network Configuration Protocol (NETCONF). [RFC6727] specifies the PSAMP MIB module as an extension of the IPFIX SELECTOR MIB module defined in [RFC6615] 🌸
In terms of development, [RFC5153] provides guidelines for the implementation and use of the IPFIX protocol, while [RFC5471] provides guidelines for testing. Finally, [RFC5472] describes what types of applications can use the IPFIX protocol and how they can use the information provided. It furthermore shows how the IPFIX framework relates to other architectures and frameworks 🫶
2. Terminology ✨
okay bestie, first up are the key words. 💅 In this doc, "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" get read exactly the way [RFC2119] (which is RFC 2119) says. Those are the BCP 14 key words, so they're serious business 🎀
The basic terms like Traffic Flow, Exporting Process, Collecting Process, Observation Points, etc. mean the same thing as in the IPFIX requirements document [RFC3917] 🦋 Some got expanded so the protocol is easier to understand, and some brand-new terms got added for the protocol itself. The definitions here and in [RFC5470] are equivalent, and anything that only matters for the IPFIX protocol lives only here ✨
Section 2.1 has a handy summary table of how the different terms connect to each other 🧚
Observation Point
An Observation Point is a location in the network where packets can be observed 👀 Examples are a line with a probe attached, a shared medium like an Ethernet-based LAN, a single port of a router, or a set of interfaces (physical or logical) on a router.
Every Observation Point is tied to an Observation Domain (more on that below), and one Observation Point can be a superset of several other Observation Points 🌸 For example, an entire line card can be one Observation Point, and the individual interfaces on that line card are the smaller Observation Points inside it.
Observation Domain
An Observation Domain is the biggest set of Observation Points whose Flow information a Metering Process can aggregate 💖 A router line card can be an Observation Domain if it's made up of several interfaces, each of which is an Observation Point. In the IPFIX Message it generates, the Observation Domain includes its Observation Domain ID, which is unique per Exporting Process. That's how the Collecting Process can tell which Observation Domain a given IPFIX Message came from on the Exporter that sent it 🫶 Every Observation Point is associated with an Observation Domain.
It is RECOMMENDED that Observation Domain IDs also be unique per IPFIX Device 🔥
Packet Treatment
"Packet Treatment" means the action(s) a forwarding device or other middlebox takes on a packet, like forwarding it, dropping it, delaying it for traffic-shaping purposes, etc. 🍓
Traffic Flow or Flow
Okay so the Internet community has a bunch of different definitions of "flow," but for IPFIX we use this one 🌈
A Flow is a set of packets or frames that pass an Observation Point during a certain time interval. Every packet in a particular Flow shares a set of common properties. Each property is the result of applying a function to the values of:
- one or more packet header fields (e.g., destination IP address), transport header fields (e.g., destination port number), or application header fields (e.g., RTP header fields [RFC3550]).
- one or more characteristics of the packet itself (e.g., number of MPLS labels, etc.).
- one or more of the fields derived from Packet Treatment (e.g., next-hop IP address, the output interface, etc.).
A packet belongs to a Flow only if it completely satisfies all the defined properties of that Flow 💅
Note that the set of packets a Flow represents can be empty, so a Flow may represent zero or more packets 🙈 Since sampling counts as Packet Treatment, this definition includes packets picked by a sampling mechanism ✨
Flow Key
Each of the fields that:
- belong to the packet header (e.g., destination IP address), or
- are a property of the packet itself (e.g., packet length), or
- are derived from Packet Treatment (e.g., Autonomous System (AS) number),
and that are used to define a Flow (meaning they're the properties all packets in the Flow have in common) are called Flow Keys 🔑 As an example, the classic '5-tuple' Flow Key is source and destination IP address, source and destination transport port, and transport protocol. It groups together all the packets going in one direction of communication on a single socket 🦋
Flow Record
A Flow Record holds info about a specific Flow that was observed at an Observation Point 📝 It has measured properties of the Flow (e.g., the total number of bytes for all the Flow's packets) and usually has characteristic properties too (e.g., source IP address).
Metering Process
The Metering Process is what generates Flow Records 🧚 Its inputs are packet headers, characteristics, and Packet Treatment observed at one or more Observation Points.
It's a set of functions that includes packet header capturing, timestamping, sampling, classifying, and maintaining Flow Records. Maintaining Flow Records can mean creating new records, updating existing ones, computing Flow statistics, deriving further Flow properties, detecting Flow expiration, passing Flow Records to the Exporting Process, and deleting Flow Records 💜
Exporting Process
The Exporting Process sends IPFIX Messages to one or more Collecting Processes 🚀 The Flow Records inside those Messages come from one or more Metering Processes.
Exporter
A device that hosts one or more Exporting Processes is called an Exporter 📦
IPFIX Device
An IPFIX Device hosts at least one Exporting Process 💜 It can host more Exporting Processes too, plus any number of Observation Points and Metering Processes.
Collecting Process
A Collecting Process receives IPFIX Messages from one or more Exporting Processes 📥 It might process or store the Flow Records it gets in those Messages, but those actions are out of scope for this document 🙈
Collector
A device that hosts one or more Collecting Processes is called a Collector 🏠
Template
A Template is an ordered sequence of <type, length> pairs. It completely specifies the structure and semantics of a particular set of information that needs to be communicated from an IPFIX Device to a Collector 🎀 Every Template can be told apart by its Template ID.
IPFIX Message
An IPFIX Message originates at the Exporting Process, carries that Exporting Process's IPFIX records, and is destined for a Collecting Process 💌 It's encapsulated at the transport layer.
Message Header
The Message Header is the first part of an IPFIX Message. It gives basic info about the message, like the IPFIX version, the length of the message, the message sequence number, etc. 📋
Template Record
A Template Record defines the structure and interpretation of fields in a Data Record 🧩
Data Record
A Data Record contains values for the parameters that a Template Record describes 💅
Options Template Record
An Options Template Record is a Template Record that defines the structure and interpretation of fields in a Data Record, and also defines how to scope the applicability of that Data Record 👑
Set
A Set is a collection of records with a similar structure, prefixed by a header 🌸 Zero or more Sets follow the Message Header in an IPFIX Message. There are three types of Sets: Template Sets, Options Template Sets, and Data Sets.
Template Set
A Template Set is one or more Template Records grouped together in an IPFIX Message 🦋
Options Template Set
An Options Template Set is one or more Options Template Records grouped together in an IPFIX Message ✨
Data Set
A Data Set is one or more Data Records of the same type, grouped together in an IPFIX Message 🍓 Each Data Record was previously defined by a Template Record or an Options Template Record.
Information Element
An Information Element is a protocol- and encoding-independent description of an attribute that may appear in an IPFIX Record 🌈 They're defined in the IANA "IPFIX Information Elements" registry [IANA-IPFIX]. The type of an Information Element sets constraints on what it can contain and also decides which encoding mechanisms are valid for it in IPFIX 🔥
Transport Session
In the Stream Control Transmission Protocol (SCTP), the Transport Session is called the SCTP association, which is uniquely identified by the SCTP endpoints [RFC4960] 💜 In TCP, it's called the TCP connection, which is uniquely identified by the combination of IP addresses and TCP ports used. In UDP, it's called the UDP session, which is uniquely identified by the combination of IP addresses and UDP ports used 🫶
2.1. Terminology Summary Table
Figure A shows a summary of IPFIX terminology 🧚
+------------------+---------------------------------------------+
| | Contents |
| +--------------------+------------------------+
| Set | Template | Record |
+------------------+--------------------+------------------------+
| Data Set | / | Data Record(s) |
+------------------+--------------------+------------------------+
| Template Set | Template Record(s) | / |
+------------------+--------------------+------------------------+
| Options Template | Options Template | / |
| Set | Record(s) | |
+------------------+--------------------+------------------------+
Figure A: Terminology Summary TableA Data Set is made up of Data Record(s) 📊 It doesn't include any Template Record. A Template Record or an Options Template Record defines the Data Record.
A Template Set contains only Template Record(s) 💖
An Options Template Set contains only Options Template Record(s) ✨
3. IPFIX Message Format ✨
okay so an IPFIX Message is basically a Message Header followed by zero or more Sets 🎀. And those Sets can be one of three types: Data Set, Template Set, or Options Template Set 💖. Here's the layout, bestie 🦋
+----------------------------------------------------+
| Message Header |
+----------------------------------------------------+
| Set |
+----------------------------------------------------+
| Set |
+----------------------------------------------------+
...
+----------------------------------------------------+
| Set |
+----------------------------------------------------+
Figure B: IPFIX Message FormatHere are some examples of IPFIX Messages, because examples are everything 🌸
- An IPFIX Message made of interleaved Template, Data, and Options Template Sets, as shown in Figure C. Here, Template and Options Template Sets are sent "on demand", before the first Data Set whose structure they define 🧚
+--------+--------------------------------------------------------+
| | +----------+ +---------+ +-----------+ +---------+ |
|Message | | Template | | Data | | Options | | Data | |
| Header | | Set | | Set | ... | Template | | Set | |
| | | | | | | Set | | | |
| | +----------+ +---------+ +-----------+ +---------+ |
+--------+--------------------------------------------------------+
Figure C: IPFIX Message: Example 1- An IPFIX Message made entirely of Data Sets, sent after the matching Template Records have been defined and transmitted to the Collecting Process, as shown in Figure D 🍓
+--------+----------------------------------------------+
| | +---------+ +---------+ +---------+ |
|Message | | Data | | Data | | Data | |
| Header | | Set | ... | Set | ... | Set | |
| | +---------+ +---------+ +---------+ |
+--------+----------------------------------------------+
Figure D: IPFIX Message: Example 2- An IPFIX Message made entirely of Template and Options Template Sets, as shown in Figure E. This kind of message is useful for defining or redefining Templates and Options Templates in bulk 🫶
+--------+-------------------------------------------------+
| | +----------+ +----------+ +----------+ |
|Message | | Template | | Template | | Options | |
| Header | | Set | ... | Set | ... | Template | |
| | | | | | | Set | |
| | +----------+ +----------+ +----------+ |
+--------+-------------------------------------------------+
Figure E: IPFIX Message: Example 33.1. Message Header Format
The Message Header is laid out like this, shown in Figure F 💅
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Version Number | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Export Time |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Observation Domain ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure F: IPFIX Message Header FormatEvery Message Header field is exported in network byte order. Here's what each field means ✨
Version
Version of IPFIX that this Message follows. The value is 0x000a for the current version, which is the NetFlow services export version 9 [RFC3954] version plus one 🌈
Length
Total length of the IPFIX Message in octets, counting the Message Header and the Set(s) 💜
Export Time
The time when the IPFIX Message Header leaves the Exporter, measured in seconds since the UNIX epoch of 1 January 1970 at 00:00 UTC and encoded as an unsigned 32-bit integer ⭐
Sequence Number
An incremental sequence counter, modulo 2^32, of all IPFIX Data Records sent in the current stream from the current Observation Domain by the Exporting Process. Each SCTP Stream counts sequence numbers separately, while all messages in a TCP connection or UDP session are treated as part of the same stream. The Collecting Process can use this value to figure out whether any IPFIX Data Records went missing. Template and Options Template Records do not increase the Sequence Number 🙈
Observation Domain ID
A 32-bit identifier for the Observation Domain, which is locally unique to the Exporting Process. The Exporting Process uses the Observation Domain ID to tell the Collecting Process which Observation Domain metered the Flows. It is RECOMMENDED that this identifier also be unique per IPFIX Device. Collecting Processes SHOULD use the Transport Session and the Observation Domain ID field to separate different export streams coming from the same Exporter. The Observation Domain ID SHOULD be 0 when no specific Observation Domain ID is relevant for the entire IPFIX Message, for example when exporting the Exporting Process Statistics, or in a hierarchy of Collectors where aggregated Data Records are exported 👑
3.2. Field Specifier Format
Okay so vendors need to be able to define their own proprietary Information Elements, maybe because they're shipping a pre-standards product or because the Information Element is commercially sensitive 🔥 This section describes the Field Specifier format for both IANA-registered Information Elements [IANA-IPFIX] and enterprise-specific Information Elements 🦋
The Information Elements are identified by their Information Element identifier. When the Enterprise bit is set to 0, the Information Element appears in [IANA-IPFIX], and the Enterprise Number MUST NOT be present. When the Enterprise bit is set to 1, the Information Element identifier identifies an enterprise-specific Information Element, and the Enterprise Number MUST be present. An example of this is in Appendix A.2.2 💖
The Field Specifier format is shown in Figure G ✨
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|E| Information Element ident. | Field Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Enterprise Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure G: Field Specifier FormatHere's what each part means 🎀
E
The Enterprise bit, which is the first bit of the Field Specifier. If this bit is zero, the Information Element identifier identifies an Information Element in [IANA-IPFIX], and the four-octet Enterprise Number field MUST NOT be present. If this bit is one, the Information Element identifier identifies an enterprise-specific Information Element, and the Enterprise Number field MUST be present 🌸
Information Element identifier
A numeric value that represents the Information Element. Refer to [IANA-IPFIX] 🧚
Field Length
The length of the corresponding encoded Information Element, in octets. Refer to [IANA-IPFIX]. The Field Length may be smaller than the one listed in [IANA-IPFIX] if the reduced-size encoding is used (see Section 6.2). The value 65535 is reserved for variable-length Information Elements (see Section 7) 💜
Enterprise Number
The IANA enterprise number [IANA-PEN] of the authority that defines the Information Element identifier in this Template Record 🍓
3.3. Set and Set Header Format
A Set is a generic term for a collection of records that share a similar structure 🌈 There are three types of Sets: Template Sets, Options Template Sets, and Data Sets. Each Set has a Set Header and one or more records. The Set Format and the Set Header Format are defined in the following sections ⭐
3.3.1. Set Format
A Set has the format shown in Figure H. The record types can be Template Records, Options Template Records, or Data Records. The record types MUST NOT be mixed within a Set 🙈
+--------------------------------------------------+
| Set Header |
+--------------------------------------------------+
| record |
+--------------------------------------------------+
| record |
+--------------------------------------------------+
...
+--------------------------------------------------+
| record |
+--------------------------------------------------+
| Padding (opt.) |
+--------------------------------------------------+
Figure H: Set FormatSet Header
The Set Header Format is defined in Section 3.3.2 👑
Record
One of the record formats: Template Record, Options Template Record, or Data Record format 💅
Padding
The Exporting Process MAY insert some padding octets so the subsequent Set starts at an aligned boundary. For security reasons, the padding octet(s) MUST be composed of octets with value zero (0). The padding length MUST be shorter than any allowable record in this Set. If you need padding for the IPFIX Message together with very short records, the padding Information Element 'paddingOctets' can be used for padding records so their length goes up to a multiple of 4 or 8 octets. Because Template Sets are always 4-octet aligned by definition, padding is only needed for other alignments, e.g., on 8-octet boundaries 🦋
3.3.2. Set Header Format ✨
Every Set comes with a common header 🎀 Here's what it looks like:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure I: Set Header FormatEvery Set Header field is exported in network format 💅 The fields work like this:
Set ID: this identifies the Set. A value of2is reserved for Template Sets, and a value of3is reserved for Options Template Sets. Values from4to255are reserved for future use, and values256and above are used for Data Sets. The Set ID values of0and1aren't used, for historical reasons [RFC3954]. 🦋Length: this is the total length of the Set in octets, including the Set Header, all records, and the optional padding. A single Set MAY contain multiple records, so theLengthvalue MUST be used to figure out where the next Set starts. 🔥
3.4. Record Format 💜
IPFIX defines three record formats, which get their own sections next: the Template Record format, the Options Template Record format, and the Data Record format. 🧚
3.4.1. Template Record Format 🌸
Templates are one of the essentials of the IPFIX record format, and they're honestly iconic ✨ They give the record format so much flexibility because they let the Collecting Process handle IPFIX Messages without needing to know how to interpret every Data Record. A Template Record can hold any mix of IANA-assigned and/or enterprise-specific Information Element identifiers.
The Template Record looks like Figure J 💖 It's made up of a Template Record Header plus one or more Field Specifiers, and those Field Specifiers are defined in Figure G above.
+--------------------------------------------------+
| Template Record Header |
+--------------------------------------------------+
| Field Specifier |
+--------------------------------------------------+
| Field Specifier |
+--------------------------------------------------+
...
+--------------------------------------------------+
| Field Specifier |
+--------------------------------------------------+
Figure J: Template Record FormatHere's the Template Record Header, shown in Figure K 🎀
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID (> 255) | Field Count |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure K: Template Record Header FormatThe Template Record Header fields are defined like this:
Template ID: each Template Record gets a unique Template ID in the range256to65535🍓 That uniqueness is local to the Transport Session and Observation Domain that generated the Template ID. Template IDs are also used as Set IDs in the Sets they describe (see Section 3.4.3), so the values0-255are reserved for special Set types (e.g., Template Sets themselves). Templates and Options Templates (see Section 3.4.2) cannot share Template IDs within a Transport Session and Observation Domain. There are no constraints on the order of Template ID allocation. Exporting Processes are free to allocate Template IDs however they see fit, so Collecting Processes MUST NOT assume incremental Template IDs, or anything about the contents of a Template based on its Template ID alone. 👑Field Count: the number of fields in this Template Record. 💅
Figure L is an example of a Template Set that mixes IANA-assigned and enterprise-specific Information Elements 🦋 It has a Set Header, a Template Header, and several Field Specifiers.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 2 | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID = 256 | Field Count = N |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|1| Information Element id. 1.1 | Field Length 1.1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Enterprise Number 1.1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| Information Element id. 1.2 | Field Length 1.2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... | ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|1| Information Element id. 1.N | Field Length 1.N |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Enterprise Number 1.N |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID = 257 | Field Count = M |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| Information Element id. 2.1 | Field Length 2.1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|1| Information Element id. 2.2 | Field Length 2.2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Enterprise Number 2.2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... | ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|1| Information Element id. 2.M | Field Length 2.M |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Enterprise Number 2.M |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Padding (opt) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure L: Template Set ExampleInformation Element ids 1.2 and 2.1 appear in [IANA-IPFIX] (Enterprise bit = 0), so they don't need an Enterprise Number to identify them 🌈
3.4.2. Options Template Record Format 🧠
Thanks to the notion of scope, the Options Template Record lets the Exporter send the Collector extra information that Flow Records alone couldn't carry. Main character energy ✨
See Section 4 for specific Options Templates used for reporting metadata about IPFIX Exporting and Metering Processes 📚
3.4.2.1. Scope 💜
The scope is only available in the Options Template Set, and it gives the context for the reported Information Elements in the Data Records 🔥
The scope is made up of one or more Information Elements, specified in the Options Template Record. At a minimum, Collecting Processes SHOULD support the following as scope: observationDomainId, exportingProcessId, meteringProcessId, templateId, lineCardId, exporterIPv4Address, exporterIPv6Address, and ingressInterface Information Elements. The IPFIX protocol doesn't stop you from using any Information Element for scope, but some Information Element types don't make sense as scope (for example, the counter Information Elements) 🙈
The IPFIX Message Header already contains the Observation Domain ID. If it isn't zero, that Observation Domain ID can be treated as an implicit scope for the Data Records in the IPFIX Message 🦋
Multiple Scope Fields MAY be present in the Options Template Record, and in that case the composite scope is the combination of the scopes. For example, if the two scopes are meteringProcessId and templateId, the combined scope is this Template for this Metering Process 💖 If a different order of Scope Fields would give a Record a different semantic meaning, then the order of Scope Fields MUST be preserved by the Exporting Process. For example, in the context of PSAMP [RFC5476], if the first scope defines the filtering function and the second scope defines the sampling function, the order really matters. Applying the sampling function first and then the filtering function could lead to different Data Records than applying the filtering function first and then the sampling function 😭
3.4.2.2. Options Template Record Format
okay so Options Template Records are giving main character energy 🎀 because they can hold any combo of IANA-assigned and/or enterprise-specific Information Element identifiers 🦋 mix and match, bestie!
An Options Template Record is made of an Options Template Record Header plus one or more Field Specifiers 💖 (the Field Specifiers are the ones from Figure G up above ✨). Here's the layout, as shown in Figure M:
+--------------------------------------------------+
| Options Template Record Header |
+--------------------------------------------------+
| Field Specifier |
+--------------------------------------------------+
| Field Specifier |
+--------------------------------------------------+
...
+--------------------------------------------------+
| Field Specifier |
+--------------------------------------------------+
Figure M: Options Template Record FormatNow the header! 👑 Figure N shows how the Options Template Record Header is laid out:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID (> 255) | Field Count |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope Field Count |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure N: Options Template Record Header FormatHere's what each header field means, no cap 💅
Template ID
Every Options Template Record gets its own unique Template ID in the range 256 to 65535 🌸 and that uniqueness is local to the Transport Session and Observation Domain that generated the Template ID. Template IDs double as Set IDs in the sets they describe (see Section 3.4.3), so values 0-255 are reserved for special Set types (e.g., Template Sets themselves) 🧚. Templates and Options Templates also can't share Template IDs within a Transport Session and Observation Domain, so no sharing, periodt! 🙅♀️ There are no constraints on the order of Template ID allocation. Exporting Processes get to allocate Template IDs however they see fit, so Collecting Processes MUST NOT assume incremental Template IDs, or anything about the contents of an Options Template based only on its Template ID 🙈 don't guess, bestie!
Field Count
This is the number of all fields in this Options Template Record, including the Scope Fields 🍓
Scope Field Count
This is how many scope fields the Options Template Record has 🔥 Scope Fields are normal Fields, except that the Collector interprets them as scope. A scope field count of N means the first N Field Specifiers in the Template Record are Scope Fields ✨ and the Scope Field Count MUST NOT be zero. Zero is a no-go, bestie 💀
Figure O is the example, and it's giving mixed energy 🌈 an Options Template Set with both IANA-assigned and enterprise-specific Information Elements. It's made of a Set Header, an Options Template Header, and several Field Specifiers 🫶
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 3 | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID = 258 | Field Count = N + M |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope Field Count = N |0| Scope 1 Infor. Element id. |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope 1 Field Length |0| Scope 2 Infor. Element id. |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope 2 Field Length | ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |1| Scope N Infor. Element id. |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope N Field Length | Scope N Enterprise Number ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
... Scope N Enterprise Number |1| Option 1 Infor. Element id. |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Option 1 Field Length | Option 1 Enterprise Number ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
... Option 1 Enterprise Number | ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |0| Option M Infor. Element id. |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Option M Field Length | Padding (optional) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure O: Options Template Set Example3.4.3. Data Record Format
Data Records ride along in Data Sets 🦋 and the format is shown in Figure P. A Data Record is just one or more Field Values, nothing extra 💜 The Template ID that the Field Values belong to is encoded in the Set Header field "Set ID", i.e., "Set ID" = "Template ID" ✨
+--------------------------------------------------+
| Field Value |
+--------------------------------------------------+
| Field Value |
+--------------------------------------------------+
...
+--------------------------------------------------+
| Field Value |
+--------------------------------------------------+
Figure P: Data Record FormatHeads up, though: Field Values do not necessarily have a length of 16 bits 😭 they're encoded according to their data type as specified in [RFC7012]. Plan accordingly, bestie 🌸
Data Record interpretation only works if the Template Record matching the Template ID is available at the Collecting Process. No template, no decoding, no vibes 💀
Figure Q shows a Data Set: a Set Header followed by several Field Values 🧚
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = Template ID | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Record 1 - Field Value 1 | Record 1 - Field Value 2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Record 1 - Field Value 3 | ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Record 2 - Field Value 1 | Record 2 - Field Value 2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Record 2 - Field Value 3 | ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Record 3 - Field Value 1 | Record 3 - Field Value 2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Record 3 - Field Value 3 | ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... | Padding (optional) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure Q: Data Set, Containing Data Records4. Specific Reporting Requirements ✨
Some specific Options Templates and Options Template Records are needed to give extra info about the Flow Records and about the Metering Process 📊
The Options Templates and Options Template Records defined in these subsections impose some constraints on the Metering Process and Exporting Process implementations 🔥 They MAY be implemented. If you do implement them, the specific Options Templates SHOULD be implemented as specified in these subsections 💅
The minimum set of Information Elements is always specified in these Specific IPFIX Options Templates 🌈 Extra Information Elements may still be used in them, though.
The Collecting Process MUST check the possible combinations of Information Elements within the Options Template Records so it can correctly interpret the following Options Templates 🫶
4.1. The Metering Process Statistics Options Template
The Metering Process Statistics Options Template describes the structure of a Data Record for reporting Metering Process statistics 📈 It SHOULD contain the following Information Elements, as defined in [IANA-IPFIX]:
(scope) observationDomainId- This Information Element MUST be defined as a Scope Field and MUST be present, unless the Observation Domain ID of the enclosing Message is non-zero.
(scope) meteringProcessId- If present, this Information Element MUST be defined as a Scope Field.
exportedMessageTotalCountexportedFlowRecordTotalCountexportedOctetTotalCount
The Exporting Process SHOULD export the Data Record specified by the Metering Process Statistics Options Template on a regular basis or based on some export policy 🦋 That periodicity or export policy SHOULD be configurable ⭐
Note that if several Metering Processes are available on the Exporter Observation Domain, the Information Element meteringProcessId MUST be specified as an additional Scope Field 🧚💖
4.2. The Metering Process Reliability Statistics Options Template 🧚
okay so this template is for reporting when the Metering Process isn't being reliable 😭 it's giving "here's exactly what got dropped." It SHOULD contain the following Information Elements, as defined in [IANA-IPFIX]:
(scope) observationDomainId🎀- This Information Element MUST be defined as a Scope Field and MUST be present, unless the Observation Domain ID of the enclosing Message is non-zero.
(scope) meteringProcessId💜- If present, this Information Element MUST be defined as a Scope Field.
ignoredPacketTotalCountignoredOctetTotalCounttime first packet ignored⏰- This is the timestamp of the first packet that the Metering Process ignored. For this timestamp, any of these timestamp Information Elements can be used:
observationTimeSeconds,observationTimeMilliseconds,observationTimeMicroseconds, orobservationTimeNanoseconds.
- This is the timestamp of the first packet that the Metering Process ignored. For this timestamp, any of these timestamp Information Elements can be used:
time last packet ignored⏰- This is the timestamp of the last packet that the Metering Process ignored. For this timestamp, any of these timestamp Information Elements can be used:
observationTimeSeconds,observationTimeMilliseconds,observationTimeMicroseconds, orobservationTimeNanoseconds.
- This is the timestamp of the last packet that the Metering Process ignored. For this timestamp, any of these timestamp Information Elements can be used:
The Exporting Process SHOULD export the Data Record specified by the Metering Process Reliability Statistics Options Template on a regular basis or based on some export policy. That periodicity or export policy SHOULD be configurable ✨
Note that if several Metering Processes are available on the Exporter Observation Domain, the Information Element meteringProcessId MUST be specified as an additional Scope Field. 🧐
Since the Metering Process Reliability Statistics Options Template contains two identical timestamp Information Elements, and since the order of the Information Elements in the Template Records is not guaranteed, the Collecting Process interprets the time interval of ignored packets as the range between the two values. See Section 5.2 for wraparound considerations 🦋
4.3. The Exporting Process Reliability Statistics Options Template 💅
The Exporting Process Reliability Statistics Options Template specifies the structure of a Data Record for reporting lack of reliability in the Exporting Process. It SHOULD contain the following Information Elements, as defined in [IANA-IPFIX]:
(scope) Exporting Process Identifier🔑- This is the identifier of the Exporting Process whose reliability is being reported. Any of these Information Elements can be used for this field:
exporterIPv4Address,exporterIPv6Address, orexportingProcessId. This Information Element MUST be defined as a Scope Field.
- This is the identifier of the Exporting Process whose reliability is being reported. Any of these Information Elements can be used for this field:
notSentFlowTotalCountnotSentPacketTotalCountnotSentOctetTotalCounttime first flow dropped⏰- This is the time at which the first Flow Record was dropped by the Exporting Process. For this timestamp, any of these timestamp Information Elements can be used:
observationTimeSeconds,observationTimeMilliseconds,observationTimeMicroseconds, orobservationTimeNanoseconds.
- This is the time at which the first Flow Record was dropped by the Exporting Process. For this timestamp, any of these timestamp Information Elements can be used:
time last flow dropped⏰- This is the time at which the last Flow Record was dropped by the Exporting Process. For this timestamp, any of these timestamp Information Elements can be used:
observationTimeSeconds,observationTimeMilliseconds,observationTimeMicroseconds, orobservationTimeNanoseconds.
- This is the time at which the last Flow Record was dropped by the Exporting Process. For this timestamp, any of these timestamp Information Elements can be used:
The Exporting Process SHOULD export the Data Record specified by the Exporting Process Reliability Statistics Options Template on a regular basis or based on some export policy. That periodicity or export policy SHOULD be configurable 🌸
Since the Exporting Process Reliability Statistics Options Template contains two identical timestamp Information Elements, and since the order of the Information Elements in the Template Records is not guaranteed, the Collecting Process interprets the time interval of dropped packets as the range between the two values. See Section 5.2 for wraparound considerations 🫶
4.4. The Flow Keys Options Template 🗝️
The Flow Keys Options Template specifies the structure of a Data Record for reporting the Flow Keys of reported Flows. A Flow Keys Data Record extends a particular Template Record that is referenced by its templateId. The Template Record is extended by specifying which of the Information Elements contained in the corresponding Data Records describe Flow properties that serve as Flow Keys of the reported Flow. 👑
The Flow Keys Options Template SHOULD contain the following Information Elements, as defined in [IANA-IPFIX]:
(scope) templateId💖- This Information Element MUST be defined as a Scope Field.
flowKeyIndicator
5. Timing Considerations ⏳
5.1. IPFIX Message Header Export Time and Flow Record Time 🕰️
The IPFIX Message Header Export Time field is the time at which the IPFIX Message Header leaves the Exporter. It uses the same encoding as the dateTimeSeconds abstract data type [RFC7012], which means it's expressed in seconds since the UNIX epoch, 1 January 1970 at 00:00 UTC, encoded as an unsigned 32-bit integer.
Certain time-related Information Elements may be expressed as an offset from this Export Time. For example, Data Records that need microsecond precision can export the flow start and end times with the flowStartMicroseconds and flowEndMicroseconds Information Elements. These encode the absolute time in microseconds in terms of the NTP epoch, 1 January 1900 at 00:00 UTC, in a 64-bit field.
Another option is to export the flowStartDeltaMicroseconds and flowEndDeltaMicroseconds Information Elements in the Data Record. These report the flow start and end time as negative offsets from the Export Time, as an unsigned 32-bit integer. This second option lowers the export bandwidth requirement, saving four bytes per timestamp, but it increases the load on the Exporter. The Exporting Process has to calculate the flowStartDeltaMicroseconds and flowEndDeltaMicroseconds of every single Data Record before exporting the IPFIX Message 🧮
It must be noted that timestamps based on the Export Time put some time constraints on the Data Records contained within the IPFIX Message. In the example of the flowStartDeltaMicroseconds and flowEndDeltaMicroseconds Information Elements, the Data Record can only contain records with timestamps within 71 minutes of the Export Time. Otherwise, the 32-bit counter wouldn't be enough to hold the flow start time offset 😬
5.2. Supporting Timestamp Wraparound 🔄
The dateTimeSeconds abstract data type [RFC7012] and the Export Time Message Header field (Section 3.1) are encoded as 32-bit unsigned integers. They're expressed as seconds since the UNIX epoch, 1 January 1970 at 00:00 UTC, as defined in [POSIX.1]. These values will wrap around on 7 February 2106 at 06:28:16 UTC 📅
To keep using the IPFIX protocol past this date, Exporting Processes SHOULD export dateTimeSeconds values and the Export Time Message Header field as the number of seconds since the UNIX epoch, 1 January 1970 at 00:00 UTC, modulo 2^32. Collecting Processes SHOULD use the current date, or other contextual information, to properly interpret dateTimeSeconds values and the Export Time Message Header field 🧠
There are similar considerations for the NTP-based dateTimeMicroseconds and dateTimeNanoseconds abstract data types [RFC7012]. Exporting Processes SHOULD export dateTimeMicroseconds and dateTimeNanoseconds values as if the NTP era [RFC5905] is implicit. Collecting Processes SHOULD use the current date, or other contextual information, to determine the NTP era so they can properly interpret dateTimeMicroseconds and dateTimeNanoseconds values in received Data Records 🤓
The dateTimeMilliseconds abstract data type will wrap around in approximately 500 billion years. The specification of how this abstract data type behaves after that time is left as a subject of a future revision of this specification 🌌
The long-term storage of files [RFC5655] for archival purposes is affected by timestamp wraparound. Using the current date to interpret timestamp values in files stored for multiple decades in the past may lead to incorrect values. Therefore, it is RECOMMENDED that such files be stored with contextual information to help interpret these timestamps 📂
6. Linkage with the Information Model 🧩
As with values in the IPFIX Message Header and Set Header, values of all Information Elements [RFC7012], except for those of the string and octetArray data types, are encoded in canonical format in network byte order (also known as big-endian byte ordering) 💜
6.1. Encoding of IPFIX Data Types 🎀
The following sections define the encoding of the data types specified in [RFC7012].
6.1.1. Integral Data Types 🔢
Integral data types (unsigned8, unsigned16, unsigned32, unsigned64, signed8, signed16, signed32, and signed64) MUST be encoded using the default canonical format in network byte order. Signed integral data types are represented in two's complement notation.
6.1.2. Address Types 🌐
Address types (macAddress, ipv4Address, and ipv6Address) MUST be encoded the same way as the integral data types, as six, four, and sixteen octets in network byte order, respectively.
6.1.3. float32 🌊
The float32 data type MUST be encoded as an IEEE binary32 floating point type as specified in [IEEE.754.2008], in network byte order as specified in Section 3.6 of [RFC1014]. Note that on little-endian machines, byte swapping of the native representation is necessary before export. The method for doing this may be implementation platform dependent.
6.1.4. float64 🌊
The float64 data type MUST be encoded as an IEEE binary64 floating point type as specified in [IEEE.754.2008], in network byte order as specified in Section 3.7 of [RFC1014]. Note that on little-endian machines, byte swapping of the native representation is necessary before export. The method for doing this may be implementation platform dependent.
6.1.5. boolean ✅
The boolean data type is specified according to the TruthValue in [RFC2579]. It is encoded as a single-octet integer per Section 6.1.1, with the value 1 for true and value 2 for false. Every other value is undefined.
6.1.6. string and octetArray 🔤
The "string" data type represents a finite-length string of valid characters of the Unicode character encoding set. The string data type MUST be encoded in UTF-8 [RFC3629] format. The string is sent as an array of zero or more octets using an Information Element of fixed or variable length. IPFIX Exporting Processes MUST NOT send IPFIX Messages containing ill-formed UTF-8 string values for Information Elements of the string data type. Collecting Processes SHOULD detect and ignore such values. See [UTF8-EXPLOIT] for background on this issue 🙈
The octetArray data type has no encoding rules. It represents a raw array of zero or more octets, with the interpretation of the octets defined in the Information Element definition 🦋
6.1.7. dateTimeSeconds 💖
okay so dateTimeSeconds is an unsigned 32-bit integer in network byte order 🧚 It counts the seconds since the UNIX epoch, which is 1 January 1970 at 00:00 UTC (as defined in [POSIX.1]). It's encoded exactly the same way as the IPFIX Message Header Export Time field, so they're literally twins 👯♀️ It can represent dates from 1 January 1970 to 7 February 2106 without wraparound, but check Section 5.2 for the wraparound details, bestie 🫶
6.1.8. dateTimeMilliseconds 🌸
dateTimeMilliseconds is an unsigned 64-bit integer in network byte order that counts the milliseconds since the UNIX epoch, 1 January 1970 at 00:00 UTC (as defined in [POSIX.1]) ✨ It can represent dates starting on 1 January 1970 and keeps going for approximately the next 500 billion years without wraparound. That's giving forever 💅
6.1.9. dateTimeMicroseconds 🦋
The dateTimeMicroseconds data type is a 64-bit field encoded according to the NTP Timestamp format, as defined in Section 6 of [RFC5905] 🎀 It's made of two unsigned 32-bit integers in network byte order, called Seconds and Fraction. The Seconds field counts the seconds since the NTP epoch, 1 January 1900 at 00:00 UTC.
The Fraction field holds the fractional part of a second in units of 1/(2^32) seconds, which is about 233 picoseconds 🔥 It can represent dates between 1 January 1900 and 8 February 2036 in the current NTP era. For wraparound stuff, see Section 5.2 🙈
Here's the tea: dateTimeMicroseconds and dateTimeNanoseconds have an identical encoding 🍓 But dateTimeMicroseconds is only meant for timestamps with microsecond precision, so the bottom 11 bits of the Fraction field SHOULD be zero and MUST be ignored for all Information Elements of this data type (as 2^11 x 233 picoseconds = .477 microseconds) 💀
6.1.10. dateTimeNanoseconds 🌈
The dateTimeNanoseconds data type is a 64-bit field encoded according to the NTP Timestamp format, as defined in Section 6 of [RFC5905] 🧚 Just like before, it's two unsigned 32-bit integers in network byte order: Seconds and Fraction. Seconds counts the seconds since the NTP epoch, 1 January 1900 at 00:00 UTC. Fraction is the fractional number of seconds in units of 1/(2^32) seconds (about 233 picoseconds), and it can represent dates between 1 January 1900 and 8 February 2036 in the current NTP era. Wraparound details live in Section 5.2 💖
Remember, dateTimeMicroseconds and dateTimeNanoseconds share an identical encoding. For dateTimeNanoseconds, there is no restriction on how the Fraction field gets interpreted ✨
6.2. Reduced-Size Encoding 💅
Information Elements encoded as signed, unsigned, or float data types MAY be encoded using fewer octets than their information model definition implies 🦋 The idea is that the smaller size is still enough for any value the Exporter might need to send. That cuts down the network bandwidth needed between the Exporter and the Collector. Also, the Information Element definitions [IANA-IPFIX] always define the maximum encoding size, so keep that in mind 🫶
Here's an example. The information model defines octetDeltaCount as an unsigned64 type, which needs 64 bits. But if the Exporter will never locally run into a value bigger than 4294967295, it can send the value as unsigned32 instead 🎀
The Exporter shows this by putting a smaller length in the Template than the one tied to the Information Element's assigned type. In the example above, the Exporter would put a length of 4 instead of 8 in the Template 🌸
Reduced-size encoding MAY be used for these integer types: unsigned64, signed64, unsigned32, signed32, unsigned16, and signed16 🔥 The signed versus unsigned property of the reported value MUST be preserved ✨ The size can drop to any number of octets smaller than the original type, as long as the value still fits. In other words, only leading zeroes get dropped. For example, an unsigned64 can shrink to 7, 6, 5, 4, 3, 2, or 1 octet(s) 🍓
Reduced-size encoding MAY also be used to reduce float64 to float32 🌈 A float32 has a smaller number range, and because its mantissa is smaller, it's less precise too. In this case, the float64 would be reduced to 4 octets 🧚
Reduced-size encoding MUST NOT be applied to any other data type defined in [RFC7012] that implies a fixed length. Those types either have internal structure (like ipv4Address or dateTimeMicroseconds) or restricted ranges that aren't good for reduced-size encoding (like dateTimeMilliseconds) 💀
Information Elements of type octetArray and string may be exported using any length, subject to the length restrictions specific to each Information Element, as noted in that Information Element's description 💖
7. Variable-Length Information Element ✨
The IPFIX Template mechanism is optimized for fixed-length Information Elements [RFC7012] 🦋 So when an Information Element has a variable length, the following mechanism MUST be used to carry the length information for both the IANA-assigned and the enterprise-specific Information Elements 🔥
In the Template Set, the Information Element Field Length is recorded as 65535. That reserved value tells the Collecting Process that the length of the Information Element will be carried in the Information Element content itself 🎀
Most of the time, the length of the Information Element will be less than 255 octets. So the following length-encoding mechanism is tuned for that common case and keeps the overhead small. The length is carried in the octet right before the Information Element, as shown in Figure R 🌸
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Length (< 255)| Information Element |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... continuing as needed |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure R: Variable-Length Information Element (IE)
(Length < 255 Octets)The length can also be encoded in 3 octets before the Information Element. That lets the length be greater than or equal to 255 octets 🧚 In this case, the first octet of the Length field MUST be 255, and the length is carried in the second and third octets, as shown in Figure S 💖
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 255 | Length (0 to 65535) | IE |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... continuing as needed |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure S: Variable-Length Information Element (IE)
(Length 0 to 65535 Octets)The octets carrying the length (either the first octet or the first three octets) MUST NOT be included in the length of the Information Element. 🌈😭
8. Template Management ✨
okay bestie, this section is all about how the Exporting Process and the Collecting Process keep their Templates and Options Templates in sync 🫶 Templates are the blueprints that tell the Collector how to read the Data Records, so getting them right is literally everything 💅
The goal is for both sides to have a consistent view of the Templates and Options Templates used to encode and decode the Records sent between them. Not always easy, ngl 😭 It gets complicated because of two things: 1) Template IDs can be reused within a Transport Session, and 2) Templates need to work even when UDP is the transport and nothing is guaranteed to arrive 💀
The Template Management mechanisms here apply to exporting IPFIX Messages over SCTP, TCP, or UDP. There are extra details for SCTP and UDP in Sections 8.3 and 8.4, respectively 🦋
The Exporting Process assigns and keeps track of Template IDs per Transport Session and Observation Domain 🌸 When a new Template Record gets created, the Exporting Process gives it an unused Template ID. The Collecting Process MUST store all the received Template Record information for the whole Transport Session, until the Template is reused or withdrawn (Section 8.1) or expires over UDP (Section 8.4). That's how it can still interpret the matching Data Records 🧚
The Collecting Process MUST NOT assume that a Template ID from a given Exporting Process means the same Template it meant in an earlier Transport Session from that same Exporting Process 🙈 And a Collecting Process MUST NOT use Templates from one Transport Session to decode Data Sets in a later one. Not me doing that, periodt 💅
If a specific Information Element is required by a Template but isn't in the packets that were observed, the Exporting Process MAY choose to export Flow Records without that Information Element, using a new Template for the Data Record 🎀
If an Information Element is needed more than once in a Template, the different occurrences SHOULD follow the logical order of how the Metering Process handled them 🔥 For example, say a selected packet goes through two hash functions and both hash values are sent in one Template. The first occurrence should belong to the first hash function in the Metering Process. Another example: when exporting the two source IP addresses of an IPv4-in-IPv4 packet, the first sourceIPv4Address occurrence SHOULD be the IPv4 address of the outer header, and the second occurrence SHOULD be the address of the inner header 👑 Collecting Processes MUST properly handle Templates with multiple identical Information Elements, no excuses 💖
The Exporting Process SHOULD send the Template Set and Options Template Set before any Data Sets that use that (Options) Template ID, so the Collector already has the Template Record before the first Data Record shows up 🌈 Data Records that match a Template Record MAY show up in the same and/or later IPFIX Message(s). Still, a Collecting Process MUST NOT assume the Data Set and its associated Template Set (or Options Template Set) come in the same IPFIX Message ⭐
A Collecting Process usually gets Template Records before Data Records, but that's not always what happens. Reordering or a Collecting Process restart over UDP can flip the order 😭 In those cases the Collecting Process MAY buffer Data Records it has no Templates for, while it waits for the Template Records that describe them. Just note that if Templates get withdrawn and redefined (Section 8.1), this can lead to wrong interpretation of Data Records 🥺
Different Observation Domains within a Transport Session MAY use the same Template ID value for different Templates, and Collecting Processes MUST properly handle that case 🦋
Options Templates and Templates that are related or depend on each other (for example, by sharing common properties as described in [RFC5473]) SHOULD be sent together in the same IPFIX Message 🫶
8.1. Template Withdrawal and Redefinition 🍓
Templates that the Exporting Process won't use anymore MAY be withdrawn by sending a Template Withdrawal. After a Template Withdrawal arrives, a Collecting Process MUST stop using that Template to interpret the Data Sets exported afterwards. Heads up: this doesn't work when UDP is carrying the IPFIX Messages, so for that case check Section 8.4 💀
A Template Withdrawal is a Template Record for the Template ID being withdrawn, with a Field Count of 0. Here's what it looks like in Figure T 👑
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = (2 or 3) | Length = 16 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID N | Field Count = 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID ... | Field Count = 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID M | Field Count = 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure T: Template Withdrawal FormatThe Set ID field MUST contain the value 2 for a Template Set Withdrawal or the value 3 for an Options Template Set Withdrawal. Multiple Template IDs MAY be withdrawn in a single Template Withdrawal, and in that case, padding MAY be used 🎀
Template Withdrawals MAY show up interleaved with Template Sets, Options Template Sets, and Data Sets in an IPFIX Message. When that happens, the Templates and Template Withdrawals shall be interpreted as taking effect in the order they appear in the IPFIX Message 🧚 An Exporting Process SHOULD NOT send a Template Withdrawal until enough time has passed for any Data Records described by the withdrawn Templates to be received and processed. Check Section 8.2 for the details on sequencing Template management actions 🌸
The end of a Transport Session implicitly withdraws all the Templates used in it, and Templates must be resent in later Transport Sessions between the Exporting Process and Collecting Process. This only applies to SCTP and TCP. For Transport Session and Template lifetime over UDP, see Sections 8.4 and 10.3.4 🦋
All Templates for a given Observation Domain MAY also be withdrawn with an All Templates Withdrawal, shown in Figure U. All Options Templates for a given Observation Domain MAY likewise be withdrawn with an All Options Templates Withdrawal, shown in Figure V 💜
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 2 | Length = 8 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID = 2 | Field Count = 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure U: All Templates Withdrawal Set Format 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 3 | Length = 8 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID = 3 | Field Count = 0 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure V: All Options Templates Withdrawal Set FormatTemplate IDs MAY be reused for new Templates by sending a new Template Record or Options Template Record for a given Template ID after the Template has been withdrawn 🌈
If a Collecting Process gets a Template Withdrawal for a Template or Options Template it doesn't currently have stored, that points to a malfunctioning or improperly implemented Exporting Process 😬 Data Records can still be received and interpreted, but the Collecting Process MUST ignore the Template Withdrawal and SHOULD log the error ⭐
If a Collecting Process gets a new Template Record or Options Template Record for an already-allocated Template ID, and that Template or Options Template is identical to the one already received, it SHOULD log the retransmission. It's not an error, though, since it doesn't change how the Data Records are interpreted 💅
If a Collecting Process gets a new Template Record or Options Template Record for an already-allocated Template ID, and that Template or Options Template is different from the one already received, that points to a malfunctioning or improperly implemented Exporting Process 🔥 Continued receipt and unambiguous interpretation of Data Records for this Template ID are no longer possible, and the Collecting Process SHOULD log the error. Any further Collecting Process actions are out of scope for this specification 🫶
8.2. Sequencing Template Management Actions
Okay so IPFIX Messages have zero guarantee of arriving in order across SCTP Streams or over UDP 😭 which means an Exporting Process MUST sequence all Template management actions (aka Template Records defining new Templates, and Template Withdrawals withdrawing them) using the Export Time field in the IPFIX Message Header 💅
An Exporting Process MUST NOT export a Data Set described by a new Template in an IPFIX Message with an Export Time before the Export Time of the IPFIX Message containing that Template. 🙅♀️ And if a new Template and a Data Set described by it show up in the same IPFIX Message, the Template Set containing the Template MUST appear before the Data Set in the Message. Template first, always, bestie 🌸
An Exporting Process MUST NOT export any Data Sets described by a withdrawn Template in IPFIX Messages with an Export Time after the Export Time of the IPFIX Message containing the Template Withdrawal withdrawing that Template. Once it's gone, it's gone 💀
Put another way, a Template describes Data Records contained in IPFIX Messages when the Export Time of those messages is between a specific start and end time, inclusive. 🦋 The start time is the Export Time of the IPFIX Message containing the Template Record. The end time is one of two things: if the Template gets withdrawn during the session, it's the Export Time of the IPFIX Message containing the Template Withdrawal for that Template; otherwise, it's the end of the Transport Session.
Even when everything is sent in order, IPFIX Messages containing Template management actions could still arrive at the Collecting Process out of order, i.e., if they're sent via UDP or via different SCTP Streams. 😵💫 Given this, Template Withdrawals and later reuse of Template IDs can seriously complicate the job of figuring out Template lifetimes at the Collecting Process. A Collecting Process MAY implement a buffer and use Export Time information to disambiguate the order of Template management actions. 🧠 If it does build one, the buffer SHOULD be configurable so it can add a delay on the order of the maximum reordering delay experienced at the Collecting Process. Also, in this case the Collecting Process's clock is totally irrelevant 🙃 it's only comparing the Export Times of Messages to each other.
8.3. Additional Considerations for Template Management over SCTP
The specifications in this section apply only to SCTP. If anything here contradicts Section 8 or Section 8.1, this section takes precedence, no debate 👑
Template Sets and Options Template Sets MAY be sent on any SCTP Stream. Data Sets sent on a given SCTP Stream MAY be represented by Template Records exported on any SCTP Stream. ✨
Template Sets and Options Template Sets MUST be sent reliably, using SCTP ordered delivery. Non-negotiable 🫶
Template Withdrawals MAY be sent on any SCTP Stream. Template Withdrawals MUST be sent reliably, using SCTP ordered delivery. Template IDs MAY be reused by sending a Template Withdrawal and/or a new Template Record on a different SCTP Stream than the stream the original Template was sent on. 🦋
There's more Template management stuff in [RFC6526], which specifies an extension that explicitly links Templates with SCTP Streams. 🎀 The tradeoff is more restrictive rules on how Template Records get assigned to SCTP Streams, but in exchange, the extension allows fast, reliable reuse of Template IDs and estimation of Data Record loss per Template. Worth it, lowkey 💅
8.4. Additional Considerations for Template Management over UDP
The specifications in this section apply only to UDP. Same deal as before: if they clash with Section 8 or Section 8.1, this section wins 💖
Since UDP has no way to reliably transmit Templates, Exporting Processes using UDP as the transport protocol MUST periodically retransmit each active Template at regular intervals. 🔁 The Template retransmission interval MUST be configurable via, for example, the templateRefreshTimeout and optionsTemplateRefreshTimeout parameters as defined in [RFC6728]. Default settings for these values are deployment- and application-specific, so it depends on who's running it 🌈
Before exporting any Data Records described by a given Template Record or Options Template Record, especially when Template IDs are being reused as described in Section 8.1, the Exporting Process SHOULD send multiple copies of the Template Record in a separate IPFIX Message, to help make sure the Collecting Process actually got it. 📬
To keep resource needs low for Templates the Exporting Process isn't using anymore, the Collecting Process MAY associate a lifetime with each Template received in a Transport Session. ⏳ Templates not refreshed by the Exporting Process within that lifetime can then be discarded by the Collecting Process. The Template lifetime at the Collecting Process MAY be exposed by a configuration parameter, or MAY be derived from observing the interval of periodic Template retransmissions from the Exporting Process. In that latter case, the Template lifetime SHOULD default to at least 3 times the observed retransmission rate. 🧮
Template Withdrawals (Section 8.1) MUST NOT be sent by Exporting Processes exporting via UDP, and MUST be ignored by Collecting Processes collecting via UDP. 🙈 Template IDs MAY be reused by Exporting Processes by exporting a new Template for the Template ID after waiting at least 3 times the retransmission delay. Be careful though: Template ID reuse may lead to incorrect interpretation of Data Records if the retransmission and lifetime aren't configured properly. 😬
When a Collecting Process receives a new Template Record or Options Template Record via UDP for an already-allocated Template ID, and that Template or Options Template is identical to the already-received one, it SHOULD NOT log the retransmission. This is just normal Template refresh over UDP, so no need to panic-log it 💜
When a Collecting Process receives a new Template Record or Options Template Record for an already-allocated Template ID, and that Template or Options Template is different from the already-received one, the Collecting Process MUST replace the Template or Options Template for that Template ID with the newly received one. This is the normal operation of Template ID reuse over UDP, iykyk 🫧
Template IDs are unique per UDP session and per Observation Domain, so at any given time, the Collecting Process SHOULD maintain the following for all the current Template Records and Options Template Records: <IPFIX Device, Exporter source UDP port, Collector IP address, Collector destination UDP port, Observation Domain ID, Template ID, Template Definition, Last Received>. 📋
9. The Collecting Process's Side 🎀
This section covers how the Collecting Process handles the IPFIX protocol, and it's the same across all transport protocols. Extra considerations for SCTP and UDP live in Sections 9.2 and 9.3, respectively. Template management at Collecting Processes is covered in Section 8. Bookmark it, it's a lot 📚
The Collecting Process MUST listen for association requests / connections to start new Transport Sessions from the Exporting Process. 👂
The Collecting Process MUST note the Information Element identifier of any Information Element that it does not understand, and MAY discard that Information Element from received Data Records. 🕵️♀️
The Collecting Process MUST accept padding in Data Records and Template Records. The padding size is the Set Length minus the size of the Set Header (4 octets for the Set ID and the Set Length), modulo the minimum Record size deduced from the Template Record. It's just math, bestie 🧮
The IPFIX protocol has a Sequence Number field in the Export header that increases with the number of IPFIX Data Records in the IPFIX Message. 🔢 A Collector can detect out-of-sequence, dropped, or duplicate IPFIX Messages by tracking the Sequence Number. A Collector SHOULD provide a logging mechanism for tracking out-of-sequence IPFIX Messages. Those out-of-sequence messages may be due to a bunch of things: Exporter resource exhaustion where it can't send messages at their creation rate, an Exporting Process reset, congestion on the network link between the Exporter and Collector, Collector resource exhaustion where it can't process messages at their arrival rate, out-of-order packet reception, duplicate packet reception, or an attacker injecting false messages. 🚨 Yikes, so many suspects 😭
9.1. Collecting Process Handling of Malformed IPFIX Messages
If the Collecting Process receives a malformed IPFIX Message, it MUST discard the IPFIX Message and SHOULD log the error. 🗑️ A malformed IPFIX Message is one that can't be interpreted because of nonsensical length values (e.g., a variable-length Information Element longer than its enclosing Set, a Set longer than its enclosing IPFIX Message, or an IPFIX Message shorter than an IPFIX Message Header). It also counts as malformed if it has a reserved Version value. That might mean a future version of IPFIX is being used for export, but in practice it happens most often when non-IPFIX data gets sent to an IPFIX Collecting Process 🙃 Note that non-zero Set padding does NOT make a message malformed, so chill about that one 🌸
As the most likely cause of malformed IPFIX Messages is a poorly implemented Exporting Process, or someone sending non-IPFIX data to an IPFIX Collecting Process, a human probably needs to step in and fix it. 🧑💻 In the meantime, the Collecting Process MAY try to rectify the situation any way it sees fit, including:
- terminating the TCP connection or SCTP connection
- using the receiver window to reduce network load from the malfunctioning Exporting Process
- buffering and saving malformed IPFIX Message(s) to help with diagnosis
- attempting to resynchronize the stream, e.g., as described in Section 10.3 of [RFC5655]
Resynchronization should only be attempted if the Collecting Process has reason to believe the error is transient. 🤞 On the other hand, the Collecting Process SHOULD stop processing IPFIX Messages from clearly malfunctioning Exporting Processes (e.g., those where the last few IPFIX Messages have been malformed). Know when to cut them off 🚫💅
9.2. Additional Considerations for SCTP Collecting Processes
As an Exporting Process may request and support more than one stream per SCTP association, the Collecting Process MUST support the opening of multiple SCTP Streams. 🌊 Multi-stream is the whole vibe, so the Collecting Process has to be ready for it ✨
9.3. Additional Considerations for UDP Collecting Processes 🎀
okay so a Transport Session for IPFIX Messages sent over UDP is defined from the Exporting Process's point of view 👀 It roughly matches the time when one Exporting Process is sending IPFIX Messages over UDP to one Collecting Process. The tea is that this is super hard for the Collecting Process to spot, so it's kinda giving guesswork 💀 That's why the Collecting Process MAY discard all Transport Session state once no IPFIX Messages have come in from a given Exporting Process within a given Transport Session for a configurable idle timeout ✨
The Collecting Process SHOULD accept Data Records even when it doesn't have the Template Record (or other definitions, like Common Properties) it needs to decode them 🫶 If those Template Records or definitions haven't shown up yet when the Data Records arrive, the Collecting Process MAY hold onto the Data Records for a short while and decode them once the Template Records or definitions finally land. While it's waiting, it compares the Export Times of the IPFIX Messages carrying the Template Records with the ones carrying the Data Records, as discussed in Section 8.2 🔥
Heads up, bestie: this trick can lead to incorrectly interpreted records if Template IDs get reused, or if other identifiers only last for a limited time 😭
10. Transport Protocol 🦋
The IPFIX Protocol Specification was designed to be transport protocol independent, so it doesn't care which transport you pick 💅 Also, the Exporter can export to multiple Collecting Processes using independent transport protocols, which is very much a main character move 👑
The IPFIX Message Header has a 16-bit Length field, which limits an IPFIX Message to 65535 octets, header included. A Collecting Process MUST be able to handle IPFIX Message lengths of up to 65535 octets 💖
Even though an Exporting Process or Collecting Process can support multiple transport protocols, Transport Sessions are bound to one transport protocol each 🧚 Transport Session state MUST NOT be migrated by an Exporting Process or Collecting Process among Transport Sessions that use different transport protocols between the same Exporting Process and Collecting Process pair. Basically, an Exporting Process that supports multiple transport protocols is conceptually multiple Exporting Processes, one per supported transport protocol 🌈 And the same goes for a Collecting Process supporting multiple transport protocols: it's conceptually multiple Collecting Processes, one per supported transport protocol 🍓
10.1. Transport Compliance and Transport Usage 🌸
SCTP [RFC4960] using the Partially Reliable SCTP (PR-SCTP) extension as specified in [RFC3758] MUST be implemented by all compliant implementations 🔥 UDP [UDP] MAY also be implemented by compliant implementations, and TCP [TCP] MAY also be implemented by compliant implementations too 💜
SCTP should be used in deployments where Exporters and Collectors are talking over links that are susceptible to congestion. With the PR-SCTP extension, SCTP is capable of providing any required degree of reliability ✨
TCP may be used in deployments where Exporters and Collectors communicate over links that are susceptible to congestion, but SCTP is preferred, because it can limit back pressure on Exporters and it has that message-versus-stream orientation 🫶
UDP may be used, but it isn't a congestion-aware protocol 😬 So in that case, the IPFIX traffic between the Exporter and Collector must be separately contained or provisioned to minimize the risk of congestion-related loss 💅
By default, the Collecting Process listens for connections on SCTP, TCP, and/or UDP port 4739 🎀 By default, it listens for secure connections on SCTP, TCP, and/or UDP port 4740 (refer to the Security Considerations section) 🔐 And by default, the Exporting Process attempts to connect to one of these ports. It MUST be possible to configure both the Exporting and Collecting Processes to use ports other than the defaults 🌷
10.2. SCTP 🐚
This section describes how IPFIX is transported over SCTP [RFC4960] using the PR-SCTP [RFC3758] extension ✨
10.2.1. Congestion Avoidance 🌊
SCTP provides the required level of congestion avoidance by design, which is honestly iconic 👑
SCTP detects congestion on the end-to-end path between the IPFIX Exporting Process and the IPFIX Collecting Process, and it limits the transfer rate to match 📉 When an IPFIX Exporting Process has records to export but notices that SCTP can't send right now, it has two options: wait until sending is possible again, or decide to drop the record. If it drops the record, the dropped export data SHOULD be accounted for, so the amount of dropped export data can be reported using the mechanism described in Section 4.3 🧾
10.2.2. Reliability 💌
The SCTP transport protocol is reliable by default, but it also has the capability to deliver messages with partial reliability [RFC3758] 🦋
Using reliable SCTP messages for IPFIX export doesn't, on its own, guarantee that every Data Record gets delivered 😭 If there's congestion on the link from the Exporting Process to the Collecting Process, or if a significant number of retransmissions are needed, the send queues on the Exporting Process may fill up. When that happens, the Exporting Process MAY suspend, export, or discard the IPFIX Messages 🫠 And if Data Records are discarded, the IPFIX Sequence Numbers used for export MUST reflect the loss of data 🔢
10.2.3. MTU 📏
SCTP provides the required IPFIX Message fragmentation service based on Path MTU (PMTU) discovery 🧩
10.2.4. Association Establishment and Shutdown 🤝
The IPFIX Exporting Process starts an SCTP association with the IPFIX Collecting Process 💖 The Exporting Process MAY establish more than one association (a connection "bundle", as SCTP calls it) to the Collecting Process 🧺
An Exporting Process MAY support more than one active association to different Collecting Processes, including the case where those Collecting Processes live on the same host 🏠
When an Exporting Process shuts down, it SHOULD shut down the SCTP association 👋
When a Collecting Process no longer wants to receive IPFIX Messages, it SHOULD shut down its end of the association. Even then, the Collecting Process SHOULD keep receiving and processing IPFIX Messages until the Exporting Process has closed its end of the association 💅
If a Collecting Process detects that the SCTP association has been abnormally terminated, it MUST continue to listen for a new association establishment 👂
If an Exporting Process detects that its SCTP association to the Collecting Process has been abnormally terminated, it SHOULD try to re-establish the association 🔁
Association timeouts SHOULD be configurable ⏱️
10.2.5. Failover 🛟
If the Collecting Process doesn't acknowledge an Exporting Process's attempt to establish an association, SCTP will automatically retry association establishment using exponential backoff 📈 The Exporter MAY log an alarm if the underlying SCTP association establishment times out, and this timeout should be configurable on the Exporter 🚨
The Exporting Process MAY open a backup SCTP association to a Collecting Process in advance, if it supports Collecting Process failover 🫶
10.2.6. Streams 🌊
An Exporting Process MAY request more than one SCTP Stream per association 🧵 Each of these streams may be used for transmitting IPFIX Messages that contain Data Sets, Template Sets, and/or Options Template Sets 🎀
Depending on what the application needs, the Exporting Process may send Data Sets with full or partial reliability, using ordered or out-of-order delivery, over any SCTP Stream established during SCTP association setup 🔀
An IPFIX Exporting Process MAY use any PR-SCTP service definition, as per Section 4 of the PR-SCTP specification [RFC3758], when using partial reliability to transmit IPFIX Messages that contain only Data Sets 🌟
However, Exporting Processes SHOULD mark such IPFIX Messages for retransmission for as long as resource or other constraints allow 🔁
10.3. UDP 🌙
This section describes how IPFIX is transported over UDP [UDP] ✨
10.3.1. Congestion Avoidance 🚦
UDP has no built-in congestion-avoidance mechanism 😬 So its use over congestion-sensitive network paths is not recommended. UDP MAY be used in deployments where Exporters and Collectors always communicate over dedicated links that are not susceptible to congestion, meaning links that are over-provisioned compared to the maximum export rate from the Exporters 🛣️
10.3.2. Reliability 📮
UDP is not a reliable transport protocol and cannot guarantee delivery of messages, so IPFIX Messages sent from the Exporting Process to the Collecting Process over UDP may get lost 💔 UDP MUST NOT be used unless the application can tolerate some loss of IPFIX Messages 🙅♀️
The Collecting Process SHOULD deduce the loss and reordering of IPFIX Data Records by looking at discontinuities in the IPFIX Sequence Number 🕵️♀️ With UDP, the IPFIX Sequence Number contains the total number of IPFIX Data Records sent for the Transport Session before this IPFIX Message was received, modulo 2^32 🔢 A Collector SHOULD detect out-of-sequence, dropped, or duplicate IPFIX Messages by tracking the Sequence Number 🎯
Exporting Processes that export IPFIX Messages via UDP MUST include a valid UDP checksum [UDP] in the UDP datagrams that carry IPFIX Messages ✅
10.3.3. MTU 📏
The maximum size of exported messages MUST be configured so that the total packet size does not exceed the PMTU 🧠 If the PMTU is unknown, a maximum packet size of 512 octets SHOULD be used 🌸
10.3.4. Session Establishment and Shutdown 🌅
UDP is connectionless, so there's no real session establishment or shutdown for IPFIX over UDP 🫥 An Exporting Process just starts sending IPFIX Messages to a Collecting Process at one point in time and stops at another. This can cause some complications in Template management, as outlined in Section 8.4 above 🙈
10.3.5. Failover and Session Duplication 👯
Because UDP isn't connection-oriented, the Exporting Process can't tell from the transport protocol that the Collecting Process has stopped being able to receive IPFIX Messages 😳 So it can't invoke a failover mechanism. However, the Exporting Process MAY duplicate the IPFIX Message to several Collecting Processes 👯♀️
10.4. TCP 🌷
This section describes how IPFIX is transported over TCP [TCP] 💜
10.4.1. Congestion Avoidance ✨
okay so TCP is basically the bestie that controls how fast data gets sent from the Exporting Process to the Collecting Process 🫶 It keeps an eye on congestion in the network AND on what the receiver can actually handle, which is so responsible of it 💅
So an IPFIX Exporting Process might not be able to send IPFIX Messages as fast as the Metering Process makes them 😭 That can happen because the network is congested, or because the Collecting Process just can't keep up. If the congestion is only temporary, the Exporting Process can buffer IPFIX Messages until it's clear to send them 🦋 But buffering has limits, both because resources are finite and because data has timeliness requirements. So if congestion is ongoing and/or severe, the Exporting Process can end up blocked 💀
When an Exporting Process has Data Records ready to export but its transmission buffer is full, and it wants to avoid getting blocked, it can choose to drop some Data Records 🌸 The dropped Data Records MUST be accounted for, so the number of lost records can be reported later, as described in Section 4.3 📊
10.4.2. Reliability 🎀
TCP ensures reliable delivery of data from the Exporting Process to the Collecting Process. Reliable is the vibe here, periodt 💖
10.4.3. MTU 🧚
okay so TCP is a stream service, not a datagram or sequential packet service, which means IPFIX Messages sent over TCP get separated using the Length field in the IPFIX Message Header 🍓 The Exporting Process can pick any valid length for the IPFIX Messages it exports, because TCP handles the segmentation for it ✨
Exporting Processes MAY choose IPFIX Message lengths lower than the maximum if they want to make sure Data Records get exported in a timely way ⏰
10.4.4. Connection Establishment and Shutdown 🌈
The IPFIX Exporting Process starts the TCP connection to the Collecting Process 🦋 An Exporting Process MAY support more than one active connection to different Collecting Processes, which includes the case where those Collecting Processes live on the same host 👑 An Exporting Process MAY also support more than one active connection to the same Collecting Process, to avoid head-of-line blocking across Observation Domains 🫶
The Exporter MAY log an alarm if the underlying TCP connection establishment times out. This timeout should be configurable on the Exporter ⚙️
When an Exporting Process shuts down, it SHOULD shut down the TCP connection 💅
When a Collecting Process no longer wants to receive IPFIX Messages, it SHOULD close its end of the connection. The Collecting Process SHOULD keep reading IPFIX Messages until the Exporting Process has closed its end 🙈
When a Collecting Process detects that the TCP connection to the Exporting Process has terminated abnormally, it MUST keep listening for a new connection 👀
When an Exporting Process detects that the TCP connection to the Collecting Process has terminated abnormally, it SHOULD try to re-establish the connection 🔁 Connection timeouts and retry schedules SHOULD be configurable. In the default configuration, an Exporting Process MUST NOT try to establish a connection more often than once per minute ⏳
10.4.5. Failover 🦋
If the Collecting Process doesn't acknowledge an attempt by the Exporting Process to establish a connection, TCP will automatically retry the connection establishment using exponential backoff 📈 The Exporter MAY log an alarm if the underlying TCP connection establishment times out. This timeout should be configurable on the Exporter 🚨
The Exporting Process MAY open a backup TCP connection to a Collecting Process in advance, if it supports Collecting Process failover 🛟
11. Security Considerations 🔐
The security considerations for the IPFIX protocol come from an analysis of potential security threats, which is discussed in the Security Considerations section of the IPFIX requirements document [RFC3917]. The requirements for IPFIX security are as follows:
- IPFIX must provide a mechanism to ensure the confidentiality of IPFIX data transferred from an Exporting Process to a Collecting Process, so that Flow Records transported via IPFIX can't be disclosed 🤫
- IPFIX must provide a mechanism to ensure the integrity of IPFIX data transferred from an Exporting Process to a Collecting Process. This prevents the injection of incorrect data or control information (e.g., Templates), and also the duplication of Messages, in an IPFIX Message stream 🛡️
- IPFIX must provide a mechanism to authenticate IPFIX Collecting and Exporting Processes, so that data can't be collected from an unauthorized Exporting Process and can't be exported to an unauthorized Collecting Process 🪪
Because IPFIX can be used to collect information for network forensics and billing, attacks designed to confuse, disable, or steal information from an IPFIX collection system may be seen as a prime objective during a sophisticated network attack 😱 This is serious, bestie 💀
An attacker who can inject false messages into an IPFIX Message stream can affect either the application using IPFIX (by falsifying data) or the IPFIX Collecting Process itself (by modifying or revoking Templates, or changing options). That's why IPFIX Message integrity matters so much 🚫
The IPFIX Messages themselves may also contain information that's valuable to an attacker. That includes information about the network configuration, plus end-user traffic and payload data, so care must be taken to keep their visibility limited to authorized users 👀 When an Information Element containing end-user payload information is exported, it SHOULD be transmitted to the Collecting Process using a means that protects its contents against eavesdropping 🕵️♀️ Suitable mechanisms include either a direct point-to-point connection that's assumed to be unavailable to attackers, or an encryption mechanism 🔒 It's the responsibility of the Collecting Process to provide a satisfactory degree of security for this collected data, including, if necessary, encryption and/or anonymization of any reported data; see Section 11.8 💖
11.1. Applicability of TLS and DTLS 🧁
Transport Layer Security (TLS) [RFC5246] and Datagram Transport Layer Security (DTLS) [RFC6347] were designed to provide the confidentiality, integrity, and authentication assurances that the IPFIX protocol requires, and they do this without needing pre-shared keys 🗝️
IPFIX Exporting Processes and Collecting Processes using TCP MUST support TLS version 1.1 and SHOULD support TLS version 1.2 [RFC5246], including the mandatory ciphersuite(s) specified in each version. IPFIX Exporting Processes and Collecting Processes using UDP or SCTP MUST support DTLS version 1.0 and SHOULD support DTLS version 1.2 [RFC6347], including the mandatory ciphersuite(s) specified in each version 🎀
Note that DTLS is selected as the security mechanism for SCTP. TLS bindings to SCTP are defined in [RFC3436], but they require that all communication be over reliable, bidirectional streams, and they also require one TLS connection per stream. That arrangement isn't compatible with the reason SCTP was chosen as an IPFIX transport protocol 🙅♀️
Note that using DTLS has a man-in-the-middle vulnerability that TLS doesn't have. It allows a message to be removed from the stream without the knowledge of either the sender or the receiver 😳 Also, when using DTLS over SCTP, an attacker could inject SCTP control information and shut down the SCTP association. That would cause a loss of IPFIX Messages if those messages are buffered outside of the SCTP association 💀 Techniques such as those described in [RFC6083] could be used to overcome these vulnerabilities 🛠️
When using DTLS over SCTP, the Exporting Process MUST make sure that each IPFIX Message is sent over the same SCTP Stream that would be used when sending that same IPFIX Message directly over SCTP. Note that DTLS may send its own control messages on stream 0 with full reliability. However, this won't interfere with the processing of stream 0 IPFIX Messages at the Collecting Process, because DTLS consumes its own control messages before passing IPFIX Messages up to the application layer 🧠
When using DTLS over SCTP or UDP, the Heartbeat Extension [RFC6520] SHOULD be used, especially on long-lived Transport Sessions, to make sure the association stays active 💓
Exporting and Collecting Processes MUST NOT request, offer, or use any version of the Secure Socket Layer (SSL), or any version of TLS prior to 1.1, because of known security vulnerabilities in earlier versions of TLS. See Appendix E of [RFC5246] for more information 🚫🔐
11.2. Usage 🧭
The IPFIX Exporting Process starts the communication with the IPFIX Collecting Process, and it acts as a TLS or DTLS client according to [RFC5246] and [RFC6347]. The IPFIX Collecting Process acts as a TLS or DTLS server 🤝 The DTLS client opens a secure connection on SCTP port 4740 of the DTLS server if SCTP is selected as the transport protocol. The TLS client opens a secure connection on TCP port 4740 of the TLS server if TCP is selected as the transport protocol. The DTLS client opens a secure connection on UDP port 4740 of the DTLS server if UDP is selected as the transport protocol 🌸
11.3. Mutual Authentication 🪞
When using TLS or DTLS, IPFIX Exporting Processes and IPFIX Collecting Processes SHOULD be identified by a certificate containing the DNS-ID, as discussed in Section 6.4 of [RFC6125]. The inclusion of Common Names (CN-IDs) in certificates that identify IPFIX Exporting Processes or Collecting Processes is NOT RECOMMENDED 🙅♀️
To prevent man-in-the-middle attacks from impostor Exporting or Collecting Processes, mutual authentication MUST be used for both TLS and DTLS. This stops the acceptance of data from an unauthorized Exporting Process and the export of data to an unauthorized Collecting Process 🤺 Exporting Processes MUST verify the reference identifiers of the Collecting Processes they export IPFIX Messages to, checking them against those stored in the certificates. Likewise, Collecting Processes MUST verify the reference identifiers of the Exporting Processes they receive IPFIX Messages from, checking them against those stored in the certificates. Exporting Processes MUST NOT export to non-verified Collecting Processes, and Collecting Processes MUST NOT accept IPFIX Messages from non-verified Exporting Processes 🚧
Exporting Processes and Collecting Processes MUST support verifying certificates against an explicitly authorized list of peer certificates identified by Common Name. They SHOULD also support verifying reference identifiers by matching the DNS-ID or CN-ID with a DNS lookup of the peer 🔍
IPFIX Exporting Processes and Collecting Processes MUST use non-NULL ciphersuites for authentication, integrity, and confidentiality 🔐✨
11.4. Protection against DoS Attacks 🛡️
Okay bestie, DoS attacks are so not it 😭 An attacker can try to take down an IPFIX collection system in two ways. Directly, by blasting huge amounts of traffic at a Collecting Process. Indirectly, by generating huge amounts of traffic for a Metering Process to measure.
Direct DoS can also be a state exhaustion move 🥀. That can happen at the transport layer (say, by opening a ton of pending connections) or inside the IPFIX Collecting Process itself (say, by sending Flow Records that are waiting on Template or scope information, or a massive pile of Options Template Records, etc.).
SCTP has a cookie-exchange mechanism built to defend against SCTP state exhaustion DoS attacks 🍓 TCP has the "SYN cookie" mechanism to mitigate state exhaustion, and SYN cookies SHOULD be used by any Collecting Process accepting TCP connections 💅 DTLS also has cookie exchange to protect against DTLS server state exhaustion ✨
Heads up, this part is important! There is no way to stop fake IPFIX Message processing (and state creation) for UDP and SCTP communication 💀 TLS and DTLS can obviously stop fake states from being created, but they are themselves prone to state exhaustion attacks 😩 So, Collector rate limiting SHOULD be used to protect TLS and DTLS (like limiting the number of new TLS or DTLS sessions per second to a sensible number) 🔥
IPFIX state exhaustion attacks can be mitigated by:
- limiting the rate at which new connections or associations will be opened by the Collecting Process
- limiting the rate at which IPFIX Messages will be accepted by the Collecting Process
- adaptively limiting the amount of state kept, particularly for records waiting for Templates
These rate and state limits MAY be provided by a Collecting Process, and if provided, the limits SHOULD be user configurable 🌸
Also, an IPFIX Collecting Process can eliminate the risk of state exhaustion attacks from untrusted nodes by requiring TLS or DTLS mutual authentication. That way, it only accepts IPFIX Messages from trusted sources 🫶
For indirect denial of service, how IPFIX behaves under overload depends on the transport protocol in use 🧚 With IPFIX over TCP, TCP congestion control makes the flow of IPFIX Messages back off and eventually stall, which blinds the IPFIX system 😭 SCTP is a little better, since some IPFIX Messages keep getting received by the Collecting Process thanks to SCTP's multiple streams and partial reliability features, which avoid head-of-line blocking. That might give some visibility into the attack 👀 UDP is kind of similar, since some datagrams may still reach the Collecting Process. That effectively applies sampling to the IPFIX Message stream, which means some information about the attack will be available 🦋
To minimize IPFIX Message loss under overload, some mechanism for service differentiation could be used to prioritize IPFIX traffic over other traffic on the same link 👑 Or, IPFIX Messages can be sent over a dedicated network. In that case, be sure the dedicated network can handle the expected peak IPFIX Message traffic 💜
11.5. When DTLS or TLS Is Not an Option 🙈
The use of DTLS or TLS might not be possible in some cases, because of performance issues or other operational concerns 😬
Without TLS or DTLS mutual authentication, IPFIX Exporting Processes and Collecting Processes can fall back on using IP source addresses to authenticate their peers 🫣 A policy of allocating Exporting Process and Collecting Process IP addresses from specified address ranges, plus using ingress filtering to prevent spoofing, can make this approach more useful. Again, completely segregating IPFIX traffic on a dedicated network, where possible, can improve security even further 🔒 In any case, open Collecting Processes (the ones that accept IPFIX Messages from any Exporting Process regardless of IP address or identity) are discouraged 🙅♀️
Modern TCP and SCTP implementations are resistant to blind insertion attacks (see [RFC4960] and [RFC6528]) ✨ UDP, however, offers no such protection. For this reason, IPFIX Message traffic transported via UDP and not secured via DTLS SHOULD be protected via segregation to a dedicated network 🌈
11.6. Logging an IPFIX Attack 📝
IPFIX Collecting Processes MUST detect potential IPFIX Message insertion or loss conditions by tracking the IPFIX Sequence Number. They SHOULD also provide a logging mechanism for reporting out-of-sequence messages 🧐 Note that an attacker may be able to exploit how out-of-sequence messages are handled at the Collecting Process, so care should be taken when handling these conditions. For example, a Collecting Process that simply resets the expected Sequence Number when it receives a later Sequence Number could be temporarily blinded by deliberate injection of later Sequence Numbers 😱
IPFIX Exporting and Collecting Processes SHOULD log any connection attempt that fails due to authentication failure. That includes being presented an unauthorized or mismatched certificate during TLS or DTLS mutual authentication, or a connection attempt from an unauthorized IP address when TLS or DTLS is not in use 🕵️♀️
IPFIX Exporting and Collecting Processes SHOULD detect and log any SCTP association reset or TCP connection reset 🪵
11.7. Securing the Collector 🏰
The security of the Collector and its implementation is important for overall security 💖 However, a complete set of security guidelines for Collector implementation is outside the scope of this document, so we'll leave that one alone 🙃
As IPFIX uses length-prefix encodings, Collector implementors should take care to detect inconsistent values that could impact IPFIX Message decoding, and make sure the Collector still operates properly when such inconsistent values show up 🧩
Specifically, IPFIX Message, Set, and variable-length Information Element lengths must be checked for consistency to avoid buffer-sizing vulnerabilities 🚨
Collector implementors should also pay special attention to UTF-8 encoding of string data types, since vulnerabilities may exist in how ill-formed UTF-8 values are interpreted. See Section 6.1.6 🎀
11.8. Privacy Considerations for Collected Data 🕊️
Flow data exported by Exporting Processes and collected by Collecting Processes typically contains information about traffic on the observed network 👀 That information may be personally identifiable and privacy-sensitive. The storage of this data must be protected with technical as well as policy means, so that the privacy of the users of the measured network is protected 🔐 A complete specification of those means is out of scope for this document. It depends on the application and the storage technology being used 🌷
12. Management Considerations 📊
[RFC6615] specifies a MIB module that defines managed objects for monitoring IPFIX Devices, including basic configuration. This MIB can be used to measure the impact of IPFIX export on the monitoring network. It contains tables covering:
- Transport Session
- Cache definition
- Observation Point definition
- Template and Options Template definition
- export features (failover, load-balancing, duplicate)
- export statistics per Process, Session, and Template
From an operational point of view, an important function of this MIB module is the Transport Session Statistical table. It holds the rate (in bytes per second) at which the Collector receives or the Exporter sends out IPFIX Messages 📈 Of particular interest to operations, the Transport Session Statistical table in Section 5.8.1 of this MIB module exposes the rate of collection or export of IPFIX Messages. That lets you measure the bandwidth used by IPFIX export 💅
[RFC6727] describes extensions to the IPFIX-SELECTOR-MIB module specified in [RFC6615]. It contains managed objects that provide information on applied packet selection functions and their parameters (filtering and sampling) 🦋
Since the IPFIX-SELECTOR-MIB [RFC6615] and PSAMP-MIB [RFC6727] modules only contain read-only objects, they cannot be used to configure IPFIX Devices 🙅♀️ [RFC6728] specifies a configuration data model for the IPFIX and PSAMP protocols, using the Network Configuration Protocol (NETCONF) 🛠️ This data model covers Selection Processes, Caches, Exporting Processes, and Collecting Processes on IPFIX and PSAMP Devices. It is defined using UML (Unified Modeling Language) class diagrams and formally specified using YANG. The configuration data is encoded in Extensible Markup Language (XML) 🌸
A few mechanisms specified alongside the IPFIX protocol can help monitor and reduce the bandwidth used for IPFIX Export 🌈
- a bandwidth-saving method for exporting redundant information in IPFIX [RFC5473]
- an efficient method for exporting bidirectional flows [RFC5103]
- a method for the definition and export of complex data structures [RFC6313]
Alternatively, PSAMP [RFC5474] can be used to export packets sampled by statistical and other methods. That may be useful for many monitoring areas that IPFIX is also suited for ✨ PSAMP also gives you control over the impact on the measured network through its sampling rate. The set of packet selection techniques (Sampling, Filtering, and hashing) standardized by PSAMP is described in [RFC5475]. PSAMP also defines an explicitly configurable export rate limit in Section 8.4 of [RFC5474] 🔥
13. IANA Considerations 🧾
IANA has updated the "IPFIX Information Elements" registry [IANA-IPFIX] so that all references that previously pointed to RFC 5101 now point to this document instead 💜
IPFIX Messages use two fields with assigned values. These are the IPFIX Version Number, which indicates which version of the IPFIX protocol was used to export an IPFIX Message, and the IPFIX Set ID, which indicates the type for each set of information within an IPFIX Message 🎀
The Information Elements used by IPFIX, and the sub-registries of Information Element values, are managed by IANA [IANA-IPFIX]. So are the Private Enterprise Numbers used by enterprise-specific Information Elements [IANA-PEN] 🏷️ This document makes no changes to these registries 🫶
The IPFIX Version Number value of 0x000a (10) is reserved for the IPFIX protocol specified in this document 🔢 Set ID values of 0 and 1 are not used, for historical reasons [RFC3954]. The Set ID value of 2 is reserved for the Template Set. The Set ID value of 3 is reserved for the Options Template Set. All other Set ID values from 4 to 255 are reserved for future use. Set ID values above 255 are used for Data Sets 💖
New assignments in either the "IPFIX Version Number" or "IPFIX Set IDs" sub-registries require a Standards Action [RFC5226], which means they are to be made via Standards Track RFCs approved by the IESG 🌟
Appendix A. IPFIX Encoding Examples ✨
This appendix isn't a normative reference, so it's all examples, no rules 🎀 It's where we walk through IPFIX encoding with real messages so you can see how the bits actually land 🦋
Okay so picture an IPFIX Message with four pieces: a Template Set, a Data Set with three Data Records, an Options Template Set, and another Data Set with two Data Records that belong to that previous Options Template Record 💅
IPFIX Message:
+--------+------------------------------------------. . .
| | +--------------+ +------------------+
|Message | | Template | | Data |
| Header | | Set | | Set | . . .
| | | (1 Template) | | (3 Data Records) |
| | +--------------+ +------------------+
+--------+------------------------------------------. . .
. . .-------------------------------------------+
+------------------+ +------------------+ |
| Options | | Data | |
. . . | Template Set | | Set | |
| (1 Template) | | (2 Data Records) | |
+------------------+ +------------------+ |
. . .-------------------------------------------+A.1. Message Header Example
Okay the Message Header is made up of these fields 🌸
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Version = 0x000a | Length = 152 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Export Time |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Sequence Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Observation Domain ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+A.2. Template Set Examples
A.2.1. Template Set Using IANA Information Elements
We want to report these Information Elements 💖
- IPv4 source IP address: sourceIPv4Address [IANA-IPFIX], with a length of 4 octets
- IPv4 destination IP address: destinationIPv4Address [IANA-IPFIX], with a length of 4 octets
- Next-hop IP address (IPv4): ipNextHopIPv4Address [IANA-IPFIX], with a length of 4 octets
- Number of packets of the Flow: packetDeltaCount [IANA-IPFIX], with a length of 4 octets
- Number of octets of the Flow: octetDeltaCount [IANA-IPFIX], with a length of 4 octets
So the Template Set ends up looking like this 🧚
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 2 | Length = 28 octets |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID 256 | Field Count = 5 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| sourceIPv4Address = 8 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| destinationIPv4Address = 12 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| ipNextHopIPv4Address = 15 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| packetDeltaCount = 2 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| octetDeltaCount = 1 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+A.2.2. Template Set Using Enterprise-Specific Information Elements
This time we want these Information Elements, and one of them is proprietary, so it's giving enterprise-specific energy 🔥
- IPv4 source IP address: sourceIPv4Address [IANA-IPFIX], with a length of 4 octets
- IPv4 destination IP address: destinationIPv4Address [IANA-IPFIX], with a length of 4 octets
- An enterprise-specific Information Element representing proprietary information, with a type of 15 and a length of 4 octets
- Number of packets of the Flow: packetDeltaCount [IANA-IPFIX], with a length of 4 octets
- Number of octets of the Flow: octetDeltaCount [IANA-IPFIX], with a length of 4 octets
And the Template Set comes out like this 👑
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 2 | Length = 32 octets |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID 257 | Field Count = 5 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| sourceIPv4Address = 8 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| destinationIPv4Address = 12 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|1| Information Element id. = 15| Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Enterprise number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| packetDeltaCount = 2 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|0| octetDeltaCount = 1 | Field Length = 4 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+A.3. Data Set Example
Here we report three Flow Records 🍓
Src IP Addr. | Dst IP Addr. | Next-Hop Addr. | Packet | Octets
| | | Number | Number
----------------------------------------------------------------
192.0.2.12 | 192.0.2.254 | 192.0.2.1 | 5009 | 5344385
192.0.2.27 | 192.0.2.23 | 192.0.2.2 | 748 | 388934
192.0.2.56 | 192.0.2.65 | 192.0.2.3 | 5 | 6534And the Data Set in wire format looks like this 🦋
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 256 | Length = 64 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 192.0.2.12 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 192.0.2.254 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 192.0.2.1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 5009 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 5344385 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 192.0.2.27 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 192.0.2.23 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 192.0.2.2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 748 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 388934 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 192.0.2.56 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 192.0.2.65 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 192.0.2.3 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 5 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 6534 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Note that padding is not necessary in this example.
A.4. Options Template Set Examples
A.4.1. Options Template Set Using IANA Information Elements
Per line card (the router has two line cards, so we're doubling the fun 🌈), we want to report these Information Elements:
- Total number of IPFIX Messages: exportedMessageTotalCount [IANA-IPFIX], with a length of 2 octets
- Total number of exported Flows: exportedFlowRecordTotalCount [IANA-IPFIX], with a length of 2 octets
The line card, which is represented by the lineCardId Information Element [IANA-IPFIX], is used as the Scope Field 🫶
Therefore, the Options Template Set will be:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 3 | Length = 24 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID 258 | Field Count = 3 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope Field Count = 1 |0| lineCardId = 141 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope 1 Field Length = 4 |0|exportedMessageTotalCount=41 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 2 |0|exportedFlowRecordTotalCo.=42|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 2 | Padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+A.4.2. Options Template Set Using Enterprise-Specific Information Elements
okay so for each line card (the router is basically two line cards stuck together 🖥️), we wanna report these Information Elements:
- Total number of IPFIX Messages: exportedMessageTotalCount [IANA-IPFIX], with a length of 2 octets
- An enterprise-specific number of exported Flows, with a type of 42 and a length of 4 octets
The line card, represented by the lineCardId Information Element [IANA-IPFIX], is the Scope Field. main character energy 💅
The format of the Options Template Set is as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 3 | Length = 28 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID 259 | Field Count = 3 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope Field Count = 1 |0| lineCardId = 141 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope 1 Field Length = 4 |0|exportedMessageTotalCount=41 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 2 |1|Information Element id. = 42 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 4 | Enterprise number ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
... Enterprise number | Padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+A.4.3. Options Template Set Using an Enterprise-Specific Scope
this time we wanna export the same stuff as in Appendix A.4.1, just with a twist 🦋
- Total number of IPFIX Messages: exportedMessageTotalCount [IANA-IPFIX], with a length of 2 octets
- Total number of exported Flows: exportedFlowRecordTotalCount [IANA-IPFIX], with a length of 2 octets
but now the info is about a proprietary scope, identified by enterprise-specific Information Element number 123 🧚
The format of the Options Template Set is now as follows:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 3 | Length = 28 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Template ID 260 | Field Count = 3 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope Field Count = 1 |1|Scope 1 Infor. El. id. = 123 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Scope 1 Field Length = 4 | Enterprise Number ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
... Enterprise Number |0|exportedMessageTotalCount=41 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 2 |0|exportedFlowRecordTotalCo.=42|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Length = 2 | Padding |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+A.4.4. Data Set Using an Enterprise-Specific Scope
now the fun part, the actual data 🍓 we're reporting these two Data Records:
Enterprise field 123 | IPFIX Message | Exported Flow Records
---------------------------------------------------------------
1 | 345 | 10201
2 | 690 | 20402 0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Set ID = 260 | Length = 20 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 345 | 10201 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 690 | 20402 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+A.5. Variable-Length Information Element Examples
A.5.1. Example of Variable-Length Information Element with Length Less Than 255 Octets
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 5 | 5-octet Information Element |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+A.5.2. Example of Variable-Length Information Element with 3-Octet Length Encoding
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 255 | 1000 | IE ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 1000-octet Information Element |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
: ... :
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... IE |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Normative References
[IANA-IPFIX]
IANA, "IP Flow Information Export (IPFIX) Entities",
<http://www.iana.org/assignments/ipfix/>.
[RFC1014] Sun Microsystems, Inc., "XDR: External Data Representation
Standard", RFC 1014, June 1987.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.
[RFC3436] Jungmaier, A., Rescorla, E., and M. Tuexen, "Transport
Layer Security over Stream Control Transmission Protocol",
RFC 3436, December 2002.
[RFC3629] Yergeau, F., "UTF-8, a transformation format of
ISO 10646", STD 63, RFC 3629, November 2003.
[RFC3758] Stewart, R., Ramalho, M., Xie, Q., Tuexen, M., and P.
Conrad, "Stream Control Transmission Protocol (SCTP)
Partial Reliability Extension", RFC 3758, May 2004.
[RFC4960] Stewart, R., Ed., "Stream Control Transmission Protocol",
RFC 4960, September 2007.
[RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing an
IANA Considerations Section in RFCs", BCP 26, RFC 5226,
May 2008.
[RFC5246] Dierks, T. and E. Rescorla, "The Transport Layer Security
(TLS) Protocol Version 1.2", RFC 5246, August 2008.
[RFC5905] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch,
"Network Time Protocol Version 4: Protocol and Algorithms
Specification", RFC 5905, June 2010.
[RFC6125] Saint-Andre, P. and J. Hodges, "Representation and
Verification of Domain-Based Application Service Identity
within Internet Public Key Infrastructure Using X.509
(PKIX) Certificates in the Context of Transport Layer
Security (TLS)", RFC 6125, March 2011.
[RFC6347] Rescorla, E. and N. Modadugu, "Datagram Transport Layer
Security Version 1.2", RFC 6347, January 2012.
[RFC6520] Seggelmann, R., Tuexen, M., and M. Williams, "Transport
Layer Security (TLS) and Datagram Transport Layer Security
(DTLS) Heartbeat Extension", RFC 6520, February 2012.
[RFC7012] Claise, B., Ed., and B. Trammell, Ed., "Information Model
for IP Flow Information Export (IPFIX)", RFC 7012,
September 2013.
[TCP] Postel, J., "Transmission Control Protocol", STD 7,
RFC 793, September 1981.
[UDP] Postel, J., "User Datagram Protocol", STD 6, RFC 768,
August 1980.Informative References ✨
okay so these are the "go read this if you want more" picks, bestie 📚 (no links, just the refs!)
[IEEE.754.2008]: Institute of Electrical and Electronics Engineers, "IEEE Standard for Floating-Point Arithmetic", IEEE Standard 754, August 2008.[IPFIX-MED-PROTO]: Claise, B., Kobayashi, A., and B. Trammell, "Operation of the IP Flow Information Export (IPFIX) Protocol on IPFIX Mediators", Work in Progress, July 2013.[IANA-PEN]: IANA, "Private Enterprise Numbers",<http://www.iana.org/assignments/enterprise-numbers/>.[POSIX.1]: IEEE 1003.1-2008, "IEEE Standard for Information Technology - Portable Operating System Interface (POSIX(R))", 2008.[RFC2579]: McCloghrie, K., Ed., Perkins, D., Ed., and J. Schoenwaelder, Ed., "Textual Conventions for SMIv2", STD 58, RFC 2579, April 1999.[RFC3550]: Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", STD 64, RFC 3550, July 2003.[RFC3917]: Quittek, J., Zseby, T., Claise, B., and S. Zander, "Requirements for IP Flow Information Export (IPFIX)", RFC 3917, October 2004.[RFC3954]: Claise, B., Ed., "Cisco Systems NetFlow Services Export Version 9", RFC 3954, October 2004.[RFC5101]: Claise, B., Ed., "Specification of the IP Flow Information Export (IPFIX) Protocol for the Exchange of IP Traffic Flow Information", RFC 5101, January 2008.[RFC5103]: Trammell, B. and E. Boschi, "Bidirectional Flow Export Using IP Flow Information Export (IPFIX)", RFC 5103, January 2008.[RFC5153]: Boschi, E., Mark, L., Quittek, J., Stiemerling, M., and P. Aitken, "IP Flow Information Export (IPFIX) Implementation Guidelines", RFC 5153, April 2008.[RFC5470]: Sadasivan, G., Brownlee, N., Claise, B., and J. Quittek, "Architecture for IP Flow Information Export", RFC 5470, March 2009.[RFC5471]: Schmoll, C., Aitken, P., and B. Claise, "Guidelines for IP Flow Information Export (IPFIX) Testing", RFC 5471, March 2009.[RFC5472]: Zseby, T., Boschi, E., Brownlee, N., and B. Claise, "IP Flow Information Export (IPFIX) Applicability", RFC 5472, March 2009.[RFC5473]: Boschi, E., Mark, L., and B. Claise, "Reducing Redundancy in IP Flow Information Export (IPFIX) and Packet Sampling (PSAMP) Reports", RFC 5473, March 2009.[RFC5474]: Duffield, N., Ed., Chiou, D., Claise, B., Greenberg, A., Grossglauser, M., and J. Rexford, "A Framework for Packet Selection and Reporting", RFC 5474, March 2009.[RFC5475]: Zseby, T., Molina, M., Duffield, N., Niccolini, S., and F. Raspall, "Sampling and Filtering Techniques for IP Packet Selection", RFC 5475, March 2009.[RFC5476]: Claise, B., Ed., Johnson, A., and J. Quittek, "Packet Sampling (PSAMP) Protocol Specifications", RFC 5476, March 2009.[RFC5477]: Dietz, T., Claise, B., Aitken, P., Dressler, F., and G. Carle, "Information Model for Packet Sampling Exports", RFC 5477, March 2009.[RFC5610]: Boschi, E., Trammell, B., Mark, L., and T. Zseby, "Exporting Type Information for IP Flow Information Export (IPFIX) Information Elements", RFC 5610, July 2009.[RFC5655]: Trammell, B., Boschi, E., Mark, L., Zseby, T., and A. Wagner, "Specification of the IP Flow Information Export (IPFIX) File Format", RFC 5655, October 2009.[RFC6083]: Tuexen, M., Seggelmann, R., and E. Rescorla, "Datagram Transport Layer Security (DTLS) for Stream Control Transmission Protocol (SCTP)", RFC 6083, January 2011.[RFC6183]: Kobayashi, A., Claise, B., Muenz, G., and K. Ishibashi, "IP Flow Information Export (IPFIX) Mediation: Framework", RFC 6183, April 2011.[RFC6313]: Claise, B., Dhandapani, G., Aitken, P., and S. Yates, "Export of Structured Data in IP Flow Information Export (IPFIX)", RFC 6313, July 2011.[RFC6526]: Claise, B., Aitken, P., Johnson, A., and G. Muenz, "IP Flow Information Export (IPFIX) Per Stream Control Transmission Protocol (SCTP) Stream", RFC 6526, March 2012.[RFC6528]: Gont, F. and S. Bellovin, "Defending against Sequence Number Attacks", RFC 6528, February 2012.[RFC6615]: Dietz, T., Ed., Kobayashi, A., Claise, B., and G. Muenz, "Definitions of Managed Objects for IP Flow Information Export", RFC 6615, June 2012.[RFC6727]: Dietz, T., Ed., Claise, B., and J. Quittek, "Definitions of Managed Objects for Packet Sampling", RFC 6727, October 2012.[RFC6728]: Muenz, G., Claise, B., and P. Aitken, "Configuration Data Model for the IP Flow Information Export (IPFIX) and Packet Sampling (PSAMP) Protocols", RFC 6728, October 2012.[UTF8-EXPLOIT]: Davis, M. and M. Suignard, "Unicode Technical Report #36: Unicode Security Considerations", The Unicode Consortium, July 2012.
Acknowledgments 🫶
big thank-you shoutout time 💖 Ganesh Sadasivan gets so much love for his huge help in the early days of the protocol spec 🌸 Juergen Quittek gets a crown 👑 for keeping IPFIX and PSAMP in sync, and Nevil Brownlee, Dave Plonka, and Andrew Johnson absolutely ate the thorough reviews 💅 Randall Stewart and Peter Lei are the SCTP experts we didn't know we needed 🦋 Martin Djernaes wrote the first essay on the SCTP section, Michael Behringer and Eric Vyncke brought the security wisdom ✨, and Michael Tuexen helped out with the DTLS section 🧚 Elisa Boschi made the SCTP sections so much better 🌈 Then there's Mark Fullmer, Sebastian Zander, Jeff Meyer, Maurizio Molina, Carter Bullard, Tal Givoly, Lutz Mark, David Moore, Robert Lowe, Paul Calato, Andrew Feren, Gerhard Muenz, Sue Hares, and many more, who all gave technical reviews and feedback 🙈 Last but definitely not least, a special mention to Adrian Farrel for his attention to management and operational aspects 🎀
Contributors 💅
these are the people who helped write the thing, so they get their own spotlight ✨
Stewart Bryant
Cisco Systems
10 New Square, Bedfont Lakes
Feltham, Middlesex TW18 8HA
United Kingdom
EMail: stbryant@cisco.comSimon Leinen
SWITCH
Werdstrasse 2
P.O. Box 8021
Zurich
Switzerland
Phone: +41 44 268 1536
EMail: simon.leinen@switch.chThomas Dietz
NEC Europe Ltd.
NEC Laboratories Europe
Network Research Division
Kurfuersten-Anlage 36
69115 Heidelberg
Germany
Phone: +49 6221 4342-128
EMail: Thomas.Dietz@nw.neclab.euAuthors' Addresses 🎀
and here's where you can find the authors themselves, bestie 🦋
Benoit Claise (editor)
Cisco Systems, Inc.
De Kleetlaan 6a b1
1831 Diegem
Belgium
Phone: +32 2 704 5622
EMail: bclaise@cisco.comBrian Trammell (editor)
Swiss Federal Institute of Technology Zurich
Gloriastrasse 35
8092 Zurich
Switzerland
Phone: +41 44 632 70 13
EMail: trammell@tik.ee.ethz.chPaul Aitken
Cisco Systems, Inc.
96 Commercial Quay
Commercial Street, Edinburgh EH6 6LX
United Kingdom
Phone: +44 131 561 3616
EMail: paitken@cisco.com📦 where this page comes from
- ietf: rfc7011 2013-09
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms