RFC 5345

Simple Network Management Protocol (SNMP) Traffic Measurements and Trace Exchange Formats

๐Ÿ“… published October 2008, from ietf

๐ŸŽ€ RFC 5345: SNMP Traffic Measurements and Trace Exchange Formats ๐ŸŽ€

okay bestie, this one is by J. Schoenwaelder from Jacobs University Bremen ๐ŸŽ“, dated October 2008. It's titled Simple Network Management Protocol (SNMP) Traffic Measurements and Trace Exchange Formats โœจ. The category is Informational ๐Ÿ’œ, so it's giving "here's some useful info" rather than "this is a binding standard." It doesn't list any RFCs it obsoletes or updates in its header, so it's not replacing or changing anything else ๐Ÿ™Œ. It's from the Network Working Group, and it's stamped as a Request for Comments under number 5345 ๐Ÿฆ‹.

Status of This Memo ๐Ÿ“œ

okay so this memo just gives info to the Internet community ๐Ÿ’…. It does not specify an Internet standard of any kind ๐Ÿ™…โ€โ™€๏ธ. Distribution is totally unlimited, so share it with everyone ๐ŸŒธ.

IESG Note ๐Ÿงš

the IESG (the folks who keep the IETF running) wants you to know a few things ๐Ÿ’€. This work is related to IETF stuff in the Operations and Management Area about SNMP, but that doesn't stop it from being published ๐Ÿ‘€. It is not a candidate for any level of Internet Standard ๐Ÿšซ. The IETF doesn't vouch for whether this RFC is fit for any purpose, and the decision to publish wasn't based on IETF review, except for the IETF Last Call on IANA's allocation of a URI and the IESG check for conflicts with IETF work ๐Ÿ™ˆ. The RFC Editor picked to publish this at its own discretion ๐ŸŽ€. If you want more info, check out RFC 3932 ๐Ÿ”— (I'm not linking it, just sayin'!).

Abstract ๐Ÿ”ฅ

okay, the vibe: SNMP (Simple Network Management Protocol) is everywhere ๐ŸŒ. People use it to monitor, control, and sometimes configure network elements ๐Ÿ› ๏ธ. The tech itself is well documented, but literally nobody really knows how it gets used in practice or what typical usage patterns look like ๐Ÿคทโ€โ™€๏ธ.

this document describes a way to do large-scale SNMP traffic measurements so we can finally understand how SNMP is used in real-world production networks ๐Ÿ“Šโœจ. It covers the motivation, the measurement approach, and the tools and data formats you'd need to pull off a study like this ๐Ÿงฐ.

it was produced within the IRTF's Network Management Research Group (NMRG), and it represents the consensus of all of the active contributors to that group ๐Ÿซถ.

Table of Contents ๐Ÿ—‚๏ธ

1. Introduction ....................................................3
2. Measurement Approach ............................................4
   2.1. Capturing Traffic Traces ...................................5
   2.2. Converting Traffic Traces ..................................6
   2.3. Filtering Traffic Traces ...................................7
   2.4. Storing Traffic Traces .....................................7
   2.5. Analyzing Traffic Traces ...................................8
3. Analysis of Traffic Traces ......................................9
   3.1. Basic Statistics ...........................................9
   3.2. Periodic versus Aperiodic Traffic ..........................9
   3.3. Message Size and Latency Distributions .....................9
   3.4. Concurrency Levels ........................................10
   3.5. Table Retrieval Approaches ................................10
   3.6. Trap-Directed Polling - Myths or Reality? .................10
   3.7. Popular MIB Definitions ...................................11
   3.8. Usage of Obsolete Objects .................................11
   3.9. Encoding Length Distributions .............................11
   3.10. Counters and Discontinuities .............................11
   3.11. Spin Locks ...............................................12
   3.12. Row Creation .............................................12
4. Trace Exchange Formats .........................................12
   4.1. XML Representation ........................................12
   4.2. CSV Representation ........................................17
5. Security Considerations ........................................18
6. IANA Considerations ............................................19
7. Acknowledgements ...............................................19
8. References .....................................................20
   8.1. Normative References ......................................20
   8.2. Informative References ....................................20

1. Introduction ๐Ÿฆ‹

okay so SNMP first showed up in the late 1980s [RFC1052] ๐Ÿ•ฐ๏ธ, and it has evolved into what we now call the SNMP version 3 Framework (SNMPv3) [RFC3410] ๐Ÿ”. SNMP is super widely deployed, but it's kinda mysterious ๐Ÿ™ˆ. Nobody's sure which protocol versions people are actually running, which features they use, how usage differs between types of networks or organizations, what info gets queried the most, or what typical interaction patterns look like in real production networks ๐Ÿค”.

there's been a bunch of research lately about SNMP performance in general [SM99][Mal02][Pat01], the impact of SNMPv3 security [DSR01][CT04], and how SNMP stacks up against Web Services [PDMQ04][PFGL04] ๐Ÿ’ป. These papers are useful for understanding design decisions and technologies, BUT some of them are kinda shaky ๐Ÿ˜ฌ. The authors often just assume certain SNMP patterns without any experimental evidence that they're right ๐Ÿ™ƒ. There are tons of speculations about how SNMP gets used in the wild, and the performance comparisons are based on limited test cases. Literally no systematic measurements have been done and published so far ๐Ÿ’€.

a lot of authors start with the ifTable of the IF-MIB [RFC2863] or the tcpConnTable of the TCP-MIB [RFC4022] for their analysis and comparisons ๐Ÿ“ˆ. Here's the tea โ˜•: there's no evidence that operations on these tables dominate SNMP traffic ๐Ÿ‘€. It's even less clear how these tables actually get read, or which optimizations real-world apps do (or don't do) ๐Ÿคทโ€โ™€๏ธ. It's also unclear what the real traffic trade-off is between periodic polling and more aperiodic bulk data retrieval ๐Ÿ”„. And we don't really get how much traffic goes to standardized MIB objects versus proprietary ones ๐Ÿงฉ. We also don't know if the mix of operations differs across operational environments, like backbone networks, access networks, or enterprise networks ๐Ÿข.

this document recommends an approach for collecting, codifying, and handling SNMP traffic traces so we can start answering some of these questions ๐Ÿ“. It also describes the tools that have been developed so network operators can collect traffic traces and share them with research groups who want to analyze and model network management interactions ๐Ÿง‘โ€๐Ÿ’ป๐Ÿ’–.

the SNMP trace collection and analysis effort was started by the research community, but network operators get to benefit too ๐Ÿ™Œ. Several new tools are being built as part of this effort to capture and analyze the traffic that management stations generate ๐Ÿ› ๏ธ. The results can then be used to make management systems more efficient and scalable ๐Ÿš€.

the measurement approach in this document is, by design, limited to studying SNMP traffic ๐ŸŽฏ. Studying other management protocols, or how protocols like SNMP affect other traffic sharing the same network resources, is left for future efforts ๐Ÿ”ฎ.

this is an Informational document from the IRTF's Network Management Research Group (NMRG), and it represents the consensus of all of the active contributors to that group ๐Ÿ’œ.

2. Measurement Approach ๐Ÿ’…

okay, this section walks through how to actually do SNMP traffic measurements and analysis ๐Ÿ—บ๏ธ. The process has five basic steps ๐Ÿชœ:

  1. Capture raw SNMP traffic traces in pcap packet capture files [1] ๐Ÿ“ฅ.
  2. Convert the raw traffic traces into a structured format that's both machine and human readable ๐Ÿค–. A suitable XML schema has been developed for this, and it captures all SNMP message details ๐Ÿงพ. There's also a more compact comma-separated values (CSV) format that only keeps the key info about SNMP messages ๐Ÿชถ.
  3. Filter the converted traces to hide or anonymize sensitive info ๐Ÿ•ต๏ธโ€โ™€๏ธ. Filtering is conceptually its own step, but it may actually get implemented as part of the conversion step for efficiency โšก.
  4. Submit the filtered traces to a repository where they can be retrieved and analyzed ๐Ÿ—„๏ธ. That repository could be public, controlled by a research group, or controlled by a network operator who commits to running analysis scripts on the repository on behalf of researchers ๐Ÿค.
  5. Analyze the traces by creating and running analysis scripts that extract and aggregate info ๐Ÿ“Š.

a bunch of these steps need network operators to get involved in the SNMP measurement projects ๐Ÿซ‚. In lots of cases, the filtered XML and CSV versions of the SNMP traces are the interface between the researchers writing the analysis scripts and the operators doing the measurement work ๐ŸŒ‰. So it's super important to have a well-defined spec for these interfaces ๐Ÿ“œโœจ.

this section also gives some advice and concrete hints on how to carry out those steps efficiently ๐Ÿ’ก. Some special tools have been developed to help operators and researchers so they don't spend forever supporting SNMP measurement projects ๐Ÿงš. The following sections go into more detail on the five steps and some of the tools ๐ŸŽ€.

2.1. Capturing Traffic Traces ๐Ÿ“ธ

okay so first up, capturing SNMP traffic traces! you can do it with packet sniffers like tcpdump [2], wireshark [3], or similar apps ๐Ÿ’… but the filter has to be right. you want pcap filter expressions that match the SNMP transport endpoints, which is typically udp and (port 161 or port 162) ๐ŸŽ€

also you need full packets, so they can't get truncated. that's tcpdump -s 0 ๐Ÿ™ˆ and placement is a whole thing. on bridged LANs, all your management traffic has to get captured and the probe needs access to every VLAN carrying it. usually that means parking the probe near the management systems and setting up dedicated monitoring ports on the bridged networks. some bridges have monitoring restrictions too, so look into that and write it down if it matters ๐Ÿงš

bestie, don't skip the placement stuff. a perfect capture from the wrong spot is still a useless capture ๐Ÿ˜ญ

it's recommended to capture at least a full week of data so you catch diurnal patterns and one cycle of weekly behavior ๐ŸŒธ operators are strongly encouraged to go even longer ๐Ÿ‘‘ for merging or splitting pcap files, you can use tcpdump and tcpslice [2] or mergecap and editcap [3] โœจ

now the checksum drama ๐Ÿ’€ some OSes offload TCP/IP work, like calculating the transport layer checksum, onto the network interface card. so traces that include traffic to or from a capturing interface with offloading can show incorrect transport layer checksums. the easiest fix is to turn checksum offloading off while capturing, if you can do that without losing too many packets ๐Ÿ“ the other option is to correct or ignore the checksums when you convert the raw pcap files later.

raw pcap files should ideally live in permanent storage, like compressed and encrypted on a CD ROM or DVD ๐Ÿ’ฟ you might need to go back to them to verify measurements, for example if bugs in the tools below get found and fixed ๐Ÿ”ฅ

every captured trace should also come with some meta data that's recorded and made available ๐Ÿ“ that includes stuff like where it was collected (the name of the network, the name of the organization that owns it, and a description of the measurement point in the network topology), when it was collected, contact info, the size of the trace, any known special events, equipment failures, or major infrastructure changes during the collection period, and so on โœจ it's also super useful to give each trace a unique identification. services like DatCat [4] let you store meta data and hand out unique identifiers, so go check them out ๐Ÿฆ‹

2.2. Converting Traffic Traces ๐Ÿ”„

raw traces in pcap format must be converted into something that's human readable and still machine readable for efficient post-processing ๐ŸŒˆ human readability lets an operator check that no sensitive data is hiding in a trace, and machine readability lets you pull out the good stuff quickly ๐Ÿ’œ

the natural pick is XML, because it's readable by humans and machines, and there are tons of tools and high-level scripting language APIs to process it ๐ŸŽ€ but XML is pretty verbose, which means more processing overhead ๐Ÿ™ˆ so the usage of XML streaming APIs is strongly suggested, since APIs that need an in-memory representation of the whole XML document don't handle large traces well ๐Ÿ˜ญ

Section 4.1 of this document defines a RELAX NG schema [OASISRNG] for representing SNMP traffic traces in XML ๐Ÿ“œ the schema captures all the relevant details of an SNMP message. also, the XML format keeps some info about the original ASN.1/BER encoding so you can analyze message sizes ๐Ÿ”

if full XML feels like too much, there's a lightweight option: comma-separated values (CSV) ๐Ÿญ it's defined in Section 4.2 and only captures selected parts of SNMP messages, so it's more compact and faster to process โšก

as we covered earlier, analysis programs that process raw pcap files should have an option to ignore incorrect checksums caused by TCP/IP offloading ๐Ÿซถ they should also be able to do IP reassembly for SNMP messages that got fragmented at the IP layer ๐Ÿงฉ

the snmpdump [5] package was built to convert raw pcap files into XML and CSV format ๐ŸŒธ the snmpdump program reads pcap, XML, or CSV files as input and outputs XML or CSV files โœจ and specific elements can be filtered out as needed to protect sensitive data ๐Ÿ”

2.3. Filtering Traffic Traces ๐Ÿงน

filtering sensitive data, like access control lists or community strings, can be done by manipulating the XML representation of an SNMP trace ๐Ÿ’… standard XSLT processors, like xsltproc [6], work for this ๐ŸŽ€ if you're into Perl, you might like picking a suitable Perl module to manipulate XML documents [7] ๐Ÿช

the snmpdump program can filter out sensitive info too, for example by deleting or clearing all XML elements whose name matches a regular expression ๐Ÿงฝ data type specific anonymization transformations that keep lexicographic ordering for values appearing in instance identifiers [HS06] can also be applied ๐Ÿฆ‹

โš ๏ธ heads up: anonymization transformations are often bound to an initialization key and depend on the data being anonymized in a given run ๐Ÿ”‘ so users must be careful when they merge data from independently anonymized traces ๐Ÿ’€ for more on anonymization techniques, check out [XFA02], [FXAM04], [PAPL06], and [RW07] ๐Ÿ“š

2.4. Storing Traffic Traces ๐Ÿ—„๏ธ

the raw pcap traces AND the XML / CSV formatted traces should be stored in a stable archive or repository โœจ that might be run by research groups (like the NMRG), network operators, or both ๐ŸŒ it's of key importance that captured traces aren't lost or modified, since they can be the basis of future research and might be needed to verify published results ๐Ÿซถ access to the archive might be restricted to people who've signed some kind of non-disclosure agreement ๐Ÿ”’

okay but here's the plot twist ๐ŸŒช๏ธ while this document recommends keeping raw traces, there are situations where that might not be feasible ๐Ÿ˜ฌ the recommendation to keep raw traces may be ignored, for example, to comply with data-protection laws or to protect a network operator from being forced to hand the data over to other organizations ๐Ÿ›ก๏ธ

lossless compression programs like gzip or bzip2 can squish even large trace files down to a size where they can be burned on DVDs for cheap long-term storage ๐Ÿ’ฟโœจ

it must be stressed again: keep the original pcap traces in addition to the XML/CSV formatted ones ๐Ÿšจ the pcap traces are the most authentic source of information ๐Ÿ‘‘ and improvements in the tool chain may mean going back to the original pcap traces and rebuilding all the intermediate formats from them ๐Ÿ”

2.5. Analyzing Traffic Traces ๐Ÿ”ฌ

scripts that analyze traffic traces must be verified for correctness โœ… ideally, all the scripts used to analyze traces will be publicly accessible so third parties can verify them ๐ŸŒท sharing scripts also lets other people repeat an analysis on other traces and extend it ๐Ÿ” it might be useful to set up a common, versioning repository for analysis scripts ๐Ÿ“‚

because XML parsers exist and CSV is so simple, trace files can be processed with tools written in almost any programming language ๐ŸŒˆ however, to build a common vocabulary and let operators easily read the scripts they run on trace files, it's suggested that analysis scripts be written in scripting languages like Perl, using suitable Perl modules to manipulate XML documents <http://perl-xml.sourceforge.net/faq/> ๐Ÿช scripting languages like Perl save development time and make scripts more accessible to operators who want to verify them before running them on trace files that might contain sensitive data ๐Ÿซถ

the snmpdump tool provides an API to process SNMP messages in C/C++ ๐Ÿ’ป coding trace analysis programs in C/C++ should in general be avoided for readability, verifiability, and portability reasons ๐Ÿ˜ค but using C/C++ might be the only option for dealing with very large traces efficiently ๐Ÿ”ฅ

last but not least: any results from analyzing a trace must be interpreted in the context of that trace ๐Ÿง  the nature of the network, the attachment point used to collect the trace, the nature of the applications generating SNMP traffic, and the events that happened during collection all clearly influence the result ๐ŸŽญ so be careful about drawing general conclusions from a potentially (too) limited data set ๐Ÿ’…

3. Analysis of Traffic Traces โœจ

this section discusses several questions you can answer by analyzing SNMP traffic traces ๐Ÿฆ‹ the questions in the subsections below are meant to be illustrative, not a complete list ๐ŸŒธ

3.1. Basic Statistics ๐Ÿ“Š

basic statistics cover things like:

  • protocol version used,
  • protocol operations used,
  • message size distribution,
  • error message type frequency, or
  • usage of authentication and encryption mechanisms.

the Object Identifier (OID) names of the objects being manipulated can be categorized into OID subtrees, for example to identify 'standardized', 'proprietary', and 'experimental' objects ๐Ÿงฉ

3.2. Periodic versus Aperiodic Traffic โฐ

SNMP is used to periodically poll devices, and also to retrieve info at the request of an operator or application ๐Ÿ” the periodic polling leads to periodic traffic patterns, while on-demand retrieval causes more aperiodic ones ๐ŸŽฒ it's worthwhile to understand the relationship between the amount of periodic and aperiodic traffic ๐Ÿ’– it'll be super interesting to see whether there are multiple levels of periodicity at different time scales ๐ŸŒˆ

periodic polling behavior may depend on the application and the polling engine it uses ๐Ÿงš for example, some management platforms let applications specify how long polled values may be kept in a cache before they're polled again ๐Ÿ—ƒ๏ธ such optimizations need to be considered when analyzing traces for periodic and aperiodic traffic ๐Ÿ‘‘

3.3. Message Size and Latency Distributions ๐ŸŽ€

okay so SNMP messages have size limits, bestie ๐Ÿ“ those come from the transport mappings and the buffers the SNMP engines have. ๐Ÿงš For the framework to keep evolving, it would be so helpful to know what the actual message size distributions look like in the wild. ๐Ÿ’… Latency distributions matter too, especially how long SNMP command responders take to process things. โฑ๏ธ Some implementations estimate networking delay by timing request-response pairs, and we'd want to know how well that actually works ๐Ÿ‘€

Also, some SNMP implementations update their counters from the underlying instrumentation using adaptive algorithms. ๐Ÿฆ‹ The updates aren't necessarily periodic, and they aren't necessarily on-demand either, which is a whole mood ๐Ÿ™ˆ That granularity of internal counter updates can change latency measurements, so it needs to be taken into account ๐Ÿ’–

3.4. Concurrency Levels ๐ŸŒธ

SNMP lets management stations retrieve information from multiple agents at the same time ๐Ÿ“ It'll be so interesting to figure out what typical concurrency levels look like on production networks. ๐Ÿ‘€ Or do management apps actually prefer to grab their data more sequentially? ๐Ÿค”

And it's also worth analyzing how many redundant requests from applications get processed by a device at almost the same moment. โœจ Concurrency level and the number of redundant requests both have implications for the caching strategies SNMP agents use, so this is lowkey important ๐Ÿซถ

3.5. Table Retrieval Approaches ๐Ÿงš

Tables can be read in a bunch of different ways, bestie ๐Ÿ“š The simplest (and honestly most inefficient ๐Ÿ’€) approach is to grab tables object-by-object in column-by-column order. ๐Ÿ™ƒ More advanced approaches read tables row-by-row, or even multiple-rows-by-multiple-rows. ๐Ÿ”ฅ Retrieval of index elements can be suppressed in most cases, and sometimes only a subset of a table's columns gets retrieved. โœจ

It would be super useful to know which of these approaches are actually used on production networks, because that has a direct impact on agent implementation techniques and caching strategies. ๐Ÿ’…

3.6. Trap-Directed Polling - Myths or Reality? ๐Ÿ”ฎ

okay so SNMP is built around something called trap-directed polling ๐ŸŒˆ Management apps are responsible for periodically polling SNMP agents to find out their status. ๐Ÿ“ก But SNMP agents can also send traps to notify managers about events, so managers can adjust their polling strategy and basically react faster than regular polling would allow. โšก

Analyzing SNMP traffic traces can reveal whether trap-directed polling is actually deployed. ๐Ÿ•ต๏ธโ€โ™€๏ธ The big question is whether SNMP notifications lead to changes in the short-term polling behavior of management stations. ๐Ÿค” It should also be investigated to what extent SNMP managers use automated procedures to track down the meaning of the event conveyed by an SNMP notification. ๐Ÿซฃ Spill the tea! โ˜•

An analysis of object identifier prefixes can pinpoint the most popular MIB modules, plus the most important object types or notification types defined by those modules. ๐ŸŽฏ That info would be so valuable for maintaining and developing these MIB modules and related ones. ๐Ÿงก

3.8. Usage of Obsolete Objects ๐Ÿ•ฐ๏ธ

Several objects from the early days have been obsoleted because they just can't represent today's networks anymore ๐Ÿ˜ญ A classic example is the ipRouteTable, which got obsoleted because it couldn't represent classless routing, the thing that was introduced and deployed on the Internet in 1993. ๐Ÿฆ– Some of these obsolete objects still show up in popular publications and research papers, which is so iconic of them ๐Ÿ™ˆ It'll be interesting to find out whether management applications still use them, or whether apps have been updated to use the replacement objects instead. ๐Ÿ”„

Depending on what's recorded in a trace, it might be possible to figure out the age of devices by looking at the values of objects like sysObjectID and sysDecr [RFC3418]. ๐Ÿ” That age can then be taken into consideration when analyzing the use of obsolete and deprecated objects. ๐Ÿง

3.9. Encoding Length Distributions ๐Ÿ“

It'll be useful to understand encoding length distributions for various data types, bestie ๐Ÿ’– Assumptions about these distributions are sometimes used to estimate SNMP message sizes so they can meet transport and buffer size constraints. ๐Ÿงฉ

3.10. Counters and Discontinuities ๐Ÿซง

Counters can experience discontinuities [RFC2578]. ๐Ÿ˜ฌ A widely used discontinuity indicator is the sysUpTime scalar of the SNMPv2-MIB [RFC3418], which can be reset through a warm start to indicate counter discontinuities. ๐Ÿ” Some MIB modules introduce more specific discontinuity indicators, e.g., the ifCounterDiscontinuityTime of the IF-MIB [RFC2863]. โœจ It'll be so interesting to study how much these objects are actually used by management applications to handle discontinuity events. ๐Ÿ‘‘

3.11. Spin Locks ๐Ÿ”’

Cooperating command generators can use advisory locks to coordinate their usage of SNMP write operations. ๐Ÿค The snmpSetSerialNo scalar of the SNMPv2-MIB [RFC3418] is the default coarse-grain coordination object. ๐ŸŽ€ It'll be interesting to find out whether there are command generators that actually coordinate themselves using these spin locks. ๐Ÿ‘€

3.12. Row Creation ๐ŸŒฑ

Row creation isn't natively supported by the protocol operations, which is kinda wild ๐Ÿ™ƒ Instead, conceptual tables that support row creation typically provide a control column that uses the RowStatus textual convention defined in the SNMPv2-TC [RFC2579] module. ๐Ÿ“‹ RowStatus itself supports different row creation modes, namely createAndWait (dribble-mode) and createAndGo (one-shot mode). ๐Ÿญ Different approaches can be used to derive the instance identifier if it doesn't have special semantics associated with it. ๐Ÿง 

It'll be useful to study which of the various row creation approaches are actually used by management applications on production networks. ๐Ÿ’…


4. Trace Exchange Formats ๐Ÿงš

4.1. XML Representation ๐Ÿซถ

The XML format was designed to keep all the information associated with SNMP messages, bestie โœจ The format is specified in RELAX NG compact notation [OASISRNC]. ๐Ÿ“œ Freely available tools such as trang [8] can convert RELAX NG compact syntax to other XML schema notations. ๐Ÿ”ง

The XML format can represent SNMPv1, SNMPv2c, and SNMPv3 messages ๐Ÿ’œ In case a new version of SNMP is introduced in the future, or existing versions get extended in ways that require changes to the XML format, a new XML format with a different namespace needs to be defined. ๐ŸŒˆ For example, you could increment the version number included in the namespace URI. ๐Ÿ”ข

Here's the full RELAX NG grammar, verbatim, because it's giving precision ๐Ÿ’…

# Relax NG grammar for the XML SNMP trace format.
#
# Published as part of RFC 5345.

default namespace = "urn:ietf:params:xml:ns:snmp-trace-1.0"

start =
  element snmptrace {
    packet.elem*
  }

packet.elem =
  element packet {
    element time-sec  { xsd:unsignedInt },
    element time-usec { xsd:unsignedInt },
    element src-ip    { ipaddress.type },
    element src-port  { xsd:unsignedInt },
    element dst-ip    { ipaddress.type },
    element dst-port  { xsd:unsignedInt },
    snmp.elem
  }

snmp.elem =
  element snmp {
    length.attrs?,
    message.elem
  }

message.elem =
  element version   { length.attrs, xsd:int },
  element community { length.attrs, xsd:hexBinary },
  pdu.elem

message.elem |=
  element version { length.attrs, xsd:int },
  element message {
    length.attrs,
    element msg-id         { length.attrs, xsd:unsignedInt },
    element max-size       { length.attrs, xsd:unsignedInt },
    element flags          { length.attrs, xsd:hexBinary },
    element security-model { length.attrs, xsd:unsignedInt }
  },
  usm.elem?,
  element scoped-pdu {
    length.attrs,
    element context-engine-id { length.attrs, xsd:hexBinary },
    element context-name      { length.attrs, xsd:string },
    pdu.elem
  }

usm.elem =
  element usm {

    length.attrs,
    element auth-engine-id    { length.attrs, xsd:hexBinary },
    element auth-engine-boots { length.attrs, xsd:unsignedInt },
    element auth-engine-time  { length.attrs, xsd:unsignedInt },
    element user              { length.attrs, xsd:hexBinary },
    element auth-params       { length.attrs, xsd:hexBinary },
    element priv-params       { length.attrs, xsd:hexBinary }
  }

pdu.elem =
  element trap {
    length.attrs,
    element enterprise        { length.attrs, oid.type },
    element agent-addr        { length.attrs, ipv4address.type },
    element generic-trap      { length.attrs, xsd:int },
    element specific-trap     { length.attrs, xsd:int },
    element time-stamp        { length.attrs, xsd:int },
    element variable-bindings { length.attrs, varbind.elem* }
  }

pdu.elem |=
  element (get-request | get-next-request | get-bulk-request |
           set-request | inform-request | snmpV2-trap |
           response | report) {
    length.attrs,
    element request-id        { length.attrs, xsd:int },
    element error-status      { length.attrs, xsd:int },
    element error-index       { length.attrs, xsd:int },
    element variable-bindings { length.attrs, varbind.elem* }
  }

varbind.elem =
  element varbind { length.attrs, name.elem, value.elem }

name.elem =
  element name { length.attrs, oid.type }

value.elem =
  element null              { length.attrs, empty } |
  element integer32         { length.attrs, xsd:int } |
  element unsigned32        { length.attrs, xsd:unsignedInt } |
  element counter32         { length.attrs, xsd:unsignedInt } |
  element counter64         { length.attrs, xsd:unsignedLong } |
  element timeticks         { length.attrs, xsd:unsignedInt } |
  element ipaddress         { length.attrs, ipv4address.type } |
  element octet-string      { length.attrs, xsd:hexBinary } |
  element object-identifier { length.attrs, oid.type } |
  element opaque            { length.attrs, xsd:hexBinary } |
  element no-such-object    { length.attrs, empty } |
  element no-such-instance  { length.attrs, empty } |
  element end-of-mib-view   { length.attrs, empty }

# The blen attribute indicates the number of octets used by the BER
# encoded tag / length / value triple.  The vlen attribute indicates
# the number of octets used by the BER encoded value alone.

length.attrs =
  ( attribute blen { xsd:unsignedShort },
    attribute vlen { xsd:unsignedShort } )?

oid.type =
  xsd:string {
    pattern =
      "(([0-1](\.[1-3]?[0-9]))|(2.(0|([1-9]\d*))))" ~
      "(\.(0|([1-9]\d*))){0,126}"
  }

# The types below are for IP addresses.  Note that SNMP's buildin
# IpAddress type only supports IPv4 addresses; IPv6 addresses are only
# introduced to cover SNMP over IPv6 endpoints.

ipv4address.type =
  xsd:string {
    pattern =
      "((0|(1[0-9]{0,2})" ~
      "|(2(([0-4][0-9]?)|(5[0-5]?)|([6-9]?)))|([3-9][0-9]?))\.){3}" ~
      "(0|(1[0-9]{0,2})" ~
      "|(2(([0-4][0-9]?)|(5[0-5]?)|([6-9]?)))|([3-9][0-9]?))"
  }

ipv6address.type =
  xsd:string {
    pattern =
      "(([0-9a-fA-F]+:){7}[0-9a-fA-F]+)|" ~
      "(([0-9a-fA-F]+:)*[0-9a-fA-F]+)?::(([0-9a-fA-F]+:)*[0-9a-fA-F]+)?"
  }
ipaddress.type = ipv4address.type | ipv6address.type

okay so this bit is the grammar wrap-up ๐Ÿ’… now here's the example! ๐Ÿงš It's an SNMP trace file in XML, showing an SNMPv1 get-next-request for the OID 1.3.6.1.2.1.1.3 (sysUpTime) and the response the agent sends back. Main character energy โœจ

<snmptrace xmlns="urn:ietf:params:xml:ns:snmp-trace-1.0">
  <packet>
    <time-sec>1147212206</time-sec>
    <time-usec>739609</time-usec>
    <src-ip>192.0.2.1</src-ip>
    <src-port>60371</src-port>
    <dst-ip>192.0.2.2</dst-ip>
    <dst-port>12345</dst-port>
    <snmp blen="42" vlen="40">
      <version blen="3" vlen="1">1</version>
      <community blen="8" vlen="6">7075626c6963</community>
      <get-next-request blen="29" vlen="27">
        <request-id blen="6" vlen="4">1804289383</request-id>
        <error-status blen="3" vlen="1">0</error-status>
        <error-index blen="3" vlen="1">0</error-index>
        <variable-bindings blen="15" vlen="13">
          <varbind blen="13" vlen="11">
            <name blen="9" vlen="7">1.3.6.1.2.1.1.3</name>
            <null blen="2" vlen="0"/>
          </varbind>
        </variable-bindings>
      </get-next-request>
    </snmp>
  </packet>
  <packet>
    <time-sec>1147212206</time-sec>
    <time-usec>762891</time-usec>
    <src-ip>192.0.2.2</src-ip>
    <src-port>12345</src-port>
    <dst-ip>192.0.2.1</dst-ip>
    <dst-port>60371</dst-port>
    <snmp blen="47" vlen="45">
      <version blen="3" vlen="1">1</version>
      <community blen="8" vlen="6">7075626c6963</community>
      <response blen="34" vlen="32">
        <request-id blen="6" vlen="4">1804289383</request-id>
        <error-status blen="3" vlen="1">0</error-status>
        <error-index blen="3" vlen="1">0</error-index>
        <variable-bindings blen="20" vlen="18">
          <varbind blen="18" vlen="16">
            <name blen="10" vlen="8">1.3.6.1.2.1.1.3.0</name>
            <unsigned32 blen="6" vlen="4">26842224</unsigned32>
          </varbind>
        </variable-bindings>
      </response>
    </snmp>
  </packet>
</snmptrace>

4.2. CSV Representation ๐Ÿฆ‹

Okay so the CSV format (comma-separated values) is the lightweight option ๐ŸŒธ It's built to capture only the most relevant info about an SNMP message. If you need literally all the information about an SNMP message, the XML format above must be used ๐Ÿ’€ Here are the CSV fields, in order:

  1. Timestamp in the format seconds.microseconds since 1970, for example, "1137764769.425484".
  2. Source IP address in dotted quad notation (IPv4), for example, "192.0.2.1", or compact hexadecimal notation (IPv6), for example, "2001:DB8::1".
  3. Source port number represented as a decimal number, for example, "4242".
  4. Destination IP address in dotted quad notation (IPv4), for example, "192.0.2.1", or compact hexadecimal notation (IPv6), for example, "2001:DB8::1".
  5. Destination port number represented as a decimal number, for example, "161".
  6. Size of the SNMP message (a decimal number) counted in octets, for example, "123". The size excludes all transport, network, and link-layer headers.
  7. SNMP message version represented as a decimal number. The version 0 stands for SNMPv1, 1 for SNMPv2c, and 3 for SNMPv3, for example, "3".
  8. SNMP protocol operation indicated by one of the keywords get-request, get-next-request, get-bulk-request, set-request, trap, snmpV2-trap, inform-request, response, report.
  9. SNMP request-id in decimal notation, for example, "1511411010".
  10. SNMP error-status in decimal notation, for example, "0".
  11. SNMP error-index in decimal notation, for example, "0".
  12. Number of variable-bindings contained in the varbind-list in decimal notation, for example, "5".
  13. For each varbind in the varbind list, three output elements are generated:
    1. Object name given as object identifier in dotted decimal notation, for example, "1.3.6.1.2.1.1.3.0".
    2. Object base type name or exception name, that is one of the following: null, integer32, unsigned32, counter32, counter64, timeticks, ipaddress, octet-string, object-identifier, opaque, no-such-object, no-such-instance, and end-of-mib-view.
    3. Object value is printed as a number if the underlying base type is numeric. An IPv4 address is rendered in the dotted quad notation and an IPv6 address is rendered in the usual hexadecimal notation. An octet string value is printed in hexadecimal format while an object identifier value is printed in dotted decimal notation. In case of an exception, the object value is empty.

Heads up, bestie: the format does not preserve the information needed to understand SNMPv1 traps ๐Ÿ˜ญ It is therefore recommended that implementations be able to convert the SNMPv1 trap format into the trap format used by SNMPv2c and SNMPv3, according to the rules defined in [RFC3584]. The activation of trap format conversion should be the user's choice ๐Ÿซถ

The following example shows an SNMP trace file in CSV format containing an SNMPv1 get-next-request message for the OID 1.3.6.1.2.1.1.3 (sysUpTime) and the response message returned by the agent. (Note that the example uses backslash line continuation marks in order to fit the example into the RFC format. Backslash line continuations are not part of the CSV format.) ๐Ÿ™ˆ

1147212206.739609,192.0.2.1,60371,192.0.2.2,12345,42,1,\
  get-next-request,1804289383,0,0,1,1.3.6.1.2.1.1.3,null,
1147212206.762891,192.0.2.2,12345,192.0.2.1,60371,47,1,\
  response,1804289383,0,0,1,1.3.6.1.2.1.1.3.0,timeticks,26842224

5. Security Considerations ๐Ÿ’€

Okay this section is serious, so listen up ๐Ÿ”ฅ SNMP traffic traces usually contain sensitive information. It is therefore necessary to (a) remove unwanted information and (b) to anonymize the remaining necessary information before traces are made available for analysis. It is recommended to encrypt traces when they are archived.

Implementations that generate CSV or XML traces from raw pcap files should have an option to suppress or anonymize values. Note that instance identifiers of tables also include values, and it might therefore be necessary to suppress or anonymize (parts of) the instance identifiers. Similarly, the packet and message headers typically contain sensitive information about the source and destination of SNMP messages as well as authentication information (community strings or user names).

Anonymization techniques can be applied to keep information in traces that could otherwise reveal sensitive information. When using anonymization, values should only be kept when the underlying data type is known and an appropriate anonymization transformation is available (filter-in principle). For values appearing in instance identifiers, it is usually desirable to maintain the lexicographic order. Special anonymization transformations that preserve this property have been developed, although their anonymization strength is usually reduced compared to transformations that do not preserve lexicographic ordering [HS06].

Big warning ๐Ÿšจ The meta data associated with traces, and in particular information about the organization owning a network and the description of the measurement point in the network topology where a trace was collected, may be misused to decide/pinpoint where and how to attack a network. Meta data therefore needs to be properly protected ๐Ÿ”’


6. IANA Considerations ๐ŸŒˆ

Per this document, IANA has registered a URI for the SNMP XML trace format namespace in the IETF XML registry [RFC3688]. Following the format in RFC 3688, the following registration has been made:

URI: "urn:ietf:params:xml:ns:snmp-trace-1.0"

Registrant Contact: The NMRG of the IRTF.

XML: N/A, the URI is an XML namespace.

7. Acknowledgements ๐Ÿซถ

This document was influenced by discussions within the Network Management Research Group (NMRG). Special thanks to Remco van de Meent for writing the initial Perl script that led to the development of the snmpdump software package, and Matus Harvan for his work on lexicographic order preserving anonymization transformations. Aiko Pras contributed ideas to Section 3, while David Harrington helped to improve the readability of this document.

Last call reviews have been received from Bert Wijnen, Aiko Pras, Frank Strauss, Remco van de Meent, Giorgio Nunzi, Wes Hardacker, Liam Fallon, Sharon Chisholm, David Perkins, Deep Medhi, Randy Bush, David Harrington, Dan Romascanu, Luca Deri, and Marc Burgess. Karen R. Sollins reviewed the document for the Internet Research Steering Group (IRSG). Jari Arkko, Pasi Eronen, Chris Newman, and Tim Polk provided helpful comments during the Internet Engineering Steering Group (IESG) review. โœจ

Part of this work was funded by the European Commission under grant FP6-2004-IST-4-EMANICS-026854-NOE. ๐Ÿ’œ


8. References ๐Ÿ“š

8.1. Normative References

  • [RFC2578] McCloghrie, K., Perkins, D., and J. Schoenwaelder, "Structure of Management Information Version 2 (SMIv2)", STD 58, RFC 2578, April 1999.
  • [OASISRNG] Clark, J. and M. Makoto, "RELAX NG Specification", OASIS Committee Specification, December 2001.
  • [OASISRNC] Clark, J., "RELAX NG Compact Syntax", OASIS Committee Specification, November 2002.
  • [RFC3584] Frye, R., Levi, D., Routhier, S., and B. Wijnen, "Coexistence between Version 1, Version 2, and Version 3 of the Internet-standard Network Management Framework", BCP 74, RFC 3584, August 2003.
  • [RFC3688] Mealling, M., "The IETF XML Registry", BCP 81, RFC 3688, January 2004.

8.2. Informative References โœจ

Okay, now the bonus reading list ๐Ÿ“š These are the extra references the doc points to for background, and they're giving research-paper realness ๐Ÿค“

  • [RFC1052] Cerf, V., "IAB Recommendations for the development of Internet network management standards", RFC 1052, April 1998. ๐ŸŒธ
  • [RFC2579] McCloghrie, K., Perkins, D., and J. Schoenwaelder, "Textual Conventions for SMIv2", STD 58, RFC 2579, April 1999. ๐Ÿ’œ
  • [RFC3418] Presuhn, R., Ed., "Management Information Base (MIB) for the Simple Network Management Protocol (SNMP)", STD 62, RFC 3418, December 2002. โœจ
  • [RFC2863] McCloghrie, K. and F. Kastenholz, "The Interfaces Group MIB", RFC 2863, June 2000. ๐Ÿฆ‹
  • [RFC3410] Case, J., Mundy, R., Partain, D., and B. Stewart, "Introduction and Applicability Statements for Internet-Standard Management Framework", RFC 3410, December 2002. ๐ŸŽ€
  • [RFC4022] Raghunarayan, R., "Management Information Base for the Transmission Control Protocol (TCP)", RFC 4022, March 2005. ๐Ÿ“
  • [PDMQ04] Pras, A., Drevers, T., van de Meent, R., and D. Quartel, "Comparing the Performance of SNMP and Web Services based Management", IEEE Transactions on Network and Service Management 1(2), November 2004. ๐Ÿ”ฅ
  • [Pat01] Pattinson, C., "A Study of the Behaviour of the Simple Network Management Protocol", Proc. 12th IFIP/IEEE Workshop on Distributed Systems: Operations and Management , October 2001. ๐Ÿงš
  • [DSR01] Du, X., Shayman, M., and M. Rozenblit, "Implementation and Performance Analysis of SNMP on a TLS/TCP Base", Proc. 7th IFIP/IEEE International Symposium on Integrated Network Management , May 2001. ๐Ÿซถ
  • [CT04] Corrente, A. and L. Tura, "Security Performance Analysis of SNMPv3 with Respect to SNMPv2c", Proc. 2004 IEEE/IFIP Network Operations and Management Symposium , April 2004. ๐Ÿ”
  • [PFGL04] Pavlou, G., Flegkas, P., Gouveris, S., and A. Liotta, "On Management Technologies and the Potential of Web Services", IEEE Communications Magazine 42(7), July 2004. ๐ŸŒˆ
  • [SM99] Sprenkels, R. and J. Martin-Flatin, "Bulk Transfers of MIB Data", Simple Times 7(1), March 1999. ๐Ÿ’…
  • [Mal02] Malowidzki, M., "GetBulk Worth Fixing", Simple Times 10(1), December 2002. ๐Ÿ’–
  • [HS06] Harvan, M. and J. Schoenwaelder, "Prefix- and Lexicographical-order-preserving IP Address Anonymization", IEEE/IFIP Network Operations and Management Symposium NOMS 2006, April 2006. ๐ŸŒธ
  • [XFA02] Xu, J., Fan, J., and M. Ammar, "Prefix-Preserving IP Address Anonymization: Measurement-based Security Evaluation and a New Cryptography-based Scheme", 10th IEEE International Conference on Network Protocols ICNP'02, November 2002. ๐Ÿฆ‹
  • [FXAM04] Fan, J., Xu, J., Ammar, M., and S. Moon, "Prefix-Preserving IP Address Anonymization", Computer Networks 46(2), October 2004. ๐ŸŽ€
  • [PAPL06] Pang, R., Allman, M., Paxson, V., and J. Lee, "The Devil and Packet Trace Anonymization", Computer Communication Review 36(1), January 2006. ๐Ÿ“
  • [RW07] Ramaswamy, R. and T. Wolf, "High-Speed Prefix-Preserving IP Address Anonymization for Passive Measurement Systems", IEEE Transactions on Networking 15(1), February 2007. ๐Ÿ‘‘

URIs

Here's the link list the references point at, bestie ๐Ÿ”—

Author's Address

Meet the author, the one and only Juergen Schoenwaelder ๐Ÿ‘‘ Here's how to reach them:

Juergen Schoenwaelder
Jacobs University Bremen
Campus Ring 1
28725 Bremen
Germany

Phone: +49 421 200-3587
EMail: j.schoenwaelder@jacobs-university.de

Now for the legal bit, which we absolutely cannot skip ๐Ÿซถ Here it is in full:

Copyright (C) The IETF Trust (2008).

This document is subject to the rights, licenses and restrictions contained in BCP 78 and at http://www.rfc-editor.org/copyright.html, and except as set forth therein, the authors retain all their rights.

This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY, THE IETF TRUST AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. ๐Ÿ˜ญ

Intellectual Property

Last but not least, the IPR stuff ๐Ÿ’ผ

The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; nor does it represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79.

Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr.

The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org. ๐ŸŒˆ

๐Ÿ“ฆ where this page comes from