RFC 5180

IPv6 Benchmarking Methodology for Network Interconnect Devices

📅 published May 2008, from ietf

IPv6 Benchmarking Methodology for Network Interconnect Devices 🎀

okay bestie, this is RFC 5180 💖 and it's an Informational doc, so it's sharing knowledge and doesn't set any Internet standard. It's by C. Popoviciu, A. Hamza, and G. Van de Velde from Cisco Systems 🦋, plus D. Dugatkin from FastSoft Inc. 🫶 It came out in May 2008. The header doesn't list anything it obsoletes or updates, so it stands on its own ✨

Status of This Memo 🌸

this memo gives info to the Internet community. it does not specify an Internet standard of any kind, and distribution of it is unlimited 🍓

Abstract 🦋

so the benchmarking methodologies in RFC 2544 are IP version independent, which is iconic 👑 but RFC 2544 doesn't cover some of the IPv6 specificities. this doc adds extra benchmarking guidelines, and together with RFC 2544 they give a more complete and realistic evaluation of the IPv6 performance of network interconnect devices 💅 IPv6 transition mechanisms are outside the scope here, so don't look for them in this doc 🙈

Table of Contents 🧚

   1. Introduction ....................................................2
   2. Existing Definitions ............................................3
   3. Tests and Results Evaluation ....................................3
   4. Test Environment Setup ..........................................3
   5. Test Traffic ....................................................4
      5.1. Frame Formats and Sizes ....................................4
           5.1.1. Frame Sizes to Be Used on Ethernet ..................5
           5.1.2. Frame Sizes to Be Used on SONET .....................5
      5.2. Protocol Addresses Selection ...............................6
           5.2.1. DUT Protocol Addresses ..............................6
           5.2.2. Test Traffic Protocol Addresses .....................7
      5.3. Traffic with Extension Headers .............................7
      5.4. Traffic Setup ..............................................9
   6. Modifiers .......................................................9
      6.1. Management and Routing Traffic .............................9
      6.2. Filters ...................................................10
           6.2.1. Filter Format ......................................10
           6.2.2. Filter Types .......................................11
   7. Benchmarking Tests .............................................12
      7.1. Throughput ................................................13
      7.2. Latency ...................................................13
      7.3. Frame Loss ................................................13
      7.4. Back-to-Back Frames .......................................13
      7.5. System Recovery ...........................................14
      7.6. Reset .....................................................14
   8. IANA Considerations ............................................14
   9. Security Considerations ........................................14
   10. Conclusions ...................................................15
   11. Acknowledgements ..............................................15
   12. References ....................................................15
      12.1. Normative References .....................................15
      12.2. Informative References ...................................16
   Appendix A.  Theoretical Maximum Frame Rates Reference ............17
      A.1.  Ethernet .................................................17
      A.2.  Packet over SONET ........................................18

1. Introduction ✨

okay so the benchmarking methodologies from RFC 2544 [9] are proving super useful for consistently evaluating IPv4 forwarding performance of network elements 🔥 following these testing and result analysis procedures makes it easy to objectively compare IPv4 performance data measured on different products and by different people 🎀 the principles in RFC 2544 are mostly IP version independent, but the protocol has kept evolving, especially version 6 (IPv6) 🌈

this doc gives benchmarking methodology recommendations for the IPv6-specific stuff, like checking the forwarding performance of traffic with extension headers, as defined in RFC 2460 [2] 💖 these recs live inside the RFC 2544 framework and complement its test and result analysis procedures ✨

the terms in this doc stay consistent with the ones in "Benchmarking Terminology for Network Interconnect Devices", RFC 1242 [7] 📚 this terminology SHOULD be consulted before using or applying the recommendations here 🫶

any methodology aspects not covered in this doc SHOULD be assumed to be handled based on the RFC 2544 recommendations 💅

2. Existing Definitions 🦋

the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this doc are to be read as described in BCP 14, RFC 2119 [1] 🌸 RFC 2119 defines how these key words get used so the intent of standards track docs is as clear as possible 💜 this doc uses these key words too, but it is not a standards track document 🙈

3. Tests and Results Evaluation 🍓

the recommended performance evaluation tests are described in Section 7 of this document 🧚 not all of them apply to every network element type, but for each device you evaluate, it is recommended to perform as many of the applicable tests described in Section 6 as possible 🔥

test execution and results analysis MUST be performed while observing generally accepted testing practices about repeatability, variance, and statistical significance of small numbers of trials 💖

4. Test Environment Setup 🌈

the test environment setup options recommended for IPv6 performance evaluation are the same as the ones in Section 6 of RFC 2544, for both single-port and multi-port scenarios ✨ single-port testing measures per-interface forwarding performance, and multi-port testing measures how well forwarding performance scales across the whole platform 💅

throughout the test, the Device Under Test (DUT) can be monitored for relevant resource (processor, memory, etc.) utilization 👑 this data can be really helpful for understanding how each DUT processes traffic and which resources need to be allocated for IPv6 🎀 it can reveal whether IPv6 traffic gets processed in hardware, by the applicable devices, under all test conditions, or whether it gets punted to the software-switched path 🦋 if this data is of interest to you, it MUST be collected out of band and be independent of any management data collected through the interfaces forwarding the test traffic ⭐

Note: during testing, either static or dynamic options for neighbor discovery can be used 💀 in the static case, the IPv6 neighbor information for the test tool is manually configured on the DUT, and the IPv6 neighbor information for the DUT is manually configured on the test tool 🫶 in the dynamic case, each device dynamically discovers the IPv6 neighbor information through the neighbor discovery protocol 🌸 the static option can be used as long as the test tool supports it 🍓 the dynamic option is preferred, where the test tool interacts with the DUT for the whole duration of the test to keep the respective neighbor caches in an active state 💜 to avoid neighbor solicitation (NS) and neighbor advertisement (NA) storms caused by the neighbor unreachability detection (NUD) mechanism [6], the test scenarios assume that the test traffic simulates end points, and that the IPv6 source and destination addresses are one hop beyond the DUT 🧚

5. Test Traffic 💖

traffic used for all tests in this doc SHOULD meet the requirements described in this section 🔥 these requirements are designed to reflect the characteristics of IPv6 unicast traffic 🌈 using the recommended IPv6 traffic profile leads to a complete evaluation of the network element performance ✨

5.1. Frame Formats and Sizes 🎀

two types of media are commonly deployed, and each SHOULD be tested if the network element supports that type of media: Ethernet and SONET (Synchronous Optical Network) 🦋 this section identifies the frame sizes that SHOULD be used for each media type 🫶 for all other media types, refer to the recommendations in RFC 2544 📚

similar to IPv4, small frame sizes help characterize the per-frame processing overhead of the DUT 💅 note that the minimum IPv6 packet size (40 bytes) is larger than that of an IPv4 packet (20 bytes) 👑 tests should compensate for this difference ✨

the frame sizes listed for IPv6 include the extension headers used in testing (see Section 5.3) 🌸 by definition, extension headers are part of the IPv6 packet payload 🍓 depending on the total length of the extension headers, using them might not be possible at the smallest frame sizes 💜

Note: test tools commonly use signatures to identify test traffic packets, so they can verify that there are no packet drops or out-of-order packets, or calculate statistics like delay and jitter 🧚 this could be why the minimum frame size selectable through the test tool might not be as low as the theoretical one presented in this doc 🙈

5.1.1. Frame Sizes to Be Used on Ethernet 🌈

Ethernet, in all its types, has become the most commonly deployed media in today's networks 💖 the following frame sizes SHOULD be used for benchmarking over this media type: 64, 128, 256, 512, 1024, 1280, and 1518 bytes 🔥

Note: the recommended 1518-byte frame size is the maximum size of an untagged Ethernet frame 🦋 according to the IEEE 802.3as standard [13], the frame size can be increased up to 2048 bytes to accommodate frame tags and other encapsulation required by the IEEE 802.1AE MAC [14] security protocol 💀 a frame size commonly used in operational environments is 1522 bytes, the max length for a VLAN-tagged frame, per 802.1D [15] ✨

Note: while jumbo frames are outside the scope of the 802.3 IEEE standard, tests SHOULD be executed with frame sizes selected based on the values supported by the device under test 🫶 examples of commonly used jumbo frame sizes are: 4096, 8192, and 9216 bytes 👑

the maximum frame rates for each frame size and the various Ethernet interface types are provided in Appendix A 🧚

5.1.2. Frame Sizes to Be Used on SONET 🦋

Packet over SONET (PoS) interfaces are commonly used for edge uplinks and high-bandwidth core links 🎀 evaluating the forwarding performance of the PoS interfaces supported by the DUT is recommended 💅 the following frame sizes SHOULD be used for this media type: 47, 64, 128, 256, 512, 1024, 1280, 1518, 2048, 4096 bytes 🌸

the theoretical maximum frame rates for each frame size and the various PoS interface types are provided in Appendix A 🍓

5.2. Protocol Addresses Selection 💜

there are two aspects of IPv6 benchmarking testing where IP address selection considerations MUST be analyzed: the selection of IP addresses for the DUT, and the selection of addresses for test traffic 🫶

5.2.1. DUT Protocol Addresses ✨

okay so IANA reserved an IPv6 address block for IPv6 benchmark testing (see Section 8) 🦋 It MAY be assumed that addresses in this block are not globally routable, and they MUST NOT be used as Internet source or destination addresses 🚫 not on the internet, bestie, they're just for the lab 🧪

Similar to Appendix C of RFC 2544, addresses from the first half of this range SHOULD be used for the ports viewed as input, and addresses from the other half of the range for the output ports 🔀 input on one side, output on the other, very organized 💅

The prefix length of the IPv6 addresses configured on the DUT interfaces MUST fall into either of the following:

Prefix length is /126, which would simulate a point-to-point link for a core router. 🛣️
Prefix length is smaller or equal to /64. 🌸

No prefix lengths SHOULD be selected in the range between 64 and 128 except the 126 value mentioned above 🙅‍♀️ so skip the in-between ones, only /126 gets to be the special exception ✨

Note that /126 prefixes might not always be the best choice for addressing point-to-point links such as back-to-back Ethernet unless the auto-provisioning mechanism is disabled 🤔 Also, not all network elements support addresses of this prefix length 💀 some devices just aren't on that vibe.

While with IPv6, the DUT interfaces can be configured with multiple global unicast addresses, the methodology described in this document does not require testing such a scenario 🙈 It is not expected that such an evaluation would bring additional data regarding the performance of the network element 💤 so we can skip that one, no notes.

The Interface ID portion of global unicast IPv6 DUT addresses SHOULD be set to ::1 🧚 There are no requirements in the selection of the Interface ID portion of the link local IPv6 addresses 🫶 link local is a free-for-all.

It is recommended that multiple iterations of the benchmark tests be conducted using the following prefix lengths: 48, 64, 126, and 128, for the advertised traffic destination prefix 📈 Other prefix lengths can be used. However, the indicated range reflects major prefix boundaries expected to be present in IPv6 routing tables, and they should be representative to establish baseline performance metrics 🌈 these are the big-deal boundaries, so the baseline has to actually look like the real world 👑

5.2.2. Test Traffic Protocol Addresses 🎀

IPv6 source and destination addresses for the test streams SHOULD belong to the IPv6 range assigned by IANA, as defined in Section 8 💜 The source addresses SHOULD belong to one half of the range and the destination addresses to the other, reflecting the DUT interface IPv6 address selection. It's giving mirror image 🪞

Tests SHOULD first be executed with a single stream leveraging a single source-destination address pair 🧁 start simple, one pair, no drama. The tests SHOULD then be repeated with traffic using a random destination address in the corresponding range 🎲 If the network element prefix lookup capabilities are evaluated, the tests SHOULD focus on the IPv6 relevant prefix boundaries: 0-64, 126, and 128 🎯

Note: When statically defined neighbors are not used, the parameters affecting Neighbor Unreachability Detection should be consistently set. The IPv6 prefix-reachable time in the router advertisement SHOULD be set to 30 seconds ⏱️ keep your settings consistent, no mixed signals, ngl.

5.3. Traffic with Extension Headers 🧚

Extension headers are an intrinsic part of the IPv6 architecture [2] 💅 They show up in all kinds of real traffic, like fragmented traffic, mobile IP-based traffic, and authenticated and encrypted traffic 📦 For these reasons, all tests described in this document SHOULD be performed with both traffic that has no extension headers and traffic that has a set of extension headers. Extension header types can be selected from the following list [2] that reflects the recommended order of multiple extension headers in a packet:

Hop-by-hop header
Destination options header
Routing header
Fragment header
Authentication header
Encapsulating security payload (ESP) header
Destination options header
Mobility header

Since extension headers are an intrinsic part of the protocol and they fulfill different roles, benchmarking of traffic containing each extension header SHOULD be executed individually 🧩 one at a time, each gets its own moment ✨

The special processing rules for the hop-by-hop extension header require a specific benchmarking approach 🔥 Unlike other extension headers, this header must be processed by each node that forwards the traffic. Tests with traffic containing these extension header types will not measure the forwarding performance of the DUT, but rather its extension-header processing capability, which is dependent on the information contained in the extension headers 🙃 The concern is that this traffic, at high rates, could have a negative impact on the operational resources of the router, and it could be used as a security threat 💀 When benchmarking with traffic that contains the hop-by-hop extension header, the goal is not to measure throughput [9] as in the case of the other extension headers, but rather to evaluate the impact of such traffic on the router. In this case, traffic with the hop-by-hop extension headers should be sent at 1%, 10%, and 50% of the interface total bandwidth 📉 Device resources must be monitored at each traffic rate to determine the impact 👀 watch the router like it's your ex's story.

Tests with traffic containing each individual extension header MUST be complemented with tests containing a chain of two or more extension headers (the chain MUST NOT contain the hop-by-hop extension header) 🚫 This chain should also exclude the ESP [5] extension header, since traffic with an encrypted payload cannot be used in tests with modifiers such as filters based on upper-layer information (see Section 5) 🔐 Since the DUT is not analyzing the content of the extension headers, any combination of extension headers can be used in testing. The extension header chain recommended for testing is:

Routing header - 24-32 bytes
Destination options header - 8 bytes
Fragment header - 8 bytes

This is a real-life extension-header chain that would be found in an IPv6 packet between two mobile nodes exchanged over an optimized path that requires fragmentation 🛸 The listed extension headers' lengths represent test tool defaults. The total length of the extension header chain SHOULD be larger than 32 bytes 📏

Extension headers add extra bytes to the payload size of the IP packets, which MUST be factored in when used in testing at low frame sizes 🧮 Their presence will modify the minimum packet size used in testing. For direct comparison between the data obtained with traffic that has extension headers and with traffic that doesn't have them at low frame size, a common value SHOULD be selected for the smallest frame size of both types of traffic ⚖️ same starting line for both, fair and square 🏁

For most cases, the network elements ignore the extension headers when forwarding IPv6 traffic. For these reasons, it is likely the performance impact related to extension headers will be observed only when testing the DUT with traffic filters that contain matching conditions for the upper-layer protocol information 🕵️‍♀️ In those cases, the DUT MUST traverse the chain of extension headers, a process that could impact performance 🫠 that's the plot twist.

5.4. Traffic Setup 🌸

All tests recommended in this document SHOULD be performed with bi-directional traffic. For asymmetric situations, tests MAY be performed with uni-directional traffic for a more granular characterization of the DUT performance. In these cases, the bi-directional traffic testing would reveal only the lowest performance between the two directions ↔️ the slowest direction is the one that gets exposed, periodt.

All other traffic profile characteristics described in RFC 2544 SHOULD be applied in this testing as well. IPv6 multicast benchmarking is outside the scope of this document 🚪 not this party, sorry multicast 🙅‍♀️

6. Modifiers 🌈

RFC 2544 underlines the importance of evaluating the performance of network elements under certain operational conditions 👑 The conditions defined in Section 11 of RFC 2544 are common to IPv4 and IPv6, except that IPv6 does not employ layer 2 or 3 broadcast frames. IPv6 does not use layer 2 or layer 3 broadcasts. This section provides additional conditions that are specific to IPv6. The suite of tests recommended in this document SHOULD be first executed in the absence of these conditions and then repeated under each of these conditions separately 🧪 baseline first, then add the chaos one condition at a time 💅

6.1. Management and Routing Traffic 🛰️

The procedures defined in Sections 11.2 and 11.3 of RFC 2544 are applicable for IPv6 management and routing update frames as well 📬 same rules, new faces.

6.2. Filters 🔍

The filters defined in Section 11.4 of RFC 2544 apply to IPv6 benchmarking as well. The filter definitions must be expanded to include upper-layer protocol information matching in order to analyze the handling of traffic with extension headers, which are an important architectural component of IPv6 🧠 we gotta look deeper to see what's really going on 👀

6.2.1. Filter Format 🎀

The filter format defined in RFC 2544 is applicable to IPv6 as well, except that the source addresses (SA) and destination addresses (DA) are IPv6 addresses. In addition to these basic filters, the evaluation of IPv6 performance SHOULD analyze the correct filtering and handling of traffic with extension headers 🫶

While the intent is not to evaluate a platform's capability to process the various extension header types, the goal is to measure performance impact when the network element must parse through the extension headers to reach upper-layer information. In IPv6, routers do not have to parse through the extension headers (other than hop-by-hop) unless, for example, upper-layer information has to be analyzed due to filters 🧐 only digging through the headers when the filter forces it, got it.

To evaluate the network element handling of IPv6 traffic with extension headers, the definition of the filters must be extended to include conditions applied to upper-layer protocol information. The following filter format SHOULD be used for this type of evaluation:

[permit|deny]  [protocol] [SA] [DA]

where permit or deny indicates the action to allow or deny a packet through the interface the filter is applied to. The protocol field is defined as:

ipv6: any IP Version 6 traffic 🌐
tcp: Transmission Control Protocol 🧵
udp: User Datagram Protocol 🦋

and SA stands for the source address and DA for the destination address 💖

The upper-layer protocols listed above are a recommended selection. However, they do not represent an all-inclusive list of upper-layer protocols that could be used in defining filters 🌸 there are more protocols out there, bestie, these are just the starter pack. The filters described in these benchmarking recommendations apply to native IPv6 traffic and upper-layer protocols (tcp, udp) transported in native IPv6 packets ✨

6.2.2. Filter Types 🎀

okay so bestie, when we're checking performance with modifiers, RFC 2544 says we run two kinds of tests: one with a single filter and one with 25 filters 🧚 We'll use the IPv6 documentation prefix [11] 2001:DB8:: for all the examples, because it's the safe pretend-address zone for this stuff 💅

Here are the single-filter examples:

Filter for TCP traffic - permit tcp 2001:DB8::1 2001:DB8::2
Filter for UDP traffic - permit udp 2001:DB8::1 2001:DB8::2
Filter for IPv6 traffic - permit ipv6 2001:DB8::1 2001:DB8::2

For the single line filter case, the network element SHOULD verify that it permits all TCP/UDP/IPv6 traffic, with or without any number of extension headers, from IPv6 SA 2001:DB8::1 to IPv6 DA 2001:DB8::2, and denies all other traffic. The vibe is "only this one lane gets through" 🚦✨

Now the 25-filter version, which is giving main character energy 👑

deny tcp 2001:DB8:1::1 2001:DB8:1::2
deny tcp 2001:DB8:2::1 2001:DB8:2::2
deny tcp 2001:DB8:3::1 2001:DB8:3::2
...
deny tcp 2001:DB8:C::1 2001:DB8:C::2
permit tcp 2001:DB8:99::1 2001:DB8:99::2
deny tcp 2001:DB8:D::1 2001:DB8:D::2
deny tcp 2001:DB8:E::1 2001:DB8:E::2
...
deny tcp 2001:DB8:19::1 2001:DB8:19::2
deny ipv6 any any

The router SHOULD deny all traffic, with or without extension headers, except TCP traffic with SA 2001:DB8:99::1 and DA 2001:DB8:99::2. Only that one permitted flow survives the whole list 🫶

7. Benchmarking Tests ✨

okay this section is where we actually run the tests 🔥 This document recommends the same benchmarking tests described in RFC 2544, while observing the DUT setup and the traffic setup considerations described above. The following sections state the test types explicitly, and they only highlight the methodology differences that might exist compared with what's described in Section 26 of RFC 2544. Less re-reading for you, bestie 💖

IPv6 has its own quirks, especially how extension header processing works, so it needs extra benchmarking steps. The tests recommended by RFC 2544 MUST be repeated for IPv6 traffic without extension headers, and for IPv6 traffic with one or multiple extension headers. No skipping, no cap 🙅‍♀️

IPv6 is getting deployed into IPv4 environments, and the two protocols are going to live side by side for a long time. So network operators care a LOT about how platforms handle both types of traffic 😭 Device resources are shared between the two, so IPv6-enabled platforms should not end up with degraded IPv4 performance. That's why IPv6 benchmarking SHOULD be performed both as a stand-alone protocol and in the context of its coexistence with IPv4 🌈

The modifiers are independent of the extension header type, so you can apply them to each of the tests above. Very flexible, very slay 💅

The benchmarking tests in this section SHOULD be performed under each of the following conditions:

Extension header specific conditions:
- i) IPv6 traffic with no extension headers
- ii) IPv6 traffic with one extension header from the list in Section 5.3
- iii) IPv6 traffic with the chain of extension headers described in Section 5.3
Coexistence-specific conditions:
- iv) IPv4 ONLY traffic benchmarking
- v) IPv6 ONLY traffic benchmarking
- vi) IPv4-IPv6 traffic mix with the ratio 90% vs 10%
- vii) IPv4-IPv6 traffic mix with the ratio 50% vs 50%
- viii) IPv4-IPv6 traffic mix with the ratio 10% vs 90%

Put the IPv6-alone tests and the coexistence tests together and you get a really big coverage matrix 🦋 At a minimum, the coexistence tests should use IPv6 traffic with no extension headers, along with the 10%-90%, 90%-10%, or IPv4/IPv6 traffic mix. Start small, then level up 🍓

The subsequent sections each describe specific tests that MUST be executed under the conditions listed above for complete benchmarking of IPv6-forwarding performance. Non-negotiable, babe 🫶