RFC 2432

Terminology for IP Multicast Benchmarking

📅 published October 1998, from ietf

RFC 2432: Terminology for IP Multicast Benchmarking ✨

Okay bestie, this one is RFC 2432, Terminology for IP Multicast Benchmarking 🦋. It's from the Network Working Group and it was written by K. Dubray of IronBridge Networks. It's marked Informational, dated October 1998. It doesn't list any RFCs it obsoletes or updates, so it's standing alone 💅.

Status of this Memo 📜

This memo provides information for the Internet community. It does not specify an Internet standard of any kind. Distribution of this memo is unlimited.

Copyright (C) The Internet Society (1998). All Rights Reserved.

Abstract 🌸

Okay so the whole point of this doc is to define terminology for benchmarking multicast IP forwarding devices 🧚. It's built on top of RFC 1242, RFC 2285, and other IETF Benchmarking Methodology Working Group (BMWG) efforts, and it's basically extending all that good work into the multicast world ✨.

The BMWG makes two main kinds of documents. Terminology documents present the benchmarks and related terms. Methodology documents define the procedures needed to collect the benchmarks named in the matching Terminology documents. Think of it as the dictionary and the recipe book working together 🍓.

1. Introduction ✨

Network forwarding devices are being asked to take one frame and deliver it to a whole bunch of destinations that belong to a particular group 🫶. That means multicast can put a different kind of strain on these devices' resources than unicast or broadcast traffic does.

The strain isn't always obvious at first glance. The only visible difference might be the Class D address on an IP multicast packet, compared to an IP unicast packet 🙈. But lots of factors can change how multicast packets get treated.

Here's a good question to ask: how does a device's architecture change the way it handles a multicast frame? Is a multicast packet processed the same way as its unicast cousin? Or is it treated as an exception and sent down a different data path? 💀 Also think about how a shared memory architecture might have a different performance profile than an architecture that explicitly passes each packet between processing entities. Totally different vibes 👑.

Forwarding architecture isn't the only thing that matters. Protocol requirements can make routers and switches look at both destination and source addresses when they decide how to forward multicast 😭. Capturing that source/destination addressing info can make the forwarding table bigger and slow down lookups. Topology stuff matters too: how much packet replication is happening, how many multicast groups the system is supporting, or whether multicast packets are wrapped inside unicast packets so they can cross non-multicast network paths. All of these can affect a system's multicast performance 🌈. If you want the full overall picture of IP multicasting, the document points you to [Se98], [Hu95], and [Mt98] 📚.

By nailing down IP multicast benchmarks and related terminology here, the hope is that detailed methodologies can be written in later documents 💜. Put together, these two efforts are trying to help make the characterization of multicast technologies and their individual implementations clinical, empirical, and consistent ✨. Knowing how a multicast forwarding device actually behaves can help a network designer deploy multicast more effectively in their own environment 🦋.

Also, this document is focused on profiling one source sending to many destinations 🎀. Some parts might need extending when you're looking at multiple sources sending to multiple destinations, so keep that in mind 👀.

2. Definition Format 🎀

This section uses the template suggested by RFC 1242 for defining a term. Here's the template:

Term to be defined.

   Definition:
      The specific definition for the term.

   Discussion:
      A brief discussion of the term, its application, or other
      information that would build understanding.

   Measurement units:
      Units used to record measurements of this term, if applicable.

   [Issues:]
      List of issues or conditions that affect this term. This field can
      present items the may impact the term's related methodology or
      otherwise restrict its measurement procedures.  This field is
      optional in this document.

   [See Also:]
      List of other terms that are relevant to the discussion of this
      term. This field is optional in this document.

2.1 Existing Terminology 📖

This document borrows terminology that's already been defined in other BMWG work 💖. The examples include, but aren't limited to, the following:

Throughput                [RFC 1242, section 3.17]
Latency                   [RFC 1242, section 3.8]
Constant Load             [RFC 1242, section 3.4]
Frame Loss Rate           [RFC 1242, section 3.6]
Overhead behavior         [RFC 1242, section 3.11]
Forwarding Rates          [RFC 2285, section 3.6]
Loads                     [RFC 2285, section 3.5]
Device Under Test (DUT)   [RFC 2285, section 3.1.1]
System Under Test (SUT)   [RFC 2285, section 3.1.2]

Note: "DUT/SUT" is a metric that may apply to a DUT or SUT. 🙃

3. Table of Defined Terms 💅

Here's the lineup of everything defined in this doc, grouped by category:

   3.1 General Nomenclature

     3.1.1 Traffic Class. (TC)
     3.1.2 Group Class. (GC)
     3.1.3 Service Class. (SC)

   3.2 Forwarding and Throughput
     3.2.1 Mixed Class Throughput (MCT).
     3.2.2 Scaled Group Forwarding Matrix (SGFM).
     3.2.3 Aggregated Multicast Throughput (AMT)
     3.2.4 Encapsulation Throughput (ET)
     3.2.5 Decapsulation Throughput (DT)
     3.2.6 Re-encapsulation Throughput (RET)

   3.3 Forwarding Latency
     3.3.1 Multicast Latency (ML)
     3.3.2 Min/Max Multicast Latency (Min/Max ML)

   3.4 Overhead
     3.4.1 Group Join Delay. (GJD)
     3.4.2 Group Leave Delay. (GLD)

   3.5 Capacity
     3.5.1 Multicast Group Capacity. (MGC)

   3.6 Interaction
     3.6.1 Burdened Response
     3.6.2 Forwarding Burdened Multicast Latency (FBML)
     3.6.3 Forwarding Burdened Join Delay (FBJD)

3.1 General Nomenclature 🌷

This section covers the general terminology that this doc and other docs will use 🧚.

3.1.1 Traffic Class. (TC) 🌈

Definition: An equivalence class of packets made up of one or more data streams.

Discussion: For this document, a Traffic Class is treated as a logical identifier. It's used to tell apart the set or sets of packets offered to the DUT 🎀.

For example, one Traffic Class could be the set of unicast packets offered to the DUT. Another Traffic Class could separate out the multicast packets going to multicast group X. And yet another Class could pick out the multicast packets going to multicast group Y 🦋.

Unless it's specified otherwise, when this document says "Class," it just means a Traffic Class ✨.

Measurement units: Not applicable.

3.1.2 Group Class. (GC) 💜

Definition: A specific type of Traffic Class where the packets in the Class are all going to one particular multicast group.

Discussion:

Measurement units: Not applicable.

3.1.3 Service Class. (SC) 👑

Definition: A specific type of Traffic Class where the packets in the Class need particular treatment, or treatments, from the network forwarding devices along the path to the packets' destination(s) 💅.

Discussion:

Measurement units: Not applicable.

3.2 Forwarding and Throughput. 🚀

This section covers the terms for measuring how well a DUT/SUT can forward packets in a multicast environment 🫶. Some of these metrics stretch the idea of throughput from RFC 1242. The idea of Forwarding Rate comes from RFC 2285 💖.

3.2.1 Mixed Class Throughput (MCT). 🍓

Definition: The maximum rate at which none of the offered frames get dropped by the device across a fixed number of ports. The offered frames are a mix of a unicast Class and a multicast Class, and they're the ones meant to be forwarded 🌸.

Discussion: Throughput usually gets measured on one kind of traffic. The load offered to the DUT is either all unicast or all multicast 🙃. But in most real networks, the traffic mix is rarely that evenly split 😭.

Using the RFC 1242 definition of throughput as the base, Mixed Class Throughput tries to show how well the DUT can handle unicast and multicast frames together in the same aggregated traffic stream ✨.

Measurement units: Frames per second

Issues: The related methodology may need to deal with the ratio of unicast packets to multicast packets 🦋.

Frame size can sometimes make a difference in frame forwarding benchmarks, so the matching methodology for this metric will need to consider the frame size distribution(s) 🎀.

3.2.2 Scaled Group Forwarding Matrix (SGFM). 🌈

Definition: A table that shows Forwarding Rate as a function of the multicast groups tested, for a fixed number of tested DUT/SUT ports 🧚.

Discussion: Scaling is a really desirable quality for lots of Internet mechanisms 💖. This benchmark is about showing how well a SUT can keep forwarding as the number of multicast groups goes up ✨.

Measurement units: Packets per second, along with the tested multicast group and port configurations 💜.

Issues: The methodology may need to reflect how the (source, group) pairing affects many multicast routing protocols 🦋.

Frame size can sometimes be a factor in frame forwarding benchmarks, so the matching methodology for this metric will need to consider the frame size distribution(s) 🌸.

3.2.3 Aggregated Multicast Throughput (AMT) 👑

Definition: The maximum rate at which none of the offered frames get dropped when they're forwarded through N destination interfaces of the same multicast group 💅.

Discussion: This is another "scaling" kind of exercise. It's designed to find out how well the DUT/SUT can handle traffic depending on how many multicast destination ports it has to support 🫶.

Measurement units: The ordered pair (N,t), where:

   N = the number of destination ports of the multicast group.
   t = the throughput, in frames per second, relative to the
   source stream.

Issues: Frame size can sometimes be a factor in frame forwarding benchmarks, so the matching methodology for this metric will need to consider the frame size distribution(s) 🎀.

3.2.4 Encapsulation Throughput (ET) 🎀

Definition: okay so this is the max rate at which frames offered to a DUT get encapsulated and correctly forwarded by the DUT without any loss 🦋

Discussion: bestie, so a super popular trick is wrapping up a packet that has a feature the device doesn't support in a format that the device does support, aka encapsulating or tunneling it. It's giving sneaky but make it make sense 💅

More specifically, encapsulation is when you take a frame, or part of a frame, and tuck it inside the payload of another frame. This benchmark is all about measuring the overhead of that translation step, so we can see what it costs ✨

Measurement units: Frames per second.

Issues: we need to think about how different frame formats affect usable bandwidth, ngl that's a whole thing 🌸

Also, frame size can sometimes matter in frame forwarding benchmarks, so the methodology for this metric will need to look at frame size distribution(s) 📊

3.2.5 Decapsulation Throughput (DT) 🌈

Definition: the max rate at which frames offered to a DUT get decapsulated and correctly forwarded by the DUT without loss. Iconic 👑

Discussion: same tunneling energy as before 💖 Like, a device might not support a feature, so the packet gets encapsulated in a format it does support. But at some point the frame needs to go back to its original format so the next destination can use it. That's where decapsulation comes in 🧚

More specifically, decapsulation is taking a frame, or part of a frame, that's embedded as the payload of another frame and returning it to the payload's proper format. This benchmark tries to characterize the overhead of that translation process, and it's giving main character energy because it's the reverse of the last one 🦋

Measurement units: Frames per second.

Issues: same drama as encapsulation. We need to consider how different frame formats impact usable bandwidth 💜

Frame size can also be a factor in frame forwarding benchmarks, so the methodology will need to consider frame size distribution(s) 🍓

3.2.6 Re-encapsulation Throughput (RET) 💅

Definition: the max rate at which frames in one encapsulated format offered to a DUT get converted into another encapsulated format and correctly forwarded by the DUT without loss 🔥

Discussion: okay one more tunneling moment, I promise 😭 A device might not support a protocol feature, so we wrap the packet in a format it does support. Then at some point, the frame might need to be converted from one encapsulation format to a different one 🫶

More specifically, re-encapsulation is taking an encapsulated payload in one format and swapping it for another encapsulated format, while keeping the original payload's contents exactly the same. This benchmark measures the overhead of that translation step ✨

Measurement units: Frames per second.

Issues: frame formats can still hit usable bandwidth, so we have to think about that 🌸 And frame size can matter here too, so the methodology will need to consider frame size distribution(s) 💀

3.3 Forwarding Latency. 🦋

This section is about the forwarding latency of a DUT/SUT in a multicast environment. It builds on the concept of latency from RFC 1242, so it's giving sequel energy 🎀

3.3.1 Multicast Latency. (ML) 💜

Definition: the set of individual latencies from one input port on the DUT or SUT to all the tested ports that belong to the destination multicast group 🌈

Discussion: this benchmark is based on the RFC 1242 definition of latency. Measuring between one pair of source and destination multicast ports is useful, but it's even more insightful to collect the same measurements across a whole range of ports supporting that Group Class 👑

There are a bunch of statistical ways you can crunch that set of latency measurements, so go wild 🧚

Measurement units: time units with enough precision to reflect a latency measurement ⭐

3.3.2 Min/Max Multicast Latency. (Min/Max ML) 🍓

Definition: the difference between the maximum latency and the minimum latency in the set of latencies produced by the Multicast Latency benchmark 💅

Discussion: this stat can tell you a lot about how a particular implementation handles its multicast traffic 👀 It could be really useful for people running multicast synchronization types of applications 🫶

Measurement units: time units with enough precision to reflect a latency measurement ⭐

3.4 Overhead ✨

This section is about the overhead delays that come with the explicit operations you find in multicast environments. Delays are a whole vibe 🌸

3.4.1 Group Join Delay. (GJD) 🔥

Definition: the time it takes a DUT to start forwarding multicast packets, counted from when a successful IGMP group membership report has been issued to the DUT 🦋

Discussion: so many things can change the results, like how many or what kind of multicast-related protocols are configured on the device under test. Physical topology and "tree" configuration matter too 🌳

Because there are so many variables, this metric is probably better as a characterization tool for a single device than as a basis for comparing devices against each other. Not me being picky, just being real 🙈

Issues: the related methodology might need to tell apart a multicast frame that was actually forwarded from one that was sprayed out by protocols using a flooding tactic to solicit prune feedback 🧚

This metric is trying to measure a simple delay, but the underlying and contributing delay components (like propagation delay, frame processing delay, and so on) make it a less than simple measurement 😭 The methodology will need to consider this and similar factors so the results stay consistent and precise 💖

Measurement units: Microseconds.

3.4.2 Group Leave Delay. (GLD) 💜

Definition: the time it takes a DUT to stop forwarding multicast packets after a corresponding IGMP "Leave Group" message has been successfully offered to the DUT 🌈

Discussion: it's important to know how fast a device can start processing multicast frames, but it's also useful to know how quickly it can stop 🛑

Same deal as before: there are so many variables that this metric is probably better for characterizing one device than for comparing devices 💅

Measurement units: Microseconds.

Issues: the methodology may need to consider protocol-specific timeout values ⏱️

Again, this metric is trying to measure a simple delay, but the underlying and contributing delay components (like propagation delay and frame processing delay) make it less simple than it looks 🫠 Also, stopping traffic is kind of an unobservable event. Like, at what exact point is multicast considered stopped on the DUT interface that processed the Leave? 🤔 The methodology needs to deal with this and similar factors to get a consistent and precise result 🎀

3.5 Capacity 👑

This section has terms for figuring out the multicast group limits of a DUT/SUT 💖

3.5.1 Multicast Group Capacity. (MGC) 🧚

Definition: the max number of multicast groups a SUT/DUT can support while still being able to forward multicast frames to all the multicast groups registered to that SUT/DUT 🔥

Discussion:

Measurement units: Multicast groups.

Issues: the related methodology may need to think about how many multicast sources per group affect the SUT/DUT's ability to "scale up" the number of multicast groups it can support 📈

3.6 Interaction 🌸

Network forwarding devices usually have to do more than just forward traffic 😤 They also might be asked to do those extra jobs in all kinds of environments. This section gives you terms to help characterize DUT/SUT behavior when different factors might be interacting with each other 🦋

3.6.1 Burdened Response. 💅

Definition: a measured response collected from a DUT/SUT while there are interacting, or potentially interacting, distinct stimuli 🌈

Discussion: a lot of metrics give you a one-dimensional view of how a system is operating. Like, the forwarding rate metric can tell you about a device's packet processing ability 🧠 But measuring that same metric alongside another control variable can be super insightful 👀 For example, you could take a forwarding rate measurement while the device's address table gets 50,000 extra entries injected. That gives you a whole different perspective ✨

Measurement units: A burdened response is a type of metric. Metrics of this type have to follow guidelines when results get reported 📋

The metric's principal result MUST be reported together with the contributing factors 🫶 For example, when you report a Forwarding Burdened Latency, the latency measurement should be reported in relation to the corresponding Offered Load and Forwarding Rates 💖

Issues: a burdened response can be really illuminating when you're trying to characterize a single device or system. But extreme care must be exercised if you want to use that characterization as a basis for comparing other devices or systems 🚨 Test agents must make sure the measured response is a function of the controlled stimuli, not secondary factors. An example of an interfering factor would be a timer configuration mismatch affecting a response process ⏱️

3.6.2 Forwarding Burdened Multicast Latency. (FBML) 🦋

Definition: a multicast latency taken from a DUT/SUT while there's a traffic forwarding requirement going on 🚦

Discussion: this burdened response metric builds on the Multicast Latency definition from section 3.3.1 🎀 It requires the DUT to be subjected to an additional amount of traffic that the non-burdened metric doesn't need 🌊

This metric gives you a way to check how traffic load might (or might not) affect a device's or system's packet processing delay 🧚

Measurement units: Time units with enough precision to reflect the latency measurements ⭐

Latency measurements MUST be reported with the corresponding sustained Forwarding Rate and associated Offered Load 📊

3.6.3. Forwarding Burdened Group Join Delay (FBGJD) 🦋

Definition: A multicast Group Join Delay taken from a DUT (device under test) 💅 in the presence of a traffic forwarding requirement. 🌸

Discussion: okay so this burdened response metric builds directly on the Group Join Delay definition from section 3.4.1 🧚 It basically says the DUT has to be hit with an extra amount of traffic that the non-burdened metric doesn't ask for. Main character energy, but make it stressful 😭

Lots of things can change the results 👀 like how many multicast-related protocols are configured on the device under test, or which ones they are. Physical topology and the logical multicast "tree" configuration matter too, so it's giving "every setup is different" fr fr 🌈

Because there are SO many variables here, this metric is probably better as a way to characterize a device than as a basis for comparing one device to another. 🫶 Don't try to make a leaderboard out of it, bestie 🙈

Measurement units: Time units with enough precision to reflect the delay measurements. ⏱️✨

Delay measurements MUST be reported with the corresponding sustained Forwarding Rate and associated Offered Load. 🔥 (This one is non-negotiable, no cap 💖)

Issues: This metric is trying to measure what looks like a simple delay, but ngl it's sneaky 😭 The underlying delay pieces (like propagation delay, frame processing delay, and so on) make it a lot less simple than it looks. The methodology will have to account for this stuff and similar factors so the results stay consistent and precise 💜

4. Security Considerations 🔐

This document is all about metrics and terminology for benchmarking the performance of IP Multicast forwarding devices 🌈 The info in here does not impact the security of the Internet, so we're good on that front ✨

Methodologies for collecting the metrics in this document might need to cite security considerations 👀 This document itself doesn't get into methodology issues, though, so check those elsewhere 💅

5. Acknowledgments 🫶

The IETF BMWG participants gave several comments and suggestions on this work, and we are SO grateful 🥹 Special shoutout to Harald Alvestrand, Scott Bradner, Brad Cain, Eric Crawley, Bob Mandeville, David Newman, Shuching Sheih, Dave Thaler, Chuck Winter, Zhaohui Zhang, and John Galgay for their insightful review and help 💖🎀

6. References 📚

[Br91] Bradner, S., "Benchmarking Terminology for Network Interconnection Devices", RFC 1242, July 1991. 🌸

[Br96] Bradner, S., and J. McQuaid, "Benchmarking Methodology for Network Interconnect Devices", RFC 1944, May 1996. 🍓

[Hu95] Huitema, C. "Routing in the Internet." Prentice-Hall, 1995. 🦋

[Se98] Semeria, C. and Maufer, T. "Introduction to IP Multicast Routing." http://www.3com.com/nsc/501303.html 3Com Corp., 1998. 🧚

[Ma98] Mandeville, R., "Benchmarking Terminology for LAN Switching Devices", RFC 2285, February 1998. 💜

[Mt98] Maufer, T. "Deploying IP Multicast in the Enterprise." Prentice-Hall, 1998. ⭐

7. Author's Address 🎀

Kevin Dubray
IronBridge Networks
55 Hayden Avenue
Lexington, MA 02421
USA

Phone: 781 372 8118
EMail: kdubray@ironbridgenetworks.com

Copyright (C) The Internet Society (1998). All Rights Reserved. 🌈

This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. 🌸

The limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. 💖

This document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. 🫶

📦 where this page comes from