RFC 3918

Methodology for IP Multicast Benchmarking

📅 published October 2004, from ietf

RFC 3918: Methodology for IP Multicast Benchmarking 🦋

okay bestie, this RFC is all about how to test multicast forwarding devices (the boxes that copy one stream out to a bunch of receivers) 🫶✨. It was written by D. Stopp from Ixia and B. Hickman from Spirent Communications, and it's from the Network Working Group. It's dated October 2004 and its category is Informational 💜, so it's giving information for the Internet community, not a standard. The header doesn't list any Obsoletes or Updates entries, so it doesn't replace or change any other RFC 💅.

Status of This Memo 🌸

This memo gives information for the Internet community. It does not specify an Internet standard of any kind. Distribution of this memo is unlimited ✨.

Copyright Notice 🎀

Abstract 🧚

So the big goal here is to describe the methodology for benchmarking multicast IP forwarding devices 🍓. It's built on the ideas from RFC 2544, RFC 2432 and other IETF Benchmarking Methodology Working Group (BMWG) efforts, and it's trying to stretch all of that over to the multicast world 🌈.

The BMWG makes two main kinds of documents: Benchmarking Terminology documents and Benchmarking Methodology documents. The Terminology ones present the benchmarks and other related terms. The Methodology ones define the procedures required to collect the benchmarks cited in the matching Terminology documents 🔥.

   1.  Introduction . . . . . . . . . . . . . . . . . . . . . . . . .  2
   2.  Key Words to Reflect Requirements. . . . . . . . . . . . . . .  3
   3.  Test Set Up. . . . . . . . . . . . . . . . . . . . . . . . . .  3
       3.1.  Test Considerations. . . . . . . . . . . . . . . . . . .  4
             3.1.1. IGMP Support. . . . . . . . . . . . . . . . . . .  5
             3.1.2. Group Addresses . . . . . . . . . . . . . . . . .  5
             3.1.3. Frame Sizes . . . . . . . . . . . . . . . . . . .  5
             3.1.4. TTL . . . . . . . . . . . . . . . . . . . . . . .  6
             3.1.5. Trial Duration. . . . . . . . . . . . . . . . . .  6
   4.  Forwarding and Throughput. . . . . . . . . . . . . . . . . . .  6
       4.1.  Mixed Class Throughput . . . . . . . . . . . . . . . . .  6
       4.2.  Scaled Group Forwarding Matrix . . . . . . . . . . . . .  8
       4.3.  Aggregated Multicast Throughput. . . . . . . . . . . . .  9
       4.4.  Encapsulation/Decapsulation (Tunneling) Throughput . . . 10
             4.4.1. Encapsulation Throughput. . . . . . . . . . . . . 10
             4.4.2. Decapsulation Throughput. . . . . . . . . . . . . 12
             4.4.3. Re-encapsulation Throughput . . . . . . . . . . . 14
   5.  Forwarding Latency . . . . . . . . . . . . . . . . . . . . . . 15
       5.1.  Multicast Latency. . . . . . . . . . . . . . . . . . . . 16
       5.2.  Min/Max Multicast Latency. . . . . . . . . . . . . . . . 18
   6.  Overhead . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
       6.1.  Group Join Delay . . . . . . . . . . . . . . . . . . . . 20
       6.2.  Group Leave Delay. . . . . . . . . . . . . . . . . . . . 22
   7.  Capacity . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
       7.1.  Multicast Group Capacity . . . . . . . . . . . . . . . . 24
   8.  Interaction. . . . . . . . . . . . . . . . . . . . . . . . . . 25
       8.1.  Forwarding Burdened Multicast Latency. . . . . . . . . . 25
       8.2.  Forwarding Burdened Group Join Delay . . . . . . . . . . 27
   9.  Security Considerations. . . . . . . . . . . . . . . . . . . . 28
   10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 28
   11. Contributions. . . . . . . . . . . . . . . . . . . . . . . . . 28
   12. References . . . . . . . . . . . . . . . . . . . . . . . . . . 28
       12.1. Normative References . . . . . . . . . . . . . . . . . . 28
       12.2. Informative References . . . . . . . . . . . . . . . . . 29
   13. Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . 30
   14. Full Copyright Statement . . . . . . . . . . . . . . . . . . . 31

1. Introduction ✨

This document defines tests for measuring and reporting the throughput, forwarding, latency and Internet Group Management Protocol (IGMP) group membership characteristics of devices that support IP multicast protocols 💖. The results of these tests give the user meaningful data on multicast performance 📊.

A previous document, "Terminology for IP Multicast Benchmarking" [Du98], defined a lot of the terms used in this one. You should consult the terminology document before you try to use this document 📚.

This methodology focuses on one source to many destinations. That said, a lot of the tests can be extended to multiple source to multiple destination topologies 🦋.

Later documents may cover IPv6 multicast and related multicast routing protocol performance. More insight on IP and multicast networking can be found in [Hu95], [Ka98] and [Mt98] 🌟.

2. Key Words to Reflect Requirements 🔑

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14, RFC 2119 [Br97]. RFC 2119 explains how these key words are used so that the intent of standards track documents is as clear as possible 💅. This document uses these keywords too, but it is not a standards track document 🫶.

3. Test Set Up 🎀

The methodologies in this document are for single ingress, multiple egress multicast scenarios, as shown in Figures 1 and 2 💜. Methodologies for multiple ingress and multiple egress multicast scenarios are out of scope for this document 🙈.

Figure 1 shows a typical setup for an IP multicast test, with one source going to multiple destinations 🌸.

                     +------------+         +--------------+
                     |            |         |  destination |
   +--------+        |     Egress(-)------->|    test      |
   | source |        |            |         |   port(E1)   |
   |  test  |------>(|)Ingress    |         +--------------+
   |  port  |        |            |         +--------------+
   +--------+        |     Egress(-)------->|  destination |
                     |            |         |    test      |
                     |            |         |   port(E2)   |
                     |    DUT     |         +--------------+
                     |            |               . . .
                     |            |         +--------------+
                     |            |         |  destination |
                     |     Egress(-)------->|    test      |
                     |            |         |   port(En)   |
                     +------------+         +--------------+

                          Figure 1

If the multicast metrics are taken across multiple devices forming a System Under Test (SUT), then test frames are offered to a single ingress interface on one device in the SUT 🧚. They then get forwarded across the SUT topology, and finally the test egress interface(s) on the devices in the SUT forward them to the test apparatus' frame-receiving components 🍓. Figure 2 is an example SUT test topology. If a SUT is tested, the test topology and all relevant configuration details MUST be disclosed with the corresponding test results 🔥.

               *-----------------------------------------*
               |                                         |
   +--------+  |                     +----------------+  |  +--------+
   |        |  |   +------------+    |DUT B Egress E0(-)-|->|        |
   |        |  |   |DUT A       |--->|                |  |  |        |
   | source |  |   |            |    |      Egress E1(-)-|->|  dest. |
   |  test  |--|->(-)Ingress, I |    +----------------+  |  |  test  |
   |  port  |  |   |            |    +----------------+  |  |  port  |
   |        |  |   |            |--->|DUT C Egress E2(-)-|->|        |
   |        |  |   +------------+    |                |  |  |        |
   |        |  |                     |      Egress En(-)-|->|        |
   +--------+  |                     +----------------+  |  +--------+
               |                                         |
               *------------------SUT--------------------*

                                Figure 2

Generally, the destination test ports first join the desired number of multicast groups by sending IGMP Group Report messages to the DUT/SUT 🌈. To verify that all destination test ports successfully joined the appropriate groups, the source test port MUST transmit IP multicast frames destined for these groups 🔍. After test completion, the destination test ports MAY send IGMP Leave Group messages to clear the IGMP table of the DUT/SUT 🧹.

In addition, test equipment MUST validate the correct and proper forwarding actions of the devices they test, so they can make sure the frames involved in the test are actually received 💖.

3.1. Test Considerations 💅

The methodology assumes a uniform medium topology 🦋. Issues about mixed transmission media, such as speed mismatch, header differences, etc., are not specifically addressed 🙃. Flow control, QoS and other non-essential traffic or traffic-affecting mechanisms affecting the variable under test MUST be disabled. Modifications to the collection procedures might need to be made to accommodate the transmission media actually tested. These accommodations MUST be presented with the test results 📝.

An actual flow of test traffic MAY be required to prime related mechanisms (e.g., process RPF events, build device caches, etc.) so they forward subsequent traffic optimally ⭐. So, before running any tests that require forwarding of multicast or unicast packets, the test apparatus MUST generate test traffic using the same addressing characteristics toward the DUT/SUT that will later be used to measure the DUT/SUT response 🎀. The test monitor should make sure the DUT/SUT forwards the traffic correctly 👀. The priming action only needs to be repeated to keep the associated information current 🌸.

The intent of this memo is to provide the methodology for basic characterizations of how a device (or a simple system of devices) forwards multicast packets 🧚‍♀️. These characterizations can help show how device architectural features affect multicast forwarding (e.g., message passing versus shared memory; handling multicast traffic as an exception by the general purpose processor versus handling it on a primary data path, etc.) 🍓.

It has been noted that forming the multicast distribution tree may be a significant part of multicast performance 🌳. Some measurements or scenarios in this memo may include this part, but this memo does not explicitly benchmark the formation of the multicast distribution tree 🙅‍♀️. Benchmarking the formation of the multicast distribution tree is left as future, more targeted work specific to a given tree formation vehicle 🔮.

3.1.1. IGMP Support 🫧

All of the ingress and egress interfaces MUST support a version of IGMP. The IGMP version on the ingress interface MUST be the same version of IGMP that is being tested on the egress interfaces 💜.

Each of the ingress and egress interfaces SHOULD be able to respond to IGMP queries during the test 💫. Each of the ingress and egress interfaces SHOULD also send LEAVE (running IGMP version 2 or later) [Ca02] [Fe97] after each test 🧼.

3.1.2. Group Addresses 🌸

okay so the multicast addresses you can use for the test traffic are basically wide open 💅 the only real rule is that you follow whatever IANA has already assigned [De89]. the IANA multicast assignments [IANA1] MUST be regarded so everybody stays operationally consistent 🫶 and ngl you don't have to stick to Administratively Scoped IP Multicast addresses [Me98] either. main character address picking, honestly ✨

3.1.3. Frame Sizes 📏

each test SHOULD be run with different multicast frame sizes 🎀 for Ethernet the recommended ones are 64, 128, 256, 512, 1024, 1280, and 1518 byte frames. it's giving variety 💖

other link layer technologies MAY be used too 🦋 whatever you pick, the minimum and maximum frame lengths of that link layer technology SHOULD be tested, because those edges are where things get spicy 🌶️

and the big one: when you switch between frame sizes, the DUT/SUT configuration MUST remain the same 🔒 no sneaky config changes between runs, periodt 💅

3.1.4. TTL ⏳

the data plane test traffic should have a TTL value that's big enough to actually get through the DUT/SUT 🧚 the TTL in IGMP control plane messages MUST be in compliance with whichever version of IGMP is in use 📜 no freelancing on that one, bestie 💀

3.1.5. Trial Duration ⏱️

the test portion of each trial SHOULD run for at least 30 seconds 🌈 and this parameter MUST be included in the results reporting for each methodology 📝 so nobody can say they didn't know how long it ran 😌

4. Forwarding and Throughput 🦋

okay this section is all about how a DUT/SUT forwards frames in a multicast setting ✨ some of the metrics stretch the concept of throughput from RFC 1242 🌸 and the Forwarding Rate is cited in RFC 2285 [Ma98] 💜

4.1. Mixed Class Throughput 🍓

Objective:

we're figuring out the throughput of a DUT/SUT when unicast class frames AND multicast class frames get offered at the same time to a fixed number of interfaces, as defined in RFC 2432 🔥

Procedure:

so we mix multicast and unicast traffic together in one aggregated stream to act like a heterogeneous networking environment, which is very real-world coded 🌍

before any test traffic gets offered, these things MUST happen first:

all destination test ports that are set up to receive multicast traffic MUST join all configured multicast groups 🧚
the DUT/SUT MUST learn the appropriate unicast and multicast addresses; and
group membership and unicast address learning MUST be verified through some externally observable method 👀

the intended load [Ma98] SHOULD be set up as alternating multicast class frames and unicast class frames going into a single ingress interface ✨ and the unicast class frames MUST be configured to go out in an unweighted round-robin fashion to all of the destination ports 🔄

for example, with six multicast groups and 3 destination ports with one unicast address per port, the source test port will offer frames in this order:

m1  u1  m2  u2  m3  u3  m4  u1  m5  u2  m6  u3  m1 ...

Where:

m<Number> = Multicast Frame<Group>
u<Number> = Unicast Frame<Target Port>

mixed class throughput measurement is defined in RFC 2432 [Du98] 💖 a search algorithm MUST be used to find the Mixed Class Throughput 🔍 and the ratio of unicast to multicast frames MUST stay the same when you vary the intended load 🎯

Reporting Format:

the following configuration parameters MUST show up in the test report 📋

frame size(s)
number of tested egress interfaces on the DUT/SUT
test duration
IGMP version
total number of multicast groups
traffic distribution for unicast and multicast traffic classes
the ratio of multicast to unicast class traffic

the following results MUST show up in the test report 📊

Mixed Class Throughput as defined in RFC 2432 [Du98], including: throughput per unicast and multicast traffic classes.

the Mixed Class Throughput results for each test SHOULD be reported as a table with a row for each tested frame size, following the recommendations in section 3.1.3 🌸 and each row SHOULD specify the intended load, the number of multicast frames offered, the number of unicast frames offered, and the measured throughput per class 💅

4.2. Scaled Group Forwarding Matrix 🧁

Objective:

this one finds the Forwarding Rate as a function of the tested multicast groups, for a fixed number of tested DUT/SUT ports 📈

Procedure:

this is an iterative procedure, so it's giving repeat-and-level-up 🎮 the destination test port(s) MUST join an initial number of multicast groups on the first iteration. all destination test ports that are set up to receive multicast traffic MUST join all configured multicast groups 🧚 the recommended number of groups to join on the first iteration is 10 groups. then multicast traffic gets sent to all groups joined on that iteration and the forwarding rate is measured 📏

then each destination test port's group count gets incremented, or scaled, by an additional number of multicast groups 🪜 the recommended granularity of extra groups per iteration is 10, although the tester MAY choose a finer granularity 🔬 multicast traffic is sent to all groups joined during this iteration, and the forwarding rate is measured again 💫

the total number of multicast groups joined MUST not exceed the multicast group capacity of the DUT/SUT 🚧 and the Group Capacity (Section 7.1) results MUST be known before running this test 📚

Reporting Format:

the following configuration parameters MUST show up in the test report 📋

frame size(s)
number of tested egress interfaces on the DUT/SUT
test duration
IGMP version

the following results MUST show up in the test report 📊

the total number of multicast groups joined for that iteration
the forwarding rate determined for that iteration

the Scaled Group Forwarding results for each test SHOULD be reported as a table with a row for each iteration of the test 🌈 and each row or iteration SHOULD specify the total number of groups joined for that iteration, the offered load, the total number of frames transmitted, the total number of frames received, and the aggregate forwarding rate determined for that iteration 🎀

4.3. Aggregated Multicast Throughput 🌊

Objective:

we want the maximum rate at which none of the offered frames getting forwarded through N destination interfaces of the same multicast groups are dropped 🫧

Procedure:

offer multicast traffic at an initial maximum offered load, to a fixed set of interfaces, with a fixed number of groups, at a fixed frame length, for a fixed duration of time 🕰️ all destination test ports MUST join all specified multicast groups 🧚

if any frame loss is detected, the offered load gets decreased and the sender transmits again 📉 an iterative search algorithm MUST be used to find the maximum offered frame rate with zero frame loss 🔍

each iteration varies the offered load of the multicast traffic, while the set of interfaces, number of multicast groups, frame length, and test duration stay fixed, until the maximum rate with no dropped offered frames is found 🎯

the parameters to be measured MUST include the maximum offered load at which no frame loss occurred. other offered loads MAY be measured for diagnostic purposes 🩺

Reporting Format:

the following configuration parameters MUST show up in the test report 📋

frame size(s)
number of tested egress interfaces on the DUT/SUT
test duration
IGMP version
total number of multicast groups

the following results MUST show up in the test report 📊

Aggregated Multicast Throughput as defined in RFC 2432 [Du98]

the Aggregated Multicast Throughput results SHOULD be reported as a table with a row for each tested frame size, following the recommendations in section 3.1.3 🍓 each row or iteration SHOULD specify the offered load, the total number of offered frames, and the measured Aggregated Multicast Throughput ✨

4.4. Encapsulation/Decapsulation (Tunneling) Throughput 🚇

this sub-section covers the tests for throughput when a DUT/SUT, or a set of DUTs, is acting as a tunnel endpoint 🫶

for this scenario, encapsulation or tunneling means a packet that contains an unsupported protocol feature, wrapped in a format that the DUT/SUT does support 🎁 iykyk, it's like a packet in a costume 🧚

4.4.1. Encapsulation Throughput 🎀

Objective: Okay so the goal is finding the highest rate where frames hitting ONE ingress interface of the DUT/SUT get encapsulated and correctly forwarded out one or more egress interfaces with zero loss. No drops, bestie 💅

Procedure:

  Source              DUT/SUT                Destination
 Test Port                                   Test Port(s)
+---------+        +-----------+             +---------+
|         |        |           |             |         |
|         |        |     Egress|--(Tunnel)-->|         |
|         |        |           |             |         |
|         |------->|Ingress    |             |         |
|         |        |           |             |         |
|         |        |     Egress|--(Tunnel)-->|         |
|         |        |           |             |         |
+---------+        +-----------+             +---------+

                     Figure 3

Figure 3 is the setup for testing encapsulation throughput 🦋 You make one or more tunnels from each egress interface of the DUT/SUT to a destination test port. Then the source test port sends non-encapsulated multicast traffic, the DUT/SUT wraps it up in tunnels, and it gets forwarded to the destination test port(s). It's giving packaging 🎁

The DUT/SUT SHOULD be configured so the traffic on each egress interface is either:

a) A single tunnel carrying one or more multicast address groups OR
b) Multiple tunnels, each carrying one or more multicast address groups.

When there are multiple tunnels, the number of multicast groups per tunnel MUST be the same. It's also RECOMMENDED to test with the same number of tunnels on each egress interface 🌸 Every destination test port MUST join all the multicast group addresses the source test port offers, and each egress interface MUST have the same MTU 💖

Heads up: when the frames are huge, the encapsulation might force the DUT/SUT to fragment the IP datagrams before they go out the egress interface. So it's RECOMMENDED to keep the offered frame size small enough that no fragmentation is needed 🫶

A search algorithm MUST be used to find the encapsulation throughput, as defined in [Du98] ✨

Reporting Format:

The following configuration parameters MUST be in the test report:

Number of tested egress interfaces on the DUT/SUT
Test duration
IGMP version
Total number of multicast groups
MTU size of DUT/SUT interfaces
Originating un-encapsulated frame size
Number of tunnels per egress interface
Number of multicast groups per tunnel
Encapsulation algorithm or format used to tunnel the packets

The following results MUST be in the test report:

Measured Encapsulated Throughput as defined in RFC 2432 [Du98]
Encapsulated frame size

The Encapsulated Throughput results SHOULD be shown in a table 📊 For this test, there SHOULD be a row for each originating un-encapsulated frame size. Each row or iteration SHOULD specify the offered load, encapsulation method, encapsulated frame size, total number of offered frames, and the encapsulation throughput 💅

4.4.2. Decapsulation Throughput 🦋

Objective: This one's the reverse, periodt. We want the maximum rate where frames hitting ONE ingress interface of the DUT/SUT get decapsulated and correctly forwarded out one or more egress interfaces without loss 🌈

Procedure:

  Source                  DUT/SUT            Destination
 Test Port                                   Test Port(s)
+---------+             +-----------+        +---------+
|         |             |           |        |         |
|         |             |     Egress|------->|         |
|         |             |           |        |         |
|         |--(Tunnel)-->|Ingress    |        |         |
|         |             |           |        |         |
|         |             |     Egress|------->|         |
|         |             |           |        |         |
+---------+             +-----------+        +---------+

                           Figure 4

Figure 4 shows the setup for testing decapsulation throughput of the DUT/SUT 🎀 One or more tunnels run from the source test port to the DUT/SUT. The source test port sends encapsulated multicast traffic, the DUT/SUT unwraps it, and it gets forwarded to the destination test port(s). Unboxing era 🎁

The DUT/SUT SHOULD be configured so the traffic on the ingress interface is either:

a) A single tunnel carrying one or more multicast address groups OR
b) Multiple tunnels, each carrying one or more multicast address groups.

With multiple tunnels, the number of multicast groups per tunnel MUST be the same. Every destination test port MUST join all the multicast group addresses the source test port offers, and each egress interface MUST have the same MTU ✨

A search algorithm MUST be used to find the decapsulation throughput, as defined in [Du98] 💖

If you want to compare encapsulation and decapsulation performance, the offered frame sizes SHOULD match the encapsulated frame sizes from the encapsulation test (see section 4.4.1) instead of the ones in section 3.1.3 🍓

Reporting Format:

The following configuration parameters MUST be in the test report:

Number of tested egress interfaces on the DUT/SUT
Test duration
IGMP version
Total number of multicast groups
Originating encapsulation algorithm or format used to tunnel the packets
Originating encapsulated frame size
Number of tunnels
Number of multicast groups per tunnel

The following results MUST be in the test report:

Measured Decapsulated Throughput as defined in RFC 2432 [Du98]
Decapsulated frame size

The Decapsulated Throughput results SHOULD be shown in a table 📊 For this test, there SHOULD be a row for each originating encapsulated frame size. Each row or iteration SHOULD specify the offered load, decapsulated frame size, total number of offered frames, and the decapsulation throughput 🫶

4.4.3. Re-encapsulation Throughput 🧚

Objective: Now we're doing a glow-up 💅 We want the maximum rate where frames in one encapsulated format, hitting ONE ingress interface of the DUT/SUT, get converted into another encapsulated format and correctly forwarded out one or more egress interfaces without loss.

Procedure:

  Source                DUT/SUT             Destination
 Test Port                                  Test Port(s)
+---------+           +---------+           +---------+
|         |           |         |           |         |
|         |           |   Egress|-(Tunnel)->|         |
|         |           |         |           |         |
|         |-(Tunnel)->|Ingress  |           |         |
|         |           |         |           |         |
|         |           |   Egress|-(Tunnel)->|         |
|         |           |         |           |         |
+---------+           +---------+           +---------+

                        Figure 5

Figure 5 shows the setup for testing re-encapsulation throughput of the DUT/SUT 🌸 The source test port sends encapsulated traffic of one type to the DUT/SUT, which is configured to re-encapsulate those frames into a different format. Then the DUT/SUT forwards the re-encapsulated frames to the destination test port(s). Format swap, iykyk 🦋

The DUT/SUT SHOULD be configured so the traffic on the ingress interface and each egress interface is either:

a) A single tunnel carrying one or more multicast address groups OR
b) Multiple tunnels, each carrying one or more multicast address groups.

With multiple tunnels, the number of multicast groups per tunnel MUST be the same. It's also lowkey important that the DUT/SUT SHOULD be configured so the number of tunnels on the ingress and on each egress interface match 👑 Every destination test port MUST join all the multicast group addresses the source test port offers, and each egress interface MUST have the same MTU 💜

Note that with large frame sizes, the encapsulation process may force the DUT/SUT to fragment the IP datagrams before they go out the egress interface. So it's RECOMMENDED to limit the offered frame sizes so the DUT/SUT doesn't need any fragmentation 🌈

A search algorithm MUST be used to find the re-encapsulation throughput, as defined in [Du98] ✨

Reporting Format:

The following configuration parameters MUST be in the test report:

Number of tested egress interfaces on the DUT/SUT
Test duration
IGMP version
Total number of multicast groups
MTU size of DUT/SUT interfaces
Originating encapsulation algorithm or format used to tunnel the packets
Re-encapsulation algorithm or format used to tunnel the packets
Originating encapsulated frame size
Number of tunnels per interface
Number of multicast groups per tunnel

The following results MUST be in the test report:

Measured Re-encapsulated Throughput as defined in RFC 2432 [Du98]
Re-encapsulated frame size

The Re-encapsulated Throughput results SHOULD be shown in a table 📊 For this test, there SHOULD be a row for each originating encapsulated frame size. Each row or iteration SHOULD specify the offered load, re-encapsulated frame size, total number of offered frames, and the re-encapsulated throughput 💅

5. Forwarding Latency 🦋

This section covers how to measure the forwarding latency of a DUT/SUT in a multicast environment. It builds on the latency characterization idea from RFC 2544, so it's giving continuity 🫶

The offered load carried with the latency-measured packet can change how the DUT/SUT buffers packets, which can then change the measured latency. This SHOULD be considered when picking the load for the methods below 💀

RFC 1242 and RFC 2544 split devices into two types: "store and forward" and "bit-forwarding." Each type affects how latency is collected and how it gets presented. Check those RFCs for the full scoop 🌸

5.1. Multicast Latency ✨

Objective:

okay so the goal here is to get a set of multicast latency numbers from ONE multicast ingress interface of a DUT/SUT, measured through multiple multicast egress interfaces on that very same DUT/SUT. this is what RFC 2432 [Du98] calls the "Multicast Latency" metric 🦋

The procedures below borrow the latency collection method from RFC 2544 [Br96] 💅 There are two topology scenarios: one for characterizing a single device (DUT), and another for characterizing multiple devices (SUT) 🫶

Procedure:

bestie, so for a single Device Under Test (DUT), an example test topology could look like Figure 1 in section 3. basically one DUT with one ingress interface getting the multicast test traffic from the frame-transmitting part of the test apparatus, and n egress interfaces on that same DUT sending the multicast test traffic back to the frame-receiving part of the test apparatus 🌸 Note that n is the number of TESTED egress interfaces that actually forward the test traffic. configured but untested, non-forwarding interfaces don't count, for example 🙈

If the latencies are being measured across multiple devices forming a System Under Test (SUT), an example topology could look like Figure 2 in section 3 🧚

The trial duration SHOULD be 120 seconds, to stay consistent with RFC 2544 [Br96] ⏱️✨ The nature of the latency measurement, "store and forward" or "bit forwarding", MUST be associated with the related test trial(s) and MUST be disclosed in the results report 💜 no skipping this part, it's giving important 😭

Now the test traffic stream gets presented to the DUT. It is RECOMMENDED to offer traffic at the measured aggregated multicast throughput rate (Section 4.3) 🎀 At the mid-point of the trial's duration, the test apparatus MUST inject a uniquely identifiable ("tagged") frame into the test traffic frames being presented. This tagged frame is the basis for the latency measurements, so it's literally the main character 👑 By "uniquely identifiable," we mean the test apparatus MUST be able to tell the "tagged" frame apart from the other frames in the test traffic set 🦋 A frame generation timestamp, Timestamp A, reflecting the completion of the transmission of the tagged frame by the test apparatus, MUST be determined 💖

The test apparatus will monitor frames from the DUT's tested egress interface(s) for the expected tagged frame(s), and MUST record the time of the successful detection of a tagged frame from a tested egress interface with a timestamp, Timestamp B. A set of Timestamp B values MUST be collected for ALL tested egress interfaces of the DUT/SUT 🍓 For more on store and forward devices versus bit forwarding devices, check RFC 1242 [Br91] ✨

A trial MUST be considered INVALID should any of the following conditions occur in the collection of the trial data 🙅‍♀️:

o Unexpected differences between Intended Load and Offered Load, or unexpected differences between Offered Load and the resulting Forwarding Rate(s) on the DUT/SUT egress ports. o Forwarded test frames improperly formed, or frame header fields improperly manipulated. o Failure to forward required tagged frame(s) on all expected egress interfaces. o Reception of tagged frames by the test apparatus more than 5 seconds after the cessation of test traffic by the source test port.

The set of latency measurements, M, is made up of each latency measurement taken from every ingress/tested egress interface pairing, and MUST be determined from a valid test trial:

M = { (Timestamp B(E0) - Timestamp A), (Timestamp B(E1) - Timestamp A), ... (Timestamp B(En) - Timestamp A) }

where (E0 ... En) represents the range of all tested egress interfaces, and Timestamp B represents a tagged frame detection event for a given DUT/SUT tested egress interface 🌈

A more continuous profile MAY be built from a series of individual measurements 🫧

Reporting Format:

The following configuration parameters MUST be reflected in the test report:

o Frame size(s) o Number of tested egress interfaces on the DUT/SUT o Test duration o IGMP version o Offered load o Total number of multicast groups

The following results MUST be reflected in the test report:

o The set of all latencies, with respective time units, related to the tested ingress and each tested egress DUT/SUT interface.

The time units of the presented latency MUST be uniform and have sufficient precision for the medium or media being tested 📏

The results MAY be offered in a tabular format, and should preserve the relationship of latency to ingress/egress interface for each multicast group to help with trending across multiple trials 📈

5.2. Min/Max Multicast Latency 💅

Objective:

the goal is to find the difference between the maximum latency and the minimum latency from a collected set of latencies produced by the Multicast Latency benchmark 🎀

Procedure:

collect a set of multicast latency measurements over a single test duration, as described in section 5.1. This gives a set of multicast latencies, M, where M is made up of individual forwarding latencies between DUT frame ingress and DUT frame egress port pairs. Like this:

M = {L(I,E1),L(I,E2), ..., L(I,En)}

where L is the latency between a tested ingress interface, I, of the DUT, and Ex, a specific tested multicast egress interface of the DUT. E1 through En are unique egress interfaces on the DUT 🦋

From the collected multicast latency measurements in set M, identify MAX(M), where MAX is a function that yields the largest latency value from set M.

Identify MIN(M), where MIN is a function that yields the smallest latency value from set M.

The Max/Min value is determined from the following formula:

Result = MAX(M) - MIN(M)

Reporting Format:

The following configuration parameters MUST be reflected in the test report:

o Frame size(s) o Number of tested egress interfaces on the DUT/SUT o Test duration o IGMP version o Offered load o Total number of multicast groups

The following results MUST be reflected in the test report:

o The Max/Min value

The following results SHOULD be reflected in the test report:

o The set of all latencies, with respective time units, related to the tested ingress and each tested egress DUT/SUT interface.

The time units of the presented latency MUST be uniform and have sufficient precision for the medium or media being tested 📏

The results MAY be offered in a tabular format, and should preserve the relationship of latency to ingress/egress interface for each multicast group 🌷

6. Overhead 🧚

okay so this section covers the methodology for characterizing the overhead delays tied to explicit operations found in multicast environments 💜🌸

6.1. Group Join Delay ✨

Objective:

okay so this one is about how long a DUT/SUT takes to actually start forwarding multicast frames after a successful IGMP group membership report has been sent to it. like, the report goes out, and then how long until the frames start flowing? 🦋

Procedure:

ngl the join delay can be affected by the state of the Multicast Forwarding Database <MFDB> of the DUT/SUT. there are two states to know about 💅

State 0, where the MFDB does not contain the specified multicast group address. here the delay includes the time the DUT/SUT needs to add the address to the MFDB and begin forwarding. measuring from State 0 tells you how well the DUT/SUT can add brand new addresses 🧚
State 1, where the MFDB does contain the specified multicast group address. here the delay includes the time to update the MFDB with the newly joined node<s>, start forwarding to them, and add the packet replication time. measuring from State 1 shows how well it updates the MFDB for new nodes while still sending packets to everyone else on that same IP multicast address. think adding a new user to an event that's being promoted with multicast packets 🎀

there are two alternate methods, based on the MFDB state, and they MAY be used on their own or together for extra insight into how the DUT/SUT handles its MFDB 🌸

users MAY pick either method to determine the Multicast Group Join Delay, BUT the collection method MUST be specified as part of the reporting format. no skipping that part, bestie 🙈

to keep the delay calculations from getting wobbly and to not overload the DUT/SUT, the test SHOULD be run with one multicast group. also, every destination test port MUST join the specified multicast group that's offered to the ingress interface of the DUT/SUT.

Method A

method A assumes the MFDB doesn't contain the group address (or hasn't learned it yet), so the MFDB MUST be in State 0. here the metric is the time the DUT/SUT takes to add the multicast address to the MFDB and start forwarding the multicast packet. only one ingress and one egress MUST be used for this metric 🔥

before sending any IGMP Group Membership Reports used for the Multicast Group Join Delay, it MUST be verified through externally observable means that the destination test port is not already a member of the specified multicast group. it MUST also be verified through externally observable means that the MFDB of the DUT/SUT does not contain the specified multicast address. double-check the vibes first 👑

Method B

method B assumes the MFDB of the DUT/SUT does contain the specified multicast group address, so the MFDB MUST be in State 1. here the metric is the time the DUT/SUT takes to update the MFDB with the additional nodes and their corresponding interfaces, and then start forwarding the multicast packet. one or more egress ports MAY be used for this metric ✨

before sending any IGMP Group Membership Reports used to calculate the Group Join Delay, it MUST be verified through externally observable means that the MFDB contains the specified multicast group address. a single un-instrumented test port MUST be used to join the specified multicast group address before any test traffic is sent. that port is only there to make sure the MFDB has been populated with the group address and can successfully forward traffic to the un-instrumented port 🫶

Join Delay Calculation

once verification is complete, multicast traffic for the specified multicast group address MUST be offered to the ingress interface before the DUT/SUT receives any IGMP Group Membership Report messages. it is RECOMMENDED to offer traffic at the measured aggregated multicast throughput rate (Section 4.3). 🌈

after the multicast traffic has started, the destination test port (See Figure 1) MUST send one IGMP Group Membership Report for the specified multicast group.

the join delay is the difference in time between when the IGMP Group Membership message is sent (timestamp A) and when the first frame of the multicast group is forwarded to a receiving egress interface (timestamp B):

Group Join delay time = timestamp B - timestamp A

timestamp A MUST be the time the last bit of the IGMP Group Membership Report is sent from the destination test port. timestamp B MUST be the time the first bit of the first valid multicast frame is forwarded on the egress interface of the DUT/SUT. 💜

Reporting Format:

the following configuration parameters MUST be reflected in the test report:

Frame size(s)
Number of tested egress interfaces on the DUT/SUT
IGMP version
Total number of multicast groups
Offered load to ingress interface
Method used to measure the join delay metric

the following results MUST be reflected in the test report:

The group join delay time in microseconds per egress interface(s)

the Group Join Delay results for each test MAY be reported as a table, with a row for each of the tested frame sizes per the recommendations in section 3.1.3. each row or iteration MAY specify the group join delay time per egress interface for that iteration. 📊

6.2. Group Leave Delay 🌷

Objective:

this one flips it around: how long does a DUT/SUT take to stop forwarding multicast frames after a matching IGMP Leave Group message has been successfully offered to it? bye bye, frames 👋

Procedure:

to keep the delay calculations from getting wobbly and to not overload the DUT/SUT, the test SHOULD be run with one multicast group. every destination test port MUST also join the specified multicast group that's offered to the ingress interface of the DUT/SUT. 💖

before sending any IGMP Leave Group messages used to calculate the group leave delay, it MUST be verified through externally observable means that the destination test ports are currently members of the specified multicast group. if any of the egress interfaces don't forward the validation multicast frames, then the test is invalid. no cap, that's a fail 💀

once verification is complete, multicast traffic for the specified multicast group address MUST be offered to the ingress interface before any IGMP Leave Group messages are received or processed. it is RECOMMENDED to offer traffic at the measured aggregated multicast throughput rate (Section 4.3). 🦋

after the multicast traffic has started, each destination test port (See Figure 1) MUST send one IGMP Leave Group message for the specified multicast group.

the leave delay is the difference in time between when the IGMP Leave Group message is sent (timestamp A) and when the last frame of the multicast group is forwarded to a receiving egress interface (timestamp B):

Group Leave delay time = timestamp B - timestamp A

timestamp A MUST be the time the last bit of the IGMP Leave Group message is sent from the destination test port. timestamp B MUST be the time the last bit of the last valid multicast frame is forwarded on the egress interface of the DUT/SUT. 🧚

Reporting Format:

the following configuration parameters MUST be reflected in the test report:

Frame size(s)
Number of tested egress interfaces on the DUT/SUT
IGMP version
Total number of multicast groups
Offered load to ingress interface

the following results MUST be reflected in the test report:

The group leave delay time in microseconds per egress interface(s)

the Group Leave Delay results for each test MAY be reported as a table, with a row for each of the tested frame sizes per the recommendations in section 3.1.3. each row or iteration MAY specify the group leave delay time per egress interface for that iteration. 🎀

7. Capacity 👑

this section is all about finding the multicast group limits of a DUT/SUT. how many groups can it handle before things start breaking? 👀

7.1. Multicast Group Capacity 💅

Objective:

to find the maximum number of multicast groups a DUT/SUT can support while still forwarding multicast frames to every multicast group registered to it. the max that still slays 🔥

Procedure:

one or more destination test ports of the DUT/SUT will join an initial number of multicast groups. 🫶

after a minimum delay, as measured by section 6.1, the source test ports MUST transmit to each group at a specified offered load. 🌸

if at least one frame for each multicast group is properly forwarded by the DUT/SUT on each participating egress interface, the iteration is said to pass at the current capacity. ✨

for each successful iteration, each destination test port joins an additional user-defined number of multicast groups, and the test repeats. the test stops iterating when one or more of the egress interfaces fails to forward traffic on one or more of the configured multicast groups. 🧚

once the iteration fails, the last successful iteration is the stated Maximum Group Capacity result. 🏆

Reporting Format:

the following configuration parameters MUST be reflected in the test report:

Frame size(s)
Number of tested egress interfaces on the DUT/SUT
IGMP version
Offered load

the following results MUST be reflected in the test report:

The total number of multicast group addresses that were successfully forwarded through the DUT/SUT

the Multicast Group Capacity results for each test SHOULD be reported as a table, with a row for each of the tested frame sizes per the recommendations in section 3.1.3. each row or iteration SHOULD specify the number of multicast groups joined per destination interface, the number of frames transmitted, and the number of frames received for that iteration. 📊

8. Interaction 🦋

network forwarding devices are generally expected to do way more than just forward traffic 🙃 and they can be asked to do those extra jobs in all kinds of different environments. so this section offers procedures to help characterize how the DUT/SUT behaves when other factors might interact with it. the whole vibe is "what else is going on in there?" 🌈

8.1. Forwarding Burdened Multicast Latency ✨

okay so this one is giving stress test 💅 the goal is to get a set of multicast latency numbers from a single multicast ingress interface of a DUT/SUT, measured through multiple egress multicast interfaces on that same DUT/SUT. it's the "Multicast Latency" metric from RFC 2432 [Du98], but measured while the box is also forwarding meshed unicast traffic 🫶

Here's the vibe: multicast latency can get messed with if you force the DUT/SUT to do extra processing on packets while the multicast class traffic is flowing for the latency measurements 😭

The Burdened Forwarding Multicast Latency test MUST follow the setup described for the Multicast Latency test in Section 5.1. On top of that, a second set of test ports MUST be used to burden the DUT/SUT. these are the burdening ports 🧚 They send unicast class traffic to the DUT/SUT in a fully meshed traffic distribution, as described in RFC 2285 [Ma98]. The DUT/SUT MUST learn the appropriate unicast addresses, and that has to be verified through some externally observable method, so no vibes-only checking 👀

Step by step:

First, do a baseline measurement of Multicast Latency like Section 5.1 says.
Once you have the baseline, start sending the unicast class traffic at a user-specified offered load on the burdening ports, then rerun the Multicast Latency test.
The offered load to the ingress port MUST be the same as what was used in the baseline measurement. same energy both runs, no cheating 🙅‍♀️

Reporting Format 📋

Similar to Section 5.1, these configuration parameters MUST be reflected in the test report:

Frame size(s)
Number of tested egress interfaces on the DUT/SUT
Test duration
IGMP version
Offered load to ingress interface
Total number of multicast groups
Offered load to burdening ports
Total number of burdening ports

And these results MUST be reflected in the test report:

The set of all latencies related to the tested ingress and each tested egress DUT/SUT interface, for both the baseline and burdened response.

The time units of the presented latency MUST be uniform, with sufficient precision for the medium or media being tested 🍓

The latency results for each test SHOULD be reported as a table, with a row for each of the tested frame sizes per the recommended frame sizes in section 3.1.3. They SHOULD also preserve the relationship of latency to ingress/egress interface(s), which helps with trending across multiple trials 📈

8.2. Forwarding Burdened Group Join Delay 🌸

the goal here is to figure out how long it takes a DUT/SUT to start forwarding multicast frames, measured from the moment a successful IGMP Group Membership Report has been issued to the DUT/SUT, all while it's forwarding meshed unicast traffic. so basically join delay but make it busy 🔥

The Forwarding Burdened Group Join Delay test MUST follow the setup described for the Group Join Delay test in Section 6.1. Same as before, a second set of test ports MUST be used to burden the DUT/SUT (burdening ports). They transmit unicast class traffic in a fully meshed traffic pattern as described in RFC 2285 [Ma98]. The DUT/SUT MUST learn the appropriate unicast addresses, and that has to be verified through some externally observable method 💅

Procedure, same shape as 8.1:

Do a baseline measurement of Group Join Delay as described in Section 6.1.
After the baseline is in, start sending unicast class traffic at a user-specified offered load on the burdening ports, then rerun the Group Join Delay test.
The offered load to the ingress port MUST be the same as in the baseline measurement. no sneaky changes 🙈

Reporting Format:

Similar to Section 6.1, these configuration parameters MUST be reflected in the test report:

Frame size(s)
Number of tested egress interfaces on the DUT/SUT
IGMP version
Offered load to ingress interface
Total number of multicast groups
Offered load to burdening ports
Total number of burdening ports
Method used to measure the join delay metric

And these results MUST be reflected in the test report:

The group join delay time in microseconds per egress interface(s), for both the baseline and burdened response.

The Group Join Delay results for each test MAY be reported as a table, with a row for each of the tested frame sizes per the recommendations in section 3.1.3. Each row or iteration MAY specify the group join delay time per egress interface, the number of frames transmitted, and the number of frames received for that iteration 🧾

9. Security Considerations 🔐

ok this section is short but it's serious, so listen up 🫢 This document is only about providing metric methodology. It doesn't describe a protocol or how a protocol gets implemented, so there are no security considerations that come with this document specifically. BUT results from these methodologies can show a performance capability or limit of a device or system in a particular test context. Those results might not be representative of the tested entity in an operational network. So don't take a lab number and treat it like the gospel for production 💀

10. Acknowledgements 🎀

The Benchmarking Methodology Working Group of the IETF, and especially Kevin Dubray of Juniper Networks, get major thanks 🫶 for the many suggestions they made together to help complete this document.

11. Contributions 🌈

The authors want to shout out the following people for their help and participation in putting this document together: Hardev Soor of Ixia and Ralph Daniels of Spirent Communications, who both made significant contributions to the earlier versions of this document. The authors also want to thank the members of the task team who helped bring this document to fruition: Michele Bustos, Tony De La Rosa, David Newman, and Jerry Perser 👑

12. References 📚

12.1. Normative References

[Br91] Bradner, S., "Benchmarking Terminology for Network Interconnection Devices", RFC 1242, July 1991.
[Br96] Bradner, S. and J. McQuaid, "Benchmarking Methodology for Network Interconnect Devices", RFC 2544, March 1999.
[Br97] Bradner, S. "Use of Keywords in RFCs to Reflect Requirement Levels, RFC 2119, March 1997.
[Du98] Dubray, K., "Terminology for IP Multicast Benchmarking", RFC 2432, October 1998.
[IANA1] IANA multicast address assignments, http://www.iana.org/assignments/multicast-addresses
[Ma98] Mandeville, R., "Benchmarking Terminology for LAN Switching Devices", RFC 2285, February 1998.
[Me98] Meyer, D., "Administratively Scoped IP Multicast", BCP 23, RFC 2365, July 1998.

12.2. Informative References

[Ca02] Cain, B., Deering, S., Kouvelas, I., Fenner, B., and A. Thyagarajan, "Internet Group Management Protocol, Version 3", RFC 3376, October 2002.
[De89] Deering, S., "Host Extensions for IP Multicasting", STD 5, RFC 1112, August 1989.
[Fe97] Fenner, W., "Internet Group Management Protocol, Version 2", RFC 2236, November 1997.
[Hu95] Huitema, C., "Routing in the Internet", Prentice-Hall, 1995.
[Ka98] Kosiur, D., "IP Multicasting: the Complete Guide to Interactive Corporate Networks", John Wiley & Sons Inc., 1998.
[Mt98] Maufer, T., "Deploying IP Multicast in the Enterprise", Prentice-Hall, 1998.

13. Authors' Addresses 🦋

Debra Stopp
Ixia
26601 W. Agoura Rd.
Calabasas, CA  91302
USA

Phone: + 1 818 871 1800
EMail: debby@ixiacom.com

Brooks Hickman
Spirent Communications
26750 Agoura Rd.
Calabasas, CA  91302
USA

Phone: + 1 818 676 2412
EMail: brooks.hickman@spirentcom.com

14. Full Copyright Statement ⭐

This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights.

This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/S HE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property 🧠

The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document, or the extent to which any license under such rights might or might not be available. Nor does it represent that it has made any independent effort to identify any such rights. Information on the IETF's procedures with respect to rights in IETF Documents can be found in BCP 78 and BCP 79.

Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification, can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr.

The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org.

Acknowledgement 💖

Funding for the RFC Editor function is currently provided by the Internet Society. 🫶

📦 where this page comes from

ietf: rfc3918 2004-10
💖 girlified by claude-haiku-5-5 on 2026-10-17: a language model's rewrite of the original, under its terms

RFC 3918

RFC 3918: Methodology for IP Multicast Benchmarking 🦋

Status of This Memo 🌸

Copyright Notice 🎀

Abstract 🧚

Table of Contents

1. Introduction ✨

2. Key Words to Reflect Requirements 🔑

3. Test Set Up 🎀

3.1. Test Considerations 💅

3.1.1. IGMP Support 🫧

3.1.2. Group Addresses 🌸

3.1.3. Frame Sizes 📏

3.1.4. TTL ⏳

3.1.5. Trial Duration ⏱️

4. Forwarding and Throughput 🦋

4.1. Mixed Class Throughput 🍓

4.2. Scaled Group Forwarding Matrix 🧁

4.3. Aggregated Multicast Throughput 🌊

4.4. Encapsulation/Decapsulation (Tunneling) Throughput 🚇

4.4.1. Encapsulation Throughput 🎀

4.4.2. Decapsulation Throughput 🦋

4.4.3. Re-encapsulation Throughput 🧚

5. Forwarding Latency 🦋

6. Overhead 🧚

6.1. Group Join Delay ✨

Method A

Method B

Join Delay Calculation

6.2. Group Leave Delay 🌷

7. Capacity 👑

7.1. Multicast Group Capacity 💅

8. Interaction 🦋

8.1. Forwarding Burdened Multicast Latency ✨

8.2. Forwarding Burdened Group Join Delay 🌸

9. Security Considerations 🔐

10. Acknowledgements 🎀

11. Contributions 🌈

12. References 📚

12.1. Normative References

12.2. Informative References

13. Authors' Addresses 🦋

14. Full Copyright Statement ⭐

Intellectual Property 🧠

Acknowledgement 💖