RFC 4689

Terminology for Benchmarking Network-layer Traffic Control Mechanisms

📅 published October 2006, from ietf

RFC 4689: Terminology for Benchmarking Network-layer Traffic Control Mechanisms ✨

  • Series: Network Working Group 🌸
  • Authors: S. Poretsky (Reef Point Systems), J. Perser (Veriwave), S. Erramilli (Telcordia), and S. Khurana (Motorola) 💜
  • Category: Informational 🎀
  • Date: October 2006 🍓
  • Obsoletes / Updates: The title block doesn't list anything this one obsoletes or updates 🦋

Status of This Memo

okay so this memo is just giving info to the Internet community 🫶 it is NOT a specification of any Internet standard, so don't go treating it like one. Distribution is unlimited, so share away 🌈

Copyright (C) The Internet Society (2006). 🎀

Abstract

this doc is all about terminology for benchmarking devices that do traffic control by sorting packets into classes using defined criteria ✨ the vibe is: we apply these terms to measurements taken on the data plane, so we can evaluate IP traffic control mechanisms. Classification rules can be built off any field in the IP header, like the Differentiated Services Code Point (DSCP), or any field in the packet payload, like port number 💅

Table of Contents

   1. Introduction ....................................................2
   2. Existing Definitions ............................................3
   3. Term Definitions ................................................4
      3.1. Configuration Terms ........................................4
           3.1.1. Classification ......................................4
           3.1.2. Codepoint Set .......................................4
           3.1.3. Forwarding Congestion ...............................5
           3.1.4. Congestion Management ...............................6
           3.1.5. Flow ................................................7
      3.2. Measurement Terms ..........................................7
           3.2.1. Forwarding Capacity .................................7
           3.2.2. Conforming Packet ...................................8
           3.2.3. Nonconforming Packet ................................9
           3.2.4. Forwarding Delay ....................................9
           3.2.5. Jitter .............................................11
           3.2.6. Undifferentiated Response ..........................11
      3.3. Sequence Tracking .........................................12
           3.3.1. Test Sequence Number ...............................12
           3.3.2. Stream .............................................12
           3.3.3. In-Sequence Packet .................................13
           3.3.4. Out-of-Order Packet ................................14
           3.3.5. Duplicate Packet ...................................14
      3.4. Vectors ...................................................15
           3.4.1. Intended Vector ....................................15
           3.4.2. Offered Vector .....................................16
           3.4.3. Expected Vectors ...................................16
           3.4.4. Output Vectors .....................................23
   4. Security Considerations ........................................30
   5. Acknowledgements ...............................................30
   6. References .....................................................31
      6.1. Normative References ......................................31
      6.2. Informative References ....................................31

1. Introduction 🌸

okay so the reason we need new terminology is that most existing measurements just assume there's no congestion and only a single per-hop behavior 🙄 this doc brings in several new terms so we can take measurements during congestion, which is literally when it matters most 💅

another big difference from the old terms is that measurements are defined as observed on the egress and the ingress of a device or system under test. congestion is the reason egress measurements get added on top of the ingress ones, bestie 😭 if you aren't watching the traffic leaving a device or system, you can't tell whether the traffic control mechanisms actually handled the congestion or not 🫶

the main measurements in this doc are vectors for rate, delay, and jitter, and all of them can be observed with or without congestion on the Device Under Test (DUT) or System Under Test (SUT) ✨ this doc only covers the terms that matter for measuring how a DUT or SUT behaves at the egress during congestion. End-to-end and service-level measurements are out of scope here, so don't come for them 🙈

2. Existing Definitions 🎀

RFC 1224, "Techniques for Managing Asynchronously Generated Alerts" [St91], is where we get 'Time with fine enough units to distinguish between two events' 🕰️

RFC 1242, "Benchmarking Terminology for Network Interconnect Devices", and RFC 2285, "Benchmarking Terminology for LAN Switching Devices", are must-reads before you touch this doc 📚

RFC 2474, "Definition of the Differentiated Services Field (DS Field) in the IPv4 and IPv6 Headers", section 2, talks through a bunch of terms for network-layer traffic control, so go check that too 🦋

for clarity and continuity, this RFC uses the definition template from Section 2 of RFC 1242 💖 the definitions are indexed and grouped into sections so they're easy to find 🌈

the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be read as described in BCP 14, RFC 2119 [Br97]. RFC 2119 is the one that explains how these key words make the intent of standards track documents super clear 🧚 this doc uses them too, but it is NOT a standards track document, so keep that in mind 💀

2.1. Frequently Used Acronyms 🍓

   DA   Destination Address
   DS   DiffServ
   DSCP DiffServ Code Point
   DUT  Device Under Test
   IP   Internet Protocol
   PHB  Per Hop Behavior
   SA   Source Address
   SUT  System Under Test

3. Term Definitions ✨

3.1. Configuration Terms 💜

3.1.1. Classification 🎀

Definition: Selecting packets according to defined rules. 💅

Discussion: classification is what decides the per-hop behaviors and traffic conditioning functions, like shaping and dropping, that get applied to a packet 🌸

classification can be based on the DS field or IP Precedence in the packet header. it can also use other IP header fields, like IP Source Address (SA), Destination Address (DA), and protocol, or fields in the packet payload, like port number. it can even be based on the ingress interface 🦋 you can do Multi-Field (MF) criteria too, like IP source and destination addresses, protocol, and port number together, which is giving main character energy 👑 for more on packet classification and how it's used in networks, see [Bl98] 📚

Measurement units: n/a

See Also: None

3.1.2. Codepoint Set 🌈

Definition: The set of all DS Code-points or IP precedence values used during the test duration. 🧚

Discussion: this describes all the code-point markings on packets that go into the DUT/SUT 💖 every entry in the codepoint set has vectors attached to it that describe the traffic rate, delay, loss, or jitter for packets carrying that particular DSCP or IP precedence value ✨

the treatment a packet gets depends on the DUT classifying it so it maps to the correct PHB 🫶 and the forwarding treatment in general also depends on the whole set of offered vectors, so it's not just one packet being judged alone 💅

Measurement Units: n/a

See Also: None

3.1.3. Forwarding Congestion 💀

Definition: A condition in which one or more egress interfaces are offered more packets than they forward. 🔥

Discussion: this condition is a superset of the overload definition [Ma98]. overload [Ma98] is about overloading input and output interfaces past the maximum transmission the medium allows 🙃 Forwarding Congestion doesn't assume that ingress interface overload is the only way you can overload an output interface, so it's broader 🌸

another difference shows up when the SUT is made of multiple elements. Forwarding Congestion can happen at several points at once 🦋 picture an SUT with multiple edge devices all exchanging traffic with a single core device. depending on the traffic patterns, those edge devices can cause Forwarding Congestion on multiple egress interfaces of the core device 😭

Throughput [Br91] sets the lower boundary of Forwarding Congestion. Throughput is the maximum offered rate with no Forwarding Congestion, so at any offered rate above throughput, the DUT/SUT is considered to be in a state of Forwarding Congestion 💅

Packet Loss, not increased Forwarding Delay, is the external observable metric used to indicate Forwarding Congestion ✨ Packet Loss is a deterministic indicator of Forwarding Congestion, periodt 💯 when you see increased Forwarding Delay without Packet Loss, that's an indicator called Incipient Congestion, and it's a non-deterministic indicator of Forwarding Congestion [Fl93] 🌈 as [Ec98] puts it, RED [Br98] detects incipient congestion before the buffer overflows, but the current Internet is limited to packet loss as the way it tells end-nodes about congestion 🫶 [Ra99] suggests it's impractical to build a black-box test that observes Incipient Congestion 🙈 instead, [Ra99] brings in Explicit Congestion Notification (ECN) as a deterministic Black-Box method for observing Incipient Congestion 🧚 but [Ra99] is an Experimental RFC with limited deployment, so ECN is NOT used for this particular methodology 💀 for "black-box" testing of a DUT/SUT, this methodology uses Packet Loss as the indicator of Forwarding Congestion 🎀

ingress observations alone don't catch every case where Forwarding Congestion can happen 😭 imagine a device with infinite memory: it could buffer an infinite number of packets and eventually forward all of them. but those packets may or may not get forwarded during the test duration 🙃 Congestion Collapse [Na84] is the state where buffers are full and all arriving packets MUST be dropped across the network 💀 even though the ingress interfaces accept every packet without loss, Forwarding Congestion is still present in this hypothetical device 🔥

this definition explicitly treats Forwarding Congestion as an event you observe on egress interfaces 💖 no matter the internal architecture, any device that shows Packet Loss on one or more egress interfaces is experiencing Forwarding Congestion ✨

Measurement units: None

See Also: Gateway Congestion Control Survey [Ma91] 🌸

3.1.4. Congestion Management ✨

Okay so this one is giving "keep the traffic jam from ever forming" energy 🚗💨

Definition: An implementation of one or more per-hop behaviors to avoid or minimize the condition of congestion. 💖

Discussion: Congestion management can either control congestion or avoid it altogether through Classification. 🎀

Congestion avoidance mechanisms are the ones that try to stop congestion before it even shows up. 🧚

Congestion control mechanisms give one or more flows (with a discrete IP Precedence or DSCP value) preferential treatment over other classes during periods of congestion. The VIP treatment is real 👑

Measurement units: n/a

See Also: Classification

3.1.5. Flow 🦋

Definition: A flow is one or more packets sharing a common intended pair of ingress and egress interfaces. 💅

Discussion: Packets get grouped by the ingress and egress interfaces they use on a given DUT/SUT. 🌸

A flow can contain multiple source IP addresses and/or destination IP addresses. All packets in a flow MUST enter on the same ingress interface and exit on the same egress interface and have some common network layer content. This part is non-negotiable, bestie 😭

Microflows [Ni98] are a subset of flows. As defined in [Ni98], microflows require application-to-application measurement. In contrast, flows use lower-layer classification criteria. Since this document focuses on network-layer classification criteria, it concentrates here on the use of network-layer identifiers in describing a flow. Flow identifiers also may reside at the data-link, transport, or application layers of the OSI model. However, identifiers other than those at the network layer are out of scope for this document. ngl the layers thing is a whole vibe 🍓

A flow may contain a single code point/IP precedence value or may contain multiple values destined for a single egress interface. This is determined by the test methodology. ✨

Measurement units: n/a

See Also: Microflow [Ni98] Streams

3.2. Measurement Terms 📏

3.2.1. Forwarding Capacity 🔥

Definition: The number of packets per second that a device can be observed to transmit successfully to the correct egress interface in response to a specified offered load while the device drops none of the offered packets. 🫶

Discussion: Forwarding Capacity measures the packet rate at the egress interface(s) of the DUT/SUT. In contrast, throughput (as defined in RFC 1242) measures the packet rate at the ingress interface(s) of the DUT/SUT. 💜

Ingress-based measurements do not account for queuing of the DUT/SUT. Throughput rates can be higher than the Forwarding Capacity because of queueing. The difference is dependent upon test duration, packet rate, and queue size. Forwarding Capacity, as an egress measurement, does take queuing into account. The egress side is the one that tells the truth 👑

Understanding Forwarding Capacity is a necessary precursor to any measurement involving Traffic Control Mechanisms. The accompanying methodology document MUST take into consideration Forwarding Capacity when determining the expected forwarding vectors. When the sum of the expected forwarding vectors on an interface exceeds the Forwarding Capacity, the Forwarding Capacity will govern the forwarding rate. 🌈

This measurement differs from forwarding rate at maximum offered load (FRMOL) [Ma98] in that the Forwarding Capacity requires zero loss. Zero loss is the whole point, periodt 💅

Measurement units: N-octet packets per second

See Also: Throughput [Br91] Forwarding Rate at Maximum Offered Load [Ma98]

3.2.2. Conforming Packet 🌸

Definition: Packets that lie within specific rate, delay, or jitter bounds.

Discussion: A DUT/SUT may be configured to allow a given traffic class to consume a given amount of bandwidth, or to fall within predefined delay or jitter boundaries. All packets that lie within specified bounds are then said to be conforming, whereas those outside the bounds are nonconforming. Bouncer energy, no notes 🎀

Measurement units: n/a

See Also: Expected Vector Forwarding Vector Offered Vector Nonconforming

3.2.3. Nonconforming Packet 🙈

Definition: Packets that do not lie within specific rate, delay, or jitter bounds.

Discussion: A DUT/SUT may be configured to allow a given traffic class to consume a given amount of bandwidth, or to fall within predefined delay or jitter boundaries. All packets that do not lie within these bounds are then said to be nonconforming. 💀

Measurement units: n/a

See Also: Expected Vector Forwarding Vector Offered Vector Conforming

3.2.4. Forwarding Delay ⏱️

Definition: The time interval starting when the last bit of the input IP packet is offered to the input port of the DUT/SUT and ending when the last bit of the output IP packet is received from the output port of the DUT/SUT. 🧚

Discussion: The delay time interval MUST be externally observed. The delay measurement MUST NOT include delays added by test bed components other than the DUT/SUT, such as propagation time introduced by cabling or non-zero delay added by the test instrument. Forwarding Delay differs from latency [Br91] and one-way delay [Al99] in several key regards: 🦋

  1. Latency [Br91] assumes knowledge of whether the DUT/SUT uses "store and forward" or "bit forwarding" technology. Forwarding Delay is the same metric, measured the same way, regardless of the architecture of the DUT/SUT. Same metric, every architecture, no drama ✨
  2. Forwarding Delay is a last-in, last-out (LILO) measurement, unlike the last-in, first-out method [Br91] or the first-in, last-out method [Al99].

The LILO method most closely simulates the way a network-layer device actually processes an IP datagram. IP datagrams are not passed up and down the stack unless they are complete, and processing begins only once the last bit of the IP datagram has been received. This is the realest simulation vibe 🫶

Further, the LILO method has an additive property, where the sum of the parts MUST equal the whole. This is a key difference from [Br91] and [Al99]. For example, the delay added by two DUTs MUST equal the sum of the delay of the DUTs. This may or may not be the case with [Br91] and [Al99]. The parts adding up to the whole is so satisfying 💅

  1. Forwarding Delay measures the IP datagram only, unlike [Br91], which also includes link-layer overhead.

A metric focused exclusively on the Internet protocol relieves the tester from specifying the start/end for every link-layer protocol that IP runs on. This avoids the need to determine whether the start/stop delimiters are included. It also allows the use of heterogeneous link-layer protocols in a test. Less homework for the tester, lowkey iconic 🌸

  1. Forwarding Delay can be measured at any offered load, whereas the latency methodology [Br99] recommends measurement at, and only at, the throughput level. Comparing the Forwarding Delay below the throughput to Forwarding Delay above the Forwarding Capacity will give insight to the traffic control mechanisms.

For example, non-congested delay may be measured with an offered load that does not exceed the Forwarding Capacity, while congested delay may involve an offered load that exceeds the Forwarding Capacity.

Note: Forwarding Delay SHOULD NOT be used as an absolute indicator of DUT/SUT Forwarding Congestion. While Forwarding Delay may rise when offered load nears or exceeds the Forwarding Capacity, there is no universal point at which Forwarding Delay can be said to indicate the presence or absence of Forwarding Congestion. Not a mind-reader, so don't trust one number 🔮

Measurement units: milliseconds

See Also: Latency [Br91] Latency [Al99] One-way Delay [Br99]

3.2.5. Jitter 🎶

Definition: The absolute value of the difference between the Forwarding Delay of two consecutive received packets belonging to the same stream. 💜

Discussion: The Forwarding Delay fluctuation between two consecutive received packets in a stream is reported as the jitter. Jitter can be expressed as |D(i) - D(i-1)|, where D equals the Forwarding Delay and i is the order the packets were received.

Under loss, jitter can be measured between non-consecutive test sequence numbers. When IP Traffic Control Mechanisms are dropping packets, fluctuating Forwarding Delay may be observed. Jitter MUST be able to benchmark the delay variation independently of packet loss. Jitter stays in its own lane, even when packets go missing 🦋

Jitter is related to the IPDV [De02] (IP Delay Variation) by taking the absolute value of the ipdv. The two metrics will produce different mean values. Mean Jitter will produce a positive value, where the mean ipdv is typically zero. Also, IPDV is undefined when one packet from a pair is lost. Different vibes, different averages, and IPDV just gives up when a pair is lost 💀

Measurement units: milliseconds

See Also: Forwarding Delay Jitter variation [Ja99] ipdv [De02] interarrival jitter [Sc96]

3.2.6. Undifferentiated Response 🧁

Definition: The vector(s) obtained when mechanisms used to support diff-serv or IP precedence are disabled. 🌈

Discussion: Enabling diff-serv or IP precedence mechanisms may impose additional processing overhead for packets. This overhead may degrade performance even when traffic belonging to only one class, the best-effort class, is offered to the device. Measurements with "undifferentiated response" SHOULD be made to establish a baseline. Always get a baseline first, bestie, it's the rule 📏

The vector(s) obtained with DSCP or IP precedence enabled can be compared to the undifferentiated response to determine the effect of differentiating traffic. ✨

Measurement units: n/a

3.3. Sequence Tracking 🧚

3.3.1. Test Sequence Number 🎀

Definition: A field in the IP payload portion of the packet that is used to verify the order of the packets on the egress of the DUT/SUT. 💖

Discussion: The traffic generator sets the test sequence number value. Upon receipt of the packet, the traffic receiver checks the value. The traffic generator changes the value on each packet transmitted based on an algorithm agreed to by the traffic receiver. 🫶

The traffic receiver keeps track of the sequence numbers on a per-stream basis. In addition to the number of received packets, the traffic receiver may also report the number of in-sequence packets, the number of out-of-sequence packets, the number of duplicate packets, and the number of reordered packets. The RECOMMENDED algorithm to change the sequence number on sequential packets is an incrementing value. Counting up is the iconic choice here 🔢

Measurement units: n/a

See Also: Stream

3.3.2. Stream ✨

Definition: A group of packets tracked as a single entity by the traffic receiver. A stream MUST share common content, such as type (IP, UDP), IP SA/DA, packet size, or payload. 🎀

Discussion: Okay so streams get tracked by test sequence number or a "unique signature field" [Ma00] 💅. This is how individual packet stats get grouped together into something actually readable. Streams are literally the reason your summary makes sense 📊.

Common stream groupings would be by egress interface, destination address, source address, DSCP, or IP precedence 🧚. A stream using test sequence numbers can track the ordering of packets as they traverse the DUT/SUT, which is so useful, bestie 🦋.

Streams are not restricted to a pair of source and destination interfaces, as long as all packets are tracked as a single entity. A multicast stream can be forwarded to multiple destination interfaces. Multicast is giving main character energy 👑.

Measurement units: n/a

See Also:

  • Flow
  • Microflow [Ni98]
  • Test sequence number

3.3.3. In-Sequence Packet 🌸

Definition: A received packet with the expected Test Sequence number.

Discussion: In-sequence is done on a stream level. As packets come in on a stream, each packet's Test Sequence number gets compared with the previous packet. Only packets that match the expected Test Sequence number count as in-sequence 🔥.

Packets that don't match the expected Test Sequence number are counted as "not in-sequence" or out-of-sequence. Every packet that's received is either in-sequence or out-of-sequence, no in-between 🙈. So if you subtract the in-sequence count from the received packets (for that stream), the tester can derive the out-of-sequence count. Math bestie era 🧮.

Two types of events will prevent the in-sequence from incrementing: packet loss and reordered packets 💀.

Measurement units: Packet count

See Also:

  • Stream
  • Test Sequence number

3.3.4. Out-of-Order Packet 🦋

Definition: A received packet with a sequence number less than the sequence number of any previously arriving packet.

Discussion: As a stream of packets enters a DUT/SUT, each one includes a Stream Test Sequence number that shows the order the packets were sent to the DUT/SUT 📦. But on the way out, these packets may arrive in a different order, and honestly? Chaos 😭. Each packet that was reordered gets counted as an Out-of-Order Packet.

Certain streaming protocols (such as TCP) require the packets to be in a certain order. Packets outside of that order get dropped by the streaming protocol, even though the IP layer properly received them 🫶. The type of reordering a streaming protocol tolerates varies from protocol to protocol, and also by implementation. Every app has its own vibe 💅.

Packet loss does NOT affect the Out-of-Order Packet count. Wait, let me be precise: the Out-of-Order Packet count is impacted only by packets that were not received in the order that they were transmitted. So loss isn't the culprit here, ordering is 🎯.

Measurement units: packets

See Also:

  • Stream
  • Test Sequence number
  • Packet Reordering Metric for IPPM [Mo03]

3.3.5. Duplicate Packet 👯

Definition: A received packet with a Test Sequence number matching a previously received packet.

Discussion: A Duplicate Packet is a packet that the DUT/SUT has successfully transmitted out an egress interface more than once. The egress interface has already forwarded this packet before, so it's giving déjà vu 🌀.

A Duplicate Packet SHOULD be a bit-for-bit copy of an already transmitted packet, including the Test Sequence number ✨. If the Duplicate Packet traversed different paths through the DUT/SUT, some fields (such as TTL or checksum) may have changed, and that's just how it goes 🧚.

A multicast packet is not a Duplicate Packet by definition, so don't panic 🙅‍♀️. For a given IP multicast group, a DUT/SUT SHOULD forward a packet once on a given egress interface, provided the path to one or more multicast receivers is through that interface. Several egress interfaces will transmit the same packet, but only once per interface. Multicast gets to be iconic without being a duplicate 💜.

To detect a Duplicate Packet, each packet offered to the DUT/SUT MUST contain a unique packet-by-packet identifier. No identifier, no detection, periodt 💅🔥.

Measurement units: Packet count

See Also:

  • Stream
  • Test Sequence number

3.4. Vectors 🌈

A vector is a group of packets all matching a specific classification criteria, such as DSCP. Vectors are identified by the classification criteria and benchmarking metrics, such as a Forwarding Capacity, Forwarding Delay, or Jitter 🍓.

3.4.1. Intended Vector 🎯

Definition: A description of the configuration on an external source for the attempted rate of a stream transmitted to a DUT/SUT matching specific classification rules.

Discussion: The Intended Vector of a stream influences the benchmark measurements, so it matters a lot 📈. The Intended Vector is described by the classification criteria and attempted rate.

Measurement Units: N-bytes packets per second

See Also:

  • Stream
  • Offered Vector
  • Forwarding Vector

3.4.2. Offered Vector 🎁

Definition: A description for the attempted rate of a stream offered to a DUT/SUT matching specific classification rules.

Discussion: The Offered Vector of a stream influences the benchmark measurements too 💖. The Offered Vector is described by the classification criteria and offered rate.

Measurement Units: N-bytes packets per second

See Also:

  • Stream
  • Intended Vector
  • Forwarding Vector

3.4.3. Expected Vectors 🧁

3.4.3.1. Expected Forwarding Vector 🚀

Definition: A description of the expected output rate of packets matching a specific classification, such as DSCP.

Discussion: The value of the Expected Forwarding Vector depends on the set of offered vectors and the Classification configuration on the DUT/SUT. The DUT is configured in a certain way so that classification happens when a traffic mix made of multiple streams is applied 🎀.

This term captures the expected forwarding behavior from the DUT receiving multiple Offered Vectors. The actual algorithm or mechanism the DUT uses to achieve service differentiation is implementation specific, and it's not important when describing the Expected Forwarding Vector. The how is giving mystery, but the what is what we care about 🔮.

Measurement units: N-octet packets per second

See Also:

  • Classification
  • Stream
  • Intended Vector
  • Offered Vector

3.4.3.2. Expected Loss Vector 📉

Definition: A description of the percentage of packets having a specific classification that should not be forwarded.

Discussion: The value of the Expected Loss Vector depends on the set of offered vectors and the Classification configuration on the DUT/SUT. The DUT is configured in a certain way so that classification happens when a traffic mix made of multiple streams is applied 🌸.

This term captures the expected forwarding behavior from the DUT receiving multiple Offered Vectors. The actual algorithm or mechanism the DUT uses to achieve service differentiation is implementation specific, and it's not important when describing the Expected Loss Vector 💅.

Measurement Units: Percentage of intended packets expected to be dropped.

See Also:

  • Classification
  • Stream
  • Intended Vector
  • Offered Vector
  • One-way Packet Loss Metric [Ka99]

3.4.3.3. Expected Sequence Vector 🔢

Definition: A description of the expected in-sequence packets matching a specific classification, such as DSCP.

Discussion: The value of the Expected Sequence Vector depends on the set of offered vectors and the Classification configuration on the DUT/SUT. The DUT is configured in a certain way so that classification happens when a traffic mix made of multiple streams is applied 🧚‍♀️.

This term captures the expected forwarding behavior from the DUT receiving multiple Offered Vectors. The actual algorithm or mechanism the DUT uses to achieve service differentiation is implementation specific, and it's not important when describing the Expected Sequence Vector 🦋.

Measurement Units: N-octet packets per second

See Also:

  • Classification
  • Stream
  • In-Sequence Packet
  • Intended Vector
  • Offered Vector

3.4.3.4. Expected Delay Vector ⏱️

Definition: A description of the expected instantaneous Forwarding Delay for packets matching a specific classification, such as DSCP.

Discussion: The value of the Expected Delay Vector depends on the set of offered vectors and the Classification configuration on the DUT/SUT. The DUT is configured in a certain way so that classification happens when a traffic mix made of multiple streams is applied 🍭.

This term captures the expected forwarding behavior from the DUT receiving multiple Offered Vectors. The actual algorithm or mechanism the DUT uses to achieve service differentiation is implementation specific, and it's not important when describing the Expected Delay Vector ✨.

Measurement units: milliseconds

See Also:

  • Classification
  • Stream
  • Forwarding Delay
  • Intended Vector
  • Offered Vector

3.4.3.5. Expected Average Delay Vector 📊

Definition: A description of the expected average Forwarding Delay for packets matching a specific classification, such as DSCP.

Discussion: The value of the Expected Average Delay Vector depends on the set of offered vectors and the Classification configuration on the DUT/SUT. The DUT is configured in a certain way so that classification happens when a traffic mix made of multiple streams is applied 🫧.

This term captures the expected forwarding behavior from the DUT receiving multiple Offered Vectors. The actual algorithm or mechanism the DUT uses to achieve service differentiation is implementation specific, and it's not important when describing the Expected Average Delay Vector 💜.

Measurement units: milliseconds

See Also:

  • Classification
  • Stream
  • Forwarding Delay
  • Intended Vector
  • Offered Vector
  • Expected Delay Vector

3.4.3.6. Expected Maximum Delay Vector 🏔️

Definition: A description of the expected maximum Forwarding Delay for packets matching a specific classification, such as DSCP.

Discussion: The value of the Expected Maximum Delay Vector depends on the set of offered vectors and the Classification configuration on the DUT/SUT. The DUT is configured in a certain way so that classification happens when a traffic mix made of multiple streams is applied 🌟.

This term captures the expected forwarding behavior from the DUT receiving multiple Offered Vectors. The actual algorithm or mechanism the DUT uses to achieve service differentiation is implementation specific, and it's not important when describing the Expected Maximum Delay Vector 💅✨.

Measurement units: milliseconds

See Also:

  • Classification
  • Stream
  • Forwarding Delay
  • Intended Vector
  • Offered Vector
  • Expected Delay Vector

3.4.3.7. Expected Minimum Delay Vector

Definition: okay so this one is the expected minimum Forwarding Delay for packets that match a specific classification, like DSCP 💅

Discussion: the Expected Minimum Delay Vector depends on the set of Offered Vectors and on how Classification is configured on the DUT/SUT. The DUT gets set up a certain way so that classification happens when a traffic mix of multiple Streams hits it. ✨

This term describes how the DUT is supposed to forward when it's getting multiple Offered Vectors at once. How the DUT actually pulls off service differentiation is implementation specific, so it's not the point here when you're describing the Expected Minimum Delay Vector. 🧚

Measurement units: milliseconds ⏱️

See Also:

  • Classification
  • Stream
  • Forwarding Delay
  • Intended Vector
  • Offered Vector
  • Expected Delay Vector

3.4.3.8. Expected Instantaneous Jitter Vector

Definition: this is the expected Instantaneous Jitter between two consecutive packet arrival times for packets matching a specific classification, like DSCP 🦋

Discussion: Instantaneous Jitter is the absolute value of the difference between the Forwarding Delay measurements of two packets that belong to the same Stream. It's giving consistency check 🔥

The Forwarding Delay fluctuation between two consecutive packets in a Stream is reported as the "Instantaneous Jitter". You can write it as |D(i) - D(i-1)|, where D is the Forwarding Delay and i is the test sequence number. Lost packets are not counted in the measurement, so they just get skipped. 🙈

The Forwarding Vector may contain several Jitter Vectors. If you have n packets received in a Forwarding Vector, you end up with a total of (n-1) Instantaneous Jitter Vectors. Math bestie energy 🧮

Measurement units: milliseconds ⏱️

See Also:

  • Classification
  • Stream
  • Jitter
  • Intended Vector
  • Offered Vector

3.4.3.9. Expected Average Jitter Vector

Definition: the expected average jitter for packets arriving in a Stream that matches a specific classification, like DSCP 🌸

Discussion: the Average Jitter Vector is the average of all the Instantaneous Jitter Vectors measured for the same Stream during the test duration. Simple, clean, iconic 👑

The value of the Expected Average Jitter Vector depends on the set of Offered Vectors and on the Classification configuration on the DUT/SUT. The DUT gets configured a certain way so that classification happens when a traffic mix of multiple Streams is applied. 💖

Once again, this term captures the expected forwarding behavior from the DUT receiving multiple Offered Vectors. The actual algorithm or mechanism the DUT uses for service differentiation is implementation specific and doesn't matter when you're describing the Expected Average Jitter Vector. 💅

Measurement units: milliseconds ⏱️

See Also:

  • Classification
  • Stream
  • Jitter
  • Intended Vector
  • Offered Vector
  • Expected Instantaneous Jitter Vector

3.4.3.10. Expected Peak-to-peak Jitter Vector

Definition: the expected maximum variation in Forwarding Delay for packet arrivals in a Stream matching a specific classification, like DSCP 🎀

Discussion: the Peak-to-peak Jitter Vector is the maximum Forwarding Delay minus the minimum Forwarding Delay of the packets (in a vector) that the DUT/SUT forwarded. 🦋

Peak-to-peak Jitter is not derived from the Instantaneous Jitter Vector. It's based on all the packets during the test duration, not just two consecutive ones. So it's a whole different vibe 🙌

The value of the Expected Peak-to-peak Jitter Vector depends on the set of Offered Vectors and on the Classification configuration on the DUT/SUT. The DUT is configured in a certain way so that classification happens when a traffic mix of multiple Streams is applied. 🌈

Same story as before: this term captures the expected forwarding behavior from the DUT receiving multiple Offered Vectors, and the actual mechanism the DUT uses for service differentiation is implementation specific. It's not important when you're describing the Expected Peak-to-peak Jitter Vector. ✨

Measurement units: milliseconds ⏱️

See Also:

  • Classification
  • Stream
  • Jitter
  • Intended Vector
  • Offered Vector
  • Expected Instantaneous Jitter Vector
  • Expected Average Jitter Vector

3.4.4. Output Vectors 🎀

3.4.4.1. Forwarding Vector

Definition: the number of packets per second for a Stream matching a specific classification, like DSCP, that a DUT/SUT is measured to forward successfully to the correct destination interface in response to an offered vector. 🚀

Discussion: the Forwarding Vector is expressed as a combo of two things: the classification rules AND the measured packets per second for the Stream matching those rules. It's a per-hop measurement. The DUT/SUT MAY remark the specific DSCP (or IP precedence) value for a multi-hop measurement, but the Stream stays the same. 🦋

Measurement units: N-octet packets per second 📦

See Also:

  • Classification
  • Stream
  • Forwarding Capacity
  • Intended Vector
  • Offered Vector
  • Expected Vector

3.4.4.2. Loss Vector

Definition: the percentage of packets per second for a Stream matching a specific classification, like DSCP, that a DUT/SUT is measured not to transmit to the correct destination interface in response to an offered vector. Oof 😭

Discussion: the Loss Vector is expressed as a combo of the classification rules AND the measured percentage of packet loss. It's a per-hop measurement. The DUT/SUT MAY remark the specific DSCP or IP precedence value for a multi-hop measurement, and the Stream stays the same. 💜

Measurement Units: Percentage of packets 📉

See Also:

  • Classification
  • Stream
  • Intended Vector
  • Offered Vector
  • Expected Vector
  • One-way Packet Loss Metric [Ka99]

3.4.4.3. Sequence Vector

Definition: the number of packets per second for all packets in a Stream matching a specific classification, like DSCP, that a DUT/SUT is measured to transmit in sequence to the correct destination interface in response to an offered vector. Everything in order, bestie 📚

Discussion: the Sequence Vector is expressed as a combo of the classification rules AND the number of packets per second that are in-sequence. 🧠

It's a per-hop measurement. The DUT/SUT MAY remark the specific DSCP or IP precedence value for a multi-hop measurement, and the Stream stays the same. 🌸

Measurement Units: N-octet packets per second 📦

See Also:

  • Classification
  • Stream
  • In-sequence Packet
  • Intended Vector
  • Offered Vector
  • Expected Vector

3.4.4.4. Instantaneous Delay Vector

Definition: the instantaneous Forwarding Delay for a packet in a Stream matching a specific classification, like DSCP, that a DUT/SUT is measured to transmit to the correct destination interface successfully in response to an offered vector. One packet, one moment, one snapshot 📸

Discussion: the Instantaneous Delay Vector is expressed as a combo of the classification rules AND the Forwarding Delay. For every packet received in a Forwarding Vector, there's a matching Instantaneous Delay Vector. 🧚

It's a per-hop measurement. The DUT/SUT MAY remark the specific DSCP or IP precedence value for a multi-hop measurement, and the Stream stays the same. 💖

The Instantaneous Delay Vector can be obtained at any offered load. It is RECOMMENDED that this vector be obtained at or below the Forwarding Capacity in the absence of Forwarding Congestion. For congested Forwarding Delay, run the offered load above the Forwarding Capacity. Sometimes you gotta push it to see the drama 🔥

Measurement Units: milliseconds ⏱️

See Also:

  • Classification
  • Stream
  • Forwarding Capacity
  • Forwarding Delay
  • Intended Vector
  • Offered Vector
  • Expected Delay Vector

3.4.4.5. Average Delay Vector

Definition: the average Forwarding Delay for packets in a Stream matching a specific classification, like DSCP, that a DUT/SUT is measured to transmit to the correct destination interface successfully in response to an offered vector. The chill version of delay 🌊

Discussion: the Average Delay Vector is expressed as a combo of the classification rules AND the average Forwarding Delay. 💅

The average Forwarding Delay is computed by averaging all the Instantaneous Delay Vectors for a given Stream. Basically, add them up and divide 🧮

It's a per-hop measurement. The DUT/SUT MAY remark the specific DSCP or IP precedence value for a multi-hop measurement, and the Stream stays the same. 🦋

The Average Delay Vector can be obtained at any offered load. It's recommended that the offered load be at or below the Forwarding Capacity in the absence of congestion. For congested Forwarding Delay, run the offered load above the Forwarding Capacity. 🌈

Measurement Units: milliseconds ⏱️

See Also:

  • Classification
  • Stream
  • Forwarding Capacity
  • Forwarding Delay
  • Intended Vector
  • Offered Vector
  • Expected Delay Vector
  • Instantaneous Delay Vector

3.4.4.6. Maximum Delay Vector

Definition: the maximum Forwarding Delay for packets in a Stream matching a specific classification, like DSCP, that a DUT/SUT is measured to transmit to the correct destination interface successfully in response to an offered vector. The worst-case moment, but make it iconic 👑

Discussion: the Maximum Delay Vector is expressed as a combo of the classification rules AND the maximum Forwarding Delay. 💀

The maximum Forwarding Delay is computed by picking the highest value from the Instantaneous Delay Vectors for a given Stream. Just find the biggest one 😤

It's a per-hop measurement. The DUT/SUT MAY remark the specific DSCP or IP precedence value for a multi-hop measurement, and the Stream stays the same. 🫶

The Maximum Delay Vector can be obtained at any offered load. It's recommended that the offered load be at or below the Forwarding Capacity in the absence of congestion. For congested Forwarding Delay, run the offered load above the Forwarding Capacity. 💜

Measurement Units: milliseconds ⏱️

See Also:

  • Classification
  • Stream
  • Forwarding Capacity
  • Forwarding Delay
  • Intended Vector
  • Offered Vector
  • Expected Delay Vector
  • Instantaneous Delay Vector

3.4.4.7. Minimum Delay Vector ✨

Definition: Okay bestie, this is the lowest Forwarding Delay 🍓 a stream gets when the DUT/SUT sends it to the right destination interface successfully after receiving an offered vector, for a classification like DSCP 💅.

Discussion: So the Minimum Delay Vector is a combo of two things: the classification rules AND the minimum Forwarding Delay. 🎀 You get the minimum by picking the lowest value out of the Instantaneous Delay Vectors for a given stream. Main character energy, literally 👑

This one is a per-hop measurement 🦋. The DUT/SUT MAY remark the specific DSCP or IP precedence value for a multi-hop measurement, but the stream stays the same ✨

You can grab the Minimum Delay Vector at any offered load 🌈. It is recommended that the offered load be at or below the Forwarding Capacity when there's no congestion. For congested Forwarding Delay, you run the offered load above the Forwarding Capacity 🔥

Measurement Units: milliseconds

See Also:

  • Classification
  • Stream
  • Forwarding Capacity
  • Forwarding Delay
  • Intended Vector
  • Offered Vector
  • Expected Delay Vector

3.4.4.8. Instantaneous Jitter Vector 🦋

Definition: This is the jitter between two back-to-back packets in a stream matching a specific classification, like DSCP 💜, that a DUT/SUT is measured to transmit to the correct destination interface successfully in response to an offered vector.

Discussion: Instantaneous Jitter is the absolute value of the difference between the Forwarding Delay measurements of two packets in the same stream. It's basically how much the delay wiggles from one packet to the next 🫶

The Instantaneous Jitter Vector is a pair of numbers 🎀 Both the specific DSCP (or IP precedence) value AND the jitter value come together to make the vector.

The Forwarding Delay fluctuation between two consecutive packets in a stream gets reported as the "Instantaneous Jitter" ✨ You can write it as |D(i) - D(i-1)|, where D is the Forwarding Delay and i is the test sequence number. Lost packets don't get counted in the measurement 💀

The Instantaneous Jitter Vector is also a per-hop measurement. The DUT/SUT MAY remark the specific DSCP or IP precedence value for a multi-hop measurement, and the stream stays the same 🌸

There can be a bunch of Instantaneous Jitter Vectors for one stream! For n packets measured, there may be (n-1) Instantaneous Jitter Vectors 🧚

Measurement units: milliseconds

See Also:

  • Classification
  • Stream
  • Forwarding Delay
  • Jitter
  • Forwarding Vector
  • Expected Vectors

3.4.4.9. Average Jitter Vector 🌈

Definition: This is the average jitter for packets in a stream matching a specific classification, like DSCP 🍓, that a DUT/SUT is measured to transmit to the correct destination interface successfully in response to an offered vector.

Discussion: You get the average jitter by averaging all the Instantaneous Jitter Vectors of the same stream measured during the test duration 🧮 The Average Jitter Vector is a combo of values too: the classification rules AND the average Jitter 💅

This is a per-hop measurement as well ✨ The DUT/SUT MAY remark the specific DSCP or IP precedence value for a multi-hop measurement, and the stream remains the same 🦋

Measurement units: milliseconds

See Also:

  • Classification
  • Stream
  • Jitter
  • Forwarding Vector
  • Expected Vector
  • Instantaneous Jitter Vector

3.4.4.10. Peak-to-peak Jitter Vector 👑

Definition: This is the maximum possible variation in the Forwarding Delay for packets in a stream matching a specific classification, like DSCP 🎀, that a DUT/SUT is measured to transmit to the correct destination interface successfully in response to an offered vector.

Discussion: Peak-to-peak Jitter is worked out by subtracting the minimum Forwarding Delay from the maximum Forwarding Delay 🔥 Wait, let's be precise here: the text says you take the maximum Forwarding Delay and subtract the minimum Forwarding Delay, of the packets the DUT/SUT forwarded. The Jitter vector is a combo of the classification rules AND the peak-to-peak Jitter 🫶

Peak-to-peak Jitter is NOT derived from the Instantaneous Jitter Vector 😭 It's based on all the packets during the test duration, not just two consecutive ones. Totally different vibe 💜

Measurement units: milliseconds

See Also:

  • Jitter
  • Forwarding Vector
  • Stream
  • Expected Vectors
  • Instantaneous Jitter Vector
  • Average Jitter Vector

4. Security Considerations 🔐

Okay, this part is seriously important, so listen up 🙈 Documents of this type don't directly affect the security of the Internet or of corporate networks, as long as benchmarking isn't performed on devices or systems connected to production networks. Don't be that person who benchmarks prod! 💀

Packets with unintended and/or unauthorized DSCP or IP precedence values may present security issues. Figuring out the security consequences of those packets is out of scope for this document, so it's not something this doc is going to answer 🫠

5. Acknowledgements 🫶

The authors gratefully acknowledge the contributions of the IETF's Benchmarking Methodology Working Group members, who reviewed this document 🌸 The authors would like to express their thanks to David Newman for his consistent and valuable assistance throughout the development of this document 💖 The authors would also like to thank Al Morton and Kevin Dubray for their ideas and support ✨

6. References 📚

6.1. Normative References

  • [Br91] Bradner, S., "Benchmarking terminology for network interconnection devices", RFC 1242, July 1991.
  • [Br97] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.
  • [Br98] Braden, B., Clark, D., Crowcroft, J., Davie, B., Deering, S., Estrin, D., Floyd, S., Jacobson, V., Minshall, G., Partridge, C., Peterson, L., Ramakrishnan, K., Shenker, S., Wroclawski, J., and L. Zhang, "Recommendations on Queue Management and Congestion Avoidance in the Internet", RFC 2309, April 1998.
  • [Ma98] Mandeville, R., "Benchmarking Terminology for LAN Switching Devices", RFC 2285, February 1998.
  • [Ni98] Nichols, K., Blake, S., Baker, F., and D. Black, "Definition of the Differentiated Services Field (DS Field) in the IPv4 and IPv6 Headers", RFC 2474, December 1998.
  • [St91] Steinberg, L., "Techniques for managing asynchronously generated alerts", RFC 1224, May 1991.

6.2. Informative References

  • [Al99] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way Delay Metric for IPPM", RFC 2679, September 1999.
  • [Bl98] Blake, S., Black, D., Carlson, M., Davies, E., Wang, Z., and W. Weiss, "An Architecture for Differentiated Service", RFC 2475, December 1998.
  • [Br99] Bradner, S. and J. McQuaid, "Benchmarking Methodology for Network Interconnect Devices", RFC 2544, March 1999.
  • [De02] Demichelis, C. and P. Chimento, "IP Packet Delay Variation Metric for IP Performance Metrics (IPPM)", RFC 3393, November 2002.
  • [Ec98] http://www3.ietf.org/proceedings/98mar/98mar-edited-135.htm
  • [Fl93] Floyd, S., and Jacobson, V., "Random Early Detection gateways for Congestion Avoidance", IEEE/ACM Transactions on Networking, V.1 N.4, August 1993, p. 397-413. URL "ftp://ftp.ee.lbl.gov/papers/early.pdf".
  • [Ja99] Davie, B., Charny, A., Bennet, J.C., Benson, K., Le Boudec, J., Courtney, W., Davari, S., Firoiu, V., and D. Stiliadis, "An Expedited Forwarding PHB (Per-Hop Behavior)", RFC 3246, March 2002.
  • [Ka99] Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way Packet Loss Metric for IPPM", RFC 2680, September 1999.
  • [Ma91] Mankin, A. and K. Ramakrishnan, "Gateway Congestion Control Survey", RFC 1254, August 1991.
  • [Ma00] Mandeville, R. and J. Perser, "Benchmarking Methodology for LAN Switching Devices", RFC 2889, August 2000.
  • [Mo03] Morton, A., Ciavattone, L., Ramachandran, G., Shalunov, S., Perser, J., "Packet Reordering Metric for IPPM", Work in Progress.
  • [Na84] Nagle, J., "Congestion control in IP/TCP internetworks", RFC 896, January 1984.
  • [Ra99] Ramakrishnan, K., Floyd, S., and D. Black, "The Addition of Explicit Congestion Notification (ECN) to IP", RFC 3168, September 2001.
  • [Sc96] Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", STD 64, RFC 3550, July 2003.

Authors' Addresses 💌

Jerry Perser
Veriwave
8770 SW Nimbus Ave.
Suite B
Beaverton, OR 97008   USA
USA

Phone: + 1 818 338 4112
EMail: jerry@perser.org


Scott Poretsky
Reef Point Systems
8 New England Executive Park
Burlington, MA 01803
USA

Phone: + 1 508 439 9008
EMail: sporetsky@reefpoint.com


Shobha Erramilli
Telcordia Technologies
331 Newman Springs Road
Red Bank, New Jersey 07701
USA

EMail: shobha@research.telcordia.com


Sumit Khurana
Motorola
7700 West Parmer Ln.
Austin, TX 78729
USA

Phone: +1 512 996 6604
Email: skhurana@motorola.com

Copyright (C) The Internet Society (2006).

This document is subject to the rights, licenses and restrictions contained in BCP 78, and except as set forth therein, the authors retain all their rights.

This document and the information contained herein are provided on an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property 🔖

The IETF takes no position regarding the validity or scope of any Intellectual Property Rights or other rights that might be claimed to pertain to the implementation or use of the technology described in this document, or the extent to which any license under such rights might or might not be available. It also doesn't represent that it has made any independent effort to identify any such rights. Information on the procedures with respect to rights in RFC documents can be found in BCP 78 and BCP 79.

Copies of IPR disclosures made to the IETF Secretariat and any assurances of licenses to be made available, or the result of an attempt made to obtain a general license or permission for the use of such proprietary rights by implementers or users of this specification, can be obtained from the IETF on-line IPR repository at http://www.ietf.org/ipr.

The IETF invites any interested party to bring to its attention any copyrights, patents or patent applications, or other proprietary rights that may cover technology that may be required to implement this standard. Please address the information to the IETF at ietf-ipr@ietf.org.

Acknowledgement 💖

Funding for the RFC Editor function is provided by the IETF Administrative Support Activity (IASA). ✨

📦 where this page comes from