RFC 5481
Packet Delay Variation Applicability Statement
RFC 5481: Packet Delay Variation Applicability Statement 🎀
okay so this is RFC 5481, and it's giving official IETF document energy 💅. It was written by A. Morton (AT&T Labs) and B. Claise (Cisco Systems, Inc.) and published in March 2009 as part of the Network Working Group series. It's categorized as Informational, which means it shares info with the Internet community and doesn't set any kind of standard 🧚. It doesn't list anything it obsoletes or updates, so it's just vibing on its own ✨.
Status of This Memo 🌸
okay so this memo is here to give info to the Internet community. It is NOT a standard of any kind, and it can be shared as widely as you want 🦋.
Copyright Notice 💜
Copyright (c) 2009 IETF Trust and the persons identified as the document authors. All rights reserved.
ngl the legal part is a lot, but the gist is: this document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents, as they were in effect when it was published. Those are the rules for your rights and restrictions, so read them carefully 🙈. The link is http://trustee.ietf.org/license-info.
It also says this document may have material from IETF Documents or IETF Contributions that were published or made public before November 10, 2008. The people who control the copyright in some of that older material may not have given the IETF Trust permission to allow modifications outside the IETF Standards Process. So unless you get a proper license from those copyright holders, this document can't be modified outside the IETF Standards Process, and derivative works can't be made outside it either. The one exceptions are formatting it for publication as an RFC or translating it into languages other than English 📖.
Abstract 🍓
okay so packet delay variation metrics show up in tons of standards docs 😭. The metric definition in RFC 3393 is super flexible, and it lets you come up with different ways to formulate delay variation by choosing different packet selection functions 🔥.
That flexibility is iconic for coverage and new ideas, BUT it can make it harder to compare independent implementations. Two different formulations of delay variation have become widely used for active measurements. This memo looks at a bunch of situations for active measurements of delay variation and what they're used for, and it recommends which of the two forms fits particular conditions and tasks best 👑.
Table of Contents 🧚
1. Introduction ....................................................4
1.1. Requirements Language ......................................5
1.2. Background Literature in IPPM and Elsewhere ................5
1.3. Organization of the Memo ...................................6
2. Purpose and Scope ...............................................7
3. Brief Descriptions of Delay Variation Uses ......................7
3.1. Inferring Queue Occupation on a Path .......................7
3.2. Determining De-Jitter Buffer Size ..........................8
3.3. Spatial Composition .......................................10
3.4. Service-Level Comparison ..................................10
3.5. Application-Layer FEC Design ..............................10
4. Formulations of IPDV and PDV ...................................10
4.1. IPDV: Inter-Packet Delay Variation ........................11
4.2. PDV: Packet Delay Variation ...............................11
4.3. A "Point" about Measurement Points ........................12
4.4. Examples and Initial Comparisons ..........................12
5. Survey of Earlier Comparisons ..................................13
5.1. Demichelis' Comparison ....................................13
5.2. Ciavattone et al. .........................................15
5.3. IPPM List Discussion from 2000 ............................16
5.4. Y.1540 Appendix II ........................................18
5.5. Clark's ITU-T SG 12 Contribution ..........................18
6. Additional Properties and Comparisons ..........................18
6.1. Packet Loss ...............................................18
6.2. Path Changes ..............................................19
6.2.1. Lossless Path Change ...............................20
6.2.2. Path Change with Loss ..............................21
6.3. Clock Stability and Error .................................22
6.4. Spatial Composition .......................................24
6.5. Reporting a Single Number (SLA) ...........................24
6.6. Jitter in RTCP Reports ....................................25
6.7. MAPDV2 ....................................................25
6.8. Load Balancing ............................................26
7. Applicability of the Delay Variation Forms and
Recommendations ................................................27
7.1. Uses ......................................................27
7.1.1. Inferring Queue Occupancy ..........................27
7.1.2. Determining De-Jitter Buffer Size (and FEC
Design) ............................................27
7.1.3. Spatial Composition ................................28
7.1.4. Service-Level Specification: Reporting a
Single Number ......................................28
7.2. Challenging Circumstances .................................28
7.2.1. Clock and Storage Issues ...........................28
7.2.2. Frequent Path Changes ..............................29
7.2.3. Frequent Loss ......................................29
7.2.4. Load Balancing .....................................29
7.3. Summary ...................................................30
8. Measurement Considerations .....................................31
8.1. Measurement Stream Characteristics ........................31
8.2. Measurement Devices .......................................32
8.3. Units of Measurement ......................................33
8.4. Test Duration .............................................33
8.5. Clock Sync Options ........................................33
8.6. Distinguishing Long Delay from Loss .......................34
8.7. Accounting for Packet Reordering ..........................34
8.8. Results Representation and Reporting ......................35
9. Security Considerations ........................................35
10. Acknowledgments ...............................................35
11. Appendix on Calculating the D(min) in PDV .....................35
12. References ....................................................36
12.1. Normative References .....................................36
12.2. Informative References ...................................371. Introduction ✨
there are so many ways to formulate packet delay variation metrics for the Internet and other packet-based networks 🌈. The IETF has a few specs for delay variation [RFC3393]. Sometimes these are called jitter [RFC3550], or even inter-arrival jitter [RFC3550], and they've been widely adopted 💖. The International Telecommunication Union - Telecommunication Standardization Sector (ITU-T) has also recommended a few delay variation metrics (they call them parameters) [Y.1540] [G.1020], and some of those are cited and used a LOT 🔥. Most of the standards above specify more than one way to measure delay variation, so the takeaway is that standards efforts have been inclusive rather than selective 🫶.
This memo uses the term "delay variation" for metrics that measure how consistently a path can transfer packets, meaning how steady the delay is 🎀. [RFC3393] and [Y.1540] both prefer this term too. Some people call this phenomenon "jitter," and the buffers that try to smooth out the variations are called de-jitter buffers 🦋. "Jitter" gets used way more broadly than packet transfer performance, with "unwanted signal variation" as its general definition. It's been used for frequency or phase variations too, like data stream rate changes or carrier signal phase noise 😭. "Delay variation" is basically self-explanatory and more precise, so that's the term this memo prefers 💅.
Most (if not all) delay variation metrics are derived metrics, meaning their definitions rely on another fundamental metric 🧚. In this case, the fundamental metric is one-way delay. Variation gets assessed by computing the difference between two individual one-way-delay measurements, or a pair of singletons. One of the delay singletons is taken as a reference, and the result is the variation relative to that reference. The variation is usually summarized across all packets in a stream using statistics 📊.
The industry has mostly implemented two specific formulations of delay variation (for one survey of the situation, see [Krzanowski]) 👑:
- Inter-Packet Delay Variation, IPDV, where the reference is the previous packet in the stream (according to sending sequence), and the reference changes for every packet in the stream. Properties of variation are coupled with packet sequence in this formulation. This form used to be called Instantaneous Packet Delay Variation in early IETF contributions, and it's similar to the packet spacing difference metric used for interarrival jitter calculations in [RFC3550] 🍓.
- Packet Delay Variation, PDV, where a single reference is chosen from the stream based on specific criteria. The most common criterion for the reference is the packet with the minimum delay in the sample. This term gets its name from a similar definition for Cell Delay Variation, an ATM performance metric [I.356] 💜.
It's important to note that the authors of the relevant delay variation standards knew there are lots of different users with different needs, so they allowed enough flexibility to formulate several metrics with different properties 🌸. That means the comparison isn't really between standards bodies or their specs. It's between specific formulations of delay variation ✨. Both Inter-Packet Delay Variation and Packet Delay Variation are compliant with [RFC3393], because different packet selection functions will produce either form 🫶.
1.1. Requirements Language 💅
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119 [RFC2119]. 🎀
1.2. Background Literature in IPPM and Elsewhere 📚✨
okay so with more people joining the measurement community every single day 🫶, it's totally possible this memo is the first IP Performance Metrics (IPPM) Working Group doc you've ever opened 🙈. no shame! this section is basically a little road map plus background on the IPPM lit, and the published specs from other standards orgs 🗺️💅
The IPPM framework [RFC2330] is the foundation for this memo and the other IPPM RFCs 🏗️. It's where key terms like singleton, sample, and statistic get defined. It also covers ways to collect samples (Poisson streams), time-related issues, and the "packet of Type-P" convention 🦋
There are two fundamental and related metrics that can be applied to every packet transfer attempt 😭: one-way loss [RFC2680] and one-way delay [RFC2679]. Both use a waiting time threshold to tell lost packets apart from delayed ones. Packets that arrive at the measurement destination within their waiting time have finite delay and aren't lost. Anything else gets designated lost, and its delay is undefined 💀. For guidance on setting that waiting time threshold, check [RFC2680] and [IPPM-Reporting] 📖.
Another fundamental metric is packet reordering, as specified in [RFC4737] 🔄. The reordering metric was defined to be "orthogonal" to packet loss, which is so iconic. A gap in a packet sequence caused by loss does not result in reordered packets. But a rearrangement of packet arrivals from their sending order does count as reordering 💜.
Derived metrics are built on the fundamental ones. The metric of primary interest here is delay variation [RFC3393], which is derived from one-way delay [RFC2680]. Another derived metric is the loss patterns metric [RFC3357], which is derived from loss 🌸.
The measured values of all metrics (both fundamental and derived) depend a lot on the stream characteristics used to collect them 🧚. Both Poisson streams [RFC3393] and Periodic streams [RFC3432] have been used with the IPDV and PDV metrics. The choice of stream specification for active measurement depends on the purpose of the characterization and the constraints of the testing environment. Periodic streams are frequently chosen for use with IPDV and PDV, because the application streams that are most sensitive to delay variation exhibit periodicity 🔥. More method-specific details are discussed in Section 8 on "Measurement Considerations" 📝.
In the ITU-T, the framework, fundamental metrics, and derived metrics for IP performance are specified in Recommendation Y.1540 [Y.1540] 🌈. [G.1020] defines additional delay variation metrics, analyzes the operation of fixed and adaptive de-jitter buffers, and describes an example adaptive de-jitter buffer emulator 🎀. Appendix II of [G.1050] describes the models for network impairments (including delay variation) that are part of a standardized IP network emulator. Those models may be useful when evaluating measurement techniques 🛠️.
1.3. Organization of the Memo 🗂️
Here's the tour 🚌. The Purpose and Scope comes next, in Section 2. Then we give a summary of the main tasks for delay variation metrics in Section 3. Section 4 defines the two primary forms of delay variation, and Section 5 presents summaries of four earlier comparisons. Section 6 adds new comparisons to the analysis, and Section 7 reviews the applicability and recommendations for each form of delay variation. Section 8 then looks at many important delay variation measurement considerations 🧐. After the Security Considerations, there's an appendix on the calculation of the minimum delay for the PDV form 💖.
2. Purpose and Scope 🎯
The IPDV and PDV formulations each have certain features that make them more suitable for one circumstance and less so for another. The whole point of this memo is to compare the two forms of delay variation, so it's obvious which one is better suited for each of many possible uses and their related circumstances 💅.
The scope is limited to the two forms of delay variation briefly described above (Inter-Packet Delay Variation and Packet Delay Variation). It also covers circumstances related to active measurement, and uses that are deemed relevant and worthy of inclusion here through IPPM Working Group consensus 🫶.
It's entirely possible that the analysis and conclusions here apply beyond the intended scope 👀. But the reader is cautioned to fully appreciate the circumstances of active measurement on IP networks before trying that. Don't skip this part, bestie ⚠️.
The scope excludes assessment of delay variation for packets with undefined delay. This is handled by conditioning the delay distribution on arrival within a reasonable waiting time, based on an understanding of the path under test and packet lifetimes ⏳. The waiting time is sometimes called the loss threshold [RFC2680]. If a packet arrives beyond this threshold, it may as well have been lost, because it's no longer useful 💀. This is consistent with [RFC3393], where the Type-P-One-way-ipdv is undefined when the destination fails to receive one or both packets in the selected pair. It's also consistent with application performance analysis to consider only arriving packets, because a finite waiting time-out is a feature of many protocols 🧚.
3. Brief Descriptions of Delay Variation Uses 🌷
This section presents a set of tasks that call for delay variation measurements 📋. Here, the memo gives several answers to the question, "How will the results be used?" for the delay variation metric ✨.
3.1. Inferring Queue Occupation on a Path 🚦
As packets travel along the path from source to destination, they pass through many network elements, including a series of router queues 🚗. Some types of delay sources along the path are constant, such as links between two locations. But the latency in each queue varies, depending on the number of packets in the queue when a particular packet arrives 😩. If one assumes that at least one of the packets in a test stream encounters virtually empty queues all along the path (and the path is stable), then the additional delay observed on other packets can be attributed to the time spent in one or more queues. Otherwise, the delay variation observed is the variation in queue time experienced by the test stream 🧠.
It's worth noting that delay variation can occur beyond IP router queues, in other communication components. Examples include media contention: DOCSIS, IEEE 802.11, and some mobile radio technologies 📡.
However, delay variation from all sources at the IP layer and below will be quantified using the two formulations discussed here 💜.
3.2. Determining De-Jitter Buffer Size 🎀
Note -- while this memo and other IPPM literature prefer the term "delay variation", the terms "jitter buffer" and the more accurate "de-jitter buffer" are widely adopted names for a component of packet communication systems. They will be used here to designate that system component 📣.
Most isochronous applications (a.k.a. real-time applications) employ a buffer to smooth out delay variation encountered on the path from source to destination 🎥. The buffer must be big enough to accommodate the expected variation of delay, or packet loss will result 😱. However, if the buffer is too large, some of the desired spontaneity of communication will be lost and conversational dynamics will be affected 🙃. So application designers need to know the range of delay variation they must accommodate, whether they're designing fixed or adaptive buffer systems 💅.
Network service providers also try to constrain delay variation to ensure the quality of real-time applications. They monitor this metric, possibly to compare with a numerical objective or Service Level Agreement 📊.
De-jitter buffer size can be expressed in units of octets of storage space for the packet stream, or in units of time that the packets are stored. It's pretty simple to convert between octets and time when the buffer read rate (in octets per second) is constant 🧮:
read_rate * storage_time = storage_octetsUnits of time are used in the discussion below ⏱️.
The objective of a de-jitter buffer is to compensate for all prior sources of delay variation and produce a packet stream with constant delay ✨. So a packet experiencing the minimum transit delay from source to destination, D_min, should spend the maximum time in a de-jitter buffer, B_max. The sum of D_min and B_max should equal the sum of the maximum transit delay (D_max) and the minimum buffer time (B_min). We have:
Constant = D_min + B_max = D_max + B_min,after rearranging terms:
B_max - B_min = D_max - D_min = range(B) = range(D)where range(B) is the range of packet buffering times, and range(D) is the range of packet transit delays from source to destination 🌈.
Packets with transit delay between the max and min spend a complementary time in the buffer and also see the constant delay 💫.
In practice, the minimum buffer time, B_min, may not be zero. And the maximum transit delay, D_max, may be a high percentile (99.9th percentile) instead of the maximum 🙈.
Note that B_max - B_min = range(B) is the range of buffering times needed to compensate for delay variation. The actual size of the buffer may be larger (where B_min > 0) or smaller than range(B) 🍓.
There must be a process to align the de-jitter buffer time with packet transit delay 🔧. This is a process to identify the packets with minimum delay and schedule their play-out time so that they spend the maximum time in the buffer. The error in the alignment process can be accounted for by a variable, A. In the equation below, the range of buffering times available to the packet stream, range(b), depends on buffer alignment with the actual arrival times of D_min and D_max:
range(b) = b_max - b_min = D_max - D_min + AHere, variable b represents the available buffer in a system with a specific alignment, A, and b_max and b_min represent the limits of the available buffer 🦋.
When A is positive, the de-jitter buffer applies more delay than necessary (where Constant = D_max + b_min + A represents one possible alignment). When A is negative, there is insufficient buffer time available to compensate for range(D) because of misalignment. Packets with D_min may be arriving too early and encountering a full buffer, or packets with D_max may be arriving too late. In either case, the packets would be discarded 😭.
In summary, the range of transit delay variation is a critical factor in determining de-jitter buffer size 💖.
3.3. Spatial Composition 🛰️
In Spatial Composition, the tasks are similar to those described above. The difference is the additional complexity of a multiple network path, where several sub-paths are measured separately and no source-to-destination measurements are available 🧩. In this case, the source-to-destination performance must be estimated, using Composed Metrics as described in [IPPM-Framework] and [Y.1541]. Note that determining the composite delay variation is not trivial: simply summing the sub-path variations is not accurate 🙅♀️.
3.4. Service-Level Comparison
okay so IP performance measurements are basically the receipts for service agreements 🧾 between providers and their customers. And the measured numbers need to look good next to what the agreement promised, or it's a whole vibe check fail 💀.
Packet delay variation is usually one of the metrics written into these agreements 💅. In theory, any formulation could go in the Service Level Agreement (SLA). But the SLA is most useful when the numbers connect to how the customer actually uses the service, and that can usually be worked out from one of the tasks above 🫶.
3.5. Application-Layer FEC Design
the design of application-layer Forward Error Correction (FEC) pieces is closely tied to de-jitter buffer design in a few ways 🦋. The FEC designer has to pick a protection interval (the time it takes to send or receive a block of packets in a constant packet rate system) that fits the packet-loss behavior, but also keeps the amount of expected delay variation in mind 👀. The system designer also has to decide how long to wait for "late" packets to show up 🕰️. Again, the range of delay variation is the relevant expression of delay variation for these tasks 💖.
4. Formulations of IPDV and PDV
this section lays out the formulations of IPDV and PDV and walks through some examples 🧚. We use the basic singleton definition from [RFC3393] (which is itself based on [RFC2679]):
"Type-P-One-way-ipdv is defined for two packets from Src to Dst selected by the selection function F, as the difference between the value of the Type-P-One-way-delay from Src to Dst at T2 and the value of the Type-P-One-Way-Delay from Src to Dst at T1".
4.1. IPDV: Inter-Packet Delay Variation
if we have packets in a stream numbered consecutively as i = 1,2,3,... inside the test interval, then IPDV(i) = D(i)-D(i-1), where D(i) is the one-way delay of the ith packet of a stream ✨.
One-way delays are the difference between timestamps applied at the two ends of the path, which is the receiver time minus the transmission time 🕑.
So D(2) = R2-T2. With this timestamp notation, it can be shown that IPDV also represents the change in inter-packet spacing between transmission and reception 🔥:
IPDV(2) = D(2) - D(1) = (R2-T2) - (R1-T1) = (R2-R1) - (T2-T1)An example selection function given in [RFC3393] is "Consecutive Type-P packets within the specified interval". This is exactly the function IPDV needs 🎯. The reference packet in the pair is the previous packet in the sending sequence.
IPDV can take on positive and negative values, and zero too 🌈. One way to analyze IPDV results is to focus on the positive excursions. However, this approach has limitations that are discussed in more detail below (see Section 5.3) 🙈.
The mean of all IPDV(i) values for a stream is usually zero. A slow delay change over the life of the stream, or a frequency error between the measurement system clocks, can result in a non-zero mean 🍓.
4.2. PDV: Packet Delay Variation
the name Packet Delay Variation is used in [Y.1540] and its predecessors. It refers to a performance parameter that is equivalent to the metric described below 💜.
The Selection Function for PDV needs two specific roles for the packets in the pair. The first packet is any Type-P packet within the specified interval. The second packet, the reference packet, is the Type-P packet within the specified interval with the minimum one-way delay 👑.
So PDV(i) = D(i)-D(min) (using the nomenclature from the IPDV section). D(min) is the delay of the packet with the lowest delay value (the minimum) over the current test interval. Values of PDV may be zero or positive, and quantiles of the PDV distribution are direct indications of delay variation 🌸.
PDV is a version of the one-way-delay distribution, shifted to the origin by normalizing to the minimum delay 🎀.
4.3. A "Point" about Measurement Points
Both IPDV and PDV are derived from the one-way-delay metric 🫧. One-way delay requires knowing the time at two points, such as the source and destination of an IP network path in end-to-end measurement. So both IPDV and PDV can be categorized as 2-point metrics, because they're derived from one-way delay 💅. Specific measurement methods may make assumptions or have a priori knowledge about one of the measurement points, but the metric definitions themselves are based on information collected at two measurement points ✨.
4.4. Examples and Initial Comparisons
Note: This material was originally presented in Slides 2 and 3 of [Morton06] 📊.
The figure below gives a sample of packet delays, calculates IPDV and PDV values, and shows a histogram for each one 😭.
Packet # 1 2 3 4 5
-------------------------------
Delay, ms 20 10 20 25 20
IPDV U -10 10 5 -5
PDV 10 0 10 15 10
| |
4| 4|
| |
3| 3| H
| | H
2| 2| H
| | H
H H 1| H H 1|H H H
H H | H H |H H H
---------+-------- +---------------
-10 -5 0 5 10 0 5 10 15
IPDV Histogram PDV Histogram
Figure 1: IPDV and PDV ComparisonThe sample has three packets with "typical" delays of 20 ms, one packet with a low delay of 10 ms (the minimum of the sample), and one packet with a 25 ms delay 🌷.
As noted above, this example shows that IPDV can take on positive and negative values, while PDV values are greater than or equal to zero 🧁. The IPDV and PDV histograms look quite different in general shape, and their ranges differ too (IPDV range = 20ms, PDV range = 15 ms) 🦄. Note that the IPDV histogram will change if the sequence of delays is modified, but the PDV histogram stays the same. PDV normalizes the one-way-delay distribution to the minimum delay and emphasizes the variation independent of the sequence of delays 💜.
5. Survey of Earlier Comparisons
this section summarizes previous work comparing these two forms of delay variation 📚.
5.1. Demichelis' Comparison
In [Demichelis], Demichelis compared early versions of two forms of delay variation 🕵️. Although the IPDV form would eventually see widespread use, the ITU-T work-in-progress he cited did not use the same reference packets as PDV. Demichelis compared IPDV with two alternatives for the PDV reference packet: the delay of the first packet in the stream, and the mean delay of the stream. Neither of these alternative references was used in practice, and they are now deprecated in favor of the minimum delay of the stream [Y.1540] 🙅♀️.
Active measurements of a transcontinental path (Torino to Tokyo) provided the data for the comparison 🌍. The Poisson test stream had a 0.764 second average inter-packet interval, with more than 58 thousand packets over 13.5 hours ⏳. Among Demichelis' observations about IPDV are the following:
- IPDV is a measure of the network's ability to preserve the spacing between packets.
- The distribution of IPDV is usually symmetrical about the origin, with a balance of negative and positive values (for the most part). The mean is usually zero, unless some long-term delay trend is present.
- IPDV singletons distinguish quick delay variations (short-term, on the order of the interval between packets) from longer-term variations.
- IPDV places reduced demands on the stability and skew of measurement clocks.
He also notes these features of PDV:
- The PDV distribution does not distinguish short-term variation from variation over the complete test interval. (Comment: PDV can be determined over any sub-intervals when the singletons are stored.)
- The location of the distribution is very sensitive to the delay of the first packet, IF this packet is used as the reference. This would be a new formulation that differs from the PDV definition in this memo. (PDV references the packet with minimum delay, so it does not have this drawback.)
- The shape of the PDV distribution is identical to the delay distribution, but shifted by the reference delay.
- Use of a common reference over measurement intervals that are longer than a typical session length may indicate more PDV than would be experienced by streams that support such sessions. (Ideally, the measurement interval should be aligned with the session length of interest, and this influences determination of the reference delay, D(min).)
- The PDV distribution characterizes the range of queue occupancies along the measurement path (assuming the path is fixed), but the range says nothing about how the variation took place 🧠.
The summary metrics used in this comparison were the number of values exceeding a +/-50ms range around the mean, the Inverse Percentiles, and the Inter-Quartile Range 📈.
5.2. Ciavattone et al. 🎀
okay so [Cia03] compared IPDV and PDV (they called PDV "delta") using a periodic packet stream that conforms to [RFC3432], with an inter-packet interval of 20 ms ⏱️
One of their comparisons was a lab setup where a queue got temporarily congested by a competing packet burst. The extra queuing delay was 85 ms to 95 ms, which is way bigger than the inter-packet interval 😭 The first packet in the stream after the competing burst spends the longest time queued, and the others get less and less queuing time until the queue drains 🫶
The authors observed that PDV reflects the extra queuing time of the packets hit by the burst, with values of 85, 65, 45, 25, and 5 ms 📉 It's also easy to tell (just look at the PDV range) that a de-jitter buffer of >85 ms would have been enough to handle the delay variation. Again, the measurement interval is a key factor in whether these observations are valid (it should have a similar length to the session interval of interest) 🦋
The IPDV values in the congested queue example are very different: 85, -20, -20, -20, -20, -5 ms 🙈 Only the positive excursion of IPDV tells you the de-jitter buffer size needed. Even though the variation exceeds the inter-packet interval, the extent of negative IPDV values is limited by that sending interval. This preference for the positive values has led some people to ignore the negative ones, or to take the absolute value of each IPDV measurement (which sacrifices key properties of IPDV in the process, like its ability to distinguish delay trends) 💀
Note that this example shows an asymmetrical IPDV distribution, because the delay variation range (85 ms) is bigger than the inter-packet spacing (20 ms) 🍓 The IPDV values 85, -20, -20, -20, -20, -5 ms have zero mean, but the left side of the distribution gets chopped off at -20 ms.
Elsewhere in the article, the authors used the range as a summary statistic for IPDV, and the 99.9th percentile minus the minimum delay as a summary statistic for delay variation, or PDV ✨
5.3. IPPM List Discussion from 2000 💌
Mike Pierce made lots of comments about a working version of [RFC3393] 🧐 One of his main points was that a delay histogram is a useful way to quantify variation. He also said the time duration of evaluation is a critical aspect.
Carlo Demichelis then mailed his comparison paper [Demichelis] to the IPPM list, as discussed in more detail above 📬
Ruediger Geib observed that both IPDV and the delay histogram (PDV) are useful, and suggested they might be applied to different variation time scales 🌈 He pointed out that loss has a significant effect on IPDV, and encouraged keeping the loss information in the arrival sequence.
Several example delay variation scenarios were discussed, including:
Packet # 1 2 3 4 5 6 7 8 9 10 11
-------------------------------------------------------
Ex. A
Lost
Delay, ms 100 110 120 130 140 150 140 130 120 110 100
IPDV U 10 10 10 10 10 -10 -10 -10 -10 -10
PDV 0 10 20 30 40 50 40 30 20 10 0
-------------------------------------------------------
Ex. B
Lost L
Delay, ms 100 110 150 U 120 100 110 150 130 120 100
IPDV U 10 40 U U -10 10 40 -20 -10 -20
PDV 0 10 50 U 20 0 10 50 30 20 0
Figure 2: Delay ExamplesClearly, the range of PDV values is 50 ms in both cases above, and this is the statistic that determines the size of a de-jitter buffer 👑 The IPDV range is minimal in response to the smooth variation in Example A (20 ms). However, IPDV responds to the faster variations in Example B (60 ms range from 40 to -20). Here the IPDV range is larger than the PDV range, so it overestimates the buffer size requirements 😬
A heuristic method to estimate buffer size using IPDV is to sum the consecutive positive or zero values as an estimate of PDV range. However, this is more complicated to assess than the PDV range, and it depends heavily on the actual sequence of IPDV values (any negative IPDV value stops the summation, and again causes an underestimate) 🥴
IPDV values can be viewed as the adjustments an adaptive de-jitter buffer would make, if it could adjust on a packet-by-packet basis 🧚 However, adaptive de-jitter buffers don't adjust this frequently, so the value of this information is unknown. The short-term variations may be useful to know in some other cases.
5.4. Y.1540 Appendix II 📚
Appendix II of [Y.1540] describes a secondary terminology for delay variation 🤓 It compares IPDV, PDV (referred to as 2-point PDV), and 1-point packet delay variation (which assumes a periodic stream and assesses variation against an ideal arrival schedule constructed at a single measurement point). This early comparison discusses some of the same considerations raised in Section 6 below 💜
5.5. Clark's ITU-T SG 12 Contribution 🌸
Alan Clark's contribution to ITU-T Study Group 12 in January 2003 analyzed the root causes of delay variation and investigated different techniques for measuring and modeling "jitter" [COM12.D98] 🔍 Clark compared a metric closely related to IPDV, Mean Packet-to-Packet Delay Variation, MPPDV = mean(abs(D(i)-D(i-1))), to the newly proposed Mean Absolute Packet Delay Variation (MAPDV2, see [G.1020]) ⚖️ One of the tasks for this study was to estimate the number of packet discards in a de-jitter buffer. Clark concluded that MPPDV did not track the ramp delay variation he associated with access link congestion (similar to Figure 2, Example A above), but MAPDV2 did 👏
Clark also briefly looked at PDV (as described in the 2002 version of [Y.1541]) ✨ He concluded that if PDV was applied to a series of very short measurement intervals (e.g., 200 ms), it could be used to determine the fraction of intervals with high packet discard rates.
6. Additional Properties and Comparisons 🦄
This section takes some of the earlier comparison areas and digs into them in more detail, and it also introduces new areas for comparison 🔎
6.1. Packet Loss 🕳️
Packet loss has a huge influence on the delay variation results, as shown in Figures 3 and 4 (L means Lost and U means Undefined) 😵💫 Figure 3 shows that in the extreme case of every other packet being lost, the IPDV metric doesn't produce any results, while PDV produces results for all arriving packets 🙃
Packet # 1 2 3 4 5 6 7 8 9 10
Lost L L L L L
---------------------------------------
Delay, ms 3 U 5 U 4 U 3 U 4 U
IPDV U U U U U U U U U U
PDV 0 U 2 U 1 U 0 U 1 U
Figure 3: Path Loss Every Other PacketIn the case of a burst of packet loss, as shown in Figure 4, both the IPDV and PDV metrics produce some results 🌊 Note that PDV still produces more values than IPDV.
Packet # 1 2 3 4 5 6 7 8 9 10
Lost L L L L L
---------------------------------------
Delay, ms 3 4 U U U U U 5 4 3
IPDV U 1 U U U U U U -1 -1
PDV 0 1 U U U U U 2 1 0
Figure 4: Burst of Packet LossIn conclusion, the PDV results are affected by the packet-loss ratio 📊 The IPDV results are affected by both the packet-loss ratio and the packet-loss distribution. In the extreme case of losing every other packet, IPDV doesn't provide any results at all 💀
6.2. Path Changes 🛤️
When there is little or no stability in the network under test, the devices trying to characterize the network are equally stressed, especially if the displayed results are used to make inferences that may not be valid 😰
Sometimes the path characteristics change during a measurement interval. The change may be due to a link or router failure, administrative changes prior to maintenance (e.g., a link-cost change), or re-optimization of routing using new information. All of these causes are usually infrequent, and network providers take appropriate measures to make sure of this 🛠️ Automatic restoration to a back-up path is seen as a desirable feature of IP networks 🙌
Frequent path changes and prolonged congestion with substantial packet loss clearly make delay variation measurements challenging 🥲
Path changes are usually accompanied by a sudden, persistent increase or decrease in one-way delay. [Cia03] gives one such example. We assume that a restoration path either accepts a stream of packets or is not used for that particular stream (e.g., no multi-path for flows) 🧩
In any case, a change in the Time to Live (TTL) (or Hop Limit) of the received packets indicates that the path is no longer the same 🚨 Transient packet reordering may also be observed with path changes, due to use of non-optimal routing while updates propagate through the network (see [Casner] and [Cia03]) 🔀
Many, if not all, packet streams experience packet loss in conjunction with a path change. However, it is certainly possible that the active measurement stream does not experience loss. This may be due to use of a long inter-packet sending interval with respect to the restoration time, and it becomes more likely as "fast restoration" techniques see wider deployment (e.g., [RFC4090]) 🏃♀️💨
Thus, there are two main cases to consider: path changes accompanied by loss, and those that are lossless from the point of view of the active measurement stream. The subsections below examine each of these cases 🫶
6.2.1. Lossless Path Change 🦋
okay so in the lossless case, a path change will typically affect only one IPDV singleton 💅. For example, the delay sequence in the figure below always produces IPDV=0 except in the one case where the value is 5 (U, 0, 0, 0, 5, 0, 0, 0, 0).
Packet # 1 2 3 4 5 6 7 8 9
Lost
------------------------------------
Delay, ms 4 4 4 4 9 9 9 9 9
IPDV U 0 0 0 5 0 0 0 0
PDV 0 0 0 0 5 5 5 5 5
Figure 5: Lossless Path ChangeBUT 👀 if the change in delay is negative and larger than the inter-packet sending interval, then more than one IPDV singleton may be affected because packet reordering is also likely to occur. not me getting plot twists in my own delay data 😭
The use of the new path and its delay variation can be quantified by treating the PDV distribution as bi-modal, and characterizing each mode separately. This would involve declaring a new path within the sample, and using a new local minimum delay as the PDV reference delay for the sub-sample (or time interval) where the new path is present. ✨
The process of detecting a bi-modal delay distribution is made difficult if the typical delay variation is larger than the delay change associated with the new path. However, information on a TTL (or Hop Limit) change or the presence of transient reordering can assist in an automated decision 🤖💖.
The effect of path changes may also be reduced by making PDV measurements over short intervals (minutes, as opposed to hours). This way, a path change will affect one sample and its PDV values. Assuming that the mean or median one-way delay changes appreciably on the new path, then subsequent measurements can confirm a path change and trigger special processing on the interval to revise the PDV result. It's giving careful bookkeeping 🧚
Alternatively, if the path change is detected, by monitoring the test packets TTL or Hop Limit, or monitoring the change in the IGP link-state database, the results of measurement before and after the path change could be kept separated, presenting two different distributions. This avoids the difficult task of determining the different modes of a multi-modal distribution. Iconic, no cap 👑
6.2.2. Path Change with Loss 🌸
If the path change is accompanied by loss, such that there are no consecutive packet pairs that span the change, then no IPDV singletons will reflect the change. This may or may not be desirable, depending on the ultimate use of the delay variation measurement. Figure 6, in which L means Lost and U means Undefined, illustrates this case 🙈.
Packet # 1 2 3 4 5 6 7 8 9
Lost L L
------------------------------------
Delay, ms 3 4 3 3 U U 8 9 8
IPDV U 1 -1 0 U U U 1 -1
PDV 0 1 0 0 U U 5 6 5
Figure 6: Path Change with LossPDV will again produce a bi-modal distribution. But here, the decision process to define sub-intervals associated with each path is further assisted by the presence of loss, in addition to TTL, reordering information, and use of short measurement intervals consistent with the duration of user sessions. It is reasonable to assume that at least loss and delay will be measured simultaneously with PDV and/or IPDV. Loss is literally the extra clue here 🔥.
IPDV does not help to detect path changes when accompanied by loss, and this is a disadvantage for those who rely solely on IPDV measurements. ngl this is the one where IPDV gets benched 💀
6.3. Clock Stability and Error ⏰
Low cost or low complexity measurement systems may be embedded in communication devices that do not have access to high stability clocks, and time errors will almost certainly be present. However, larger time-related errors (~1 ms) may offer an acceptable trade-off for monitoring performance over a large population (the accuracy needed to detect problems may be much less than required for a scientific study, ~0.01 ms for example) 🍓.
Maintaining time accuracy <<1 ms has typically required access to dedicated time receivers at all measurement points. Global positioning system (GPS) receivers have often been installed to support measurements. The GPS installation conditions are fairly restrictive, and many prospective measurement efforts have found the deployment complexity and system maintenance too difficult. Real-life struggle bus 😩.
As mentioned above, [Demichelis] observed that PDV places greater demands on clock synchronization than for IPDV. This observation deserves more discussion. Synchronization errors have two components: time-of-day errors and clock-frequency errors (resulting in skew). ✨
Both IPDV and PDV are sensitive to time-of-day errors when attempting to align measurement intervals at the source and destination. Gross misalignment of the measurement intervals can lead to lost packets, for example, if the receiver is not ready when the first test packet arrives. However, both IPDV and PDV assess delay differences, so the error present in any two one-way-delay singletons will cancel as long as the error is constant. So, the demand for NTP or GPS synchronization comes primarily from one-way-delay measurement time-of-day accuracy requirements. Delay variation and measurement interval alignment are relatively less demanding. 🫶
Skew is a measure of the change in clock time over an interval with respect to a reference clock. Both IPDV and PDV are affected by skew, but the error sensitivity in IPDV singletons is less because the intervals between consecutive packets are rather small, especially when compared to the overall measurement interval. Since PDV computes the difference between a single reference delay (the sample minimum) and all other delays in the measurement interval, the constraint on skew error is greater to attain the same accuracy as IPDV. Again, use of short PDV measurement intervals (on the order of minutes, not hours) provides some relief from the effects of skew error. Thus, the additional accuracy demand of PDV can be expressed as a ratio of the measurement interval to the inter-packet spacing. 📐
A practical example is a measurement between two hosts, one with a synchronized clock and the other with a free-running clock having 50 parts per million (ppm) long term accuracy.
- If IPDV measurements are made on packets with a 1 second spacing, the maximum singleton error will be
1 x 5 x 10^-5 seconds, or0.05 ms. - If PDV measurements are made on the same packets over a 60 second measurement interval, then the delay variation due to the max free-running clock error will be
60 x 5 x 10-5 seconds, or3 msdelay variation error from the first packet to the last.
Therefore, the additional accuracy required for equivalent PDV error under these conditions is a factor of 60 more than for IPDV. This is a rather extreme scenario, because time-of-day error of 1 second would accumulate in ~5.5 hours, potentially causing the measurement interval alignment issue described above. 😅
If skew is present in a sample of one-way delays, its symptom is typically a nearly linear growth or decline over all the one-way-delay values. As a practical matter, if the same slope appears consistently in the measurements, then it may be possible to fit the slope and compensate for the skew in the one-way-delay measurements, thereby avoiding the issue in the PDV calculations that follow. See [RFC3393] for additional information on compensating for skew. slope-fitting is a whole vibe 📈
Values for IPDV may have non-zero mean over a sample when clock skew is present. This tends to complicate IPDV analysis when using the assumptions of a zero mean and a symmetric distribution. 🌈
There is a third factor related to clock error and stability: this is the presence of a clock-synchronization protocol (e.g., NTP) and the time-adjustment operations that result. When a time error is detected (typically on the order of a few milliseconds), the host clock frequency is continuously adjusted to reduce the time error. If these adjustments take place during a measurement interval, they may appear as delay variation when none was present, and therefore are a source of error (regardless of the form of delay variation considered). Ghost jitter, the worst kind of plot device 👻💜.
6.4. Spatial Composition 🧩
ITU-T Recommendation [Y.1541] gives a provisional method to compose a PDV metric using PDV measurement results from two or more sub-paths. Additional methods are considered in [IPPM-Spatial].
PDV has a clear advantage at this time, since there is no validated method to compose an IPDV metric. In addition, IPDV results depend greatly on the exact sequence of packets and may not lend themselves easily to the composition problem, where segments must be assumed to have independent delay distributions. PDV really said "pick me" here 💅🎀.
6.5. Reporting a Single Number (SLA) 📊
Despite the risk of over-summarization, measurements must often be displayed for easy consumption. If the right summary report is prepared, then the "dashboard" view correctly indicates whether there is something different and worth investigating further, or that the status has not changed. The dashboard model restricts every instrument display to a single number. The packet network dashboard could have different instruments for loss, delay, delay variation, reordering, etc., and each must be summarized as a single number for each measurement interval. The single number summary statistic is a key component of SLAs, where a threshold on that number must be met x% of the time. 🦋
The simplicity of the PDV distribution lends itself to this summarization process (including use of the percentiles, median or mean). An SLA of the form "no more than x% of packets in a measurement interval shall have PDV >= y ms, for no less than z% of time" is relatively straightforward to specify and implement. [Y.1541] introduced the notion of a pseudo-range when setting an objective for the 99.9th percentile of PDV. The conventional range (max-min) was avoided for several reasons, including stability of the maximum delay. The 99.9th percentile of PDV is helpful to performance planners (seeking to meet some user-to-user objective for delay) and in design of de-jitter buffer sizes, even those with adaptive capabilities. Percentiles understood the assignment 🧠✨.
IPDV does not lend itself to summarization so easily. The mean IPDV is typically zero. As the IPDV distribution will have two tails (positive and negative), the range or pseudo-range would not match the needed de-jitter buffer size. Additional complexity may be introduced when the variation exceeds the inter-packet sending interval, as discussed above (in Sections 5.2 and 6.2.1). Should the Inter-Quartile Range be used? Should the singletons beyond some threshold be counted (e.g., mean +/- 50 ms)? A strong rationale for one of these summary statistics has yet to emerge. IPDV is giving unresolved storyline 😭🌸.
When summarizing IPDV, some prefer the simplicity of the single-sided distribution created by taking the absolute value of each singleton result, abs(D(i)-D(i-1)). This approach sacrifices the two-sided inter-arrival spread information in the distribution. It also makes the evaluation using percentiles more confusing, because a single late packet that exceeds the variation threshold will cause two pairs of singletons to fail the criteria (one positive, the other negative converted to positive). The single-sided PDV distribution is an advantage in this category. simple, clean, and honestly iconic 💖
6.6. Jitter in RTCP Reports ✨
Okay so, [RFC3550] Section 6.4.1 is where you find how the "inter-arrival jitter" field in an RTP Control Protocol (RTCP) report gets calculated 🎀. There's even a sample implementation tucked into an Appendix, which is so helpful of them 🫶.
The RTCP "interarrival jitter" value can be worked out from IPDV singletons 💅. Now if there's packet reordering (the kind defined in [RFC4737]), the Jitter estimates based on IPDV might come out a little different, slightly. That's because [RFC3550] says to use the order packets are received in 🙈.
Here's the tea: you can't just turn PDV singletons into IPDV singletons without going back to the original delay singletons, and there's no clear relationship between PDV and the [RFC3550] "interarrival jitter" either 💀.
6.7. MAPDV2 🦋
MAPDV2 stands for Mean Absolute Packet Delay Variation (version) 2, and it's specified in [G.1020] 🌸. The algorithm keeps a smoothed running estimate of the mean delay, using the one-way delays of 16 previous packets. Then it compares the current one-way delay to that estimated mean, works out the means of the positive and negative deviations separately, and adds those two deviation means together to get MAPVDV2 💜. Basically you get a MAPDV2 singleton for every single packet that shows up, so it's usually worth summarizing further 🧚.
Neither IPDV nor PDV forms help with computing MAPDV2 💅.
6.8. Load Balancing 🌈
Okay, network traffic load balancing is when you split packet traffic so it's spread more evenly across two or more equally viable paths 🔥. The paths get picked based on IGP cost metrics, but the delay depends on how the physical path is laid out ✨. Usually the balancing happens per flow, so packets don't see delay variation from zipping along different physical paths 🦋.
If the sample has test packets with different characteristics, like different IP addresses or ports, you could end up with multi-modal delay distributions 👑. PDV makes it possible to spot multiple modes ✨. IPDV might also reveal that multiple paths are in use when you have a mixed-flow sample, but the different delay modes aren't easy to split up and analyze separately 🙈.
So should the delay singletons from multiple addresses/ports go in the same sample? Should each mode get characterized on its own? (This question also applies to the Path Change case.) Honestly, it depends on what the measurement is for 💖.
For de-jitter buffer sizing or assessing queue occupation, you should characterize the modes separately 💅. Flows only experience one mode on a stable path anyway. Using a single flow description (address/port combination) in each sample makes the analysis so much simpler 🫶. If you want to find multiple modes, you can collect samples with different flow attributes, then compare the delay distributions from each sample to characterize the paths 🌸.
For capacity planning and routing optimization, characterizing the modes separately could be a real advantage 🔥. Network-wide capacity planning (not just link capacity planning) takes in the core traffic matrix, which is a matrix of traffic moving from every source to every destination in the network 🍓. Put that core traffic matrix together with routing information (usually the link state database of a routing protocol) in a capacity planning tool, and you can see the paths the traffic flows along and optimize routing based on link utilization 👑. When equal cost multiple paths (ECMPs) are in use, the traffic gets load balanced across multiple paths 🌈. If each mode of the IP delay multi-modal distribution can be tied to a specific path, then the delay performance becomes an extra optimization parameter, i.e., routing optimization based on the IP delay variation metric 💜. For example, you could stop load balancing across ECMPs so Voice over IP (VoIP) calls only get routed over the path with the lower IP delay variation 📞✨. Obviously, any changes can produce new delay performance measurements, so there has to be a verification step to make sure you got the outcome you wanted 🙈.
7. Applicability of the Delay Variation Forms and Recommendations 👑
Based on the comparisons of IPDV and PDV we just went through, this section matches the attributes of each form with the tasks we covered earlier 💅. We'll start with the more general circumstances first 🦋.
7.1. Uses 🌸
7.1.1. Inferring Queue Occupancy 🫧
The PDV distribution is anchored at the minimum delay observed in the measurement interval 💖. When the sample minimum matches the true minimum delay of the path, the PDV distribution is equivalent to the queuing time distribution the test stream experienced ✨. If the minimum delay isn't the true minimum, then the PDV distribution still captures the variation in queuing time, but some extra queuing time is experienced that we don't know about 🙈. You can summarize the PDV distribution with the mean, median, and other statistics 📊.
IPDV can capture the difference in queuing time from one packet to the next 🔥, but that's a different distribution from the queue occupancy that PDV reveals 💅.
7.1.2. Determining De-Jitter Buffer Size (and FEC Design) 🎀
This task is complimentary to the problem of inferring queue occupancy through measurement 🫶. Again, using the sample minimum as the reference delay for PDV gives you a distribution that's super relevant to de-jitter buffer size 🌈. That's because the minimum delay is the alignment point for the smoothing operation of de-jitter buffers 🧚. A de-jitter buffer that's ideally aligned with the delay variation adds zero buffer time to packets with the longest accommodated network delay (anything with a longer delay gets discarded) 💀. So a packet that experiences the minimum network delay should be aligned to wait the maximum length of the de-jitter buffer 👑. With that alignment, the stream gets smoothed with no unnecessary delay added ✨. Figure 5 of [G.1020] illustrates the ideal relationship between network delay variation and buffer time 🦋.
The PDV distribution is useful for this task too, but different statistics are preferred 💜. The range (max-min) or the 99.9th percentile of PDV (pseudo-range) are closely related to the buffer size needed to accommodate the observed network delay variation 🍓.
The PDV distribution also goes straight at the FEC waiting time question 🔥. When the PDV distribution has a 99th percentile of 10 ms, waiting 10 ms longer than the FEC protection interval will let 99% of late packets arrive and get used in the FEC block 🌸.
In some cases, the positive excursions (or series of positive excursions) of IPDV might help approximate the de-jitter buffer size 🤔. But there's no guarantee a good buffer estimate will come out of it, especially when the delay varies as a positive trend over several test packets 😭.
7.1.3. Spatial Composition 🧩
PDV has a clear advantage right now, since there's no validated method to compose an IPDV metric 💅.
7.1.4. Service-Level Specification: Reporting a Single Number 📈
The one-sided PDV distribution can be constrained with a single statistic, like an upper percentile, which is why it's preferred 👑. The IPDV distribution is two-sided, usually has zero mean, and after years of experience, no universal summary statistic tied to a physical quantity has emerged 🙃.
7.2. Challenging Circumstances 🌧️
Just so you know, measuring delay variation might not be the main concern when things are unstable and unreliable 💜.
7.2.1. Clock and Storage Issues ⏰
When there's appreciable skew between measurement system clocks, IPDV has an advantage, because PDV would need processing over the entire sample to remove the skew error 🫶. But significant skew can invalidate IPDV analysis assumptions, like the zero-mean and symmetric-distribution characteristics 😵. Small skew might be within the error tolerance, and then both PDV and IPDV results are usable 🌸. There may be a portion of the skew, measurement interval, and required accuracy 3-D space where IPDV has an advantage, depending on the specific measurement specifications 🦋.
Neither form of delay variation is more suited than the other to on-the-fly summarization without memory 🧠. That might be one reason [RFC3550] RTCP Jitter and MAPDV2 in [G.1020] have caught on in low-cost systems 💖.
7.2.2. Frequent Path Changes 🛤️
If the network under test has frequent path changes, on the order of several new routes per minute, IPDV seems to isolate the delay variation on each path from the transient effect of the path change 🔥. This is especially true if there's packet loss at the time of the path change 💀. But if you want to use IPDV to indicate path changes, it can't do that when the change comes with loss 🙅♀️.
It's possible to make meaningful PDV measurements when paths are unstable, but a lot rides on the algorithms that infer path change and try to divide the sample on path change boundaries 🧐.
When path changes are frequent and cause packet loss, delay variation is probably less important than the loss episodes, so turn your attention to the loss metric instead 👑.
7.2.3. Frequent Loss 📉
If the network under test has frequent loss, PDV may produce a larger set of singletons for the sample than IPDV 💅. That's because IPDV needs consecutive packet arrivals to assess delay variation, while PDV finds any packet arrival useful 🫶. The worst case is when no consecutive packets arrive, so the entire IPDV sample is undefined, yet PDV can still produce a sample based on the packets that did arrive 😭.
7.2.4. Load Balancing ⚖️
PDV distributions offer the most straightforward way to identify that a sample of packets traversed multiple paths 🌈. For de-jitter buffer sizing or assessing queue occupation with PDV, you should use a sample with a single flow, because flows only experience one mode on a stable path and it keeps the analysis simple 💖.
7.3. Summary ✨
okay so bestie, here's the whole comparison in one glance 👀💅
+---------------+----------------------+----------------------------+
| Comparison | PDV = D(i)-D(min) | IPDV = D(i)-D(i-1) |
| Area | | |
+---------------+----------------------+----------------------------+
| Challenging | Less sensitive to | Preferred when path |
| Circumstances | packet loss, and | changes are frequent or |
| | simplifies analysis | when measurement clocks |
| | when load balancing | exhibit some skew |
| | or multiple paths | |
| | are present | |
|---------------|----------------------|----------------------------|
| Spatial | All validated | Has sensitivity to |
| Composition | methods use this | sequence and spacing |
| of DV metric | form | changes, which tends to |
| | | break the requirement for |
| | | independent distributions |
| | | between path segments |
|---------------|----------------------|----------------------------|
| Determine | "Pseudo-range" | No reliable relationship, |
| De-Jitter | reveals this | but some heuristics |
| Buffer Size | property by | |
| Required | anchoring the | |
| | distribution at the | |
| | minimum delay | |
|---------------|----------------------|----------------------------|
| Estimate of | Distribution has | No reliable relationship |
| Queuing Time | one-to-one | |
| and Variation | relationship on a | |
| | stable path, | |
| | especially when | |
| | sample min = true | |
| | min | |
|---------------|----------------------|----------------------------|
| Specification | One constraint | Distribution is two-sided, |
| Simplicity: | needed for | usually has zero mean, and |
| Single Number | single-sided | no universal summary |
| SLA | distribution, and | statistic that relates to |
| | easily related to | a physical quantity |
| | quantities above | |
+---------------+----------------------+----------------------------+
Summary of Comparisons8. Measurement Considerations 🔥
this part is all about the practical stuff, like how you actually measure delay variation without messing it up, and it gives the two formulations we compared extra attention 🧐
8.1. Measurement Stream Characteristics
okay so, as we said back in Section 1.2, the stream you send for measuring really matters for the results 🫶 there are two main ways people collect samples. There's Poisson sampling, described in [RFC2330], and Periodic sampling, described in [RFC3432]. Poisson was meant to give you an unbiased sample of performance, while Periodic is for when you have a "known bias of interest" 💜 Periodic streams are required to have random start times and a limited duration. The reason is so they don't accidentally sync up with some other periodic process, and so congestion-aware senders don't lock onto the stream and give you weird, atypical results 🙈 Each new stream should get its own random start time.
It's worth knowing that [RFC3393] was being developed at the same time as [RFC3432]. Because of that, all the stream metrics in [RFC3393] are written around the Poisson sampling method 📝
Periodic sampling shows up a lot in delay variation measurements, and there are a few reasons why 🧚
- Lots of application streams that care about delay variation are also periodic, so they show that bias of interest. If the application sends with a constant packet spacing, that same spacing can be the inter-packet gap for your test stream. VoIP streams often use 20 ms spacing, so that's an obvious pick for an Active stream. This works for both the IPDV and PDV forms.
- The spacing between packets decides whether the stream sees short-range dependency or only long-range dependency, as looked into in [Li.Mills]. Packet spacing also shapes the IPDV distribution and how sensitive the stream is to reordering. For example, with 20 ms spacing, the IPDV distribution can't go below -20 ms unless packets get reordered 👀
- When the send spacing and send rate are constant, the measurement process can make a few simplifying assumptions. For example, you can compare the inter-arrival times at the destination against an ideal sending schedule. That allows a one-point measurement of delay variation (described in [Y.1540]) that approximates the IPDV form. There are also simplified methods that approximate PDV (some are discussed in Appendix II of [Y.1541]) 💅
- Analyzing truncated or non-symmetrical IPDV distributions gets a lot easier. Delay variations bigger than the periodic sending interval can cause multiple singleton values stuck at the negative limit of the packet spacing (see Section 5.2 and [Cia03]). Only packet reordering can push things past that negative spacing limit 🌸
Even though IPDV is all about inter-packet delay differences, both Poisson [Demichelis] and Periodic [Li.Mills] streams have been used. Those references show the different kinds of analysis you can do 🦋
The reasons to use a Poisson distribution are covered in [RFC2330] 💖 The main points are that it avoids predicting when samples happen, avoids syncing up with periodic events that show up in networks, and avoids inducing synchronization with congestion-aware senders. When you use a Poisson stream with IPDV, the distribution reflects inter-packet delay variation across lots of different time scales (or packet spacings). That unbiased sampling does bring a whole new layer of complexity to analyzing IPDV distributions 🌀
8.2. Measurement Devices
one big thing about measurement devices is whether they can store singletons (individual measurements) 📦 That usually goes hand in hand with how much local calculation they can do. For example, an embedded measurement device with limited storage will likely only give you a few statistics on the delay variation distribution. Dedicated measurement systems, on the other hand, can store every singleton and allow for detailed analysis. With the original singletons, you can calculate either form of delay variation later on ✨
So systems with limited storage have to pick their metrics and summary statistics ahead of time 🫠 If you want both IPDV and PDV statistics, you have to collect the supporting information as the packets arrive. For instance, the PDV range and high percentiles can be figured out later, as long as the minimum and several of the largest delays were stored while the measurement was running 🧠
8.3. Units of Measurement
both IPDV and PDV can be summarized as a range in milliseconds 📏
With IPDV, it makes sense to report a positive percentile. An inter-quantile range is a good fit because it captures both the positive and negative tails (e.g., 5% to 95%). If the IPDV distribution is symmetric around a mean of zero, then reporting the positive side of the distribution is enough 🎀
With PDV, it's enough to specify the upper percentile (e.g., 99.9%) 💜
8.4. Test Duration
throughout this memo, we've recommended test intervals on the order of minutes ⏱️ In their paper looking at how stable Internet path properties are [Zhang.Duff], Zhang et al. concluded that consistency shows up on the order of minutes for the performance metrics they looked at (loss, delay, and throughput) on the paths they measured 🌈
If you want to estimate a bigger interval from performance measured in small intervals, that's called temporal aggregation, and it's covered in the Metric Composition Framework [IPPM-Framework] 🧚
The main recommendation here is to test for durations that are similar in length to the session time you care about 💖 This applies to both IPDV and PDV, but it's maybe more relevant for PDV, since the duration decides how often D_min gets determined and how big the associated sample is 🍓
8.5. Clock Sync Options
just like with one-way-delay measurements, keeping local clocks synchronized is a big deal for delay variation measurements ⏰
There are a few options for doing that 🛰️
- Global Positioning System receivers
- In some parts of the world, Cellular Code Division Multiple Access (CDMA) systems distribute timing signals that come from GPS and can be traced to UTC.
- Network Time Protocol [RFC1305] is a convenient choice in many cases, but it usually has lower accuracy than the options above.
When clock sync is inconvenient or has noticeable errors, round-trip measurements can give a cumulative picture of the delay variation on both directions of the path 🔄 The catch is that delay distributions are rarely symmetrical, so it's hard to infer much about one-way-delay variation from round-trip measurements 😬 Measurements on asymmetrical paths also add complications for the one-way-delay metric.
8.6. Distinguishing Long Delay from Loss
lost packets and really delayed packets are separated by a waiting time threshold ⏳ Packets that show up at the measurement destination within their waiting time have a finite delay and aren't lost. Anything else gets marked as lost, and its delay is undefined 💀 For guidance on setting that waiting time threshold, check out [RFC2680] and [IPPM-Reporting].
Basically, [IPPM-Reporting] suggests using a long waiting time to characterize the network, and then revising the results for specific application delay thresholds as needed 💅
8.7. Accounting for Packet Reordering
packet reordering, defined in [RFC4737], is basically an extreme form of delay variation, where the order packets arrive in is different from the order they were sent in 🌀
PDV results don't care about packet arrival order, and reordering only affects them by showing the more extreme variation 🫶
IPDV results change when reordering is present, because they depend on the sequence of delays of arriving packets. The biggest example of this is how the negative tail of the distribution gets truncated 🙈
- When there's no reordering, the negative tail is limited by the sending time spacing between packets.
- If reordering happens (and the reordered packets aren't discarded), the negative tail can take on any value (in principle).
In general, measurement systems should be able to detect when the sequence has changed 👀 If IPDV measurements are made without paying attention to packet arrival order, IPDV will be under-reported when reordering happens 😭
8.8. Results Representation and Reporting
okay so every reference that talks about delay variation has ideas for how to show or report the results 📊✨. Interested readers should look through the different options, bestie 💖.
For example, [IPPM-Reporting] suggests reporting a pseudo-range of delay variation. You calculate it as the difference between a high percentile of delay and the minimum delay 🦋. The 99.9th percentile minus the minimum gives a value you can compare with objectives in [Y.1541] 🎀.
9. Security Considerations 🔐💀
Security is NOT a bit, babe. The security considerations for any active measurement of live networks apply here too 😭. Go check the "Security Considerations" sections in [RFC2330], [RFC2679], [RFC3393], [RFC3432], and [RFC4656] 🙈.
Security considerations do not help you pick between the PDV and IPDV forms of delay variation 🌸. Measurements with either metric have exactly the same security issues, so it's a no-brainer 💅.
10. Acknowledgments 🫶
The authors want to thank Phil Chimento 💜. He suggested using conditional distributions of delay to deal with packet loss, and he encouraged them to "write the memo" after hearing "the talk" on this topic at IETF 65 🎤✨. They also acknowledge constructive comments from Alan Clark, Loki Jorgenson, Carsten Schmoll, and Robert Holley 🌈. Iconic supporting cast, no notes 👑.
11. Appendix on Calculating the D(min) in PDV 🧮
Practitioners have been asking some really good questions, and this section is here to answer them 💭:
- How is this D_min calculated? Is it DV(99%) as mentioned in [Krzanowski]?
- Do we need to keep all the values from the interval, then take the minimum? Or do we keep the minimum from previous intervals?
So, the D_min used as the reference delay for PDV calculations is simply the minimum delay of all packets in the current sample 🍓. The usual single-value summary of the PDV distribution is D_(99.9th percentile) minus D_min ✨.
It may be appropriate to split the packets into subsets and revise the minimum value during a sample 🦋. For example, if you can be certain the path has changed by checking the Time to Live or Hop Count of arriving packets, that may be enough reason to reset the minimum for packets on the new path 🛤️. There's also a simpler approach: use samples collected over short evaluation intervals, on the order of minutes ⏱️💅. Intervals with path changes may be more interesting from the loss or one-way-delay perspective, since they might be failing one or more SLAs, so it may not be necessary to do delay variation analysis for them 😬. Short evaluation intervals are preferred for measurements that serve as a basis for troubleshooting, since the results are available to report soon after collection 🧚.
It is not necessary to store all delay values in a sample when storage is a major concern 💾. You can find D_min by comparing each new singleton value with the current value and replacing it when required 🙌. In a sample with 5000 packets, you can also evaluate the 99.9th percentile with limited storage 🤯. One method stores the top 50 delay singletons and revises the top value list each time 50 more packets arrive 🎀.
12. References 📚
12.1. Normative References
[RFC2119]Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997.[RFC2330]Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, "Framework for IP Performance Metrics", RFC 2330, May 1998.[RFC2679]Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way Delay Metric for IPPM", RFC 2679, September 1999.[RFC2680]Almes, G., Kalidindi, S., and M. Zekauskas, "A One-way Packet Loss Metric for IPPM", RFC 2680, September 1999.[RFC3393]Demichelis, C. and P. Chimento, "IP Packet Delay Variation Metric for IP Performance Metrics (IPPM)", RFC 3393, November 2002.[RFC3432]Raisanen, V., Grotefeld, G., and A. Morton, "Network performance measurement with periodic streams", RFC 3432, November 2002.[RFC4090]Pan, P., Swallow, G., and A. Atlas, "Fast Reroute Extensions to RSVP-TE for LSP Tunnels", RFC 4090, May 2005.[RFC4656]Shalunov, S., Teitelbaum, B., Karp, A., Boote, J., and M. Zekauskas, "A One-way Active Measurement Protocol (OWAMP)", RFC 4656, September 2006.[RFC4737]Morton, A., Ciavattone, L., Ramachandran, G., Shalunov, S., and J. Perser, "Packet Reordering Metrics", RFC 4737, November 2006.
12.2. Informative References
[COM12.D98]Clark, A., "Analysis, measurement and modelling of Jitter", ITU-T Delayed Contribution COM 12 - D98, January 2003.[Casner]Casner, S., Alaettinoglu, C., and C. Kuan, "A Fine-Grained View of High Performance Networking", NANOG 22, May 20-22, 2001,<http://www.nanog.org/mtg-0105/agenda.html>.[Cia03]Ciavattone, L., Morton, A., and G. Ramachandran, "Standardized Active Measurements on a Tier 1 IP Backbone", IEEE Communications Magazine, p. 90-97, June 2003.[Demichelis]Demichelis, C., "Packet Delay Variation Comparison between ITU-T and IETF Draft Definitions", November 2000,<http://www.advanced.org/ippm/archive.3/att-0075/01-pap02.doc>.[G.1020]ITU-T, "Performance parameter definitions for the quality of speech and other voiceband applications utilizing IP networks", ITU-T Recommendation G.1020, 2006.[G.1050]ITU-T, "Network model for evaluating multimedia transmission performance over Internet Protocol", ITU-T Recommendation G.1050, November 2005.[I.356]ITU-T, "B-ISDN ATM Layer Cell Transfer Performance", ITU-T Recommendation I.356, March 2000.[IPPM-Framework]Morton, A., "Framework for Metric Composition", Work in Progress, October 2008.[IPPM-Reporting]Morton, A., Ramachandran, G., and G. Maguluri, "Reporting Metrics: Different Points of View", Work in Progress, January 2009.[IPPM-Spatial]Morton, A. and E. Stephan, "Spatial Composition of Metrics", Work in Progress, July 2008.[Krzanowski]Presentation at IPPM, IETF-64, "Jitter Definitions: What is What?", November 2005.[Li.Mills]Li, Q. and D. Mills, "The Implications of Short-Range Dependency on Delay Variation Measurement", Second IEEE Symposium on Network Computing and Applications, 2003.[Morton06]Morton, A., "A Brief Jitter Metrics Comparison, and not the last word, by any means...", slide presentation at IETF 65, IPPM Session, March 2006.[RFC1305]Mills, D., "Network Time Protocol (Version 3) Specification, Implementation", RFC 1305, March 1992.[RFC3357]Koodli, R. and R. Ravikanth, "One-way Loss Pattern Sample Metrics", RFC 3357, August 2002.[RFC3550]Schulzrinne, H., Casner, S., Frederick, R., and V. Jacobson, "RTP: A Transport Protocol for Real-Time Applications", STD 64, RFC 3550, July 2003.[Y.1540]ITU-T, "Internet protocol data communication service - IP packet transfer and availability performance parameters", ITU-T Recommendation Y.1540, November 2007.[Y.1541]ITU-T, "Network Performance Objectives for IP-Based Services", ITU-T Recommendation Y.1541, February 2006.[Zhang.Duff]Zhang, Y., Duffield, N., Paxson, V., and S. Shenker, "On the Constancy of Internet Path Properties", Proceedings of ACM SIGCOMM Internet Measurement Workshop, November 2001.
Authors' Addresses 💌
Al Morton
AT&T Labs
200 Laurel Avenue South
Middletown, NJ 07748
USA
Phone: +1 732 420 1571
Fax: +1 732 368 1192
EMail: acmorton@att.com
URI: http://home.comcast.net/~acmacm/
Benoit Claise
Cisco Systems, Inc.
De Kleetlaan 6a b1
Diegem, 1831
Belgium
Phone: +32 2 704 5622
EMail: bclaise@cisco.comAnd that's a wrap on RFC 5481, bestie 🌈💖✨
📦 where this page comes from
- ietf: rfc5481 2009-03
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms