RFC 2679

A One-way Delay Metric for IPPM

๐Ÿ“… published September 1999, from ietf

RFC 2679 โœจ A One-way Delay Metric for IPPM

๐ŸŽ€ Title block tea ๐ŸŽ€

  • Number: RFC 2679 ๐Ÿฆ‹
  • Authors: G. Almes, S. Kalidindi, and M. Zekauskas, all from Advanced Network & Services ๐Ÿ’…
  • Category: Standards Track ๐Ÿ‘‘
  • Date: September 1999 ๐ŸŒธ
  • Obsoletes / Updates: the header doesn't list anything it obsoletes or updates ๐Ÿ™ˆ

1. Status of this Memo ๐Ÿ’–

okay so this doc is a Internet standards track protocol for the whole Internet community ๐ŸŒ, and it's literally asking for discussion and suggestions for improvements ๐Ÿซถ. If you want to know where it stands officially, check the current edition of "Internet Official Protocol Standards" (STD 1) ๐Ÿ“œ. Distribution of this memo is unlimited, so share it with everyone bestie ๐Ÿ’œ.

Copyright (C) The Internet Society (1999). All Rights Reserved. ยฉ๏ธ no cap, this is the legal bit and it's staying exactly like this ๐Ÿ’…

2. Introduction ๐Ÿงš

This memo defines a metric for one-way delay of packets across Internet paths โœจ. It builds on ideas from the IPPM Framework document, RFC 2330 [1], so you should be familiar with that one first ๐Ÿ“š.

This memo is designed to be parallel in structure to a companion document for Packet Loss ("A One-way Packet Loss Metric for IPPM") [2] ๐Ÿค.

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be read as described in RFC 2119 [6] ๐Ÿ”ฅ. RFC 2119 was written with protocols in mind, but the same key words get used here for similar reasons ๐Ÿ’œ. They make sure measurements from two different implementations can actually be compared, and they call out moments when an implementation could mess with the network ๐Ÿ™ˆ.

The structure of the memo goes like this ๐ŸŒˆ:

  • A 'singleton' analytic metric, called Type-P-One-way-Delay, gets introduced to measure one single observation of one-way delay โœจ.
  • Using that singleton, a 'sample' called Type-P-One-way-Delay-Poisson-Stream gets introduced. It measures a sequence of singleton delays taken at times from a Poisson process ๐Ÿฆ‹.
  • Using that sample, several 'statistics' of the sample get defined and discussed ๐Ÿ“Š.

This progression from singleton to sample to statistics, with clear separation between each step, is important ๐Ÿ‘‘.

Whenever a technical term from the IPPM Framework document shows up for the first time in this memo, it gets a trailing asterisk. So "term*" just means "term" is defined in the Framework ๐ŸŽ€.

2.1. Motivation:

okay the reason one-way delay of a Type-P* packet from a source host* to a destination host is useful comes down to a few things ๐Ÿ’…:

  • Some applications don't perform well (or at all) if end-to-end delay between hosts is big compared to some threshold value ๐Ÿ˜ญ.
  • Erratic variation in delay makes it hard (or impossible) to support lots of real-time applications ๐Ÿ’€.
  • The bigger the delay, the harder it is for transport-layer protocols to keep high bandwidths going ๐Ÿ”ฅ.
  • The minimum value of this metric tells you the delay caused only by propagation and transmission ๐Ÿงš.
  • The minimum value also tells you the delay you'll likely see when the path* you're on is lightly loaded ๐ŸŒธ.
  • Values above the minimum tell you about the congestion present on the path ๐ŸŒŠ.

Measuring one-way delay instead of round-trip delay is motivated by these factors โœจ:

  • In today's Internet, the path from a source to a destination can be different from the path going back from the destination to the source ("asymmetric paths"). Different sequences of routers get used for the forward and reverse paths ๐Ÿ™ˆ. So round-trip measurements actually measure two different paths mashed together ๐Ÿ’€. Measuring each path on its own shows the performance difference between the two paths, which might cross different Internet service providers, and even totally different kinds of networks (like research versus commodity networks, or ATM versus packet-over-SONET) ๐Ÿฆ‹.
  • Even when the two paths are symmetric, they can have really different performance characteristics because of asymmetric queueing ๐Ÿ˜ญ.
  • An application's performance can depend mostly on performance in one direction ๐ŸŽ€. For example, a file transfer using TCP may depend more on the direction the data flows in, rather than the direction the acknowledgements travel ๐Ÿ’….
  • In quality-of-service (QoS) enabled networks, provisioning in one direction can be totally different from provisioning in the reverse direction, so the QoS guarantees differ ๐Ÿ‘‘. Measuring each path independently lets you verify both guarantees ๐Ÿ’œ.

It is outside the scope of this document to say exactly how delay metrics would be applied to specific problems ๐Ÿ™ˆ.

2.2. General Issues Regarding Time โฐ

Comment: the terminology below is different from what ITU-T documents define (for example G.810, "Definitions and terminology for synchronization networks", and I.356, "B-ISDN ATM layer cell transfer performance"). It is consistent with the IPPM Framework document instead ๐Ÿง . These differences come from different backgrounds: ITU-T documents have a telephony origin, while the authors of this document (and the Framework) have a computer systems background ๐Ÿ’ป. The terms defined below have no direct ITU-T equivalent, so a rough mapping gets given after the definitions. Just watch out for one potential confusion: this memo's definition of "clock" is the computer operating systems one, meaning a time-of-day clock, while the ITU-T definition of clock means a frequency reference โš ๏ธ.

Whenever a time (aka a moment in history) is mentioned here, it's understood to be measured in seconds (and fractions of seconds) relative to UTC โฑ๏ธ.

As the Framework document explains more fully, there are four distinct but related notions of clock uncertainty ๐ŸŒˆ:

  • synchronization* measures how far two clocks agree on what time it is. For example, the clock on one host might be 5.4 msec ahead of the clock on a second host ๐Ÿซถ. {Comment: a rough ITU-T equivalent is "time error".}
  • accuracy* measures how far a given clock agrees with UTC. For example, the clock on a host might be 27.1 msec behind UTC ๐Ÿ•ฐ๏ธ. {Comment: a rough ITU-T equivalent is "time error from UTC".}
  • resolution* measures how precise a given clock is. For example, the clock on an old Unix host might tick only once every 10 msec, so its resolution is only 10 msec ๐Ÿ”. {Comment: a very rough ITU-T equivalent is "sampling period".}
  • skew* measures how accuracy, or synchronization, changes over time ๐Ÿ“‰. For example, a clock might gain 1.3 msec per hour, so it's 27.1 msec behind UTC at one point and only 25.8 msec an hour later ๐Ÿ˜ญ. In that case, we say the clock has a skew of 1.3 msec per hour relative to UTC, which threatens accuracy ๐Ÿ’€. We can also talk about the skew of one clock relative to another clock, which threatens synchronization ๐Ÿ’…. {Comment: a rough ITU-T equivalent is "time drift".}

3. A Singleton Definition for One-way Delay ๐Ÿฆ‹

3.1. Metric Name:

Type-P-One-way-Delay โœจ

3.2. Metric Parameters:

  • Src, the IP address of a host ๐ŸŒธ
  • Dst, the IP address of a host ๐ŸŒธ
  • T, a time โฐ

3.3. Metric Units:

The value of a Type-P-One-way-Delay is either a real number, or an undefined (informally, infinite) number of seconds ๐Ÿ”ข.

3.4. Definition:

For a real number dT, the definition goes: the Type-P-One-way-Delay from Src to Dst at T is dT means that Src sent the first bit of a Type-P packet to Dst at wire-time* T, and that Dst received the last bit of that packet at wire-time T+dT ๐ŸŽฏ.

Then there's the undefined case: the Type-P-One-way-Delay from Src to Dst at T is undefined (informally, infinite) means that Src sent the first bit of a Type-P packet to Dst at wire-time T, and Dst never received that packet ๐Ÿ’€.

Suggestions for what to report along with metric values show up in Section 3.8, after a discussion of the metric, the methodologies for measuring it, and error analysis ๐Ÿ“.

3.5. Discussion:

Type-P-One-way-Delay is a relatively simple analytic metric, and we believe it'll afford effective ways of measuring things ๐Ÿง .

These issues are likely to come up in practice ๐Ÿ‘€:

  • Real delay values will be positive, so it doesn't make sense to report a negative value as a real delay ๐Ÿ™…. Still, a single zero or negative delay value can be useful as part of a stream when you're trying to figure out the distribution of a stream of delay values ๐Ÿ“Š.
  • Delay values are often as low as the 100 usec to 10 msec range, so it's super important for Src and Dst to synchronize really closely โฑ๏ธ. GPS systems are one way to synchronize to within several 10s of usec ๐Ÿ›ฐ๏ธ. Ordinary NTP might only get you synchronization within several msec, and that depends on the stability and symmetry of delay properties among the NTP agents used. And that delay is exactly what we're trying to measure ๐Ÿ™ƒ. A mix of some GPS-based NTP servers and a conservatively designed and deployed set of other NTP servers should yield good results, but that's still untested ๐Ÿงช.
  • A given methodology has to include a way to figure out whether a delay value is infinite, or just very large (with the packet still on its way to Dst) ๐Ÿค”. As Mahdavi and Paxson [4] noted, simple upper bounds could be used, like the 255 seconds theoretical upper bound on the lifetimes of IP packets [5] ๐Ÿ•ฐ๏ธ. But good engineering, including an understanding of packet lifetimes, will be needed in practice ๐Ÿ› ๏ธ.
    • {Comment: for many applications of these metrics, the harm in treating a large delay as infinite might be zero or very small. A TCP data packet, for example, that only arrives after several multiples of the RTT may as well have been lost ๐Ÿ™ˆ.}
  • If the packet is duplicated along the path (or paths) so that multiple non-corrupt copies arrive at the destination, the packet still counts as received, and the first copy to arrive determines the packet's one-way delay ๐Ÿฅ‡.
  • If the packet is fragmented and, for whatever reason, reassembly doesn't happen, then the packet gets deemed lost ๐Ÿ’”.

3.6. Methodologies: ๐Ÿงชโœจ

okay so bestie, this part is literally where the methodology lives ๐Ÿ’… As with other Type-P-* metrics, the detailed methodology depends on the Type-P (like protocol number, UDP/TCP port number, size, or precedence) ๐ŸŽ€

Generally, for a given Type-P, the methodology goes like this ๐Ÿ‘‘

+  Arrange that Src and Dst are synchronized; that is, that they have
   clocks that are very closely synchronized with each other and each
   fairly close to the actual time.

Src and Dst MUST NOT be left to vibe on their own clocks. They need clocks that are very closely synchronized with each other, and each one fairly close to the actual time ๐Ÿ•ฐ๏ธ๐Ÿ’–

Wait, I'm sorry, I have to flag this one: the original says "arrange that," not a key word, so I'm keeping it exactly as the spec has it. No extra force here ๐Ÿ™ˆ
+  At the Src host, select Src and Dst IP addresses, and form a test
   packet of Type-P with these addresses.  Any 'padding' portion of
   the packet needed only to make the test packet a given size should
   be filled with randomized bits to avoid a situation in which the
   measured delay is lower than it would otherwise be due to
   compression techniques along the path.

At the Src host, pick the Src and Dst IP addresses and build a test packet of Type-P using them ๐Ÿฆ‹ Any padding that's only there to make the packet a given size should be filled with randomized bits ๐ŸŒˆ That way the measured delay doesn't come out lower than it really is because of compression techniques along the path ๐Ÿซถ

+  At the Dst host, arrange to receive the packet.

At the Dst host, set things up so the packet actually gets received ๐Ÿงš

+  At the Src host, place a timestamp in the prepared Type-P packet,
   and send it towards Dst.

At the Src host, stamp the prepared Type-P packet with a timestamp and send it toward Dst ๐ŸŒธ

+  If the packet arrives within a reasonable period of time, take a
   timestamp as soon as possible upon the receipt of the packet.  By
   subtracting the two timestamps, an estimate of one-way delay can
   be computed.  Error analysis of a given implementation of the
   method must take into account the closeness of synchronization
   between Src and Dst.  If the delay between Src's timestamp and the
   actual sending of the packet is known, then the estimate could be
   adjusted by subtracting this amount; uncertainty in this value
   must be taken into account in error analysis.  Similarly, if the
   delay between the actual receipt of the packet and Dst's timestamp
   is known, then the estimate could be adjusted by subtracting this
   amount; uncertainty in this value must be taken into account in
   error analysis.  See the next section, "Errors and Uncertainties",
   for a more detailed discussion.

If the packet arrives within a reasonable period of time, Dst takes a timestamp as soon as possible after receiving it ๐Ÿ’… Subtracting the two timestamps gives you an estimate of one-way delay โœจ Error analysis for any given implementation MUST take into account how closely Src and Dst are synchronized ๐Ÿ”ฅ

Hold up, the original says "must" here in lowercase, so I'm keeping it lowercase too. Look at me being precise ๐Ÿค“

If the delay between Src's timestamp and the actual sending of the packet is known, the estimate could be adjusted by subtracting that amount, and the uncertainty in that value must be taken into account in error analysis ๐ŸŽ€ Same deal on the other side: if the delay between the actual receipt of the packet and Dst's timestamp is known, the estimate could be adjusted by subtracting it, and that uncertainty also has to be in the error analysis ๐Ÿฆ‹ For a deeper dive, check out the next section, "Errors and Uncertainties" ๐Ÿง โœจ

+  If the packet fails to arrive within a reasonable period of time,
   the one-way delay is taken to be undefined (informally, infinite).
   Note that the threshold of 'reasonable' is a parameter of the
   methodology.

If the packet doesn't arrive within a reasonable period of time, the one-way delay is undefined (informally, infinite) ๐Ÿ’€ And just so you know, the threshold of 'reasonable' is a parameter of the methodology ๐ŸŒˆ

Issues such as the packet format, the means by which Dst knows when to expect the test packet, and the means by which Src and Dst are synchronized are outside the scope of this document ๐Ÿงš {Comment: We plan to document elsewhere our own work in describing such more detailed implementation techniques and we encourage others to as well.} ๐Ÿ’œ


3.7. Errors and Uncertainties: ๐Ÿ™ˆ

okay this section is about the messy stuff, the errors and uncertainties ๐Ÿ˜ญ The description of any specific measurement method should include an accounting and analysis of various sources of error or uncertainty ๐Ÿซถ The Framework document gives general guidance on this, but here are the specifics for delay metrics ๐Ÿ”ฅ

+  Errors or uncertainties due to uncertainties in the clocks of the
   Src and Dst hosts.

+  Errors or uncertainties due to the difference between 'wire time'
   and 'host time'.

In addition, the loss threshold may affect the results ๐ŸŒธ Each of these gets discussed in more detail below, along with a section ("Calibration") on accounting for these errors and uncertainties ๐Ÿ’…


okay bestie, the clocks are the first thing to get messy ๐Ÿ•ฐ๏ธ The uncertainty in a measurement of one-way delay is related, in part, to uncertainties in the clocks of the Src and Dst hosts ๐Ÿ’– We refer to the clock used to measure when the packet was sent from Src as the source clock ๐Ÿฆ‹ and the clock used to measure when the packet was received by Dst as the destination clock ๐ŸŒˆ The observed time when the packet was sent by the source clock is Tsource โœจ and the observed time when the packet was received by the destination clock is Tdest ๐ŸŽ€ Alluding to the notions of synchronization, accuracy, resolution, and skew mentioned in the Introduction, here's the tea ๐Ÿ“

+  Any error in the synchronization between the source clock and the
   destination clock will contribute to error in the delay
   measurement.  We say that the source clock and the destination
   clock have a synchronization error of Tsynch if the source clock
   is Tsynch ahead of the destination clock.  Thus, if we know the
   value of Tsynch exactly, we could correct for clock
   synchronization by adding Tsynch to the uncorrected value of
   Tdest-Tsource.

Any error in synchronization between the source clock and the destination clock adds error to the delay measurement ๐Ÿซถ If the source clock is Tsynch ahead of the destination clock, they have a synchronization error of Tsynch ๐ŸŒธ So if we know Tsynch exactly, we can correct for it by adding Tsynch to the uncorrected value of Tdest-Tsource ๐Ÿงš

+  The accuracy of a clock is important only in identifying the time
   at which a given delay was measured.  Accuracy, per se, has no
   importance to the accuracy of the measurement of delay.  When
   computing delays, we are interested only in the differences
   between clock values, not the values themselves.

The accuracy of a clock only matters for figuring out when a given delay was measured ๐Ÿ’… Accuracy by itself doesn't affect how accurate the delay measurement is ๐Ÿ™ˆ When we compute delays, we only care about the differences between clock values, not the values themselves โœจ

+  The resolution of a clock adds to uncertainty about any time
   measured with it.  Thus, if the source clock has a resolution of
   10 msec, then this adds 10 msec of uncertainty to any time value
   measured with it.  We will denote the resolution of the source
   clock and the destination clock as Rsource and Rdest,
   respectively.

The resolution of a clock adds to the uncertainty of any time measured with it ๐Ÿ”ฅ So if the source clock has a resolution of 10 msec, that adds 10 msec of uncertainty to any time value measured with it ๐Ÿ’€ We'll call the resolution of the source clock Rsource and the resolution of the destination clock Rdest ๐Ÿฆ‹

+  The skew of a clock is not so much an additional issue as it is a
   realization of the fact that Tsynch is itself a function of time.
   Thus, if we attempt to measure or to bound Tsynch, this needs to
   be done periodically.  Over some periods of time, this function
   can be approximated as a linear function plus some higher order
   terms; in these cases, one option is to use knowledge of the
   linear component to correct the clock.  Using this correction, the
   residual Tsynch is made smaller, but remains a source of
   uncertainty that must be accounted for.  We use the function
   Esynch(t) to denote an upper bound on the uncertainty in
   synchronization.  Thus, |Tsynch(t)| <= Esynch(t).

Skew is less a separate problem and more a realization that Tsynch is itself a function of time ๐Ÿ•ฐ๏ธ So if we try to measure or bound Tsynch, that has to happen periodically ๐ŸŒˆ Over some periods, this function can be approximated as a linear function plus some higher order terms, and in those cases one option is to use knowledge of the linear component to correct the clock ๐Ÿซถ With that correction, the residual Tsynch gets smaller, but it's still a source of uncertainty that must be accounted for ๐Ÿ’– We use Esynch(t) to denote an upper bound on the uncertainty in synchronization, so |Tsynch(t)| <= Esynch(t) ๐Ÿ‘‘

Taking these items together, we note that naive computation Tdest-
Tsource will be off by Tsynch(t) +/- (Rsource + Rdest).  Using the
notion of Esynch(t), we note that these clock-related problems
introduce a total uncertainty of Esynch(t)+ Rsource + Rdest.  This
estimate of total clock-related uncertainty should be included in the
error/uncertainty analysis of any measurement implementation.

Putting all of that together: a naive Tdest-Tsource will be off by Tsynch(t) +/- (Rsource + Rdest) ๐Ÿ˜ญ Using Esynch(t), these clock-related problems add up to a total uncertainty of Esynch(t)+ Rsource + Rdest โœจ This estimate of total clock-related uncertainty should be included in the error/uncertainty analysis of any measurement implementation ๐Ÿ”ฅ


last section, bestie ๐Ÿซถ As we defined one-way delay, we'd like to measure the time between when the test packet leaves the network interface of Src and when it (completely) arrives at the network interface of Dst ๐Ÿ’… We call these "wire times" ๐ŸŒธ If software on Src and Dst does the timing, though, that software can only directly measure the time between when Src grabs a timestamp just prior to sending the test packet and when Dst grabs a timestamp just after receiving it ๐Ÿ˜ญ We call these two points "host times" ๐Ÿฆ‹

To the extent that the difference between wire time and host time is
accurately known, this knowledge can be used to correct for host time
measurements and the corrected value more accurately estimates the
desired (wire time) metric.

To the extent that the difference between wire time and host time is accurately known, you can use that knowledge to correct host time measurements, and the corrected value more accurately estimates the desired wire time metric โœจ

To the extent, however, that the difference between wire time and
host time is uncertain, this uncertainty must be accounted for in an
analysis of a given measurement method.  We denote by Hsource an
upper bound on the uncertainty in the difference between wire time
and host time on the Src host, and similarly define Hdest for the Dst
host.  We then note that these problems introduce a total uncertainty
of Hsource+Hdest.  This estimate of total wire-vs-host uncertainty
should be included in the error/uncertainty analysis of any
measurement implementation.

To the extent that the difference between wire time and host time is uncertain, that uncertainty must be accounted for in an analysis of a given measurement method ๐Ÿ”ฅ Hsource is an upper bound on the uncertainty in the difference between wire time and host time on the Src host, and Hdest is defined the same way for the Dst host ๐Ÿ’– Together these problems add up to a total uncertainty of Hsource+Hdest ๐Ÿ‘‘ This estimate of total wire-vs-host uncertainty should be included in the error/uncertainty analysis of any measurement implementation ๐ŸŒˆ

3.7.3. Calibration ๐ŸŽ€

Okay so generally, the measured values can be broken down like this ๐Ÿงฎ

measured value = true value + systematic error + random error

If the systematic error (the constant bias in measured values) can be determined, it can be compensated for in the reported results ๐Ÿ’…

reported value = measured value - systematic error

therefore

reported value = true value + random error

The goal of calibration is to figure out the systematic and random error the instruments create themselves, in as much detail as possible โœจ At a minimum, we need a bound ("e") so that the reported value lands in the range (true value - e) to (true value + e) at least 95 percent of the time ๐ŸŽฏ We call "e" the calibration error for the measurements. It shows how repeatable the values from the measurement instrument are, basically how closely an actual delay of 30 ms gets reported as 30 ms ๐Ÿซถ {Comment: 95 percent was chosen because (1) some confidence level is desirable to be able to remove outliers, which will be found in measuring any physical property; (2) a particular confidence level should be specified so that the results of independent implementations can be compared; and (3) even with a prototype user-level implementation, 95% was loose enough to exclude outliers.} ๐ŸŒธ

From the last two sections, you can bound the error in measurements by working out every individual uncertainty and adding them together, like this:

Esynch(t) + Rsource + Rdest + Hsource + Hdest.

Lowkey though, reasonable bounds are possible on both pieces ๐Ÿฆ‹ The clock-related uncertainty (the first three terms) and the host-related uncertainty (the last two terms) can be bounded with careful design and by calibrating the instruments on a known, isolated network in a lab ๐Ÿงช

For example, clock-related uncertainties get way smaller when you use a GPS time source ๐Ÿ›ฐ๏ธ The sum of Esynch(t) + Rsource + Rdest is small, and it stays bounded for the whole measurement because of that global time source. Iconic ๐Ÿ‘‘

The host-related uncertainties, Hsource + Hdest, could be bounded by connecting two instruments back-to-back with a high-speed serial link or an isolated LAN segment ๐Ÿ”Œ In this setup, repeated measurements are all measuring the same one-way delay.

If the test packets are small, a network connection like that has a minimal delay that can be approximated as zero ๐Ÿ“ So the measured delay only contains systematic and random error from the instrumentation. The "average value" of repeated measurements is the systematic error, and the variation is the random error.

One way to compute the systematic error, and the random error to a 95% confidence, is to repeat the experiment many times, at least hundreds of tests ๐Ÿ’œ The systematic error would then be the median. The random error could then be found by removing the systematic error from the measured values. The 95% confidence interval would be the range from the 2.5th percentile to the 97.5th percentile of these deviations from the true value. The calibration error "e" could then be taken to be the largest absolute value of these two numbers, plus the clock-related uncertainty. {Comment: as described, this bound is relatively loose since the uncertainties are added, and the absolute value of the largest deviation is used. As long as the resulting value is not a significant fraction of the measured values, it is a reasonable bound. If the resulting value is a significant fraction of the measured values, then more exact methods will be needed to compute the calibration error.} ๐Ÿ™ˆ

Note that random error depends on measurement load ๐Ÿ“ˆ For example, if one instrument is measuring lots of paths, that can increase interrupts, process scheduling, and disk I/O (like recording the measurements), and all of that may increase the random error in measured singletons. So on top of minimal load measurements to find the systematic error, calibration measurements should be run with the same measurement load the instruments will see in the field ๐Ÿ’…

We wish to reiterate that this statistical treatment is about calibrating the instrument. It's used to "calibrate the meter stick" and say how well the meter stick reflects reality ๐Ÿ“โœจ

On top of calibrating the instruments for finite one-way delay, two checks should be made to make sure packets reported as losses were really lost ๐Ÿ•ต๏ธโ€โ™€๏ธ First, the threshold for loss should be verified. In particular, make sure the "reasonable" threshold is actually reasonable: it should be very unlikely that a packet arrives after the threshold value, so the number of packets lost over an interval isn't sensitive to the error bound on measurements. Second, consider the possibility that a packet arrives at the network interface but is lost due to congestion on that interface, or to other resource exhaustion (e.g. buffers) in the instrument ๐Ÿฅน

3.8. Reporting the metric: ๐Ÿ“

The calibration and context in which the metric is measured MUST be carefully considered, and SHOULD always be reported along with metric results. We now present four items to consider: the Type-P of test packets, the threshold of infinite delay (if any), error calibration, and the path traversed by the test packets. This list is not exhaustive; any additional information that could be useful in interpreting applications of the metrics should also be reported. ๐Ÿ’–

3.8.1. Type-P ๐ŸŽŸ๏ธ

As noted in the Framework document [1], the value of the metric may depend on the type of IP packets used to make the measurement, or "type-P". The value of Type-P-One-way-Delay could change if the protocol (UDP or TCP), port number, size, or arrangement for special treatment (e.g., IP precedence or RSVP) changes. The exact Type-P used to make the measurements MUST be accurately reported. ๐Ÿ”ฅ

3.8.2. Loss threshold ๐Ÿšง

In addition, the threshold (or methodology to distinguish) between a large finite delay and loss MUST be reported. ๐Ÿง 

3.8.3. Calibration results ๐Ÿ“Š

  • If the systematic error can be determined, it SHOULD be removed from the measured values. โœจ
  • You SHOULD also report the calibration error, e, such that the true value is the reported value plus or minus e, with 95% confidence (see the last section.) ๐ŸŒˆ
  • If possible, the conditions under which a test packet with finite delay is reported as lost due to resource exhaustion on the measurement instrument SHOULD be reported. ๐Ÿ’€

3.8.4. Path ๐Ÿ›ฃ๏ธ

Finally, the path traversed by the packet SHOULD be reported, if possible. In general it is impractical to know the precise path a given packet takes through the network. The precise path may be known for certain Type-P on short or stable paths. If Type-P includes the record route (or loose-source route) option in the IP header, and the path is short enough, and all routers* on the path support record (or loose-source) route, then the path will be precisely recorded. This is impractical because the route must be short enough, many routers do not support (or are not configured for) record route, and use of this feature would often artificially worsen the performance observed by removing the packet from common-case processing. However, partial information is still valuable context ๐Ÿงญ For example, if a host can choose between two links* (and hence two separate routes from Src to Dst), then the initial link used is valuable context. {Comment: For example, with Merit's NetNow setup, a Src on one NAP can reach a Dst on another NAP by either of several different backbone networks.} ๐Ÿฆ‹

4. A Definition for Samples of One-way Delay ๐Ÿงš

Given the singleton metric Type-P-One-way-Delay, we now define one particular sample of such singletons. The idea of the sample is to select a particular binding of the parameters Src, Dst, and Type-P, then define a sample of values of parameter T. The means for defining the values of T is to select a beginning time T0, a final time Tf, and an average rate lambda, then define a pseudo-random Poisson process of rate lambda, whose values fall between T0 and Tf. The time interval between successive values of T will then average 1/lambda. ๐ŸŽ€

{Comment: Note that Poisson sampling is only one way of defining a sample. Poisson has the advantage of limiting bias, but other methods of sampling might be appropriate for different situations. We encourage others who find such appropriate cases to use this general framework and submit their sampling method for standardization.} ๐Ÿซถ

4.1. Metric Name: ๐Ÿท๏ธ

Type-P-One-way-Delay-Poisson-Stream

4.2. Metric Parameters: ๐Ÿงฉ

  • Src, the IP address of a host
  • Dst, the IP address of a host
  • T0, a time
  • Tf, a time
  • lambda, a rate in reciprocal seconds

4.3. Metric Units: ๐Ÿ“

A sequence of pairs; the elements of each pair are:

  • T, a time, and
  • dT, either a real number or an undefined number of seconds.

The values of T in the sequence are monotonic increasing. Note that T would be a valid parameter to Type-P-One-way-Delay, and that dT would be a valid value of Type-P-One-way-Delay. ๐Ÿ’œ

4.4. Definition: ๐Ÿ’…

Given T0, Tf, and lambda, we compute a pseudo-random Poisson process beginning at or before T0, with average arrival rate lambda, and ending at or after Tf. Those time values greater than or equal to T0 and less than or equal to Tf are then selected. At each of the times in this process, we obtain the value of Type-P-One-way-Delay at this time. The value of the sample is the sequence made up of the resulting <time, delay> pairs. If there are no such pairs, the sequence is of length zero and the sample is said to be empty. ๐ŸŒธ

4.5. Discussion: ๐Ÿ—ฃ๏ธ

The reader should be familiar with the in-depth discussion of Poisson sampling in the Framework document [1], which includes methods to compute and verify the pseudo-random Poisson process. ๐Ÿ“š

We specifically do not constrain the value of lambda, except to note the extremes. If the rate is too large, then the measurement traffic will perturb the network, and itself cause congestion. If the rate is too small, then you might not capture interesting network behavior. {Comment: We expect to document our experiences with, and suggestions for, lambda elsewhere, culminating in a "best current practices" document.} ๐Ÿ“

Since a pseudo-random number sequence is employed, the sequence of times, and hence the value of the sample, is not fully specified. Pseudo-random number generators of good quality will be needed to achieve the desired qualities. ๐ŸŽฒ

The sample is defined in terms of a Poisson process both to avoid the effects of self-synchronization and also to capture a sample that is statistically as unbiased as possible. {Comment: there is, of course, no claim that real Internet traffic arrives according to a Poisson arrival process.} The Poisson process is used to schedule the delay measurements. The test packets will generally not arrive at Dst according to a Poisson distribution, since they are influenced by the network. ๐ŸŒ

All the singleton Type-P-One-way-Delay metrics in the sequence will have the same values of Src, Dst, and Type-P. ๐Ÿฆ‹

Note also that, given one sample that runs from T0 to Tf, and given new time values T0' and Tf' such that T0 <= T0' <= Tf' <= Tf, the subsequence of the given sample whose time values fall between T0' and Tf' is also a valid Type-P-One-way-Delay-Poisson-Stream sample. ๐Ÿฅฐ

4.6. Methodologies:

okay so the methodologies basically come straight from two things ๐Ÿ’…

  • picking the specific times, using the specified Poisson arrival process, and
  • the methodologies discussion we already did for the singleton Type-P-One-way-Delay metric ๐Ÿฆ‹

bestie, we have to handle out-of-order arrival of test packets super carefully ๐Ÿ˜ญ it's totally possible for the Src to send one test packet at TS[i], then send a second one (later) at TS[i+1], and the Dst gets the second test packet at TR[i+1] and then gets the first one (later) at TR[i]. the packets swap places on the way, and that's giving chaos energy ๐ŸŒ€

4.7. Errors and Uncertainties:

so there are the errors and uncertainties for measuring each singleton value that makes up the sample, obviously ๐Ÿ™„ but we also need to check how accurate the Poisson process is, with respect to the wire-times when the test packets get sent. things can go wrong here for a few reasons, like problems with the pseudo-random number techniques that generate the Poisson arrival process, or jitter in the value of Hsource (which we already mentioned as an uncertainty in the singleton delay metric). the Framework document shows how to use the Anderson-Darling test to check that a Poisson process is accurate over small time frames โœจ {Comment: The goal is to ensure that test packets are sent "close enough" to a Poisson schedule, and avoid periodic behavior.} basically we want the sends to feel random but not too random, and definitely not on a regular beat ๐Ÿฅ

4.8. Reporting the metric:

you MUST report the calibration and context for the underlying singletons along with the stream. no skipping this, periodt ๐Ÿ’€

(see "Reporting the metric" for Type-P-One-way-Delay for the details ๐ŸŽ€)


5. Some Statistics Definitions for One-way Delay โœจ

okay now that we've got the sample metric, Type-P-One-way-Delay-Poisson-Stream, we can look at some stats you can pull from it ๐Ÿ“Š these are mostly here to show you what's possible, not to be the only way to do things ๐Ÿ’œ

5.1. Type-P-One-way-Delay-Percentile

so, given a Type-P-One-way-Delay-Poisson-Stream and a percent X between 0% and 100%, this is the Xth percentile of all the dT values in the Stream. when we compute it, undefined values get treated as infinitely large ๐Ÿ™ˆ which means the percentile itself could be undefined (informally, infinite). also, the Type-P-One-way-Delay-Percentile is undefined if the sample is empty. nothing to see here, just vibes ๐Ÿซ 

example time ๐Ÿ“ say we take a sample and get these results:

Stream1 = <
<T1, 100 msec>
<T2, 110 msec>
<T3, undefined>
<T4, 90 msec>
<T5, 500 msec>
>

then the 50th percentile would be 110 msec, since 90 msec and 100 msec are smaller and 110 msec and 'undefined' are larger ๐ŸŽฏ

note that if there's a real chance a packet with finite delay gets reported as lost, then a high percentile (90th or 95th) might come out as infinite instead of finite ๐Ÿ˜ฌ be careful with those!

5.2. Type-P-One-way-Delay-Median

given a Type-P-One-way-Delay-Poisson-Stream, this is the median of all the dT values in the Stream ๐Ÿ’– when computing the median, undefined values are treated as infinitely large (same as before, we're consistent bestie ๐Ÿซถ). and just like Type-P-One-way-Delay-Percentile, Type-P-One-way-Delay-Median is undefined if the sample is empty.

as the Framework document points out, the median only differs from the 50th percentile when the sample has an even number of values. in that case you take the mean of the two central values ๐Ÿงš

example: say we take a sample and get:

Stream2 = <
<T1, 100 msec>
<T2, 110 msec>
<T3, undefined>
<T4, 90 msec>
>

so the median would be 105 msec, the mean of 100 msec and 110 msec, which are the two central values โœจ math is slaying today ๐Ÿ”ฅ

5.3. Type-P-One-way-Delay-Minimum

given a Type-P-One-way-Delay-Poisson-Stream, this is the minimum of all the dT values in the Stream. when computing it, undefined values are treated as infinitely large. so the minimum could be undefined (informally, infinite) if every single dT value is undefined ๐Ÿ˜ญ and yep, the Type-P-One-way-Delay-Minimum is undefined if the sample is empty too.

in the example above, the minimum would be 90 msec ๐ŸŒธ

5.4. Type-P-One-way-Delay-Inverse-Percentile

given a Type-P-One-way-Delay-Poisson-Stream and a time duration threshold, this is the fraction of all the dT values in the Stream that are less than or equal to the threshold. the result could be as low as 0% (if every dT value goes over the threshold) or as high as 100% ๐ŸŒˆ and Type-P-One-way-Delay-Inverse-Percentile is undefined if the sample is empty.

in the example above, the Inverse-Percentile of 103 msec would be 50% ๐Ÿฆ‹


6. Security Considerations

okay this section is SERIOUS bestie, no cap ๐Ÿšจ but I'm still gonna explain it like a friend ๐Ÿซถ

conducting Internet measurements raises both security and privacy concerns. this memo doesn't specify an implementation of the metrics, so it doesn't directly affect the security of the Internet or the applications that run on it. however, implementations of these metrics must be mindful of security and privacy concerns ๐Ÿ”

there are two kinds of security concerns: potential harm caused by the measurements, and potential harm to the measurements. the measurements could cause harm because they're active, which means they inject packets into the network ๐Ÿ˜ฌ the measurement parameters MUST be carefully selected so the measurements inject trivial amounts of additional traffic into the networks they measure ๐Ÿ’… if they inject "too much" traffic, they can skew the results, and in extreme cases cause congestion and denial of service. not it being the main character of a DDoS ๐Ÿ’€

the measurements themselves could be harmed by routers giving measurement traffic a different priority than "normal" traffic, or by an attacker injecting fake measurement traffic. if routers can recognize measurement traffic and treat it separately, the measurements won't reflect actual user traffic ๐Ÿ™ˆ and if an attacker injects artificial traffic that gets accepted as legitimate, the loss rate will be artificially lowered. so the measurement methodologies SHOULD include appropriate techniques to reduce the probability that measurement traffic can be distinguished from "normal" traffic ๐Ÿ•ต๏ธโ€โ™€๏ธ authentication techniques, such as digital signatures, may be used where appropriate to guard against injected traffic attacks ๐Ÿ”‘

the privacy concerns of network measurement are limited by the active measurements described in this memo. unlike passive measurements, there can be no release of existing user data ๐ŸŽ€ phew, we're safe on that one.


7. Acknowledgements

special thanks to Vern Paxson of Lawrence Berkeley Labs for his helpful comments on issues of clock uncertainty and statistics ๐Ÿซถ thanks also to Garry Couch, Will Leland, Andy Scherrer, Sean Shapira, and Roland Wittig for several useful suggestions ๐Ÿ’œ the real ones ๐Ÿ˜ญ


8. References

[1] Paxson, V., Almes, G., Mahdavi, J. and M. Mathis, "Framework for IP Performance Metrics", RFC 2330, May 1998. ๐Ÿ“š

[2] Almes, G., Kalidindi, S. and M. Zekauskas, "A One-way Packet Loss Metric for IPPM", RFC 2680, September 1999. ๐Ÿ“š

[3] Mills, D., "Network Time Protocol (v3)", RFC 1305, April 1992. โฐ

[4] Mahdavi J. and V. Paxson, "IPPM Metrics for Measuring Connectivity", RFC 2678, September 1999. ๐Ÿ“š

[5] Postel, J., "Internet Protocol", STD 5, RFC 791, September 1981. ๐ŸŒ

[6] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. ๐Ÿ”‘

[7] Bradner, S., "The Internet Standards Process -- Revision 3", BCP 9, RFC 2026, October 1996. ๐Ÿ“œ


9. Authors' Addresses

these are the authors and how to reach them ๐Ÿ’Œ

Guy Almes
Advanced Network & Services, Inc.
200 Business Park Drive
Armonk, NY  10504
USA

Phone: +1 914 765 1120
EMail: almes@advanced.org


Sunil Kalidindi
Advanced Network & Services, Inc.
200 Business Park Drive
Armonk, NY  10504
USA

Phone: +1 914 765 1128
EMail: kalidindi@advanced.org


Matthew J. Zekauskas
Advanced Network & Services, Inc.
200 Business Park Drive
Armonk, NY 10504
USA

Phone: +1 914 765 1112
EMail: matt@advanced.org

Copyright (C) The Internet Society (1999). All Rights Reserved. ๐ŸŒธ

this document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. however, this document itself may not be modified in any way, such as by removing the copyright notice or references to the Internet Society or other Internet organizations, except as needed for the purpose of developing Internet standards in which case the procedures for copyrights defined in the Internet Standards process must be followed, or as required to translate it into languages other than English. ๐ŸŒ

the limited permissions granted above are perpetual and will not be revoked by the Internet Society or its successors or assigns. โ™พ๏ธ

this document and the information contained herein is provided on an "AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERING TASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. ๐Ÿซ 

Acknowledgement

funding for the RFC Editor function is currently provided by the Internet Society ๐Ÿ’œ thank you for keeping the lights on, fr fr โœจ

๐Ÿ“ฆ where this page comes from