RFC 8250
IPv6 Performance and Diagnostic Metrics (PDM) Destination Option
RFC 8250 ✨ IPv6 Performance and Diagnostic Metrics (PDM) Destination Option
Okay bestie, let's get into it 💅 This is RFC 8250, published by the Internet Engineering Task Force (IETF) in September 2017. It's a Standards Track document (ISSN 2070-1721), written by N. Elkins from Inside Products, R. Hamilton from Chemical Abstracts Service, and M. Ackermann from BCBS Michigan. The header doesn't list anything it obsoletes or updates, so no drama there 🫶
Abstract 🦋
Okay so the big idea is this: when the internet feels slow, we need to figure out why. This doc describes little optional headers that ride along in each packet, carrying sequence numbers and timing info 🕐 Those numbers become the raw material for measurements, and you can read them live or look back on them later.
The spec here is the Performance and Diagnostic Metrics (PDM) Destination Options header. The doc also covers the field limits, the calculations, and how PDM gets used for measurement. It's giving detective work 🔍
Status of This Memo 📜
This is an Internet Standards Track document.
It's a product of the Internet Engineering Task Force (IETF), so it represents the consensus of the IETF community. It went through public review and got the green light from the Internet Engineering Steering Group (IESG). If you want more on Internet Standards, check out Section 2 of RFC 7841 💖
For the current status, any errata, and how to send feedback, head to https://www.rfc-editor.org/info/rfc8250.
Copyright Notice ©️
Copyright (c) 2017 IETF Trust and the persons identified as the document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) as they were in effect on the date of publication. Please actually read these, because they spell out your rights and restrictions for this document 📖 Any Code Components pulled out of this document must include Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions. They're provided without warranty, as described in the Simplified BSD License.
Table of Contents 🗺️
1. Background ......................................................3
1.1. Terminology ................................................3
1.2. Rationale for Defined Solution .............................4
1.3. IPv6 Transition Technologies ...............................4
2. Measurement Information Derived from PDM ........................5
2.1. Round-Trip Delay ...........................................5
2.2. Server Delay ...............................................5
3. Performance and Diagnostic Metrics Destination Option Layout ....6
3.1. Destination Options Header .................................6
3.2. Performance and Diagnostic Metrics Destination Option ......6
3.2.1. PDM Layout ..........................................6
3.2.2. Base Unit for Time Measurement ......................8
3.3. Header Placement ...........................................9
3.4. Header Placement Using IPsec ESP Mode ......................9
3.4.1. Using ESP Transport Mode ...........................10
3.4.2. Using ESP Tunnel Mode ..............................10
3.5. Implementation Considerations .............................10
3.5.1. PDM Activation .....................................10
3.5.2. PDM Timestamps .....................................10
3.6. Dynamic Configuration Options .............................11
3.7. Information Access and Storage ............................11
4. Security Considerations ........................................11
4.1. Resource Consumption and Resource Consumption Attacks .....11
4.2. Pervasive Monitoring ......................................12
4.3. PDM as a Covert Channel ...................................12
4.4. Timing Attacks ............................................13
5. IANA Considerations ............................................13
6. References .....................................................14
6.1. Normative References ......................................14
6.2. Informative References ....................................14
Appendix A. Context for PDM .......................................15
A.1. End-User Quality of Service (QoS) ..........................15
A.2. Need for a Packet Sequence Number (PSN) ....................15
A.3. Rationale for Defined Solution .............................15
A.4. Use PDM with Other Headers .................................16
Appendix B. Timing Considerations .................................16
B.1. Calculations of Time Differentials .........................16
B.2. Considerations of This Time-Differential Representation ....18
B.2.1. Limitations with This Encoding Method ..................18
B.2.2. Loss of Precision Induced by Timer Value Truncation ....19
Appendix C. Sample Packet Flows ...................................20
C.1. PDM Flow - Simple Client-Server Traffic ....................20
C.1.1. Step 1 .................................................20
C.1.2. Step 2 .................................................21
C.1.3. Step 3 .................................................21
C.1.4. Step 4 .................................................23
C.1.5. Step 5 .................................................24
C.2. Other Flows ................................................24
C.2.1. PDM Flow - One-Way Traffic .............................24
C.2.2. PDM Flow - Multiple-Send Traffic .......................25
C.2.3. PDM Flow - Multiple-Send Traffic with Errors ...........26
Appendix D. Potential Overhead Considerations .....................28
Acknowledgments ...................................................30
Authors' Addresses ................................................301. Background 🌸
Okay, so to assess performance problems, measurements based on optional sequence numbers and timing may be embedded in each packet. Those measurements may be read in real time or after the fact. Very flexible, very iconic 💅
As defined in RFC 8200 [RFC8200], destination options are carried by the IPv6 Destination Options extension header. These options hold optional info that only needs to be examined by the IPv6 node named as the destination address in the IPv6 header. Routers and other "middleboxes" don't need to look at it 🙈 This document specifies the Performance and Diagnostic Metrics (PDM) destination option. It also covers the field limits, the calculations, and how PDM gets used in measurement.
1.1. Terminology 📚
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.
1.2. Rationale for Defined Solution 💡
The current IPv6 specification doesn't provide timing. It also doesn't have a similar field in the IPv6 main header or in any extension header. That's the gap the IPv6 PDM destination option fills ✨
The perks are:
- Real measurements of actual transactions 📊
- Ability to span organizational boundaries with consistent instrumentation 🤝
- No time synchronization needed between session partners ⏱️
- Ability to handle all transport protocols (TCP, UDP, the Stream Control Transmission Protocol (SCTP), etc.) in a uniform way 🧚
PDM lets you figure out quickly whether a latency problem lives in the network or in the server (application). Basically it's a fast way to do triage 🚑 For more on the background and usage of PDM, see Appendix A.
1.3. IPv6 Transition Technologies 🦋
On the road to full IPv6 deployment, transition technologies like translation or tunneling may get used. An IPv6 packet containing PDM might get dropped when those technologies are in play. For example, a translation technique (IPv6 to IPv4) that doesn't support or recognize the IPv6 Destination Options extension header may just drop the packet instead of translating it without the extension header. Rude 💀
Some devices in the network may also not handle multiple IPv6 extension headers correctly, and that includes the IPv6 Destination Option. For example, adding the PDM header to a packet may push the Layer 4 information to a spot in the packet where filtering logic can't see it, and then the packet may get dropped. The good news is this situation is expected to become rare over time 🌈
2. Measurement Information Derived from PDM 📈
Each packet holds info about the sender and receiver. In IP, that identifying info is called a "5-tuple" 🎀
The 5-tuple consists of:
SADDR: IP address of the sender
SPORT: Port for the sender
DADDR: IP address of the destination
DPORT: Port for the destination
PROTC: Upper-layer protocol (TCP, UDP, ICMP, etc.)PDM contains the following base fields (scale fields intentionally left out):
PSNTP : Packet Sequence Number This Packet
PSNLR : Packet Sequence Number Last Received
DELTATLR: Delta Time Last Received
DELTATLS: Delta Time Last SentThere are other fields in PDM for storing time scaling factors too. We'll get into those in Section 3 🧠
Combine this info with the 5-tuple and you can measure the following metrics:
- Round-trip delay 🔁
- Server delay 🖥️
2.1. Round-Trip Delay 🏓
Round-trip network delay is the time it takes for a packet to go from a source host to a destination host and then back to the source host. This measurement has already been defined, and its advantages and disadvantages are discussed in "A Round-trip Delay Metric for IPPM" [RFC2681] 🧐
2.2. Server Delay 🐢
Server delay is the interval between when a device receives a packet and when the first corresponding packet gets sent back in response. This might be "server processing time". It could also be a delay caused by acknowledgments. Server processing time includes the time the stack and application take together to return the response. The stack delay may be related to network performance too 👀 If this aggregate time is seen as a problem, and there's a need to clearly tell apart application processing time from stack delay (including whatever the network causes), then more client-based measurements are needed.
3. Performance and Diagnostic Metrics Destination Option Layout 🧩
3.1. Destination Options Header 🎁
The IPv6 Destination Options extension header [RFC8200] carries optional info that only needs to be examined by a packet's destination node(s). The Destination Options header is identified by a Next Header value of 60 in the immediately preceding header, and it's defined in RFC 8200 [RFC8200]. The IPv6 Performance and Diagnostic Metrics (PDM) destination option is implemented as an IPv6 Option carried in the Destination Options header. The main thing to remember: PDM does not require time synchronization ⏰💖
3.2.1. PDM Layout ✨
okay bestie, let's get into the actual layout 🎀 PDM carries six fields, and they're lined up like a squad:
SCALEDTLR: Scale for Delta Time Last Received
SCALEDTLS: Scale for Delta Time Last Sent
PSNTP : Packet Sequence Number This Packet
PSNLR : Packet Sequence Number Last Received
DELTATLR : Delta Time Last Received
DELTATLS : Delta Time Last SentAlignment is a whole vibe 🫶 The PDM fields follow the same convention IPv6 uses, so the multi-octet values in the Option Data field land on natural boundaries. Basically, a field that's n octets wide sits at an integer multiple of n octets from the start of the header, where n is 1, 2, 4, or 8 [RFC8200]. Neat, right? 💅
Here's how the TLV encoding looks:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Option Type | Option Length | ScaleDTLR | ScaleDTLS |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| PSN This Packet | PSN Last Received |
|-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Delta Time Last Received | Delta Time Last Sent |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Option Type 🔥
The Option Type is 0x0F. Same deal as IPv6 in general: the two high-order bits are how the option says how it should be processed. For PDM, those two bits MUST be set to 00. ✨
The third high-order bit tells you whether the Option Data can change on the way to the packet's final destination. For PDM, that third bit MUST be 0. So no changing data en route, bestie 🙈
Option Length 📏
This is an 8-bit unsigned integer that counts the octets in the option, not including the Option Type and Option Length fields. It MUST be set to 10. Ten, no more, no less 💖
Scale Delta Time Last Received (SCALEDTLR) 🍓
An 8-bit unsigned integer. It's the scaling value for the DELTATLR field, and it can be anything from 0 to 255. For the deep dive on timing and how the scaling values get formatted, go check out Appendix B 🧚
Scale Delta Time Last Sent (SCALEDTLS) 🌸
An 8-bit signed integer. It's the scaling value for the DELTATLS field, with possible values from 0 to 255.
Packet Sequence Number This Packet (PSNTP) 🦋
A 16-bit unsigned integer. This one will wrap 🔄 It's meant for analyzing packet traces.
It starts at a random number and goes up by one for each packet in the session flow of the 5-tuple. The random start is there to make it harder for someone to spoof and insert packets. Sneaky, iykyk 🕵️
Operating systems MUST implement a separate packet sequence number counter per 5-tuple. Every flow gets its own counter, no sharing!! 👑
Packet Sequence Number Last Received (PSNLR) 💜
A 16-bit unsigned integer. It holds the PSNTP of the packet last received on the 5-tuple. It's initialized to 0 🌈
Delta Time Last Received (DELTATLR) ⭐
A 16-bit unsigned integer, set according to the scale in SCALEDTLR:
Delta Time Last Received =
(send time packet n - receive time packet (n - 1))Delta Time Last Sent (DELTATLS) 💫
A 16-bit unsigned integer, set according to the scale in SCALEDTLS:
Delta Time Last Sent =
(receive time packet n - send time packet (n - 1))3.2.2. Base Unit for Time Measurement ⏱️
A time differential is always a whole number in a CPU. The unit is what makes the number mean something, whether that's hours, seconds, or nanoseconds 🤓 For PDM, the base time unit is 1 attosecond (asec). That gives every IP stack and hardware implementation a common unit to scale from, which is so iconic 👑
Here's the tea: PDM has to measure differentials that are super tiny and ones in a Delay/Disruption Tolerant Networking (DTN) environment, where delays can be huge 🌍 Squeezing a range that wide into just 16 bits means you need some scaling. Obsessed with that flex 😭
The conversion from the CPU's native time base to attoseconds sounds like it'd give you an astronomically big number, but it's actually straightforward. You multiply by the right power of 10. Then, to fit the value into DELTATLR or DELTATLS, you shift it by some number of bits and keep the 16 high-order (most significant) bits. The number of bits you shifted becomes the scaling factor, stored as SCALEDTLR or SCALEDTLS, respectively. For more on this, see Appendix B 📖
3.3. Header Placement 🎀
The PDM destination option is placed as defined in RFC 8200 [RFC8200]. You might have a choice of where the Destination Options header goes. If you're using Encapsulating Security Payload (ESP) mode, check Section 3.4 of this document for how to place the PDM Destination Options header 🦋
Here's a rule that's giving strict: for each IPv6 packet header, PDM MUST NOT appear more than once. ❌ An encapsulated packet, though, MAY contain a separate PDM for each encapsulated IPv6 header. Each layer gets its own, bestie 💖
3.4. Header Placement Using IPsec ESP Mode 🔐
IPsec ESP is defined in [RFC4303] and it's super widely used. Section 3.1.1 of [RFC4303] covers where Destination Options headers go.
PDM placement depends on whether ESP is in tunnel mode or transport mode 🧚
With ESP, there's no 5-tuple, because there are no port numbers 🙃 So an ESP flow should be identified only by SADDR, DADDR, and PROTC. The Security Parameter Index (SPI) numbers SHOULD be ignored when you're considering the flow that PDM information is measured over.
3.4.1. Using ESP Transport Mode 🚚
Destination options can go before or after ESP, or both. But if you're using PDM in ESP transport mode, PDM MUST be placed after the ESP header so it doesn't leak information 🤫
3.4.2. Using ESP Tunnel Mode 🌀
In tunnel mode, both the outer and inner sets of IP headers can have destination options before or after ESP, or both 💜 A tunnel endpoint that creates a new packet can decide to use PDM on its own, separate from whatever PDM the original packet used. That's how the two tunnel endpoints can measure delay between each other ⭐
3.5. Implementation Considerations 🛠️
3.5.1. PDM Activation 🔘
An implementation should provide an interface to turn PDM on or off. This specification RECOMMENDS having PDM off by default. Safe and cozy 🧸
PDM MUST NOT be turned on just because a packet arrives with a PDM header. Another device could have spoofed it, so don't trust it blindly 🙅♀️
3.5.2. PDM Timestamps ⏰
PDM timestamps are meant to separate wire time from server or host time. Heads up though, they might have to count some host processing time as network latency 🙈
Section 10.2 of RFC 2330 [RFC2330] ("Framework for IP Performance Metrics") describes two notions of "wire time". These are only defined in terms of an Internet host H observing an Internet link L at a particular location:
- For a given IP packet P, the "wire arrival time" of P at H on L is the first time T at which any bit of P has appeared at H's observational position on L.
- For a given IP packet P, the "wire exit time" of P at H on L is the first time T at which all the bits of P have appeared at H's observational position on L.
This specification doesn't pin down H's exact observational position on L. That's up to the deployment setups 🏗️ Still, the spot where PDM timestamps are taken SHOULD be as close to the physical network interface as possible. Not every implementation can hit that ideal level of measurement, and that's okay 🫶
3.6. Dynamic Configuration Options 🎛️
If the PDM Destination Options header is used, it MAY be turned on for all packets flowing through the host. It can also be applied to an upper-layer protocol (TCP, UDP, SCTP, etc.), a local port, or an IP address only. It's up to the implementation to decide 💅
3.7. Information Access and Storage 📦
PDM measurement information may be made accessible to higher layers or the user itself. Just like turning PDM on, the implementation may also provide an interface to indicate whether it's received 👀
PDM information may be stored, if desired. If a packet with PDM information arrives and the information should be stored, the upper layers may be notified. The implementation SHOULD define a configurable maximum lifetime after which the information can be removed, plus a configurable maximum amount of memory that can be allocated for PDM information. Keep it tidy, bestie ✨
4. Security Considerations 🚨
PDM may introduce some new security weaknesses. Okay, serious moment 🥺
4.1. Resource Consumption and Resource Consumption Attacks 💥
PDM needs to calculate the deltas for time and keep track of the sequence numbers. That means control blocks that live in memory may be kept at the end hosts, one per 5-tuple 🧠
A limit on how much memory is used SHOULD be implemented.
Without a memory limit, any time a control block is kept in memory, an attacker can try to misuse those control blocks to cause excessive resource consumption. That could be used to compromise the end host. 😱
PDM is used only at the end hosts, and memory is used only at the end host, not at routers or middleboxes. Phew 💖
4.2. Pervasive Monitoring 👁️
Since PDM passes in the clear, there's a concern that the data could be used to fingerprint the system or somehow learn about the contents of the payload 😬
Let's talk fingerprinting of the end host first. Seeing the pattern of deltas or the absolute values could hint at the speed of the end host, like whether it's a very fast system or an older, slow device. That could be useful to an attacker. However, if the attacker has access to PDM, they also have access to the entire packet, and they could make that same deduction just from the time frames between packets without PDM 🙄
As for deducing the content of the payload, like web pages, user names, user passwords, and so on, it looks to us like PDM is pretty unhelpful there 🙌 That said, being able to separate wire time from processing time might give an attacker extra information. It's conceivable that an attacker could try to figure out the type of application in use by noting the server time and payload length. Some encryption algorithms try to obfuscate the packet length to avoid exactly that kind of vulnerability. In the future, encryption algorithms may want to obfuscate the server time too 🔐🦋
4.3. PDM as a Covert Channel ✨
Okay so PDM gives you a bunch of fields in the packet that could be used to leak data 👀. But ngl, there's no real reason to think PDM would be picked over some other part of the payload or another extension header 💅.
A firewall or another device could sanity-check the PDM fields, but randomly assigned sequence numbers and delta times are expected to vary a LOT, so that's messy 🌀. The biggest problem, though, is how an attacker would even get access to PDM to leak anything. They'd have to either compromise the end host or be a Man in the Middle (MitM). Either one could change the fields, but then the other end host would be getting sequence numbers and deltas that make zero sense 🙈.
It's conceivable that someone could compromise an end host and make it start sending packets with PDM without the host knowing 😳. But again, the bigger problem is that the end host got compromised in the first place. Once that's happened, the attacker probably has way better ways to leak data 💀.
Having said that, if a PDM-aware middlebox or an implementation (destination host) notices some number of "nonsensical" sequence numbers or timing information, it could take action. It could block that traffic, discard it, or send an alert 🚨🫶.
4.4. Timing Attacks ⏱️
The fact that PDM can help separate node processing time from network latency is what makes it so valuable for performance monitoring 📈. BUT, that same characteristic is exactly what could be misused to set up new types of timing attacks against protocols and implementations 😭.
Depending on the cryptographic protocol being used, it may be possible to leak the credentials of the device 🔐. For example, if an attacker can see that PDM is being used, they might use PDM to launch a timing attack against the keying material used by the cryptographic protocol. Yikes, bestie 💀.
An implementation may want to make sure PDM is only enabled for certain IP addresses or only for some ports 🎯. Also, the implementation SHOULD require an explicit restart of monitoring after a certain time period (for example, after 1 hour). That way PDM isn't accidentally left on after, say, debugging is done 🧹✨.
Even so, if you're using PDM, a user "Consent to be Measured" SHOULD be a prerequisite 📋🎀. Consent is common in enterprises and with some subscription services, so it's not weird. The actual wording of "Consent to be Measured" will differ by site, but it SHOULD make clear that the traffic is being measured for Quality of Service (QoS) and to help with diagnostics. It should also make clear that there may be potential risks from certain vulnerabilities if the traffic is captured during a diagnostic session 🦋.
5. IANA Considerations 🌸
IANA has registered a Destination Option Type assignment with the act bits set to 00 and the chg bit set to 0 from the "Destination Options and Hop-by-Hop Options" sub-registry of the "Internet Protocol Version 6 (IPv6) Parameters" registry [RFC2780] at <https://www.iana.org/assignments/ipv6-parameters/> 🗂️💜.
Hex Value Binary Value Description Reference
act chg rest
---------------------------------------------------------------------
0x0F 00 0 01111 Performance and RFC 8250
Diagnostic Metrics (PDM)6. References 📚
6.1. Normative References
- [RFC1122] Braden, R., Ed., "Requirements for Internet Hosts - Communication Layers", STD 3, RFC 1122, DOI 10.17487/RFC1122, October 1989,
<https://www.rfc-editor.org/info/rfc1122>. - [RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997,
<https://www.rfc-editor.org/info/rfc2119>. - [RFC2681] Almes, G., Kalidindi, S., and M. Zekauskas, "A Round-trip Delay Metric for IPPM", RFC 2681, DOI 10.17487/RFC2681, September 1999,
<https://www.rfc-editor.org/info/rfc2681>. - [RFC2780] Bradner, S. and V. Paxson, "IANA Allocation Guidelines For Values In the Internet Protocol and Related Headers", BCP 37, RFC 2780, DOI 10.17487/RFC2780, March 2000,
<https://www.rfc-editor.org/info/rfc2780>. - [RFC4303] Kent, S., "IP Encapsulating Security Payload (ESP)", RFC 4303, DOI 10.17487/RFC4303, December 2005,
<https://www.rfc-editor.org/info/rfc4303>. - [RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017,
<https://www.rfc-editor.org/info/rfc8174>. - [RFC8200] Deering, S. and R. Hinden, "Internet Protocol, Version 6 (IPv6) Specification", STD 86, RFC 8200, DOI 10.17487/RFC8200, July 2017,
<https://www.rfc-editor.org/info/rfc8200>.
6.2. Informative References
- [RFC2330] Paxson, V., Almes, G., Mahdavi, J., and M. Mathis, "Framework for IP Performance Metrics", RFC 2330, DOI 10.17487/RFC2330, May 1998,
<https://www.rfc-editor.org/info/rfc2330>. - [TCPM] Scheffenegger, R., Kuehlewind, M., and B. Trammell, "Encoding of Time Intervals for the TCP Timestamp Option", Work in Progress, draft-trammell-tcpm-timestamp-interval-01, July 2013.
Appendix A. Context for PDM 🧠
A.1. End-User Quality of Service (QoS)
The timing values in PDM embedded in the packet are used to estimate QoS as experienced by an end-user device ⏳.
For many applications, the key user performance indicator is response time. When the end user is an individual, they're generally indifferent to what's happening along the network. What they really care about is how long it takes to get a response back 😤. And this isn't just about personal convenience. In many cases, rapid response is critical to the business being conducted 💼.
Low, reliable, and acceptable response times are NOT just "nice to have" 🚑. On many networks, the impact can be financial hardship or can endanger human life. In some cities, the emergency police contact system runs over IP, all levels of law enforcement use IP networks, and transactions on our stock exchanges are settled using IP networks 📞📈. The critical nature of these activities to our daily lives and financial well-being demands a simple solution to support response-time measurements 🫶.
A.2. Need for a Packet Sequence Number (PSN)
When you're doing network diagnostics on an end-to-end connection, it often becomes necessary to isolate the factors along the network path that are causing problems 🔍. Diagnostic data may be collected at multiple places along the path (if possible), or at the source and destination. Then, in post-collection processing, the diagnostic data for each packet at different observation points has to be matched for proper measurements 🧩. A sequence number in each packet gives you a sufficient basis for that matching process. If need be, the timing fields can be used along with the sequence number to ensure uniqueness 🔢.
This method of collecting data along the path is especially useful for figuring out where packet loss or packet corruption is happening 🕵️♀️.
The packet sequence number needs to be unique in the context of the session (5-tuple) 💖.
A.3. Rationale for Defined Solution
One of the important jobs of PDM is letting you quickly dispatch the right set of diagnosticians 🚀. Within network or server latency there can be lots of components, and the diagnostician's job is to rule each one out until the culprit is found 🕵️.
PDM fits into this diagnostic picture by quickly telling you how to escalate 🆙. It points to either the network area or the server area. Within server latency, PDM does NOT tell you whether the bottleneck is in the IP stack, the application, or buffer allocation. Within network latency, PDM does NOT tell you which network segment or middlebox is at fault 🙃.
What PDM does tell you is whether the problem is in the network or the server 🎉.
A.4. Use PDM with Other Headers 🎀
For diagnostics, you may want to use PDM with other headers (Layer 2, Layer 3, etc). For example, if a technician (or tool) is looking at a packet capture, they'd have the Layer 2 header, IP header (v6 or v4), TCP header, UDP header, ICMP header, SCTP header, or other headers all available inside that capture 📦. Everything gets looked at together to make sense of the packet flow. The technician or processing tool can analyze, report, or ignore the PDM data, as necessary 🧐.
For an example of how PDM can help with TCP retransmission problems, please look at Appendix C 🔁.
Appendix B. Timing Considerations ⏰
B.1. Calculations of Time Differentials ⏱️
Okay bestie, so when SCALEDTLR or SCALEDTLS shows up, the processing described here applies to both of them equally 💖. No need to say it twice!
The time counter in a CPU is a binary whole number counting milliseconds (msec), microseconds (usec), or even picoseconds (psec) 🤓. Turning any of those into attoseconds (asec) means multiplying by 10 raised to some exponent. Peep this table of equalities:
Base value = # of sec = # of asec 1000s of asec
--------------- ------------- ------------- -------------
1 second 1 sec 10**18 asec 1000**6 asec
1 millisecond 10**-3 sec 10**15 asec 1000**5 asec
1 microsecond 10**-6 sec 10**12 asec 1000**4 asec
1 nanosecond 10**-9 sec 10**9 asec 1000**3 asec
1 picosecond 10**-12 sec 10**6 asec 1000**2 asec
1 femtosecond 10**-15 sec 10**3 asec 1000**1 asecSo say your time differential is in microseconds 🧁. Each microsecond is 10**12 asec, so you multiply by 10**12 to get the attoseconds. If it's in nanoseconds, you multiply by 10**9 instead. Easy peasy 🎀.
The result is a binary value that needs to get shortened by a bunch of bits so it fits into the 16-bit PDM delta field. The next step is to divide by 2 until the value fits in just 16 significant bits ✨. The exponent in the last column of the table helps here: your initial scaling factor is that exponent multiplied by 10. That's the minimum number of low-order bits that get shifted out or discarded 💅. It's basically dividing the time value by 1024 raised to that exponent.
The value might still be too big for 16 bits, which is totally fine, because you can normalize it by shifting out more bits (dividing by 2) until it fits in the delta field 🫶. Every extra bit you shift out gets added to the scaling factor. That scaling factor, the total number of low-order bits dropped, is your SCALEDTLR or SCALEDTLS value 🔥.
Let's do an example. Say an app has these start and finish timer values (hex, in microseconds):
Finish: 27C849234 usec (02:57:58.997556)
-Start: 27C83F696 usec (02:57:58.957718)
========== ============== ==========================
Difference 9B9E usec 0.039838 sec or 39838 usecTo convert that differential to binary attoseconds, multiply the microseconds by 10**12. Then divide by 1024**4, or just discard 40 bits from the right 🍓. You end up with 36232, which is 8D88 in hex, with a scaling factor (SCALEDTLR/SCALEDTLS) of 40. Iconic 👑.
Now presume the time differential is bigger, like 32.311072 seconds, which is 32311072 usec. Each microsecond is 10**12 asec, so multiply by 10**12 to get the hex value 1C067FCCAE8120000. Using the initial scaling factor of 40, drop the last 10 characters (40 bits), which leaves 1C067FC. That won't fit in a delta field because it's 25 bits long 😭. So shift right another 9 bits, which gives a delta value of E033 with a resulting scaling factor of 49.
When the time differential is a small number, the exponent trick doesn't help you pick the scaling value, no matter what time unit you started with 🙈. Take 3 seconds, for example:
3 seconds = 3*10**18 asec (decimal)
= 29A2241AF62C0000 asec (hexadecimal)If you just truncate the last 60 bits, you end up with a delta value of 2 and a scaling factor of 60. But what you actually wanted was a delta value with more significant digits 💀. The most precision you can store this value with in 16 bits is A688, with a scaling factor of 46. Much better, periodt 💅.
B.2. Considerations of This Time-Differential Representation 🧠
There are two things to think about with this representation of a time differential ✨. First: are there any limits on the maximum or minimum time differential you can express with a delta value and a scaling factor? Second: how much imprecision does this method introduce? Let's get into it 🦋.
B.2.1. Limitations with This Encoding Method 🚧
The DELTATLS and DELTATLR fields only store the 16 most significant bits of the time-differential value 💜. So the range, not counting the scaling factor, goes from 0 to 65535, which is a max of 2**16 - 1. This method is described further in [TCPM] 📚.
The actual magnitude of the time differential is set by the scaling factor. SCALEDTLR and SCALEDTLS are 8-bit unsigned integers, so the scaling factor goes from 0 to 255 🌈. The smallest number you can represent is a 1 in the delta field with a 0 in the matching scale field. That's 1 attosecond! So yes, PDM can measure extremely tiny time differentials 🔬.
On the other end, the max delta value is 65535, or FFFF in hex. If you use the max scale value of 255, the time differential you can represent is 65535*2**255, which is over 3*10**55 years. Basically forever 😭🫶. So there's really no practical limit on the time differential that can be represented.
B.2.2. Loss of Precision Induced by Timer Value Truncation 🌸
PDM stores the DELTATLR and DELTATLS values as 16-bit unsigned integers 🎀. So any time the precision is more than those 16 bits, the trailing bits get truncated, and you lose a bit of precision 💔.
Any time-differential value smaller than 65536 asec can be stored exactly in DELTATLR or DELTATLS, because it needs at most 16 bits to represent ✨.
Since PDM measures time differentials in attoseconds, the range of values that would get truncated to the same encoded value is 2**((Scale) - 1) asec 🧮.
For example, the smallest time differential that gets truncated to fit into a delta field is:
1 0000 0000 0000 0000 = 65536 asecThat one gets encoded as a delta value of 8000 (hex) with a scaling factor of 1. The value:
1 0000 0000 0000 0001 = 65537 asecalso gets encoded as a delta value of 8000 with a scaling factor of 1. And this one is actually the largest value that gets truncated to that same encoded value 🙈. When the scale value is 1, the value range works out to 2**1 - 1, or 1 asec. You can see that's the difference between the minimum and maximum values 🦋.
The scaling factor is defined as the number of low-order bits truncated to shrink the value so it fits into a 16-bit delta field 💅. If, for example, you had to truncate 12 bits, the loss of precision would depend on which bits got cut. The range of those values is:
0000 0000 0000 = 0 asec
to
1111 1111 1111 = 4095 asecSo the minimum loss of precision is 0 asec, where the delta value exactly represents the time differential, and the maximum loss of precision is 4095 asec 😤. As mentioned above, a scaling factor of 12 means the max loss of precision is 2**12 - 1 asec, which is 4095 asec.
Now compare that loss to the actual time differential. The range of actual time-differential values that would have this loss of precision goes from:
1000 0000 0000 0000 0000 0000 0000 = 2**27 asec or 134217728 asec
to
1111 1111 1111 1111 1111 1111 1111 = 2**28 - 1 asec or 268435455 asecGranted, these are small values, but the point is that any value between these two will have a max loss of precision of 4095 asec. That's about 0.00305% for the first value, as encoded, and at most 0.001526% for the second 🥹. These max-loss percentages hold for all scaling values too, so you can always count on them 💖.
Appendix C. Sample Packet Flows 🛼
C.1. PDM Flow - Simple Client-Server Traffic 📦
Here's a sample simple flow for PDM, with one packet sent from Host A and one packet received by Host B 💌. PDM doesn't need time synchronization between Host A and Host B, which is honestly such a relief 🙌. The calculations for meaningful network diagnostics metrics are shown below each packet sent or received 🧁.
C.1.1. Step 1 🎬
Packet 1 is sent from Host A to Host B. Host A's time is initially set to 10:00AM ⏰.
The sender saves the time and the packet sequence number internally 🧠. The packet sequence number and the delta times get sent in the packet.
Packet 1
+----------+ +----------+
| | | |
| Host | ----------> | Host |
| A | | B |
| | | |
+----------+ +----------+
PDM Contents:
PSNTP : Packet Sequence Number This Packet: 25
PSNLR : Packet Sequence Number Last Received: -
DELTATLR : Delta Time Last Received: -
SCALEDTLR: Scale of Delta Time Last Received: 0
DELTATLS : Delta Time Last Sent: -
SCALEDTLS: Scale of Delta Time Last Sent: 0
Internally, within the sender, Host A, it must keep:
Packet Sequence Number of the last packet sent: 25
Time the last packet was sent: 10:00:00Note: the initial PSNTP from Host A starts at a random number, in this case 25 🎲. The times in these examples are in seconds, just to keep things simple 🫧.
C.1.2. Step 2 📬
Packet 1 is received at Host B. Its time is set to 1 hour later than Host A's, so 11:00AM in this case 🕚.
Internally, within the receiver, Host B, it must note the following:
Packet Sequence Number of the last packet received: 25
Time the last packet was received : 11:00:03Note: this timestamp is in Host B time. It has nothing whatsoever to do with Host A time 🙅♀️. The packet sequence number of the last packet received becomes PSNLR, which gets sent out in the packet Host B sends in the next step 💜. The timestamp of the packet last received (noted above) is the input for calculating the DELTATLR value that goes out in that same next-step packet 🦋.
C.1.3. Step 3 ✨
okay so Packet 2 is sent by Host B to Host A 🦋. Quick note: the initial packet sequence number (PSNTP) from Host B starts at a random number, and in this case that number is 12 🎲. Before sending, Host B does a little calculation of deltas 🧮. Since Host B knows when it's sending the packet and when it got the previous one, it can do this:
DELTATLR = send time (packet 2) - receive time (packet 1)Note: both the send time and the receive time are saved inside Host B. They do not travel in the packet 🙅♀️. Only the change in values (the delta) goes in the packet, and that's the DELTATLR value 💖.
Assume that inside Host B we have this:
Packet Sequence Number of the last packet received: 25
Time the last packet was received: 11:00:03
Packet Sequence Number of this packet: 12
Time this packet is being sent: 11:00:07So now we can work out the delta to send in the packet:
4 seconds = 11:00:07 - 11:00:03 = 3782DACE9D900000 asecThis is the derived metric: server delay ⏱️. The time scaling factors have to be converted. Here the time differential is DE0B and the scaling factor is 2E, which is 46 in decimal 🔥. Then those values, along with the packet sequence numbers, get sent to Host A like this:
Packet 2
+----------+ +----------+
| | | |
| Host | <---------- | Host |
| A | | B |
| | | |
+----------+ +----------+
PDM Contents:
PSNTP : Packet Sequence Number This Packet: 12
PSNLR : Packet Sequence Number Last Received: 25
DELTATLR : Delta Time Last Received: DE0B (4 seconds)
SCALEDTLR: Scale of Delta Time Last Received: 2E (46 decimal)
DELTATLS : Delta Time Last Sent: -
SCALEDTLS: Scale of Delta Time Last Sent: 0The one metric still left to calculate is the round-trip delay 🎀. Host A will work that out when it receives packet 2 👑.
C.1.4. Step 4 🌸
Packet 2 lands at Host A. Remember that Host A's clock is set 1 hour earlier than Host B's 🕐. Internally, Host A has to note this:
Packet Sequence Number of the last packet received: 12
Time the last packet was received : 10:00:12Note: this timestamp is in Host A time. It has nothing whatsoever to do with Host B time 🙅♀️.
So Host A can now calculate the total end-to-end time, like this:
End-to-End Time = Time Last Received - Time Last SentFor example, packet 25 was sent by Host A at 10:00:00, and packet 12 was received by Host A at 10:00:12, so:
End-to-End time = 10:00:12 - 10:00:00 or 12 (server and network
round-trip delay combined).This time may also be called "total overall Round-Trip Time (RTT)", which includes network RTT and host response time 🧠.
We're going to call this derived metric "Delta Time Last Sent" (DELTATLS) ✨.
Now the round-trip delay can be calculated. The formula is:
Round-trip delay =
(Delta Time Last Sent - Delta Time Last Received)Or, with our numbers:
Round-trip delay = 12 - 4 or 8The only problem at this point is that all of these metrics live inside Host A and aren't exposed in any packet 😭. To fix that, we need a third packet 🦋.
Note: this simple example assumes one send and one receive. That's only done to explain how PDM works. When multiple packets come back, you'd take the time in the last packet in the sequence. The calculations for those timings and the smart processing of the data are the job of post-processing 💅.
C.1.5. Step 5 🎀
Packet 3 is sent from Host A to Host B 💜.
+----------+ +----------+
| | | |
| Host | ----------> | Host |
| A | | B |
| | | |
+----------+ +----------+
PDM Contents:
PSNTP : Packet Sequence Number This Packet: 26
PSNLR : Packet Sequence Number Last Received: 12
DELTATLR : Delta Time Last Received: 0
SCALEDTLS: Scale of Delta Time Last Received 0
DELTATLS : Delta Time Last Sent: A688 (scaled value)
SCALEDTLR: Scale of Delta Time Last Received: 30 (48 decimal)To calculate the two-way delay, any packet-capture device can look at these packets and do what it needs to do 🔍🦋.
C.2. Other Flows 🧚
So far we've looked at a simple flow where one packet is sent and one comes back. Let's see how PDM is useful in other kinds of flows 💖.
C.2.1. PDM Flow - One-Way Traffic 🌈
The flow on a particular session doesn't have to be send-and-receive 🙈. Let's look at a few other situations. In a one-way flow, you might see something like this:
Note: the time is expressed in generic units to keep things simple. These values don't represent attoseconds, microseconds, or any other real unit of time ⏳.
Packet Sender PSN PSN Delta Time Delta Time
This Packet Last Recvd Last Recvd Last Sent
=====================================================================
1 Server 1 0 0 0
2 Server 2 0 0 5
3 Server 3 0 0 12
4 Server 4 0 0 20What does this mean, and how is it useful?
In a one-way flow, only the Delta Time Last Sent gets used. Remember, Delta Time Last Sent is the gap between one packet being sent from a device and the next one going out. It's a measure of throughput for the sender, from the sender's point of view 📈. In other words, it shows how fast the application itself (stack time included) can send packets 🏃♀️.
How might that help? Say you're having a performance issue at the client and you see that packet 2, for example, was sent 5 time units after the server sent packet 1, but it takes 10 times that long to reach the destination 😱. Then you can safely conclude that there are delays on the path, somewhere other than the server, that might be causing the delivery problem for that packet 🛣️. Those delays could come from network links, middleboxes, and so on 🔧.
True one-way traffic is pretty rare 🤷♀️. What people usually mean by "one-way" traffic is an application like FTP, where a group of packets (say, a TCP window's worth) gets sent and then the sender waits for an acknowledgment ⏸️. That kind of flow actually falls into the "multiple-send" traffic model 💅.
C.2.2. PDM Flow - Multiple-Send Traffic 💜
Let's assume two packets are sent from the server, and then the client sends an ACK. A TCP flow does this, per RFC 1122 [RFC1122], Section 4.2.3 🦋. Packets 1 and 2 come from the server, then an ACK comes from the client, and packet 4 starts a second sequence from the server 🔁.
Packet Sender PSN PSN Delta Time Delta Time
This Packet Last Recvd Last Recvd Last Sent
=====================================================================
1 Server 1 0 0 0
2 Server 2 0 0 5
3 Client 1 2 20 0
4 Server 3 1 10 15How might this be used? 🤔
Notice that in packet 3, the client has a Delta Time Last Received value of 20. Remember:
DELTATLR = send time (packet 3) - receive time (packet 2)So what do we know now? In this case, Delta Time Last Received is the time the client took to send its next packet, which is its processing time ⚙️.
How to interpret this depends on what's actually being sent 👀. Remember that PDM isn't used on its own. It supplements the fields found in other headers 🎀. Here are two examples:
- The client is sending a standalone TCP ACK. You'd spot this by looking at the payload length in the IPv6 header and the TCP Acknowledgment field in the TCP header. So in this case, the client takes 20 time units to send back the ACK. That may or may not be interesting 🤷♀️.
- The client is sending data with the packet. Again, you'd find this by looking at the payload length in the IPv6 header and the TCP Acknowledgment field in the TCP header. So in this case, the client takes 20 time units to send back data. This could represent "User Think Time" 🧠. Again, it may or may not be interesting on its own. But if there's a performance problem receiving data at the server, then combined with RTT or other packet timing information, it can be super interesting 🔥.
Of course, you also need to look at the PSN Last Received field to make sure you're interpreting this data correctly. In other words, you want to confirm that the Delta Time Last Received corresponds to the packet you care about 🔍💖.
The benefit of PDM is that this information is now available in a uniform manner for all applications and all protocols, without extensive changes to applications 🙌✨.
C.2.3. PDM Flow - Multiple-Send Traffic with Errors ✨
Okay bestie, now it gets messy 😭 Let's look at how PDM can help when TCP has to retransmit, and what extra info it adds to the TCP header 🦋
Assume that three packets are sent with each send from the server.
From the server, this is what is seen:
Pkt Sender PSN PSN Delta Time Delta Time TCP Data
This Pkt Last Recvd Last Recvd Last Sent SEQ Bytes
=====================================================================
1 Server 1 0 0 0 123 100
2 Server 2 0 0 5 223 100
3 Server 3 0 0 5 333 100The client, however, does not receive all the packets. 💔 From the client, this is what is seen for the packets sent from the server:
Pkt Sender PSN PSN Delta Time Delta Time TCP Data
This Pkt Last Recvd Last Recvd Last Sent SEQ Bytes
=====================================================================
1 Server 1 0 0 0 123 100
2 Server 3 0 0 5 333 100Let's assume that the server now retransmits the packet. 🔁 (Obviously, a duplicate acknowledgment sequence for fast retransmit or a retransmit timeout would occur. To illustrate the point, these packets are being left out.)
So, if a TCP retransmission is done, then from the client, this is what is seen for the packets sent from the server:
Pkt Sender PSN PSN Delta Time Delta Time TCP Data
This Pkt Last Recvd Last Recvd Last Sent SEQ Bytes
=====================================================================
1 Server 4 0 0 30 223 100The server has resent the old packet 2 with a TCP sequence number of 223. 🎀 The retransmitted packet now has a PSN This Packet value of 4.
The Delta Time Last Sent is 30 -- in other words, the time between sending the packet with a PSN of 3 and this current packet. ⏱️
Let's say that packet 4 is lost again. 😩 Then, after some amount of time (RTO), the packet with a TCP sequence number of 223 is resent.
From the client, this is what is seen for the packets sent from the server:
Pkt Sender PSN PSN Delta Time Delta Time TCP Data
This Pkt Last Recvd Last Recvd Last Sent SEQ Bytes
====================================================================
1 Server 5 0 0 60 223 100If this packet now arrives at the destination, one has a very good idea that packets exist that are being sent from the server as retransmissions and not arriving at the client. 👀 This is because the PSN of the resent packet from the server is 5 rather than 4. If we had used the TCP sequence number alone, we would never have seen this situation. The TCP sequence number in all situations is 223. 🙈
This situation would be experienced by the user of the application (the human being actually sitting somewhere) as "hangs" or long delays between packets. 😤 On large networks, to diagnose problems such as these where packets are lost somewhere on the network, one has to take multiple traces to find out exactly where.
The first thing to do is to start with doing a trace at the client and the server, so that we can see if the server sent a particular packet and the client received it. If the client did not receive it, then we start tracking back to trace points at the router right after the server and the router right before the client. Did they get these packets that the server has sent? This is a time-consuming activity. 🥱
With PDM, we can speed up the diagnostic time because we may be able to use only the trace taken at the client to see what the server is sending. ✨💅
Appendix D. Potential Overhead Considerations 🧮
One might wonder about the potential overhead of PDM. First, PDM is entirely optional. That is, a site may choose to implement PDM or not, as they wish. If they are happy with the costs of PDM versus the benefits, then the choice should be theirs. 💖
Below is a table outlining the potential overhead in terms of additional time to deliver the response to the end user for various assumed RTTs:
Bytes RTT Bytes Bytes New Overhead
in Packet Per Millisec in PDM RTT
====================================================================
1000 1000 milli 1 16 1016.000 16.000 milli
1000 100 milli 10 16 101.600 1.600 milli
1000 10 milli 100 16 10.160 0.160 milli
1000 1 milli 1000 16 1.016 0.016 milliBelow are two examples of actual RTTs for packets traversing large enterprise networks. 🏢
The first example is for packets going to multiple business partners:
Bytes RTT Bytes Bytes New Overhead
in Packet Per Millisec in PDM RTT
====================================================================
1000 17 milli 58 16 17.360 0.360 milliThe second example is for packets at a large enterprise customer within a data center. Notice that the scale is now in microseconds rather than milliseconds: 🤏
Bytes RTT Bytes Bytes New Overhead
in Packet Per Microsec in PDM RTT
====================================================================
1000 20 micro 50 16 20.320 0.320 microAs with other diagnostic tools, such as packet traces, a certain amount of processing time will be required to create and process PDM. Since PDM is lightweight (has only a few variables), we expect the processing time to be minimal. 🌸
Acknowledgments 🫶
The authors would like to thank Keven Haining, Al Morton, Brian Trammell, David Boyes, Bill Jouris, Richard Scheffenegger, and Rick Troth for their comments and assistance. We would also like to thank Tero Kivinen and Jouni Korhonen for their detailed and perceptive reviews. 💜
Authors' Addresses 📮
Nalini Elkins Inside Products, Inc. 36A Upper Circle Carmel Valley, CA 93924 United States of America
Phone: +1 831 659 8360 Email: nalini.elkins@insidethestack.com URI: http://www.insidethestack.com
Robert M. Hamilton Chemical Abstracts Service A Division of the American Chemical Society 2540 Olentangy River Road Columbus, Ohio 43202 United States of America
Phone: +1 614 447 3600 x2517 Email: rhamilton@cas.org URI: http://www.cas.org
Michael S. Ackermann Blue Cross Blue Shield of Michigan P.O. Box 2888 Detroit, Michigan 48231 United States of America
Phone: +1 310 460 4080 Email: mackermann@bcbsm.com URI: http://www.bcbsm.com
📦 where this page comes from
- ietf: rfc8250 2017-09
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms