RFC 8039

Multipath Time Synchronization

📅 published December 2016, from ietf

RFC 8039: Multipath Time Synchronization 🦋

Quick intel first, bestie: here's the title block so you know who made this and what kind of doc it is 💅

  • RFC number: 8039 🎀
  • Title: Multipath Time Synchronization
  • Published by: Internet Engineering Task Force (IETF)
  • Authors: A. Shpiner (Mellanox), R. Tse (Microsemi), C. Schelp (Oracle), and T. Mizrahi (Marvell) 👩‍💻
  • Category: Experimental 🧪
  • ISSN: 2070-1721
  • Date: December 2016 🎄
  • Obsoletes / Updates: the header doesn't list anything this RFC obsoletes or updates 🌸

Abstract ✨

okay so clock sync protocols are literally everywhere in IP networks 🕰️ NTP has been the classic for years, and PTP has been blowing up over the last few years. As time-sensitive apps level up, clock accuracy needs keep getting stricter, so these protocols have to be super accurate 😭 This memo describes a multipath approach to PTP and NTP over IP networks. The protocols can run at the same time over multiple communication paths between the master and slave clocks, and the protocols themselves don't get changed 🙌 The multipath approach can seriously boost clock accuracy, security, and fault tolerance. And it stays backward compatible with nodes that don't support the multipath functionality, which is so considerate 🫶

Status of This Memo 📜

This document is not an Internet Standards Track specification. It's published for examination, experimental implementation, and evaluation 🧪

This document defines an Experimental Protocol for the Internet community. It's a product of the Internet Engineering Task Force (IETF) and represents the consensus of the IETF community. It went through public review and got approved for publication by the Internet Engineering Steering Group (IESG). Not every IESG-approved doc is a candidate for any level of Internet Standard, so see Section 2 of RFC 7841 for the deets 💅

Info about the current status of this doc, any errata, and how to send feedback is at http://www.rfc-editor.org/info/rfc8039 🔗

Copyright (c) 2016 IETF Trust and the persons identified as the document authors. All rights reserved.

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, because they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they're provided without warranty as described in the Simplified BSD License. Read the fine print, babe 🎀

Table of Contents 🗂️

   1. Introduction ....................................................3
   2. Conventions Used in This Document ...............................4
      2.1. Abbreviations ..............................................4
      2.2. Terminology ................................................4
   3. Multiple Paths in IP Networks ...................................5
      3.1. Load Balancing .............................................5
      3.2. Using Multiple Paths Concurrently ..........................5
      3.3. Two-Way Paths ..............................................5
   4. Solution Overview ...............................................6
      4.1. Path Configuration and Identification ......................6
      4.2. Combining ..................................................6
   5. Multipath Time Synchronization over IP Networks .................7
      5.1. Overview ...................................................7
      5.2. Single-Ended Multipath Synchronization .....................8
           5.2.1. Single-Ended MPPTP Synchronization Message
                  Exchange ............................................8
           5.2.2. Single-Ended MPNTP Synchronization Message
                  Exchange ............................................9
      5.3. Dual-Ended Multipath Synchronization ......................10
           5.3.1. Dual-Ended MPPTP Synchronization Message Exchange ..10
           5.3.2. Dual-Ended MPNTP Synchronization Message Exchange ..11
      5.4. Using Traceroute for Path Discovery .......................12
      5.5. Using Unicast Discovery for MPPTP .........................13
   6. Combining Algorithm ............................................13
   7. Security Considerations ........................................14
   8. Scope of the Experiment ........................................14
   9. References .....................................................15
      9.1. Normative References ......................................15
      9.2. Informative References ....................................15
   Acknowledgments ...................................................17
   Authors' Addresses ................................................17

1. Introduction 🌈

The two most common time sync protocols in IP networks are (1) the Network Time Protocol [NTP] and (2) the Precision Time Protocol (PTP), which is defined in the IEEE 1588 standard [IEEE1588] 🕰️

The accuracy of time sync protocols depends directly on how stable and how symmetric the propagation delays are in both directions between the master and slave clocks. Depending on the underlying network, time sync protocol packets can get hit with variable network latency or path asymmetry (e.g., [ASYMMETRY] [ASYMMETRY2]). Time-sensitive apps keep getting pickier, so accuracy requirements are getting more and more stringent 😤 Using a single network path ties the slave clock's accuracy really tightly to how that specific path behaves. That path might suffer from temporary congestion, faults, or malicious attacks 💀 Relying on multiple clock servers, like NTP does, solves these problems, but it needs active maintenance of multiple accurate sources in the network, and that's not always possible 🙃 Transparent Clocks (TCs) in PTP fix the congestion problem by taking the queuing time out of the delay calculations, but they don't address security or fault-tolerance aspects 🫠

                                   ____
                            ______/    \_____
                        ___/                 \____
                   ____/                          \
       ____       /           path 1              /           ___
      /    \     /    ________________________    \          /   \
     /Master\____\___/                        \____\________/Slave\
     \Clock /    /   \________ _______________/     \       \Clock/
      \____/     \                                  /        \__ /
                  \____       path 2             __/
                       \_______           ______/
                               \_________/

                      Figure 1: Multipath Connection

Master and slave clocks are often connected through more than one path in the network, like in Figure 1. So [SLAVEDIV] suggested that a time sync protocol can run over multiple paths, and that brings several perks 🎁 First, it can significantly increase clock accuracy, as shown in [SLAVEDIV]. Second, this approach adds security by helping mitigate man-in-the-middle attacks against the time sync protocol [DELAY-ATT]. Third, using multiple paths concurrently gives you an inherent failure protection mechanism 🛡️

This document introduces Multipath PTP (MPPTP) and Multipath NTP (MPNTP). The functionality of the multipath approach is defined at the network layer, and it doesn't require any changes in PTP or NTP. Main character energy, no notes 👑

MPPTP and MPNTP are defined over IP networks. Since IP networks typically combine ECMP routing, that property gets leveraged for the multiple paths used in MPPTP and MPNTP 🦋 The key property of the multipath approach is that clocks in the network can use more than one IP address. Each {master IP, slave IP} address pair defines a path. Depending on the network topology and configuration, the IP combination pairs can form multiple diverse paths for the multipath sync protocols to use. It has been shown [MULTI] that using multiple IP addresses over the wide Internet really does let two endpoints reach multiple diverse paths 🌐

This document introduces two variants of the multipath approach. The first is (1) a variant that requires both master and slave nodes to support the multipath functionality, and it's called the dual-ended variant. The second is (2) a backward-compatible variant that lets a multipath clock connect to a conventional single-path clock, and it's called the single-ended variant 🎀

2. Conventions Used in This Document 📚

2.1. Abbreviations 🔤

  • BMC: Best Master Clock [IEEE1588] 🏆
  • ECMP: Equal-Cost Multipath 🛣️
  • LAN: Local Area Network 🏠
  • MPNTP: Multipath Network Time Protocol 💜
  • MPPTP: Multipath Precision Time Protocol 💖
  • NTP: Network Time Protocol [NTP] ⏰
  • PTP: Precision Time Protocol [IEEE1588] ⌚

2.2. Terminology 🗣️

In NTP terminology, a time sync protocol runs between a client and a server, while PTP uses the terms 'master' and 'slave'. Throughout this document, the sections that refer to both PTP and NTP generically use the terms 'master' and 'slave'. No drama, just consistency 🫶

3. Multiple Paths in IP Networks 🌸

3.1. Load Balancing ⚖️

Traffic sent across IP networks is often load-balanced across multiple paths. The load-balancing decisions are typically based on packet header fields: source and destination addresses, Layer 4 ports, the Flow Label field in IPv6, etc. 🧚

There are three common load-balancing criteria: per-destination, per-flow, and per-packet. Per-destination load balancers make a load-balancing decision based on the destination IP address. Per-flow load balancers use various fields in the packet header, e.g., IP addresses and Layer 4 ports, for the decision. Per-packet load balancers use flow-blind techniques such as round-robin, and they don't base the choice on the packet content 🎰

3.2. Using Multiple Paths Concurrently 🧵

To use the diverse paths that go through per-destination load balancers or per-flow load balancers, the packet transmitter can vary the IP addresses in the packet header. The analysis in [PARIS2] shows that a significant majority of flows on the Internet go through per-destination or per-flow load balancing. It presents statistics that 72% of flows go through per-destination load balancing and 39% of flows go through per-flow load balancing, while only a negligible part of flows go through per-packet load balancing. So the vast majority of Internet traffic is load-balanced based on packet header fields. Iconic, honestly 💅

The approaches in this document are based on varying the source and destination IP addresses in the packet header. Extensions that also vary the UDP ports have been considered. However, some existing implementations of PTP and NTP use fixed UDP port values in both the source and destination UDP port fields, so they don't allow that approach 🙈

3.3. Two-Way Paths 🦋

okay so here's a big one bestie 👀 packets going from A to B do not have to take the same road back from B to A 🛣️ no cap. So for a master-slave connection we define a two-way path as a pair of one-way paths: one from master to slave, and one from slave to master 🫶

If it's possible, a traffic engineering approach can be used to check that time sync traffic always goes through bidirectional two-way paths. That means each two-way path uses the same route forward and back, which gives us propagation time symmetry 💅 But in the general case, the forward and reverse directions don't have to share a path, so it's not guaranteed. ✨

4. Solution Overview ✨

The multipath time sync protocols we're about to talk about are built from two building blocks 🧱 The first is path configuration and identification. The second is the algorithm the slave uses to combine the info it gets from all those different paths 🔥

4.1. Path Configuration and Identification

The master and slave clocks have to be able to figure out which path their outgoing protocol packets take, and identify the path an incoming packet came in on 🗺️ A path is defined by a {master IP, slave IP} address pair. The synchronization protocol message exchange runs independently through each path 🦋

Each IP address pair defines a two-way path, so the clocks can bind a transmitted packet to a specific path, or identify the path of an incoming packet. Iconic behavior 🙌

If possible, the routing tables across the network should be configured with multiple traffic-engineered paths between the pair of clocks. If you carefully configure the routers in these networks, you can create diverse paths for each IP address pair between two clocks. 🌈 But in public and provider networks, the load-balancing behavior is hidden from end users 😭 In that case, the actual number of paths may be less than the number of IP address pairs, since some of the pairs may share common paths 💀

4.2. Combining

There are lots of ways to combine the time info received from the different paths 🧚 The output of the combining algorithm is the accurate time offset. We'll talk more about combining methods in Section 6 💜

5. Multipath Time Synchronization over IP Networks 🌸

5.1. Overview

This section covers two variants of MPPTP and MPNTP: single-ended multipath time synchronization and dual-ended multipath time synchronization 👑 In the first variant, only the slave implements the multipath approach, and the master has no idea it's happening. In the second variant, all the clocks use multiple paths 🔥

The dual-ended variant gives you more path diversity, because it uses multiple IP addresses at both ends (master and slave). The single-ended variant only uses multiple addresses at the slave 🫶 That's why single-ended can interoperate with existing implementations that don't use multiple paths. The two can totally coexist in the same network, and each slave picks the connection(s) it wants to make with the available masters 💅 A dual-ended slave could switch to single-ended mode if it doesn't see any dual-ended masters available. A single-ended slave could connect to a single IP address of a dual-ended master. Main character energy 🎀

Multipath time sync in both variants requires clocks to use multiple IP addresses. This comes with a trade-off ⚖️ A lot of IP addresses means a lot of diverse paths, which gives you the slave diversity advantages we covered in Section 1. But a lot of IP addresses is also more costly, needs a more redundant network topology, and adds extra management overhead 😩

If possible, the set of IP addresses for each clock should be chosen so that the paths you end up with are as different as possible 🦋 If the load-balancing rules in the network are known, you can pick the IP addresses in a way that enforces path diversity. Even if the load-balancing scheme isn't known, a careful choice of IP addresses can still raise the probability of path diversity ✨ For example, picking multiple addresses with different prefixes is likely to produce higher path diversity, since BGP routers are more likely to route those different prefixes through different routes 🌈

Network Address Translation (NAT) may significantly reduce how well multipath synchronization works in some cases 🙈 For example, if a master uses multiple IP addresses that all get translated to a single IP address, the path diversity can drop dramatically compared to a network without NAT. So path discovery should be used to identify the possible paths between the master and slave. Path discovery is discussed further in Section 5.4 🔍

Using multiple IP addresses or multiple interfaces is a well-established concept, and lots of applications and protocols use it today, e.g., [MPTCP] 💖 Using multiple interfaces brings some challenges and issues, which were presented and discussed in [MIF] 🧐

The descriptions in this section refer to the end-to-end scheme of PTP, but they're similarly applicable to the peer-to-peer scheme 🫧 MPNTP, as described in this document, refers to the NTP client-server mode, although the concepts here could be extended to include the symmetric variant too 🧚

Multipath synchronization by nature requires protocol messages to be sent as unicast 💅 Specifically in PTP, the following messages must be sent as unicast in MPPTP: Sync, Delay_Req, Delay_Resp, PDelay_Req, PDelay_Resp, Follow_Up, and PDelay_Resp_Follow_Up. Note that [IEEE1588] allows these messages to be sent as either multicast or unicast 🤷‍♀️

5.2. Single-Ended Multipath Synchronization

In the single-ended approach, only the slave knows that multiple paths are being used 🙈 The master is totally agnostic to it. This lets you have a hybrid network, where some clocks are multipath clocks and others are conventional one-path clocks 🌸 A single-ended multipath clock shows up to the network as N independent clocks, using N IP addresses and N clockIdentity [IEEE1588] values (in PTP) 💜 So the slave's multiple identities are transparent from the master's point of view. The master treats each identity as a separate slave clock. Iconic 👑

5.2.1. Single-Ended MPPTP Synchronization Message Exchange

Here's how the single-ended MPPTP message exchange goes 📝

  • Each single-ended MPPTP clock has a fixed set of N IP addresses and N corresponding clockIdentity values. Each clock arbitrarily picks one of its IP addresses and clockIdentity values to be its clock primary identity 🎀
  • A single-ended MPPTP port only sends Announce messages from its primary identity, according to the BMC algorithm 📣
  • The BMC algorithm at each clock uses the received Announce messages to determine the master 👀
  • A single-ended MPPTP port that's in the 'slave' state uses unicast negotiation to ask the master to transmit unicast messages to each of the N slave clockIdentity values. The slave port periodically sends N Signaling messages to the master, using each of its N identities. The Signaling message includes the REQUEST_UNICAST_TRANSMISSION TLV [IEEE1588] 🫶
  • The master periodically sends unicast Sync messages from its primary identity, which is identified by the sourcePortIdentity [IEEE1588] and IP address, to each of the slave identities 🧚
  • When the slave gets a Sync message, it identifies the path using the destination IP address. The slave sends a Delay_Req unicast message to the primary identity of the master. The Delay_Req goes out using the slave identity that matches the path the Sync came in on. Note that the rate of Delay_Req messages may be lower than the Sync message rate, so a Sync message isn't necessarily followed by a Delay_Req 🌈
  • The master, responding to a Delay_Req from the slave, sends back a Delay_Resp message using the IP address and sourcePortIdentity from the Delay_Req message 💅
  • When the slave gets the Delay_Resp message, it identifies the path using the destination IP address and the requestingPortIdentity [IEEE1588]. The slave can then compute the corresponding path delay and the offset from the master ✨
  • The slave combines the information from all of the negotiated paths 💖

5.2.2. Single-Ended MPNTP Synchronization Message Exchange

Now the single-ended MPNTP version 🌊

  • A single-ended MPNTP client has N separate identities, i.e., N IP addresses. The assumption is that the server information, including its IP address, is already known to the NTP clients. This is a fair assumption, since the address(es) of the NTP server(s) are usually given to the NTP client through configuration 🙌
  • A single-ended MPNTP client starts NTP with an NTP server N times, once for each of its N identities 🔁
  • NTP is maintained between the server and each of the N client identities 🫶
  • The client sends NTP messages to the master using each of its N identities 📨
  • The server responds to the client's NTP messages using the IP address from the received NTP packet 🎯
  • When the client gets an NTP packet, it uses the IP destination address to identify the path it came through, and it uses the time info accordingly 🧠
  • The client combines the information from all paths 👑

5.3. Dual-Ended Multipath Synchronization

In dual-ended multipath synchronization, each clock has N IP addresses 🦋 Time sync messages get exchanged between some of the combinations of {master IP, slave IP} addresses, which allows multiple paths between the master and slave. Heads up: the actual number of paths between the master and slave may be less than the number of {master IP, slave IP} address pairs you chose 😬

Once the multiple two-way connections are established, a separate synchronization protocol exchange instance runs through each of them 🌸

5.3.1. Dual-Ended MPPTP Synchronization Message Exchange ✨

okay so here's how the dual-ended MPPTP message exchange goes 🫶

  • Every clock has N IP addresses but uses a single clockIdentity 🎀
  • The BMC algorithm at each clock picks the master 👑. The master is identified by its clockIdentity, so other clocks can figure out all the IP addresses it uses.
  • When a clock sends an Announce message, it sends it from each of its IP addresses, with its clockIdentity 📣
  • A dual-ended MPPTP port in the 'slave' state uses unicast negotiation to ask the master to send unicast messages to some or all of its N_s IP addresses 💌. This negotiation happens one pair at a time, between a slave IP address and the matching master IP address that the slave wants to connect to. The slave port keeps sending Signaling messages to the master, using some or all of its N_s IP addresses as the source and aiming at the master's N_m IP addresses. The Signaling message includes the REQUEST_UNICAST_TRANSMISSION TLV [IEEE1588] 🦋

(N_s and N_m are the number of IP addresses of the slave and the master, respectively. 🌸)

  • The master keeps sending unicast Sync messages from each of its IP addresses to the matching slave IP addresses that got a unicast connection negotiated 🧚
  • When the slave gets a Sync message, it figures out its path using the {source IP, destination IP} addresses. Then it sends a Delay_Req unicast message with the source and destination IP addresses swapped from the Sync message 💅. Heads up: Delay_Req messages may come less often than Sync messages, so a Sync message isn't always followed by a Delay_Req 🙈
  • When the master gets a Delay_Req from the slave, it replies with a Delay_Resp message. It uses the sourcePortIdentity from the Delay_Req message and swaps the IP addresses from the Delay_Req ✨
  • When the slave gets the Delay_Resp, it identifies the path using the {source IP, destination IP} address pair. Then it can work out the path delay and the offset from the master 🔥
  • The slave combines the information from all of the negotiated paths 👑

5.3.2. Dual-Ended MPNTP Synchronization Message Exchange 🌈

the MPNTP message exchange goes like this 💜

  • Each NTP clock has a set of N IP addresses. The assumption is that the server info, including its multiple IP addresses, is known to the NTP clients 📇
  • The MPNTP client picks N_svr server IP addresses and N_c client IP addresses, then starts N_svr*N_c instances of the protocol, one for each {server IP, client IP} address pair 🎀. That lets the client combine the information from the N_s*N_c paths 🦋

(N_svr and N_c are the number of IP addresses of the server and the client, respectively, that the client chooses to connect with. 💖)

  • The client sends NTP messages to the master using each of the source-destination address combinations 📨
  • The server replies to the client's NTP messages using the IP address combination from the received NTP packet 🔁
  • Using the {source IP, destination IP} address pair in the received packets, the client identifies the path and does its computations for each path 🧮
  • The client combines the information from all paths 💅

5.4. Using Traceroute for Path Discovery 🗺️

so the approach we've covered so far uses multiple IP addresses on a single clock to make multiple paths. But even though each two-way path is defined by a different {master IP, slave IP} address pair, some of those pairs might go through exactly the same network path, which makes them redundant 😭

Traceroute-based path discovery can be used to filter out only the IP addresses that get diverse paths 🌸. 'Paris traceroute' [PARIS] and 'TraceFlow' [TRACEFLOW] are examples of tools that discover the paths between two points in the network 🔍. Just note that this filtering only works if the Traceroute implementation uses the same IP addresses and UDP ports as the synchronization protocol packets 🙈. Some Traceroute implementations vary the UDP ports, so they may not be effective at spotting and filtering redundant paths in synchronization protocols 💀

Traceroute-based filtering can be done by both master and slave nodes, or it can be limited to running only on slave nodes to cut down the overhead on the master 🧚. For networks that guarantee the timing packets take the same path in both the forward and reverse directions, path discovery should only be performed at the slave 🫶

Network routes change over time, so path discovery and redundant path filtering should be performed periodically ⏰. Two {master IP, slave IP} address pairs that produce two diverse paths might get rerouted onto the same paths, so the set of addresses each clock uses should be reassessed regularly 🦋

5.5. Using Unicast Discovery for MPPTP 📡

like we talked about earlier, MPPTP uses Announce messages and the BMC algorithm to find the master 👑. The unicast discovery option of PTP can be used as an alternative ✨

When unicast discovery is in use, the MPPTP slave ports keep a list of the master's IP addresses 📋. The slave port uses unicast negotiation to ask the master for unicast service, like this:

  • In single-ended MPPTP, the slave uses unicast negotiation from each of its identities to the master's (only) identity 💫
  • In dual-ended MPPTP, the slave uses unicast negotiation from its IP addresses, each to a matching master IP address, to request unicast synchronization messages 🎀

After that, the message exchange carries on as described in Sections 5.2.1 and 5.3.1 💖

The unicast discovery option can be used in networks that don't support multicast, or in networks where the master clocks are known ahead of time 🌈. Most importantly, unicast discovery avoids multicasting Announce messages 🙌

6. Combining Algorithm 🧠

the earlier sections covered how to create the multiple paths and get the time info the slave algorithm needs 🛤️. Once the time info comes in through each path, the slave should use a combining algorithm, which squishes the info from the different paths into a single clock 🫧. People have suggested lots of ways to combine info from different paths or different clocks, e.g., [NTP] [SLAVEDIV] [HIGH-AVAI] [KALMAN] 📚. The choice of combining algorithm is local to the slave and doesn't affect interoperability, so this document doesn't define a specific method for the slave to use 🤷‍♀️. The combining algorithm should be picked carefully based on the system properties, because different combining algorithms bring different advantages ⚖️. For example, some combining algorithms (e.g., [NTP] [DELAY-ATT]) are meant to hold up against security attacks 🛡️, while others (e.g., [KALMAN]) handle random delay variation better 🌊

7. Security Considerations 🔐

the security side of time synchronization protocols is covered in detail in [RFC7384] 🕵️‍♀️. The methods in this document run a time synchronization protocol through redundant paths, which lets us detect and mitigate man-in-the-middle attacks, as described in [DELAY-ATT] 🚨. Specifically, multipath synchronization can mitigate these threats (as per [RFC7384]):

  • Packet manipulation (Section 3.2.1 of [RFC7384]) 🙅‍♀️
  • Packet interception and removal (Section 3.2.5 of [RFC7384]) 🥷
  • Packet delay manipulation (Section 3.2.6 of [RFC7384]) ⏳
⚠️ Heads up, this is important: when you use multiple paths, those paths may partly overlap. That means an attack in a shared segment of those paths is NOT fixed by the redundancy 😱. Also, an on-path attacker might, in some cases, have access to more than one router or might be able to hop from one router to another 💀. So when you use multiple paths, it really matters that the paths are as diverse and as independent as possible, which makes the redundancy scheme tougher against on-path attacks 🛡️

Also, the multipath approach asks the master (or NTP server) to dedicate more resources to each slave (client) than the usual single-path approach does 🥵. So well-known Distributed Denial-of-Service (DDoS) attacks might get amplified when the multipath approach is turned on 📈

8. Scope of the Experiment 🧪

this memo is published as an Experimental RFC 🔬. The point of the experimental period is to let the community analyze and verify the methods defined here 🫶. An experimental evaluation of some of these methods has been published in [MULTI] 📊. It's expected that the experimental period will let the community investigate and verify the methods further 👀. The experiment is expected to last no less than two years from the publication of this document ⏳💖

9. References 📚

9.1. Normative References 🌟

[IEEE1588] IEEE Instrumentation and Measurement Society, "IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems", IEEE Std 1588-2008, DOI 10.1109/IEEESTD.2008.4579760.

[NTP] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch, "Network Time Protocol Version 4: Protocol and Algorithms Specification", RFC 5905, DOI 10.17487/RFC5905, June 2010, <http://www.rfc-editor.org/info/rfc5905>.

9.2. Informative References 📚✨

Okay bestie, these are the receipts 🧾💅 Every paper and RFC the spec leans on, so you can go read the source material yourself if you're curious.

  • [ASYMMETRY] He, Y., Faloutsos, M., Krishnamurthy, S., and B. Huffaker, "On routing asymmetry in the Internet", IEEE GLOBECOM, DOI 10.1109/GLOCOM.2005.1577769, December 2005. This one's about routes going one way and coming back a different way, which is so real 🦋
  • [ASYMMETRY2] Pathak, A., Pucha, H., Zhang, Y., Hu, C., and Z. Mao, "A measurement study of internet delay asymmetry", International Conference on Passive and Active Network Measurement 2008, DOI 10.1007/978-3-540-79232-1_19, April 2008. Same vibe as the one above, but it's all about how the delay goes lopsided 🌸
  • [DELAY-ATT] Mizrahi, T., "A Game Theoretic Analysis of Delay Attacks against Time Synchronization Protocols", IEEE International Symposium on Precision Clock Synchronization for Measurement, Control and Communication (ISPCS), DOI 10.1109/ISPCS.2012.6336612, September 2012. Delay attacks on time sync, analysed with game theory. Villain arc, but make it academic 😈⏱️
  • [HIGH-AVAI] Ferrari, P., Flammini, A., Rinaldi, S., and G. Prytz, "High availability IEEE 1588 nodes over IEEE 802.1 aq Shortest Path Bridging networks", IEEE International Symposium on Precision Clock Synchronization for Measurement, Control and Communication (ISPCS), DOI 10.1109/ISPCS.2013.6644760, September 2013. Keeping IEEE 1588 nodes alive and well over Shortest Path Bridging networks 🫶
  • [KALMAN] Giorgi, G. and C. Narduzzi, "Kalman filtering for multi-path network synchronization", IEEE International Symposium on Precision Clock Synchronization for Measurement, Control and Communication (ISPCS), DOI 10.1109/ISPCS.2014.6948693, September 2014. Kalman filters for syncing clocks across multiple paths, it's giving math-core 🧠✨
  • [MIF] Blanchet, M. and P. Seite, "Multiple Interfaces and Provisioning Domains Problem Statement", RFC 6418, DOI 10.17487/RFC6418, November 2011, <http://www.rfc-editor.org/info/rfc6418>. The problem statement for devices with multiple interfaces and provisioning domains. Very relatable 💜
  • [MPTCP] Ford, A., Raiciu, C., Handley, M., and O. Bonaventure, "TCP Extensions for Multipath Operation with Multiple Addresses", RFC 6824, DOI 10.17487/RFC6824, January 2013, <http://www.rfc-editor.org/info/rfc6824>. This is the MPTCP spec, and it's a whole icon 👑
  • [MULTI] Shpiner, A., Revah, Y., and T. Mizrahi, "Multi-path Time Protocols", IEEE International Symposium on Precision Clock Synchronization for Measurement, Control and Communication (ISPCS), DOI 10.1109/ISPCS.2013.6644754, September 2013. Multi-path time protocols, the original vibe for this whole doc 🌈
  • [PARIS] Augustin, B., Friedman, T., and R. Teixeira, "Measuring Load-balanced Paths in the Internet", 7th ACM SIGCOMM conference on Internet measurement (IMC '07), DOI 10.1145/1298306.1298329, October 2007. How to measure load-balanced paths without getting confused by them 🍓
  • [PARIS2] Augustin, B., Friedman, T., and R. Teixeira, "Measuring Multipath Routing in the Internet", IEEE/ACM Transactions on Networking, 19(3), pp. 830-840, DOI 10.1109/TNET.2010.2096232, June 2011. The journal follow-up to the one above. Sequel energy, and it's better 🔥
  • [RFC7384] Mizrahi, T., "Security Requirements of Time Protocols in Packet Switched Networks", RFC 7384, DOI 10.17487/RFC7384, October 2014, <http://www.rfc-editor.org/info/rfc7384>. Security requirements for time protocols. The security bit, and it's serious 🔐🥺
  • [SLAVEDIV] Mizrahi, T., "Slave Diversity: Using Multiple Paths to Improve the Accuracy of Clock Synchronization Protocols", IEEE International Symposium on Precision Clock Synchronization for Measurement, Control and Communication (ISPCS), DOI 10.1109/ISPCS.2012.6336621, September 2012. Use several paths to make clock sync more accurate. Diversity hire, but for packets 🦋💖
  • [TRACEFLOW] Narasimhan, J., Venkataswami, B., Groves, R., and P. Hoose, "Traceflow", Work in Progress, draft-janapath-intarea-traceflow-00, January 2012. An early draft, still in progress, so treat it as a sneak peek 👀

Acknowledgments 🎀

The authors wanna thank Yoram Revah for his contribution to this work 🫶 They also gratefully acknowledge the useful comments provided by Peter Meyer, Doug Arnold, Joe Abley, Zhen Cao, Watson Ladd, and Mirja Kuehlewind, as well as other comments received from the TICTOC working group participants 💜 Everyone who helped make this happen, you're iconic!

Authors' Addresses 💌

Here's where to find the authors if you have questions, bestie 📬

Alex Shpiner
Mellanox Technologies, Ltd.
Hakidma 26
Ofer Industrial Park
Yokneam, 2069200
Israel

Email: alexshp@mellanox.com


Richard Tse
Microsemi
8555 Baxter Place
Burnaby, BC  V5A 4V7
Canada

Email: Richard.Tse@microsemi.com


Craig Schelp
Oracle

Email: craig.schelp@oracle.com


Tal Mizrahi
Marvell
6 Hamada St.
Yokneam, 2066721
Israel

Email: talmi@marvell.com
📦 where this page comes from