RFC 5905
Network Time Protocol Version 4: Protocol and Algorithms Specification
RFC 5905 at a Glance 🎀
okay bestie, this is RFC 5905, and the title is Network Time Protocol Version 4: Protocol and Algorithms Specification 🕰️✨. It's from the IETF, and it's a Standards Track doc, so it's the real deal, not just vibes. It was published in June 2010 📅.
The authors are D. Mills (University of Delaware), J. Martin, Ed. (ISC, with Martin as the editor), J. Burbank, and W. Kasch. Burbank and Kasch are both at JHU/APL 🧑🔬👩🔬. The ISSN is 2070-1721 🔢.
This doc obsoletes RFC 1305 and RFC 4330, so it's officially replacing those. The older NTPv3 spec is retired and this one takes over 🫶.
Abstract ✨
okay so NTP is literally how computer clocks across the Internet stay in sync, and it's so iconic 🕰️💅. This doc describes NTP version 4 (NTPv4). It's backwards compatible with NTPv3, which is the one described in RFC 1305, plus the earlier versions too 🔄.
NTPv4 comes with a modified protocol header so it can handle the IPv6 address family 🌐🦋. It also upgrades the mitigation and discipline algorithms, which pushes the accuracy down to the tens of microseconds on modern workstations and fast LANs. Absolutely unhinged precision 😭🔥.
There's a dynamic server discovery scheme, so in a lot of cases you don't have to configure specific servers by hand 🧚. It fixes some errors from the NTPv3 design and implementation, and it adds an optional extension mechanism 💖.
Status of This Memo 🧾
This is an Internet Standards Track document 🌈.
This doc is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community, it went through public review, and the Internet Engineering Steering Group (IESG) approved it for publication 🫶. If you want more info on Internet Standards, check out Section 2 of RFC 5741 📚.
For the current status, any errata, and how to send feedback, go to http://www.rfc-editor.org/info/rfc5905 🔗.
Copyright Notice 💜
The copyright is Copyright (c) 2010 IETF Trust and the persons identified as the document authors, and all rights are reserved 📜.
This doc is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents, which you can find at http://trustee.ietf.org/license-info, as they stood on the publication date. Read those carefully, bestie, because they spell out your rights and restrictions for this document 👀. Any Code Components pulled out of this doc have to include the Simplified BSD License text, as described in Section 4.e of the Trust Legal Provisions, and they come with no warranty, same as the Simplified BSD License says 🧾.
This doc might include material from IETF Documents or IETF Contributions that were published or made public before November 10, 2008. Some of the people who own the copyright on that older material might not have let the IETF Trust allow modifications outside the IETF Standards Process 🙈. So unless you get a proper license from those copyright holders, this doc can't be modified outside the IETF Standards Process, and you can't make derivative works outside it either. The only exceptions are formatting it for publication as an RFC or translating it into languages other than English 🌸.
Table of Contents 📚
1. Introduction ....................................................4
1.1. Requirements Notation ......................................5
2. Modes of Operation ..............................................6
3. Protocol Modes ..................................................6
3.1. Dynamic Server Discovery ...................................7
4. Definitions .....................................................8
5. Implementation Model ...........................................10
6. Data Types .....................................................12
7. Data Structures ................................................16
7.1. Structure Conventions .....................................16
7.2. Global Parameters .........................................16
7.3. Packet Header Variables ...................................17
7.4. The Kiss-o'-Death Packet ..................................24
7.5. NTP Extension Field Format ................................25
8. On-Wire Protocol ...............................................26
9. Peer Process ...................................................30
9.1. Peer Process Variables ....................................31
9.2. Peer Process Operations ...................................33
10. Clock Filter Algorithm ........................................37
11. System Process ................................................39
11.1. System Process Variables .................................40
11.2. System Process Operations ................................41
11.2.1. Selection Algorithm ...............................43
11.2.2. Cluster Algorithm .................................44
11.2.3. Combine Algorithm .................................45
11.3. Clock Discipline Algorithm ...............................47
12. Clock-Adjust Process ..........................................51
13. Poll Process ..................................................51
13.1. Poll Process Variables ...................................51
13.2. Poll Process Operations ..................................52
14. Simple Network Time Protocol (SNTP) ...........................54
15. Security Considerations .......................................55
16. IANA Considerations ...........................................58
17. Acknowledgements ..............................................59
18. References ....................................................59
18.1. Normative References .....................................59
18.2. Informative References ...................................59
Appendix A. Code Skeleton .......................................61
A.1. Global Definitions .......................................61
A.1.1. Definitions, Constants, Parameters .....................61
A.1.2. Packet Data Structures .................................65
A.1.3. Association Data Structures ............................66
A.1.4. System Data Structures .................................68
A.1.5. Local Clock Data Structures ............................69
A.1.6. Function Prototypes ....................................69
A.2. Main Program and Utility Routines ..........................70
A.3. Kernel Input/Output Interface ..............................73
A.4. Kernel System Clock Interface ..............................74
A.5. Peer Process ...............................................76
A.5.1. receive() ..............................................77
A.5.2. clock_filter() .........................................85
A.5.3. fast_xmit() ............................................88
A.5.4. access() ...............................................89
A.5.5. System Process .........................................90
A.5.6. Clock Adjust Process ..................................103
A.5.7. Poll Process ..........................................1041. Introduction ✨
okay so this doc defines NTPv4 (Network Time Protocol version 4) 🕰️💖, which is the thing basically everyone uses to keep their system clocks synced across a whole bunch of distributed time servers and clients. It covers the core architecture, the protocol, the state machines, the data structures, and the algorithms. NTPv4 is an upgrade on NTPv3 as described in [RFC1305], and it also grows out of Simple NTP version 4 (SNTPv4) as described in [RFC4330] (SNTPv4 is basically a subset of NTPv4 🧁). This document obsoletes [RFC1305] and [RFC4330], so those are officially retired 🫡. There are some small tweaks to a few protocol header fields, but they don't break interoperability between NTPv4 and the older NTP and SNTP versions, so everyone can still talk to each other 🫶.
Here's the big picture. The NTP subnet model has a bunch of widely accessible primary time servers that are synced by wire or radio to national standards ⏱️. The whole point of NTP is to pass timekeeping info from those primary servers down to secondary time servers and clients, over both private networks and the public Internet 🌐. The algorithms are precisely tuned to handle the errors that can pop up from network disruptions, server failures, and possible hostile actions 😤. Servers and clients are set up so values flow toward the clients from the root primary servers, through branching secondary servers, kind of like a family tree 🌳✨.
NTPv4 is a glow-up 💅. It fixes significant shortcomings in the NTPv3 design, squashes certain bugs, and adds new features. The biggest thing is the expanded NTP timestamp definitions, which encourage using the floating double data type all throughout the implementation 🔥. Because of that, time resolution is better than one nanosecond, and frequency resolution is less than one nanosecond per second 🤯. There's also a new clock discipline algorithm that reacts better to fluctuations in system clock hardware frequency 🎀. Typical primary servers on modern machines are precise within a few tens of microseconds ⚡. Typical secondary servers and clients on fast LANs are within a few hundred microseconds, with poll intervals up to 1024 seconds (that was the max with NTPv3) 🍓. With NTPv4, servers and clients are precise within a few tens of milliseconds, with poll intervals up to 36 hours 😭🌈.
The main body of this document describes the core protocol and data structures you need to interoperate between conforming implementations 🦋. Appendix A is the full-featured example, a skeleton program with data structures and code segments for the core algorithms, plus the mitigation algorithms used to boost reliability and accuracy 🧚. The skeleton program and the other descriptions in this doc apply to one particular implementation, but they are NOT the only way to implement the required functions, so don't panic if yours looks different 💜. Everything in Appendix A is non-normative: it's just examples meant to illustrate how the protocol works, and it's NOT a requirement for a conforming implementation 🙈. The NTPv3 symmetric key authentication scheme described here was carried over from NTPv3, while the Autokey public key authentication scheme, which is new in NTPv4, is described in [RFC5906] 🔐.
The NTP protocol has modes of operation (described in Section 2) that use the data types from Section 6 and the data structures from Section 7 💅. The implementation model in Section 5 is based on a threaded, multi-process architecture, though other architectures could work too 🧵. The on-wire protocol in Section 8 is a returnable-time design. It only depends on measured clock offsets, and it does NOT require reliable message delivery 🌸. Reliable delivery like TCP [RFC0793] can actually make the NTP packet less reliable, since retries would increase the delay value and cause other errors 💀. The synchronization subnet is a self-organizing, hierarchical, master-slave network, with synchronization paths picked by a shortest-path spanning tree and a defined metric 🌲. Multiple masters (primary servers) can exist, but there is no requirement for an election protocol 👑.
This document includes material from [ref9], which has flow charts and equations that just don't fit in RFC format 📊. There is also a ton of extra info in [ref7], including a deep technical analysis and performance assessment of the protocol and algorithms in this document 🤓. The reference implementation is available at www.ntp.org 🫶.
The rest of this document has a lot of variables and math expressions 🧮. Some variables are Greek characters, which are spelled out by their full case-sensitive name. For example, DELTA refers to the uppercase Greek character, while delta refers to the lowercase one 🇬🇷. Subscripts are written with _, so for example theta_i means the lowercase Greek character theta with subscript i (or, phonetically, "theta sub i") 🎓. In this document, all time values are in seconds (s), and all frequencies are given as fractional frequency offsets (FFOs), which are pure numbers. It's often easier to express FFOs in parts per million (ppm) 💖.
1.1. Requirements Notation
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119] 📜✨.
2. Modes of Operation
okay the vibe here is that an NTP implementation operates as a primary server, a secondary server, or a client 🎀. A primary server is synced to a reference clock that's directly traceable to UTC (think GPS, Galileo, that kind of thing) 🛰️. A client syncs to one or more upstream servers, but it does NOT provide synchronization to any dependent clients 🙅♀️. A secondary server has one or more upstream servers AND one or more downstream servers or clients, so it's the middle child 🧃. All servers and clients who are fully NTPv4-compliant MUST implement the entire suite of algorithms described in this document 💅. To keep big NTP subnets stable, secondary servers SHOULD be fully NTPv4-compliant 🌈. Alternative algorithms MAY be used, but their output MUST be identical to the algorithms described in this specification 🔥.
3. Protocol Modes
so there are three NTP protocol variants: symmetric, client/server, and broadcast 🦋. Each one is linked to an association mode (basically a description of the relationship between two NTP speakers), as shown in Figure 1 🧚. Also, persistent associations get mobilized on startup and are NEVER demobilized, they're forever 💜. Ephemeral associations get mobilized when a packet shows up and get demobilized on error or timeout, so they're more of a come-and-go situation 🌸.
+-------------------+-------------------+------------------+
| Association Mode | Assoc. Mode Value | Packet Mode Value|
+-------------------+-------------------+------------------+
| Symmetric Active | 1 | 1 or 2 |
| Symmetric Passive | 2 | 1 |
| Client | 3 | 4 |
| Server | 4 | 3 |
| Broadcast Server | 5 | 5 |
| Broadcast Client | 6 | N/A |
+-------------------+-------------------+------------------+
Figure 1: Association and Packet ModesIn the client/server variant 🤝, a persistent client sends packet mode 4 packets to a server, and the server sends back packet mode 3 packets. Servers provide synchronization to one or more clients, but they do NOT accept synchronization from them (no takebacks 🙅♀️). A server can also be a reference clock driver that gets time straight from a standard source, like a GPS receiver or a telephone modem service 📡. In this variant, clients pull synchronization from servers 🫳.
In the symmetric variant, a peer acts as both a server and a client, using either a symmetric active or a symmetric passive association 🪞. A persistent symmetric active association sends symmetric active (mode 1) packets to a symmetric active peer association. Or, if no matching association exists, an ephemeral symmetric passive association can be mobilized when a symmetric active packet arrives 🌀. That association sends symmetric passive (mode 2) packets and sticks around until error or timeout. Peers both push and pull synchronization to and from each other, which is very much a mutual-aid situation 🫶. For this document, a peer operates like a client, so any reference to client also implies peer 💅.
In the broadcast variant 📢, a persistent broadcast server association sends periodic broadcast server (mode 5) packets that multiple clients can receive. When a broadcast server packet arrives with no matching association, an ephemeral broadcast client (mode 6) association gets mobilized and sticks around until error or timeout 🦋. It's useful to do an initial volley, where the client runs in client mode and swaps several packets with the server. That calibrates the propagation delay and runs the Autokey security protocol, and then the client goes back to broadcast client mode 🔐. A broadcast server pushes synchronization out to clients and other servers 📣.
Loosely following conventions from the telephone industry 📞, the level of each server in the hierarchy is defined by a stratum number. Primary servers are stratum one, and secondary servers at each lower level get a stratum number one greater than the level above them 🪜. As the stratum number goes up, accuracy degrades, depending on the network path and how stable the system clock is 😬. Mean errors (measured by synchronization distances) increase roughly in proportion to the stratum number and the measured round-trip delay 📉.
As standard practice, timing network topology should be organized to avoid timing loops and minimize the synchronization distance 🧠. In NTP, the subnet topology is determined using a variant of the Bellman-Ford distributed routing algorithm, which computes the shortest-path spanning tree rooted on the primary servers 🌳. Because of this design, the algorithm automatically reorganizes the subnet to produce the most accurate and reliable time, even when there are failures in the timing network 👑✨.
3.1. Dynamic Server Discovery ✨
okay so this part is about how a client can find servers all by itself 🧚 there are two special associations, manycast client and manycast server, and together they handle the dynamic server discovery thing 💖 there are two flavors of manycast client association: persistent and ephemeral.
The persistent manycast client sends client (mode 3) packets to a designated IPv4 or IPv6 broadcast or multicast group address 📡 designated manycast servers that are within range of the time-to-live (TTL) field in the packet header listen for packets with that address 👂 if a server is suitable for synchronization, it sends back an ordinary server (mode 4) packet using the client's unicast address 🫶 once the client gets that packet, it mobilizes an ephemeral client (mode 3) association 🦋 and that ephemeral client association persists until error or timeout, periodt 💅
Now the search part. A manycast client keeps sending packets to look for a minimum number of associations 🔍 it starts with a TTL equal to one and keeps adding one to it until the minimum number of associations is made, or until the TTL hits its maximum value 📈 if the TTL reaches its max and still not enough associations got mobilized, the client stops transmitting for a time-out period to clear all associations, and then it starts the whole search cycle over again 😭 but if a minimum number of associations has been mobilized, the client switches to sending one packet per time-out period to maintain them 🌸 fun fact (well, rule fact): field constraints limit the minimum value to 1 and the maximum to 255 🔢 these limits may be tuned for individual application needs 🎀
The ephemeral associations are lowkey competitive 👑 as new ephemeral associations get mobilized, the client runs the mitigation algorithms described in Sections 10 and 11.2 on the best candidates out of the population, and the remaining ephemeral associations get timed out and demobilized ✂️ so in the end, the population only includes the best candidates, the ones that most recently responded with an NTP packet to discipline the system clock ⭐ iconic behavior, honestly 🙌
4. Definitions ✨
okay, this section is where all the technical terms get defined, so pay attention bestie 🤓 a timescale is a frame of reference where time is expressed as the value of a monotonically increasing binary counter with an indefinite number of bits 🧮 it counts in seconds and fractions of a second, when a decimal point is employed 🍓 the Coordinated Universal Time (UTC) timescale is defined by ITU-R TF.460 [ITU-R_TF.460] 🌍 and under the auspices of the Metre Convention of 1865, in 1975 the CGPM [CGPM] strongly endorsed the use of UTC as the basis for civil time 🗓️
the UTC timescale represents mean solar time as disseminated by national standards laboratories ☀️ the system time is represented by the system clock maintained by the hardware and operating system 🖥️ the goal of the NTP algorithms is to minimize both the time difference and the frequency difference between UTC and the system clock 🎯 when those differences have been reduced below nominal tolerances, the system clock is said to be synchronized to UTC 🫶
the date of an event is the UTC time at which the event takes place 📅 dates are ephemeral values designated with uppercase T 🩷 and running time is another timescale that is coincident to the synchronization function of the NTP program 🏃♀️
a timestamp T(t) represents either the UTC date or the time offset from UTC at running time t 🕰️ which meaning is intended should be clear from the context 💜 now let T(t) be the time offset, R(t) the frequency offset, and D(t) the aging rate (the first derivative of R(t) with respect to t) 🧠 then, if T(t_0) is the UTC time offset determined at t = t_0, the UTC time offset at time t is:
T(t) = T(t_0) + R(t_0)(t-t_0) + 1/2 * D(t_0)(t-t_0)^2 + e,where e is a stochastic error term discussed later in this document 🎲 the D(t) term is important when characterizing precision oscillators, but it's ordinarily neglected for computer oscillators 🖨️ in this document, all time values are in seconds (s) and all frequency values are in seconds-per-second (s/s) ⏱️ it's sometimes convenient to express frequency offsets in parts-per-million (ppm), where 1 ppm is equal to 10^(-6) s/s 🧁
it's important in computer timekeeping applications to assess how the timekeeping function is performing 📊 the NTP performance model includes four statistics that get updated each time a client makes a measurement with a server 🔄 the offset (theta) is the maximum-likelihood time offset of the server clock relative to the system clock 🎀 the delay (delta) is the round-trip delay between the client and server 🏓 the dispersion (epsilon) is the maximum error inherent in the measurement, and it increases at a rate equal to the maximum disciplined system clock frequency tolerance (PHI), typically 15 ppm 📈 and the jitter (psi) is defined as the root-mean-square (RMS) average of the most recent offset differences, representing the nominal error in estimating the offset 🌊
those theta, delta, epsilon, and psi statistics measure the system clock relative to each server clock separately 👯 but the NTP protocol also has mechanisms to combine the statistics of several servers so the system clock can be disciplined and calibrated more accurately 💪 the system offset (THETA) is the maximum-likelihood offset estimate for the server population 💖 the system jitter (PSI) is the nominal error in estimating the system offset 🔮 the delta and epsilon statistics get accumulated at each stratum level from the reference clock to produce the root delay (DELTA) and root dispersion (EPSILON) statistics 🪜 and the synchronization distance (LAMBDA), equal to EPSILON + DELTA / 2, represents the maximum error due to all causes 🌈 the detailed formulations of these statistics are given in Section 11.2 📖 they're available to the dependent applications so they can assess the performance of the synchronization function 🫧 literally the stats queen behavior 👑✨
5. Implementation Model ✨
okay so Figure 2 is the architecture of a typical multi-threaded implementation 🧵 and bestie, it's giving two processes dedicated to each server 💅: a peer process that receives messages from the server or reference clock, and a poll process that transmits messages to the server or reference clock 🫶
.....................................................................
. Remote . Peer/Poll . System . Clock .
. Servers . Processes . Process .Discipline.
. . . . Process .
.+--------+. +-----------+. +------------+ . .
.| |->| |. | | . .
.|Server 1| |Peer/Poll 1|->| | . .
.| |<-| |. | | . .
.+--------+. +-----------+. | | . .
. . ^ . | | . .
. . | . | | . .
.+--------+. +-----------+. | | +-----------+. .
.| |->| |. | Selection |->| |. +------+ .
.|Server 2| |Peer/Poll 2|->| and | | Combine |->| Loop | .
.| |<-| |. | Cluster | | Algorithm |. |Filter| .
.+--------+. +-----------+. | Algorithms |->| |. +------+ .
. . ^ . | | +-----------+. | .
. . | . | | . | .
.+--------+. +-----------+. | | . | .
.| |->| |. | | . | .
.|Server 3| |Peer/Poll 3|->| | . | .
.| |<-| |. | | . | .
.+--------+. +-----------+. +------------+ . | .
....................^.........................................|......
| . V .
| . +-----+ .
+--------------------------------------| VFO | .
. +-----+ .
. Clock .
. Adjust .
. Process .
............Figure 2: Implementation Model 🎀
All of these processes work on one shared data structure called an association 🌸. It holds the statistics we just covered, plus a bunch of other stuff that's explained in Section 9. A client sends packets to one or more servers, then processes the returned packets when they arrive. The server swaps the source and destination addresses and ports, overwrites certain fields in the packet, and sends it back right away (in the client/server mode) or sometime later (in the symmetric modes) ⏰. Every time an NTP message comes in, we compute the offset theta between the peer clock and the system clock, along with the related statistics delta, epsilon, and psi 🧚.
The system process holds the selection, cluster, and combine algorithms, and they work together to mitigate among the various servers and reference clocks 👑. Their goal is to find the most accurate and reliable candidates to sync the system clock. The selection algorithm uses Byzantine fault detection principles to toss out the presumably incorrect candidates, called "falsetickers" from the incident population 💀. What's left are the good candidates, called "truechimers" ✨. A truechimer is a clock that keeps its timekeeping accurate to a previously published and trusted standard. A falseticker is a clock that shows misleading or inconsistent time. Next up, the cluster algorithm uses statistical principles to find the most accurate set of truechimers 📊. Finally, the combine algorithm works out the final clock offset by statistically averaging the surviving truechimers 🍓.
The clock discipline process is a system process that controls the time and frequency of the system clock 🦋. Here the system clock is represented as a variable frequency oscillator (VFO). Timestamps struck from the VFO close the feedback loop that keeps the system clock time on track 🔁. The clock discipline process also has a buddy, the clock-adjust process, which runs once each second to inject a computed time offset and maintain constant frequency ⚡. The RMS average of past time offset differences represents the nominal error or system clock jitter. The RMS average of past frequency offset differences represents the oscillator frequency stability or frequency wander. These terms get their precise meaning in Section 11.3 🙈.
A client sends messages to each server with a poll interval of 2^tau seconds, based on the poll exponent tau 🌈. In NTPv4, tau ranges from 4 (16 s) to 17 (36 h). The clock discipline algorithm decides the value of tau so it matches the loop-time constant T_c = 2^tau 💜. In client/server mode, the server responds immediately. In symmetric modes, though, each of the two peers manages tau as a function of the current system offset and system jitter, so they might not agree on the same value 😭. The dynamic behavior of the clock discipline algorithm has to be carefully controlled to keep the NTP subnet at large stable 🛡️. So the peers need to agree on a common tau equal to the minimum poll exponent of both peers 🫶. The NTP protocol has provisions to negotiate this value properly 🤝.
The implementation model includes some means to set and adjust the system clock ⏱️. The operating system is assumed to provide two functions. One sets the time directly, for example the Unix settimeofday() function. The other adjusts the time in small increments, advancing or retarding it by a designated amount, for example the Unix adjtime() function 🎀. In this and the following references, parentheses after a name mean we're talking about a function, not a simple variable 🧠. In the intended design, the clock discipline process uses the adjtime() function if the adjustment is less than a designated threshold, and the settimeofday() function if it's above the threshold. The manner in which this is done, and the value of the threshold, are described in Section 10 🦋.
6. Data Types ✨
okay so bestie, NTP time values are all twos-complement 🫶 and bits are numbered big-endian, starting at zero on the left (the high-order side) 🎀. The bit-numbering stuff is from Appendix A of [RFC0791], so go check that if you wanna see where it comes from 🧚. There are three NTP time formats: a 128-bit date format, a 64-bit timestamp format, and a 32-bit short format, all shown in Figure 3 💅.
The 128-bit date format is the main character 👑 and you only use it when you've got enough storage and word size. Its 64-bit signed seconds field spans 584 billion years (literally a whole era of vibes 😭), and its 64-bit fraction field resolves .05 attosecond (that's 0.5e-18). To make it easier to map between formats, the seconds field gets split into a 32-bit Era Number field and a 32-bit Era Offset field 🌈. NTP can't produce eras by itself, and it doesn't really need to. When you do need them, you can grab them from outside sources like the filesystem or dedicated hardware 🔥.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Seconds | Fraction |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
NTP Short Format
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Seconds |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Fraction |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
NTP Timestamp Format
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Era Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Era Offset |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
| Fraction |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
NTP Date Format
Figure 3: NTP Time FormatsThe 64-bit timestamp format shows up in packet headers and anywhere else the word size is tight 🍓. It has a 32-bit unsigned seconds field that spans 136 years, plus a 32-bit fraction field that resolves 232 picoseconds 🦋. The 32-bit short format is for delay and dispersion header fields, where you don't need the full resolution and range of the others 🌸. It's just a 16-bit unsigned seconds field and a 16-bit fraction field 💜.
In the date and timestamp formats, the prime epoch (aka the base date of era 0) is 0 h 1 January 1900 UTC, which is the moment when all bits are zero ⭐. Okay, real talk: strictly speaking, UTC didn't exist before 1 January 1972 🙈. But it's convenient to pretend it's been around forever, even if we've totally lost track of the historic leap seconds 💀. Dates are measured relative to the prime epoch. Values above zero mean times after that date, and values below zero mean times before it 🌈. Also, the Era Offset field of the date format and the Seconds field of the timestamp format mean the same thing 🫶.
Timestamps are unsigned, and any operations on them give you a result in the same era or the adjacent one 🔥. Era 0 covers everything from the prime epoch until sometime in 2036, when the timestamp field wraps around and the base date for era 1 gets set 😭. In either format, a value of zero is a special case that means unknown or unsynchronized time ✨. Figure 4 has a bunch of historic NTP dates along with their Modified Julian Day (MJD), NTP era, and NTP timestamp, so you can see how it all lines up 🧚.
+-------------+------------+-----+---------------+------------------+
| Date | MJD | NTP | NTP Timestamp | Epoch |
| | | Era | Era Offset | |
+-------------+------------+-----+---------------+------------------+
| 1 Jan -4712 | -2,400,001 | -49 | 1,795,583,104 | 1st day Julian |
| 1 Jan -1 | -679,306 | -14 | 139,775,744 | 2 BCE |
| 1 Jan 0 | -678,491 | -14 | 171,311,744 | 1 BCE |
| 1 Jan 1 | -678,575 | -14 | 202,939,144 | 1 CE |
| 4 Oct 1582 | -100,851 | -3 | 2,873,647,488 | Last day Julian |
| 15 Oct 1582 | -100,840 | -3 | 2,874,597,888 | First day |
| | | | | Gregorian |
| 31 Dec 1899 | 15019 | -1 | 4,294,880,896 | Last day NTP Era |
| | | | | -1 |
| 1 Jan 1900 | 15020 | 0 | 0 | First day NTP |
| | | | | Era 0 |
| 1 Jan 1970 | 40,587 | 0 | 2,208,988,800 | First day UNIX |
| 1 Jan 1972 | 41,317 | 0 | 2,272,060,800 | First day UTC |
| 31 Dec 1999 | 51,543 | 0 | 3,155,587,200 | Last day 20th |
| | | | | Century |
| 8 Feb 2036 | 64,731 | 1 | 63,104 | First day NTP |
| | | | | Era 1 |
+-------------+------------+-----+---------------+------------------+
Figure 4: Interesting Historic NTP DatesNow the resolution and precision bit 🎀. Let p be the number of significant bits in the second fraction. Then the clock resolution is 2^(-p), in seconds 💖. To keep bias low and make timestamps harder for an intruder to predict, the non-significant bits should be set to an unbiased random bit string 🔐. The clock precision is the running time it takes to read the system clock, in seconds. Heads up, the precision you get this way can be bigger or smaller than the resolution 🙈. The term rho, which is the precision used in the protocol, is whichever of the two is larger ✨.
The only arithmetic you're allowed to do on dates and timestamps is twos-complement subtraction, and it gives you a 127-bit or 63-bit signed result 🧮. It's super important that the first-order differences between two dates keep the full 128-bit precision, and the first-order differences between two timestamps keep the full 64-bit precision 🚨. The differences are usually small compared to the seconds span, so you can convert them to floating double format for more processing without losing precision 💅.
Something iconic to know: twos-complement arithmetic doesn't tell signed and unsigned values apart (though comparisons can take the sign into account). Only the conditional branch instructions care about that 🤓. So even though the document separates signed dates from unsigned timestamps, they get processed the same way 🫶. People worry that 64-bit timestamp math spanning an era, like what could happen in 2036, might overrun 😰. But honestly? If the client is set within 68 years of the server before the protocol starts, you still get correct values even when the client and server are in adjacent eras 🎉.
Some time values use exponent format, like the precision, time constant, and poll interval 🌸. These are 8-bit signed integers in log2 (log base 2) seconds 🧠. The only arithmetic you can do on them is increment and decrement ➕➖. To keep things simple, when this document says a variable by name, it means the exponentiated value. So the poll interval is 1024 s. But if you reference it by name and exponent, you get the actual value, so the poll exponent is 10 🦋.
To convert system time in any format into NTP date and timestamp formats, you need the number of seconds s from the prime epoch to the system time. To find the integer era and timestamp given s:
era = s / 2^(32) and timestamp = s - era * 2^(32),and this works for both positive and negative dates 💫. To find s given the era and timestamp:
s = era * 2^(32) + timestamp.Converting between NTP and system time can get a little messy, and it's out of scope for this document 😅. Also, era 0 has one more day than most other eras. That won't happen again until the year 2400, in era 3 📅.
In the state variable descriptions coming up, an explicit reference to an integer type means a 32-bit unsigned integer 🔢. That makes bounds checks easier, since you only need to define the upper limit 🧚. If there's no explicit type, the default is a 64-bit floating double. Any exceptions will get called out when they come up 💜.
7. Data Structures 💖
the NTP state machines are defined in the sections that follow 🦋. The state variables are grouped into classes by what they do: packet headers, peer and poll processes, the system process, and the clock discipline process 🌈. Packet variables hold the NTP header values in transmitted and received packets 📦. Peer and poll variables hold the contents of the association for each server, separately 🤝. System variables show the state of the server as its dependent clients see it 👑. Clock discipline variables cover the internal workings of the clock discipline algorithm ⚙️. There's an example of all this in Appendix A 🎀.
7.1. Structure Conventions 🎀
To tell apart variables that have the same name but get used in different processes, the naming convention in Figure 5 is used 💅. A receive packet variable v is a member of the packet structure r, so its fully qualified name is r.v. Same idea for the rest: x.v is a transmit packet variable, p.v is a peer variable, s.v is a system variable, and c.v is a clock discipline variable ✨. Each association has its own set of peer variables. There's only one set of system variables and one set of clock variables 🫶.
+------+---------------------------------+
| Name | Description |
+------+---------------------------------+
| r. | receive packet header variable |
| x. | transmit packet header variable |
| p. | peer/poll variable |
| s. | system variable |
| c. | clock discipline variable |
+------+---------------------------------+
Figure 5: Prefix Conventions7.2. Global Parameters ✨
Okay so, on top of the variable classes, this doc also defines a bunch of global parameters 🎀. The ones with values are in Figure 6:
+-----------+-------+----------------------------------+
| Name | Value | Description |
+-----------+-------+----------------------------------+
| PORT | 123 | NTP port number |
| VERSION | 4 | NTP version number |
| TOLERANCE | 15e-6 | frequency tolerance PHI (s/s) |
| MINPOLL | 4 | minimum poll exponent (16 s) |
| MAXPOLL | 17 | maximum poll exponent (36 h) |
| MAXDISP | 16 | maximum dispersion (16 s) |
| MINDISP | .005 | minimum dispersion increment (s) |
| MAXDIST | 1 | distance threshold (1 s) |
| MAXSTRAT | 16 | maximum stratum number |
+-----------+-------+----------------------------------+Figure 6: Global Parameters
These are the only global parameters you need for interop 🫶, but any real implementation is gonna want a bigger collection 💅. Appendix A.1.1 has the ones used by the skeleton for the mitigation algorithms, the clock discipline algorithm, and related implementation-dependent functions 🧚. Some of these are literally cast in stone, like the NTP port number the IANA assigned and the version number for NTPv4 itself 🔥. Others, like the frequency tolerance (also called PHI), are based on an assumption about the worst-case behavior of a system clock once it's synced and then allowed to drift after its sources become unreachable 👀. And the minimum and maximum parameters are the limits on state variables, as described in later sections 🌸.
While they're shown with fixed values here, some implementations may make them adjustable by configuration commands 💜. For instance, the reference implementation works out PRECISION as log2 of the minimum time it takes, over several iterations, to read the system clock ✨.
7.3. Packet Header Variables
The most important state variables from the outside are the packet header variables in Figure 7 and below 🦋. The NTP packet header is made of a whole number of 32-bit (4 octet) words in network byte order 🍓. The packet has three parts: the header itself, one or more optional extension fields, and an optional message authentication code (MAC) 🫶. The header is identical to the NTPv3 header and earlier versions 🌈. The optional extension fields are used by the Autokey public key cryptographic algorithms described in [RFC5906] 🔐. The optional MAC is used by both Autokey and the symmetric key cryptographic algorithm described in this RFC 💖.
+-----------+------------+-----------------------+
| Name | Formula | Description |
+-----------+------------+-----------------------+
| leap | leap | leap indicator (LI) |
| version | version | version number (VN) |
| mode | mode | mode |
| stratum | stratum | stratum |
| poll | poll | poll exponent |
| precision | rho | precision exponent |
| rootdelay | delta_r | root delay |
| rootdisp | epsilon_r | root dispersion |
| refid | refid | reference ID |
| reftime | reftime | reference timestamp |
| org | T1 | origin timestamp |
| rec | T2 | receive timestamp |
| xmt | T3 | transmit timestamp |
| dst | T4 | destination timestamp |
| keyid | keyid | key ID |
| dgst | dgst | message digest |
+-----------+------------+-----------------------+Figure 7: Packet Header Variables
The NTP packet is a UDP datagram [RFC0768] 🦋. Some fields span multiple words, and others are packed into smaller fields inside a single word 🧩. The NTP packet header in Figure 8 has 12 words, then optional extension fields, and finally an optional message authentication code (MAC) made of the Key Identifier field and the Message Digest field 💅.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|LI | VN |Mode | Stratum | Poll | Precision |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Root Delay |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Root Dispersion |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Reference ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
+ Reference Timestamp (64) +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
+ Origin Timestamp (64) +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
+ Receive Timestamp (64) +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
+ Transmit Timestamp (64) +
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
. .
. Extension Field 1 (variable) .
. .
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
. .
. Extension Field 2 (variable) .
. .
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Key Identifier |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| |
| dgst (128) |
| |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+Figure 8: Packet Header Format
The extension fields add optional capabilities, like the Autokey security protocol [RFC5906] 🔑. They're laid out so the packet can be parsed without knowing what the extension field functions do 🧠. The MAC is used by both Autokey and the symmetric key authentication scheme 🫧.
Here's the list of packet header variables, shown in Figure 7 and described in detail below 📝. Except for a minor variation when using the IPv6 address family, these fields are backwards compatible with NTPv3 🙌. The packet header fields apply to both transmitted packets (x prefix) and received packets (r prefix) 🦋. In Figure 8, the size of some multiple-word fields is shown in bits if it isn't the default 32 bits 🔢. The basic header runs from the start of the packet to the end of the Transmit Timestamp field 💜.
The fields and their packet variables (in parentheses) work like this 🎀:
LI Leap Indicator (leap): 2-bit integer warning of an impending leap second to be inserted or deleted in the last minute of the current month, with values defined in Figure 9 ⏰.
+-------+----------------------------------------+
| Value | Meaning |
+-------+----------------------------------------+
| 0 | no warning |
| 1 | last minute of the day has 61 seconds |
| 2 | last minute of the day has 59 seconds |
| 3 | unknown (clock unsynchronized) |
+-------+----------------------------------------+Figure 9: Leap Indicator
VN Version Number (version): 3-bit integer representing the NTP version number, currently 4 🌸.
Mode (mode): 3-bit integer representing the mode, with values defined in Figure 10 🦋.
+-------+--------------------------+
| Value | Meaning |
+-------+--------------------------+
| 0 | reserved |
| 1 | symmetric active |
| 2 | symmetric passive |
| 3 | client |
| 4 | server |
| 5 | broadcast |
| 6 | NTP control message |
| 7 | reserved for private use |
+-------+--------------------------+Figure 10: Association Modes
Stratum (stratum): 8-bit integer representing the stratum, with values defined in Figure 11 🏆.
+--------+-----------------------------------------------------+
| Value | Meaning |
+--------+-----------------------------------------------------+
| 0 | unspecified or invalid |
| 1 | primary server (e.g., equipped with a GPS receiver) |
| 2-15 | secondary server (via NTP) |
| 16 | unsynchronized |
| 17-255 | reserved |
+--------+-----------------------------------------------------+Figure 11: Packet Stratum
It is customary to map the stratum value 0 in received packets to MAXSTRAT (16) in the peer variable p.stratum, and to map p.stratum values of MAXSTRAT or greater to 0 in transmitted packets 🪄. This lets reference clocks, which normally sit at stratum 0, be conveniently mitigated using the same clock selection algorithms used for external sources (see Appendix A.5.5.1 for an example) 💖.
Okay bestie, the Poll field 🔁 is an 8-bit signed integer for the longest gap allowed between successive messages, measured in log2 seconds. The suggested default limits for minimum and maximum poll intervals are 6 and 10, respectively 💖
Precision ⏱️ is also an 8-bit signed integer, and it tells you how precise the system clock is, in log2 seconds. A value of -18 means about one microsecond ✨ You can figure out the precision when the service first starts up by taking the minimum time from several rounds of reading the system clock.
Root Delay (rootdelay) 🐢 is the total round-trip delay to the reference clock, written in NTP short format.
Root Dispersion (rootdisp) 🌫️ is the total dispersion to the reference clock, also in NTP short format.
Reference ID (refid) 🆔 is a 32-bit code that identifies the particular server or reference clock. What it means depends on the stratum field 🧠 For stratum 0 (unspecified or invalid), it's a four-character ASCII [RFC1345] string called the "kiss code," used for debugging and monitoring 🔍 For stratum 1 (reference clock), it's a four-octet, left-justified, zero-padded ASCII string assigned to that clock. IANA keeps the authoritative list of Reference Identifiers, but any string starting with the ASCII character "X" is reserved for unregistered experimentation and development 🧪 Here are the identifiers from Figure 12 that have been used as ASCII IDs 👑
+------+----------------------------------------------------------+
| ID | Clock Source |
+------+----------------------------------------------------------+
| GOES | Geosynchronous Orbit Environment Satellite |
| GPS | Global Position System |
| GAL | Galileo Positioning System |
| PPS | Generic pulse-per-second |
| IRIG | Inter-Range Instrumentation Group |
| WWVB | LF Radio WWVB Ft. Collins, CO 60 kHz |
| DCF | LF Radio DCF77 Mainflingen, DE 77.5 kHz |
| HBG | LF Radio HBG Prangins, HB 75 kHz |
| MSF | LF Radio MSF Anthorn, UK 60 kHz |
| JJY | LF Radio JJY Fukushima, JP 40 kHz, Saga, JP 60 kHz |
| LORC | MF Radio LORAN C station, 100 kHz |
| TDF | MF Radio Allouis, FR 162 kHz |
| CHU | HF Radio CHU Ottawa, Ontario |
| WWV | HF Radio WWV Ft. Collins, CO |
| WWVH | HF Radio WWVH Kauai, HI |
| NIST | NIST telephone modem |
| ACTS | NIST telephone modem |
| USNO | USNO telephone modem |
| PTB | European telephone modem |
+------+----------------------------------------------------------+
Figure 12: Reference IdentifiersAbove stratum 1 (secondary servers and clients) 🛰️➡️💻 the Reference ID is the server's own identifier, and it's how you catch timing loops 🔁 If you're on the IPv4 address family, it's the four-octet IPv4 address. If you're on IPv6, it's the first four octets of the MD5 hash of the IPv6 address. Lowkey tricky: when an NTPv4 server is talking to an NTPv3 client over IPv6, the Reference Identifier field looks like a random value, so a timing loop might not get detected 🙈
Reference Timestamp 🕰️ is when the system clock was last set or corrected, written in NTP timestamp format.
Origin Timestamp (org) 📤 is the time at the client when the request left for the server, also in NTP timestamp format.
Receive Timestamp (rec) 📥 is the time at the server when the request arrived from the client, in NTP timestamp format.
Transmit Timestamp (xmt) 📨 is the time at the server when the response left for the client, in NTP timestamp format.
Destination Timestamp (dst) 🎯 is the time at the client when the reply arrived from the server, in NTP timestamp format.
Note: the Destination Timestamp isn't an actual header field 🙅♀️ It gets determined when the packet arrives and is made available in the packet buffer data structure.
If NTP has access to the physical layer, the timestamps are tied to the beginning of the symbol after the start of frame ⚡ If it doesn't, implementations should try to tie the timestamp to the earliest accessible point in the frame 🧩
The MAC is made of the Key Identifier followed by the Message Digest 🔐 The message digest (aka the cryptosum) gets calculated as in [RFC1321] over all the NTP-header and optional extension fields, but not the MAC itself. Also, Extension Field n just points you to Section 7.5 for its format 🎀
Key Identifier (keyid) 🗝️ is a 32-bit unsigned integer that the client and server use to designate a secret 128-bit MD5 key.
Message Digest (digest) 🧾 is a 128-bit MD5 hash computed over the key, followed by the NTP packet header and extension fields (but not the Key Identifier or Message Digest fields).
It should be noted that the MAC computation used here is different from the ones in [RFC1305] and [RFC4330], but it matches how existing implementations already generate a MAC 💅
7.4. The Kiss-o'-Death Packet 💀
If the Stratum field is 0, which means unspecified or invalid, the Reference Identifier field can carry messages for status reporting and access control 📣 These are called Kiss-o'-Death (KoD) packets, and the ASCII messages they carry are called kiss codes. The KoD packets got their name because an early use was telling clients to stop sending packets that broke server access controls 🚫 Kiss codes can give useful info to an intelligent client, whether it's NTPv4 or SNTPv4 🧠 Each kiss code is a four-character ASCII string that's left justified and zero filled 🧸 The strings are designed for character displays and log files 📋 Figure 13 has the list of currently defined kiss codes.
Okay, the big rule: recipients of kiss codes MUST inspect them, and in these cases they take these actions 👀
a. For kiss codes DENY and RSTR, the client MUST demobilize any associations to that server and stop sending packets to that server 🛑
b. For kiss code RATE, the client MUST immediately reduce its polling interval to that server, and keep reducing it each time it receives a RATE kiss code 🐌
c. Kiss codes beginning with the ASCII character "X" are for unregistered experimentation and development, and they MUST be ignored if not recognized 🧪
d. Other than the above conditions, KoD packets have no protocol significance and are discarded after inspection 🗑️
+------+------------------------------------------------------------+
| Code | Meaning |
+------+------------------------------------------------------------+
| ACST | The association belongs to a unicast server. |
| AUTH | Server authentication failed. |
| AUTO | Autokey sequence failed. |
| BCST | The association belongs to a broadcast server. |
| CRYP | Cryptographic authentication or identification failed. |
| DENY | Access denied by remote server. |
| DROP | Lost peer in symmetric mode. |
| RSTR | Access denied due to local policy. |
| INIT | The association has not yet synchronized for the first |
| | time. |
| MCST | The association belongs to a dynamically discovered server.|
| NKEY | No key found. Either the key was never installed or is |
| | not trusted. |
| RATE | Rate exceeded. The server has temporarily denied access |
| | because the client exceeded the rate threshold. |
| RMOT | Alteration of association from a remote host running |
| | ntpdc. |
| STEP | A step change in system time has occurred, but the |
| | association has not yet resynchronized. |
+------+------------------------------------------------------------+
Figure 13: Kiss CodesHeads up, this one's a bit of a rule 🚨 The Receive Timestamp and the Transmit Timestamp (the one set by the server) are undefined in a KoD packet. They MUST NOT be relied upon to have valid values, and they MUST be discarded 🙅♀️
7.5. NTP Extension Field Format 🧩
In NTPv4, one or more extension fields can get slotted in after the header and before the MAC 🎀 The MAC is always present whenever an extension field is. Apart from defining the field format, this document doesn't use the field contents at all 🤷♀️ Each extension field carries a request or response message in the format shown in Figure 14.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Field Type | Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
. .
. Value .
. .
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Padding (as needed) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 14: Extension Field FormatAll extension fields are zero-padded out to a word boundary (that's four octets) 🪶 The Field Type field depends on the specific function it's for, and it isn't explained further here. The minimum field length that holds the required fields is four words (16 octets), but a maximum field length is still to be established 📏
The Length field is a 16-bit unsigned integer. It tells you the length of the entire extension field in octets, and that includes the Padding field 💜
8. On-Wire Protocol ✨
okay so the heart of the NTP on-wire protocol is the core mechanism that swaps time values between servers, peers, and clients 🕰️ and honestly? it's giving resilience. it's inherently resistant to lost or duplicate packets, bestie 💖 data integrity comes from the IP and UDP checksums, and no flow control or retransmission facilities are provided or necessary. the protocol uses timestamps, which are either pulled out of packet headers or struck from the system clock when a packet arrives or leaves 🎀 timestamps are precision data, so they should be restruck in the case of link-level retransmission and corrected for the time to compute a MAC in transmit. the way they're so careful about this 😭
NTP messages use two communication modes: one-to-one and one-to-many, which people usually call unicast and broadcast. for this doc, "broadcast" just means any available one-to-many mechanism 🦋 for IPv4 that's either IPv4 broadcast or IPv4 multicast, and for IPv6 it's IPv6 multicast. IANA allocated the IPv4 multicast address 224.0.1.1 and the IPv6 multicast address ending :101, with the prefix decided by scoping rules 🌈 any other non-allocated multicast address may also be used alongside those allocated ones.
the on-wire protocol uses four timestamps, t1 through t4, plus three state variables: org, rec, and xmt. see Figure 15 🧚 this is the most general case, where each of two peers, A and B, independently measures the offset and delay relative to the other. the packet timestamps are in lowercase and the state variables are in uppercase, so you can tell them apart 🍓 the state variables get copied from the packet timestamps when a packet arrives or leaves.
t2 t3 t6 t7
+---------+ +---------+ +---------+ +---------+
| 0 | | t1 | | t3 | | t5 |
+---------+ +---------+ +---------+ +---------+
| 0 | | t2 | | t4 | | t6 | Packet
+---------+ +---------+ +---------+ +---------+ Timestamps
| t1 | |t3=clock | | t5 | |t7=clock |
+---------+ +---------+ +---------+ +---------+
|t2=clock | |t6=clock |
+---------+ +---------+
Peer B
+---------+ +---------+ +---------+ +---------+
org | T1 | | T1 | | t5<>T1? | | T5 |
+---------+ +---------+ +---------+ +---------+ State
rec | T2 | | T2 | | T6 | | T6 | Variables
+---------+ +---------+ +---------+ +---------+
xmt | 0 | | T3 | | t3=T3? | | T7 |
+---------+ +---------+ +---------+ +---------+
t2 t3 t6 t7
---------------------------------------------------------
/\ \ /\ \
/ \ / \
/ \ / \
/ \/ / \/
---------------------------------------------------------
t1 t4 t5 t8
t1 t4 t5 t8
+---------+ +---------+ +---------+ +---------+
| 0 | | t1 | | t3 | | t5 |
+---------+ +---------+ +---------+ +---------+
| 0 | | t2 | | t4 | | t6 | Packet
+---------+ +---------+ +---------+ +---------+ Timestamps
|t1=clock | | t3 | |t5=clock | | t7 |
+---------+ +---------+ +---------+ +---------+
|t4=clock | |t8=clock |
+---------+ +---------+
Peer A
+---------+ +---------+ +---------+ +---------+
org | 0 | | t3<>0? | | T3 | | t7<>T3? |
+---------+ +---------+ +---------+ +---------+ State
rec | 0 | | T4 | | T4 | | T8 | Variables
+---------+ +---------+ +---------+ +---------+
xmt | T1 | | t1=T1? | | T5 | | t5=T5? |
+---------+ +---------+ +---------+ +---------+
Figure 15: On-Wire Protocolin the figure, the first packet A sends has only the origin timestamp t1, which then gets copied to T1 🌸 B receives it at t2, copies t1 to T1, and copies the receive timestamp t2 to T2. right now or some time later at t3, B sends a packet to A containing t1 and t2 plus the transmit timestamp t3, and all three get copied to their matching state variables 💅 A receives the packet at t4, which has the three timestamps t1, t2, and t3 and also its own destination timestamp t4. those four timestamps are used to work out the offset and delay of B relative to A, as explained below.
before the xmt and org state variables get updated, there are two sanity checks to protect against duplicate, bogus, or replayed packets 🛡️ in the exchange above, a packet counts as duplicate or replay if the transmit timestamp t3 in the packet matches the org state variable T3. a packet is bogus if the origin timestamp t1 in the packet doesn't match the xmt state variable T1. in either case, the state variables are updated and then the packet is discarded 🙈 to protect against replay of the last transmitted packet, the xmt state variable is set to zero right after a successful bogus check. this is the kind of detail that makes me obsessed.
the four most recent timestamps, T1 through T4, are used to compute the offset of B relative to A:
theta = T(B) - T(A) = 1/2 * [(T2-T1) + (T3-T4)]and the round-trip delay:
delta = T(ABA) = (T4-T1) - (T3-T2).⚠️ okay this part is important! the quantities in parentheses are computed from 64-bit unsigned timestamps and come out as signed values with 63 significant bits plus sign. these can represent dates from 68 years in the past to 68 years in the future. BUT the offset and delay are computed as sums and differences of those values, which hold 62 significant bits and two sign bits, so they can only represent unambiguous values from 34 years in the past to 34 years in the future. in other words, the time of the client must be set within 34 years of the server before the service is started. this is a fundamental limitation with 64-bit integer arithmetic 😭
in implementations where floating double arithmetic is available, the first-order differences can be converted to floating double and the second-order sums and differences computed in that arithmetic ✨ since the second-order terms are typically very small compared to the timestamp magnitudes, there's no loss in significance, and the unambiguous range is restored from 34 years to 68 years. slay 👑
in some scenarios where the client's initial frequency offset is relatively large and the actual propagation time is small, the delay computation can become negative 💀 for instance, if the frequency difference is 100 ppm and the interval T4-T1 is 64 s, the apparent delay is -6.4 ms. since negative values are misleading in later computations, the value of delta should be clamped to not less than s.rho, where s.rho is the system precision described in Section 11.1, expressed in seconds.
the discussion above assumes the most general case, where two symmetric peers independently measure the offsets and delays between them. with a stateless server, the protocol can be simplified 🧚♀️ a stateless server copies T3 and T4 from the client packet to T1 and T2 of the server packet, and then tacks on the transmit timestamp T3 before sending it back to the client. additional details for filling in the remaining protocol fields are given in a Section 9 and following sections and in the appendix.
one more thing, and this is the tea 🍵 the on-wire protocol as described resists replay of a server response packet. however, it does NOT resist replay of the client request packet, which would result in a server reply packet with new values of T2 and T3, leading to an incorrect offset and delay. this vulnerability can be avoided by setting the xmt state variable to zero after computing the offset and delay 🫶
9. Peer Process 🦋
the process descriptions to follow include a listing of the important state variables, followed by an overview of the process operations implemented as routines 📝 frequent reference is made to the skeleton in the appendix. the skeleton includes C-language fragments that describe the functions in more detail, and it includes the parameters, variables, and declarations needed for a conforming NTPv4 implementation. however, a working implementation may need many additional variables and routines 🌈
the peer process is called when a server or peer packet arrives 🎀 it runs the on-wire protocol to determine the clock offset and round-trip delay, and it computes statistics used by the system and poll processes. peer variables get instantiated in the association data structure when that structure is initialized, and they're updated by arriving packets. there's a peer process, a poll process, and an association process for each server, so it's a whole squad 💜
9.1. Peer Process Variables ✨
okay bestie, Figures 16, 17, 18, and 19 are basically the cheat sheet 📋💖 They list the common names, the formula names, and a quick description of each peer variable. The common names and formula names are totally interchangeable, but the formula names exist so the equations in this spec are easier to read 🧮✨ Unless it says otherwise, every peer variable has the prefix p 🦋
+---------+----------+-----------------------+
| Name | Formula | Description |
+---------+----------+-----------------------+
| srcaddr | srcaddr | source address |
| srcport | srcport | source port |
| dstaddr | dstaddr | destination address |
| dstport | destport | destination port |
| keyid | keyid | key identifier key ID |
+---------+----------+-----------------------+
Figure 16: Peer Process Configuration Variables +-----------+------------+---------------------+
| Name | Formula | Description |
+-----------+------------+---------------------+
| leap | leap | leap indicator |
| version | version | version number |
| mode | mode | mode |
| stratum | stratum | stratum |
| ppoll | ppoll | peer poll exponent |
| rootdelay | delta_r | root delay |
| rootdisp | epsilon_r | root dispersion |
| refid | refid | reference ID |
| reftime | reftime | reference timestamp |
+-----------+------------+---------------------+
Figure 17: Peer Process Packet Variables +------+---------+--------------------+
| Name | Formula | Description |
+------+---------+--------------------+
| org | T1 | origin timestamp |
| rec | T2 | receive timestamp |
| xmt | T3 | transmit timestamp |
| t | t | packet time |
+------+---------+--------------------+
Figure 18: Peer Process Timestamp Variables +--------+---------+-----------------+
| Name | Formula | Description |
+--------+---------+-----------------+
| offset | theta | clock offset |
| delay | delta | round-trip delay|
| disp | epsilon | dispersion |
| jitter | psi | jitter |
| filter | filter | clock filter |
| tp | t_p | filter time |
+--------+---------+-----------------+
Figure 19: Peer Process Statistics VariablesHere's the vibe for the configuration variables 🎀 They usually get set up when the association is mobilized, either from a config file or when the very first packet shows up from an association you've never seen before 👀
srcaddr: this is the IP address of the remote server or reference clock 🖥️ It becomes the destination IP address in packets sent from this association.
srcport: this is the UDP port number of the server or reference clock 🔌 It becomes the destination port number in packets sent from this association. When you're operating in symmetric modes (1 and 2), this field must contain the NTP port number PORT (123) that the IANA assigned 💜 In other modes, it can contain any number that fits local policy.
dstaddr: this is the IP address of the client 🧚 It becomes the source IP address in packets sent from this association.
dstport: this is the UDP port number of the client, which is ordinarily the NTP port number PORT (123) assigned by the IANA 🌸 It becomes the source port number in packets sent from this association.
keyid: this is the symmetric key ID for the 128-bit MD5 key used to generate and verify the MAC 🔐 The client and the server or peer can use different values, but they must map to the same key. It's giving secret handshake 🤝
The variables in Figure 17 get updated from the packet header every time a packet arrives 📦 They're interpreted the same way as the packet variables with the same names. For later processing, it's super convenient to convert the NTP short format packet values r.rootdelay and r.rootdisp into floating doubles as peer variables 🍓
The variables in Figure 18 include the timestamps that get exchanged by the on-wire protocol in Section 8 ⏱️ The t variable is the seconds counter c.t associated with these values. The c.t variable is maintained by the clock-adjust process described in Section 12, and it counts the seconds since the service was started 🌈 The variables in Figure 19 include the statistics computed by the clock_filter() routine described in Section 10 📊 The tp variable is the seconds counter associated with those values 💅
9.2. Peer Process Operations ✨
Okay bestie, the receive routine is where the magic happens 🦋 Every time a packet shows up, this is the flow that decides what happens next. An example is the receive() routine in Appendix A.5.1. There's no specific method required for access control 💅, though it is recommended that implementations include some kind of scheme, and it's pretty similar to lots of others already out in the wild. The access() routine in Appendix A.5.4 shows how to do access restrictions with an access control list (ACL). Format checks are like the bouncers at the door 🎀: they make sure the field length and alignment are correct, the version number is acceptable (1-4), and the extension field syntax is right, if there is any.
Authentication isn't strictly required 🙈 but if you do implement it, the MD5-keyed hash algorithm described in [RFC1321] must be supported.
Next, the association table gets searched for a matching source address and source port, for example with the find_assoc() routine in Appendix A.5.1. Figure 20 is a dispatch table 🍓 The columns are the packet mode, the rows are the association mode, and wherever they intersect, the processing gets sent to one of the steps below.
+------------------+---------------------------------------+
| | Packet Mode |
+------------------+-------+-------+-------+-------+-------+
| Association Mode | 1 | 2 | 3 | 4 | 5 |
+------------------+-------+-------+-------+-------+-------+
| No Association 0 | NEWPS | DSCRD | FXMIT | MANY | NEWBC |
| Symm. Active 1 | PROC | PROC | DSCRD | DSCRD | DSCRD |
| Symm. Passive 2 | PROC | ERR | DSCRD | DSCRD | DSCRD |
| Client 3 | DSCRD | DSCRD | DSCRD | PROC | DSCRD |
| Server 4 | DSCRD | DSCRD | DSCRD | DSCRD | DSCRD |
| Broadcast 5 | DSCRD | DSCRD | DSCRD | DSCRD | DSCRD |
| Bcast Client 6 | DSCRD | DSCRD | DSCRD | DSCRD | PROC |
+------------------+-------+-------+-------+-------+-------+
Figure 20: Peer Dispatch TableDSCRD. This one's a non-fatal protocol violation caused by a programming error, a long-delayed packet, or a replayed packet. The peer process discards the packet and exits.
ERR. This one's a fatal protocol violation, caused by the same kinds of things: a programming error, a long-delayed packet, or a replayed packet. The peer process discards the packet, demobilizes the symmetric passive association, and exits. 💀
FXMIT. This is when a client (mode 3) packet shows up and matches no association (mode 0). If the destination address isn't a broadcast address, the server builds a server (mode 4) packet and sends it back to the client without keeping any state ✨ The server packet header is put together as shown in Figure 21, and an example is the fast_xmit() routine in Appendix A.5.3. It's built from the receive packet and the system variables. If the s.rootdelay and s.rootdisp system variables are stored as floating double, they must be converted to NTP short format first.
+-----------------------------------+
| Packet Variable --> Variable |
+-----------------------------------+
| r.leap --> p.leap |
| r.mode --> p.mode |
| r.stratum --> p.stratum |
| r.poll --> p.ppoll |
| r.rootdelay --> p.rootdelay |
| r.rootdisp --> p.rootdisp |
| r.refid --> p.refid |
| r.reftime --> p.reftime |
| r.keyid --> p.keyid |
+-----------------------------------+
Figure 21: Receive Packet HeaderNow, if authentication fails, the server sends back a special message called a crypto-NAK 🚫 It has the normal NTP header data from Figure 8, but the MAC is four octets of zeros. The client MAY accept or reject the data in that message. After all of that, the peer process exits.
If the destination address is a multicast address, the sender is operating in manycast client mode 📡 If the packet is valid and the server stratum is less than the client stratum, the server sends an ordinary server (mode 4) packet, but this one uses its unicast destination address. A crypto-NAK is not sent if authentication fails. After these actions, the peer process exits.
MANY: This is a server (mode 4) packet that matches no association. Ordinarily it only happens when a manycast server replies to a multicast client packet that was sent earlier. If the packet is valid, an ordinary client (mode 3) association gets mobilized, and everything keeps going as if the association had been mobilized by the configuration file. 🧚
NEWBC. This is a broadcast (mode 5) packet that matches no association. The client mobilizes either a client (mode 3) or a broadcast client (mode 6) association. Examples are in the mobilize() and clear() routines in Appendix A.2. After that, the packet gets validated and the peer variables get initialized. An example is the packet() routine in Appendix A.5.1.1.
If the implementation doesn't support any extra security or calibration functions, the association mode is set to broadcast client (mode 6) and the peer process exits. Implementations supporting public key authentication MAY run the Autokey or an equivalent security protocol 🔐 Implementations SHOULD set the association mode to 3 and run a short client/server exchange to figure out the propagation delay. Once that's done, the association mode is set to 6 and the peer process keeps going in listen-only mode 👂 Heads up, a mode-6 packet is reserved for the NTP monitor and control functions, which is totally different from a mode-6 association. Don't mix those up, bestie 🙈
NEWPS. This is a symmetric active (mode 1) packet that matches no association. The client mobilizes a symmetric passive (mode 2) association. An example is in the mobilize() and clear() routines in Appendix A.2. Processing continues in the PROC section just below.
PROC. This is a packet that matches an existing association. The packet timestamps get checked really carefully so that invalid, duplicate, or bogus packets don't slip through 🕵️♀️ The additional checks are summarized in Figure 22. Note that every packet, including a crypto-NAK, only counts as valid if it survives these tests.
+--------------------------+----------------------------------------+
| Packet Type | Description |
+--------------------------+----------------------------------------+
| 1 duplicate packet | The packet is at best an old duplicate |
| | or at worst a replay by a hacker. |
| | This can happen in symmetric modes if |
| | the poll intervals are uneven. |
| 2 bogus packet | |
| 3 invalid | One or more timestamp fields are |
| | invalid. This normally happens in |
| | symmetric modes when one peer sends |
| | the first packet to the other and |
| | before the other has received its |
| | first reply. |
| 4 access denied | The access controls have blacklisted |
| | the source. |
| 5 authentication failure | The cryptographic message digest does |
| | not match the MAC. |
| 6 unsynchronized | The server is not synchronized to a |
| | valid source. |
| 7 bad header data | One or more header fields are invalid. |
+--------------------------+----------------------------------------+
Figure 22: Packet Error ChecksProcessing then keeps going by copying the packet variables to the peer variables, like in Figure 21 ✨ An example is the packet() routine in Appendix A.5.1.1. The receive() routine handles tests 1-5 from Figure 22, and the packet() routine handles tests 6-7. If any errors pop up, the packet gets discarded and the peer process exits. ❌
The on-wire protocol works out the clock offset theta and the round-trip delay delta from the four most recent timestamps, as described in Section 8 🧮 Technically, you could do every calculation except the first-order timestamp differences in fixed-point arithmetic, but it's way easier to convert those first-order differences to floating doubles and do the rest in that arithmetic. That's what gets assumed in the description that follows.
Next up, the 8-bit p.reach shift register in the poll process (Section 13) tells us whether the server is reachable and whether the data are fresh 👀 The register shifts left by one bit whenever a packet gets sent, and the rightmost bit is set to zero. As valid packets come in, the rightmost bit gets set to one. If the register has any nonzero bits, the server counts as reachable; otherwise, it's unreachable. The peer poll interval might have changed since the last packet, so the host poll interval gets reviewed too. An example is the poll_update() routine in Appendix A.5.7.2.
The dispersion statistic epsilon(t) is the maximum error from the frequency tolerance and the time since the last packet was sent 💜 It gets initialized as:
epsilon(t_0) = r.rho + s.rho + PHI * (T4-T1)This happens when the measurement is taken at t_0, according to the seconds counter. Here, r.rho is the packet precision from Section 7.3, and s.rho is the system precision from Section 11.1, both in seconds. These terms are there to account for the uncertainty in reading the system clock on both the server and the client. Lowkey important stuff, iykyk 🫶
After that, the dispersion grows at a constant rate PHI. In other words, at time t, we get:
epsilon(t) = epsilon(t_0) + PHI * (t-t_0)With the default value PHI = 15 ppm, that adds up to about 1.3 s per day 🌈 Since the argument t gets dropped from here on, the dispersion is just written as epsilon. The remaining statistics are computed by the clock filter algorithm, which is up next 🦋
10. Clock Filter Algorithm ✨
okay so the clock filter algorithm lives inside the peer process, and its whole job is to groom the stream of on-wire data 💅 so it picks out the samples most likely to show accurate time. It spits out the variables in Figure 19, including the offset (theta), delay (delta), dispersion (epsilon), jitter (psi), and time of arrival (t) 🦋. The mitigation algorithms use these to work out the best and final offset for disciplining the system clock, and they also use them to judge server health and whether a server is suitable for synchronization. literally the main character of this whole process 👑
The clock filter keeps the most recent sample tuples (theta, delta, epsilon, t) in the filter structure, which acts like an 8-stage shift register 🧚. Tuples get saved in the order the packets arrive. Quick warning: here, t is the packet time of arrival according to the seconds counter, so don't mix it up with the peer variable tp 🙈
So how does it make sure there are enough samples and tosses out the old stale stuff? 🍓 At the start, every stage holds the dummy tuple (0, MAXDISP, MAXDISP, 0). As valid packets roll in, they get shifted into the filter, the old tuples fall off the end, and eventually only valid tuples are left. ✨
If the three low-order bits of the reach register are zero, that means three poll intervals have gone by with no valid packets received 😭. In that case the poll process calls the clock filter with a dummy tuple, just as if it had come in from the network. If this keeps going for eight poll intervals, the register goes right back to its initial condition.
Next, the shift register stages get copied into a temporary list, and that list gets sorted by increasing delta 🎀. Let i index the stages starting from the lowest delta. If the first tuple epoch t_0 is not later than the last valid sample epoch tp, the routine just exits and leaves the current peer variables alone. Otherwise, let epsilon_i be the dispersion of the ith entry, and then:
i=n-1
--- epsilon_i
epsilon = \ ----------
/ (i+1)
--- 2
i=0This gives you the peer dispersion p.disp. Heads up, epsilon gets overloaded here, meaning it can be an input to the clock filter or an output. The meaning should be clear from context 💜
Some observer notes, because these are so good 🤩:
- (a) If all stages hold the dummy tuple with dispersion MAXDISP, the computed dispersion comes out a little less than 16 s.
- (b) Every time a valid tuple gets shifted into the register, the dispersion drops by a little less than half, depending on the dispersion of that valid tuple.
- (c) After the fourth valid packet, the dispersion is usually a little less than 1 s. That's the assumed value of the MAXDIST parameter, which the selection algorithm uses to decide whether the peer variables are acceptable 🔥
Now let the first stage offset in the sorted list be theta_0. For the other stages, in any order, the jitter is the RMS average:
+----- -----+^1/2
| n-1 |
| --- |
1 | \ 2 |
psi = -------- * | / (theta_0-theta_j) |
(n-1) | --- |
| j=1 |
+----- -----+Here n is the number of valid tuples in the filter, and n has to be greater than 1 🫶. To keep things consistent and avoid divide exceptions in other calculations, psi gets bounded from below by the system precision s.rho, expressed in seconds. Jitter isn't usually a major factor when ranking server quality, but it's a valuable indicator of fundamental timekeeping performance and network congestion state 🌈
The one that matters most to the mitigation algorithms is the peer synchronization distance, which comes from the delay and dispersion:
lambda = (delta / 2) + epsilon.Epsilon, and therefore lambda, increase at rate PHI. Lambda isn't a state variable, since it gets recalculated every time it's used. It's a component of the root synchronization distance, and the mitigation algorithms use it as a metric to judge how good the time from each server is ⭐
It's important to note that NTPv4 is different from NTPv3 here. NTPv4 associations do NOT show a timeout by setting the stratum to 16 and the leap indicator to 3 💀 (okay so I'm yelling a little, but it's only because it's the vibe!). Instead, the association variables keep the values from when the last packet arrived. Since lambda increases with time, eventually the synchronization distance goes over the distance threshold MAXDIST, and then the association is considered unfit for synchronization 🙅♀️
If you want to see an example implementation, check out the clock_filter() routine in Appendix A.5.2 🎀
11. System Process ✨
Every time a new sample (theta, delta, epsilon, jitter, t) gets produced by the clock filter algorithm, the mitigation algorithms scan all the peer processes 🔍. Those algorithms are the selection, cluster, combine, and clock discipline algorithms, and they all live in the system process. The selection algorithm checks every association and casts off the falsetickers, which are the ones with demonstrably incorrect time, so only the truechimers are left as the result 💖. Over a series of rounds, the cluster algorithm discards whichever association is statistically furthest from the centroid, until a specified minimum number of survivors is left 🦋. The combine algorithm then makes the best and final statistics on a weighted average basis ✨. Finally, the final offset gets passed to the clock discipline algorithm, which steers the system clock to the correct time 🫶
The cluster algorithm picks one of the survivors to be the system peer 👑. The statistics attached to that peer (theta, delta, epsilon, jitter, t) are used to build the system variables. Those get inherited by dependent servers and clients, and they're also made available to other applications running on the same machine 🌸
11.1. System Process Variables
Figure 23 sums up the common names, formula names, and a short description of each system variable. Unless noted otherwise, every variable has the assumed prefix s 🎀
+-----------+------------+------------------------+
| Name | Formula | Description |
+-----------+------------+------------------------+
| t | t | update time |
| p | p | system peer identifier |
| leap | leap | leap indicator |
| stratum | stratum | stratum |
| precision | rho | precision |
| offset | THETA | combined offset |
| jitter | PSI | combined jitter |
| rootdelay | DELTA | root delay |
| rootdisp | EPSILON | root dispersion |
| v | v | survivor list |
| refid | refid | reference ID |
| reftime | reftime | reference time |
| NMIN | 3 | minimum survivors |
| CMIN | 1 | minimum candidates |
+-----------+------------+------------------------+
Figure 23: System Process VariablesExcept for the t, p, offset, and jitter variables and the NMIN and CMIN constants, the variables have the same format and interpretation as the peer variables with the same name. The NMIN and CMIN parameters get used by the selection and cluster algorithms, which we'll cover in the next section 🧚
The t variable is the seconds counter at the time of the last update. You can see an example in the clock_update() routine in Appendix A.5.5.4 ⏰. The p variable is the system peer identifier, which is determined by the cluster() routine in Section 11.2.2 👑. The precision variable has the same format as the packet variable with the same name. Precision is defined as the larger of the resolution and the time it takes to read the clock, in log2 units. For instance, a mains-frequency clock ticking at 60 Hz has a precision of 16 ms, even when the system clock hardware represents time down to the nanosecond 🤓
The offset and jitter variables are determined by the combine algorithm in Section 11.2.3. They represent the best and final offset and jitter used to discipline the system clock 💅
Initially, all variables are cleared to zero. After that, leap is set to 3 (unsynchronized) and stratum is set to MAXSTRAT (16). Remember that MAXSTRAT gets mapped to zero in the transmitted packet 🙈
11.2. System Process Operations
Figure 24 sums up the system process operations performed by the clock select routine. The selection algorithm in Section 11.2.1 produces a majority clique of presumed correct candidates (truechimers) based on agreement principles 🤝. The cluster algorithm in Section 11.2.2 discards outliers to produce the most accurate survivors ✨. The combine algorithm in Section 11.2.3 provides the best and final offset for the clock discipline algorithm. There's an example in Appendix A.5.5.6 🦋
If the selection algorithm can't produce a majority clique, or it can't produce at least CMIN survivors, the system process exits without disciplining the system clock 😭. If it succeeds, the cluster algorithm selects the statistically best candidate as the system peer, and that peer's variables get inherited as the system variables 🎉
+-----------------+
| clock_select() |
+-----------------+
................................|...........
. V .
. yes +---------+ +-----------------+ .
. +--| accept? | | scan candidates | .
. | +---------+ | | .
. V no | | | .
. +---------+ | | | .
. | add peer| | | | .
. +---------- | | | .
. | V | | .
. +---------->-->| | .
. | | .
. Selection Algorithm +-----------------+ .
.................................|..........
V
no +-------------------+
+-------------| survivors? |
| +-------------------+
| | yes
| V
| +-------------------+
| | Cluster Algorithm |
| +-------------------+
| |
| V
V yes +-------------------+
|<------------| n < CMIN? |
| +-------------------+
V |
+-----------------+ V no
| s.p = NULL | +-------------------+
+-----------------+ | s.p = v_0.p |
| +-------------------+
V |
+-----------------+ V
| return (UNSYNC) | +-------------------+
+-----------------+ | return (SYNC) |
+-------------------+
Figure 24: Clock Select Routine11.2.1. Selection Algorithm ✨
okay bestie, so the selection and cluster algorithms are described separately but get mashed together in the code skeleton 🫶 the selection one is about finding an intersection interval that holds a majority clique of truechimers. It uses Byzantine agreement ideas that Marzullo [ref6] first proposed, but with a glow-up for accuracy 💅 You can see the big picture below, and the detailed version lives in the first half of the clock_select() routine in Appendix A.5.5.1.
First, anyone who's unusable under the protocol rules gets detected and tossed out, which is what the accept() routine in Appendix A.5.5.3 does 🚫 Then every remaining candidate gets a set of tuples (p, type, edge). Here p is the association identifier, and type marks the upper (+1), middle (0), and lower (-1) endpoints of a correctness interval centered on theta for that candidate. So each candidate gives three tuples: lowpoint (p, -1, theta - lambda), midpoint (p, 0, theta), and highpoint (p, +1, theta + lambda), where lambda is the root synchronization distance. There's an example of this math in the rootdist() routine in Appendix A.5.1.1 🧮 The steps go like this:
- For each of m associations, place three tuples as defined above on the candidate list.
- Sort the tuples on the list by the edge component. Order the lowpoint, midpoint, and highpoint of these intervals from lowest to highest. Set the number of falsetickers f = 0.
- Set the number of midpoints d = 0. Set c = 0. Scan from lowest endpoint to highest. Add one to c for every lowpoint, subtract one for every highpoint, add one to d for every midpoint. If c >= m - f, stop; set l = current lowpoint.
- Set c = 0. Scan from highest endpoint to lowest. Add one to c for every highpoint, subtract one for every lowpoint, add one to d for every midpoint. If c >= m - f, stop; set u = current highpoint.
- Is d = f and l < u? If yes, then follow step 5A; else, follow step 5B.
5A. Success: the intersection interval is [l, u].
5B. Add one to f. Is f < (m / 2)? If yes, then go to step 3 again. If no, then go to step 6.
- Failure; a majority clique could not be found. There are no suitable candidates to discipline the system clock. 😭
The full algorithm is in Appendix A.5.5.1 🌸 It starts by assuming there are no falsetickers (f = 0) and hunts for a non-empty intersection interval that holds the midpoints of all correct servers, aka truechimers. If it can't find a non-empty interval, it bumps the assumed number of falsetickers up by one and tries again. Once it finds a non-empty interval and the number of falsetickers is less than the number of truechimers, boom, a majority clique is found 👑 At that point the midpoint of each truechimer (theta) is what the cluster algorithm gets to work with.
If there's no majority clique, or the number of truechimers is less than CMIN, then there aren't enough candidates to discipline the system clock 🥲 CMIN is the minimum number of servers that still fits the correctness requirements. Suspicious operators would crank CMIN up so multiple redundant servers are around for the algorithms to handle properly. But for historic reasons, the default value for CMIN is one 🙈
11.2.2. Cluster Algorithm ✨
The majority clique candidates get put on the survivor list v as tuples (p, theta_p, psi_p, lambda_p) 🎀 Here p is an association identifier, theta_p, psi_p, and stratum_p are the current offset, jitter, and stratum of association p, and lambda_p is a merit factor equal to stratum_p * MAXDIST + lambda, where lambda is the root synchronization distance for association p. The cluster algorithm processes that list below, and there's an example in the second half of the clock_select() algorithm in Appendix A.5.5.1 🦋
- Let (p, theta_p, psi_p, lambda_p) represent a survivor candidate.
- Sort the candidates by increasing
lambda_p. Let n be the number of candidates andNMINthe minimum required number of survivors. - For each candidate, compute the selection jitter
psi_s:
+----- -----+^1/2
| n-1 |
| --- |
| 1 \ 2 |
psi_s = | ---- * / (theta_s - theta_j) |
| n-1 --- |
| j=1 |
+----- -----+- Select
psi_maxas the candidate with maximumpsi_s. - Select
psi_minas the candidate with minimumpsi_p. - Is
psi_max<psi_minor n <=NMIN? If yes, follow step 6A; otherwise, follow step 6B.
6A. Done. The remaining candidates on the survivor list are ranked in the order of preference. The first entry on the list represents the system peer; its variables are used later to update the system variables. 👑
6B. Delete the outlier candidate with psi_max; reduce n by one and go back to step 3.
This algorithm runs in rounds, and each round tosses the statistical outlier with the biggest selection jitter psi_s 🗑️ But if psi_s is less than the minimum peer jitter psi_p, then tossing outliers won't help anymore. That, plus the minimum number of survivors, are the terminating conditions. When it wraps up, the final value of psi_max gets saved as the system selection jitter PSI_s for later use ✨
11.2.3. Combine Algorithm ✨
The clock combine route takes the leftover survivors and makes the best, final data for the clock discipline algorithm 💖 It crunches the peer offset and jitter stats into the combined system offset THETA and system peer jitter PSI_p. Each server's stat gets weighted by the reciprocal of its root synchronization distance, then the result gets normalized. There's an example in the clock_combine() routine in Appendix A.5.5.5 🧁
The combined THETA gets passed to the clock update routine. The first candidate on the survivor list is nominated as the system peer with identifier p. The system peer jitter PSI_p is one piece of the system jitter PSI, and it gets used together with the selection jitter PSI_s to make the system jitter:
PSI = [(PSI_s)^2 + (PSI_p)^2]^1/2Every time an update comes in from the system peer, the clock update routine gets called 📨 By rule, an update gets discarded if its time of arrival p.t is not strictly later than the last update used s.t. The labels IGNOR, PANIC, ADJ, and STEP are return codes from the local clock routine, which the next section covers 🔍
IGNORE means the update got ignored as an outlier 🙅 PANIC means the offset is bigger than the panic threshold PANICT (1000 s) and SHOULD cause the program to exit with a diagnostic message to the system log 🚨 STEP means the offset is under the panic threshold but bigger than the step threshold STEPT (125 ms). In this case, the clock gets stepped to the correct offset. Since that means all peer data have been invalidated, all associations MUST be reset and the client begins as at initial start. Major reset energy 🔄
ADJ means the offset is under the step threshold, so it's a valid update ✅ In this case, the system variables get updated from the peer variables, as shown in Figure 25:
+-------------------------------------------+
| System Variable <-- System Peer Variable | |
+-------------------------------------------+
| s.leap <-- p.leap |
| s.stratum <-- p.stratum + 1 |
| s.offset <-- THETA |
| s.jitter <-- PSI |
| s.rootdelay <-- p.delta_r + delta |
| s.rootdisp <-- p.epsilon_r + p.epsilon + |
| p.psi + PHI * (s.t - p.t) |
| + |THETA| |
| s.refid <-- p.refid |
| s.reftime <-- p.reftime |
| s.t <-- p.t |
+-------------------------------------------+
Figure 25: System Variables UpdateThere's an important detail that the figure doesn't show 👀 The dispersion increment (p.epsilon + p.psi + PHI * (s.t - p.t) + |THETA|) is bounded from below by MINDISP. In subnets with super fast processors and networks, and really tiny delay and dispersion, this forces a monotone-definite increase in s.rootdisp (EPSILON). That's what stops loops between peers operating at the same stratum 🔁🚫
The system variables are available to dependent application programs as nominal performance statistics 📊 The system offset THETA is the clock offset relative to the available synchronization sources. The system jitter PSI is an estimate of the error in figuring out this value, which is also called the expected error. The root delay DELTA is the total round-trip delay relative to the primary server. The root dispersion EPSILON is the dispersion that piles up over the network from the primary server. Finally, the root synchronization distance is defined as:
LAMBDA = EPSILON + DELTA / 2,which represents the maximum error due to all causes, and it's the thing designated as the root synchronization distance 🦋
There's an example of the clock update routine in Appendix A.5.5.4 🌈
11.3. Clock Discipline Algorithm ✨
okay so the NTPv4 clock discipline algorithm (we'll just call it "discipline" from here, bestie 🫶) is basically two super different feedback control systems working as one team 👯♀️. In a phase-locked loop (PLL) design, periodic phase updates every mu seconds are used directly to minimize the time error and, indirectly, the frequency error. In a frequency-locked loop (FLL) design, periodic frequency updates at intervals mu go straight at the frequency error and indirectly fix the time error. As shown in [ref7], a PLL usually works better when network jitter is the main problem, while an FLL works better when oscillator wander is the main problem 🌊. This section gives you the outline of how the NTPv4 design works. If you want the deep dive on the design principles (plus a performance analysis), go read [ref7] 📚.
The discipline is built as the feedback control system in Figure 26 🔁. Here theta_r is the combine algorithm offset (the reference phase) and theta_c is the VFO offset (the control phase). Each update spits out a signal V_d, which is just the instantaneous phase difference theta_r - theta_c. The clock filter for each server works like a tapped delay line, with the output coming from whichever tap the clock filter algorithm picks 🎀. The selection, cluster, and combine algorithms gather data from all those filters and cook up the signal V_s 🍳. Then the loop filter, which has impulse response F(t), makes the signal V_c. That signal controls the VFO frequency omega_c and, through it, the integral of the phase theta_c, which is what closes the loop 💖. The V_c signal is generated by the clock-adjust process in Section 12. The detailed equations are best presented in the routines of Appendices A.5.5.6 and A.5.6.1 🧚.
theta_r + +---------\ +----------------+
NTP --------->| Phase \ V_d | | V_s
theta_c - | Detector ------>| Clock Filter |----+
+-------->| / | | |
| +---------/ +----------------+ |
| |
----- |
/ \ |
| VFO | |
\ / |
----- ....................................... |
^ . Loop Filter . |
| . +---------+ x +-------------+ . |
| V_c . | |<-----| | . |
+------.-| Clock | y | Phase/Freq |<---------+
. | Adjust |<-----| Prediction | .
. | | | | .
. +---------+ +-------------+ .
.......................................Figure 26: Clock Discipline Feedback Loop
Normally the pseudo-linear feedback loop I just described does the job of disciplining the system clock, no problem 💅. BUT there are moments where a non-linear algorithm is way better 😤. Case one: the discipline starts without knowing the intrinsic clock frequency. The pseudo-linear loop takes several hours to get an accurate measurement, and for most of that time the poll interval can't go up. The non-linear loop below does it in 15 minutes, which is giving speed ⚡. Case two: sometimes there are bursts of big jitter because of congested network links 🚦. The state machine below resists error bursts lasting less than 15 minutes, so it doesn't freak out over a little drama 🙈.
Figure 27 is your cheat sheet for the variables and parameters 📝. It shows the variable (lowercase) or parameter (uppercase) name, the formula name, and a short description. Unless noted otherwise, all variables have an assumed prefix c. The variables t, tc, state, hyster, and count are integers, and everything else is a floating double 🧮. Each one's job gets explained in the algorithm descriptions below, so no need to panic yet 🫶.
+--------+------------+--------------------------+
| Name | Formula | Description |
+--------+------------+--------------------------+
| t | timer | seconds counter |
| offset | theta | combined offset |
| resid | theta_r | residual offset |
| freq | phi | clock frequency |
| jitter | psi | clock offset jitter |
| wander | omega | clock frequency wander |
| tc | tau | time constant (log2) |
| state | state | state |
| adj | adj | frequency adjustment |
| hyster | hyster | hysteresis counter |
| STEPT | 125 | step threshold (.125 s) |
| WATCH | 900 | stepout thresh(s) |
| PANICT | 1000 | panic threshold (1000 s) |
| LIMIT | 30 | hysteresis limit |
| PGATE | 4 | hysteresis gate |
| TC | 16 | time constant scale |
| AVG | 8 | averaging constant |
+--------+------------+--------------------------+Figure 27: Clock Discipline Variables and Parameters
The process terminates immediately if the offset is greater than the panic threshold PANICT (1000 s). That's a hard stop, no cap 🛑. The state transition function is described by the rstclock() function in Appendix A.5.5.7, and Figure 28 shows the table that routine uses 🗺️. It has four columns: the state name, the predicate and action if the offset theta is less than the step threshold, the predicate and actions otherwise, and some comments.
+-------+---------------------+-------------------+--------------+
| State | theta < STEP | theta > STEP | Comments |
+-------+---------------------+-------------------+--------------+
| NSET | ->FREQ | ->FREQ | no frequency |
| | adjust time | step time | file |
+-------+---------------------+-------------------+--------------+
| FSET | ->SYNC | ->SYNC | frequency |
| | adjust time | step time | file |
+-------+---------------------+-------------------+--------------+
| SPIK | ->SYNC | if < 900 s ->SPIK | outlier |
| | adjust freq | else ->SYNC | detected |
| | adjust time | step freq | |
| | | step time | |
+-------+---------------------+-------------------+--------------+
| FREQ | if < 900 s ->FREQ | if < 900 s ->FREQ | initial |
| | else ->SYNC | else ->SYNC | frequency |
| | step freq | step freq | |
| | adjust time | adjust time | |
+-------+---------------------+-------------------+--------------+
| SYNC | ->SYNC | if < 900 s ->SPIK | normal |
| | adjust freq | else ->SYNC | operation |
| | adjust time | step freq | |
| | | step time | |
+-------+---------------------+-------------------+--------------+Figure 28: State Transition Function
In the table entries, the next state is marked with the arrow ->, and the actions are listed below it 🏹. Actions like adjust time and adjust frequency get handled by the PLL/FLL feedback loop in the local_clock() routine. A step clock action sets the clock directly, but only after the stepout threshold WATCH (900 s) has passed, and only when the offset is more than the step threshold STEPT (.125 s). This is how it resists clock steps during extreme network congestion 🚧, which is honestly so smart.
The jitter (psi) and wander (omega) statistics are computed with an exponential average using weight factor AVG 📈. The time constant exponent (tau) gets decided by comparing psi with the magnitude of the current offset theta. If the offset is greater than PGATE (4) times the clock jitter, the hysteresis counter hyster goes down by two; otherwise it goes up by one 🎢. If hyster climbs to the upper limit LIMIT (30), tau goes up by one. If it drops to the lower limit -LIMIT (-30), tau goes down by one. Normally tau hovers near MAXPOLL, but it quickly drops if a temperature spike causes a frequency surge 🌡️. Hot take: the clock is very dramatic, and this algorithm is the emotional support it needs 🫠.
12. Clock-Adjust Process 🎀
The actual clock-adjust process runs at one-second intervals 🕐. It adds the frequency correction plus a fixed percentage of the residual offset theta_r. The theta_r is basically the exponential decay of the theta value that the loop filter made at each update 📉. The TC parameter scales the time constant so it matches the poll interval, because convenience is everything 💅. Also, the dispersion EPSILON goes up by PHI every second, so it keeps growing, lowkey like my to-do list 📋.
The clock-adjust process has a timer interrupt facility that drives the seconds counter c.t. It starts at zero when the service starts and goes up by one each second ⏱️. At each interrupt, the clock_adjust() routine gets called to pull in the clock discipline time and frequency adjustments. Then the associations get scanned to see if the seconds counter equals or exceeds the p.next state variable (we'll meet it in the next section 👀). If it does, the poll process gets called to send a packet and work out the next p.next value 📦.
An example of the clock-adjust process is the clock_adjust() routine in Appendix A.5.6.1 🧩.
13. Poll Process 🦋
Each association has a poll process that runs at regular intervals to construct and send packets in symmetric, client, and broadcast server associations 📬. It runs continuously, whether or not the servers are reachable, because it has to keep the clock filter and reach register in shape 🧹. Main character energy, always on the job 👑✨.
13.1. Poll Process Variables
Okay bestie, Figure 29 is basically your cheat sheet 📝 It lists the common names, the formula names, and a quick description for the poll process variables (the lowercase ones 💜) and the parameters (the UPPERCASE ones 💅). Unless noted otherwise, all variables have assumed prefix p ✨
+---------+---------+--------------------+
| Name | Formula | Description |
+---------+---------+--------------------+
| hpoll | hpoll | host poll exponent |
| last | last | last poll time |
| next | next | next poll time |
| reach | reach | reach register |
| unreach | unreach | unreach counter |
| UNREACH | 24 | unreach limit |
| BCOUNT | 8 | burst count |
| BURST | flag | burst enable |
| IBURST | flag | iburst enable |
+---------+---------+--------------------+Figure 29: Poll Process Variables and Parameters
The poll process variables are stored in the association data structure, right alongside the peer process variables 🫶 Here's the tea on each one:
hpoll: a signed integer for the poll exponent, in log2 seconds 🔥last: an integer for the seconds counter when the most recent packet was sentnext: an integer for the seconds counter when the next packet is to be sentreach: an 8-bit integer shift register shared by the peer and poll processes 🌸unreach: an integer for how many seconds the server has been unreachable 💀
I'll call out the parameters as we go 🦋
13.2. Poll Process Operations
Remember from before: once every second, the clock-adjust process gets called ⏰ It goes through each association one at a time and calls the poll routine on each. If the time for the next poll message is greater than the seconds counter, the routine bails right away 🏃♀️
Symmetric (modes 1, 2), client (mode 3), and broadcast server (mode 5) associations routinely send packets 📦 A broadcast client (mode 6) association also runs the routine to update the reach and unreach variables, but it does not send packets 🙈 The poll process calls the transmit process to send a packet. If it's in a burst (burst > 0), nothing further happens except calling the poll update routine to set the next poll interval.
If it's not in a burst, the reach variable gets shifted left by one bit, with zero replacing the rightmost bit. If the server hasn't been heard from for the last three poll intervals, the clock filter routine gets called to increase the dispersion. There's an example in Appendix A.5.7.3 if you want to see it in action 👀
If the BURST flag is lit and the server is reachable and a valid source of synchronization is available, the client sends a burst of BCOUNT (8) packets at each poll interval 💖 The gap between packets in the burst is two seconds. This is super useful for measuring jitter accurately with long poll intervals 🎀 If the IBURST flag is lit and this is the first packet sent when the server has been unreachable, the client sends a burst. This helps quickly bring the synchronization distance below the distance threshold so the clock can sync up ⭐
If the P_MANY flag is lit in the p.flags word of the association, this is a manycast client association 🧚 Manycast client associations send client mode packets to designated multicast group addresses at MINPOLL intervals. The association starts out with a TTL of 1. If by the time of the next poll there are fewer than MINCLOCK servers have been mobilized, the ttl is increased by one. If the ttl reaches the limit TTLMAX, without finding MINCLOCK servers, the poll interval increases until it reaches BEACON, and then it starts over from the beginning 🔄
The poll() routine has a feature that backs off the poll interval if the server becomes unreachable 🐢 If reach is nonzero, the server is reachable and unreach is set to zero. Otherwise, unreach is incremented by one for each poll, up to the maximum UNREACH. After that, hpoll is increased by one for each poll, which doubles the poll interval up to the maximum MAXPOLL determined by the poll_update() routine 📈 When the server becomes reachable again, unreach is set to zero, hpoll is reset to the tc system variable, and operation resumes normally 💜
A packet is sent by the transmit process 💌 Some header values are copied from the peer variables left by a previous packet, and others come from the system variables. Figure 30 shows which values get copied into each header field. In implementations that use floating double data types for root delay and root dispersion, these must be converted to NTP short format 🍓 Everything else is either copied intact from the peer and system variables or struck as a timestamp from the system clock ⏱️
+-----------------------------------+
| Packet Variable <-- Variable |
+-----------------------------------+
| x.leap <-- s.leap |
| x.version <-- s.version |
| x.mode <-- s.mode |
| x.stratum <-- s.stratum |
| x.poll <-- s.poll |
| x.precision <-- s.precision |
| x.rootdelay <-- s.rootdelay |
| x.rootdisp <-- s.rootdisp |
| x.refid <-- s.refid |
| x.reftime <-- s.reftime |
| x.org <-- p.xmt |
| x.rec <-- p.dst |
| x.xmt <-- clock |
| x.keyid <-- p.keyid |
| x.digest <-- md5 digest |
+-----------------------------------+Figure 30: xmit_packet Packet Header
The poll update routine gets called when a valid packet is received and immediately after a poll message has been sent 🫧 If it's in a burst, the poll interval is fixed at 2 s. Otherwise, the host poll exponent hpoll is set to the minimum of ppoll from the last packet received and hpoll from the poll routine, but never less than MINPOLL or greater than MAXPOLL 🦋 So the clock discipline can be oversampled but not undersampled. This is necessary to preserve subnet dynamic behavior and protect against protocol errors 🛡️
The poll exponent gets converted to an interval, which is added to the last poll time variable to determine the value of the next poll time variable. Finally, the last poll time variable is set to the current seconds counter ✨
14. Simple Network Time Protocol (SNTP)
Primary servers and clients that follow a subset of NTP, called the Simple Network Time Protocol (SNTPv4) [RFC4330], do not need to implement the mitigation algorithms described in Section 9 and following sections 🌈 SNTP is meant for primary servers with a single reference clock, as well as for clients with a single upstream server and no dependent clients. The fully developed NTPv4 implementation is meant for secondary servers with multiple upstream servers and multiple downstream servers or clients 💅 Other than those considerations, NTP and SNTP servers and clients are completely interoperable and can be mixed together in NTP subnets 🫶
An SNTP primary server that implements the on-wire protocol described in Section 8 has no upstream servers except a single reference clock 🕰️ In principle, it looks just like an NTP primary server that has the mitigation algorithms, and is therefore capable of mitigating between multiple reference clocks 🤓
When an SNTP primary server gets a client request, it builds and sends the reply packet as described in Figure 31. Note that the dispersion field in the packet header must be updated as described in Section 5 📝
+-----------------------------------+
| Packet Variable <-- Variable |
+-----------------------------------+
| x.leap <-- s.leap |
| x.version <-- r.version |
| x.mode <-- 4 |
| x.stratum <-- s.stratum |
| x.poll <-- r.poll |
| x.precision <-- s.precision |
| x.rootdelay <-- s.rootdelay |
| x.rootdisp <-- s.rootdisp |
| x.refid <-- s.refid |
| x.reftime <-- s.reftime |
| x.org <-- r.xmt |
| x.rec <-- r.dst |
| x.xmt <-- clock |
| x.keyid <-- r.keyid |
| x.digest <-- md5 digest |
+-----------------------------------+Figure 31: fast_xmit Packet Header
An SNTP client that implements the on-wire protocol has a single server and no dependent clients 📱 It can work with any subset of the NTP on-wire protocol. The simplest approach uses only the transmit timestamp of the server packet and ignores all other fields. However, the extra complexity of implementing the full on-wire protocol is minimal, so a full implementation is encouraged 💖
15. Security Considerations ✨
okay bestie, NTP security is kind of a big deal and honestly more intense than most distributed services 💀. The authentication part and the time-sync part are tangled together. Reliable time sync needs crypto keys that only work during a certain time window, but you can only enforce that window if the servers and clients are reliably synced to UTC. It's a loop. 🔄 On top of that, NTP is hierarchical by nature, so time and trust flow down from the primary servers at the root, through secondary servers, to the clients at the leaves 🌳.
A client can only claim to have authentic time for its apps if every server on the path up to the primary servers is authenticated. Each server authenticates the lower stratum servers below it, and by induction that authenticates the lowest stratum (primary) servers too. Also, authenticated does NOT automatically mean correct 🙈. Like, the signature can be perfect and the time can still be wrong. A client runs a bunch of associations with different servers at once and uses a carefully built agreement algorithm to pluck the truechimers out of the crowd, which might include falsetickers 🎀.
The NTP spec assumes the intruder wants to inject fake time values, mess with the protocol, or clog the network, servers, or clients with junk packets until resources run out and legit apps get denied service 😤. NTP already has a bunch of defenses built into its architecture, protocol, and algorithms 🛡️. The on-wire timestamp exchange is inherently resistant to spoofing, packet loss, and replay attacks 💖. The clock filter, selection, and clustering algorithms are designed to defend against evil cliques of Byzantine traitors 👑. These algorithms aren't necessarily built to beat a determined intruder, but along with the sanity checks they've worked really well over the years to deflect servers that are misbehaving while presumably still being friendly 🦋. Still, these mechanisms do NOT securely identify and authenticate servers to clients 😭. Without extra protection, an intruder can pull off any or all of these:
- An intruder can intercept and archive packets forever, along with all the public values ever generated and sent over the net.
- An intruder can generate packets faster than the server, network, or client can process them, especially if those packets need expensive cryptographic computations.
- In a wiretap attack, the intruder can intercept, modify, and replay a packet. It cannot permanently stop the original packet from being forwarded, so it can't break the wire, only tell lies and congest it 🧚. Usually the modified packet can't reach the victim before the original does, and it doesn't have the server's private keys or identity parameters.
- In a middleman or masquerade attack, the intruder sits between the server and client, so it can intercept, modify, and replay a packet AND stop the original from being forwarded. The middleman still doesn't have the server's private keys 🕵️.
The NTP security model also assumes these limitations:
- Public key algorithms take a relatively long time to run, and the time varies a lot. In general, time sync performance gets badly degraded if these algorithms have to run on every single NTP packet 🐢.
- In some modes of operation, it isn't feasible for a server to keep state variables for every client. It is, however, feasible to regenerate them for a client when a packet arrives from that client 🍓.
- The lifetime of cryptographic values has to be enforced, and that needs a reliable system clock. The catch is that the sources that sync the system clock have to be trusted. This circular dependence between the timekeeping and authentication functions needs special handling 🌀.
- Client security functions must involve only public values sent over the net. Private values must never be disclosed beyond the machine they were created on, except for a special trusted agent (TA) assigned for that purpose 🔐.
Here's the big difference from SSH 💅. In SSH the client has to be securely authenticated to the server, but in NTP it's the other way around: the server has to be securely authenticated to the client. In SSH, each interface address can be bound to a different name, as returned by a reverse-DNS query. With that design, separate public/private key pairs may be needed for each interface address that has a distinct name. The perceived upside is that the security compartment can be different for each interface, so a firewall could, for example, require some interfaces to authenticate the client and let others skip it 🔥.
For NTP as specified here, NTP broadcast clients are vulnerable to disruption from misbehaving or hostile SNTP or NTP broadcast servers anywhere on the Internet 😬. You can cut that disruption down in a few ways. Filtering can limit NTP clients' access to known or trusted NTP broadcast servers, which keeps malicious traffic from ever reaching the clients ✨. Cryptographic authentication at the client means only timing info from properly signed NTP messages gets used to sync its clock 🫶. For even higher levels of authentication, you can use the Autokey mechanism [RFC5906] 👑.
Section 8 describes a potential security concern with replayed client requests. Following the recommendations in that section protects against those attacks 🛡️.
It's worth noting that this specification describes an existing implementation. The MD5 algorithm's security shortfalls are well known, but its use in the NTP specification is consistent with widescale deployment across the Internet community 🌈.
16. IANA Considerations 🎀
UDP/TCP Port 123 was previously assigned by IANA for this protocol 🦋. IANA has assigned the IPv4 multicast group address 224.0.1.1 and the IPv6 multicast address ending :101 for NTP 📡. This document introduces NTP extension fields so future extensions can be developed. A particular extension is identified by the Field Type sub-field within the extension field 🧩. IANA has established and will maintain a registry for Extension Field Types for this protocol, and it starts out with no initial entries 📋. As new needs come up, new Extension Field Types may be defined. Following the policies in [RFC5226], new values are to be defined by IETF Review 🔍.
IANA also created a new registry for NTP Reference Identifier codes. It includes the current codes from Section 7.3 and can be extended on a First-Come-First-Serve (FCFS) basis 💜. The registry looks like this:
+------+----------------------------------------------------------+
| ID | Clock Source |
+------+----------------------------------------------------------+
| GOES | Geosynchronous Orbit Environment Satellite |
| GPS | Global Position System |
| ... | ... |
+------+----------------------------------------------------------+
Figure 32: Reference Identifier CodesIANA created another new registry, this time for NTP Kiss-o'-Death codes 🌸. It includes the current codes from Section 7.4 and can be extended on a FCFS basis too. The format is:
+------+------------------------------------------------------------+
| Code | Meaning |
+------+------------------------------------------------------------+
| ACST | The association belongs to a unicast server. |
| AUTH | Server authentication failed. |
| ... | ... |
+------+------------------------------------------------------------+
Figure 33: Kiss CodesFor both Reference Identifiers and Kiss-o'-Death codes, IANA is requested to never assign a code beginning with the character "X". That letter is reserved for experimentation and development, so hands off 🙅♀️✨.
17. Acknowledgements 🫶
Big thank-yous to the editors' technical reviewers and text contributors: Karen O'Donoghue, Brian Haberman, Greg Dowd, Mark Elliot, Harlan Stenn, Yaakov Stein, Stewart Bryant, and Danny Mayer 💖🌟. These people are literally the reason this doc is good, iykyk 👑.
18. References 📚
18.1. Normative References
[RFC0768] Postel, J., "User Datagram Protocol", STD 6, RFC 768,
August 1980.
[RFC0791] Postel, J., "Internet Protocol", STD 5, RFC 791,
September 1981.
[RFC0793] Postel, J., "Transmission Control Protocol", STD 7,
RFC 793, September 1981.
[RFC1321] Rivest, R., "The MD5 Message-Digest Algorithm",
RFC 1321, April 1992.
[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate
Requirement Levels", BCP 14, RFC 2119, March 1997.18.2. Informative References
[CGPM] Bureau International des Poids et Mesures, "Comptes
Rendus de la 15e CGPM", 1976.
[ITU-R_TF.460] International Telecommunications Union, "ITU-R TF.460
Standard-frequency and time-signal emissions",
February 2002.
[RFC1305] Mills, D., "Network Time Protocol (Version 3)
Specification, Implementation and Analysis",
RFC 1305, March 1992.
[RFC1345] Simonsen, K., "Character Mnemonics and Character
Sets", RFC 1345, June 1992.
[RFC4330] Mills, D., "Simple Network Time Protocol (SNTP)
Version 4 for IPv4, IPv6 and OSI", RFC 4330,
January 2006.
[RFC5226] Narten, T. and H. Alvestrand, "Guidelines for Writing
an IANA Considerations Section in RFCs", BCP 26,
RFC 5226, May 2008.
[RFC5906] Haberman, B., Ed. and D. Mills, "Network Time
Protocol Version 4: Autokey Specification", RFC 5906,
June 2010.
[ref6] Marzullo and S. Owicki, "Maintaining the time in a
distributed system", ACM Operating Systems Review 19,
July 1985.
[ref7] Mills, D.L., "Computer Network Time Synchronization -
the Network Time Protocol", CRC Press, 304 pp, 2006.
[ref9] Mills, D.L., Electrical and Computer Engineering
Technical Report 06-6-1, NDSS, June 2006, "Network
Time Protocol Version 4 Reference and Implementation
Guide", 2006.Appendix A. Code Skeleton ✨
okay bestie, so this appendix is basically a code skeleton 🦋 It's a set of definitions, structs, and little code fragments that show how the protocol and algorithms work, without all the messy details of a real implementation. It is NOT an executable. Don't try to compile it and expect magic 💀
Most of the reference implementation's features are in here, except: no reference clock support, no public key (Autokey) cryptography, no huff-n'-puff filter, no anti-clockhop hysteresis, and no monitoring stuff. A bunch of the tunable values from the reference implementation are just hardcoded constants here 🎀 There's also only minimal kiss-o'-death handling, and no code that responds to it.
It's not built to be fast or compact, just to show the algorithms clearly enough that you can understand how they work 🧚 The skeleton has eight segments in total: a header segment that every other segment includes, a code segment for the main program, kernel I/O and system clock interfaces, and then the peer, system, clock_adjust, and poll processes. They show up in that order below, with the definitions and variables each process needs ✨
A.1. Global Definitions
A.1.1. Definitions, Constants, Parameters
#include <math.h> /* avoids complaints about sqrt() */
#include <sys/time.h> /* for gettimeofday() and friends */
#include <stdlib.h> /* for malloc() and friends */
#include <string.h> /* for memset() */
/*
* Data types
*
* This program assumes the int data type is 32 bits and the long data
* type is 64 bits. The native data type used in most calculations is
* floating double. The data types used in some packet header fields
* require conversion to and from this representation. Some header
* fields involve partitioning an octet, here represented by individual
* octets.
*
* The 64-bit NTP timestamp format used in timestamp calculations is
* unsigned seconds and fraction with the decimal point to the left of
* bit 32. The only operation permitted with these values is
* subtraction, yielding a signed 31-bit difference. The 32-bit NTP
* short format used in delay and dispersion calculations is seconds and
* fraction with the decimal point to the left of bit 16. The only
* operations permitted with these values are addition and
* multiplication by a constant.
*
* The IPv4 address is 32 bits, while the IPv6 address is 128 bits. The
* message digest field is 128 bits as constructed by the MD5 algorithm.
* The precision and poll interval fields are signed log2 seconds.
*/
typedef unsigned long long tstamp; /* NTP timestamp format */
typedef unsigned int tdist; /* NTP short format */
typedef unsigned long ipaddr; /* IPv4 or IPv6 address */
typedef unsigned long digest; /* md5 digest */
typedef signed char s_char; /* precision and poll interval (log2) */
/*
* Timestamp conversion macroni
*/
#define FRIC 65536. /* 2^16 as a double */
#define D2FP(r) ((tdist)((r) * FRIC)) /* NTP short */
#define FP2D(r) ((double)(r) / FRIC)
#define FRAC 4294967296. /* 2^32 as a double */
#define D2LFP(a) ((tstamp)((a) * FRAC)) /* NTP timestamp */
#define LFP2D(a) ((double)(a) / FRAC)
#define U2LFP(a) (((unsigned long long) \
((a).tv_sec + JAN_1970) << 32) + \
(unsigned long long) \
((a).tv_usec / 1e6 * FRAC))
/*
* Arithmetic conversions
*/
#define LOG2D(a) ((a) < 0 ? 1. / (1L << -(a)) : \
1L << (a)) /* poll, etc. */
#define SQUARE(x) (x * x)
#define SQRT(x) (sqrt(x))
/*
* Global constants. Some of these might be converted to variables
* that can be tinkered by configuration or computed on-the-fly. For
* instance, the reference implementation computes PRECISION on-the-fly
* and provides performance tuning for the defines marked with % below.
*/
#define VERSION 4 /* version number */
#define MINDISP .01 /* % minimum dispersion (s) */
#define MAXDISP 16 /* maximum dispersion (s) */
#define MAXDIST 1 /* % distance threshold (s) */
#define NOSYNC 0x3 /* leap unsync */
#define MAXSTRAT 16 /* maximum stratum (infinity metric) */
#define MINPOLL 6 /* % minimum poll interval (64 s)*/
#define MAXPOLL 17 /* % maximum poll interval (36.4 h) */
#define MINCLOCK 3 /* minimum manycast survivors */
#define MAXCLOCK 10 /* maximum manycast candidates */
#define TTLMAX 8 /* max ttl manycast */
#define BEACON 15 /* max interval between beacons */
#define PHI 15e-6 /* % frequency tolerance (15 ppm) */
#define NSTAGE 8 /* clock register stages */
#define NMAX 50 /* maximum number of peers */
#define NSANE 1 /* % minimum intersection survivors */
#define NMIN 3 /* % minimum cluster survivors */
/*
* Global return values
*/
#define TRUE 1 /* boolean true */
#define FALSE 0 /* boolean false */
/*
* Local clock process return codes
*/
#define IGNORE 0 /* ignore */
#define SLEW 1 /* slew adjustment */
#define STEP 2 /* step adjustment */
#define PANIC 3 /* panic - no adjustment */
/*
* System flags
*/
#define S_FLAGS 0 /* any system flags */
#define S_BCSTENAB 0x1 /* enable broadcast client */
/*
* Peer flags
*/
#define P_FLAGS 0 /* any peer flags */
#define P_EPHEM 0x01 /* association is ephemeral */
#define P_BURST 0x02 /* burst enable */
#define P_IBURST 0x04 /* intial burst enable */
#define P_NOTRUST 0x08 /* authenticated access */
#define P_NOPEER 0x10 /* authenticated mobilization */
#define P_MANY 0x20 /* manycast client */
/*
* Authentication codes
*/
#define A_NONE 0 /* no authentication */
#define A_OK 1 /* authentication OK */
#define A_ERROR 2 /* authentication error */
#define A_CRYPTO 3 /* crypto-NAK */
/*
* Association state codes
*/
#define X_INIT 0 /* initialization */
#define X_STALE 1 /* timeout */
#define X_STEP 2 /* time step */
#define X_ERROR 3 /* authentication error */
#define X_CRYPTO 4 /* crypto-NAK received */
#define X_NKEY 5 /* untrusted key */
/*
* Protocol mode definitions
*/
#define M_RSVD 0 /* reserved */
#define M_SACT 1 /* symmetric active */
#define M_PASV 2 /* symmetric passive */
#define M_CLNT 3 /* client */
#define M_SERV 4 /* server */
#define M_BCST 5 /* broadcast server */
#define M_BCLN 6 /* broadcast client */
/*
* Clock state definitions
*/
#define NSET 0 /* clock never set */
#define FSET 1 /* frequency set from file */
#define SPIK 2 /* spike detected */
#define FREQ 3 /* frequency mode */
#define SYNC 4 /* clock synchronized */
#define min(a, b) ((a) < (b) ? (a) : (b))
#define max(a, b) ((a) < (b) ? (b) : (a))A.1.2. Packet Data Structures
okay so this part is where the packet layouts get defined 🎀 The receive and transmit packets can carry an optional message authentication code (MAC), made of a key identifier (keyid) plus a message digest (mac in the receive struct, dgst in the transmit struct). NTPv4 also supports optional extension fields that sit after the header and before the MAC, but those aren't described here 💅
Quick note for the receive side: the dst timestamp isn't part of the packet itself. It gets captured when the packet arrives and is handed back in the receive buffer along with the buffer length and data 🦋 Also, some of the char fields are actually packed together in the real header, but the packing details are left out here 🌸
/*
* The receive and transmit packets may contain an optional message
* authentication code (MAC) consisting of a key identifier (keyid) and
* message digest (mac in the receive structure and dgst in the transmit
* structure). NTPv4 supports optional extension fields that
* are inserted after the header and before the MAC, but these are
* not described here.
*
* Receive packet
*
* Note the dst timestamp is not part of the packet itself. It is
* captured upon arrival and returned in the receive buffer along with
* the buffer length and data. Note that some of the char fields are
* packed in the actual header, but the details are omitted here.
*/
struct r {
ipaddr srcaddr; /* source (remote) address */
ipaddr dstaddr; /* destination (local) address */
char version; /* version number */
char leap; /* leap indicator */
char mode; /* mode */
char stratum; /* stratum */
char poll; /* poll interval */
s_char precision; /* precision */
tdist rootdelay; /* root delay */
tdist rootdisp; /* root dispersion */
char refid; /* reference ID */
tstamp reftime; /* reference time */
tstamp org; /* origin timestamp */
tstamp rec; /* receive timestamp */
tstamp xmt; /* transmit timestamp */
int keyid; /* key ID */
digest mac; /* message digest */
tstamp dst; /* destination timestamp */
} r;
/*
* Transmit packet
*/
struct x {
ipaddr dstaddr; /* source (local) address */
ipaddr srcaddr; /* destination (remote) address */
char version; /* version number */
char leap; /* leap indicator */
char mode; /* mode */
char stratum; /* stratum */
char poll; /* poll interval */
s_char precision; /* precision */
tdist rootdelay; /* root delay */
tdist rootdisp; /* root dispersion */
char refid; /* reference ID */
tstamp reftime; /* reference time */
tstamp org; /* origin timestamp */
tstamp rec; /* receive timestamp */
tstamp xmt; /* transmit timestamp */
int keyid; /* key ID */
digest dgst; /* message digest */
} x;A.1.3. Association Data Structures ✨
okay so this is where the association stuff lives 🧚 Heads up bestie: the t member in these structs refers to process time, not real time. Process time goes up by one second for every elapsed second of real time 🕰️
/*
* Filter stage structure. Note the t member in this and other
* structures refers to process time, not real time. Process time
* increments by one second for every elapsed second of real time.
*/
struct f {
tstamp t; /* update time */
double offset; /* clock ofset */
double delay; /* roundtrip delay */
double disp; /* dispersion */
} f;
/*
* Association structure. This is shared between the peer process
* and poll process.
*/
struct p {
/*
* Variables set by configuration
*/
ipaddr srcaddr; /* source (remote) address */
ipaddr dstaddr; /* destination (local) address */
char version; /* version number */
char hmode; /* host mode */
int keyid; /* key identifier */
int flags; /* option flags */
/*
* Variables set by received packet
*/
char leap; /* leap indicator */
char pmode; /* peer mode */
char stratum; /* stratum */
char ppoll; /* peer poll interval */
double rootdelay; /* root delay */
double rootdisp; /* root dispersion */
char refid; /* reference ID */
tstamp reftime; /* reference time */
#define begin_clear org /* beginning of clear area */
tstamp org; /* originate timestamp */
tstamp rec; /* receive timestamp */
tstamp xmt; /* transmit timestamp */
/*
* Computed data
*/
double t; /* update time */
struct f f[NSTAGE]; /* clock filter */
double offset; /* peer offset */
double delay; /* peer delay */
double disp; /* peer dispersion */
double jitter; /* RMS jitter */
/*
* Poll process variables
*/
char hpoll; /* host poll interval */
int burst; /* burst counter */
int reach; /* reach register */
int ttl; /* ttl (manycast) */
#define end_clear unreach /* end of clear area */
int unreach; /* unreach counter */
int outdate; /* last poll time */
int nextdate; /* next poll time */
} p;A.1.4. System Data Structures 💅
this part covers the chime list, which the intersection algorithm uses, and the survivor list, which the clustering algorithm uses. Then we get the system structure itself, the main boss 👑
/*
* Chime list. This is used by the intersection algorithm.
*/
struct m { /* m is for Marzullo */
struct p *p; /* peer structure pointer */
int type; /* high +1, mid 0, low -1 */
double edge; /* correctness interval edge */
} m;
/*
* Survivor list. This is used by the clustering algorithm.
*/
struct v {
struct p *p; /* peer structure pointer */
double metric; /* sort metric */
} v;
/*
* System structure
*/
struct s {
tstamp t; /* update time */
char leap; /* leap indicator */
char stratum; /* stratum */
char poll; /* poll interval */
char precision; /* precision */
double rootdelay; /* root delay */
double rootdisp; /* root dispersion */
char refid; /* reference ID */
tstamp reftime; /* reference time */
struct m m[NMAX]; /* chime list */
struct v v[NMAX]; /* survivor list */
struct p *p; /* association ID */
double offset; /* combined offset */
double jitter; /* combined jitter */
int flags; /* option flags */
int n; /* number of survivors */
} s;A.1.5. Local Clock Data Structures 🎀
and here's the local clock struct, which keeps track of the clock's state, offset, frequency, and jitter and wander 🌈
/*
* Local clock structure
*/
struct c {
tstamp t; /* update time */
int state; /* current state */
double offset; /* current offset */
double last; /* previous offset */
int count; /* jiggle counter */
double freq; /* frequency */
double jitter; /* RMS jitter */
double wander; /* RMS wander */
} c;A.1.6. Function Prototypes 🦋
okay so this is the full lineup of function prototypes, grouped by process. It's giving organized, no cap 💖
/*
* Peer process
*/
void receive(struct r *); /* receive packet */
void packet(struct p *, struct r *); /* process packet */
void clock_filter(struct p *, double, double, double); /* filter */
double root_dist(struct p *); /* calculate root distance */
int fit(struct p *); /* determine fitness of server */
void clear(struct p *, int); /* clear association */
int access(struct r *); /* determine access restrictions */
/*
* System process
*/
int main(); /* main program */
void clock_select(); /* find the best clocks */
void clock_update(struct p *); /* update the system clock */
void clock_combine(); /* combine the offsets */
/*
* Local clock process
*/
int local_clock(struct p *, double); /* clock discipline */
void rstclock(int, double, double); /* clock state transition */
/*
* Clock adjust process
*/
void clock_adjust(); /* one-second timer process */
/*
* Poll process
*/
void poll(struct p *); /* poll process */
void poll_update(struct p *, int); /* update the poll interval */
void peer_xmit(struct p *); /* transmit a packet */
void fast_xmit(struct r *, int, int); /* transmit a reply packet */
/*
* Utility routines
*/
digest md5(int); /* generate a message digest */
struct p *mobilize(ipaddr, ipaddr, int, int, int, int); /* mobilize */
struct p *find_assoc(struct r *); /* search the association table */
/*
* Kernel interface
*/
struct r *recv_packet(); /* wait for packet */
void xmit_packet(struct x *); /* send packet */
void step_time(double); /* step time */
void adjust_time(double); /* adjust (slew) time */
tstamp get_time(); /* read time */A.2. Main Program and Utility Routines 👑
here's the main program and its helpers, starting with the definitions. The main() function sets up the system variables, then the local clock, then mobilizes the configured associations, and then loops over incoming packets 🔁
/*
* Definitions
*/
#define PRECISION -18 /* precision (log2 s) */
#define IPADDR 0 /* any IP address */
#define MODE 0 /* any NTP mode */
#define KEYID 0 /* any key identifier */
/*
* main() - main program
*/
int
main()
{
struct p *p; /* peer structure pointer */
struct r *r; /* receive packet pointer */
/*
* Read command line options and initialize system variables.
* The reference implementation measures the precision specific
* to each machine by measuring the clock increments to read the
* system clock.
*/
memset(&s, sizeof(s), 0);
s.leap = NOSYNC;
s.stratum = MAXSTRAT;
s.poll = MINPOLL;
s.precision = PRECISION;
s.p = NULL;
/*
* Initialize local clock variables
*/
memset(&c, sizeof(c), 0);
if (/* frequency file */ 0) {
c.freq = /* freq */ 0;
rstclock(FSET, 0, 0);
} else {
rstclock(NSET, 0, 0);
}
c.jitter = LOG2D(s.precision);
/*
* Read the configuration file and mobilize persistent
* associations with specified addresses, version, mode, key ID,
* and flags.
*/
while (/* mobilize configurated associations */ 0) {
p = mobilize(IPADDR, IPADDR, VERSION, MODE, KEYID,
P_FLAGS);
}
/*
* Start the system timer, which ticks once per second. Then,
* read packets as they arrive, strike receive timestamp, and
* call the receive() routine.
*/
while (0) {
r = recv_packet();
r->dst = get_time();
receive(r);
}
return(0);
}the mobilize() function is where an association gets born ✨ It allocates the memory, fills in the fields, and then clears the association:
/*
* mobilize() - mobilize and initialize an association
*/
struct p
*mobilize(
ipaddr srcaddr, /* IP source address */
ipaddr dstaddr, /* IP destination address */
int version, /* version */
int mode, /* host mode */
int keyid, /* key identifier */
int flags /* peer flags */
)
{
struct p *p; /* peer process pointer */
/*
* Allocate and initialize association memory
*/
p = malloc(sizeof(struct p));
p->srcaddr = srcaddr;
p->dstaddr = dstaddr;
p->version = version;
p->hmode = mode;
p->keyid = keyid;
p->hpoll = MINPOLL;
clear(p, X_INIT);
p->flags = flags;
return (p);
}find_assoc() is the search function. It goes through the association table looking for a match on source address and mode, and returns NULL if nothing matches 🔍
/*
* find_assoc() - find a matching association
*/
struct p /* peer structure pointer or NULL */
*find_assoc(
struct r *r /* receive packet pointer */
)
{
struct p *p; /* dummy peer structure pointer */
/*
* Search association table for matching source
* address, source port and mode.
*/
while (/* all associations */ 0) {
if (r->srcaddr == p->srcaddr && r->mode == p->hmode)
return(p);
}
return (NULL);
}and last but not least, md5(), which computes the keyed digest. The MAC it returns is the 32-bit key ID followed by the 128-bit digest 🔐
/*
* md5() - compute message digest
*/
digest
md5(
int keyid /* key identifier */
)
{
/*
* Compute a keyed cryptographic message digest. The key
* identifier is associated with a key in the local key cache.
* The key is prepended to the packet header and extension fields
* and the result hashed by the MD5 algorithm as described in
* RFC 1321. Return a MAC consisting of the 32-bit key ID
* concatenated with the 128-bit digest.
*/
return (/* MD5 digest */ 0);
}A.3. Kernel Input/Output Interface 🌸
this is the kernel interface for sending and receiving packets. The details are deliberately vague and depend on the operating system, so it's a placeholder vibe 🙈
/*
* Kernel interface to transmit and receive packets. Details are
* deliberately vague and depend on the operating system.
*
* recv_packet - receive packet from network
*/
struct r /* receive packet pointer*/
*recv_packet() {
return (/* receive packet r */ 0);
}
/*
* xmit_packet - transmit packet to network
*/
void
xmit_packet(
struct x *x /* transmit packet pointer */
)
{
/* send packet x */
}A.4. Kernel System Clock Interface ⭐
last stop, the system clock utilities 🕐 There are three time formats in play: native (Unix), NTP, and floating double. The get_time() routine hands back NTP long format, and the Unix routines take their arguments as signed 32-bit words in seconds and microseconds (timeval) or nanoseconds (timespec). step_time() and adjust_time() take signed floating doubles. Also, the simplified code is only for illustration and hasn't been verified, so don't ship it as-is 💜
/*
* System clock utility functions
*
* There are three time formats: native (Unix), NTP, and floating
* double. The get_time() routine returns the time in NTP long format.
* The Unix routines expect arguments as a structure of two signed
* 32-bit words in seconds and microseconds (timeval) or nanoseconds
* (timespec). The step_time() and adjust_time() routines expect signed
* arguments in floating double. The simplified code shown here is for
* illustration only and has not been verified.
*/
#define JAN_1970 2208988800UL /* 1970 - 1900 in seconds */
/*
* get_time - read system time and convert to NTP format
*/
tstamp
get_time()
{ struct timeval unix_time;
/*
- There are only two calls on this routine in the program. One
- when a packet arrives from the network and the other when a
- packet is placed on the send queue. Call the kernel time of
- day routine (such as gettimeofday()) and convert to NTP
- format. / gettimeofday(&unix_time, NULL); return (U2LFP(unix_time)); } /
- step_time() - step system time to given offset value / void step_time( double offset / clock offset */ ) { struct timeval unix_time; tstamp ntp_time;
/*
- Convert from double to native format (signed) and add to the
- current time. Note the addition is done in native format to
- avoid overflow or loss of precision. */ gettimeofday(&unix_time, NULL); ntp_time = D2LFP(offset) + U2LFP(unix_time); unix_time.tv_sec = ntp_time >> 32; unix_time.tv_usec = (long)(((ntp_time - unix_time.tv_sec) <<
- / FRAC * 1e6); settimeofday(&unix_time, NULL); }
/*
- adjust_time() - slew system clock to given offset value / void adjust_time( double offset / clock offset */ ) { struct timeval unix_time; tstamp ntp_time;
/*
- Convert from double to native format (signed) and add to the
- current time. */ ntp_time = D2LFP(offset); unix_time.tv_sec = ntp_time >> 32; unix_time.tv_usec = (long)(((ntp_time - unix_time.tv_sec) <<
- / FRAC * 1e6); adjtime(&unix_time, NULL); }
<pink>okay so the clock stuff is wrapped up 🕰️✨</pink> <lavender>step_time just yeets the clock to a new spot, and adjust_time eases it there gently 🫶</lavender> <mint>now we're entering the peer process era 💅💜</mint>
## A.5. Peer Process
/*
- A crypto-NAK packet includes the NTP header followed by a MAC
- consisting only of the key identifier with value zero. It tells
- the receiver that a prior request could not be properly
- authenticated, but the NTP header fields are correct. *
- A kiss-o'-death packet is an NTP header with leap 0x3 (NOSYNC) and
- stratum 16 (MAXSTRAT). It tells the receiver that something
- drastic has happened, as revealed by the kiss code in the refid
- field. The NTP header fields may or may not be correct. / /
- Peer process parameters and constants / #define SGATE 3 / spike gate (clock filter / #define BDELAY .004 / broadcast delay (s) */
/*
- Dispatch codes / #define ERR -1 / error / #define DSCRD 0 / discard packet / #define PROC 1 / process packet / #define BCST 2 / broadcast packet / #define FXMIT 3 / client packet / #define MANY 4 / manycast packet / #define NEWPS 5 / new symmetric passive client / #define NEWBC 6 / new broadcast client */
/*
- Dispatch matrix
- active passv client server bcast / int table[7][5] = { / nopeer / { NEWPS, DSCRD, FXMIT, MANY, NEWBC }, / active / { PROC, PROC, DSCRD, DSCRD, DSCRD }, / passv / { PROC, ERR, DSCRD, DSCRD, DSCRD }, / client / { DSCRD, DSCRD, DSCRD, PROC, DSCRD }, / server / { DSCRD, DSCRD, DSCRD, DSCRD, DSCRD }, / bcast / { DSCRD, DSCRD, DSCRD, DSCRD, DSCRD }, / bclient / { DSCRD, DSCRD, DSCRD, DSCRD, PROC} }; /
- Miscellaneous macroni *
- This macro defines the authentication state. If x is 0,
- authentication is optional; otherwise, it is required. */ #define AUTH(x, y) ((x) ? (y) == A_OK : (y) == A_OK || \ (y) == A_NONE)
/*
- These are used by the clear() routine */ #define BEGIN_CLEAR(p) ((char *)&((p)->begin_clear)) #define END_CLEAR(p) ((char *)&((p)->end_clear)) #define LEN_CLEAR (END_CLEAR((struct p *)0) - \ BEGIN_CLEAR((struct p *)0))
### A.5.1. receive()
/*
- receive() - receive packet and decode modes / void receive( struct r *r / receive packet pointer */ )
<pink>okay bestie, the receive() party is still going</pink> 💖 <lavender>the access checks, the authentication outcomes, and the big timestamp gauntlet all keep stacking up in here</lavender> 💅
{ struct p p; / peer structure pointer / int auth; / authentication code / int has_mac; / size of MAC / int synch; / synchronized switch */
/*
- Check access control lists. The intent here is to implement
- a whitelist of those IP addresses specifically accepted
- and/or a blacklist of those IP addresses specifically
- rejected. There could be different lists for authenticated
- clients and unauthenticated clients. / if (!access(r)) return; / access denied */
/*
- The version must not be in the future. Format checks include
- packet length, MAC length and extension field lengths, if
- present. */
if (r->version > VERSION /* or format error /) return; / format error */
/*
- Authentication is conditioned by two switches that can be
- specified on a per-client basis. *
- P_NOPEER do not mobilize an association unless
- authenticated.
- P_NOTRUST do not allow access unless authenticated
- (implies P_NOPEER). *
- There are four outcomes: *
- A_NONE the packet has no MAC.
- A_OK the packet has a MAC and authentication
- succeeds.
- A_ERROR the packet has a MAC and authentication fails.
- A_CRYPTO crypto-NAK. The MAC has four octets only. *
- Note: The AUTH (x, y) macro is used to filter outcomes. If x
- is zero, acceptable outcomes of y are NONE and OK. If x is
- one, the only acceptable outcome of y is OK. */
has_mac = /* length of MAC field / 0; if (has_mac == 0) { auth = A_NONE; / not required / } else if (has_mac == 4) { auth = A_CRYPTO; / crypto-NAK / } else { if (r->mac != md5(r->keyid)) auth = A_ERROR; / auth error / else auth = A_OK; / auth OK */ }
/*
- Find association and dispatch code. If there is no
- association to match, the value of p->hmode is assumed NULL. / p = find_assoc(r); switch(table[(unsigned int)(p->hmode)][(unsigned int)(r->mode)]) { /
- Client packet and no association. Send server reply without
- saving state. */ case FXMIT:
/*
- If unicast destination address, send server packet.
- If authentication fails, send a crypto-NAK packet. */
/* not multicast dstaddr / if (0) { if (AUTH(p->flags & P_NOTRUST, auth)) fast_xmit(r, M_SERV, auth); else if (auth == A_ERROR) fast_xmit(r, M_SERV, A_CRYPTO); return; / M_SERV packet sent */ }
/*
- This must be manycast. Do not respond if we are not
- synchronized or if our stratum is above the
- manycaster. */ if (s.leap == NOSYNC || s.stratum > r->stratum) return;
/*
- Respond only if authentication is OK. Note that the
- unicast address is used, not the multicast. */ if (AUTH(p->flags & P_NOTRUST, auth)) fast_xmit(r, M_SERV, auth); return;
/*
- New manycast client ephemeral association. It is mobilized
- in the same version as in the packet. If authentication
- fails, ignore the packet. Verify the server packet by
- comparing the r->org timestamp in the packet with the p->xmt
- timestamp in the multicast client association. If they
- match, the server packet is authentic. Details omitted. */
case MANY: if (!AUTH(p->flags & (P_NOTRUST | P_NOPEER), auth)) return; /* authentication error */
p = mobilize(r->srcaddr, r->dstaddr, r->version, M_CLNT, r->keyid, P_EPHEM); break;
/*
- New symmetric passive association. It is mobilized in the
- same version as in the packet. If authentication fails,
- send a crypto-NAK packet. If restrict no-moblize, send a
- symmetric active packet instead. / case NEWPS: if (!AUTH(p->flags & P_NOTRUST, auth)) { if (auth == A_ERROR) fast_xmit(r, M_SACT, A_CRYPTO); return; / crypto-NAK packet sent / } if (!AUTH(p->flags & P_NOPEER, auth)) { fast_xmit(r, M_SACT, auth); return; / M_SACT packet sent */ } p = mobilize(r->srcaddr, r->dstaddr, r->version, M_PASV, r->keyid, P_EPHEM); break;
/*
- New broadcast client association. It is mobilized in the
- same version as in the packet. If authentication fails,
- ignore the packet. Note this code does not support the
- initial volley feature in the reference implementation. / case NEWBC: if (!AUTH(p->flags & (P_NOTRUST | P_NOPEER), auth)) return; / authentication error */
if (!(s.flags & S_BCSTENAB)) return; /* broadcast not enabled */
p = mobilize(r->srcaddr, r->dstaddr, r->version, M_BCLN, r->keyid, P_EPHEM); break; /* processing continues */
/*
- Process packet. Placeholdler only. / case PROC: break; / processing continues */
/*
- Invalid mode combination. We get here only in case of
- ephemeral associations, so the correct action is simply to
- toss it. / case ERR: clear(p, X_ERROR); return; / invalid mode combination */
/*
- No match; just discard the packet. / case DSCRD: return; / orphan abandoned */ }
/*
- Next comes a rigorous schedule of timestamp checking. If the
- transmit timestamp is zero, the server is horribly broken. / if (r->xmt == 0) return; / invalid timestamp */
/*
- If the transmit timestamp duplicates a previous one, the
- packet is a replay. / if (r->xmt == p->xmt) return; / duplicate packet */
/*
- If this is a broadcast mode packet, skip further checking.
- If the origin timestamp is zero, the sender has not yet heard
- from us. Otherwise, if the origin timestamp does not match
- the transmit timestamp, the packet is bogus. */
synch = TRUE; if (r->mode != M_BCST) { if (r->org == 0) synch = FALSE; /* unsynchronized */
else if (r->org != p->xmt) synch = FALSE; /* bogus packet */ }
/*
- Update the origin and destination timestamps. If
- unsynchronized or bogus, abandon ship. / p->org = r->xmt; p->rec = r->dst; if (!synch) return; / unsynch */
/*
- The timestamps are valid and the receive packet matches the
- last one sent. If the packet is a crypto-NAK, the server
- might have just changed keys. We demobilize the association
- and wait for better times. / if (auth == A_CRYPTO) { clear(p, X_CRYPTO); return; / crypto-NAK */ }
/*
- If the association is authenticated, the key ID is nonzero
- and received packets must be authenticated. This is designed
- to avoid a bait-and-switch attack, which was possible in past
- versions. / if (!AUTH(p->keyid || (p->flags & P_NOTRUST), auth)) return; / bad auth */
/*
- Everything possible has been done to validate the timestamps
- and prevent bad guys from disrupting the protocol or
- injecting bogus data. Earn some revenue. */ packet(p, r); }
### A.5.1.1. packet() 🦋
<mint>next up is packet(), the function that actually does the offset, delay, and dispersion math</mint> ✨
/*
- packet() - process packet and compute offset, delay, and
- dispersion. / void packet( struct p *p, / peer structure pointer / struct r *r / receive packet pointer */ )
{ double offset; /* sample offsset / double delay; / sample delay / double disp; / sample dispersion */
/*
- By golly the packet is valid. Light up the remaining header
- fields. Note that we map stratum 0 (unspecified) to MAXSTRAT
- to make stratum comparisons simpler and to provide a natural
- interface for radio clock drivers that operate for
- convenience at stratum 0. */ p->leap = r->leap; if (r->stratum == 0) p->stratum = MAXSTRAT; else p->stratum = r->stratum; p->pmode = r->mode; p->ppoll = r->poll; p->rootdelay = FP2D(r->rootdelay); p->rootdisp = FP2D(r->rootdisp); p->refid = r->refid; p->reftime = r->reftime;
/*
- Verify the server is synchronized with valid stratum and
- reference time not later than the transmit time. */
if (p->leap == NOSYNC || p->stratum >= MAXSTRAT) return; /* unsynchronized */
/*
- Verify valid root distance. / if (r->rootdelay / 2 + r->rootdisp >= MAXDISP || p->reftime > r->xmt) return; / invalid header values */
poll_update(p, p->hpoll); p->reach |= 1;
/*
- Calculate offset, delay and dispersion, then pass to the
- clock filter. Note carefully the implied processing. The
- first-order difference is done directly in 64-bit arithmetic,
- then the result is converted to floating double. All further
- processing is in floating-double arithmetic with rounding
- done by the hardware. This is necessary in order to avoid
- overflow and preserve precision. *
- The delay calculation is a special case. In cases where the
- server and client clocks are running at different rates and
- with very fast networks, the delay can appear negative. In
- order to avoid violating the Principle of Least Astonishment,
- the delay is clamped not less than the system precision. */ if (p->pmode == M_BCST) { offset = LFP2D(r->xmt - r->dst); delay = BDELAY; disp = LOG2D(r->precision) + LOG2D(s.precision) + PHI * 2 * BDELAY; } else { offset = (LFP2D(r->rec - r->org) + LFP2D(r->dst - r->xmt)) / 2; delay = max(LFP2D(r->dst - r->org) - LFP2D(r->rec - r->xmt), LOG2D(s.precision)); disp = LOG2D(r->precision) + LOG2D(s.precision) + PHI * LFP2D(r->dst - r->org); } clock_filter(p, offset, delay, disp); }
### A.5.2. clock_filter() 🎀
Okay so this one is the filter that picks the best of your last eight samples. 💅 It's giving "keep the good ones, toss the spiky ones" energy.
/*
- clock_filter(p, offset, delay, dispersion) - select the best from the
- latest eight delay/offset samples. / void clock_filter( struct p *p, / peer structure pointer / double offset, / clock offset / double delay, / roundtrip delay / double disp / dispersion / ) { struct f f[NSTAGE]; / sorted list */ double dtemp; int i;
/*
- The clock filter contents consist of eight tuples (offset,
- delay, dispersion, time). Shift each tuple to the left,
- discarding the leftmost one. As each tuple is shifted,
- increase the dispersion since the last filter update. At the
- same time, copy each tuple to a temporary list. After this,
- place the (offset, delay, disp, time) in the vacated
- rightmost tuple. */ for (i = 1; i < NSTAGE; i++) { p->f[i] = p->f[i - 1]; p->f[i].disp += PHI * (c.t - p->t); f[i] = p->f[i]; } p->f[0].t = c.t; p->f[0].offset = offset; p->f[0].delay = delay; p->f[0].disp = disp; f[0] = p->f[0];
/*
- Sort the temporary list of tuples by increasing f[].delay.
- The first entry on the sorted list represents the best
- sample, but it might be old. */ dtemp = p->offset; p->offset = f[0].offset; p->delay = f[0].delay; for (i = 0; i < NSTAGE; i++) { p->disp += f[i].disp / (2 ^ (i + 1)); p->jitter += SQUARE(f[i].offset - f[0].offset); } p->jitter = max(SQRT(p->jitter), LOG2D(s.precision));
/*
- Prime directive: use a sample only once and never a sample
- older than the latest one, but anything goes before first
- synchronized. */ if (f[0].t - p->t <= 0 && s.leap != NOSYNC) return;
/*
- Popcorn spike suppressor. Compare the difference between the
- last and current offsets to the current jitter. If greater
- than SGATE (3) and if the interval since the last offset is
- less than twice the system poll interval, dump the spike.
- Otherwise, and if not in a burst, shake out the truechimers. */ if (fabs(p->offset - dtemp) > SGATE * p->jitter && (f[0].t - p->t) < 2 * s.poll) return;
p->t = f[0].t; if (p->burst == 0) clock_select(); return; }
/*
- fit() - test if association p is acceptable for synchronization / int fit( struct p *p / peer structure pointer / ) { /
- A stratum error occurs if (1) the server has never been
- synchronized, (2) the server stratum is invalid. */ if (p->leap == NOSYNC || p->stratum >= MAXSTRAT) return (FALSE);
/*
- A distance error occurs if the root distance exceeds the
- distance threshold plus an increment equal to one poll
- interval. */ if (root_dist(p) > MAXDIST + PHI * LOG2D(s.poll)) return (FALSE);
/*
- A loop error occurs if the remote peer is synchronized to the
- local peer or the remote peer is synchronized to the current
- system peer. Note this is the behavior for IPv4; for IPv6
- the MD5 hash is used instead. */ if (p->refid == p->dstaddr || p->refid == s.refid) return (FALSE);
/*
- An unreachable error occurs if the server is unreachable. */ if (p->reach == 0) return (FALSE);
return (TRUE); }
/*
- clear() - reinitialize for persistent association, demobilize
- for ephemeral association. / void clear( struct p *p, / peer structure pointer / int kiss / kiss code */ ) { int i;
/*
- The first thing to do is return all resources to the bank.
- Typical resources are not detailed here, but they include
- dynamically allocated structures for keys, certificates, etc.
- If an ephemeral association and not initialization, return
- the association memory as well. / / return resources */ if (s.p == p) s.p = NULL;
if (kiss != X_INIT && (p->flags & P_EPHEM)) { free(p); return; }
/*
- Initialize the association fields for general reset. */ memset(BEGIN_CLEAR(p), LEN_CLEAR, 0); p->leap = NOSYNC; p->stratum = MAXSTRAT; p->ppoll = MAXPOLL; p->hpoll = MINPOLL; p->disp = MAXDISP; p->jitter = LOG2D(s.precision); p->refid = kiss; for (i = 0; i < NSTAGE; i++) p->f[i].disp = MAXDISP;
/*
- Randomize the first poll just in case thousands of broadcast
- clients have just been stirred up after a long absence of the
- broadcast server. */ p->outdate = p->t = c.t; p->nextdate = p->outdate + (random() & ((1 << MINPOLL) - 1)); }
### A.5.3. fast_xmit() ✨
This is the reply maker. 🦋 It stamps the packet with our info and sends it back to whoever asked.
/*
- fast_xmit() - transmit a reply packet for receive packet r / void fast_xmit( struct r *r, / receive packet pointer / int mode, / association mode / int auth / authentication code */ ) { struct x x;
/*
- Initialize header and transmit timestamp. Note that the
- transmit version is copied from the receive version. This is
- for backward compatibility. */
x.version = r->version; x.srcaddr = r->dstaddr; x.dstaddr = r->srcaddr; x.leap = s.leap; x.mode = mode; if (s.stratum == MAXSTRAT) x.stratum = 0; else x.stratum = s.stratum; x.poll = r->poll; x.precision = s.precision; x.rootdelay = D2FP(s.rootdelay); x.rootdisp = D2FP(s.rootdisp); x.refid = s.refid; x.reftime = s.reftime; x.org = r->xmt; x.rec = r->dst; x.xmt = get_time();
/*
- If the authentication code is A.NONE, include only the
- header; if A.CRYPTO, send a crypto-NAK; if A.OK, send a valid
- MAC. Use the key ID in the received packet and the key in
- the local key cache. */ if (auth != A_NONE) { if (auth == A_CRYPTO) { x.keyid = 0; } else { x.keyid = r->keyid; x.dgst = md5(x.keyid); } } xmit_packet(&x); }
### A.5.4. access() 🌸
Access control is the bouncer. 💅 Its list is checked in order, and the first matching source address decides which restrict word comes back.
/*
- access() - determine access restrictions / int access( struct r *r / receive packet pointer */ )
{ /*
- The access control list is an ordered set of tuples
- consisting of an address, mask, and restrict word containing
- defined bits. The list is searched for the first match on
- the source address (r->srcaddr) and the associated restrict
- word is returned. / return (/ access bits */ 0); }
## A.5.5. System Process 🧚
This is where the big decisions happen. 👑 The clock selection is the main event, so let's get into it.
### A.5.5.1. clock_select() 💜
Okay bestie, here's the function that goes looking for the best clocks. 🔥
/*
- clock_select() - find the best clocks */ void
<rainbow>Part 20 done, more to come! ✨🌈</rainbow>
clock_select() {
struct p *p, *osys; /* peer structure pointers */
double low, high; /* correctness interval extents */
int allow, found, chime; /* used by intersection algorithm */
int n, i, j;
/*
* We first cull the falsetickers from the server population,
* leaving only the truechimers. The correctness interval for
* association p is the interval from offset - root_dist() to
* offset + root_dist(). The object of the game is to find a
* majority clique; that is, an intersection of correctness
* intervals numbering more than half the server population.
*
* First, construct the chime list of tuples (p, type, edge) as
* shown below, then sort the list by edge from lowest to
* highest.
*/
osys = s.p;
s.p = NULL;
n = 0;
while (fit(p)) {
s.m[n].p = p;
s.m[n].type = +1;
s.m[n].edge = p->offset + root_dist(p);
n++;
s.m[n].p = p;
s.m[n].type = 0;
s.m[n].edge = p->offset;
n++;
s.m[n].p = p;
s.m[n].type = -1;
s.m[n].edge = p->offset - root_dist(p);
n++;
}
/*
* Find the largest contiguous intersection of correctness
* intervals. Allow is the number of allowed falsetickers;
* found is the number of midpoints. Note that the edge values
* are limited to the range +-(2 ^ 30) < +-2e9 by the timestamp
* calculations.
*/
low = 2e9; high = -2e9;
for (allow = 0; 2 * allow < n; allow++) {
/*
* Scan the chime list from lowest to highest to find
* the lower endpoint.
*/
found = 0;
chime = 0;
for (i = 0; i < n; i++) {
chime -= s.m[i].type;
if (chime >= n - found) {
low = s.m[i].edge;
break;
}
if (s.m[i].type == 0)
found++;
}
/*
* Scan the chime list from highest to lowest to find
* the upper endpoint.
*/
chime = 0;
for (i = n - 1; i >= 0; i--) {
chime += s.m[i].type;
if (chime >= n - found) {
high = s.m[i].edge;
break;
}
if (s.m[i].type == 0)
found++;
}
/*
* If the number of midpoints is greater than the number
* of allowed falsetickers, the intersection contains at
* least one truechimer with no midpoint. If so,
* increment the number of allowed falsetickers and go
* around again. If not and the intersection is
* non-empty, declare success.
*/
if (found > allow)
continue;
if (high > low)
break;
}
/*
* Clustering algorithm. Construct a list of survivors (p,
* metric) from the chime list, where metric is dominated first
* by stratum and then by root distance. All other things being
* equal, this is the order of preference.
*/
s.n = 0;
for (i = 0; i < n; i++) {
if (s.m[i].edge < low || s.m[i].edge > high)
continue;
p = s.m[i].p;
s.v[n].p = p;
s.v[n].metric = MAXDIST * p->stratum + root_dist(p);
s.n++;
}
/*
* There must be at least NSANE survivors to satisfy the
* correctness assertions. Ordinarily, the Byzantine criteria
* require four survivors, but for the demonstration here, one
* is acceptable.
*/
if (s.n < NSANE)
return;
/*
* For each association p in turn, calculate the selection
* jitter p->sjitter as the square root of the sum of squares
* (p->offset - q->offset) over all q associations. The idea is
* to repeatedly discard the survivor with maximum selection
* jitter until a termination condition is met.
*/
while (1) {
struct p *p, *q, *qmax; /* peer structure pointers */
double max, min, dtemp;
max = -2e9; min = 2e9;
for (i = 0; i < s.n; i++) {
p = s.v[i].p;
if (p->jitter < min)
min = p->jitter;
dtemp = 0;
for (j = 0; j < n; j++) {
q = s.v[j].p;
dtemp += SQUARE(p->offset - q->offset);
}
dtemp = SQRT(dtemp);
if (dtemp > max) {
max = dtemp;
qmax = q;
}
}
/*
* If the maximum selection jitter is less than the
* minimum peer jitter, then tossing out more survivors
* will not lower the minimum peer jitter, so we might
* as well stop. To make sure a few survivors are left
* for the clustering algorithm to chew on, we also stop
* if the number of survivors is less than or equal to
* NMIN (3).
*/
if (max < min || n <= NMIN)
break;
/*
* Delete survivor qmax from the list and go around
* again.
*/
s.n--;
}
/*
* Pick the best clock. If the old system peer is on the list
* and at the same stratum as the first survivor on the list,
* then don't do a clock hop. Otherwise, select the first
* survivor on the list as the new system peer.
*/
if (osys->stratum == s.v[0].p->stratum)
s.p = osys;
else
s.p = s.v[0].p;
clock_update(s.p);
}
Okay so this is the big one, bestie 💅✨ `clock_select()` is basically the bouncer at the club 🪩. It goes through every peer and kicks out the **falsetickers** (the liars 🙅♀️) so only the **truechimers** get in. Each peer gets a little correctness interval, and the function hunts for the biggest overlap that more than half the crew agrees on. Main character energy for the honest clocks 👑
The chime list is built as tuples of `(p, type, edge)`, sorted by edge, and then the function loops with a rising `allow` count to find the intersection. Lowkey the most iconic part of this whole thing 🫶
After that, the survivors get ranked by stratum first and root distance second. If there aren't enough survivors (`NSANE`), it just bails 😭. Then it keeps tossing out whichever survivor has the worst selection jitter, but stops once that won't help or there are only `NMIN` left. Finally it picks a system peer and calls `clock_update()`, and it's **not** going to hop to a new clock if the old one is still at the same stratum as the top survivor 🦋 no clock hopping, periodt.
### A.5.5.2. root_dist()
Next up is `root_dist()`, the function that measures how far off a peer could be from the primary server 🌸 The idea is half the total delay plus the total dispersion plus the peer jitter, all wrapped up in one number. Very neat, very precise 🧚
/*
- root_dist() - calculate root distance / double root_dist( struct p *p / peer structure pointer */ ) {
/*
- The root synchronization distance is the maximum error due to
- all causes of the local clock relative to the primary server.
- It is defined as half the total delay plus total dispersion
- plus peer jitter. */ return (max(MINDISP, p->rootdelay + p->delay) / 2 + p->rootdisp + p->disp + PHI * (c.t - p->t) + p->jitter); }
### A.5.5.3. accept()
Now `accept()` is the gatekeeper deciding whether an association is even allowed to sync 💖 It checks for stratum errors (never synced, or an invalid stratum), distance errors (root distance too big, plus one poll interval of wiggle room), loop errors (the peer is synced to us or to our current system peer, and yes, IPv6 uses an MD5 hash for this instead 🙈), and unreachable peers. If any of those are a problem, it's a big fat `FALSE` 🚫
/*
- accept() - test if association p is acceptable for synchronization / int accept( struct p *p / peer structure pointer / ) { /
- A stratum error occurs if (1) the server has never been
- synchronized, (2) the server stratum is invalid. */ if (p->leap == NOSYNC || p->stratum >= MAXSTRAT) return (FALSE);
/*
- A distance error occurs if the root distance exceeds the
- distance threshold plus an increment equal to one poll
- interval. */
if (root_dist(p) > MAXDIST + PHI * LOG2D(s.poll)) return (FALSE);
/*
- A loop error occurs if the remote peer is synchronized to the
- local peer or the remote peer is synchronized to the current
- system peer. Note this is the behavior for IPv4; for IPv6
- the MD5 hash is used instead. */ if (p->refid == p->dstaddr || p->refid == s.refid) return (FALSE);
/*
- An unreachable error occurs if the server is unreachable. */ if (p->reach == 0) return (FALSE);
return (TRUE); }
### A.5.5.4. clock_update()
Here's where the actual clock gets updated 🎀 Old samples get skipped, because we never reuse a sample or use the same one twice (no recycling, bestie ♻️🙅♀️). Then the survivor offsets get combined and `local_clock()` tells us whether the news is good or bad. If the offset is way too big (**PANIC**), the program just exits 💀. If it's over the step threshold (**STEP**), every association gets reset and starts fresh, so no stale vibes 🌈. If it's under the threshold (**SLEW**, the normal case ✨), the system variables get copied over from the peer, including the stratum (plus one), the root delay, and a root dispersion with a lower clamp. **IGNORE** just skips the update while a direct frequency measurement is happening 🧚
/*
- clock_update() - update the system clock / void clock_update( struct p *p / peer structure pointer */ ) { double dtemp;
/*
- If this is an old update, for instance, as the result of a
- system peer change, avoid it. We never use an old sample or
- the same sample twice. */ if (s.t >= p->t) return;
/*
- Combine the survivor offsets and update the system clock; the
- local_clock() routine will tell us the good or bad news. / s.t = p->t; clock_combine(); switch (local_clock(p, s.offset)) { /
- The offset is too large and probably bogus. Complain to the
- system log and order the operator to set the clock manually
- within PANIC range. The reference implementation includes a
- command line option to disable this check and to change the
- panic threshold from the default 1000 s as required. */ case PANIC: exit (0);
/*
- The offset is more than the step threshold (0.125 s by
- default). After a step, all associations now have
- inconsistent time values, so they are reset and started
- fresh. The step threshold can be changed in the reference
- implementation in order to lessen the chance the clock might
- be stepped backwards. However, there may be serious
- consequences, as noted in the white papers at the NTP project
- site. / case STEP: while (/ all associations */ 0) clear(p, X_STEP); s.stratum = MAXSTRAT; s.poll = MINPOLL; break;
/*
- The offset was less than the step threshold, which is the
- normal case. Update the system variables from the peer
- variables. The lower clamp on the dispersion increase is to
- avoid timing loops and clockhopping when highly precise
- sources are in play. The clamp can be changed from the
- default .01 s in the reference implementation. */ case SLEW: s.leap = p->leap; s.stratum = p->stratum + 1; s.refid = p->refid; s.reftime = p->reftime; s.rootdelay = p->rootdelay + p->delay; dtemp = SQRT(SQUARE(p->jitter) + SQUARE(s.jitter)); dtemp += max(p->disp + PHI * (c.t - p->t) + fabs(p->offset), MINDISP); s.rootdisp = p->rootdisp + dtemp; break;
/*
- Some samples are discarded while, for instance, a direct
- frequency measurement is being made. */ case IGNORE: break; } }
### A.5.5.5. clock_combine()
Last bit for this chunk is `clock_combine()`, which is where the survivor offsets get blended together 🦋
/*
- clock_combine() - combine offsets */ clock_combine()
{
struct p *p; /* peer structure pointer */
double x, y, z, w;
int i;
/*
* Combine the offsets of the clustering algorithm survivors
* using a weighted average with weight determined by the root
* distance. Compute the selection jitter as the weighted RMS
* difference between the first survivor and the remaining
* survivors. In some cases, the inherent clock jitter can be
* reduced by not using this algorithm, especially when frequent
* clockhopping is involved. The reference implementation can
* be configured to avoid this algorithm by designating a
* preferred peer.
*/
y = z = w = 0;
for (i = 0; s.v[i].p != NULL; i++) {
p = s.v[i].p;
x = root_dist(p);
y += 1 / x;
z += p->offset / x;
w += SQUARE(p->offset - s.v[0].p->offset) / x;
}
s.offset = z / y;
s.jitter = SQRT(w / y);
}
### A.5.5.6. local_clock() ✨
okay bestie, this is the clock discipline era 💅 the function below is where the local clock gets its whole glow-up, and it's giving main character energy 🦋
/*
- Clock discipline parameters and constants / #define STEPT .128 / step threshold (s) / #define WATCH 900 / stepout threshold (s) / #define PANICT 1000 / panic threshold (s) / #define PLL 65536 / PLL loop gain / #define FLL MAXPOLL + 1 / FLL loop gain / #define AVG 4 / parameter averaging constant / #define ALLAN 1500 / compromise Allan intercept (s) / #define LIMIT 30 / poll-adjust threshold / #define MAXFREQ 500e-6 / frequency tolerance (500 ppm) / #define PGATE 4 / poll-adjust gate */
/*
- local_clock() - discipline the local clock / int / return code / local_clock( struct p *p, / peer structure pointer / double offset / clock offset from combine() / ) { int state; / clock discipline state / double freq; / frequency / double mu; / interval since last update */ int rval; double etemp, dtemp;
/*
- If the offset is too large, give up and go home. */ if (fabs(offset) > PANICT) return (PANIC);
/*
- Clock state machine transition function. This is where the
- action is and defines how the system reacts to large time
- and frequency errors. There are two main regimes: when the
- offset exceeds the step threshold and when it does not. / rval = SLEW; mu = p->t - s.t; freq = 0; if (fabs(offset) > STEPT) { switch (c.state) { /
- In S_SYNC state, we ignore the first outlier and
- switch to S_SPIK state. */ case SYNC: state = SPIK; return (rval);
/*
- In S_FREQ state, we ignore outliers and inliers. At
- the first outlier after the stepout threshold,
- compute the apparent frequency correction and step
- the time. */ case FREQ: if (mu < WATCH) return (IGNORE);
freq = (offset - c.offset) / mu; /* fall through to S_SPIK */
/*
- In S_SPIK state, we ignore succeeding outliers until
- either an inlier is found or the stepout threshold is
- exceeded. */ case SPIK: if (mu < WATCH) return (IGNORE);
/* fall through to default */
/*
- We get here by default in S_NSET and S_FSET states
- and from above in S_FREQ state. Step the time and
- clamp down the poll interval. *
- In S_NSET state, an initial frequency correction is
- not available, usually because the frequency file has
- not yet been written. Since the time is outside the
- capture range, the clock is stepped. The frequency
- will be set directly following the stepout interval. *
- In S_FSET state, the initial frequency has been set
- from the frequency file. Since the time is outside
- the capture range, the clock is stepped immediately,
- rather than after the stepout interval. Guys get
- nervous if it takes 17 minutes to set the clock for
- the first time. *
- In S_SPIK state, the stepout threshold has expired
- and the phase is still above the step threshold.
- Note that a single spike greater than the step
- threshold is always suppressed, even at the longer
- poll intervals. */ default:
/*
- This is the kernel set time function, usually
- implemented by the Unix settimeofday() system
- call. */ step_time(offset); c.count = 0; s.poll = MINPOLL; rval = STEP; if (state == NSET) { rstclock(FREQ, p->t, 0); return (rval); } break; } rstclock(SYNC, p->t, 0); } else {
/*
- Compute the clock jitter as the RMS of exponentially
- weighted offset differences. This is used by the
- poll-adjust code. */ etemp = SQUARE(c.jitter); dtemp = SQUARE(max(fabs(offset - c.last), LOG2D(s.precision))); c.jitter = SQRT(etemp + (dtemp - etemp) / AVG); switch (c.state) {
/*
- In S_NSET state, this is the first update received
- and the frequency has not been initialized. The
- first thing to do is directly measure the oscillator
- frequency. */ case NSET: rstclock(FREQ, p->t, offset); return (IGNORE);
/*
- In S_FSET state, this is the first update and the
- frequency has been initialized. Adjust the phase,
- but don't adjust the frequency until the next update. */ case FSET: rstclock(SYNC, p->t, offset); break;
/*
- In S_FREQ state, ignore updates until the stepout
- threshold. After that, correct the phase and
- frequency and switch to S_SYNC state. */ case FREQ: if (c.t - s.t < WATCH) return (IGNORE);
freq = (offset - c.offset) / mu; break;
/*
- We get here by default in S_SYNC and S_SPIK states.
- Here we compute the frequency update due to PLL and
- FLL contributions. */ default:
/*
- The FLL and PLL frequency gain constants
- depending on the poll interval and Allan
- intercept. The FLL is not used below one
- half the Allan intercept. Above that the
- loop gain increases in steps to 1 / AVG. / if (LOG2D(s.poll) > ALLAN / 2) { etemp = FLL - s.poll; if (etemp < AVG) etemp = AVG; freq += (offset - c.offset) / (max(mu, ALLAN) * etemp); } /
- For the PLL the integration interval
- (numerator) is the minimum of the update
- interval and poll interval. This allows
- oversampling, but not undersampling. */ etemp = min(mu, LOG2D(s.poll)); dtemp = 4 * PLL * LOG2D(s.poll); freq += offset * etemp / (dtemp * dtemp); rstclock(SYNC, p->t, offset); break; } }
/*
- Calculate the new frequency and frequency stability (wander).
- Compute the clock wander as the RMS of exponentially weighted
- frequency differences. This is not used directly, but can,
- along with the jitter, be a highly useful monitoring and
- debugging tool. */ freq += c.freq; c.freq = max(min(MAXFREQ, freq), -MAXFREQ); etemp = SQUARE(c.wander); dtemp = SQUARE(freq); c.wander = SQRT(etemp + (dtemp - etemp) / AVG);
/*
- Here we adjust the poll interval by comparing the current
- offset with the clock jitter. If the offset is less than the
- clock jitter times a constant, then the averaging interval is
- increased; otherwise, it is decreased. A bit of hysteresis
- helps calm the dance. Works best using burst mode. */ if (fabs(c.offset) < PGATE * c.jitter) { c.count += s.poll; if (c.count > LIMIT) { c.count = LIMIT; if (s.poll < MAXPOLL) { c.count = 0; s.poll++; } } } else { c.count -= s.poll << 1; if (c.count < -LIMIT) { c.count = -LIMIT; if (s.poll > MINPOLL) {
c.count = 0; s.poll--; } } } return (rval); }
### A.5.5.7. rstclock() 🎀
this one is the state machine's little helper, and it's lowkey the reset button 🌸
/*
- rstclock() - clock state machine / void rstclock( int state, / new state / double offset, / new offset / double t / new update time / ) { /
- Enter new state and set state variables. Note, we use the
- time of the last clock filter sample, which must be earlier
- than the current time. */ c.state = state; c.last = c.offset = offset; s.t = t; }
### A.5.6. Clock Adjust Process 🦋
and now we're entering the clock adjust era 💖✨
### A.5.6.1. clock_adjust() 🌸
okay so this is the once-a-second heartbeat of the clock 💖 it ticks the process time forward, grows the dispersion a bit every second, nudges the offset toward zero, and checks on every association. literally the most consistent little routine ever 🫶
/*
- clock_adjust() - runs at one-second intervals */ void clock_adjust() { double dtemp;
/*
- Update the process time c.t. Also increase the dispersion
- since the last update. In contrast to NTPv3, NTPv4 does not
- declare unsynchronized after one day, since the dispersion
- threshold serves this function. When the dispersion exceeds
- MAXDIST (1 s), the server is considered unfit for
- synchronization.
*/ c.t++; s.rootdisp += PHI;
/*
- Implement the phase and frequency adjustments. The gain
- factor (denominator) is not allowed to increase beyond the
- Allan intercept. It doesn't make sense to average phase
- noise beyond this point and it helps to damp residual offset
- at the longer poll intervals. */ dtemp = c.offset / (PLL * min(LOG2D(s.poll), ALLAN)); c.offset -= dtemp;
/*
- This is the kernel adjust time function, usually implemented
- by the Unix adjtime() system call. */ adjust_time(c.freq + dtemp);
/*
- Peer timer. Call the poll() routine when the poll timer
- expires. / while (/ all associations / 0) { struct p *p; / dummy peer structure pointer */
if (c.t >= p->nextdate) poll(p); }
/*
- Once per hour, write the clock frequency to a file. / /
- if (c.t % 3600 == 3599)
- write c.freq to file */ }
The big thing to notice is the comment about NTPv3 vs NTPv4 🙈 NTPv4 doesn't call things unsynchronized after a day. Instead, the dispersion threshold does that job, and once the dispersion passes MAXDIST (1 s), the server is just considered unfit for synchronization. Iconic restraint, honestly ✨
---
## A.5.7. Poll Process 🦋
okay now we're in the poll zone 🧚 these are the constants that run the whole show, and they're giving very specific vibes, so don't mess with them 💅
/*
- Poll process parameters and constants / #define UNREACH 12 / unreach counter threshold / #define BCOUNT 8 / packets in a burst / #define BTIME 2 / burst interval (s) */
### A.5.7.1. poll() 💜
this one decides when a packet actually gets sent for an association. it handles broadcasting, manycasting, and the usual unicast flow, plus bursts and the reachability register. main character energy, no cap 👑
/*
- poll() - determine when to send a packet for association p-> / void poll( struct p *p / peer structure pointer */ ) { int hpoll; int oreach;
/*
- This routine is called when the current time c.t catches up
- to the next poll time p->nextdate. The value p->outdate is
- the last time this routine was executed. The poll_update()
- routine determines the next execution time p->nextdate. *
- If broadcasting, just do it, but only if we are synchronized. */ hpoll = p->hpoll; if (p->hmode == M_BCST) { p->outdate = c.t; if (s.p != NULL) peer_xmit(p); poll_update(p, hpoll); return; }
/*
- If manycasting, start with ttl = 1. The ttl is increased by
- one for each poll until MAXCLOCK servers have been found or
- ttl reaches TTLMAX. If reaching MAXCLOCK, stop polling until
- the number of servers falls below MINCLOCK, then start all
- over. */ if (p->hmode == M_CLNT && p->flags & P_MANY) { p->outdate = c.t; if (p->unreach > BEACON) { p->unreach = 0; p->ttl = 1; peer_xmit(p); } else if (s.n < MINCLOCK) { if (p->ttl < TTLMAX) p->ttl++; peer_xmit(p); } p->unreach++; poll_update(p, hpoll); return; } if (p->burst == 0) {
/*
- We are not in a burst. Shift the reachability
- register to the left. Hopefully, some time before
- the next poll a packet will arrive and set the
- rightmost bit. */ oreach = p->reach; p->outdate = c.t; p->reach = p->reach << 1; if (!(p->reach & 0x7)) clock_filter(p, 0, 0, MAXDISP); if (!p->reach) {
/*
- The server is unreachable, so bump the
- unreach counter. If the unreach threshold
- has been reached, double the poll interval
- to minimize wasted network traffic. Send a
- burst only if enabled and the unreach
- threshold has not been reached. */ if (p->flags & P_IBURST && p->unreach == 0) { p->burst = BCOUNT; } else if (p->unreach < UNREACH) p->unreach++; else hpoll++; p->unreach++; } else {
/*
- The server is reachable. Set the poll
- interval to the system poll interval. Send a
- burst only if enabled and the peer is fit. */ p->unreach = 0; hpoll = s.poll; if (p->flags & P_BURST && fit(p)) p->burst = BCOUNT; } } else {
/*
- If in a burst, count it down. When the reply comes
- back the clock_filter() routine will call
- clock_select() to process the results of the burst. / p->burst--; } /
- Do not transmit if in broadcast client mode. */ if (p->hmode != M_BCLN) peer_xmit(p); poll_update(p, hpoll); }
The reachability register is basically a little shift-left diary 📖 every poll it slides left, and if a packet shows up, it sets the rightmost bit. If nothing has shown up for a while, the unreach counter starts climbing, and once it hits the threshold, the poll interval doubles to save traffic. Bursts are the hype part: they send a few packets fast, and then clock_select() gets to sort out the results 🔥
### A.5.7.2. poll_update() 🍓
this is the little scheduler that figures out when the next poll should happen. it gets called from both the poll() and packet() routines, which is why there's a note about a tiny race ngl 🙈
/*
- poll_update() - update the poll interval for association p *
- Note: This routine is called by both the packet() and poll() routine.
- Since the packet() routine is executed when a network packet arrives
- and the poll() routine is executed as the result of timeout, a
- potential race can occur, possibly causing an incorrect interval for
- the next poll. This is considered so unlikely as to be negligible. / void poll_update( struct p *p, / peer structure pointer / int poll / poll interval (log2 s) / ) { /
- This routine is called by both the poll() and packet()
- routines to determine the next poll time. If within a burst
- the poll interval is two seconds. Otherwise, it is the
- minimum of the host poll interval and peer poll interval, but
- not greater than MAXPOLL and not less than MINPOLL. The
- design ensures that a longer interval can be preempted by a
- shorter one if required for rapid response. */ p->hpoll = max(min(MAXPOLL, poll), MINPOLL); if (p->burst > 0) { if (p->nextdate != c.t) return; else p->nextdate += BTIME; } else {
/*
- While not shown here, the reference implementation
- randomizes the poll interval by a small factor. */ p->nextdate = p->outdate + (1 << max(min(p->ppoll, p->hpoll), MINPOLL)); }
/*
- It might happen that the due time has already passed. If so,
- make it one second in the future. */ if (p->nextdate <= c.t) p->nextdate = c.t + 1; }
The vibe here is "shorter intervals can jump the queue," which is so relatable 🧚 a longer interval can get bumped by a shorter one when a quick response is needed. Also, inside a burst the interval is two seconds, and if the due time already slipped by, it just gets pushed one second into the future 🌈
### A.5.7.3. peer_xmit() 🎀
and finally, the packet-sender 💌 it fills in the header, grabs the transmit timestamp, and then either sends the packet or, if the key situation is broken, resets the association and waits.
/*
- transmit() - transmit a packet for association p / void peer_xmit( struct p *p / peer structure pointer / ) { struct x x; / transmit packet */
/*
- Initialize header and transmit timestamp */ x.srcaddr = p->dstaddr; x.dstaddr = p->srcaddr; x.leap = s.leap; x.version = p->version; x.mode = p->hmode; if (s.stratum == MAXSTRAT) x.stratum = 0; else x.stratum = s.stratum; x.poll = p->hpoll; x.precision = s.precision; x.rootdelay = D2FP(s.rootdelay); x.rootdisp = D2FP(s.rootdisp); x.refid = s.refid; x.reftime = s.reftime; x.org = p->org; x.rec = p->rec; x.xmt = get_time(); p->xmt = x.xmt;
/*
- If the key ID is nonzero, send a valid MAC using the key ID
- of the association and the key in the local key cache. If
- something breaks, like a missing trusted key, don't send the
- packet; just reset the association and stop until the problem
- is fixed. / if (p->keyid) if (/ p->keyid invalid */ 0) { clear(p, X_NKEY); return; } x.dgst = md5(p->keyid); xmit_packet(&x); }
---
# Authors' Addresses 💌
the crew behind this spec, with their contact details exactly as they were written 🫶
Dr. David L. Mills University of Delaware Newark, DE 19716 US
Phone: +1 302 831 8247 EMail: mills@udel.edu
Jim Martin (editor) Internet Systems Consortium 950 Charter Street Redwood City, CA 94063 US
Phone: +1 650 423 1378 EMail: jrmii@isc.org
Jack Burbank The Johns Hopkins University Applied Physics Laboratory 11100 Johns Hopkins Road Laurel, MD 20723-6099 US
Phone: +1 443 778 7127 EMail: jack.burbank@jhuapl.edu
William Kasch The Johns Hopkins University Applied Physics Laboratory 11100 Johns Hopkins Road Laurel, MD 20723-6099 US
Phone: +1 443 778 7463 EMail: william.kasch@jhuapl.edu
📦 where this page comes from
- ietf: rfc5905 2010-06
- 💖 girlified by claude-haiku-5-5 on : a language model's rewrite of the original, under its terms