RFC 8633

Network Time Protocol Best Current Practices

📅 published July 2019, from ietf

RFC 8633: Network Time Protocol Best Current Practices 🕰️✨

Okay bestie, let's start with the title block so you know who's behind this 💅. This is RFC 8633, a Best Current Practice (so it's BCP 223 🎀). D. Reilly edited it, with H. Stenn and D. Sibold as co-authors. Reilly is at Orolia USA, Stenn is at the Network Time Foundation, and Sibold is at PTB 🧚. It came out in July 2019 under ISSN 2070-1721. It doesn't list anything it obsoletes or updates, so it's standing on its own, no cap 💖.

Abstract

NTP (the Network Time Protocol) is honestly one of the OG protocols on the Internet 👑, and people have been using it since it first came out. This doc is a collection of best practices for how NTP servers and clients generally get run on the Internet 🌈. It's got recommendations for keeping time stable, accurate, and secure. It's targeted at NTP version 4 as described in RFC 5905 🫶.

Status of This Memo

This memo documents an Internet Best Current Practice 🌸.

The IETF made this one. It represents the consensus of the IETF community, it got public review, and the Internet Engineering Steering Group (IESG) approved it for publication. If you want more on BCPs, check Section 2 of RFC 7841 💜.

Info about the current status of this document, any errata, and how to send feedback is at https://www.rfc-editor.org/info/rfc8633 🦋.

Copyright (c) 2019 IETF Trust and the persons identified as the document authors. All rights reserved. 🎀

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, because they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions, and they are provided without warranty as described in the Simplified BSD License 🔥.

Table of Contents

1.  Introduction  . . . . . . . . . . . . . . . . . . . . . . . .   4
  1.1.  Requirements Language . . . . . . . . . . . . . . . . . .   4
2.  General Network Security Best Practices . . . . . . . . . . .   4
  2.1.  BCP 38  . . . . . . . . . . . . . . . . . . . . . . . . .   4
3.  NTP Configuration Best Practices  . . . . . . . . . . . . . .   5
  3.1.  Keeping NTP Up to Date  . . . . . . . . . . . . . . . . .   5
  3.2.  Using Enough Time Sources . . . . . . . . . . . . . . . .   5
  3.3.  Using a Diversity of Reference Clocks . . . . . . . . . .   6
  3.4.  Control Messages  . . . . . . . . . . . . . . . . . . . .   7
  3.5.  Monitoring  . . . . . . . . . . . . . . . . . . . . . . .   7
  3.6.  Using Pool Servers  . . . . . . . . . . . . . . . . . . .   8
  3.7.  Leap-Second Handling  . . . . . . . . . . . . . . . . . .   8
    3.7.1.  Leap Smearing . . . . . . . . . . . . . . . . . . . .   9
4.  NTP Security Mechanisms . . . . . . . . . . . . . . . . . . .  10
  4.1.  Pre-Shared Key Approach . . . . . . . . . . . . . . . . .  11
  4.2.  Autokey . . . . . . . . . . . . . . . . . . . . . . . . .  11
  4.3.  Network Time Security . . . . . . . . . . . . . . . . . .  11
  4.4.  External Security Protocols . . . . . . . . . . . . . . .  12
5.  NTP Security Best Practices . . . . . . . . . . . . . . . . .  12
  5.1.  Minimizing Information Leakage  . . . . . . . . . . . . .  12
  5.2.  Avoiding Daemon Restart Attacks . . . . . . . . . . . . .  13
  5.3.  Detection of Attacks through Monitoring . . . . . . . . .  14
  5.4.  Kiss-o'-Death Packets . . . . . . . . . . . . . . . . . .  15
  5.5.  Broadcast Mode Only on Trusted Networks . . . . . . . . .  15
  5.6.  Symmetric Mode Only with Trusted Peers  . . . . . . . . .  16
6.  NTP in Embedded Devices . . . . . . . . . . . . . . . . . . .  16
  6.1.  Updating Embedded Devices . . . . . . . . . . . . . . . .  16
  6.2.  Server Configuration  . . . . . . . . . . . . . . . . . .  17
7.  NTP over Anycast  . . . . . . . . . . . . . . . . . . . . . .  17
8.  IANA Considerations . . . . . . . . . . . . . . . . . . . . .  18
9.  Security Considerations . . . . . . . . . . . . . . . . . . .  19
10. References  . . . . . . . . . . . . . . . . . . . . . . . . .  19
  10.1.  Normative References . . . . . . . . . . . . . . . . . .  19
  10.2.  Informative References . . . . . . . . . . . . . . . . .  20
Appendix A.  Best Practices Specific to the Network Time
             Foundation Implementation  . . . . . . . . . . . . .  23
  A.1.  Use Enough Time Sources . . . . . . . . . . . . . . . . .  23
  A.2.  NTP Control and Facility Messages . . . . . . . . . . . .  23
  A.3.  Monitoring  . . . . . . . . . . . . . . . . . . . . . . .  24
  A.4.  Leap-Second File  . . . . . . . . . . . . . . . . . . . .  24
  A.5.  Leap Smearing . . . . . . . . . . . . . . . . . . . . . .  25
  A.6.  Configuring ntpd  . . . . . . . . . . . . . . . . . . . .  25
  A.7.  Pre-Shared Keys . . . . . . . . . . . . . . . . . . . . .  25
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . .  26
Authors' Addresses  . . . . . . . . . . . . . . . . . . . . . . .  26

1. Introduction 🌷

NTP version 4 (NTPv4) has been widely used since it was published as [RFC5905]. This document is a collection of best practices for running NTP clients and servers 💅.

The recommendations here are meant to help operators distribute time on their networks more accurately and securely 🎀. They're meant to apply generally across a broad range of networks. Some specific networks might have higher accuracy requirements that call for extra techniques beyond what's written here 🧐.

The best practices covered include general network security, time-protocol-specific security, and NTP server and client configuration. NTP operation in embedded devices is covered too 🔌.

This document also has info for protocol implementors who want to build their own implementations compliant with RFC 5905 🛠️.

1.1. Requirements Language

Okay, so the key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here 📖✨.

2. General Network Security Best Practices 🔐

2.1. BCP 38

Lowkey so many network attacks rely on messing with the IP source address of a packet so it points to a different IP address than the computer it actually came from 🙈. UDP-based protocols, like NTP, are generally more susceptible to spoofing attacks than connection-oriented protocols are 💀. NTP control messages can generate a lot of data in response to a small query, which makes them an attractive vector for distributed denial-of-service attacks (NTP Control messages are discussed further in Section 3.4). One documented instance of this kind of attack can be found in [DDOS], with more discussion in [IMC14] and [NDSS14] 😭.

BCP 38 [RFC2827] came out in 2000 to give some level of remediation against address-spoofing attacks 🛡️. It calls for filtering outgoing and incoming traffic so the source and destination IP addresses make sense for the expected flow of traffic on each network interface 🦋. It is RECOMMENDED that ISPs and large corporate networks implement ingress and egress filtering 💜. More info is at [BCP38WIKI] 🍓.

3. NTP Configuration Best Practices ⚙️

This section gives best practices for NTP configuration and operation ✨. The practices specific to the Network Time Foundation implementation, including example configuration directives valid at the time of this writing, are compiled in Appendix A 🧚.

3.1. Keeping NTP Up to Date

There are multiple versions and implementations of the NTP protocol in use across lots of different platforms 🌈. The practices in this document are meant to apply generally to any implementation of [RFC5905]. NTP users should pick an implementation that's actively maintained 🫶. Users should keep up to date on any known attacks against their chosen implementation and deploy updates containing security fixes as soon as it's practical 🔥.

3.2. Using Enough Time Sources

Okay this part is so cool 😭. An NTP implementation that's compliant with [RFC5905] takes the available sources of time and feeds that timing data into sophisticated intersection, clustering, and combining algorithms to get the best estimate of the correct time 🕰️. The algorithms themselves are beyond the scope of this document. If you're curious, read [RFC5905] or the detailed description of NTP in [MILLS2006] 🤓.

  • If there's only one source of time, the answer is obvious 🙃. It might not be a great source, but it's the only one that can be considered. Any issue with the time at the source will get passed on to the client 💀.
  • If there are two sources of time and they align well enough, the best time can be calculated easily 🎀. But if one source fails, the solution degrades to the single-source solution above. And if the two sources don't agree, it's hard to know which one is correct without using info from outside the protocol 😬.
  • If there are three sources of time, there's more data to converge on the best calculated time, and that time is more likely to be accurate ✨. Losing one of the sources (by becoming unreachable or unusable) can be tolerated. But at that point, the solution degrades to the two-source solution 🦋.
  • Having four or more sources of time is better, as long as the sources are diverse (Section 3.3). If one of them develops a problem, there are still at least three other time sources 💖.

This analysis assumes that a majority of the servers used in the solution are honest, even if some are inaccurate 🧐. Operators should be aware that if an attacker is in control of the network, the time coming from all servers could be compromised 🚨.

Operators who care about keeping accurate time SHOULD use at least four independent, diverse sources of time 👑. Four sources give sufficient backup in case one goes down. If four sources aren't available, operators MAY use fewer sources, which is subject to the risks outlined above 🫶.

But even with four or more sources of time, systemic problems can still happen 😩. One example involves the leap-smearing concept detailed in Section 3.7.1. For several hours before and after the June 2015 leap second, several operators configured their NTP servers with leap smearing while others didn't. Many NTP end nodes couldn't determine an accurate time source, because two of their four sources of time gave them consistent UTC/POSIX time, while the other two gave them consistent leap-smeared time 💀. This is just one of many potential causes of disagreement among time sources 🌸.

Operators are advised to monitor all time sources in use 👀. If time sources don't generally align, operators are encouraged to investigate the cause and either correct the problems or stop using defective servers 🧹. See Section 3.5 for more information 💅.

3.3. Using a Diversity of Reference Clocks 🌈

When you're running servers with hardware reference clocks attached, it is suggested that you use different types of reference clocks. Having a diverse crew of sources with independent implementations means any one issue is way less likely to cause a service interruption 🦋

But ngl, ask yourself: are all your clocks from the same vendor? 💀 They may have the same bugs! Even devices from different vendors might not be truly independent if they share common elements. Same base chipset? Same firmware version on all of them? Chipset and firmware bugs can happen, and they're way harder to diagnose than application software bugs 😭 When having the correct time is super critical, it's ultimately up to operators to make sure their sources are independent enough, even if they aren't under the operator's control 🫶

A systemic problem with time from any satellite navigation service is possible, and it has happened 🛰️ Sunspot activity can make a satellite or radio-based time source unusable. Depending on the application requirements, operators may need to consider backup scenarios for the rare circumstance when the satellite system is faulty or unavailable 🌸

3.4. Control Messages ✨

Some implementations of NTPv4 provide NTP control messages (also known as Mode 6 messages) 🧚 These were originally specified in Appendix B of [RFC1305], which defined NTPv3. They don't appear in the NTPv4 specification [RFC5905], which obsoletes the NTPv3 specification [RFC1305], but they're still used. At the time of writing, work is being done to formally document the structure of these control messages for use with NTPv4 in [CTRLMSG].

NTP control messages are designed to permit monitoring and optionally authenticated control of NTP and its configuration. Used properly, these facilities give vital debugging and performance information and control 🔧 But when abused, they can be a vector for amplification attacks 😱 For this reason, it is RECOMMENDED that public-facing NTP servers block NTP control message queries from outside their organization 🚫

The ability to use NTP control messages beyond their basic monitoring capabilities SHOULD be limited to authenticated sessions that provide a 'controlkey'. It can also be limited through mechanisms outside the NTP specification, such as Access Control Lists, that only allow access from approved IP addresses 🔐

The NTP control message responses are much larger than the corresponding queries. Thus, they can be abused in high-bandwidth DDoS attacks 💥 Section 2.1 gives more information on how to protect against this abuse by implementing BCP 38.

3.5. Monitoring 👀

Operators SHOULD use their NTP implementation's remote monitoring capabilities to quickly identify servers that are out of sync and ensure correct functioning of the service. Operators SHOULD also monitor system logs for messages so that problems and abuse attempts can be quickly identified 🕵️‍♀️

If a system starts receiving NTP Reply packets from a remote time server that don't correspond to any requests the system sent, that can mean an attacker is forging that system's IP address in requests to the remote time server 🙈 The goal of this attack is to hurt the availability of time for the targeted system. Based on these forged packets, the remote time server might throttle or rate-limit packets, or even stop sending packets to the targeted system 😤

If a system is a broadcast client and its system log shows it's receiving early time messages from its server, that's a sign somebody may be forging packets from a broadcast server (broadcast client and server modes are defined in Section 3 of [RFC5905]) 📡

If a server's system log shows messages indicating it's receiving NTP timestamps that are much earlier than the current system time, then either the system clock is unusually fast or somebody is trying to launch a replay attack against that server 🌀

3.6. Using Pool Servers 🏊

It only takes a small amount of bandwidth and system resources to synchronize one NTP client, but NTP servers that can service tens of thousands of clients take more resources to run. Network operators and advanced users who want to synchronize their computers MUST only synchronize to servers that they have permission to use 🙅‍♀️

The NTP Pool Project is a group of volunteers who have donated their computing and bandwidth resources to freely distribute time from primary time sources to others on the Internet 🙌 The time is generally of good quality but comes with no guarantee whatsoever. If you're interested in using this pool, please review their instructions at [POOLUSE].

Vendors can obtain their own subdomain that's part of the NTP Pool Project 💼 This offers vendors the ability to safely make use of the time distributed by the pool for their devices. Details are available at [POOLVENDOR].

If there's a need to synchronize many computers, an operator may want to run local NTP servers that are synchronized to the NTP Pool Project. NTP users on that operator's networks can then be synchronized to those local NTP servers 🏠

3.7. Leap-Second Handling ⏱️

UTC is kept in agreement with the Universal Time UT1 [SOLAR] to within +/- 0.9 seconds by the insertion (or possibly deletion) of a leap second. UTC is an atomic time scale, whereas UT1 is based on the rotational rate of the earth 🌍 Leap seconds are not introduced at a fixed rate. They are announced by the International Earth Rotation and Reference Systems Service (IERS) in its Bulletin C [IERS] when necessary to keep UTC and UT1 aligned.

NTP time is based on the UTC timescale, and the protocol has the capability to broadcast leap-second information. Some global navigation satellite systems (like GPS) or radio transmitters (like DCF77) broadcast leap-second information too 📻 If an NTP client is synced to an NTP server that provides leap-second notification, the client will get advance notification of impending leap seconds automatically ✨

Since the length of the UT1 day generally slowly increases [SOLAR], all leap seconds that have been introduced since the practice started in 1972 have been positive leap seconds, where a second is added to UTC. NTP also supports a negative leap second, where a second is removed from UTC if it ever becomes necessary ✂️

While earlier versions of NTP contained some ambiguity about when a leap second broadcast by a server should be applied by a client, RFC 5905 is clear that leap seconds are only applied on the last day of a month 📅 However, because some older clients may apply it at the end of the current day, it is RECOMMENDED that NTP servers wait until the last day of the month before broadcasting leap seconds. Doing this will prevent older clients from applying a leap second at the wrong time. When implementing this recommendation, operators should ensure that clients are not configured to use polling intervals greater than 24 hours so the leap-second notification is not missed ⏰

In circumstances where an NTP server is not receiving leap-second information from an automated source, certain organizations maintain files that are updated every time a new leap second is announced:

3.7.1. Leap Smearing 🧈

Some NTP installations make use of a technique called leap smearing. With this method, instead of introducing an extra second (or eliminating a second) in a leap-second event, NTP time is adjusted in small increments over a comparably large window of time (called the smear interval) around the leap-second event. The smear interval should be large enough for the time to be adjusted at a low rate, so that clients will follow the smeared time without objecting. Periods ranging from two to twenty-four hours have been used successfully. During the adjustment window, all the NTP clients' times may be offset from UTC by as much as a full second, depending on the implementation. However, all clients will generally agree on what time they think it is 🫠

The purpose of leap smearing is to enable systems that don't deal with the leap-second event properly to function consistently, at the expense of fidelity to UTC during the smear window. During a standard leap-second event, a minute will have 61 (or possibly 59) seconds, and some applications (and even some OSs) are known to have problems with that 🙃

Operators who have legal obligations or other strong requirements to be synchronized with UTC or civil time SHOULD NOT use leap smearing, because the distributed time cannot be guaranteed to be traceable to UTC during the smear interval 🚨

Clients that are connected to leap-smearing servers MUST NOT apply the standard NTP leap-second handling. These clients must never have a leap-second file loaded, and the smearing servers must never advertise to clients for which a leap second is pending.

Any use of leap-smearing servers should be limited to within a single, well-controlled environment. Leap smearing MUST NOT be used for public-facing NTP servers, as they will disagree with non-smearing servers (as well as UTC) during the leap smear interval, and there is no standardized way for a client to detect that a server is using leap smearing.

bestie, this is the big one: public-facing servers and leap smearing do NOT mix 🚫 Some public-facing servers may still be configured this way despite this guidance, so keep an eye out 👀

System administrators are advised to be aware of impending leap seconds and how the servers (inside and outside their organization) they are using deal with them. Individual clients MUST NOT be configured to use a mixture of smeared and non-smeared servers. If a client uses smeared servers, the servers it uses must all have the same leap smear configuration 💜

4. NTP Security Mechanisms 🔐

In the standard configuration, NTP packets are exchanged unprotected between client and server. An adversary that is able to become a man in the middle is therefore able to drop, replay, or modify the content of the NTP packet, which leads to degradation of the time synchronization or transmission of false time information 😈 A threat analysis for time-synchronization protocols is given in [RFC7384].

NTP provides two internal security mechanisms to protect the authenticity and integrity of the NTP packets. Both measures protect the NTP packet by means of a Message Authentication Code (MAC) 🛡️ Neither of them encrypts the NTP's payload, because this payload information is not considered to be confidential 🫶

4.1. Pre-Shared Key Approach 🔐

okay so this approach uses one symmetric key to calculate the MAC, and that's what protects the authenticity and integrity of the packets in an association 💖. but here's the catch: NTP has no built-in way to swap keys between the nodes 😭. So each association's keys MUST be exchanged securely by external means, and they MUST be protected from disclosure 🫶. bestie, it's RECOMMENDED that every association gets its own unique key 💅. It's also RECOMMENDED that participants agree to refresh keys periodically 🔄, but NTP doesn't help with that at all 🙈. Each communication partner has to track its keys in its own local key storage 🗝️.

[RFC5905] specifies the MD5 hash algorithm for the MAC calculation 🧁, though other algorithms may be supported too. But MD5 is now considered too weak and unsuitable for cryptographic usage 💀. [RFC6151] has more on its weaknesses. Implementations based on AES-128-CMAC [RFC8573] are coming soon, and users SHOULD use that when it's available ✨.

Some implementations store the key in clear text 😬, so it MUST only be readable by the NTP process. An NTP client also has to be able to link a key to a particular server so it can set up a protected association 🔗. How that linkage works is implementation specific. Once it's applied, the key is trusted until the link is removed 👑.

4.2. Autokey 🔑

[RFC5906] specifies the Autokey protocol 📜. It came out in 2010 to provide automated key management and authentication of NTP servers. However, security researchers have found vulnerabilities [AUTOKEY] in Autokey 🙀.

Autokey SHOULD NOT be used. ❌

4.3. Network Time Security 🌸

Work is in progress on an enhanced replacement for Autokey ✨. Check out [NTSFORNTP] for more info 🦋.

4.4. External Security Protocols 🛡️

If applicable, external security protocols such as IPsec and MACsec can be applied to boost the integrity and authenticity protection of NTP time-synchronization packets 💪. Heads up though: using them can decrease time-synchronization performance [RFC7384] 🐢. So operators are advised to carefully evaluate whether that performance hit still meets their specific timing requirements 🧐.

🚨 Note that none of the security measures described in Section 4 can prevent packet delay manipulation attacks on NTP. Such delay attacks can target time-synchronization packets sent as clear text or even within an encrypted tunnel. These attacks are described further in Section 3.2.6 of [RFC7384].

5. NTP Security Best Practices 🎀

This section lists some general NTP security practices 📝, but these issues may (or may not) have been mitigated in particular versions of particular implementations. Contact the maintainers of the relevant implementation for more information 📮.

5.1. Minimizing Information Leakage 🤫

The base NTP packet leaks important info (including reference ID and reference time) that may be used in attacks [NDSS16] [CVE-2015-8138] [CVE-2016-1548] 😱. A remote attacker can learn this by sending mode 3 queries to a target system and inspecting the fields in the mode 4 response packet 👀. NTP control queries also leak important info (including reference ID and expected origin timestamp, etc.) that may be used in attacks [CVE-2015-8139] 💀. A remote attacker can learn that too, just by sending control queries and inspecting the leaked info in the response.

As such, mechanisms outside of the NTP protocol, such as Access Control Lists, SHOULD be used to limit exposure of this info to allowed IP addresses and keep it away from remote attackers not on the list 🚧. Hosts SHOULD only respond to NTP control queries from authorized parties 🙅‍♀️.

An NTP client that does not provide time on the network can additionally log and drop incoming mode 3 timing queries from unexpected sources 📒. Note well that the easiest way to monitor the status of an NTP instance is to send it a mode 3 query, so it may not be desirable to drop all mode 3 queries 🤔. As an alternative, operators SHOULD either filter mode 3 queries from outside their networks or make sure mode 3 queries are allowed only from trusted systems or networks 🧱.

A "leaf-node host" is a host that uses NTP solely for the purpose of adjusting its own system time 🍓. It isn't expected to provide time to other hosts, and it relies exclusively on NTP's basic mode to take time from a set of servers (that is, the host sends mode 3 queries to its servers and receives mode 4 responses from these servers containing timing information.) To minimize information leakage, leaf-node hosts SHOULD drop all incoming NTP packets except mode 4 response packets that come from known sources. An exception to this can be made if a leaf-node host is being actively monitored, in which case incoming packets from the monitoring server can be allowed 👀.

Please refer to [DATAMIN] for more information 📚.

5.2. Avoiding Daemon Restart Attacks 🔄

[RFC5905] says NTP clients should not accept time shifts greater than the panic threshold. Specifically, RFC 5905 says "PANIC means the offset is greater than the panic threshold PANICT (1000 s) and SHOULD cause the program to exit with a diagnostic message to the system log." 🪧

However, this behavior can be exploited by attackers, as described in [NDSS16], when the following two conditions hold 😈:

  1. The operating system automatically restarts the NTP client when it quits. Modern UNIX and UNIX-like operating systems are swapping traditional init systems for process supervisors, such as systemd, which can be configured to automatically restart any daemons that quit 🔁. This is the default in CoreOS and Arch Linux 🐧. As of the time of this writing, it looks likely to become the default in other systems as they migrate legacy init scripts to process supervisors such as systemd 👀.
  2. The NTP client is configured to ignore the panic threshold on all restarts 🙃.

In such cases, if the attacker can send the target an offset that exceeds the panic threshold, the client will quit. Then, when it restarts, it ignores the panic threshold and accepts the attacker's large offset 😱.

Operators need to be aware that when operating with the above two conditions, the panic threshold offers no protection from attacks 🚨. The natural solution is not to run hosts with these conditions 🙅‍♀️.

Specifically, operators SHOULD NOT ignore the panic threshold in all cold-start situations unless sufficient oversight and checking is in place to make sure that this type of attack cannot happen 🧐.

As an alternative, the following steps MAY be taken by operators to mitigate the risk of attack:

  • Monitor the NTP system log to detect when the NTP daemon quit due to a panic event, as this could be a sign of an attack.
  • Request manual intervention when a timestep larger than the panic threshold is detected.
  • Configure the ntp client to only ignore the panic threshold in a cold-start situation.
  • Increase the minimum number of servers required before the NTP client adjusts the system clock. This will make the NTP client wait until enough trusted sources of time agree before declaring the time to be correct.

In addition, the following steps SHOULD be taken by those who implement the NTP protocol:

  • Prevent the NTP daemon from taking time steps that set the clock to a time earlier than the compile date of the NTP daemon.
  • Prevent the NTP daemon from putting 'INIT' in the reference ID of its NTP packets upon initializing. This will make it more difficult for attackers to know when the daemon reboots.

5.3. Detection of Attacks through Monitoring 🕵️‍♀️

Operators SHOULD monitor their NTP instances to detect attacks 👀. Many known attacks on NTP have particular signatures. Common attack signatures include:

  1. Bogus packets - A packet whose origin timestamp does not match the value that is expected by the client.
  2. Zero origin packet - A packet with an origin timestamp set to zero [CVE-2015-8138].
  3. A packet with an invalid cryptographic MAC.

The observation of many such packets could indicate that the client is under attack 🚨.

5.4. Kiss-o'-Death Packets 💀

The "Kiss-o'-Death" (KoD) packet includes a rate-management mechanism where a server can tell a misbehaving client to reduce its query rate ✋. KoD packets in general (and the RATE packet in particular) are defined in Section 7.4 of [RFC5905]. It is RECOMMENDED that all NTP devices respect these packets and back off when asked to do so by a server 🙏. This is even more important for an embedded device, which may not have an exposed control interface for NTP 🔌.

That said, a client MUST only accept a KoD packet if it has a valid origin timestamp 🧾. Once a RATE packet is accepted, the client should increase its poll interval value (thus decreasing its polling rate) to a reasonable maximum. This maximum can vary by implementation but should not exceed a poll interval value of 13 (two hours) ⏰. The mechanism to determine how much to increase the poll interval value is undefined in [RFC5905]. If the client uses the poll interval value sent by the server in the RATE packet, it MUST NOT simply accept any value. Using large interval values may create a vector for a denial-of-service attack that causes the client to stop querying its server [NDSS16] 😵.

The KoD rate-management mechanism relies on clients behaving properly in order to be effective. Some clients ignore the RATE packet entirely, and other poorly implemented clients might unintentionally increase their poll rate and simulate a denial-of-service attack 🙈. Server administrators are advised to be prepared for this and take measures outside of the NTP protocol to drop packets from misbehaving clients when these clients are detected 🛑.

Kiss-o'-Death (KoD) packets can be used in denial-of-service attacks. Thus, the observation of even just one RATE packet with a high poll value could be sign that the client is under attack. And KoD packets are commonly accepted even when not cryptographically authenticated, which increases the risk of denial-of-service attacks 😭.

5.5. Broadcast Mode Only on Trusted Networks 📡

Per [RFC5905], NTP's broadcast mode is authenticated using symmetric key cryptography. The broadcast server and all its broadcast clients share a symmetric cryptographic key, and the broadcast server uses this key to append a MAC to the broadcast packets it sends 🔑.

Importantly, all broadcast clients that listen to this server have to know the cryptographic key. This means that any client can use this key to send valid broadcast messages that look like they come from the broadcast server. Thus, a rogue broadcast client can use its knowledge of this key to attack the other broadcast clients 😈.

For this reason, an NTP broadcast server and all its clients have to trust each other. Broadcast mode SHOULD only be run from within a trusted network 🏠.

okay bestie, let's get into it 💅

5.6. Symmetric Mode Only with Trusted Peers

Okay so picture this: two peers, Alice and Bob, can both push and pull time syncing to and from each other 🫶. They can do it with either ephemeral symmetric passive (mode 2) packets or persistent symmetric active (NTP mode 1) packets. The persistent association gets preconfigured and kicked off at the active peer, but Bob, the passive peer, is NOT preconfigured for it 🙈. So when Bob gets a mode 1 NTP packet from Alice, he mobilizes a brand new ephemeral association if he doesn't already have one.

Here's the lowkey scary part 💀: an arbitrary attacker can totally try to mess with Bob's time by asking Bob to become its symmetric passive peer. Not it, not on my watch 😭.

So for this reason, a host SHOULD only allow symmetric passive associations to be established with trusted peers 🔥. More specifically, a host SHOULD require each of its symmetric passive associations to be cryptographically authenticated 🔐, and each symmetric passive association SHOULD be authenticated under a different cryptographic key 🗝️. Iconic behavior, periodt ✨.

6. NTP in Embedded Devices 🧚

Ngl, computing is getting SO ubiquitous 🌈. There are going to be a ton of tiny embedded devices that need accurate time ⏰. These devices may not have a persistent battery-backed clock, so using NTP to set the correct time on power-up may be critical for them to work properly 💜. They may also not have a traditional user interface 🙃, but if they connect to the Internet, they're subject to the exact same security threats as traditional deployments. Main character energy doesn't exempt anyone 💀.

6.1. Updating Embedded Devices 🎀

Vendors of embedded devices are advised to pay attention to the current state of protocol security issues and bugs in their chosen implementation 🔍. Why? Because their customers can't update their NTP implementation on their own 😤. Those devices may have a single firmware upgrade, provided by the manufacturer, that updates all capabilities at once 🍓. Which means the vendor takes on the responsibility of making sure their devices have an up-to-date and secure NTP implementation 👑.

Vendors of embedded devices SHOULD include the ability to update the list of NTP servers used by the device 🌸.

There's a whole catalog of NTP server abuse incidents, some involving embedded devices, on the Wikipedia page for NTP Server Misuse and Abuse [MISUSE] 📚. Go look, it's wild 😭.

6.2. Server Configuration 🦋

Vendors of embedded devices with preconfigured NTP servers need to think carefully about which servers they use 🤔. There are several public-facing NTP servers out there, but they may not be ready to handle requests from thousands of new devices on the Internet 😬. Vendors MUST only preconfigure NTP servers that they have permission to use 🚫.

Vendors are encouraged to invest resources into providing their own time servers for their devices to connect to 💖. This can be done through the NTP Pool Project, as documented in Section 3.6 ✨.

Vendors should read [RFC4085] 📖, which advises against embedding globally routable IP addresses in products and offers several better alternatives. Iykyk 🙈.

7. NTP over Anycast 🌍

Anycast is described in BCP 126 [RFC4786] (see also [RFC7094]) 🌐. The vibe: a single IP address is assigned to multiple servers, and routers direct packets to the closest active server 🧭.

Anycast is super common for Internet services at known IP addresses, like DNS 💅. It can also be used in large organizations to simplify configuring lots of NTP clients 🎀. Each client can be set up with the same NTP server IP address, and a pool of anycast servers can be deployed to handle those requests 🫧. New servers can be added to or taken from the pool, and other than a temporary loss of service while a server is taken down, these changes can be transparent to the clients ✨. Seamless, honestly iconic 👑.

Note well 🚨: using a single anycast address for NTP has its own potential issues. Each client will likely end up using a single time server source 😵. A key element of a robust NTP deployment is each client using multiple sources of time 🕰️. With multiple time sources, a client analyzes the various sources, selects the good ones, and disregards the poor ones 🧐. If a single anycast address is used, that analysis won't happen 🙅‍♀️. This can be mitigated by creating multiple, separate anycast pools so clients can have multiple sources of time while still getting the configuration perks of the anycast pools 💜.

If clients are connected to an NTP server via anycast, the client doesn't know which particular server it's connected to 🫠. As anycast servers enter and leave the network, or the network topology changes, the server a client is connected to may change 🔄. This may cause a small shift in time from the client's perspective when that server changes 🌙. Extreme cases where the network topology changes rapidly could cause the server a client sees to change rapidly too, which can lead to larger time inaccuracies 💀. It is RECOMMENDED that network operators only deploy anycast NTP in environments where they know these small shifts can be tolerated by the applications running on the clients being synchronized this way 🌷.

Configuration of an anycast interface is independent of NTP 🧩. Clients will ALWAYS connect to the closest server, even if that server is having NTP issues 😩. It is RECOMMENDED that anycast NTP implementations have an independent method of monitoring the performance of NTP on a server 🔍. If the server isn't performing to specification, it should remove itself from the anycast network 🙋‍♀️. It is also RECOMMENDED that each anycast NTP server have an alternative method of access, such as an alternate unicast IP address, so its performance can be checked independently of the anycast routing scheme 💖.

One useful application in large networks is a hybrid unicast/anycast approach 🦋. Stratum 1 NTP servers can be deployed with unicast interfaces at several sites 🏢. Each site may have several Stratum 2 servers with two Ethernet interfaces, or a single interface that can support multiple addresses 🔌. One interface has a unique unicast IP address. The second has an anycast IP interface (with a shared IP address per location) 📍. The unicast interfaces can be used to get time from the Stratum 1 servers globally (and perhaps peer with the other Stratum 2 servers at their site) 🌏. Clients at each site can be configured to use the shared anycast address for their site, which keeps their config simple 🎀. Keeping anycast routing restricted on a per-site basis minimizes disruption at the client if its closest anycast server changes 🫶. Each Stratum 2 server can be uniquely identified on its unicast interface to make monitoring easier 📊.

8. IANA Considerations 📝

This document has no IANA actions 🌸.

9. Security Considerations 🔐

Time is a fundamental component of security on the Internet ⏳. Without a reliable source of current time, many common web authentication schemes fall apart 😱. For example, expired credentials can get accepted, or messages meant to be processed only once can be replayed 💀. Literally the worst.

Much of this document directly addresses how to secure NTP servers 🛡️. In particular, see Section 2, Section 4, and Section 5 👀.

There are several general threats to time-synchronization protocols, which are discussed in [RFC7384] 📚.

[NTSFORNTP] specifies the Network Time Security (NTS) mechanism and applies it to NTP 🔒. Readers are encouraged to check the status of the document and make use of the methods it describes 💅.

10. References 📚

10.1. Normative References 🎀

[RFC2119] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, <https://www.rfc-editor.org/info/rfc2119>.

[RFC2827] Ferguson, P. and D. Senie, "Network Ingress Filtering: Defeating Denial of Service Attacks which employ IP Source Address Spoofing", BCP 38, RFC 2827, DOI 10.17487/RFC2827, May 2000, <https://www.rfc-editor.org/info/rfc2827>.

[RFC4085] Plonka, D., "Embedding Globally-Routable Internet Addresses Considered Harmful", BCP 105, RFC 4085, DOI 10.17487/RFC4085, June 2005, <https://www.rfc-editor.org/info/rfc4085>.

[RFC4786] Abley, J. and K. Lindqvist, "Operation of Anycast Services", BCP 126, RFC 4786, DOI 10.17487/RFC4786, December 2006, <https://www.rfc-editor.org/info/rfc4786>.

[RFC5905] Mills, D., Martin, J., Ed., Burbank, J., and W. Kasch, "Network Time Protocol Version 4: Protocol and Algorithms Specification", RFC 5905, DOI 10.17487/RFC5905, June 2010, <https://www.rfc-editor.org/info/rfc5905>.

[RFC8174] Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, <https://www.rfc-editor.org/info/rfc8174>.

10.2. Informative References ✨

Appendix A. Best Practices Specific to the Network Time Foundation Implementation 💖

The Network Time Foundation (NTF) makes a super popular NTP implementation called ntpd [NTPDOWN]. It's basically a glow-up of the very first NTP implementations that David Mills built at the University of Delaware. 🎀 This appendix has extra recommendations just for this implementation, and they're valid at the time of writing.

A.1. Use Enough Time Sources 🕰️

Along with the recommendation in Section 3.2, ntpd gives you the 'pool' directive. Starting with ntp-4.2.6, using this directive in the ntp.conf file spins up enough associations to give you robust time service. It also kicks out poor servers and adds new ones as needed. ✨ You can use the 'minclock' and 'maxclock' options of the 'tos' command to override the default numbers for how many servers get discovered through the 'pool' directive.

A.2. NTP Control and Facility Messages 🎛️

On top of NTP control messages, ntpd also offers Mode 7 commands for monitoring and configuration. 👀

If Mode 7 has been explicitly enabled for more than basic monitoring, it should be limited to authenticated sessions that provide a 'requestkey'.

As mentioned above, there are two general ways to use Mode 6 and Mode 7 requests. 💅 The first way is to query ntpd for information, and you can switch this off with:

restrict ... noquery

The second way is to modify ntpd's behavior. Changing ntpd's configuration needs an authenticated session by default, and if no authentication keys have been specified, no modifications can be made. 🔐 For extra protection, you can control the ability to make these modifications with:

restrict ... nomodify

Users can stop their NTP servers from considering query/configuration traffic by default by adding the following to their ntp.conf file:

restrict default -4 nomodify notrap nopeer noquery

restrict default -6 nomodify notrap nopeer noquery

restrict source nomodify notrap noquery

A.3. Monitoring 📊

The ntpd implementation allows remote monitoring. Access to this service is generally controlled by the "noquery" directive in NTP's configuration file (ntp.conf) via a "restrict" statement. The syntax reads:

restrict address mask address_mask noquery

If a system is using broadcast mode and is running ntp-4.2.8p6 or later, use the fourth field of the ntp.keys file to specify the IPs of machines that are allowed to serve time to the group. 📣

A.4. Leap-Second File 🪂

Leap-second files need ntpd 4.2.6 or later. After you fetch the leap-second file onto the server, add this line to ntpd.conf to apply and use the file, swapping in the proper path:

leapfile "/path/to/leap-file"

You might need to restart ntpd to apply this change. 🔄

ntpd servers with a manually configured leap-second file will ignore a leap-second information broadcast from upstream NTP servers until the leap-second file expires. If no valid leap-second file is available, then a leap-second notification from an attached reference clock is always accepted by ntpd. 🛰️

If no valid leap-second file is available, a leap-second notification may be accepted from upstream NTP servers. As of ntp-4.2.6, a majority of servers must provide the notification before it's accepted. Before 4.2.6, a leap-second notification was accepted if a single upstream server out of a group of configured servers provided one. 😬 That led to misbehavior when single NTP servers sent an invalid leap second warning, e.g., due to a faulty GPS receiver in one server. The behavior was once chosen because in the "early days", there was a greater chance that leap-second information would only be available from a very limited number of sources. 🦖

A.5. Leap Smearing 🧈

Leap smearing arrived in ntpd versions 4.2.8.p3 and 4.3.47 in response to client requests. 🙌 Support for it isn't configured by default and must be added at compile time. Also, no leap smearing happens unless a leap smear interval is specified in ntpd.conf. For more information, refer to [LEAPSEC]. 🫶

A.6. Configuring ntpd 🛠️

See [CONFIGNTP] for additional information on configuring ntpd. 📖

A.7. Pre-Shared Keys 🗝️

Each communication partner has to add the key information to their key file in the form:

keyid type key

where "keyid" is a number between 1 and 65534, inclusive; "type" is an ASCII character that defines the key format; and "key" is the key itself. 🔑

An ntpd client sets up a protected association by appending the option "key keyid" to the server statement in ntp.conf:

server address key keyid

Substitute the server address in the "address" field and the numerical keyid you want to use with that server in the "keyid" field. 🎯

A key is deemed trusted when its keyid is added to the list of trusted keys by the "trustedkey" statement in ntp.conf:

trustedkey keyid_1 keyid_2 ... keyid_n

Starting with ntp-4.2.8p7, the ntp.keys file accepts an optional fourth column, a comma-separated list of IPs that are allowed to serve time. Use this feature. 💜 Note, however, that an adversarial client that knows the symmetric broadcast key could still easily spoof its source IP to an IP that is allowed to serve time. 😭 This is easy to do because the origin timestamp on broadcast mode packets is not validated by the client. By contrast, client/server and symmetric modes do require origin timestamp validation, which makes it more difficult to spoof packets [CCR16]. 🛡️

Acknowledgments 🫶

the authors wanna give a huge shoutout to Sue Graves, Samuel Weiler, Lisa Perdue, Karen O'Donoghue, David Malone, Sharon Goldberg, Martin Burnicki, Miroslav Lichvar, Daniel Fox Franke, Robert Nagy, and Brian Haberman 💖 these people helped make this doc happen, and we are so grateful. literally iconic squad 👑

Authors' Addresses 🦋

here's where you can find the authors. Denis is the editor, so he's the one keeping the whole doc in order 💅

Denis Reilly (editor)
Orolia USA
1565 Jefferson Road, Suite 460
Rochester, NY  14623
United States of America

Email: denis.reilly@orolia.com


Harlan Stenn
Network Time Foundation
P.O. Box 918
Talent, OR  97540
United States of America

Email: stenn@nwtime.org


Dieter Sibold
Physikalisch-Technische Bundesanstalt
Bundesallee 100
Braunschweig  D-38116
Germany

Phone: +49-(0)531-592-8420
Fax:   +49-531-592-698420
Email: dieter.sibold@ptb.de
📦 where this page comes from