VALKEY-LATENCY-MONITOR(7)

πŸ“¦
version 9.0.6 Β· 2 versions
, in fedora 44

✨ NAME ✨

valkey-latency-monitor β€” latency monitoring, bestie. Yes it's a real manual page from the Valkey Manual, and yes we're gonna read the whole thing together πŸ’–


🌸 DESCRIPTION 🌸

Okay so, real talk: Valkey gets used for some seriously demanding stuff. Like, a huge number of queries per second per instance, but ALSO with strict latency requirements β€” both for the average response time AND for the worst-case latency. Both. At the same time. Iconic behavior honestly πŸ‘‘

And here's the thing: even though Valkey is an in-memory system, it still has to deal with the operating system in different ways β€” for example, in the context of persisting to disk. Not to mention Valkey implements a rich set of commands. Some commands are fast and run in constant or logarithmic time. Other commands are slower O(N) commands that can cause latency spikes 😭

And finally β€” and this is the big one β€” Valkey is single threaded. Usually that's a flex: it's an advantage from the point of view of the amount of work it can perform per core, and in the latency figures it is able to provide. BUT it poses a challenge for latency, because that single thread must be able to perform certain tasks incrementally β€” for example, key expiration β€” in a way that does not impact the other clients that are served. She's doing a lot, okay?? πŸ’…

So for all these reasons, there's a feature called Latency Monitoring that helps the user check and troubleshoot possible latency problems. Latency monitoring is composed of the following conceptual parts:

  • Latency hooks that sample different latency-sensitive code paths.
  • Time series recording of latency spikes, split by different events.
  • Reporting engine to fetch raw data from the time series.
  • Analysis engine to provide human-readable reports and hints according to the measurements.

The rest of this document covers the latency monitoring subsystem details. For more information about the general topic of Valkey and latency, see valkey-latency(7) Valkey latency problems troubleshooting πŸ¦‹


πŸŽ€ Events and time series πŸŽ€

Different monitored code paths have different names and are called events. For example, command is an event that measures latency spikes of possibly slow command executions, while fast-command is the event name for the monitoring of the O(1) and O(log N) commands. Other events are less generic and monitor specific operations performed by Valkey. For example, the fork event only monitors the time taken by Valkey to execute the fork(2) system call. Very specific. Very focused. We love a specialist πŸ’–

A latency spike is an event that takes more time to run than the configured latency threshold. There is a separate time series associated with every monitored event. This is how the time series work:

  • Every time a latency spike happens, it is logged in the appropriate time series.
  • Every time series is composed of 160 elements.
  • Each element is a pair made of a Unix timestamp of the time the latency spike was measured and the number of milliseconds the event took to execute.
  • Latency spikes for the same event that occur in the same second are merged by taking the maximum latency. Even if continuous latency spikes are measured for a given event, which could happen with a low threshold, at least 160 seconds of history are available.
  • Records the all-time maximum latency for every element.

The framework monitors and logs latency spikes in the execution time of these events:

  • command: regular commands.
  • fast-command: O(1) and O(log N) commands.
  • fork: the fork(2) system call.
  • rdb-unlink-temp-file: the unlink(2) system call.
  • aof-fsync-always: the fsync(2) system call when invoked by the appendfsync allways policy.
  • aof-write: writing to the AOF β€” a catchall event for write(2) system calls.
  • aof-write-pending-fsync: the write(2) system call when there is a pending fsync.
  • aof-write-active-child: the write(2) system call when there are active child processes.
  • aof-write-alone: the write(2) system call when no pending fsync and no active child process.
  • aof-fstat: the fstat(2) system call.
  • aof-rename: the rename(2) system call for renaming the temporary file after completing BGREWRITEAOF.
  • aof-rewrite-diff-write: writing the differences accumulated while performing BGREWRITEAOF.
  • active-defrag-cycle: the active defragmentation cycle.
  • expire-cycle: the expiration cycle.
  • eviction-cycle: the eviction cycle.
  • eviction-del: deletes during the eviction cycle.

πŸ’œ How to enable latency monitoring πŸ’œ

Okay so what counts as "high latency" for one use case might be completely fine for another. It's giving ✨relative✨. Some applications may require that all queries be served in less than 1 millisecond. For other applications, it may be acceptable for a small amount of clients to experience a 2 second latency on occasion. No judgement, everyone's built different 🫢

The first step to enable the latency monitor is to set a latency threshold in milliseconds. Only events that take longer than the specified threshold will be logged as latency spikes. The user should set the threshold according to their needs. For example, if the application requires a maximum acceptable latency of 100 milliseconds, the threshold should be set to log all the events blocking the server for a time equal or greater to 100 milliseconds.

Enable the latency monitor at runtime in a production server with the following command:

CONFIG SET latency-monitor-threshold 100

Monitoring is turned off by default (threshold set to 0), even if the actual cost of latency monitoring is near zero. While the memory requirements of latency monitoring are very small, there is no good reason to raise the baseline memory usage of a Valkey instance that is working well. She's thriving, don't disturb her nap 😴


πŸ‘‘ Report information with the LATENCY command πŸ‘‘

The user interface to the latency monitoring subsystem is the LATENCY command. Like many other Valkey commands, LATENCY accepts subcommands that modify its behavior. These subcommands are:

  • LATENCY LATEST β€” returns the latest latency samples for all events.
  • LATENCY HISTORY β€” returns latency time series for a given event.
  • LATENCY RESET β€” resets latency time series data for one or more events.
  • LATENCY GRAPH β€” renders an ASCII-art graph of an event's latency samples.
  • LATENCY DOCTOR β€” replies with a human-readable latency analysis report.

Refer to each subcommand's documentation page for further information. Go read them. Do it. For me πŸ’…


That's the whole page bestie β€” no skips, no summaries, every single event name and option made it through the vibe check ✨

And that's a wrap! This was VALKEY-LATENCY-MONITOR(7) from the Valkey Manual, version 9.0.0, dated 2025-10-21 β€” the source/version and date, mentioned here in the sign-off like a little goodbye kiss πŸŽ€ mwah, go check your latency, periodt πŸ’–πŸ¦‹πŸŒΈ

πŸ“¦ where this page comes from