VALKEY-CLIENT-SIDE-CACHING(7)

📦
version 9.0.6 · 2 versions
, in fedora 44

✨ NAME ✨

valkey-client-side-caching — Client-side caching 💅

okay bestie, welcome to the VALKEY-CLIENT-SIDE-CACHING(7) manual page, and no cap this one is a banger. it's all about client-side caching, which is a whole technique for making services go zoom. let's get into it. 🦋

🌸 DESCRIPTION 🌸

okay so client-side caching? it's a technique used to create high performance services. the whole vibe is that it exploits the memory available on application servers — and bestie, those are usually distinct computers compared to the Valkey nodes — to store some subset of the Valkey information directly in the application side. 💾✨

normally when data is required, the application servers ask the Valkey about such information, like in the following diagram:

              +-------------+                                +----------+
              |             | ------- GET user:1234 -------> |          |
              | Application |                                |  Valkey  |
              |             | <---- username = Alice ------- |          |
              +-------------+                                +----------+

when client-side caching is used, the application will store the reply of popular queries directly inside the application memory, so that it can reuse such replies later, without contacting the Valkey again:

              +-------------+                                +----------+
              |             |                                |          |
              | Application |       ( No chat needed )       |  Valkey  |
              |             |                                |          |
              +-------------+                                +----------+
              | Local cache |
              |             |
              | user:1234 = |
              | username    |
              | Alice       |
              +-------------+

while the application memory used for the local cache may not be very big, the time needed in order to access the local computer memory is orders of magnitude smaller compared to accessing a networked service like a Valkey. since often the same small percentage of data are accessed frequently, this pattern can greatly reduce the latency for the application to get data and, at the same time, the load in the Valkey side. it's giving efficiency. 🔥

moreover there are many datasets where items change very infrequently. for instance, most user posts in a social network are either immutable or rarely edited by the user. adding to this the fact that usually a small percentage of the posts are very popular, either because a small set of users have a lot of followers and/or because recent posts have a lot more visibility, it is clear why such a pattern can be very useful. periodt. 💅

usually the two key advantages of client-side caching are:

  1. Data is available with a very small latency.
  2. The Valkey system receives less queries, allowing it to serve the same dataset with a smaller number of nodes.

🧠 There are two hard problems in computer science...

a problem with the above pattern is how to invalidate the information that the application is holding, in order to avoid presenting stale data to the user. for example after the application above locally cached the information for user:1234, Alice may update her username to Flora. yet the application may continue to serve the old username for user:1234. awkward. 😭

sometimes, depending on the exact application we are modeling, this isn’t a big deal, so the client will just use a fixed maximum “time to live” for the cached information. once a given amount of time has elapsed, the information will no longer be considered valid. more complex patterns, when using Valkey, leverage the Pub/Sub system in order to send invalidation messages to listening clients. this can be made to work but is tricky and costly from the point of view of the bandwidth used, because often such patterns involve sending the invalidation messages to every client in the application, even if certain clients may not have any copy of the invalidated data. moreover every application query altering the data requires to use the PUBLISH command, costing the Valkey more CPU time to process this command. highkey expensive. 💀

regardless of what schema is used, there is a simple fact: many very large applications implement some form of client-side caching, because it is the next logical step to having a fast store or a fast cache server. for this reason Valkey implements direct support for client-side caching, in order to make this pattern much simpler to implement, more accessible, reliable, and efficient. slay. ✨

👑 The Valkey implementation of client-side caching

the Valkey client-side caching support is called Tracking, and has two modes:

  • in the default mode, the server remembers what keys a given client accessed, and sends invalidation messages when the same keys are modified. this costs memory in the server side, but sends invalidation messages only for the set of keys that the client might have in memory.
  • in the broadcasting mode, the server does not attempt to remember what keys a given client accessed, so this mode costs no memory at all in the server side. instead clients subscribe to key prefixes such as object: or user:, and receive a notification message every time a key matching a subscribed prefix is touched.

to recap, for now let’s forget for a moment about the broadcasting mode, to focus on the first mode. we’ll describe broadcasting in more detail later. bestie, stay tuned. 🎀

  1. Clients can enable tracking if they want. Connections start without tracking enabled.
  2. When tracking is enabled, the server remembers what keys each client requested during the connection lifetime (by sending read commands about such keys).
  3. When a key is modified by some client, or is evicted because it has an associated expire time, or evicted because of a maxmemory policy, all the clients with tracking enabled that may have the key cached, are notified with an invalidation message.
  4. When clients receive invalidation messages, they are required to remove the corresponding keys, in order to avoid serving stale data.

this is an example of the protocol:

  • Client 1 -> Server: CLIENT TRACKING ON
  • Client 1 -> Server: GET foo
  • (The server remembers that Client 1 may have the key “foo” cached)
  • (Client 1 may remember the value of “foo” inside its local memory)
  • Client 2 -> Server: SET foo SomeOtherValue
  • Server -> Client 1: INVALIDATE “foo”

this looks great superficially, but if you imagine 10k connected clients all asking for millions of keys over long living connection, the server ends up storing too much information. for this reason Valkey uses two key ideas in order to limit the amount of memory used server-side and the CPU cost of handling the data structures implementing the feature:

  • the server remembers the list of clients that may have cached a given key in a single global table. this table is called the Invalidation Table. the invalidation table can contain a maximum number of entries. if a new key is inserted, the server may evict an older entry by pretending that such key was modified (even if it was not), and sending an invalidation message to the clients. doing so, it can reclaim the memory used for this key, even if this will force the clients having a local copy of the key to evict it.
  • inside the invalidation table we don’t really need to store pointers to clients’ structures, that would force a garbage collection procedure when the client disconnects: instead what we do is just store client IDs (each Valkey client has a unique numerical ID). if a client disconnects, the information will be incrementally garbage collected as caching slots are invalidated.
  • there is a single keys namespace, not divided by Valkey numbers. so if a client is caching the key foo in Valkey 2, and some other client changes the value of the key foo in Valkey 3, an invalidation message will still be sent. this way we can ignore Valkey numbers reducing both the memory usage and the implementation complexity.

💅 Two connections mode

using the new version of the Valkey protocol, RESP3, it is possible to run the data queries and receive the invalidation messages in the same connection. however many client implementations may prefer to implement client-side caching using two separated connections: one for data, and one for invalidation messages. for this reason when a client enables tracking, it can specify to redirect the invalidation messages to another connection by specifying the “client ID” of a different connection. many data connections can redirect invalidation messages to the same connection, this is useful for clients implementing connection pooling. the two connections model is the only one that is also supported for RESP2 (which lacks the ability to multiplex different kind of information in the same connection).

here’s an example of a complete session using the Valkey protocol in the old RESP2 mode involving the following steps: enabling tracking redirecting to another connection, asking for a key, and getting an invalidation message once the key gets modified.

to start, the client opens a first connection that will be used for invalidations, requests the connection ID, and subscribes via Pub/Sub to the special channel that is used to get invalidation messages when in RESP2 modes (remember that RESP2 is the usual Valkey protocol, and not the more advanced protocol that you can use, optionally, using the HELLO command):

(Connection 1 -- used for invalidations)
CLIENT ID
:4
SUBSCRIBE __redis__:invalidate
*3
$9
subscribe
$20
__redis__:invalidate
:1

now we can enable tracking from the data connection:

(Connection 2 -- data connection)
CLIENT TRACKING on REDIRECT 4
+OK

GET foo
$3
bar

the client may decide to cache "foo" => "bar" in the local memory.

a different client will now modify the value of the “foo” key:

(Some other unrelated connection)
SET foo bar
+OK

as a result, the invalidations connection will receive a message that invalidates the specified key.

(Connection 1 -- used for invalidations)
*3
$7
message
$20
__redis__:invalidate
*1
$3
foo

the client will check if there are cached keys in this caching slot, and will evict the information that is no longer valid.

note that the third element of the Pub/Sub message is not a single key but is a Valkey array with just a single element. since we send an array, if there are groups of keys to invalidate, we can do that in a single message. in case of a flush (FLUSHALL or FLUSHDB), a null message will be sent.

okay so THE most important thing to get about client-side caching when you're doing RESP2 + a Pub/Sub connection to read those invalidation messages? The Pub/Sub thing is entirely a trick 💅. Like, it's a hack so old client implementations could be reused. The message is not actually broadcast to a channel and picked up by everybody subscribed to it. Only the connection you named in the REDIRECT argument of the CLIENT command actually receives that Pub/Sub message — which makes the whole feature way more scalable, bestie. Slay.

When you're on RESP3 instead, invalidation messages come through as push messages (either on the same connection, or on the secondary connection if you're using redirection). Go read the RESP3 spec if you want the deets 📬.

🌸 What tracking tracks

So basically, by default, clients do not need to tell the server which keys they're caching. Every key mentioned in the context of a read-only command gets tracked by the server, because hypothetically it could be cached. 👑

Obvious glow-up: the client never has to report what it's caching. And honestly, in a lot of client implementations that's exactly what you want — a really solid move is to just cache everything that isn't already cached, FIFO style. Like, cache a fixed number of objects; every new thing you fetch gets cached, and the oldest cached object gets yeeted. Fancier implementations might instead drop the least-used object or whatever. ✨

But heads up: if there's write traffic on the server, caching slots are gonna get invalidated over time, no matter what. And in general, when the server assumes "what we get, we also cache", you're making a tradeoff:

  1. It's more efficient when the client tends to cache a lot of stuff with a policy that welcomes new objects.
  2. The server is forced to retain more data about the client's keys.
  3. The client receives useless invalidation messages about objects it never cached. 💀

So yeah — there's an alternative, and it's right below.

🎀 Opt-in caching

Some client implementations only wanna cache selected keys, and tell the server explicitly what they will and won't cache. This costs more bandwidth when you cache new objects, but it reduces both the amount of data the server has to remember and the number of invalidation messages the client gets. Balance, bestie. ⚖️

To do this, tracking has to be enabled with the OPTIN option:

CLIENT TRACKING on REDIRECT 1234 OPTIN

In this mode, by default, keys mentioned in read queries are not supposed to be cached. Instead, when a client wants to cache something, it has to send a special command immediately before the actual command that retrieves the data:

CLIENT CACHING YES
+OK
GET foo
"bar"

The CACHING command affects the command executed immediately after it. But! If the next command is MULTI, then all the commands in the transaction get tracked. Same vibe with Lua scripts — all the commands the script executes get tracked. 🧚

🦋 Broadcasting mode

So far we've been talking about the first client-side caching model that Valkey implements. There's another one, called broadcasting, and it looks at the problem from a totally different tradeoff: it consumes zero memory on the server side, but sends more invalidation messages to clients. Here's the main character energy:

  • Clients enable client-side caching with the BCAST option, specifying one or more prefixes using the PREFIX option. For instance: CLIENT TRACKING on REDIRECT 10 BCAST PREFIX object: PREFIX user:. If no prefix is specified at all, the prefix is assumed to be the empty string, so the client will receive invalidation messages for every key that gets modified. If one or more prefixes are used instead, only keys matching one of the specified prefixes get sent in the invalidation messages.
  • The server does not store anything in the invalidation table. Instead it uses a different Prefixes Table, where each prefix is associated to a list of clients.
  • No two prefixes can track overlapping parts of the keyspace. For instance, having the prefix "foo" and "foob" would not be allowed, since they would both trigger an invalidation for the key "foobar". However, just using the prefix "foo" is sufficient.
  • Every time a key matching any of the prefixes is modified, all the clients subscribed to that prefix will receive the invalidation message. 📣
  • The server will consume CPU proportional to the number of registered prefixes. If you have just a few, it's hard to see any difference. With a big number of prefixes the CPU cost can become quite large. 😭
  • In this mode the server can perform the optimization of creating a single reply for all the clients subscribed to a given prefix, and send the same reply to all. This helps to lower the CPU usage. ✨

💜 The NOLOOP option

By default, client-side tracking will send invalidation messages to the client that modified the key. Sometimes clients want that, because they implement very basic logic that doesn't involve automatically caching writes locally. But more advanced clients might wanna cache even the writes they're doing in the local in-memory table. In that case, getting an invalidation message immediately after the write is a problem — it forces the client to evict the value it literally just cached. 🙈

For that situation you can use the NOLOOP option: it works in both normal and broadcasting mode. With this option, clients can tell the server they don't want to receive invalidation messages for keys that they themselves modified. Periodt.

🌈 Avoiding race conditions

When you're implementing client-side caching and redirecting invalidation messages to a different connection, be aware there's a possible race condition. Check out this example interaction — we'll call the data connection "D" and the invalidation connection "I":

[D] client -> server: GET foo
[I] server -> client: Invalidate foo (somebody else touched it)
[D] server -> client: "bar" (the reply of "GET foo")

See the problem? The reply to the GET was slower to reach the client, so we got the invalidation message before the actual data — which is already no longer valid. So we'll keep serving a stale version of the foo key. 😭

To dodge this, it's a good idea to populate the cache with a placeholder when we send the command:

Client cache: set the local copy of "foo" to "caching-in-progress"
[D] client-> server: GET foo.
[I] server -> client: Invalidate foo (somebody else touched it)
Client cache: delete "foo" from the local cache.
[D] server -> client: "bar" (the reply of "GET foo")
Client cache: don't set "bar" since the entry for "foo" is missing.

Good news: this race condition is not possible when you use a single connection for both data and invalidation messages, because in that case the order of the messages is always known. 🫶

🍓 What to do when losing connection with the server

Same energy here: if we lose the connection on the socket we use to get invalidation messages, we could end up with stale data. To avoid that, we need to do these things:

  1. Make sure that if the connection is lost, the local cache is flushed.
  2. Both when using RESP2 with Pub/Sub, or RESP3, ping the invalidation channel periodically (yes, you can send PING commands even when the connection is in Pub/Sub mode!). If the connection looks broken and we can't receive ping backs, after a maximum amount of time, close the connection and flush the cache. 🚨

⭐ What to cache

Clients might wanna run internal statistics on how many times a given cached key was actually served in a request, so they can figure out what's actually worth caching. In general:

  • We don't want to cache many keys that change continuously.
  • We don't want to cache many keys that are requested very rarely.
  • We want to cache keys that are requested often and change at a reasonable rate. For an example of a key not changing at a reasonable rate, think of a global counter that is continuously INCRemented.

That said, simpler clients may just evict data using some random sampling, remembering only the last time a given cached value was served, and trying to evict keys that weren't served recently. 🤷♀️

🧚 Other hints for implementing client libraries

  • Handling TTLs: make sure you also request the key TTL and set the TTL in the local cache if you want to support caching keys with a TTL.
  • Putting a max TTL on every key is a good idea, even if it has no TTL. This protects against bugs or connection issues that would make the client have old data in the local copy.
  • Limiting the amount of memory used by clients is absolutely needed. There must be a way to evict old keys when new ones are added.

👑 Limiting the amount of memory used by Valkey

Be sure to configure a suitable value for the maximum number of keys remembered by Valkey — or alternatively use the BCAST mode, which consumes no memory at all on the Valkey side. Note that the memory consumed by Valkey when BCAST is not used is proportional to both the number of keys tracked and the number of clients requesting such keys. 📊

💖 that's the whole page, bestie — you ate that. this is from Valkey 9.0.0, dated 2025-10-21, and it's the VALKEY-CLIENT-SIDE-CACHING(7) manual. go forth and cache responsibly ✨🦋🎀
📦 where this page comes from