VALKEY-HYPERLOGLOGS(7)

💖 valkey-hyperloglogs — bestie edition 💖

okay so this is the manual page for valkey-hyperloglogs (yes, that's the manual name, it's giving very "I live in section 7"), and we're about to talk about HyperLogLog, aka the data structure that counts your whole entire fandom without needing to remember a single one of their names. iconic behavior honestly. 🦋

---

✨ NAME ✨

valkey-hyperloglogs — HyperLogLog 💅

---

🌸 DESCRIPTION 🌸

HyperLogLog is a probabilistic data structure that estimates the cardinality of a set. And because it's probabilistic, it's literally out here trading perfect accuracy for efficient space utilization. that's the deal, that's the whole vibe. 🎀

The HyperLogLog implementation uses up to 12 KB and provides a standard error of 0.81%.

okay so here's the tea. counting unique items usually means you need an amount of memory proportional to the number of items you want to count, because you have to remember the elements you've already seen in the past so you don't count them twice. so much remembering. exhausting. 😭

BUT. a whole set of algorithms exists that trade memory for precision: they return an estimated measure with a standard error, which, in the case of the Valkey implementation for HyperLogLog, is less than 1%. and the actual magic of this algorithm is that you no longer need an amount of memory proportional to the number of items counted — instead you get to use a constant amount of memory. 12k bytes in the worst case, or a lot less if your HyperLogLog (we'll just call them HLL from now on, we're on nickname basis fr fr) has seen very few elements.

HLLs in Valkey, while technically a different data structure, are encoded as a String, so you can call GET to serialize a HLL, and SET to deserialize it back to the server. sneaky? no. iconic? yes. 👑

Conceptually the HLL API is like using Sets to do the same task. You would SADD every observed element into a set, and would use SCARD to check the number of elements inside the set, which are unique since SADD will not re-add an existing element.

And while you don't really add items into an HLL — because the data structure only contains a state that does not include actual elements — the API is the same:

• Every time you see a new element, you add it to the count with PFADD.
• When you want to retrieve the current approximation of unique elements added using the PFADD command, you can use the PFCOUNT command. If you need to merge two different HLLs, the PFMERGE command is available. And since HLLs provide approximate counts of unique elements, the result of the merge will give you an approximation of the number of unique elements across both source HLLs.

127.0.0.1:6379> PFADD bikes Hyperion Deimos Phoebe Quaoar
(integer) 1
127.0.0.1:6379> PFCOUNT bikes
(integer) 4
127.0.0.1:6379> PFADD commuter_bikes Salacia Mimas Quaoar
(integer) 1
127.0.0.1:6379> PFMERGE all_bikes bikes commuter_bikes
OK
127.0.0.1:6379> PFCOUNT all_bikes
(integer) 6

Some examples of use cases for this data structure is counting unique queries performed by users in a search form every day, number of unique visitors to a web page and other similar cases.

Valkey is also able to perform the union of HLLs, so please check the valkey-commands(7) full documentation for more information. 🫶

🌈 Use cases 🌈

Anonymous unique visits of a web page (SaaS, analytics tools) — the classic, the blueprint, the moment.

This application answers these questions:

• How many unique visits has this page had on this day?
• How many unique users have played this song?
• How many unique users have viewed this video?

Note: Storing the IP address or any other kind of personal identifier is against the law in some countries, which makes it impossible to get unique visitor statistics on your website. so like… be careful, bestie. ⚠️

One HyperLogLog is created per page (video/song) per period, and every IP/identifier is added to it on every visit.

💜 Basic commands 💜

• PFADD adds an item to a HyperLogLog.
• PFCOUNT returns an estimate of the number of items in the set.
• PFMERGE combines two or more HyperLogLogs into one.

See the valkey-commands(7) complete list of HyperLogLog commands.

⚡ Performance ⚡

Writing (PFADD) to and reading from (PFCOUNT) the HyperLogLog is done in constant time and space. Merging HLLs is O(n), where n is the number of sketches. fast fast fast. 🏎️

🚧 Limits 🚧

The HyperLogLog can estimate the cardinality of sets with up to 18,446,744,073,709,551,616 (2^64) members. that's a lot of members. that's everybody. 💀

📚 Learn more 📚

• This blog post on the HyperLogLog data structure has a lot of details about the data structure and its implementation in Valkey.

---

and that's the whole page, periodt. 🎀 this one's from Valkey 9.0.0, dated 2025-10-21, and it lives under the manual name VALKEY-HYPERLOGLOGS(7). mwah, go count some uniques 💖✨🦋