PFCOUNT(3valkey)
π PFCOUNT β okay bestie, let's count some uniques π
So this is the Valkey Command Manual page for PFCOUNT, and no cap, it's lowkey the most iconic little counter in the whole HyperLogLog fam. Sit down, hydrate, let's get into it. π¦
β¨ NAME β¨
pfcount β returns the approximated cardinality of the set(s) observed by the HyperLogLog key(s).
(Translation: it tells you roughly how many unique things you've dumped into your HLL. Emphasis on roughly, we'll get there. π )
π SYNOPSIS π
PFCOUNT key [key...]
πΈ DESCRIPTION πΈ
Okay so here's the vibe. When you call this with one single key, it gives you back the approximated cardinality that the HyperLogLog data structure stored at that variable computed for you. And if that variable literally does not exist? You get 0. Not an error. Just zero. Chill. β¨
When you call it with multiple keys, it returns the approximated cardinality of the union of all those HyperLogLogs β like, it internally merges the HyperLogLogs stored at all the keys you handed it into a temporary HyperLogLog and counts that. Merging era. π
The whole point of the HyperLogLog data structure is that you can count unique elements in a set using just a tiny constant amount of memory β specifically 12k bytes for every HyperLogLog (plus a few bytes for the key itself). That's it. That's the whole flex. π
And the returned cardinality of the observed set is not exact, but approximated with a standard error of 0.81%. So like, close. Bestie-close. Not exact-close.
For example β say you wanna take the count of all the unique search queries performed in a day. Your program needs to call valkey-pfadd(7) PFADD every single time a query is processed. Then the estimated number of unique queries can be retrieved with PFCOUNT at any time. Obsessed with how easy that is, honestly. π
Note: as a side effect of calling this function, it is possible that the HyperLogLog is modified, since the last8bytes encode the latest computed cardinality for caching purposes. SoPFCOUNTis technically a write command. Yes bestie, a "read" that writes. It's giving main character energy with a secret. π
π REPLY π
valkey-protocol(7) Integer reply: the approximated number of unique elements observed via PFADD.
π§ COMPLEXITY π§
O(1) with a very small average constant time when called with a single key. O(N) with N being the number of keys, and much bigger constant times, when called with multiple keys.
π ACL CATEGORIES π
@hyperloglog @read @slow
β HISTORY β
- Available since:
2.8.9
π₯ EXAMPLES π₯
127.0.0.1:6379> PFADD hll foo bar zap
(integer) 1
127.0.0.1:6379> PFADD hll zap zap zap
(integer) 0
127.0.0.1:6379> PFADD hll foo bar
(integer) 0
127.0.0.1:6379> PFCOUNT hll
(integer) 3
127.0.0.1:6379> PFADD some-other-hll 1 2 3
(integer) 1
127.0.0.1:6379> PFCOUNT hll some-other-hll
(integer) 6
Look at that last one β 3 + 3 = 6, union slay. π
π¦ Performances π¦
When PFCOUNT is called with a single key, performances are excellent β even though in theory the constant times to process a dense HyperLogLog are high. Why is it still fast? Because PFCOUNT uses caching to remember the cardinality previously computed, and that rarely changes since most PFADD operations will not update any register. Hundreds of operations per second are possible. Periodt. π
When PFCOUNT is called with multiple keys, an on-the-fly merge of the HyperLogLogs is performed, which is slow. And on top of that, the cardinality of the union can't be cached, so when used with multiple keys PFCOUNT may take a time in the order of magnitude of the millisecond, and should be not abused. Ma'am, please. Be gentle. π
The user should take in mind that single-key and multiple-keys executions of this command are semantically different and have different performances. Not the same thing wearing a different outfit. They're genuinely different.
πΈ HyperLogLog representation πΈ
HyperLogLogs are represented using a double representation:
- the sparse representation β suitable for HLLs counting a small number of elements (resulting in a small number of registers set to non-zero value), and
- the dense representation β suitable for higher cardinalities.
Valkey automatically switches from the sparse to the dense representation when needed. It just knows. β¨
The sparse representation uses a run-length encoding optimized to store efficiently a big number of registers set to zero. The dense representation is a String of 12288 bytes in order to store 16384 6-bit counters. The need for the double representation comes from the fact that using 12k (which is the dense representation memory requirement) to encode just a few registers for smaller cardinalities is extremely suboptimal. Like, wasteful. Not cute. π
Both representations are prefixed with a 16 bytes header, that includes a magic, an encoding / version field, and the cached cardinality estimation computed, stored in little endian format (the most significant bit is 1 if the estimation is invalid since the HyperLogLog was updated since the cardinality was computed). iykyk. π
The HyperLogLog, being a String, can be retrieved with valkey-get(7) GET and restored with valkey-set(7) SET. Calling valkey-pfadd(7) PFADD, PFCOUNT or valkey-pfmerge(7) PFMERGE commands with a corrupted HyperLogLog is never a problem β it may return random values but does not affect the stability of the server. Most of the times when corrupting a sparse representation, the server recognizes the corruption and returns an error. So like, it'll be fine, but it also might just clown on you with random numbers. π
The representation is neutral from the point of view of the processor word size and endianness, so the same representation is used by 32 bit and 64 bit processor, big endian or little endian. Universal. Iconic. π
More details about the HyperLogLog implementation can be found in this blog post. The source code of the implementation in the hyperloglog.c file is also easy to read and understand, and includes a full specification for the exact encoding used for the sparse and dense representations. Go read it bestie, it's actually good. π«Ά
π SEE ALSO π
pfadd(3valkey), pfdebug(3valkey), pfmerge(3valkey), pfselftest(3valkey)
and that's the whole page!! version 9.0.0, dated 2025-10-21 β ty for reading, go count some uniques, ily, mwah πβ¨ππ¦
π¦ where this page comes from
- fedora 44: valkey-doc 9.0.3-2.fc44 Β· CC-BY-SA-4.0
- fedora 44: valkey-doc 9.0.6-1.fc44 Β· CC-BY-SA-4.0
- π girlified by deepseek-ai/DeepSeek-V4.1-Flash on : a language model's rewrite of the original, under its terms