Skip to main content
Version: 2.x

HyperLogLog

A HyperLogLog is a probabilistic structure that estimates how many distinct elements it has seen, using a small and fixed amount of memory. This page documents the three commands that ZIO Redis exposes for it.

All commands are methods on the Redis service. Keys and elements are encoded with the configured CodecSupplier, so any type with a ZIO Schema Schema can be used for both. See Getting Started for how to build the layer.

Commandsโ€‹

The commands are grouped below by purpose:

MethodRedis CommandResultPurpose
Redis#pfAddPFADDBooleanAdds elements to a HyperLogLog
Redis#pfCountPFCOUNTLongEstimates the number of distinct elements
Redis#pfMergePFMERGEUnitMerges several HyperLogLogs into a destination key

None of these commands needs returning, because their results do not depend on a user type.

Adding Elementsโ€‹

Redis#pfAdd takes a key and at least one element. It returns true if the internal registers of the HyperLogLog changed, and false otherwise:

import zio._
import zio.redis._

def trackVisitors(redis: Redis): IO[RedisError, Boolean] =
redis.pfAdd("visitors:today", "alice", "bob", "carol")

Adding an element that was already counted usually leaves the registers unchanged, so the result is false in that case.

Counting Elementsโ€‹

Redis#pfCount returns the approximate number of distinct elements added to the key. When more than one key is given, it returns the approximate size of the union of all of them:

import zio._
import zio.redis._

def countVisitors(redis: Redis): IO[RedisError, (Long, Long)] =
for {
today <- redis.pfCount("visitors:today")
both <- redis.pfCount("visitors:today", "visitors:yesterday")
} yield (today, both)

A key that does not exist counts as an empty HyperLogLog.

Merging HyperLogLogsโ€‹

Redis#pfMerge takes a destination key followed by at least one source key. It stores the union of the sources in the destination, and the sources are left untouched:

import zio._
import zio.redis._

def mergeVisitors(redis: Redis): IO[RedisError, Long] =
for {
_ <- redis.pfMerge("visitors:week", "visitors:today", "visitors:yesterday")
count <- redis.pfCount("visitors:week")
} yield count

Next Stepsโ€‹

  • Sets store exact members when an approximation is not acceptable.
  • Keys covers expiry and deletion, which also apply to HyperLogLog keys.

ZIO Chat

Hello! ๐Ÿ˜Š I'm your ZIO Chat Bot, designed to be your go-to resource for all things ZIO! Whether you're new to this powerful functional effect system or a seasoned pro, I'm here to assist you. ZIO empowers you to build concurrent and resilient applications in Scala, offering referential transparency, composable data types, type-safety, and more. So, go ahead and ask me anything about ZIO โ€“ I'm here to provide the answers you need! ๐Ÿค–