Skip to main content
Version: 1.0.18

Ref

Ref[A] models a mutable reference to a value of type A in which we can store immutable data. The two basic operations are set, which fills the Ref with a new value, and get, which retrieves its current content. All operations on a Ref are atomic and thread-safe, providing a reliable foundation for synchronizing concurrent programs.

Ref is ZIO's analog to something like a State Monad in more Haskell-Oriented FP. We don't need State Monad in ZIO, because we have Refs. Refs allow us to get and set state, or update it.

When we write stateful applications, we need some mechanism to manage our state. We need a way to update the in-memory state in a functional way. So this is why we need Refs.

Refs are:

  • Purely Functional and Referential Transparent
  • Concurrent-Safe and Lock-free
  • Update and Modify atomically

Concurrent Stateful Application​

Refs are building blocks for writing concurrent stateful applications. Without Ref or something equivalently, we can't do that. Anytime we need to share information between multiple fibers, and those fibers have to update the same information, they need to communicate through something that provides the guarantee of atomicity. So Refs can update the state in an atomic way, consistent and isolated from all other concurrent updates.

Refs are concurrent-safe. we can share the same Ref among many fibers. All of them can update Ref concurrently. We don't have to worry about race conditions. Even we have ten thousand fibers all updating the same Ref as long as they are using atomic update and modify functions, we will have zero race conditions.

Operations​

The Ref has lots of operations. Here we are going to introduce the most important and common ones. Also, note that Ref is a type alias for ZRef. ZRef has many type parameters. Basically, all of these type parameters on ZRef are useful for the more advanced operators. So as a not advanced user, don't worry about them.

make​

Ref is never empty and it always contains something. We can create Ref by providing the initial value to the make, which is a constructor of the Ref data type. We should pass an immutable value of type A to the constructor, and it returns an UIO[Ref[A]] value:

def make[A](a: A): UIO[Ref[A]]

As we can see, the output is wrapped in UIO, which means creating Ref is effectful. Whenever we make, update, or modify the Ref, we are doing some effectful operation, this is why their output is wrapped in UIO. It helps the API remain referential transparent.

Let's create some Refs from immutable values:

val counterRef = Ref.make(0)
// counterRef: UIO[Ref[Int]] = zio.ZIO$EffectTotal@34507957
val stringRef = Ref.make("initial") 
// stringRef: UIO[Ref[String]] = zio.ZIO$EffectTotal@3564252 

sealed trait State
case object Active  extends State
case object Changed extends State
case object Closed  extends State

val stateRef = Ref.make(Active) 
// stateRef: UIO[Ref[Active.type]] = zio.ZIO$EffectTotal@1c10a424

Warning:

The big mistake to creating Ref is trying to store mutable data inside it. It doesn't work. The only way to use a Ref is to store immutable data inside it, otherwise, it does not provide us atomic guarantees, and we can have collisions and race conditions.

As we mentioned above, we shouldn't create Ref from a mutable variable. The following snippet compiles, but it leads us to race conditions due to improper use of make:

// Compiles but don't work properly
var init = 0
// init: Int = 0
val counterRef = Ref.make(init)
// counterRef: UIO[Ref[Int]] = zio.ZIO$EffectTotal@7a7980cf

So we should change the init to be immutable:

val init = 0
// init: Int = 0
val counterRef = Ref.make(init)
// counterRef: UIO[Ref[Int]] = zio.ZIO$EffectTotal@4dd4c416

get​

The get method returns the current value of the reference. Its return type is IO[EB, B]. Which B is the value type of returning effect and in the failure case, EB is the error type of that effect.

def get: IO[EB, B]

As the make and get methods of Ref are effectful, we can chain them together with flatMap. In the following example, we create a Ref with initial value, and then we acquire the current state with the get method:

Ref.make("initial")
   .flatMap(_.get)
   .flatMap(current => putStrLn(s"current value of ref: $current"))

We can use syntactic sugar representation of flatMap series with for-comprehension:

for {
  ref   <- Ref.make("initial")
  value <- ref.get
} yield assert(value == "initial")

Note that, there is no way to access the shared state outside the monadic operations.

set​

The set method atomically writes a new value to the Ref.

for {
  ref   <- Ref.make("initial")
  _     <- ref.set("update")
  value <- ref.get
} yield assert(value == "update")

update​

With update, we can atomically update the state of Ref with a given pure function. A function that we pass to the update needs to be a pure function, it needs to be deterministic and free of side effects.

def update(f: A => A): IO[E, Unit]

Assume we have a counter, we can increase its value with the update method:

val counterInitial = 0
for {
  counterRef <- Ref.make(counterInitial)
  _          <- counterRef.update(_ + 1)
  value <- counterRef.get
} yield assert(value == 1)

Note:

The update is not the composition of get and set, this composition is not concurrently safe. So whenever we need to update our state, we should not compose get and set to manage our state in a concurrent environment. Instead, we should use the update operation which modifies its Ref atomically.

The following snippet is not concurrent safe:

// Unsafe State Management
object UnsafeCountRequests extends zio.App {
  import zio.console._

  def request(counter: Ref[Int]) = for {
    current <- counter.get
    _ <- counter.set(current + 1)
  } yield ()

  private val initial = 0
  private val program =
    for {
      ref <- Ref.make(initial)
      _ <- request(ref) zipPar request(ref)
      rn <- ref.get
      _ <- putStrLn(s"total requests performed: $rn")
    } yield ()

  override def run(args: List[String]) = program.exitCode
}

The above snippet doesn't behave deterministically. This program sometimes print 2 and sometime print 1. So let's fix that issue by using update which behaves atomically:

// Unsafe State Management
object CountRequests extends zio.App {
  import zio.console._

  def request(counter: Ref[Int]): ZIO[Console, Nothing, Unit] = {
    for {
      _ <- counter.update(_ + 1)
      reqNumber <- counter.get
      _ <- putStrLn(s"request number: $reqNumber").orDie
    } yield ()
  }

  private val initial = 0
  private val program =
    for {
      ref <- Ref.make(initial)
      _ <- request(ref) zipPar request(ref)
      rn <- ref.get
      _ <- putStrLn(s"total requests performed: $rn").orDie
    } yield ()

  override def run(args: List[String]) = program.exitCode
}

Here is another use case of update to write repeat combinator:

def repeat[E, A](n: Int)(io: IO[E, A]): IO[E, Unit] =
  Ref.make(0).flatMap { iRef =>
    def loop: IO[E, Unit] = iRef.get.flatMap { i =>
      if (i < n)
        io *> iRef.update(_ + 1) *> loop
      else
        IO.unit
    }
    loop
  }

modify​

modify is a more powerful version of the update. It atomically modifies its Ref with the given function and, also computes a return value. The function that we pass to the modify needs to be a pure function; it needs to be deterministic and free of side effects.

def modify[B](f: A => (B, A)): IO[E, B]

Remember the CountRequest example. What if we want to log the number of each request, inside the request function? Let's see what happen if we write that function with the composition of update and get methods:

// Unsafe in Concurrent Environment
def request(counter: Ref[Int]) = {
  for {
    _  <- counter.update(_ + 1)
    rn <- counter.get
    _  <- putStrLn(s"request number received: $rn")
  } yield ()
}

What happens if between running the update and get, another update in another fiber performed? This function doesn't perform in a deterministic fashion in concurrent environments. So we need a way to perform get and set and get atomically. This is why we need the modify method. Let's fix the request function to do that atomically:

// Safe in Concurrent Environment
def request(counter: Ref[Int]) = {
  for {
    rn <- counter.modify(c => (c + 1, c + 1))
    _  <- putStrLn(s"request number received: $rn")
  } yield ()
}

AtomicReference in Java​

For Java programmers, we can think of Ref as an AtomicReference. Java has a java.util.concurrent.atomic package and that package contains AtomicReference, AtomicLong, AtomicBoolean and so forth. We can think of Ref as being an AtomicReference. It has roughly the same power, the same guarantees, and the same limitations. It packages it up in a higher-level context and of course, makes it ZIO friendly.

Ref vs. State Monad​

Basically Ref allows us to have all the power of State Monad inside ZIO. State Monad lacks two important features that we use in real-life application development:

  1. Concurrency Support
  2. Error Handling

Concurrency​

State Monad is its effect system that only includes state. It allows us to do pure stateful computations. We can only get, set and update related computations to managing the state. State Monad updates its state with series of stateful computations sequentially, but we can't use the State Monad to do async or concurrent computations. But Refs have great support on concurrent and async programming.

Error Handling​

In real-life applications, we need error handling. In most real-life stateful applications, we will involve some database IO and API calls and or some concurrent and sync stuff that it can fail in different ways along the path of execution. So besides the state management, we need a way to do error handling. The State Monad doesn't have the ability to model error management.

We can combine State Monad and Either Monad with StateT monad transformer, but it imposes massive performance overhead. It doesn't buy us anything that we can't do with a Ref. So it is an anti-pattern. In the ZIO model, errors are encoded in effects and Ref utilizes that. So besides state management, we have the ability to error-handling without any further work.

State Transformers​

Those who live on the dark side of mutation sometimes have it easy; they can add state everywhere like it's Christmas. Behold:

var idCounter = 0
def freshVar: String = {
  idCounter += 1
  s"var${idCounter}"
}
val v1 = freshVar
val v2 = freshVar
val v3 = freshVar

As functional programmers, we know better and have captured state mutation in the form of functions of type S => (A, S). Ref provides such an encoding, with S being the type of the value, and modify embodying the state mutation function.

Ref.make(0).flatMap { idCounter =>
  def freshVar: UIO[String] =
    idCounter.modify(cpt => (s"var${cpt + 1}", cpt + 1))

  for {
    v1 <- freshVar
    v2 <- freshVar
    v3 <- freshVar
  } yield ()
}

Building more sophisticated concurrency primitives​

Ref is low-level enough that it can serve as the foundation for other concurrency data types.

Semaphores are a classic abstract data type for controlling access to shared resources. They are defined as a triple S = (v, P, V) where v is the number of units of the resource that are currently available, and P and V are operations that respectively decrement and increment v; P will only complete when v is non-negative and must wait if it isn't.

Well, with Refs, that's easy to do! The only difficulty is in P, where we must fail and retry when either v is negative or its value has changed between the moment we read it and the moment we try to update it. A naive implementation could look like:

sealed trait S {
  def P: UIO[Unit]
  def V: UIO[Unit]
}

object S {
  def apply(v: Long): UIO[S] =
    Ref.make(v).map { vref =>
      new S {
        def V = vref.update(_ + 1).unit

        def P = (vref.get.flatMap { v =>
          if (v < 0)
            IO.fail(())
          else
            vref.modify(v0 => if (v0 == v) (true, v - 1) else (false, v)).flatMap {
              case false => IO.fail(())
              case true  => IO.unit
            }
        } <> P).unit
      }
    }
}

Let's rock these crocodile boots we found the other day at the market and test our semaphore at the night club, yee-haw:

import zio.duration.Duration
import zio.clock._
import zio.console._
import zio.random._

val party = for {
  dancefloor <- S(10)
  dancers <- ZIO.foreachPar(1 to 100) { i =>
    dancefloor.P *> nextDouble.map(d => Duration.fromNanos((d * 1000000).round)).flatMap { d =>
      putStrLn(s"${i} checking my boots") *> sleep(d) *> putStrLn(s"${i} dancing like it's 99")
    } *> dancefloor.V
  }
} yield ()

It goes without saying you should take a look at ZIO's own Semaphore, it does all this and more without wasting all those CPU cycles while waiting.

Polymorphic Refs​

Ref[A] is actually a type alias for ZRef[Nothing, Nothing, A, A]. The type signature of ZRef is:

trait ZRef[+EA, +EB, -A, +B]

A ZRef is a polymorphic, purely functional description of a mutable reference. The fundamental operations of a ZRef are set and get. set takes a value of type A and sets the reference to a new value, potentially failing with an error of type EA. get gets the current value of the reference and returns a value of type B, potentially failing with an error of type EB.

When the error and value types of the ZRef are unified, that is, it is a ZRef[E, E, A, A], the ZRef also supports atomic modify and update operations as discussed above.

A simple use case is passing out read-only or write-only views of a reference:

for {
  ref       <- Ref.make(false)
  readOnly  = ref.readOnly
  writeOnly = ref.writeOnly
  _         <- writeOnly.set(true)
  value     <- readOnly.get
} yield value