Shiny Source Code Explained: the use of R6

Did you know you can find several of R’s Object-Oriented Programming (OOP) systems in Shiny? One of them is R6. Shiny uses R6 classes in several places: for setting up the reactive environment, managing a Shiny session, and in functions like reactiveValues and ExtendedTask. So how does it work, and why is R6 being used? Find out in this edition of “Shiny Source Code Explained”!

Objected-Oriented Programming in R

When it comes to OOP in R, there’s much to choose from. The most important systems are S3, S4 and R6. And if that wasn’t enough, there’s also a new (or rather: latest) system in the works: S7. Each of these systems have their own strengths and weaknesses, and that’s probably why a package like Shiny implements various systems as well. In Shiny, you can find both S3 and R6. While we will primarily focus on the use of R6, we will also touch upon the use of S3. As you will see, both of these systems are used for the same functions. One of those functions is reactiveValues.

reactiveValues in Shiny

It’s one of my favourite reactive building blocks in Shiny: reactiveValues. The reactiveValues function is great to store and manage state across your Shiny application. It behaves like a reactive list, where accessing or updating values triggers reactive expressions that depend on those values. Under the hood, Shiny implements reactiveValues using an R6 class, leveraging one of R’s available OOP systems. To top it al off, there’s also an S3 wrapper class that forms the main interface.

Sounds like magic? I can imagine! So let’s start by demystifying the source code in two parts: the R6 class, and the S3 wrapper.

The use of R6 for reactiveValues

Long code snippet ahead! Don't worry if you don't grasp it all - just scan through it to get an idea of what the code looks like. I will explain the key parts in more detail.

				
					# ReactiveValues ------------------------------------------------------------

ReactiveValues <- R6Class(
  'ReactiveValues',
  portable = FALSE,
  public = list(
    # For debug purposes
    .reactId = character(0),
    .label = character(0),
    .values = 'Map',
    .metadata = 'Map',
    # A map of Dependents objects, one for each key
    .dependents = 'Map',
    # Dependents for the list of all names, including hidden
    .namesDeps = 'Dependents',
    # Dependents for all values, including hidden
    .allValuesDeps = 'Dependents',
    # Dependents for all values
    .valuesDeps = 'Dependents',
    .dedupe = logical(0),
    # Key, asList(), or names() have been retrieved
    .hasRetrieved = list(),
    # All names, in insertion order. The names are also stored in the .values
    # object, but it does not preserve order.
    .nameOrder = character(0),


    initialize = function(
      dedupe = TRUE,
      label = paste0('reactiveValues', p_randomInt(1000, 10000))
    ) {
      .reactId <<- nextGlobalReactId()
      .label <<- label
      .values <<- Map$new()
      .metadata <<- Map$new()
      .dependents <<- Map$new()
      .hasRetrieved <<- list(names = FALSE, asListAll = FALSE, asList = FALSE)
      .namesDeps <<- Dependents$new(reactId = rLog$namesIdStr(.reactId))
      .allValuesDeps <<- Dependents$new(reactId = rLog$asListAllIdStr(.reactId))
      .valuesDeps <<- Dependents$new(reactId = rLog$asListIdStr(.reactId))
      .dedupe <<- dedupe
    },

    get = function(key) {
      # get value right away to use for logging
      keyValue <- .values$get(key)

      if (!.dependents$containsKey(key)) {
        # If we got here, this is the first time someone has tried to access
        # this key.
        rLog$defineKey(.reactId, keyValue, key, .label, getCurrentContext()$.domain)

        reactKeyId <- rLog$keyIdStr(.reactId, key)
        .dependents$set(key, Dependents$new(reactKeyId))
      }

      # Register the "downstream" reactive which is accessing this value, so
      # that we know to invalidate them when this value changes.
      .dependents$get(key)$register()

      if (isFrozen(key))
        reactiveStop()

      keyValue
    },

    set = function(key, value, force = FALSE) {
      # if key exists
      #   if it is the same value, return
      #
      # update value of `key`
      #
      # if key exists
      #   if `key` has been read,
      #     log `update key`
      #     ## (invalidate key later in code)
      # else # if new key
      #   if `names()` have been read,
      #     log `update names()`
      #     invalidate `names()`
      #
      # if hidden
      #   if asListAll has been read,
      #     log `update asList(all.names = TRUE)`
      #     invalidate `asListAll`
      # else # not hidden
      #   if asList has been read,
      #     log `update asList()`
      #     invalidate `asList`
      #
      # update value of `key`
      # invalidate all deps of `key`

      domain <- getDefaultReactiveDomain()
      hidden <- substr(key, 1, 1) == "."

      key_exists <- .values$containsKey(key)

      if (key_exists && !isTRUE(force) && .dedupe && identical(.values$get(key), value)) {
        return(invisible())
      }

      # If it's new, append key to the name order
      if (!key_exists) {
        .nameOrder[length(.nameOrder) + 1] <<- key
      }

      # set the value for better logging
      .values$set(key, value)

      # key has been depended upon
      if (.dependents$containsKey(key)) {
        rLog$valueChangeKey(.reactId, key, value, domain)
        .dependents$get(key)$invalidate()
      }

      # only invalidate if there are deps
      if (!key_exists && isTRUE(.hasRetrieved$names)) {
        rLog$valueChangeNames(.reactId, .values$keys(), domain)
        .namesDeps$invalidate()
      }

      if (hidden) {
        if (isTRUE(.hasRetrieved$asListAll)) {
          rLog$valueChangeAsListAll(.reactId, .values$values(), domain)
          .allValuesDeps$invalidate()
        }
      } else {
        if (isTRUE(.hasRetrieved$asList)) {
          react_vals <- .values$values()
          react_vals <- react_vals[!grepl("^\\.", base::names(react_vals))]
          # leave as is. both object would be registered to the listening object
          rLog$valueChangeAsList(.reactId, react_vals, domain)
          .valuesDeps$invalidate()
        }
      }

      invisible()
    },

    mset = function(lst) {
      lapply(base::names(lst),
             function(name) {
               self$set(name, lst[[name]])
             })
    },

    names = function() {
      if (!isTRUE(.hasRetrieved$names)) {
        domain <- getDefaultReactiveDomain()
        rLog$defineNames(.reactId, .nameOrder, .label, domain)
        .hasRetrieved$names <<- TRUE
      }
      .namesDeps$register()
      return(.nameOrder)
    },

    # Get a metadata value. Does not trigger reactivity.
    getMeta = function(key, metaKey) {
      # Make sure to use named (not numeric) indexing into list.
      metaKey <- as.character(metaKey)
      .metadata$get(key)[[metaKey]]
    },

    # Set a metadata value. Does not trigger reactivity.
    setMeta = function(key, metaKey, value) {
      # Make sure to use named (not numeric) indexing into list.
      metaKey <- as.character(metaKey)

      if (!.metadata$containsKey(key)) {
        .metadata$set(key, list())
      }

      m <- .metadata$get(key)
      m[[metaKey]] <- value
      .metadata$set(key, m)
    },

    # Mark a value as frozen If accessed while frozen, a shiny.silent.error will
    # be thrown.
    freeze = function(key, invalidate = FALSE) {
      domain <- getDefaultReactiveDomain()
      rLog$freezeReactiveKey(.reactId, key, domain)
      setMeta(key, "frozen", TRUE)

      if (invalidate) {
        # Force an invalidation
        self$set(key, NULL, force = TRUE)
      }
    },

    thaw = function(key) {
      domain <- getDefaultReactiveDomain()
      rLog$thawReactiveKey(.reactId, key, domain)
      setMeta(key, "frozen", NULL)
    },

    isFrozen = function(key) {
      isTRUE(getMeta(key, "frozen"))
    },

    toList = function(all.names=FALSE) {
      listValue <- .values$mget(.nameOrder)
      if (!all.names) {
        listValue <- listValue[!grepl("^\\.", base::names(listValue))]
      }
      if (all.names) {
        if (!isTRUE(.hasRetrieved$asListAll)) {
          domain <- getDefaultReactiveDomain()
          rLog$defineAsListAll(.reactId, listValue, .label, domain)
          .hasRetrieved$asListAll <<- TRUE
        }
        .allValuesDeps$register()
      }

      if (!isTRUE(.hasRetrieved$asList)) {
        domain <- getDefaultReactiveDomain()
        # making sure the value being recorded is with `all.names = FALSE`
        rLog$defineAsList(.reactId, listValue[!grepl("^\\.", base::names(listValue))], .label, domain)
        .hasRetrieved$asList <<- TRUE
      }
      .valuesDeps$register()

      return(listValue)
    }

  )
)

I warned you for the amount of code involved! You can find it back in reactives.R. If you’ve worked with R6 before, it looks quite familiar though. The R6 class is created with the function R6::R6Class(). This function needs two arguments: the classname, which is “ReactiveValues”, and a list of public fields or methods that will make up the interface you can interact with, public. There are a bunch of fields for internal housekeeping and debugging purposes, like .reactId, .label and .values. After the fields you can find the methods, like initialize, get, and set.

To make it bit easier to follow, let’s focus on how you you would normally use reactive values in Shiny:

You initialize your reactiveValues object, by calling the reactiveValues function with initial key-value pairs.
You get values somewhere in your app.
You set updated or new values somewhere in your app.
Whenever a value changes, all dependents get notified and are re-calculated. There’s dependency tracking and invalidation.

We call the steps above the reactiveValues lifecycle.

Initialization

It all starts with creating the reactiveValues object, where you, as a Shiny user, call the function like so:

				
					values <- reactiveValues(a = 1, b = 2)

Under the hood, Shiny calls ReactiveValues$new(), whose behaviour is determined by whatever is in the initialize method of the R6 class. We’ll come back to how Shiny exactly ends up using the R6 class a little bit later. For now, we’ll just assume that somewhere in the process, the initialize function is triggered:

				
					    initialize = function(
      dedupe = TRUE,
      label = paste0('reactiveValues', p_randomInt(1000, 10000))
    ) {
      .reactId <<- nextGlobalReactId()
      .label <<- label
      .values <<- Map$new()
      .metadata <<- Map$new()
      .dependents <<- Map$new()
      .hasRetrieved <<- list(names = FALSE, asListAll = FALSE, asList = FALSE)
      .namesDeps <<- Dependents$new(reactId = rLog$namesIdStr(.reactId))
      .allValuesDeps <<- Dependents$new(reactId = rLog$asListAllIdStr(.reactId))
      .valuesDeps <<- Dependents$new(reactId = rLog$asListIdStr(.reactId))
      .dedupe <<- dedupe
    }

As you can see, the initialize function sets up several internal fields, or values, including:

.values: stores the actual data as key-value pairs.
.metadata: to manage and store non-reactive properties of reactive values.
.dependents: Tracks which reactive expressions depend on each key.

It also creates an unique .reactId and a .label. Note that the majority of these fields are created by R6 classes too! You can recognise them by $new(). So if you feel like exploring more R6 in Shiny, you can check out the Map and Dependents classes! Spoiler alert: at the end of this article we take a brief look at the second class which covers dependents.

Reading a value

You create a reactiveValues object for a reason: you want to read those values somewhere. To read one or more values from your reactiveValues object, you need a reactive context, like a text output:

				
					output$text <- renderText({
    values$a
})

It’s the get method in the ReactieValues R6 class that actually takes care of reading values:

				
					    get = function(key) {
      # get value right away to use for logging
      keyValue <- .values$get(key)

      if (!.dependents$containsKey(key)) {
        # If we got here, this is the first time someone has tried to access
        # this key.
        rLog$defineKey(.reactId, keyValue, key, .label, getCurrentContext()$.domain)

        reactKeyId <- rLog$keyIdStr(.reactId, key)
        .dependents$set(key, Dependents$new(reactKeyId))
      }

      # Register the "downstream" reactive which is accessing this value, so
      # that we know to invalidate them when this value changes.
      .dependents$get(key)$register()

      if (isFrozen(key))
        reactiveStop()

      keyValue
    }

Your reactive expression needs to be re-calculated every time the value updates. Therefore, your reactive expression becomes a dependent. So, somewhere Shiny needs to keep track of those dependents. The first bit in the get method takes care of that. It starts by checking if the key has been accessed before. If not, it creates a new Dependents object for it via Dependents$new() (yes, one of the other R6 classes mentioned earlier!).

The second bit registers the current reactive context, e.g. your output$text, as dependent on this key.

Eventually, it will return the key value.

Setting a value

You’re probably going to set new or updated values for your keys in your app, which you would normally do like this:

				
					values$a <- 3
values[['b']] <- 4

To make sure those new or updated values get stored correctly, the set method is used:

				
					    set = function(key, value, force = FALSE) {
      domain <- getDefaultReactiveDomain()
      hidden <- substr(key, 1, 1) == "."

      key_exists <- .values$containsKey(key)

      if (key_exists && !isTRUE(force) && .dedupe && identical(.values$get(key), value)) {
        return(invisible())
      }

      # If it's new, append key to the name order
      if (!key_exists) {
        .nameOrder[length(.nameOrder) + 1] <<- key
      }

      # set the value for better logging
      .values$set(key, value)

      # key has been depended upon
      if (.dependents$containsKey(key)) {
        rLog$valueChangeKey(.reactId, key, value, domain)
        .dependents$get(key)$invalidate()
      }

      # only invalidate if there are deps
      if (!key_exists && isTRUE(.hasRetrieved$names)) {
        rLog$valueChangeNames(.reactId, .values$keys(), domain)
        .namesDeps$invalidate()
      }

      if (hidden) {
        if (isTRUE(.hasRetrieved$asListAll)) {
          rLog$valueChangeAsListAll(.reactId, .values$values(), domain)
          .allValuesDeps$invalidate()
        }
      } else {
        if (isTRUE(.hasRetrieved$asList)) {
          react_vals <- .values$values()
          react_vals <- react_vals[!grepl("^\\.", base::names(react_vals))]
          # leave as is. both object would be registered to the listening object
          rLog$valueChangeAsList(.reactId, react_vals, domain)
          .valuesDeps$invalidate()
        }
      }

      invisible()
    }

This methods takes care of the following:

It checks if the key already exists and whether the value has changed. If it didn’t change (and some other conditions), it will return(invisible()). Or, in simpler terms: do nothing.
If the value is new, it adds the key to .nameOrder and stores the new value. As the name suggest, .nameOrder makes sure the order of keys is preserved.
If the value is new or updated and if reactive dependencies are registered for that key, they are invalidated (meaning that they will be notified that the value has changed, and start re-calculating).

Dependency tracking and invalidation

The .dependents are important. As you could see above, when the value of a key changes, the dependents will be invalidated. This happens through one of the methods beloning to the Dependent R6 class: invalidate.

The two methods of the ReactiveValues class that we have discussed, namely get and set, make good use of this Dependent class.

In the set method of the ReactiveValues class, dependents get invalidated with .dependents$get(key)$invalidate().
In the the get method of the ReactiveValues class a new instance of the Dependent class is created with Dependents$new().

Can you still follow it?! There are R6 classes everywhere!

To give you a better idea, this is what that Dependent R6 class looks like:

				
					Dependents <- R6Class(
  'Dependents',
  portable = FALSE,
  class = FALSE,
  public = list(
    .reactId = character(0),
    .dependents = 'Map',

    initialize = function(reactId = NULL) {
      .reactId <<- reactId
      .dependents <<- Map$new()
    },
    # ... ignored, use to be depLabel and depId, not used anymore
    register = function(...) {
      ctx <- getCurrentContext()
      if (!.dependents$containsKey(ctx$id)) {

        # must wrap in if statement as ctx react id could be NULL
        #   if options(shiny.suppressMissingContextError = TRUE)
        if (is.character(.reactId) && is.character(ctx$.reactId)) {
          rLog$dependsOn(ctx$.reactId, .reactId, ctx$id, ctx$.domain)
        }

        if (ctx$isWeak()) {
          .dependents$set(ctx$id, rlang::new_weakref(ctx))
        } else {
          .dependents$set(ctx$id, ctx)
        }

        ctx$onInvalidate(function() {
          rLog$dependsOnRemove(ctx$.reactId, .reactId, ctx$id, ctx$.domain)
          .dependents$remove(ctx$id)
        })
      }
    },
    # at times, the context is run in a ctx$onInvalidate(...) which has no runtime context
    invalidate = function(log = TRUE) {
      if (isTRUE(log)) {

        domain <- getDefaultReactiveDomain()
        rLog$invalidateStart(.reactId, NULL, "other", domain)
        on.exit(
          rLog$invalidateEnd(.reactId, NULL, "other", domain),
          add = TRUE
        )
      }
      lapply(
        .dependents$values(sort = TRUE),
        function(ctx) {
          if (rlang::is_weakref(ctx)) {
            ctx <- rlang::wref_key(ctx)
            if (is.null(ctx)) {
              # Can get here if weakref target was GC'd
              return()
            }
          }
          ctx$invalidate()
          NULL
        }
      )
    }
  )
)

For understanding reactiveValues, you only need to remember that the two methods, register and invalidate, are called at appropriate times when getting and setting keys and their values in the ReactiveValues class.

The S3 wrapper class

You have to do quite some digging to understand what’s going on in the R6 class. And you also saw that it’s not just one R6 class: there are many. Yet, as a “regular” Shiny user, you don’t need to know anything about these classes. That’s because there’s an additional layer: the S3 wrapper class.

				
					reactiveValues <- function(...) {
  args <- list2(...)
  if ((length(args) > 0) && (is.null(names(args)) || any(names(args) == "")))
    rlang::abort("All arguments passed to reactiveValues() must be named.")

  values <- .createReactiveValues(ReactiveValues$new())

  # Use .subset2() instead of [[, to avoid method dispatch
  .subset2(values, 'impl')$mset(args)
  values
}

So it’s not only R6 that let’s the reactiveValues function work the way it does. The main interface you actually interact with as Shiny user, is written in S3. This S3 wrapper makes it easier to interact with the ReactiveValues R6 class (and its friends). The reactiveValues() function abstracts away the complexity of the underlying R6 class, allowing users to work with reactive values as if they were just using a list.

As you can see in the code above, the ReactiveValues R6 class is created with ReactiveValues$new(), and stored in an object called values. Eventually the values object is returned. This way, you don’t have to worry about anything R6 related, but can just happily use the function as a simple list!

Why R6?

Now you know that Shiny uses R6, it could make you wonder “why”. Why did the Shiny package developers choose for R6?

While we can’t look into the minds of the Shiny package developers, it is quite clear that there’s a lot to keep track of when it comes to reactive values. It’s about state management. We constantly need to be aware of which keys there are, what their values are, which reactive expressions depend on them… And all of that information needs to be available in the reactiveValues object created by the user.

R6 is perfect for this:

R6 Classes encapsulate both data and methods, which is very convenient for managing state.
R6 Classes are mutable: you can very simply modify properties of R6 objects without needing to return a new object.

Of course there’s much more to say about R6 and its usability, but for that I’ll refer you to Object-Oriented Programming in R with R6.

Object-Oriented Programming: do you need to know anything about it?

No, you don’t need to know anything about Object-Oriented Programming as a Shiny user. The brilliant developers of the Shiny package made sure that you don’t have to worry about all those technicalities. But if you want to understand Shiny better, and even want to contribute to it, you do need to know about OOP. The reason is simple: it’s used throughout the package and it’s hard to get around it. Just by looking at one reactive construct, reactiveValues, we can already see how much OOP is involved through both the R6 and S3 systems.

Learn more about OOP

Curious to learn more? I have written a couple of articles about OOP already:

And let’s not forget this brilliant book:

Advanced R by Hadley Wickham

Shiny Source Code Explained: other editions

I like digging through source code. It’s like an adventure where you discover more with every turn you take. So far, I’ve written two other editions of “Shiny Source Code Explained”:

I hope you enjoy these little Shiny adventures as much as I do!

Wrap up

There are lots of mysteries to uncover when it comes to the Shiny source code. I hope this series is a fun way to learn more about (perhaps) your favourite R package!

☕️ Was this useful to you and would you like to support me? You can buy me a coffee!

📚 Want to learn more about Shiny? Discover my online Shiny courses: ProductioniZing Shiny , CustomiZing WidgetZ and Mobile StructureZ.

I provide R and Shiny consultancy. Do you need help with a project? Reach out and let’s have a chat!