Shiny’s reactive system has a few key players, and one of them is observers. The idea is pretty straightforward: whenever there’s a change in a reactive value or input, you want to do something. Shiny observers make that happen. But there’s more than meets the eye: observers have their little secrets!
Before we can uncover any secrets, it’s good to take a step back first and focus on what observers actually are. Which functions are we talking about, and what characteristics do Shiny observers have? After that, it’s time to get into Shiny’s Source Code to see how observers actually work under the hood. What hidden methods exist, and how can you use them in your next Shiny app? All that (and more) in this edition of “Shiny Source Code Explained”!
Table of Contents
Observers: what do they do?
Observers, such as observe(), observeEvent(), and the more recent combination with bindEvent(), play a crucial role in Shiny’s reactive programming paradigm. They allow you to define reactive behaviour based on changes in input values or specific events. They do something: you can call an API, update a database, or print something on the console. Observers cause side-effects.
Observers respond in real-time to these events, triggering actions or updating outputs accordingly. In other words, they are eager: they are immediately executed when changes occur.
Besides being eager, observers are also forgetful. They don’t remember or store values.
Another key aspect of Shiny observers is how they handle errors. When an error occurs within an observer, such as in an observe() block, the error terminates the entire app’s execution. This behavior ensures that inconsistencies or incomplete updates are avoided, maintaining the integrity of your app’s state. By terminating the app on error, Shiny prevents users from interacting with a potentially unstable app. That’s good, because nobody wants that. This is different from reactives, for example. When an error occurs within a reactive expression, e.g. reactive(), it does not propagate to other parts of the app. The error is isolated within the reactive expression, allowing the app to continue functioning without termination.
So far so good. The eager Shiny observers are handy for side-effects, like calling an API. But what if you want to control that eagerness a little bit? Well, hello bindEvent()! With bindEvent() you can literally bind an event to an observer, such as the click of a button. You always use it in combination with observe() (or reactive()). The use of bindEvent() has been recommended since Shiny 1.6, and without going into too much detail, it is preferred over observeEvent() (and eventReactive()) because it can work with cache and it can work with render* functions. It is more flexible.
observe({
# Do something on button click, but not on text change
print(input$text)
}) |> bindEvent(input$button)
Observers vs outputs
In Shiny, outputs are typically considered observers rather than reactives. Outputs are not explicitly created using functions like observe(), or a combination of observe() and bindEvent(). Instead, they are defined using specific output functions like renderPlot(), renderText(), or renderTable(), among others. These output functions generate the necessary reactive expressions behind the scenes to update the output based on reactive dependencies.
If you look at a reactive graph, you’ll see outputs and observers in a terminal node. They do something. When an output function is used, it creates an observer that monitors the reactive dependencies it relies on. These dependencies can be reactive values, reactive expressions, or reactive conductors like user inputs or reactive data. When any of these dependencies change, the observer associated with the output function is triggered, causing the output to update and reflect the new values or changes.
So observers and outputs are closely related. You can think of outputs as having a special side-effect: updating the HTML in the user’s browser so the correct thing is being displayed.
Right, now we got that covered it’s time to take a look at how these eager reactive building blocks are handled in the place where all the magic happens: Shiny’s Source Code!
Shiny Source Code
Lots is happening in Shiny’s Source Code, so it’s to be expected that there’s a long code snippet that makes up the code for observers. The starting point is the Observer R6 class. We won’t go into the details of R6 classes and why Shiny uses them right now, but you can read more about that in a previous edition of “Shiny Source Code Explained”.
The Observer R6 class contains a list of public fields or methods that will make up the interface you can interact with, public. In reactives.R, you can see a bunch of these methods in Shiny’s Source Code for the Observer class:
# Observer ------------------------------------------------------------------
Observer <- R6Class(
'Observer',
portable = FALSE,
public = list(
.reactId = character(0),
.func = 'function',
.label = character(0),
.domain = 'ANY',
.priority = numeric(0),
.autoDestroy = logical(0),
.autoDestroyHandle = 'ANY',
.invalidateCallbacks = list(),
.execCount = integer(0),
.onResume = 'function',
.suspended = logical(0),
.destroyed = logical(0),
.prevId = character(0),
.ctx = NULL,
initialize = function(observerFunc, label, suspended = FALSE, priority = 0,
domain = getDefaultReactiveDomain(),
autoDestroy = TRUE, ..stacktraceon = TRUE) {
if (length(formals(observerFunc)) > 0)
rlang::abort(c(
"Can't make an observer from a function that takes arguments.",
"Only functions without arguments can become observers."
))
if (grepl("\\s", label, perl = TRUE)) {
funcLabel <- "<observer>"
} else {
funcLabel <- paste0("<observer:", label, ">")
}
.func <<- wrapFunctionLabel(observerFunc, funcLabel, ..stacktraceon = ..stacktraceon)
.label <<- label
.domain <<- domain
.priority <<- normalizePriority(priority)
.execCount <<- 0L
.suspended <<- suspended
.onResume <<- function() NULL
.destroyed <<- FALSE
.prevId <<- ''
.autoDestroy <<- FALSE
.autoDestroyHandle <<- NULL
setAutoDestroy(autoDestroy)
.reactId <<- nextGlobalReactId()
rLog$defineObserver(.reactId, .label, .domain)
# Defer the first running of this until flushReact is called
.createContext()$invalidate()
},
.createContext = function() {
ctx <- Context$new(.domain, .label, type='observer', prevId=.prevId, reactId = .reactId)
.prevId <<- ctx$id
if (!is.null(.ctx)) {
# If this happens, something went wrong.
warning("Created a new context without invalidating previous context.")
}
.ctx <<- ctx
ctx$onInvalidate(function() {
# Context is invalidated, so we don't need to store a reference to it
# anymore.
.ctx <<- NULL
lapply(.invalidateCallbacks, function(invalidateCallback) {
invalidateCallback()
NULL
})
continue <- function() {
ctx$addPendingFlush(.priority)
if (!is.null(.domain)) {
.domain$incrementBusyCount()
}
}
if (.suspended == FALSE)
continue()
else
.onResume <<- continue
})
ctx$onFlush(function() {
hybrid_chain(
{
if (!.destroyed) {
shinyCallingHandlers(run())
}
},
catch = function(e) {
if (cnd_inherits(e, "shiny.silent.error")) {
return()
}
printError(e)
if (!is.null(.domain)) {
.domain$unhandledError(e, close = TRUE)
}
},
finally = .domain$decrementBusyCount
)
})
return(ctx)
},
run = function() {
ctx <- .createContext()
.execCount <<- .execCount + 1L
ctx$run(.func)
},
onInvalidate = function(callback) {
"Register a callback function to run when this observer is invalidated.
No arguments will be provided to the callback function when it is
invoked."
.invalidateCallbacks <<- c(.invalidateCallbacks, callback)
},
setPriority = function(priority = 0) {
"Change this observer's priority. Note that if the observer is currently
invalidated, then the change in priority will not take effect until the
next invalidation--unless the observer is also currently suspended, in
which case the priority change will be effective upon resume."
.priority <<- normalizePriority(priority)
},
setAutoDestroy = function(autoDestroy) {
"Sets whether this observer should be automatically destroyed when its
domain (if any) ends. If autoDestroy is TRUE and the domain already
ended, then destroy() is called immediately."
if (.autoDestroy == autoDestroy) {
return(.autoDestroy)
}
oldValue <- .autoDestroy
.autoDestroy <<- autoDestroy
if (autoDestroy) {
if (!.destroyed && !is.null(.domain)) { # Make sure to not try to destroy twice.
if (.domain$isEnded()) {
destroy()
} else {
.autoDestroyHandle <<- onReactiveDomainEnded(.domain, .onDomainEnded)
}
}
} else {
if (!is.null(.autoDestroyHandle))
.autoDestroyHandle()
.autoDestroyHandle <<- NULL
}
invisible(oldValue)
},
suspend = function() {
"Causes this observer to stop scheduling flushes (re-executions) in
response to invalidations. If the observer was invalidated prior to this
call but it has not re-executed yet (because it waits until onFlush is
called) then that re-execution will still occur, because the flush is
already scheduled."
.suspended <<- TRUE
},
resume = function() {
"Causes this observer to start re-executing in response to invalidations.
If the observer was invalidated while suspended, then it will schedule
itself for re-execution (pending flush)."
if (.suspended) {
.suspended <<- FALSE
.onResume()
.onResume <<- function() NULL
}
invisible()
},
destroy = function() {
"Prevents this observer from ever executing again (even if a flush has
already been scheduled)."
# Make sure to not try to destory twice.
if (.destroyed)
return()
suspend()
.destroyed <<- TRUE
if (!is.null(.autoDestroyHandle)) {
.autoDestroyHandle()
}
.autoDestroyHandle <<- NULL
if (!is.null(.ctx)) {
.ctx$invalidate()
}
},
.onDomainEnded = function() {
if (isTRUE(.autoDestroy)) {
destroy()
}
}
)
)
initialize is a method, as is run, suspend, destroy, and the others that are listed. You can call any of the Observer class methods using $methodName on the assigned object. This happens in Shiny’s Source Code, but as a Shiny user you can also call these methods if you assign an observer to an object. We will come back to this later.
New instances of observers are created by calling Observer$new(). When called, its behaviour is determined by whatever is in the initialize method of the R6 class. See it is some sort of shortcut. New instances are created in the observe() function, which can also be found in reactives.R:
observe <- function(
x,
env = parent.frame(),
quoted = FALSE,
...,
label = NULL,
suspended = FALSE,
priority = 0,
domain = getDefaultReactiveDomain(),
autoDestroy = TRUE,
..stacktraceon = TRUE)
{
check_dots_empty()
func <- installExprFunction(x, "func", env, quoted)
label <- funcToLabel(func, "observe", label)
o <- Observer$new(
func,
label = label,
suspended = suspended,
priority = priority,
domain = domain,
autoDestroy = autoDestroy,
..stacktraceon = ..stacktraceon
)
invisible(o)
}
As you can see, an object called o is created and it basically passes along whatever the user has specified in the observe() call. o is returned at the end of the function. Perhaps you didn’t expect the code to be this short, right? So what happens when Observer$new() is being called? As said earlier, it’s the initialize method of the R6 class that is executed first. Let’s zoom in on that bit:
Observer <- R6Class(
'Observer',
portable = FALSE,
public = list(
...
initialize = function(observerFunc, label, suspended = FALSE, priority = 0,
domain = getDefaultReactiveDomain(),
autoDestroy = TRUE, ..stacktraceon = TRUE) {
if (length(formals(observerFunc)) > 0)
rlang::abort(c(
"Can't make an observer from a function that takes arguments.",
"Only functions without arguments can become observers."
))
if (grepl("\\s", label, perl = TRUE)) {
funcLabel <- "<observer>"
} else {
funcLabel <- paste0("<observer:", label, ">")
}
.func <<- wrapFunctionLabel(observerFunc, funcLabel, ..stacktraceon = ..stacktraceon)
.label <<- label
.domain <<- domain
.priority <<- normalizePriority(priority)
.execCount <<- 0L
.suspended <<- suspended
.onResume <<- function() NULL
.destroyed <<- FALSE
.prevId <<- ''
.autoDestroy <<- FALSE
.autoDestroyHandle <<- NULL
setAutoDestroy(autoDestroy)
.reactId <<- nextGlobalReactId()
rLog$defineObserver(.reactId, .label, .domain)
# Defer the first running of this until flushReact is called
.createContext()$invalidate()
}
...
)
)
The following happens in this method:
- First there is some function validation and label formatting.
Then, several internal properties (
.label,.domain,.priority,.execCount,.suspended,.destroyed,.prevId) are initialized with default or provided values. For example,.execCountstarts at0L(indicating no executions yet),.destroyedisFALSEbecause the observer is active, and.suspendedis set based on the providedsuspendedargument.Then we get to auto destruction. It’s a bit confusing, but pay close attention here, as there are actually two values:
autoDestroywith and without a dot. In R6, the dot (.) prefix in field names is typically used as a naming convention to indicate that a field is “private” or “internal” to the class..autoDestroyis an internal property of theObserverclass. It’s set toFALSE.autoDestroy(without the dot) is an argument to theinitialize()method, which allows the user to specify whether the observer should auto-destroy when ending the reactive context, aka the Shiny session. By default this isTRUE. Probably this is used for separation of internal state and user inputs, as the.autoDestroyfield holds the actual internal value, whileautoDestroy(the argument) is just a temporary value for initialization. This separation is also handy to keep track of old and new values, which you can see in thesetAutoDestroymethod of our Observer R6 class. TL;DR: Shiny makes sure to auto-destroy observers whenever you want them to- which is most likely always.
Now let’s zoom in on the last line in the initialization method, .createContext$invalidate(), where the .createContext method is being called and invalidated. This method looks like this:
# Observer ------------------------------------------------------------------
Observer <- R6Class(
'Observer',
portable = FALSE,
public = list(
...
.createContext = function() {
ctx <- Context$new(.domain, .label, type='observer', prevId=.prevId, reactId = .reactId)
.prevId <<- ctx$id
if (!is.null(.ctx)) {
# If this happens, something went wrong.
warning("Created a new context without invalidating previous context.")
}
.ctx <<- ctx
ctx$onInvalidate(function() {
# Context is invalidated, so we don't need to store a reference to it
# anymore.
.ctx <<- NULL
lapply(.invalidateCallbacks, function(invalidateCallback) {
invalidateCallback()
NULL
})
continue <- function() {
ctx$addPendingFlush(.priority)
if (!is.null(.domain)) {
.domain$incrementBusyCount()
}
}
if (.suspended == FALSE)
continue()
else
.onResume <<- continue
})
ctx$onFlush(function() {
hybrid_chain(
{
if (!.destroyed) {
shinyCallingHandlers(run())
}
},
catch = function(e) {
if (cnd_inherits(e, "shiny.silent.error")) {
return()
}
printError(e)
if (!is.null(.domain)) {
.domain$unhandledError(e, close = TRUE)
}
},
finally = .domain$decrementBusyCount
)
})
return(ctx)
}
...
)
)
The .createContext() method takes care of creating a new reactive context for the observer. It’s the foundation. It makes use of Context, also an R6 class, tracks dependencies and ensures that when reactive values change, the observer invalidates itself and schedules execution. The Context method responsible for this is onFlush(), which you can take a closer look at here. The onFlush method registers a function to be called before the next time (if once = TRUE) or every time (if once = FALSE) Shiny flushes the reactive system.
So back to the initialization, where the context immediately invalidates itself, using .createContext()$invalidate(), deferring the first execution until the next flush cycle. In Shiny, a flush cycle is the process of executing reactive updates. This happens in a certain order and makes sure updates are processed efficiently across the entire app. But wait a minute… We defer something by actually invalidating?! Calling .createContext()$invalidate() might seem counterintuitive because invalidate() is what normally triggers execution, but in this case, it’s actually a mechanism to defer execution until the next flush cycle. Observers execute as part of a reactive flush cycle, not immediately when invalidated. It marks the observer with a flag that says, “Hey, I need to run!”, but the actual execution waits for Shiny’s flush cycle to process everything properly. This ensures the observer runs after all other pending changes in a structured way. Clever thinking!
Observer$new(), the observer() function doesn’t do anything else besides returning the o object with invisible(), which sounds a bit mysterious. Why is that being used? Two reasons:
- Remember that in Shiny,
observe()is called for its side effects, and users typically don’t need to see the observer object in the console.invisible()prevents automatic printing of the object when the function is executed interactively. - However, returning the observer object allows advanced users to store it in a variable. This comes in handy when users want to call some of the methods.
Methods
As we’ve seen in the Observer R6 class, there are various methods that can be used if you assign your observer to an object:
destroy(): a method that you can call on observers to stop them from executing ever again, even if they are currently scheduled for re-execution.suspend(): stop scheduling flushes (re-executions) in response to invalidationsresume(): start re-executing in response to invalidationssetPriority(): change the observer’s prioritysetAutoDestroy(): as previously discussed, this sets whether this observer should be automatically destroyed when its domain endsonInvalidate(callback): a callback function that gets triggered whenever the observer is invalidated
Much of these things you won’t be using on a daily basis, but when you find yourself in a complex reactivity problem, knowing about these methods can come in handy. How? In the next section we’ll take a look at a practical example using the destroy() method.
Practical example: destroying observers
Imagine you have an app that lets users dynamically create sports pages based on the sports they want to see. You’re a good and organized Shiny dev, so you build a sports page module. Whenever a user generates a sports page, you insert the module in the UI and you attach the module on the server side. So far so good.
However, a user can change her/his mind. At some stage, she/he doesn’t want to see a particular sport anymore. You remove the page from the UI and you think that’s the end of the story. But before you know it, that annoying user is changing their minds again, and they decide to add the particular sport back.
UI wise that’s fine, but the user will notice some strange behaviour: observers trigger not once, but TWICE. That’s because all the observers you attach to the “domain” will be there until the “domain”, aka the Shiny session, stops being alive. And it will just keep attaching the “same” observers when you call your server module again. That’s observers piling up.
For this problem, there are two solutions:
- You just keep track of which sports already had their server part “loaded”, and don’t call the server part again whenever an already loaded sport has been chosen OR
- You make sure you destroy observers whenever a sport is removed
The second one is a bit cleaner: you don’t need those observers whenever the particular sports page is not there. If you want to get rid of a module, you might as well do it properly. It would look something like this:
library(shiny)
library(bslib)
sportsPageUI <- function(id, page_name) {
ns <- NS(id)
tagList(
h2(page_name),
card(
card_header("General Info"),
card_body(
textOutput(ns("general_info")),
actionButton(ns("notification"), "Show notification")
)
)
)
}
sportsPageServer <- function(id, chosen_sport, r) {
moduleServer(id, function(input, output, session) {
output$general_info <- renderText({
sprintf("%s is the sports you have chosen!", chosen_sport)
})
my_observer <-
observe({
showNotification(sprintf("Welcome to the %s page!", chosen_sport))
}) |> bindEvent(input$notification)
r$observer_list[[id]] <- my_observer
})
}
ui <- page_navbar(
id = "navbar_id",
title = "Destroy modules",
nav_menu(title = "Sports",
icon = icon("basketball-ball"),
nav_panel(title = "Add sports",
icon = icon("cogs"),
selectInput("sports_to_add",
label = "Choose sports to add",
choices = c("Baseball",
"Basketball",
"Football",
"Tennis"),
selected = NULL,
multiple = TRUE)
)
)
)
server <- function(input, output, session) {
r <- reactiveValues(active_pages = NULL,
observer_list = list())
observe({
chosen_sports <- input$sports_to_add
lapply(chosen_sports, function(this_sport) {
if (this_sport %in% r$active_pages) {
# the sport already has a page, and is chosen by the user, so don't do anything
return()
}
nav_insert(id = "navbar_id",
target = "Add sports",
nav = nav_panel(
title = this_sport,
icon = icon("plus-circle"),
sportsPageUI(id = this_sport, page_name = this_sport)
),
position = "after")
sportsPageServer(id = this_sport, chosen_sport = this_sport, r = r)
})
if (setdiff(r$active_pages, chosen_sports) |> length() > 0) {
remove_sports <- setdiff(r$active_pages, chosen_sports)
for (sport in remove_sports) {
nav_remove(id = "navbar_id",
target = sport)
# destroy any existing observers
r$observer_list[[sport]]$destroy()
}
}
r$active_pages <- chosen_sports
}) |> bindEvent(input$sports_to_add)
}
shinyApp(ui, server)
In the sportsPageServer module, you can see that the notification observer is assigned to an object, my_observer. It is then stored in a list with an id (the sport), as a reactive value using the stratégie du petit r. We can access currently “active” observers using r$observer_list.
If we combine that approach with a reactive value that stores the active pages, r$active pages, we can make sure we destroy any observer that has an id that corresponds to a sport that is not chosen anymore. We simply call the destroy method from the Observer R6 class: r$observer_list[[sport]]$destroy(). We can do this because we assigned our observers to an object.
Of course, this is a very niche problem, but it demonstrates how you could end up with observers triggering multiple times and how you can destroy them to prevent that. You can also think about scenarios where you would want to run an observer not just once, but also not infinitely. A simple counter that keeps track of the number of runs and destroying an observer with the destroy method when that counter reaches a certain value already does the trick.
As said earlier, the “secret” methods of observers can come in quite handy! Luckily you know all about them now.
Shiny Source Code Explained: other editions
The “Shiny Source Code Explained” series keeps expanding! If you want to learn more, you can check out previous editions:
Wrap up
I dig through the Shiny Source Code so you don’t have to. I hope you get a deeper understanding of Shiny this way- I certainly do! If you want to support me, you can:
☕️ Buy me a coffee!
📚 Take a look at my online Shiny courses: ProductioniZing Shiny , CustomiZing WidgetZ and Mobile StructureZ.
🤓 Let me help you with your R and Shiny questions. Do you need help with your project? Let’s have a chat!